#### Description
Protobuf has two possible encodings for a repeated field with wire type
VARINT/I32/I64:
- repeat the field tag for each value (like for repeated fields with
other wire types)
- use a "packed" representation: use a single field tag with wire type
LEN containing all the values
The packed representation is the default, but [the Protobuf
documentation](https://protobuf.dev/programming-guides/encoding/#repeated)
notes that:
> Protocol buffer parsers must be able to parse repeated fields that
were compiled as packed as if they were not packed, and vice versa. This
permits adding [packed=true] to existing fields in a forward- and
backward-compatible way.
The current Protobuf parser generated by pdatagen expects the packed
representation and returns an error if the "unpacked" representation is
provided instead, causing slightly off-spec clients to fail exporting
telemetry to the Collector. This PR fixes that by implementing parsing
of unpacked repeated fields.
#### Link to tracking issue
Fixes#13727
#### Testing
I added a test in `pdata/internal` which "manually" encodes a
`HistogramDataPoint` and an `ExponentialHistogramDataPoint_Buckets`
using unpacked fields, and checks that we successfully unmarshal them.
Profile payloads are also heavily affected, but I didn't add tests for
them to avoid the tests failing when there are breaking changes in the
profile proto.
This feature is protected by a featuregate that is not yet enabled and
only marked as alpha. Preliminary results show > 20% CPU performance
improvement and at least 40% allocations improvement. I still need to
generate UnmarshalProto for AnyValue which should improve even more
results.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Also, this PR improves JSON unmarshal logic by avoiding stack calls
caused by using the Read[Array|Object]CB and replace with simple
non-recursive iterations.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
The only non-test change is in the Unmarshal proto but that generates
exactly the same code using the "defaultValue" for the message.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Add support to generate the proto Size by pdatagen. Not yet used, will
update after more tests are added.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Remove the complication about internal/public for all funcs that only
need origin. Also improves significant testing.
This simplified a lot the fields (no more isCommon/isBaseCommon/etc.)
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
This PR allows to add tests for CopyOrig directly in the internal
package, also enables us to move more things to internal (including
tests) for Marshal/Unmarshal.
After this PR is merged we can generate the ProtoSize and other proto
helpers in internal.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Main idea is to have all funcs that work on "origin" in internal, so
that we simplify access to them (see field access to this). Also will
remove duplicate generated code like copying the slice of KeyValue
(followup PR).
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Damien Mathieu
Jade Guiton
Bogdan Drutu