* [chore] use license shortform
To remain consistent w/ contrib repo, see https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/22052
Signed-off-by: Alex Boten <aboten@lightstep.com>
* make goporto
Signed-off-by: Alex Boten <aboten@lightstep.com>
---------
Signed-off-by: Alex Boten <aboten@lightstep.com>
Implement the fixes proposed in #7508. Otherwise, using a batchprocessor with Config{} passes validation but acts badly at runtime there is both a zero-duration timer and the call to sendItems never returns, constantly sending empty batches.
- Instrument batch processor with OpenTelemetry Go.
- Also fixed a small typo on the bucket definition of the batch_send_size_bytes view of OpenCensus.
The main reason is to remove the circular dependency between the config (including sub-packages) and component. Here is the current state:
* component depends on config
* config/sub-package[grpc, http, etc.] depends on config & component
Because of this "circular" dependency, we cannot split for example "config" into its own module, only if all the other config sub-packages are also split.
Signed-off-by: Bogdan <bogdandrutu@gmail.com>
The main motivation is because we want to be able to extend these Marshaler/Unmarshaler implementation with possible options.
Because of that we have 2 options: 1. add Option to each New func, OR 2. expose the structs so that later users can configure different options inside these structs, similar with `jsonpb.Marshaler`.
I implemented the version 2 since it is simpler, and less code, and also common in the industry.
Signed-off-by: Bogdan <bogdandrutu@gmail.com>
Signed-off-by: Bogdan <bogdandrutu@gmail.com>
Split all `pdata` related code by type and move it from `model` to the new module `pdata`.
- `model/pdata` and `model/otlp` are moved to `pdata/plog`, `pdata/pmetric` and `pdata/ptrace`.
- `model/otlpgrpc` is moved to `pdata/plogotlp`, `pdata/pmetricotlp` and `pdata/ptraceotlp`.
Now all the API in `model` except for `model/semconv` is deprecated.
* Deprecation of pdata methods following OTLP v0.15.0
Adding deprecation methods for the following pdata methods:
- InstrumentationLibrary is now Scope
- InstrumentationLibrarySpans is now ScopeSpans
- InstrumentationLibraryLogs is now ScopeLogs
- InstrumentationLibraryMetrics is now ScopeMetrics
* update changelog
* add deprecated method on resource spans/metrics/logs
* add InstrumentationLibrary deprecation method
- Removed all deprecated metric messages and corresponding conversion code and tests.
- Removed deprecated_code from span status and corresponding conversions and tests.
- Renamed logs field to log_records in InstrumentationLibraryLogs message.
The batch size was 100 and the number of datapoints being generated was also 100 so at times, the batch processor would be sending the data before the timeout was reached, which is what this particular test is trying to test. To fix this, I'm increasing the batch send size to 101.
This PR is to rename `config.NewID` to `config.NewComponentID` and `config.NewIDFromString` to `config.NewComponentIDFromString` for naming consistency.
**Link to tracking Issue:**
Fixes#4105
* all: remove OtlpProtoSize in favor of Sizer interface
* add {Metrics|Traces|Logs}Sizer, fix test commenting
* move size tests to pb_test, fix metrics+logs parameter names
* uncommented batch_processor tests + adjusted them to use *Sizer
* I forgot BenchmarkTraceSizeBytes :(
* Add docs for NewProtobof*Sizer + update docs for NewProtobuf*Marshaler
* cast *Marshaler to *Sizer for now
* add casts to batch_processor_test
The new module go.opentelemetry.io/collector/model will be created later when the internal data will
also be moved.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
**Link to tracking Issue:** #2754
This change introduces a BenchmarkBatchMetricProcessor that stress tests batching logic.
Results before:
`BenchmarkBatchMetricProcessor-12 20000 80614 ns/op`
Results after the change:
`BenchmarkBatchMetricProcessor-12 20000 96184 ns/op`
* The reordering could happen if during processing of an item other items were added to the newItems channel. This is fixed by avoiding to re-add the left items to the channel.
* The logic of spliting was wrong by forcing sendBatchSize instead of sendBatchMaxSize when split was called.
* Fix logic if a very large message is received > 2x sendBatchMaxSize to call export multiple times instead of once and reset the timer.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
* Error checking on unit tests
* Add back license text
* Retrigger CI
* Wrap defer calls with `func`
Arguments are evaluated immediately on defer calls
* Update config/configauth/oidc_server_test.go
* Use t.Cleanup
* Change ioutil.ReadAll by resp.Body.Close()
* Retrigger CI (flaky tests)
I added a Shutdown() test that does basic verification of the behavior of the
Shutdown() function. More verifications can be added later.
The test revealed a bug in batchprocessor Shutdown() function which would
not wait until all pending data was drained.
Currently for slices of pointers, pdata exposes elements that are pointers to the pointer in the slice (double pointers in the internal implementation). Because of this users have to deal with possible nil values, have to initialize elements (in some cases) etc, but in reality the elements in the slice cannot be nil, so this just adds extra unnecessary complexity. This is possible because:
* Gogo proto (and protobuf) will not unmarshal any nil element in a slice;
* Our APIs to add elements/remove elements from the slice will guarantee that we never have a nil element in the slice between [0, len-1];
This is an important change because will allow us to change the internal representation (use slice of pointers or non-pointers) without affecting the public API.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
This is a preparation for enabling per component telemetry level:
* Views are global and they must always be installed; they are no-op if nobody records data;
* Views are global and they must always contain all the labels; In case of measurements being recorded without one of the labels (for perfomance reasons)
the extra label defined does not have impact, because all the timeseries will have that label as "null" (missing).
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
* Remove legacy metrics, they were marked as legacy for ~12 months
Fixes https://github.com/open-telemetry/opentelemetry-collector/issues/1082
This PR removes:
- Legacy receiver/exporter metrics:
- otelcol/receiver/received_spans
- otelcol/receiver/dropped_spans
- otelcol/receiver/received_timeseries
- otelcol/receiver/dropped_timeseries
- otelcol/exporter/received_spans
- otelcol/exporter/dropped_spans
- otelcol/exporter/received_timeseries
- otelcol/exporter/dropped_timeseries
- otelcol/exporter/received_logs
- otelcol/exporter/dropped_logs
- For processors remove the legacy metrics (new metrics have the same data, different names):
- e.g. "spans_dropped" -> "processor/spans_dropped"
- e.g. "batch_send_size_bytes" -> "processor/batch/batch_send_size_bytes"
All the new metrics were enabled when using the --new-metrics flag. The PR also removes two flags:
- "--new-metrics"
- "--legacy-metrics"
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
* Update changelog
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
* Fix comments from review
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Bogdan Drutu
Alex Boten
Joshua MacDonald
Daniel Jaglowski
Gustavo Paiva
Dmitrii Anoshin
Corbin Phelps
Nathan Vērzemnieks
Tigran Najaryan
Pablo Baeyens
James
Nathan Dias
kisieland
Patryk Matyjasek
Ben Keith