* Remove legacy metrics, they were marked as legacy for ~12 months
Fixes https://github.com/open-telemetry/opentelemetry-collector/issues/1082
This PR removes:
- Legacy receiver/exporter metrics:
- otelcol/receiver/received_spans
- otelcol/receiver/dropped_spans
- otelcol/receiver/received_timeseries
- otelcol/receiver/dropped_timeseries
- otelcol/exporter/received_spans
- otelcol/exporter/dropped_spans
- otelcol/exporter/received_timeseries
- otelcol/exporter/dropped_timeseries
- otelcol/exporter/received_logs
- otelcol/exporter/dropped_logs
- For processors remove the legacy metrics (new metrics have the same data, different names):
- e.g. "spans_dropped" -> "processor/spans_dropped"
- e.g. "batch_send_size_bytes" -> "processor/batch/batch_send_size_bytes"
All the new metrics were enabled when using the --new-metrics flag. The PR also removes two flags:
- "--new-metrics"
- "--legacy-metrics"
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
* Update changelog
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
* Fix comments from review
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Fixes https://github.com/open-telemetry/opentelemetry-collector/issues/1888
Previously the zipkin receiver parses string tags to other datatypes (int/bool/float), this adds the functionality back under a new configuration option `parse_string_tags` in the zipkin receiver.
To unify the behavior with the zipkin v1 I also disabled the parsing by default and check the same configuration option.
* fixed all Jaeger translation diffs from correctness tests
* fixed Jaeger translations where Resource is missing or empty
* improve test coverage
* initial dev of otlp to zipkin translation
* initial dev of otlp to zipkin translation
* initial dev of zipkin v2 to otlp translation
* initial dev of zipkin v2 to otlp translation
* initial dev of zipkin v2 to otlp translation
* initial dev of zipkin v2 to otlp translation
* initial dev of zipkin v1 to otlp translation
* change zipkin receiver and exporter to use internal traces
* improve event/annotation translations
* fix broken zipkin tests
* convert zipken receiver/exporter to new factory signature
* fixed receiver config
* add code from master
* fix broken test
* fix PR requested changes
* translation fixes
* improve perf
* added more test coverage
* changed timestamp validations to only check to nearest millisecond
* fix new lint issues and PR comments
* improve timestamp validation
* improve timestamp validation
* fix missed timestamp validation
* added missing goldendata examples
* fix generator tests
* fix PR comments
* improve test coverage
Fixes https://github.com/open-telemetry/opentelemetry-collector/issues/975
Please look at the individual commits, most of this is just moving code around.
Moves zipkin v2 trace conversion code into translator/trace/zipkin, previously it was in the receiver
Use the same tag parsing logic for both zipkin v1 and v2
While zero IDs are considered by most formats to be an error the Collector should not drop data because of that. This follows the same pattern already adopted for the Jaeger receiver.
As we are preparing for Beta release we want to cleanup publicly exported
types and interfaces and do all necessary refactoring and breaking changes
now, before the Beta release. We will have a lot less leeway for breaking
changes after the Beta release.
Component related type declarations are now all in the `component` package.
This makes it possible for the interfaces to reference each other. This
was were very restricted earlier because component interfaces were in 5
different packages and many proposals were impossible to implement because
they would result in circular dependencies between packages.
(An example upcoming new capability that is enabled by this refactoring
is for components to query the Host for other components and for factories).
List of changes in this commit:
- Move all factory interfaces and component interfaces to component package.
- Rename old factories and components interfaces to use "Old" suffix for clarity.
- Eliminate forced checks that components implement factories. This is already
enforced by the compiler when the factory is added to the Defaults() and
was unnecessary code.
- Eliminated some unnecessary codes (removed overall over 200 lines).
- Run `go mod tidy` on testbed.
Warning: this is a breaking change to publicly exported types and function
signatures. We announced that a breaking change is comming. Once we agree
to merge this commit we will need to announce the exact list of changes
and guide component authors to modify their components accordingly.
Future changes:
- Once all components are migrated to the new internal representation,
delete all "Old"-suffixed definitions. This will likely be done while
we are still in Beta phase, before the Stable release.
Previously these span kinds were not supported because OpenCensus
does not have equivalent span kinds and they were translated as
"Unspecified" span kind when received by Zipkin receiver.
An earlier fix added this support for Zipkin V1 format. This fix
extends the support to Zipkin V2 format.
Testing done:
- Added a unit test to verify translations between ZipkinV2 and OC.
Client package will help identify clients connecting to different
receivers and enable usecases where processors might depend on some
information about the client. For example, a rate-limiting processor
might need some information about clients to figure out when and how to
rate-limit. Also, the upcoming kubernetes processor depends on this as
it needs the IP address of the clients to properly identify resources.
Submitting this as an independent PR as it is generic enough and not
tied to any single processor or usecase.
This PR adds support to the OC, Zipkin and Jaeger GRPC receivers. I'll implement the
same feature for older jaeger receivers in a follow up PR.
This change fixes inconsistencies in component interfaces. Motivation:
- Uniformness results in reduction of code that currently has to
deal with differences.
- Processor.Start is missing and is important for allowing processors
to communicate with the Host.
What's changed:
- Introduced Component interface.
- Unified Host interface.
- Added a Start function to processors (via Component interface).
- Start/Shutdown is now called for Processors from service start/shutdown.
- Receivers, Exporters, Processors, Extensions now embed Component interface.
- Replaced StartTraceReception/StartMetricsReception by single Start function for receivers.
- Replaced StopTraceReception/StopMetricsReception by single Shutdown function for receivers.
Note: before merging this we need to announce the change in Gitter since it
breaks existing implementations in contrib (although the fix is easy).
Resolves#477Resolves#262
This allows other to import and extend the zipkin receiver for different
use cases. For example, people can now wrap the server handler and add
HTTP middlewares to it.
* Improve batching for Zipkin spans
The local end point of zipkin typically includes the ip address and that can have a very high cardinality making batching inefficient since the batch processor groups according to the node configuration. This change stops storing this information on the node and put it on the span attributes. The zipkin exporter recognize the attributes and it is capable of returning local and remote endpoint of a span to its original values.
Fixed zipkin receiver TestConversionRoundtrip: the function testutils.GenerateNormalizedJSON was not checking for errors when marshaling/unmarshaling and the TestConversionRoundtrip was passing because both JSON objects were "null" what was not intended. This changes fixes the test and the helper function so similar doesn't repeat for other tests.
Since the dependency with zipkin exporter of OpenCensus was removed 'go mod tidy' was also executed for root and testbed.
* Do not change trace data when exporting to zipkin
This keeps the internal trace data unchanged. In order to avoid redundancy on zipkin spans the data that can be represented as local or remote zipkin endpoint is removed only from zipkin spans.
* Add few comments and explain a hard coded value on make
* Simplify attribute lookup
* Add zipkin Format to TestConversionRoundtrip
* Remove memory limiting changes
We paused on that until further design conversations, in order to avoid confusion with this incomplete feature let's remove it from the code base.
* Recover test that covered more than back presssure
* gofmt config_test.go
* Make Zipkin receiver respond to host ingestion status
This change makes the Zipkin receiver respond to received data according to the host status regarding ingestion.
* Nit cleanup
* Use better type to handle back pressure setting
* Refactoring: remove old OC Collector code
Initial phase of removal of old OC code that is still present on the repo.
* Rename package application to service
Add Receiver Host Interface
This interface allows better communication between receivers and the code hosting them. This becomes important when the host needs to request receivers to stop ingestion due to limited resources.
* Change census-instrumentation to open-telemetry and update authors
census-instrumentation/opencensus is now open-telemetry/opentelemetry
"OpenCensus Authors" is now "OpenTelemetry Authors"
"Copyright 2018" is now "Copyright 2019"
Fix go fmt
* Make all receivers take "next" on creation
This is required in order to move the code to use factories for the receivers. The "next" parameter will be passed to the factory and then to the receiver constructor, this way Start[Trace|Metrics]Reception doesn't need the next parameter anymore. Since typically the Start* methods launch goroutines took the opportunity to add a channel to report asynchronous errors.
Opportunistic change: renamed WrapWithSpanSink and related variables.
Remaing items from previous commit
* Improve test coverage for zipkin-scribe
Improving code coverage for zipkin-scribe that went a tad down with the initial PR.
* Improve a little code coverage zipkin receiver
* Fix test broken during rebase
* Rename [Trace|Metrics]DataProcessor to [Trace|Metrics]Processor
* Rename TraceProcessor to TraceConsumer as discussed.
* Add empty_test for processor.
* Add tests for [trace|span]_id conversion from ints, and change code to use this.
* Add tests for the returned errors.
* Fix comments for public functions in big_endian_convertor.
* Fix typo.
* For receivers and exporters record number of received and dropped spans.
* Shorter tag names, unify tags between metrics/trace.
* Record when errors, add TODOs for better handling errors.
Bogdan Drutu