Need to re-generate the new internal metrics, but it is too much code to change,
so will move this to dataold (was internal anyway, so no public breaking change).
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
As we are preparing for Beta release we want to cleanup publicly exported
types and interfaces and do all necessary refactoring and breaking changes
now, before the Beta release. We will have a lot less leeway for breaking
changes after the Beta release.
Component related type declarations are now all in the `component` package.
This makes it possible for the interfaces to reference each other. This
was were very restricted earlier because component interfaces were in 5
different packages and many proposals were impossible to implement because
they would result in circular dependencies between packages.
(An example upcoming new capability that is enabled by this refactoring
is for components to query the Host for other components and for factories).
List of changes in this commit:
- Move all factory interfaces and component interfaces to component package.
- Rename old factories and components interfaces to use "Old" suffix for clarity.
- Eliminate forced checks that components implement factories. This is already
enforced by the compiler when the factory is added to the Defaults() and
was unnecessary code.
- Eliminated some unnecessary codes (removed overall over 200 lines).
- Run `go mod tidy` on testbed.
Warning: this is a breaking change to publicly exported types and function
signatures. We announced that a breaking change is comming. Once we agree
to merge this commit we will need to announce the exact list of changes
and guide component authors to modify their components accordingly.
Future changes:
- Once all components are migrated to the new internal representation,
delete all "Old"-suffixed definitions. This will likely be done while
we are still in Beta phase, before the Stable release.
Initial updates to migrate processor metrics to obsreport package, ie.: the new metrics.
Cleaned-up a bit some of the processor metrics and spelled out the rule names for new metrics.
Related to https://github.com/open-telemetry/opentelemetry-collector/issues/141
Testing: Added test for the processor common metrics, validated manually that legacy metrics were still working
Change "batches_dropped" to a .Sum(), and emit 0s for them on processor start.
Motivation:
Currently with batches_dropped being a .Count() we end up with a missing metric for bad_batches until one occurs. This makes discovering the "error" metrics I want to watch kind of annoying as I can't just look in the default prom metric list and choose what I want to dashboard / alert on.
I also think that for things we KNOW are 0, we should be emitting a 0. If we send 8000 batches, the bad_batches shouldn't be absent it should be 0. I do think that view.Count()'s should be initialized to 0 as well ( but I may not know the whole story there. )
I can add these to the other processors if we think this is a good idea. For now I just hoped the metric name is correct in my current production dashboards.
Testing:
I added some happy path metric tests to the processor. I can figure out a way to add bad path tests, it will just require a bunch of plumbing I think.
Documentation: I think the metrics as a whole need better documentation, for example I "fail_sends" in the exporter wasn't actually an error. Maybe a distinction between data loss events or not?
This is part 1 of the task to introduce capabilities and declare
processor's intent to mutate or not mutate consumed data for the
purpose of optimizing pipeline data ownership.
If a processor declares that they mutate data the pipeline
that the processor is in will use cloning fan out connector.
This is done only if the pipeline shares a receiver with another
pipeline.
This ensures that it is safe for processor modify the data (because
pipelines work concurrently) and avoids cloning for pipelines that
consume data from the same receiver but do not modify it.
For more details see:
https://github.com/open-telemetry/opentelemetry-collector/issues/372
Bogdan Drutu
Jay Camp
Pavol Loffay
Bruno Garcia
Dmitrii Anoshin
Tigran Najaryan
Paulo Janotti
Ian Quick
Ruslan Abelharisov
Yang Song