<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
This upgrades protobuf/pdata to
[v1.7.0](https://github.com/open-telemetry/opentelemetry-proto/releases/tag/v1.7.0).
Note: due to the move of the lookup tables, merging profiles required
more non-trivial work.
In order to facilitate this review, merging is therefore currently
disabled for profiles. We will bring it back in a separate PR.
Note: this needs its contrib counterpart before it can be moved out of
draft.
Depends on
https://github.com/open-telemetry/opentelemetry-collector/pull/12856Resolves#12676
This is a reboot of #11311, incorporating metrics defined in the
[component telemetry
RFC](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/rfcs/component-universal-telemetry.md)
and attributes added in #12617.
The basic pattern is:
- When building any pipeline component which produces data, wrap the
"next consumer" with instrumentation to measure the number of items
being passed. This wrapped consumer is then passed into the constructor
of the component.
- When building any pipeline component which consumes data, wrap the
component itself. This wrapped consumer is saved onto the graph node so
that it can be retrieved during graph assembly.
---------
Co-authored-by: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Implements new name for attribute as documented in #12951.
Note: The obsconsumer package is unused until #12812 is merged so this
is not a breaking change.
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
<!-- Issue number if applicable -->
Adds a CodeCov status badge unless explicitly disabled.
I have disabled this in core until we roll this out in contrib to show
the Go SIG and move codecovgen to build tools
#### Link to tracking issue
Updates open-telemetry/opentelemetry-collector-contrib/issues/39583
Discussed offline in relation to #12812
Introduction of this gate had some unintended side effects, such as
removing attributes from loggers.
---------
Co-authored-by: Jade Guiton <jade.guiton@datadoghq.com>
Subset of #12812
This internal package defines wrappers around consumers. These are
useful for instrumenting the component graph, so that we can generate
telemetry describing data as it is passed in between components.
Currently, this supports only a single counter metric, but in the near
future it can be enhanced to automatically capture multiple metrics
(e.g. item count & size), and potentially spans and/or logs as well.
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
The GetFactory method is used by a few receivers in the contrib package
(e.g. receiver_creator). This PR adds a new interface to the
hostcapabilities package so it can be cast correctly from other
packages/components.
<!-- Issue number if applicable -->
#### Link to tracking issue
Fixes #
<!--Describe what testing was performed and which tests were added.-->
#### Testing
<!--Describe the documentation added.-->
#### Documentation
<!--Please delete paragraphs that you did not use before submitting.-->
---------
Co-authored-by: Jade Guiton <jade.guiton@datadoghq.com>
#### Description
Fork of #12384 to showcase how component attributes can be injected into
scope attributes instead of log/metric/span attributes. See that PR for
more context.
To see the diff from the previous PR, filter changes starting from the
"Prototype using scope attributes" commit.
#### Link to tracking issue
Resolves#12217
Also incidentally resolves#12213 and resolves#12117
#### Testing
I updated the existing tests to check for scope attributes, and did some
manual testing with a debug exporter to check that the scope attributes
are added/removed properly.
---------
Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Wraps the component ID and type into the component's start retuned
error. If `telemetry::logs::level` is < error (e.g fatal), the service
error message does not give information about which pipeline component
failed:
Current error message:
```
$ otelcontribcol --config config.yaml
Error: cannot start pipelines: start function error
2024/12/10 10:07:25 collector server run finished with error: cannot start pipelines: start function error
```
With these changes:
```
$ otelcontribcol --config config.yaml
Error: cannot start pipelines: failed to start geoip processor: start
function error
2024/12/10 09:58:29 collector server run finished with error: cannot
start pipelines: failed to start geoip processor: start function error
```
<!-- Issue number if applicable -->
#### Link to tracking issue
https://github.com/open-telemetry/opentelemetry-collector/issues/10426
<!--Describe what testing was performed and which tests were added.-->
#### Testing
<!--Describe the documentation added.-->
#### Documentation
<!--Please delete paragraphs that you did not use before submitting.-->
---------
Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
#### Description
Upgrading the collector to use opentelemetry-go-contrib
v1.35.0/v0.60.0/v0.29.0/v0.15.0/v0.10.0/v0.8.0/v0.7.0
See changelog here:
https://github.com/open-telemetry/opentelemetry-go-contrib/releases/tag/v1.35.0
<!-- Issue number if applicable -->
#### Link to tracking issue
none available
<!--Describe what testing was performed and which tests were added.-->
#### Testing
I ran `make all` but happy to run other targets if needed
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
create `service/hostcapabilities` package to expose `GetModuleInfos()`
from service/host. Also moves getExporters interface for
`GetExporters()` to this package.
<!-- Issue number if applicable -->
#### Link to tracking issue
Addresses planned work from #12296
<!--Describe what testing was performed and which tests were added.-->
#### Testing
none, creates interface for existing function
<!--Describe the documentation added.-->
#### Documentation
changelog yaml
<!--Please delete paragraphs that you did not use before submitting.-->
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
As mentioned in this
https://github.com/open-telemetry/opentelemetry-collector/issues/8721#issuecomment-1813468623,
the error message for unused connectors currently lacks specific
pipeline names, making debugging more difficult.
This PR enhances the error message by including pipeline names in the
`[signal/name]` format, consistent with how they appear in
`config.yaml`. This provides a better context for identifying
misconfigurations.
<!-- Issue number if applicable -->
#### Link to tracking issue
Related to #8721
<!--Describe what testing was performed and which tests were added.-->
#### Testing
A few scenarios and example output are given below. I will do additional
testing and add unit tests if necessary.
**1. Used as a receiver but not used as an exporter with 1 signal**
<details>
<summary><strong>config.yaml</strong></summary>
```yaml
receivers:
otlp:
protocols:
grpc:
exporters:
debug:
connectors:
forward:
service:
pipelines:
logs/in:
receivers: [otlp]
processors: []
exporters: [debug]
logs/out:
receivers: [forward]
processors: []
exporters: [debug]
```
</details>
Main Branch Output:
```
Error: failed to build pipelines: connector "forward" used as receiver in logs pipeline but not used in any supported exporter pipeline
```
Proposed Output:
```
Error: failed to build pipelines: connector "forward" used as receiver in [logs/out] pipeline but not used in any supported exporter pipeline
```
**2. Plain**
<details>
<summary><strong>config.yaml</strong></summary>
```yaml
receivers:
otlp:
protocols:
grpc:
exporters:
debug:
connectors:
forward:
service:
pipelines:
traces:
receivers: [ otlp ]
processors: [ ]
exporters: [ forward ]
metrics:
receivers: [ forward ]
processors: [ ]
exporters: [ debug ]
```
</details>
Main Branch Output:
```
Error: failed to build pipelines: connector "forward" used as exporter in traces pipeline but not used in any supported receiver pipeline
```
Proposed Output:
```
Error: failed to build pipelines: connector "forward" used as exporter in [traces] pipeline but not used in any supported receiver pipeline
```
**3. Multiple pipeline**
<details>
<summary><strong>config.yaml</strong></summary>
```yaml
receivers:
otlp:
protocols:
grpc:
exporters:
debug:
connectors:
forward:
service:
pipelines:
logs/in:
receivers: [otlp]
processors: []
exporters: [forward]
logs/in2:
receivers: [ otlp ]
processors: [ ]
exporters: [ forward ]
logs/out:
receivers: [otlp]
processors: []
exporters: [debug]
traces:
receivers: [ otlp ]
processors: [ ]
exporters: [ forward ]
metrics:
receivers: [ forward ]
processors: [ ]
exporters: [ debug ]
```
</details>
Main Branch Output:
```
Error: failed to build pipelines: connector "forward" used as exporter in logs pipeline but not used in any supported receiver pipeline
```
Proposed Output:
```
Error: failed to build pipelines: connector "forward" used as exporter in [logs/in2 logs/in] pipeline but not used in any supported receiver pipeline
```
---------
Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Add `component.Type` parameter to `NewNopSettings` And deprecate
`NewNopSettingsWithType` cc @mx-psi
<!-- Issue number if applicable -->
#### Link to tracking issue
Relevant to #12305
<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updated
<!--Describe the documentation added.-->
#### Documentation
Added
<!--Please delete paragraphs that you did not use before submitting.-->
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
fix metadata.yaml github_project, add a go:generate instruction to
confmap and a banner to README
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
<!-- Issue number if applicable -->
Creates `NewNopSettingsWithType` function on test modules and deprecates
the `NewNopSettings` functions on those modules. Replace all usages of
`NewNopSettings` with `NewNopSettingsWithType`.
Part of #12305 but applied to all component kinds.
#### Link to tracking issue
Updates #12221
#### Description
At the moment, `receiver`, `scraper`, `processor`, `exporter`, and
`extension` each have their own version of the `MakeFactoryMap`
function, which takes a list of Factories and returns it as a map, with
the component name as the key (and an error if there is overlap).
Because all versions are near-identical except for the Factory type
used, and this function is only used in tests and in the code generated
by OCB (`components.go`), it was suggested to make it generic and move
it to the `otelcol` package.
This PR does exactly that. The old `MakeFactoryMap` functions are
deprecated (and may be removed alongside their tests in a future
release). I also had to make a few other odd changes to fit the new
dependency graph.
#### Link to tracking issue
Resolves#12222
---------
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Moves `LoggerWithout` to an internal package so that we can keep on
experimenting on it while mark component 1.0 on v0.121.0.
This does not need a changelog since the change has not been released
<!-- Issue number if applicable -->
#### Link to tracking issue
Updates #12217
Implements the logger described in
https://github.com/open-telemetry/opentelemetry-collector/issues/12217
Alternative to #12057Resolves#11814
`component/componentattribute`:
- Initializes new module
- Defines constants for component telemetry attribute keys
- Defines a `zapcore.Core` which can remove attributes from the root
logger
`service`:
- Rebases component instantiation on attribute sets
- Internal constructors for attribute sets for each component type
- Constructs loggers from `componentattribute`
`otlpreceiver`:
- Uses `componentattribute` to remove `otelcol.signal` attribute from
logger
`memorylimiter`:
- Uses `componentattribute` to remove `otelcol.signal`,
`otelcol.pipeline.id` and `otelcol.component.id` attributes from logger
### This PR
- adds the githubgen tool as a dependency in internal/tools
- uses githubgen to generate codeowners and issue template files
- updates lots of metadata files by
- taking the existing codeowners file and feeding the info from there
back into the component metadata.yaml files or creating new
metadata.yaml files where none existed yet
- adds distributions.yaml as a basis the mostly already existing
`distributions:` keys in metadata.yaml files (needed for githubgen to
work correctly)
- adds relevant make commands to make the githubgen tool usage mostly
transparent to users
This change is a prerequisite to be able to ping codeowners reliably
with automated tooling as a next step.
Part of #11562
---------
Signed-off-by: Moritz Wiesinger <moritz.wiesinger@dynatrace.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
<!-- Issue number if applicable -->
Moves `extension.Settings.ModuleInfo` to a `service.Host.GetModuleInfo`
method.
This field probably fits better on `extension.Settings` or even in
`component.BuildInfo`, but since its contents are a bit in flux per
discussions like #12283, I would like to move it here to not stabilize
it alongside the rest of `extension`.
The field is not used in any public code on Github, so I think it won't
have a huge impact to remove it in one go, happy to do this in two steps
if preferred.
One relevant difference is that this information is no longer available
at extension build time, but rather after the `Start` method is called.
Another relevant difference is that this is now available for all
component kinds, not just extensions. This could be worked around (we
could pass a different host depending on the component kind) but I don't
see the use right now.
#### Link to tracking issue
Updates #12283
This is the last functionality needed to replace
`metadatatest.Telemetry` with `componenttest.Telemetry`.
The `metadatatest.Telemetry` will be deprecated next release to give
time to change contrib since it is a large change.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
This PR also solves one more issues with the new interface, which is the
ability to record multiple observations for a single async metric using
one callback.
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
A follow-up PR of #12179 , cc @bogdandrutu
Note: This PR also removes those metric assertions which is actually
testing metrics that doesn't belong to the package. They might be added
due to the limitation of the AssertMetrics function. e.g. changes in
`processor/memorylimitprocessor/memorylimiter_test.go`
<!-- Issue number if applicable -->
#### Link to tracking issue
Relevant to #12179
<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updated
<!--Describe the documentation added.-->
#### Documentation
n/a
<!--Please delete paragraphs that you did not use before submitting.-->
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Completion of #12167 , cc @bogdandrutu
Updated one test to verify the change, will update the rest ones after
this PR merges.
<!-- Issue number if applicable -->
#### Link to tracking issue
n/a
<!--Describe what testing was performed and which tests were added.-->
#### Testing
Added
<!--Describe the documentation added.-->
#### Documentation
Added
<!--Please delete paragraphs that you did not use before submitting.-->
This PR does a couple of things that I couldn't quite split up so I put
together a PR w/ individual commits to help reviewers get through it.
This PR does the following:
1. update `go.opentelemetry.io/contrib/config` package to latest. this
brings in breaking changes. in order to prevent those breaking changes
from impacting end users, i've also added a layer of config unmarshaling
2. updates the collector to instantiate the meter provider (and
exporters) via the config package. this allows us to remove all the code
in `otelinit`. the reason for including this change was that
unmarshaling the config was causing circular dependencies i didn't want
to address by moving code that could be deleted around.
Replacement for
https://github.com/open-telemetry/opentelemetry-collector/pull/11458.
Fixes
https://github.com/open-telemetry/opentelemetry-collector/issues/12021
---------
Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
<!-- Issue number if applicable -->
Removes `level` field. This will be added back in the future once #11754
is completed.
#### Link to tracking issue
Updates #11061
This PR creates a new internal package which defines the correct set of
attributes for each kind of component.
This is a subset of #12057 which is broken off in order to reduce the
overall size of that PR. As such, this package will not actually be used
until #12057 is merged.
Subset of #12057
This PR adds a test to validate the expected number of instances of each
component. This framework becomes more useful once singleton components
are explicitly supported.
This PR just cleans up some naming in the `testcomponents` package, so
that no component kind has a monopoly on the generic version of a
variable or function name.
Subset of #12057
#### Description
In #10058 I mentioned:
> There is a tangentially related issue with PermanentErrors and the
underlying finite state machine that governs transitions between
statuses. Currently, a PermanentError is a final state. That is, once a
component enters this state, no further transitions are allowed. In
light of the work I did on the alternative health check extension, I
believe we should allow a transition from PermanentError to Stopping to
consistently prioritize lifecycle events for components. This transition
also make sense from a practical perspective. A component in a
PermanentError state is one that has been started and is running,
although in a likely degraded state. The collector will call shutdown on
the component (when the collector is shutting down) and we should allow
the status to reflect that.
This PR makes the suggested change and updates the documentation to
reflect that. As this is an internal change, I have not included a
changelog. Also note, we can close#10058 after this as we've already
removed status aggregation from core during the recent component status
refactor.
<!-- Issue number if applicable -->
#### Link to tracking issue
Fixes#10058
<!--Describe what testing was performed and which tests were added.-->
#### Testing
units
<!--Describe the documentation added.-->
#### Documentation
Updated docs/component-status.md and associated diagram.
<!--Please delete paragraphs that you did not use before submitting.-->
Co-authored-by: Tyler Helmuth <12352919+TylerHelmuth@users.noreply.github.com>
Co-authored-by: Antoine Toulme <atoulme@splunk.com>
#### Description
[Thelper](https://golangci-lint.run/usage/linters/#thelper) detects
tests helpers which is not start with t.Helper() method.
This only enables rules for naming and paramter order
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Damien Mathieu
Bogdan Drutu
Daniel Jaglowski
Alex Boten
Pablo Baeyens
Roger Coll
Matthieu MOREL
Jade Guiton
Matthew Sainsbury
John L. Peterson (Jack)
Sudipto Baral
Chao Weng
Antoine Toulme
Moritz Wiesinger
Matthew Wear
Dmitrii Anoshin