<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
<!-- Issue number if applicable -->
Adds a CodeCov status badge unless explicitly disabled.
I have disabled this in core until we roll this out in contrib to show
the Go SIG and move codecovgen to build tools
#### Link to tracking issue
Updates open-telemetry/opentelemetry-collector-contrib/issues/39583
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
This RFC will help us explore the use cases Optional types solve and
move us toward a resolution for whether to adopt these in our config.
Related to #10266
---------
Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
Co-authored-by: Pablo Baeyens <pablo.baeyens@datadoghq.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
fix metadata.yaml github_project, add a go:generate instruction to
confmap and a banner to README
### This PR
- adds the githubgen tool as a dependency in internal/tools
- uses githubgen to generate codeowners and issue template files
- updates lots of metadata files by
- taking the existing codeowners file and feeding the info from there
back into the component metadata.yaml files or creating new
metadata.yaml files where none existed yet
- adds distributions.yaml as a basis the mostly already existing
`distributions:` keys in metadata.yaml files (needed for githubgen to
work correctly)
- adds relevant make commands to make the githubgen tool usage mostly
transparent to users
This change is a prerequisite to be able to ping codeowners reliably
with automated tooling as a next step.
Part of #11562
---------
Signed-off-by: Moritz Wiesinger <moritz.wiesinger@dynatrace.com>
### Context
The [Pipeline Component Telemetry
RFC](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/rfcs/component-universal-telemetry.md)
was recently accepted (#11406). The document states the following
regarding error monitoring:
> For both [consumed and produced] metrics, an `outcome` attribute with
possible values `success` and `failure` should be automatically
recorded, corresponding to whether or not the corresponding function
call returned an error. Specifically, consumed measurements will be
recorded with `outcome` as `failure` when a call from the previous
component the `ConsumeX` function returns an error, and `success`
otherwise. Likewise, produced measurements will be recorded with
`outcome` as `failure` when a call to the next consumer's `ConsumeX`
function returns an error, and `success` otherwise.
[Observability requirements for stable pipeline
components](https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#observability-requirements)
were also recently merged (#11772). The document states the following
regarding error monitoring:
> The goal is to be able to easily pinpoint the source of data loss in
the Collector pipeline, so this should either:
> - only include errors internal to the component, or;
> - allow distinguishing said errors from ones originating in an
external service, or propagated from downstream Collector components.
Because errors are typically propagated across `ConsumeX` calls in a
pipeline (except for components with an internal queue like
`processor/batch`), the error observability mechanism proposed by the
RFC implies that Pipeline Telemetry will record failures for every
component interface upstream of the component that actually emitted the
error, which does not match the goals set out in the observability
requirements, and makes it much harder to tell which component errors
are coming from from the emitted telemetry.
### Description
This PR amends the Pipeline Component Telemetry RFC with the following:
- restrict the `outcome=failure` value to cases where the error comes
from the very next component (the component on which `ConsumeX` was
called);
- add a third possible value for the `outcome` attribute: `rejected`,
for cases where an error observed at an interface comes from further
downstream (the component did not "fail", but its output was
"rejected");
- propose a mechanism to determine which of the two values should be
used.
The current proposal for the mechanism is for the pipeline
instrumentation layer to wrap errors in an unexported `downstream`
struct, which upstream layers could check for with `errors.As` to check
whether the error has already been "attributed" to a component. This is
the same mechanism currently used for tracking permanent vs. retryable
errors. Please check the diff for details.
### Possible alternatives
There are a few alternatives to this amendment, which were discussed as
part of the observability requirements PR:
- loosen the observability requirements for stable components to not
require distinguishing internal errors from downstream ones → makes it
harder to identify the source of an error;
- modify the way we use the `Consumer` API to no longer propagate errors
upstream → prevents proper propagation of backpressure through the
pipeline (although this is likely already a problem with the `batch`
prcessor);
- let component authors make their own custom telemetry to solve the
problem → higher barrier to entry, especially for people wanting to
opensource existing components.
---------
Co-authored-by: Pablo Baeyens <pablo.baeyens@datadoghq.com>
<!--Describe the documentation added.-->
#### Documentation
This is a documentation-only change to fix some typos in the Pipeline
Component Telemetry RFC doc.
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
<!-- Issue number if applicable -->
Proposal for creating a new `collector-release-approvers` group.
Announced at:
- #otel-collector-dev on 2024-10-30:
https://cloud-native.slack.com/archives/C07CCCMRXBK/p1730307025302339
- Collector SIG on 2024-11-05 (TBD)
The stakeholders for this PR are:
- @open-telemetry/collector-approvers
- @open-telemetry/collector-contrib-approvers
---------
Co-authored-by: Andrzej Stencel <andrzej.stencel@elastic.co>
This was done in the [specification
repo](6c626defb7),
and allows us to use a github action instead of install npm packages as
part of the build process (which kept bringing security warnings back)
Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Based on the experience of this process going through once, I am
suggesting we make the following changes:
1. Require announcing the final comment period via a comment and on the
#otel-collector-dev channel.
2. Require that all conversations are resolved. Allow RFC author to
resolve any conversation, but require them to leave a comment explaining
why
<!-- Issue number if applicable -->
#### Link to tracking issue
Fixes#11737
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
This PR adds a RFC for normalized telemetry across all pipeline
components. See
https://github.com/open-telemetry/opentelemetry-collector/issues/11343
edit by @mx-psi:
- Announced on #otel-collector-dev on 2024-10-23:
https://cloud-native.slack.com/archives/C07CCCMRXBK/p1729705290741179
- Announced on the Collector SIG meeting from 2024-10-30
---------
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
Co-authored-by: Damien Mathieu <42@dmathieu.com>
Co-authored-by: William Dumont <william.dumont@grafana.com>
Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
#### Description
Define RFC merge process in the `docs/rfc/README.md`. This is meant to
be a tool for making decisions about hard stuff :)
---------
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
#### Description
This is a stripped-down version of
https://github.com/open-telemetry/opentelemetry-collector/pull/10121
that eschews implementation details in favor of answering questions
about _what_ we want to achieve instead of how we will achieve it. A
section from that PR has been reproduced here to cover a few approaches
for future discussions.
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
<!-- Issue number if applicable -->
- Adds new `expandedValue` struct that holds the original string
representation if available for values resolved from a provider.
- Removes any mention of `expandedValue` in the public API by adding a
`sanitize` step before returning any `Get`s or `ToStringMap`s.
- Adds new decoding hook that checks if the target field is of `string`
type and uses the string representation in that case.
#### Link to tracking issue
Fixes#10605, Fixes#10405, Fixes#10659
<!--Describe what testing was performed and which tests were added.-->
#### Testing
<!--Describe the documentation added.-->
This changes the behavior in some cases, I update the test cases.
#### Documentation
<!--Please delete paragraphs that you did not use before submitting.-->
| ENV value | ${ENV} before unification | ${ENV} in v0.105.0 (also
${env:ENV} before unification) | Value after this PR |
|----------------------------|----------------------------|---------------------------------------------------------|----------------------------|
| foo\nbar | foo\nbar | foo bar | foo\nbar |
| 1111:1111:1111:1111:1111:: | 1111:1111:1111:1111:1111:: | **Error** |
1111:1111:1111:1111:1111:: |
| "0123" | "0123" | 0123 | "0123" |
**Description:** Adds an RFC about how environment variable resolution
should work
**Link to tracking Issue:** Fixes#9515, relates to:
- #8215
- #8565
- #9162
- #9531
- #9532
---------
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
Roger Coll
Pablo Baeyens
Daniel Jaglowski
Evan Bradley
Joshua MacDonald
Bogdan Drutu
Antoine Toulme
Moritz Wiesinger
Jade Guiton
Curtis Robert
Alex Boten
Nathan Baulch
Tyler Helmuth
Damien Mathieu