#### Description
There are two main changes here:
- We move the componentattribute logger core wrapping and Logger to
LoggerProvider tee'ing to the service package. This will enable the
wrapping to be performed independently of the implementation of
telemetry providers.
- We move away from using zapcore.NewTeeCore, and introduce a new
zapcore.Core implementation that only copies log entries accepted by the
native Zap core to the OTel LoggerProvider.
The second change is necessary due to the first: because the
LoggerProvider will in the future be injectable, and because the
telemetry configuration will be opaque to the service package, we must
always tee the logs from the zap logger to the LoggerProvider. When
using zapcore.NewTeeCore, the resulting core will duplicate all entries
to all cores - which is not what we want.
There's also some light refactoring of tests, and the service package
tests are now ~30s faster by replacing a sleep with assert.Eventually.
#### Link to tracking issue
Preparation work for
https://github.com/open-telemetry/opentelemetry-collector/issues/4970
#### Testing
Unit tests pass
#### Documentation
N/A
---------
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
#### Context
PR #12617, which implemented the injection of component-identifying
attributes into the `zap.Logger` provided to components, introduced
significant additional memory use when the Collector's pipelines contain
many components (#13014). This was because we would call
`zapcore.NewSamplerWithOptions` to wrap the specialized logger core of
each Collector component, which allocates half a megabyte's worth of
sampling counters.
This problem was mitigated in #13015 by moving the sampling layer to a
different location in the logger core hierarchy. This meant that
Collector users that do not export their logs through OTLP and only use
stdout-based logs no longer saw the memory increase.
#### Description
This PR aims to provide a better solution to this issue, by using the
`reflect` library to clone zap's sampler core and set a new inner core,
while reusing the counter allocation.
(This may also be "more correct" from a sampling point of view, ie. we
only have one global instance of the counters instead of one for console
logs and one for each component's OTLP-exported logs, but I'm not sure
if anyone noticed the difference anyway).
#### Link to tracking issue
Fixes#13014
#### Testing
A new test was added which checks that the log counters are shared
between two sampler cores with different attributes.
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Avoid re-creating sampler counters every time we wrap with attributes.
<!-- Issue number if applicable -->
#### Link to tracking issue
Updates #13014
<!--Describe what testing was performed and which tests were added.-->
#### Testing
<!--Describe the documentation added.-->
#### Documentation
<!--Please delete paragraphs that you did not use before submitting.-->
---------
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Co-authored-by: Jade Guiton <jade.guiton@datadoghq.com>
#### Description
Propagate `telemetry.resource` to the internal logs.
#### Link to tracking issue
Fixes#12582
Signed-off-by: Israel Blancas <iblancasa@gmail.com>
Discussed offline in relation to #12812
Introduction of this gate had some unintended side effects, such as
removing attributes from loggers.
---------
Co-authored-by: Jade Guiton <jade.guiton@datadoghq.com>
#### Description
Fork of #12384 to showcase how component attributes can be injected into
scope attributes instead of log/metric/span attributes. See that PR for
more context.
To see the diff from the previous PR, filter changes starting from the
"Prototype using scope attributes" commit.
#### Link to tracking issue
Resolves#12217
Also incidentally resolves#12213 and resolves#12117
#### Testing
I updated the existing tests to check for scope attributes, and did some
manual testing with a debug exporter to check that the scope attributes
are added/removed properly.
---------
Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
#### Description
[gofumpt](https://golangci-lint.run/usage/linters/#gofumpt) is a
stricter format than gofmt, while being backwards compatible.
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
OTel collector log messages in Windows are currently displayed in
scientific notation, e.g.:
```
1.7194132367192364e+09 error scraperhelper/scrapercontroller.go:197 Error scraping metrics …
```
This fix will cause the log messages to use ISO8601 timestamps instead.
<!--Describe what testing was performed and which tests were added.-->
#### Testing
Manual testing.
<!--Describe the documentation added.-->
#### Documentation
The previous behavior was undocumented, so no documentation changes
needed.
<!--Please delete paragraphs that you did not use before submitting.-->
---------
Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com>
#### Description
[whitespace](https://golangci-lint.run/usage/linters/#whitespace) is a
linter that checks for unnecessary newlines at the start and end of
functions.
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
The tracer and logger provider were instantiating different resources
objects that didn't have a `service.instance.id` attribute. This change
fixes that by instantiating the SDK in the service and passing it to the
factory via the telemetry.Settings struct.
This also removes the duplicate code to instantiate the SDK multiple
times, which will be useful when we move to instantiating MeterProvider
via the config SDK. This bug is blocking the change to bump up the
dependency on the config package.
There are a few alternatives that were considered:
1. could set the resource's service.instance.id on the config object
instead. this seemed messy as it would be editing the config struct and
the instantiation of the SDK would remain duplicated.
2. update the factory to pass in a resource object option, i didn't want
to update the NewFactory func.
3. update the CreateLogger, CreateTracerProvider to receive either a
resource or sdk parameter, both of those seemed incorrect as well.
---------
Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com>
This allows us to use the otel-go/config package to support configuring
external destinations for logs. I'm putting this in draft to gather
community feedback on whether this is a desirable feature for the
collector.
I used the following configuration with this PR to send data to an OTLP
backend:
```yaml
telemetry:
logs:
processors:
- batch:
exporter:
otlp:
protocol: http/protobuf
endpoint: https://api.honeycomb.io:443
headers:
"x-honeycomb-team": "${env:HONEYCOMB_API_KEY}"
```
This allowed me to see logs in my backend:
---------
Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Switches `service/telemetry` to a factory pattern. To avoid adding a lot
of public API in one go:
1. the actual factory builder is in an internal package
2. I have not added the `CreateMeterProvider` method yet
There are two goals with this: one is to make progress on #4970, the
other is to allow initializing telemetry sooner:
<!-- Issue number if applicable -->
#### Link to tracking issue
Updates #4970.
<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updates existing tests to use `NewFactory`
Andrew Wilkins
Jade Guiton
Israel Blancas
Bogdan Drutu
Daniel Jaglowski
Matthieu MOREL
igorpeshansky
Alex Boten
Pablo Baeyens