Commit Graph

19 Commits

Author SHA1 Message Date
Daniel Jaglowski dc8e2ddd9c
Permanently enable 'telemetry.newPipelineTelemetry' feature gate (#12856)
Discussed offline in relation to #12812

Introduction of this gate had some unintended side effects, such as
removing attributes from loggers.

---------

Co-authored-by: Jade Guiton <jade.guiton@datadoghq.com>
2025-04-17 15:57:35 +00:00
Jade Guiton 54c13a9217
Inject component-identifying scope attributes (#12617)
#### Description

Fork of #12384 to showcase how component attributes can be injected into
scope attributes instead of log/metric/span attributes. See that PR for
more context.

To see the diff from the previous PR, filter changes starting from the
"Prototype using scope attributes" commit.

#### Link to tracking issue
Resolves #12217 
Also incidentally resolves #12213 and resolves #12117

#### Testing
I updated the existing tests to check for scope attributes, and did some
manual testing with a debug exporter to check that the scope attributes
are added/removed properly.

---------

Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
2025-03-28 12:15:25 +00:00
Israel Blancas 4397725dff
[processor/memorylimiter] Add profiles support to memorylimiter processor (#12454)
#### Description
Add profiles support to the memorylimiter processor.

#### Link to tracking issue
Fixes #12453

Signed-off-by: Israel Blancas <iblancasa@gmail.com>
2025-02-27 12:53:19 +00:00
Chao Weng 6de29ce169 Add `component.Type` parameter to `NewNopSettings` And deprecate `NewNopSettingsWithType` (#12452)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Add `component.Type` parameter to `NewNopSettings` And deprecate
`NewNopSettingsWithType` cc @mx-psi

<!-- Issue number if applicable -->
#### Link to tracking issue
Relevant to #12305 

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updated

<!--Describe the documentation added.-->
#### Documentation
Added

<!--Please delete paragraphs that you did not use before submitting.-->
2025-02-21 11:17:45 +00:00
Pablo Baeyens a4ae175111
[*test] Add NewNopSettingsWithType everywhere (#12357)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

<!-- Issue number if applicable -->

Creates `NewNopSettingsWithType` function on test modules and deprecates
the `NewNopSettings` functions on those modules. Replace all usages of
`NewNopSettings` with `NewNopSettingsWithType`.

Part of #12305 but applied to all component kinds.

#### Link to tracking issue
Updates #12221
2025-02-13 16:56:20 +00:00
Pablo Baeyens f87b93a8f9
[chore] Move LoggerWithout temporarily to internal package (#12334)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Moves `LoggerWithout` to an internal package so that we can keep on
experimenting on it while mark component 1.0 on v0.121.0.

This does not need a changelog since the change has not been released

<!-- Issue number if applicable -->
#### Link to tracking issue
Updates #12217
2025-02-12 10:35:58 +00:00
Daniel Jaglowski 5d5fb21acf
Introduce component logger with appropriate attributes (#12259)
Implements the logger described in
https://github.com/open-telemetry/opentelemetry-collector/issues/12217

Alternative to #12057

Resolves #11814

`component/componentattribute`:
- Initializes new module
- Defines constants for component telemetry attribute keys
- Defines a `zapcore.Core` which can remove attributes from the root
logger

`service`:
- Rebases component instantiation on attribute sets
- Internal constructors for attribute sets for each component type
- Constructs loggers from `componentattribute`

`otlpreceiver`:
- Uses `componentattribute` to remove `otelcol.signal` attribute from
logger

`memorylimiter`:
- Uses `componentattribute` to remove `otelcol.signal`,
`otelcol.pipeline.id` and `otelcol.component.id` attributes from logger
2025-02-06 16:53:20 +00:00
Bogdan Drutu 98230db9d3
Deprecate funcs that repeate processor in name (#11310)
Similar with
https://github.com/open-telemetry/opentelemetry-collector/pull/11287

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
2024-09-30 15:31:44 -07:00
Matthieu MOREL 37f783308e
[chore]: enable require-error rule from testifylint (#11199)
#### Description

Testifylint is a linter that provides best practices with the use of
testify.

This PR enables
[require-error](https://github.com/Antonboom/testifylint?tab=readme-ov-file#require-error)
rule from [testifylint](https://github.com/Antonboom/testifylint)

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-09-18 15:02:22 -07:00
Alex Boten 9907ba50df
[processor] deprecate CreateSettings -> Settings (#10336)
This deprecates CreateSettings in favour of Settings.
NewNopCreateSettings is also being deprecated in favour of
NewNopSettings
    
Part of #9428

---------

Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com>
2024-06-06 09:34:53 -07:00
splunkertimn a85b37df73
[extension/memorylimiter] add memory limiter extension, a copy of memorylimiter processor (#8964)
**Description:**

Following
https://github.com/open-telemetry/opentelemetry-collector/issues/8632,
this change introduces memory limiter as an extension. This allows
us to place the component to reject incoming connections due to limited
memory, providing better protection from running out of memory.

missing feature: receiver fairness. issue where a receiver hogs all the
resource can happen.

**Link to tracking Issue:** <Issue number if applicable>
 https://github.com/open-telemetry/opentelemetry-collector/issues/8632

---------

Co-authored-by: Dmitry Anoshin <anoshindx@gmail.com>
2024-01-23 10:44:46 -08:00
Dmitrii Anoshin b81d4efd05
[processor/memory_limiter] Update config validation (#9059)
- Fix names of the config fields that are validated in the error
messages
  - Move the validation from start to the initialization phrase
2023-12-12 10:13:56 -08:00
Alex Boten 80d704deb4
[chore] use license shortform (#7694)
* [chore] use license shortform

To remain consistent w/ contrib repo, see https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/22052

Signed-off-by: Alex Boten <aboten@lightstep.com>

* make goporto

Signed-off-by: Alex Boten <aboten@lightstep.com>

---------

Signed-off-by: Alex Boten <aboten@lightstep.com>
2023-05-18 13:11:17 -07:00
Daniel Jaglowski fdef8b181a
Deprecate `Processor` related structs and functions in favor of `processor` package (#6709) 2022-12-09 08:09:27 -08:00
Bogdan d6ed8246b6 Deprecate all types and funcs in config package
The main reason is to remove the circular dependency between the config (including sub-packages) and component. Here is the current state:
* component depends on config
* config/sub-package[grpc, http, etc.] depends on config & component

Because of this "circular" dependency, we cannot split for example "config" into its own module, only if all the other config sub-packages are also split.

Signed-off-by: Bogdan <bogdandrutu@gmail.com>
2022-11-05 10:04:49 -07:00
Bogdan Drutu e8dd9536f5
Enable errorlint and fix all warnings (#5364)
On top of the errorlint errors, also changes `fmt.Errorf("string literal")` with `errors.New("string literal")`.

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
2022-05-16 10:36:41 -07:00
Dani Louca 52d651861c
[processor/memorylimiter] Only drops traces, not logs or metrics
Signed-off-by: Dani Louca <dlouca@splunk.com>

**Description:** 

This change 062e64f1d7 caused the memory limiter to "only" start the `checkMemLimits` routine for the ml instance used by the traces processor . 
In other words, metrics and logs processor will NOT [drop/refuse](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiterprocessor/memorylimiter.go#L206) data and will pass them down to the next consumer regardless of the current memory pressure as their instance of ml->forcingDrop will not be set.

The simplest solution, is to call start for each processor (metrics, logs, traces) , but this will not be efficient as we'll be running 3 instances of `checkMemLimits`, ie: multiple GC .
But at the same we need to allow multiple instances, with different configs, example: `memory_limiter/another` and `memory_limiter`

````
extensions:
  memory_ballast:
    size_mib: 4

receivers:
  otlp:
    protocols:
      grpc:
      http:
processors:
  memory_limiter:
    check_interval: 2s
    limit_mib: 10

  memory_limiter/another:
    check_interval: 1s
    limit_mib: 100
exporters:
  logging:
    logLevel: info

service:
  telemetry:
    logs:
      level: "info"
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [logging]
    metrics/default:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [logging]
    traces:
      receivers: [otlp]
      processors: [memory_limiter/another]
      exporters: [logging]
  extensions: [memory_ballast]
````

The fix adds a global map to keep track of the different instance and add ~~sync once~~ mutex for the start and shutdown call, so only the first processor can launch the `checkMemLimits` routine and the last one to call `shutdown` to take it down.
If shutdown was called and no `checkMemLimits` has started, then we'll return an error message; unit tests were updated to handle this.


**Testing:** 
Tested with above config and using splunk otel instance with valid data.
Made sure only a single `checkMemLimits` is running when there is a single config for memory-limiter and more than one when we have multiple.
I also verified that under memory pressure, when we pass the soft limit, all data types, traces, logs and metrics are getting dropped.

One we agree on this solution, I will look into adding more unit test to validate the change
2022-02-24 13:13:32 -05:00
Bogdan Drutu 0b4e5ec792
memorylimiterprocessor: Remove deprecated config (already no-op) ballast_size_mib (#4365)
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
2021-11-04 16:49:54 -07:00
alrex 3577bd05d4
rename memorylimiter -> memorylimiterprocessor (#4064) 2021-09-17 03:01:07 -07:00