**Description:**
Following
https://github.com/open-telemetry/opentelemetry-collector/issues/8632,
this change introduces memory limiter as an extension. This allows
us to place the component to reject incoming connections due to limited
memory, providing better protection from running out of memory.
missing feature: receiver fairness. issue where a receiver hogs all the
resource can happen.
**Link to tracking Issue:** <Issue number if applicable>
https://github.com/open-telemetry/opentelemetry-collector/issues/8632
---------
Co-authored-by: Dmitry Anoshin <anoshindx@gmail.com>
* [chore] use license shortform
To remain consistent w/ contrib repo, see https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/22052
Signed-off-by: Alex Boten <aboten@lightstep.com>
* make goporto
Signed-off-by: Alex Boten <aboten@lightstep.com>
---------
Signed-off-by: Alex Boten <aboten@lightstep.com>
Contributes to https://github.com/open-telemetry/opentelemetry-collector/issues/1084
- Clarify what the memory limiter does.
- Set expectations from receivers, how they are supposed to react
when the memory limiter refuses the data.
- Add a test that demonstrates that memory limiter does not lose data
if the receiver and exporter behave according to the contract.
All receivers must adhere to this contract. See for example
an issue opened against filelog receiver:
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/20511
Note that there are no functional changes to the memory limiter.
Future work: one additional thing we can do is implement a backoff
logic in the memory limiter. When in memory limited mode the processor
can introduce pauses before it returns from the ConsumeLogs/Traces/Metrics
call. This will allow to slow down the inflow of data into the Collector
and give time for the pipeline to clear and memory usage to return to the
normal. This needs to be explored further.
The main reason is to remove the circular dependency between the config (including sub-packages) and component. Here is the current state:
* component depends on config
* config/sub-package[grpc, http, etc.] depends on config & component
Because of this "circular" dependency, we cannot split for example "config" into its own module, only if all the other config sub-packages are also split.
Signed-off-by: Bogdan <bogdandrutu@gmail.com>
* chore: adds porto and fixes vanity imports.
* chore: fixes target overriding.
* chore: fixes install of porto.
* chore: includes porto as a tool.
* chore: upgrades porto to check internals.
* chore: rebase and update vanity import.
* chore: removes unnecessary space.
* chore: rollsback vanity import in generated files.
splunkertimn
Dmitrii Anoshin
Alex Boten
Tigran Najaryan
Bogdan Drutu
José Carlos Chávez