opentelemetry-collector/exporter
Tyler Helmuth 77bb849aa0
[component] Refactor to use pipeline.ID and pipeline.Signal (#11204)
#### Description
Depends on
https://github.com/open-telemetry/opentelemetry-collector/pull/11209

This PR is a non-breaking implementation of
https://github.com/open-telemetry/opentelemetry-collector/pull/10947. It
adds a new module, `pipeline`, which houses a `pipeline.ID` and
`pipeline.Signal`. `pipeline.ID` is used to identify a pipeline within
the service. `pipeline.Signal` is uses to identify the signal associated
to a pipeline.

I do this work begrudgingly. As the PR shows, this is a huge refactor
when done in a non-breaking way, will require 3 full releases, and
doesn't benefit our [End Users or, in my opinion, our Component
Developers or Collector Library
Users](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CONTRIBUTING.md#target-audiences).
I view this refactor as a Nice-To-Have, not a requirement for Component
1.0.

<!-- Issue number if applicable -->
#### Link to tracking issue
Works towards
https://github.com/open-telemetry/opentelemetry-collector/issues/9429
2024-09-23 07:38:59 -07:00
..
debugexporter [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
exporterbatcher [chore]: enable require-error rule from testifylint (#11199) 2024-09-18 15:02:22 -07:00
exporterhelper [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
exporterprofiles [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
exporterqueue [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
exportertest [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
internal [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
loggingexporter [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
nopexporter [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
otlpexporter [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
otlphttpexporter [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
Makefile split exporter into its own module (#7240) 2023-03-03 13:13:16 -08:00
README.md [chore] Simplify general information in receivers/exporter readme (#9336) 2024-01-22 09:55:32 -08:00
exporter.go [connector,exporter,receiver,extension,processor] remove deprecated funcs/structs (#10423) 2024-07-19 08:13:50 -07:00
exporter_test.go [chore]: enable require-error rule from testifylint (#11199) 2024-09-18 15:02:22 -07:00
go.mod [component] Refactor to use pipeline.ID and pipeline.Signal (#11204) 2024-09-23 07:38:59 -07:00
go.sum fix(deps): update module google.golang.org/grpc to v1.66.2 (#11187) 2024-09-18 10:33:58 +02:00
package_test.go [chore] remove unused opencensus code (#9108) 2024-01-30 10:19:42 -08:00

README.md

General Information

An exporter defines how the pipeline data leaves the collector.

This repository hosts the following exporters available in traces, metrics and logs pipelines (sorted alphabetically):

The contrib repository has more exporters available in its builds.

Configuring Exporters

Exporters are configured via YAML under the top-level exporters tag.

The following is a sample configuration for the exampleexporter.

exporters:
  # Exporter 1.
  # <exporter type>:
  exampleexporter:
    # <setting one>: <value one>
    endpoint: 1.2.3.4:8080
    # ...
  # Exporter 2.
  # <exporter type>/<name>:
  exampleexporter/settings:
    # <setting two>: <value two>
    endpoint: 0.0.0.0:9211

An exporter instance is referenced by its full name in other parts of the config, such as in pipelines. A full name consists of the exporter type, '/' and the name appended to the exporter type in the configuration. All exporter full names must be unique.

For the example above:

  • Exporter 1 has full name exampleexporter.
  • Exporter 2 has full name exampleexporter/settings.

Exporters are enabled upon being added to a pipeline. For example:

service:
  pipelines:
    # Valid pipelines are: traces, metrics or logs
    # Trace pipeline 1.
    traces:
      receivers: [examplereceiver]
      processors: []
      exporters: [exampleexporter, exampleexporter/settings]
    # Trace pipeline 2.
    traces/another:
      receivers: [examplereceiver]
      processors: []
      exporters: [exampleexporter, exampleexporter/settings]

Data Ownership

When multiple exporters are configured to send the same data (e.g. by configuring multiple exporters for the same pipeline):

  • exporters not configured to mutate the data will have shared access to the data
  • exporters with the Capabilities to mutate the data will receive a copy of the data

Exporters access export data when ConsumeTraces/ConsumeMetrics/ConsumeLogs function is called. Unless exporter's capabalities include mutation, the exporter MUST NOT modify the pdata.Traces/pdata.Metrics/pdata.Logs argument of these functions. Any approach that does not mutate the original pdata.Traces/pdata.Metrics/pdata.Logs is allowed without the mutation capability.

Proxy Support

Beyond standard YAML configuration as outlined in the individual READMEs above, exporters that leverage the net/http package (all do today) also respect the following proxy environment variables:

  • HTTP_PROXY
  • HTTPS_PROXY
  • NO_PROXY

If set at Collector start time then exporters, regardless of protocol, will or will not proxy traffic as defined by these environment variables.