open-telemetry
/
opentelemetry-collector
mirror of https://github.com/open-telemetry/opentelemetry-collector.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
opentelemetry-collector/processor/batchprocessor
History
Alex Boten 4e29c769bb
splitting batch/memorylimiter processors into their own modules (#6427) 
Fixes #6188, #6192, #6193
 		2022-11-03 08:32:19 -07:00
..
testdata [chore] change config tests to unmarshal only the config for that component (#5895) 2022-08-11 06:57:13 -07:00
Makefile splitting batch/memorylimiter processors into their own modules (#6427) 2022-11-03 08:32:19 -07:00
README.md [docs]fix some grammatical mistakes in readmes (#5607) 2022-07-05 11:58:00 -07:00
batch_processor.go Deprecate New[JSON|Proto][Marshaler|Unmarshale], expose the structs (#6340) 2022-10-18 15:29:37 -07:00
batch_processor_test.go Deprecate New[JSON|Proto][Marshaler|Unmarshale], expose the structs (#6340) 2022-10-18 15:29:37 -07:00
config.go Remove usage of unnecessary omitempty (#5256) 2022-04-25 10:43:13 -07:00
config_test.go [chore] change config tests to unmarshal only the config for that component (#5895) 2022-08-11 06:57:13 -07:00
factory.go Replace deprecate With[Traces|Metrics|Logs][Receiver|Processor|Exporter] with the new definition (#5759) 2022-07-29 11:38:34 -07:00
factory_test.go Move ValidateConfig from configcheck to configtest (#3956) 2021-09-16 03:44:59 -07:00
go.mod splitting batch/memorylimiter processors into their own modules (#6427) 2022-11-03 08:32:19 -07:00
go.sum splitting batch/memorylimiter processors into their own modules (#6427) 2022-11-03 08:32:19 -07:00
metrics.go Adds vanity import check (#4180) 2021-10-12 13:47:36 -07:00
metrics_test.go Remove level from all the MetricViews calls (#2149) 2020-11-16 23:11:27 -05:00
splitlogs.go Split `model` by signal type and move it to the new `pdata` module (#5168) 2022-04-11 16:00:26 -07:00
splitlogs_test.go Remove usages of deprecated pdata Clone methods (#6170) 2022-09-28 18:25:22 -07:00
splitmetrics.go Replace MetricDataType with MetricType (#6127) 2022-09-20 17:06:49 -07:00
splitmetrics_test.go [pdata] Rename MetricAggregationTemporality to AggregationTemporality (#6253) 2022-10-07 09:37:47 -07:00
splittraces.go Split `model` by signal type and move it to the new `pdata` module (#5168) 2022-04-11 16:00:26 -07:00
splittraces_test.go Remove usages of deprecated pdata Clone methods (#6170) 2022-09-28 18:25:22 -07:00

README.md

Batch Processor

Status
Stability traces beta
metrics beta
logs beta
Supported pipeline types traces, metrics, logs
Distributions core, contrib

The batch processor accepts spans, metrics, or logs and places them into batches. Batching helps better compress the data and reduce the number of outgoing connections required to transmit the data. This processor supports both size and time based batching.

It is highly recommended to configure the batch processor on every collector. The batch processor should be defined in the pipeline after the memory_limiter as well as any sampling processors. This is because batching should happen after any data drops such as sampling.

Please refer to config.go for the config spec.

The following configuration options can be modified:

  • send_batch_size (default = 8192): Number of spans, metric data points, or log records after which a batch will be sent regardless of the timeout.
  • timeout (default = 200ms): Time duration after which a batch will be sent regardless of size.
  • send_batch_max_size (default = 0): The upper limit of the batch size. 0 means no upper limit of the batch size. This property ensures that larger batches are split into smaller units. It must be greater than or equal to send_batch_size.

Examples:

processors:
  batch:
  batch/2:
    send_batch_size: 10000
    timeout: 10s

Refer to config.yaml for detailed examples on using the processor.