open-telemetry
/
opentelemetry-collector
mirror of https://github.com/open-telemetry/opentelemetry-collector.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,447 Commits 132 Branches 4,405 Tags 206 MiB
Commit Graph

7 Commits

Author SHA1 Message Date
Matthieu MOREL 0b978307c2
[chore]: enable gofumpt linter in connector, consumer, exporter, extension and featuregate (#11854) 
#### Description

[gofumpt](https://golangci-lint.run/usage/linters/#gofumpt) is a
stricter format than gofmt, while being backwards compatible.

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
 2024-12-12 19:25:06 +00:00
Bogdan Drutu d660596439
Fix small comments in exporter helper (#11650) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2024-11-14 08:16:04 -08:00
Bogdan Drutu 4f2a8d3ef0
[chore] Remove unused/unnecessary queue.Sizer types (#11455) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2024-10-15 19:37:48 -07:00
Sindy Li 28d0d57c35
[exporter] exporter queue Read() (#11396) 
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

This PR adds a public function `GetNextItem` to queue (both persistent
queue and bounded memory queue)

Why this change?
Instead of blocking until consumption of the item is done, we would like
to separate the API for reading and committing consumption.

Before:
`Consume(consumeFunc)`

After:
`idx, item = Read()`
`OnProcessingFinished(idx)`

<!-- Issue number if applicable -->
#### Link to tracking issue

https://github.com/open-telemetry/opentelemetry-collector/issues/8122
https://github.com/open-telemetry/opentelemetry-collector/issues/10368

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
 2024-10-11 10:17:38 -07:00
Sindy Li fed6dfe76d
[exporter] internal/persistent_queue::OnProcessingFinished is changed to a class function instead of a callback (#11338) 
#### Description

Why this change?
Each request from the queue contains multiple items, and those items
could be merge-split into multiple batches when they are sent out (see
https://github.com/open-telemetry/opentelemetry-collector/issues/8122
for more about exporter batcher). We would like to book-keep those
cases, and only call `onProcessingFinished` when all such batches has
gone out. In this PR, `onProcessingFinished` is changed from a callback
to a method function because it is easier to book keep index instead of
functions.

#### Link to tracking issue
https://github.com/open-telemetry/opentelemetry-collector/issues/8122
https://github.com/open-telemetry/opentelemetry-collector/issues/10368

#### Testing
`exporter/internal/queue/persistent_queue_test.go`

#### Documentation

This is an internal change invisible to the users.

---------

Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>
 2024-10-04 19:59:22 -07:00
Dmitrii Anoshin b7b7e51d1d
[chore] [exporterhelper] Integrate capacity limiting into the communication channel (#9232) 
Integrate capacity limiting into internal channels used by both memory
and persistent queues. Otherwise, with the independent capacity limiter,
it's hard to ensure that queue size is always accurate going forward.

Benchmarks before:
```
goos: darwin
goarch: arm64
Benchmark_QueueUsage_1000_requests-10      	    3252	    325010 ns/op	  246059 B/op	      10 allocs/op
Benchmark_QueueUsage_100000_requests-10    	      39	  29811116 ns/op	24002870 B/op	      10 allocs/op
Benchmark_QueueUsage_10000_items-10        	    3404	    349753 ns/op	  246052 B/op	      10 allocs/op
Benchmark_QueueUsage_1M_items-10           	      40	  29415583 ns/op	24002858 B/op	      10 allocs/op
BenchmarkPersistentQueue_TraceSpans
BenchmarkPersistentQueue_TraceSpans/#traces:_1_#spansPerTrace:_1-10         	  338180	      3836 ns/op	    2851 B/op	      78 allocs/op
BenchmarkPersistentQueue_TraceSpans/#traces:_1_#spansPerTrace:_10-10        	   81369	     15822 ns/op	   14598 B/op	     289 allocs/op
BenchmarkPersistentQueue_TraceSpans/#traces:_10_#spansPerTrace:_10-10       	   13066	     90155 ns/op	  130087 B/op	    2417 allocs/op
```

Benchmarks after:
```
Benchmark_QueueUsage_1000_requests-10      	    4210	    278175 ns/op	  246055 B/op	      10 allocs/op
Benchmark_QueueUsage_100000_requests-10    	      42	  25835945 ns/op	24002968 B/op	      10 allocs/op
Benchmark_QueueUsage_10000_items-10        	    4376	    279571 ns/op	  246056 B/op	      10 allocs/op
Benchmark_QueueUsage_1M_items-10           	      42	  26483907 ns/op	24002995 B/op	      10 allocs/op
BenchmarkPersistentQueue_TraceSpans
BenchmarkPersistentQueue_TraceSpans/#traces:_1_#spansPerTrace:_1-10         	  328268	      4251 ns/op	    2854 B/op	      78 allocs/op
BenchmarkPersistentQueue_TraceSpans/#traces:_1_#spansPerTrace:_10-10        	  101683	     12238 ns/op	   14582 B/op	     289 allocs/op
BenchmarkPersistentQueue_TraceSpans/#traces:_10_#spansPerTrace:_10-10       	   13382	     86464 ns/op	  130154 B/op	    2417 allocs/op
```
 2024-05-04 09:20:48 -07:00
Dmitrii Anoshin bdbb5f3d51
[exporterhelper] Add WithRequestQueue option to the exporter (#8853) 
Introduce a way to enable queue in the new exporter helper with a
developer interface suggested in
https://github.com/open-telemetry/opentelemetry-collector/pull/8248#discussion_r1302102261.

The new configuration interface for the end users provides a new
`queue_size_items` option to limit the queue by a number of spans, log
records, or metric data points. The previous way to limit the queue by
number of requests is preserved under the same field, `queue_size,`
which will later be deprecated through a longer transition process.

Tracking issue:
https://github.com/open-telemetry/opentelemetry-collector/issues/8122
 2024-02-03 09:24:05 -08:00