open-telemetry
/
opentelemetry-collector
mirror of https://github.com/open-telemetry/opentelemetry-collector.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,201 Commits 132 Branches 4,405 Tags 206 MiB
Commit Graph

35 Commits

Author SHA1 Message Date
Bogdan Drutu ffb332b37b
Add capabilities to consumer, remove from processor (#2770) 
* Add capabilities to consumer, remove from processor

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* Rename baseConsumer

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-05-11 16:13:47 -07:00
Bogdan Drutu 7b39a894e1
Fix batch processor traces reorder, improve performance (#3107) 
Benchmarks Before:

```
goos: darwin
goarch: amd64
pkg: go.opentelemetry.io/collector/processor/batchprocessor
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkSplitTraces
BenchmarkSplitTraces-16    	    8277	    135600 ns/op	  222440 B/op	    1725 allocs/op
PASS
```

Benchmarks After:

```
goos: darwin
goarch: amd64
pkg: go.opentelemetry.io/collector/processor/batchprocessor
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkSplitTraces
BenchmarkSplitTraces-16    	    9896	    108060 ns/op	  172409 B/op	    1372 allocs/op
PASS
```

Benchmarks Reference Clone:

```
goos: darwin
goarch: amd64
pkg: go.opentelemetry.io/collector/processor/batchprocessor
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkCloneSpans
BenchmarkCloneSpans-16    	   12393	     97528 ns/op	  167896 B/op	    1303 allocs/op
PASS
```

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-05-05 15:19:29 -07:00
Bogdan Drutu 665e9657ac
Use config.ID for Processors config (#3077) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-05-03 14:46:10 -07:00
Bogdan Drutu 9df19a1b45
Fix batchprocessor to avoid reordering and send max size (#3029) 
* The reordering could happen if during processing of an item other items were added to the newItems channel. This is fixed by avoiding to re-add the left items to the channel.
* The logic of spliting was wrong by forcing sendBatchSize instead of sendBatchMaxSize when split was called.
* Fix logic if a very large message is received > 2x sendBatchMaxSize to call export multiple times instead of once and reset the timer.

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-04-26 16:01:08 -07:00
Bogdan Drutu 25da8cfb98
Small cleanups in batchprocessor (#3013) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-04-26 13:52:33 -07:00
Bogdan Drutu d6a0f9ab76
Fix comments for exported funcs/vars/consts (#2938) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-04-14 14:55:23 -07:00
Bogdan Drutu a409f2a1da
Remove duplicate metrics from memorylimiter (#2841) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-04-02 15:03:54 -07:00
gregoryfranklin 49ddca96c1
Support max batch size for logs (#2736) 2021-03-24 12:41:34 -07:00
Bogdan Drutu 0c6757ec71
Rename [Traces|Metrics|Logs]Consumer to [Traces|Metrics|Logs] (#2761) 
* Rename [Traces|Metrics|Logs]Consumer to [Traces|Metrics|Logs]

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* Update internal/testcomponents/example_exporter.go

Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>

Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
 2021-03-23 08:04:44 -07:00
Bogdan Drutu 94a6cdde67
Rename pdata Size to OtlpProtoSize, fix some other comments (#2726) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-03-19 14:30:25 -04:00
Bogdan Drutu 85804ff257
Rename pdata.Logs.SizeBytes to Size, consistent with metrics/traces (#2694) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2021-03-17 15:21:25 -07:00
Tigran Najaryan 0961efb54f
Fix Shutdown behavior for batchprocessor (#2537) 
I added a Shutdown() test that does basic verification of the behavior of the
Shutdown() function. More verifications can be added later.

The test revealed a bug in batchprocessor Shutdown() function which would
not wait until all pending data was drained.
 2021-03-02 13:43:58 -05:00
Daniel Dyla e479242ac7
Support max batch size for metrics (#2422) 
* Support max batch size for metrics

* send batch max supported by trace and metric

* Avoid extra allocations
 2021-02-09 12:28:23 -08:00
Ben Keith ab843b20c2
batchprocessor: Fix shutdown race (#1967) 
The newItem channel cannot be closed ever since Shutdown is not
guaranteed to be called after all Consume* calls to the same processor.

Even after Shutdown is finished, there is not hard guarantee that
nothing will ever call Consume*, so we need to just leave the channel
open until GC will clean it up.
 2020-11-03 10:00:41 -08:00
Bogdan Drutu a3fa0a9166
Rename processor component.TraceProcessor to component.TracesProcessor, and equivalent Create method feature request (#2026) 
Fixes #1975

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-10-28 18:11:25 -04:00
Bogdan Drutu 99cb5b244a
Rename consumer.TraceConsumer to consumer.TracesConsumer (#1981) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-10-20 10:56:30 -07:00
Bogdan Drutu e7e6693926
Expose telemetry level in the configtelemetry (#1796) 
Next PR will add a config setting that can be embedded in every component config.

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-09-18 16:11:27 -07:00
Bogdan Drutu d119b89225
Move the metrics definition to pdata. Remove internal/data definition (#1720) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-09-03 12:18:32 -07:00
Ben Keith 889948e2fa
Batch Processor: Log Support (#1723) 
This mimics the same logic as metrics for logs in the batchprocessor.
 2020-09-02 18:33:26 -04:00
Bogdan Drutu e9d67ce381
Change batch processor to use the new internal metrics structs (#1699) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-08-31 15:33:52 -07:00
Bogdan Drutu 2a54e941a4
Move more old internal structs to old files, and rename funcs with Old (#1662) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-08-27 21:53:55 -07:00
Bogdan Drutu 42ac0eeafc
Move previous internal metrics package to dataold (#1658) 
Need to re-generate the new internal metrics, but it is too much code to change,
so will move this to dataold (was internal anyway, so no public breaking change).

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-08-27 20:19:39 -07:00
Jay Camp 1e65674799
Update copyright (#1597) 
* Update copyright

* rebase
 2020-08-19 18:25:44 -07:00
Pavol Loffay 815e760588
Add batch size bytes metric to batch processor (#1270) 
* Add batch size bytes metrics

Signed-off-by: Pavol Loffay <ploffay@redhat.com>

* increase test coverage

Signed-off-by: Pavol Loffay <ploffay@redhat.com>

* Fix metrics

Signed-off-by: Pavol Loffay <ploffay@redhat.com>

* Fix test

Signed-off-by: Pavol Loffay <ploffay@redhat.com>

Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-07-22 11:14:25 -07:00
Pavol Loffay 5c7db8cce7
Fix overflowing batch size (#1310) 
* Fix overflowing batch size

Signed-off-by: Pavol Loffay <ploffay@redhat.com>

* Use max size

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
 2020-07-14 08:04:37 -07:00
Bogdan Drutu 29cd959efc
Set batch processor channel size to num cpus, use exporter sink (#1330) 
Fixes #1070

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-07-13 12:36:02 -07:00
Bogdan Drutu f3608576be
Cleanup batch processor, remove duplicate code and logic (#1323) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-07-12 08:07:00 -07:00
Pavol Loffay 97d9d86c88
Add batch size metric (#1241) 
* Add batch size metric to otel collector

Signed-off-by: Pavol Loffay <ploffay@redhat.com>

* Rename variable

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
 2020-07-07 16:30:22 -07:00
Juraci Paixão Kröhling 64403a6683
Handle errors in exporters (#1259) 
Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
 2020-07-03 06:41:46 -07:00
Constance Caramanolis 4074b3c654
Add the ability to batch metrics as a processor. (#1060) 
* Add batching for metrics

* Update documentation

* Improve test coverage
 2020-06-02 10:23:26 -07:00
Bruno Garcia 8aa2731844
Remove year from copyright (#964) 2020-05-26 12:44:22 -04:00
Bogdan Drutu a637b41c22
Change entire repo to use the new vanityurl go.opentelemetry.io/collector (#977) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-05-15 12:04:05 -07:00
Bogdan Drutu ca4b8231de
Rename MoveTo to MoveAndAppendTo. (#875) 
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
 2020-04-27 08:06:35 -07:00
Dmitrii Anoshin 27277a675f
Switch queued_retry processor to internal data model (#824) 2020-04-21 17:45:43 -07:00
Dmitrii Anoshin 277aa581ad
Switch batch_processor to internal data model (#832) 
New internal data structure allows to batch together traces from different resources. This simplify batching a lot. This commit removes most of the functionality for handling many batches per node and uses only one batch with one ticker.
 2020-04-15 16:51:27 -07:00