#### Description
[gofumpt](https://golangci-lint.run/usage/linters/#gofumpt) is a
stricter format than gofmt, while being backwards compatible.
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
#### Description
[perfsprint](https://golangci-lint.run/usage/linters/#perfsprint) checks
that fmt.Sprintf can be replaced with a faster alternative.
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
#### Description
[whitespace](https://golangci-lint.run/usage/linters/#whitespace) is a
linter that checks for unnecessary newlines at the start and end of
functions.
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
#### Description
This PR moves queue start and shutdown outside of queue consumer.
Consumer should be in charge of "consuming" the queue but not managing
the life cycle.
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
This PR adds a public function `GetNextItem` to queue (both persistent
queue and bounded memory queue)
Why this change?
Instead of blocking until consumption of the item is done, we would like
to separate the API for reading and committing consumption.
Before:
`Consume(consumeFunc)`
After:
`idx, item = Read()`
`OnProcessingFinished(idx)`
<!-- Issue number if applicable -->
#### Link to tracking issue
https://github.com/open-telemetry/opentelemetry-collector/issues/8122https://github.com/open-telemetry/opentelemetry-collector/issues/10368
<!--Describe what testing was performed and which tests were added.-->
#### Testing
<!--Describe the documentation added.-->
#### Documentation
<!--Please delete paragraphs that you did not use before submitting.-->
#### Description
Why this change?
Each request from the queue contains multiple items, and those items
could be merge-split into multiple batches when they are sent out (see
https://github.com/open-telemetry/opentelemetry-collector/issues/8122
for more about exporter batcher). We would like to book-keep those
cases, and only call `onProcessingFinished` when all such batches has
gone out. In this PR, `onProcessingFinished` is changed from a callback
to a method function because it is easier to book keep index instead of
functions.
#### Link to tracking issue
https://github.com/open-telemetry/opentelemetry-collector/issues/8122https://github.com/open-telemetry/opentelemetry-collector/issues/10368
#### Testing
`exporter/internal/queue/persistent_queue_test.go`
#### Documentation
This is an internal change invisible to the users.
---------
Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>
#### Description
This change fixes a potential deadlock bug for persistent queue.
There is a race condition in persistent queue that caused `used` in
`sizedChannel` to become out of sync with `ch` len. This causes `Offer`
to be deadlocked in specific race condition. For example:
1. Multiple consumers are calling Consume
2. Multiple producers are calling Offer to insert into the queue
a. All elements are taken from consumers. ch is empty
3. One consumer completes consume, calls onProcessingFinished
a. Inside sizedChannel, syncSize is invoked, used is reset to 0 when
other consumers are still waiting for lock to consume
4. More Offer is called inserting elements -> used and ch len should
equal
5. As step 3a consumers completes, used is decreased -> used is lower
than ch len
a. More Offer is called inserting since used is below capacity. however,
ch is full.
b. goroutine calling offer is holding the mutex but can’t release it as
ch is full.
c. no consumer can acquire mutex to complete previous
onProcessingFinished
This change returns an error if channel is full instead of waiting for
it to unblock.
#### Link to tracking issue
Fixes #
https://github.com/open-telemetry/opentelemetry-collector/issues/11015
#### Testing
- Added concurrent test in persistent queue that can reproduce the
problem(note: need to re-run it 100 times as the race condition is not
consistent).
- Added unit test for sizedChannel
#### Documentation
Added comment in the block explaining it
---------
Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>
This deprecates CreateSettings in favour of Settings.
NewNopCreateSettings is also being deprecated in favour of
NewNopSettings
Part of #9428
Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com>
This deprecates CreateSettings in favour of Settings.
NewNopCreateSettings is also being deprecated in favour of
NewNopSettings
Part of #9428
~Follows
https://github.com/open-telemetry/opentelemetry-collector/pull/10333~
---------
Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com>
Matthieu MOREL
Bogdan Drutu
Sindy Li
renovate[bot]
Tyler Helmuth
Alex Boten
Eric L
Dmitrii Anoshin
Alex Boten