Fix#5996
### Correctness
From the [OTel
specification](88bffeac48/specification/common/README.md (attribute-limits)):
> - set an attribute value length limit such that for each attribute
value:
> - if it is a string, if it exceeds that limit (counting any character
in it as 1), SDKs MUST truncate that value, so that its length is at
most equal to the limit...
Our current implementation truncates on number of bytes not characters.
Unit tests are added/updated to validate this fix and prevent
regressions.
### Performance
```
goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/trace
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
│ commit-b6264913(old).txt │ commit-54c61ac2(new).txt │
│ sec/op │ sec/op vs base │
Truncate/Unlimited-8 1.2300n ± 7% 0.8757n ± 3% -28.80% (p=0.000 n=10)
Truncate/Zero-8 2.341n ± 2% 1.550n ± 9% -33.77% (p=0.000 n=10)
Truncate/Short-8 31.6800n ± 3% 0.9960n ± 4% -96.86% (p=0.000 n=10)
Truncate/ASCII-8 8.821n ± 1% 3.567n ± 3% -59.57% (p=0.000 n=10)
Truncate/ValidUTF-8-8 11.960n ± 1% 7.163n ± 1% -40.10% (p=0.000 n=10)
Truncate/InvalidUTF-8-8 56.35n ± 0% 37.34n ± 18% -33.74% (p=0.000 n=10)
Truncate/MixedUTF-8-8 81.83n ± 1% 50.00n ± 1% -38.90% (p=0.000 n=10)
geomean 12.37n 4.865n -60.68%
│ commit-b6264913(old).txt │ commit-54c61ac2(new).txt │
│ B/op │ B/op vs base │
Truncate/Unlimited-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/Zero-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/Short-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/ASCII-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/ValidUTF-8-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/InvalidUTF-8-8 16.00 ± 0% 16.00 ± 0% ~ (p=1.000 n=10) ¹
Truncate/MixedUTF-8-8 32.00 ± 0% 32.00 ± 0% ~ (p=1.000 n=10) ¹
geomean ² +0.00% ²
¹ all samples are equal
² summaries must be >0 to compute geomean
│ commit-b6264913(old).txt │ commit-54c61ac2(new).txt │
│ allocs/op │ allocs/op vs base │
Truncate/Unlimited-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/Zero-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/Short-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/ASCII-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/ValidUTF-8-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/InvalidUTF-8-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
Truncate/MixedUTF-8-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
geomean ² +0.00% ²
¹ all samples are equal
² summaries must be >0 to compute geomean
```
#### Values shorter than limit
This is the default code path. Most attribute values will be shorter
than the default 128 limit that users will not modify.
The current code, `safeTruncate` requires a full iteration of the value
to determine it is valid and under the limit.
The replacement, `truncate`, first checks if the number of bytes in the
value are less than or equal to the limit (which guarantees the number
of characters are less than or equal to the limit) and returns
immediately. This will mean that invalid encoding less than the limit is
not changed, which meets the specification requirements.
#### Values longer than the limit
For values who's number of bytes exceeds the limit, they are iterated
only once with the replacement, `truncate`.
In comparison, the current code, `safeTruncate`, can iterate the string
up to three separate times when the string contains invalid characters.
```go
type interInst struct {
x int
}
type inter interface {
}
var sink []inter
func BenchmarkX(b *testing.B) {
sink = make([]inter, b.N)
for i := 0; i < b.N; i++ {
sink[i] = &interInst{}
}
clear(sink)
sink = sink[:0]
runtime.GC()
var ms runtime.MemStats
runtime.ReadMemStats(&ms)
b.Log(b.N, ms.Frees) // Frees is the cumulative count of heap objects freed.
}
```
```
clear:
ioz_test.go:35: 1 589
ioz_test.go:35: 100 711
ioz_test.go:35: 10000 10729
ioz_test.go:35: 1000000 1010750 <-- 1m+ freed
ioz_test.go:35: 16076874 17087643
ioz_test.go:35: 19514749 36602412
```
```
no clear:
ioz_test.go:35: 1 585
ioz_test.go:35: 100 606
ioz_test.go:35: 10000 725
ioz_test.go:35: 1000000 10745 <-- some "overhead" objects freed, not the slice.
ioz_test.go:35: 16391445 1010765
ioz_test.go:35: 21765238 17402230
```
This is documented at https://go.dev/wiki/SliceTricks:
> NOTE If the type of the element is a pointer or a struct with pointer
fields, which need to be garbage collected, the above implementations of
Cut and Delete have a potential memory leak problem: some elements with
values are still referenced by slice a’s underlying array, just not
“visible” in the slice. Because the “deleted” value is referenced in the
underlying array, the deleted value is still “reachable” during GC, even
though the value cannot be referenced by your code. If the underlying
array is long-lived, this represents a leak.
Followed by examples of how zeroing out the slice elements solves
the problem. This PR does the same.
Good day,
While working on
https://github.com/open-telemetry/opentelemetry-go/pull/5858 I found
some other possible improvements.
This PR:
- Adds an early return to `SetAttributes` when no attributes are
provided.
- Only increases `s.attributes` to guarantee that there is enough space
for elements to be added.
- Fixes and issue where `truncateAttr` was not used when a attribute was
being updated in `addOverCapAttrs`.
Thanks for reviewing and please let me know if any changes are needed.
---------
Co-authored-by: Damien Mathieu <42@dmathieu.com>
Rather than the deprecated InstrumentationLibrary
This is a replacement for
https://github.com/open-telemetry/opentelemetry-go/pull/3104, as after
this, there is no usage of `instrumentation.Library` within the SDK
anymore.
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
Co-authored-by: Sam Xie <sam@samxie.me>
Fix#5343
- Update the `evictionQueue` to log when it drops a value
- Update the `evictionQueue` to be declared over an `[T any]` parameter
so it knows what to log when it is dropping a value and to reduce the
`interface{}` allocation
- Add a `clone` method to replace the now unneeded
`interfaceArrayTo*Array` functions.
- Update the `recordingSpan` to log once that is dropped an attribute
when limits are reached.
* record links with empty span context
* add global trace state
* fix test comments and changelog
---------
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
* Update README.md
* Remove 1.20 support from CI workflows
* Update all go mod
* Add changelog entry
* Update go mod tidy target
* Run go mod tidy
* Replace sliceEqualFunc with slices.EqualFunc
* Replace grow with slices.Grow
* Replace ensureAttributesCapacity with slices.Grow
* Replace conditional with min
* Use slices module for slice comparison in metricdatatest
* Add trace/embedded
* Update trace impl to use trace/embedded
* Add noop pkg to replace no-op impl in trace pkg
* Use trace/embedded in global impl
* Use trace/embedded in SDK impl
* Update opencensus bridge
* Update opentracing bridge
* Add changes to changelog
* Update trace/doc.go
Co-authored-by: David Ashpole <dashpole@google.com>
---------
Co-authored-by: David Ashpole <dashpole@google.com>
* Use gofumpt instead of gofmt in golangci-lint conf
* Run gofumpt fixes
* Format generated templates
---------
Co-authored-by: Chester Cheung <cheung.zhy.csu@gmail.com>
* Upgrade all use of semconv to v1.21.0
* Add change to changelog
* Add AIX and ZOS OS support
* Upgrade semconv for merged changes
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* docs(typos): Run codespell to fix typos
There were a lot of typos through the repository, so I ran
[codespell][], a tool for automatically fixing typos, to fix them.
```console
make codespell
```
There's already a tool called [misspell][] that's supposed to take care
of this, but misspell hasn't been updated for 6 years, and it doesn't
seem to be catching any of the typos that codespell can.
[codespell]: https://github.com/codespell-project/codespell
[misspell]: https://github.com/client9/misspell
* Revert and ignore spelling for Consequentially
* Add GH workflow for codespell
* Revert GH Workflow and Makefile for codespell
Per @pellared, since there's no instructions for setting up codespell,
it was suggested that the changes for setting up a workflow and section
in Makefile include instructions for setting up codespell as well.
* Revert spelling on consequently
---------
Co-authored-by: Chester Cheung <cheung.zhy.csu@gmail.com>
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
* Pre-allocate spanProcessorStates slice
* Make sync.Once a non-pointer
It doesn't need to be a pointer, can be part of the struct to avoid allocating a separate object for it
* getSpanProcessors() helper
* Add tests for UnregisterSpanProcessor()
* Deprecate Library and move all uses to Scope
* Add PR number to changelog
* Don't change signatures in stable modules
* Revert some changes
* Rename internal struct names
* A bit more renaming
* Update sdk/trace/span.go
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
* Update based on feedback
* Revert change
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
* No wrap RELEASING Semantic Convention Generation section
* Initial generator
* Update template render
* Add exception and schema templates
* Add semconv/internal http unification
* Add http template
* Add licenses header
* Embed the templates
* Update static version in schema tmpl
* Add semconv-generate target to Makefile
Use this target to generate versions of the semconv packages.
* Generate semconv packages
* Update RELEASING to use make semconv-generate
* Add comments to semconvkit
* Generate semconv/v1.8.0
* Use new version of semconv
* Add changes to changelog
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
* Replace recordingSpan attributes implementation
Instead of an LRU strategy for cap-ing span attributes, comply with the
specification and drop last added. To do this, the attributesmap is
replaced with a slice of attributes.
* Remove attributesmap files
* Refine addition algorithm
Unify duplicated code.
Fix deduplication algorithm.
Fix droppedAttributes to always be returned, even if the span has no
attributes.
* Unify span SetAttributes tests
* Doc fix to attr drop order in changelog
* Test span and snapshot attrs
* fix lint
* Add tests for recordingSpan method defaults
* Comment why pre-allocation is not done
* Correct grammar in recordingSpan allocation comment
* Update sdk/trace/tracer.go
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
* Do not store TracerProvider fields in span
Instead of keeping a reference to the span's Tracer, and therefore also
it's TracerProvider, and the associated resource and spanLimits just
keep the reference to the Tracer. Refer to the TracerProvider fields
when needed instead.
* Make span refer to the inst lib via the Tracer
Instead of holding a field in the span, refer to the field in the parent
Tracer.
* Optimize evictedQueue impl and use
Avoid unnecessary allocations in the recordingSpan by using an
evictedQueue type instead of a pointer to one.
Lazy allocate the evictedQueue queue to prevent unnecessary operations
for spans without any use of the queue.
Document the evictedQueue
* Fix grammar
* Add semconv/v1.7.0 package
Generated from the v1.7.0 release of the specification using the
semconvgen tool manually. This also updates the project to use this
version of the semconv package.
This does not include the prior, and missing, v1.5.0 and v1.6.0 versions
of the semconv package. They are planned to be added in a follow-on PR.
* Update CHANGELOG.md
* Refactor startSpanInternal into a tracer method
The span creation and configuration process is split across the tracer
Start method and the startSpanInternal function, each living in
different files. This adds confusion when developing. It requires the
developer to remember certain parts of the configuration happen in one
place or the other.
This unifies the creation and configuration of a new span. It makes this
unification by creating a newSpan method on the tracer. This method will
do all the configuration needed for a new span other than setting the
execution task tracker. This remains a part of the Start method to
preserve any existing timing and tracing the already exists.
* Add a non-recording span to the SDK
* Assign returned context from runtime trace task
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
* Removed DroppedAttributeCount field from Link struct
* Reintroduced DroppedAttributeCount with a wrapper struct
* Added Link type in otel/sdk package and used it instead of Trace API equivalent
* Added changelog entry
* Linting fix in changelog
* Deleted duplicated changelog section added by mistake
* Expanded changelog entry and moved it under the 'Added' section
* Explicitly mentioned otel/trace library in changelog entry under 'Removed' section to avoid ambiguity
* Ensure that no-op tracer still progates non-recording spans
Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
* Add `TracerProvider` to the `trace.Span` interface
Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
* Remove config from oteltest.Tracer as it can be accessed through the provider
Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>
Tyler Yahn
Damien Mathieu
Mikhail Mazurskiy
Warnar Boekkooi
Anton Manakin
qcheng
Robert Pająk
Liz Fong-Jones
Charlie Le
Damien Mathieu
Ziqi Zhao
Bogdan Drutu
wdullaer
Joshua MacDonald
Craig Pastro
Brad Topol
Bogdan Drutu
Srikanth Chekuri
Bhautik Pipaliya
Nelson Ghezzi
David Ashpole
Ali Josie
Anthony Mirabella