Directly update:
- go.opentelemetry.io/otel/* from v1.26.0 to v1.27.0
- go.opentelemetry.io/contrib/* from v0.51.0 to v0.52.0
Indirectly update:
- google.golang.org/protobuf from v1.33.0 to v1.34.0
This update breaks some of our existing otel grpc interceptors, but in
return allows us to use the newer grpc StatsHandler mechanism, while
still filtering out health-check requests.
Fixes https://github.com/letsencrypt/boulder/issues/7235
This upgrades otel to v1.15.0, and the /contrib/ packages to v0.41.0.
Several dependencies are upgraded as dependencies, notably grpc.
This contains a change to grpc, only mapping some grpc.Errors into span
errors if it's Unknown, DeadlineExceeded, Unimplemented, Internal,
Unavailable, or DataLoss, which should be helpful for us as we use grpc
errors semantically in Boulder, especially NotFound.
Add a new shared config stanza which all boulder components can use to
configure their Open Telemetry tracing. This allows components to
specify where their traces should be sent, what their sampling ratio
should be, and whether or not they should respect their parent's
sampling decisions (so that web front-ends can ignore sampling info
coming from outside our infrastructure). It's likely we'll need to
evolve this configuration over time, but this is a good starting point.
Add basic Open Telemetry setup to our existing cmd.StatsAndLogging
helper, so that it gets initialized at the same time as our other
observability helpers. This sets certain default fields on all
traces/spans generated by the service. Currently these include the
service name, the service version, and information about the telemetry
SDK itself. In the future we'll likely augment this with information
about the host and process.
Finally, add instrumentation for the HTTP servers and grpc
clients/servers. This gives us a starting point of being able to monitor
Boulder, but is fairly minimal as this PR is already somewhat unwieldy:
It's really only enough to understand that everything is wired up
properly in the configuration. In subsequent work we'll enhance those
spans with more data, and add more spans for things not automatically
traced here.
Fixes https://github.com/letsencrypt/boulder/issues/6361
---------
Co-authored-by: Aaron Gable <aaron@aarongable.com>
Upgrade grpc to v1.53.0, as preparation for introducing OpenTelemetry,
which depends on that grpc version.
Two changes to our own code were necessitated by upstream changes:
1. Add a stub implementation of GetOrBuildProducer: this was added to
the balancer.SubConn interface by grpc v1.51.0
2. Change use of Endpoint field to Endpoint() method: the field was
removed and replaced by a method in
https://github.com/grpc/grpc-go/pull/5852. This also means that our
tests can't set the .Endpoint field, so the tests are updated to use the
.URL field instead, and a helper has been added to make that easy.
Part of #6361
Diff: https://github.com/prometheus/client_golang/compare/v1.7.1...v1.12.1
Changes:
* 1.12.1 / 2022-01-29
- [BUGFIX] Make the Go 1.17 collector concurrency-safe
- Use simpler locking in the Go 1.17 collector
- [BUGFIX] Reduce granularity of histogram buckets for Go 1.17 collector
- [ENHANCEMENT] API client: make HTTP reads more efficient
* 1.12.0 / 2022-01-19
- [CHANGE] example/random: Move flags and metrics into main()
- [FEATURE] API client: Support wal replay status api
- [FEATURE] Use the runtime/metrics package for the Go collector for 1.17+
- [ENHANCEMENT] API client: Update /api/v1/status/tsdb to include headStats
- [ENHANCEMENT] promhttp: Check validity of method and code label values
* 1.11.0 / 2021-06-07
- [CHANGE] Add new collectors package.
- [CHANGE] prometheus.NewExpvarCollector is deprecated, use collectors.NewExpvarCollector instead.
- [CHANGE] prometheus.NewGoCollector is deprecated, use collectors.NewGoCollector instead.
- [CHANGE] prometheus.NewBuildInfoCollector is deprecated, use collectors.NewBuildInfoCollector instead.
- [FEATURE] Add new collector for database/sql#DBStats.
- [FEATURE] API client: Add exemplars API support.
- [ENHANCEMENT] API client: Add newer fields to Rules API.
- [ENHANCEMENT] API client: Add missing fields to Targets API.
* 1.10.0 / 2021-03-18
- [CHANGE] Minimum required Go version is now 1.13.
- [CHANGE] API client: Add matchers to LabelNames and LabesValues.
- [FEATURE] API client: Add buildinfo call.
- [BUGFIX] Fix build on riscv64.
* 1.9.0 / 2020-12-17
- [FEATURE] NewPidFileFn helper to create process collectors for processes whose PID is read from a file.
- [BUGFIX] promhttp: Prevent endless loop in InstrumentHandler... middlewares with invalid metric or label names.
* 1.8.0 / 2020-10-15
- [CHANGE] API client: Use time.Time rather than string for timestamps in RuntimeinfoResult.
- [FEATURE] Export MetricVec to facilitate implementation of vectors of custom Metric types.
- [FEATURE] API client: Support /status/tsdb endpoint.
- [ENHANCEMENT] API client: Enable GET fallback on status code 501.
- [ENHANCEMENT] Remove Metric references after reslicing to free up more memory.
Additional transitive dependency updates:
* https://github.com/prometheus/common/compare/v0.10.0...v0.32.1
* https://github.com/prometheus/procfs/compare/v0.1.3...v0.7.3
* https://github.com/golang/appengine/compare/v1.6.5...v1.6.6
* cb27e3aa20...8632dd7979
* 0f9fa26af8...5a964db013
protoc now generates grpc code in a separate file from protobuf code.
Also, grpc servers are now required to embed an "unimplemented"
interface from the generated .pb.go file, which provides forward
compatibility.
Update the generate.go files since the invocation for protoc has changed
with the split into .pb.org and _grpc.pb.go.
Fixes#5368
Note: This dep bump introduces a harmless, but annoying, error log
to our service startup output of the form:
> E203318 boulder-ra 2PvBvwg [AUDIT] ccResolverWrapper: error parsing service config: no JSON service config provided
We previously addressed this with the upstream project
(30f4150eec)
but the problem has returned. Filed https://github.com/letsencrypt/boulder/issues/4628
as a follow-up to chase this down.
Unit tests are confirmed to pass:
```
~/go/src/google.golang.org/grpc$ git log --pretty=format:'%h' -n 1
1a3960e
~/go/src/google.golang.org/grpc$ go test ./...
ok google.golang.org/grpc 18.163s
? google.golang.org/grpc/backoff [no test files]
? google.golang.org/grpc/balancer [no test files]
? google.golang.org/grpc/balancer/base [no test files]
ok google.golang.org/grpc/balancer/grpclb 15.491s
? google.golang.org/grpc/balancer/grpclb/grpc_lb_v1 [no test files]
ok google.golang.org/grpc/balancer/roundrobin 0.349s
? google.golang.org/grpc/balancer/weightedroundrobin [no test files]
? google.golang.org/grpc/benchmark [no test files]
? google.golang.org/grpc/benchmark/benchmain [no test files]
? google.golang.org/grpc/benchmark/benchresult [no test files]
? google.golang.org/grpc/benchmark/client [no test files]
ok google.golang.org/grpc/benchmark/flags 0.001s
? google.golang.org/grpc/benchmark/grpc_testing [no test files]
ok google.golang.org/grpc/benchmark/latency 1.005s
ok google.golang.org/grpc/benchmark/primitives 0.001s [no tests to run]
? google.golang.org/grpc/benchmark/server [no test files]
? google.golang.org/grpc/benchmark/stats [no test files]
? google.golang.org/grpc/benchmark/worker [no test files]
? google.golang.org/grpc/binarylog/grpc_binarylog_v1 [no test files]
? google.golang.org/grpc/channelz/grpc_channelz_v1 [no test files]
ok google.golang.org/grpc/channelz/service 0.009s
ok google.golang.org/grpc/codes 0.002s
? google.golang.org/grpc/connectivity [no test files]
ok google.golang.org/grpc/credentials 0.017s
ok google.golang.org/grpc/credentials/alts 0.003s
? google.golang.org/grpc/credentials/alts/internal [no test files]
ok google.golang.org/grpc/credentials/alts/internal/authinfo 0.003s
ok google.golang.org/grpc/credentials/alts/internal/conn 0.079s
ok google.golang.org/grpc/credentials/alts/internal/handshaker 0.039s
ok google.golang.org/grpc/credentials/alts/internal/handshaker/service 0.007s
? google.golang.org/grpc/credentials/alts/internal/proto/grpc_gcp [no test files]
? google.golang.org/grpc/credentials/alts/internal/testutil [no test files]
? google.golang.org/grpc/credentials/google [no test files]
ok google.golang.org/grpc/credentials/internal 0.005s
? google.golang.org/grpc/credentials/oauth [no test files]
? google.golang.org/grpc/encoding [no test files]
? google.golang.org/grpc/encoding/gzip [no test files]
ok google.golang.org/grpc/encoding/proto 0.025s
? google.golang.org/grpc/examples/features/authentication/client [no test files]
? google.golang.org/grpc/examples/features/authentication/server [no test files]
? google.golang.org/grpc/examples/features/cancellation/client [no test files]
? google.golang.org/grpc/examples/features/cancellation/server [no test files]
? google.golang.org/grpc/examples/features/compression/client [no test files]
? google.golang.org/grpc/examples/features/compression/server [no test files]
? google.golang.org/grpc/examples/features/deadline/client [no test files]
? google.golang.org/grpc/examples/features/deadline/server [no test files]
? google.golang.org/grpc/examples/features/debugging/client [no test files]
? google.golang.org/grpc/examples/features/debugging/server [no test files]
? google.golang.org/grpc/examples/features/encryption/ALTS/client [no test files]
? google.golang.org/grpc/examples/features/encryption/ALTS/server [no test files]
? google.golang.org/grpc/examples/features/encryption/TLS/client [no test files]
? google.golang.org/grpc/examples/features/encryption/TLS/server [no test files]
? google.golang.org/grpc/examples/features/errors/client [no test files]
? google.golang.org/grpc/examples/features/errors/server [no test files]
? google.golang.org/grpc/examples/features/interceptor/client [no test files]
? google.golang.org/grpc/examples/features/interceptor/server [no test files]
? google.golang.org/grpc/examples/features/keepalive/client [no test files]
? google.golang.org/grpc/examples/features/keepalive/server [no test files]
? google.golang.org/grpc/examples/features/load_balancing/client [no test files]
? google.golang.org/grpc/examples/features/load_balancing/server [no test files]
? google.golang.org/grpc/examples/features/metadata/client [no test files]
? google.golang.org/grpc/examples/features/metadata/server [no test files]
? google.golang.org/grpc/examples/features/multiplex/client [no test files]
? google.golang.org/grpc/examples/features/multiplex/server [no test files]
? google.golang.org/grpc/examples/features/name_resolving/client [no test files]
? google.golang.org/grpc/examples/features/name_resolving/server [no test files]
? google.golang.org/grpc/examples/features/proto [no test files]
? google.golang.org/grpc/examples/features/proto/echo [no test files]
? google.golang.org/grpc/examples/features/reflection/server [no test files]
? google.golang.org/grpc/examples/features/retry/client [no test files]
? google.golang.org/grpc/examples/features/retry/server [no test files]
? google.golang.org/grpc/examples/features/wait_for_ready [no test files]
? google.golang.org/grpc/examples/helloworld/greeter_client [no test files]
? google.golang.org/grpc/examples/helloworld/greeter_server [no test files]
? google.golang.org/grpc/examples/helloworld/helloworld [no test files]
ok google.golang.org/grpc/examples/helloworld/mock_helloworld 0.003s
? google.golang.org/grpc/examples/route_guide/client [no test files]
ok google.golang.org/grpc/examples/route_guide/mock_routeguide 0.005s
? google.golang.org/grpc/examples/route_guide/routeguide [no test files]
? google.golang.org/grpc/examples/route_guide/server [no test files]
ok google.golang.org/grpc/grpclog 0.003s
? google.golang.org/grpc/grpclog/glogger [no test files]
ok google.golang.org/grpc/health 0.063s
? google.golang.org/grpc/health/grpc_health_v1 [no test files]
? google.golang.org/grpc/internal [no test files]
? google.golang.org/grpc/internal/backoff [no test files]
? google.golang.org/grpc/internal/balancerload [no test files]
ok google.golang.org/grpc/internal/binarylog 0.026s
ok google.golang.org/grpc/internal/buffer 0.002s
ok google.golang.org/grpc/internal/cache 0.653s
ok google.golang.org/grpc/internal/channelz 0.005s
? google.golang.org/grpc/internal/envconfig [no test files]
? google.golang.org/grpc/internal/grpcrand [no test files]
ok google.golang.org/grpc/internal/grpcsync 0.002s
ok google.golang.org/grpc/internal/grpctest 0.002s
ok google.golang.org/grpc/internal/leakcheck 4.083s
ok google.golang.org/grpc/internal/proto/grpc_service_config 0.002s
ok google.golang.org/grpc/internal/resolver/dns 1.620s
? google.golang.org/grpc/internal/resolver/passthrough [no test files]
? google.golang.org/grpc/internal/syscall [no test files]
ok google.golang.org/grpc/internal/testutils 0.002s
ok google.golang.org/grpc/internal/transport 81.078s
ok google.golang.org/grpc/internal/wrr 0.008s
? google.golang.org/grpc/interop [no test files]
? google.golang.org/grpc/interop/alts/client [no test files]
? google.golang.org/grpc/interop/alts/server [no test files]
? google.golang.org/grpc/interop/client [no test files]
? google.golang.org/grpc/interop/fake_grpclb [no test files]
? google.golang.org/grpc/interop/grpc_testing [no test files]
? google.golang.org/grpc/interop/http2 [no test files]
? google.golang.org/grpc/interop/server [no test files]
? google.golang.org/grpc/keepalive [no test files]
ok google.golang.org/grpc/metadata 0.004s
ok google.golang.org/grpc/naming 0.156s
? google.golang.org/grpc/peer [no test files]
ok google.golang.org/grpc/reflection 0.010s
? google.golang.org/grpc/reflection/grpc_reflection_v1alpha [no test files]
? google.golang.org/grpc/reflection/grpc_testing [no test files]
? google.golang.org/grpc/reflection/grpc_testingv3 [no test files]
? google.golang.org/grpc/resolver [no test files]
? google.golang.org/grpc/resolver/dns [no test files]
? google.golang.org/grpc/resolver/manual [no test files]
? google.golang.org/grpc/resolver/passthrough [no test files]
? google.golang.org/grpc/serviceconfig [no test files]
ok google.golang.org/grpc/stats 0.046s
? google.golang.org/grpc/stats/grpc_testing [no test files]
ok google.golang.org/grpc/status 0.008s
? google.golang.org/grpc/stress/client [no test files]
? google.golang.org/grpc/stress/grpc_testing [no test files]
? google.golang.org/grpc/stress/metrics_client [no test files]
? google.golang.org/grpc/tap [no test files]
ok google.golang.org/grpc/test 30.190s
ok google.golang.org/grpc/test/bufconn 0.204s
? google.golang.org/grpc/test/codec_perf [no test files]
? google.golang.org/grpc/test/go_vet [no test files]
? google.golang.org/grpc/test/grpc_testing [no test files]
? google.golang.org/grpc/xds/experimental [no test files]
ok google.golang.org/grpc/xds/internal 0.003s
ok google.golang.org/grpc/xds/internal/balancer 5.113s
ok google.golang.org/grpc/xds/internal/balancer/edsbalancer 1.264s
ok google.golang.org/grpc/xds/internal/balancer/lrs 0.246s
ok google.golang.org/grpc/xds/internal/balancer/orca 0.002s
ok google.golang.org/grpc/xds/internal/client 0.004s
? google.golang.org/grpc/xds/internal/proto [no test files]
? google.golang.org/grpc/xds/internal/proto/udpa/data/orca/v1 [no test files]
? google.golang.org/grpc/xds/internal/proto/udpa/service/orca/v1 [no test files]
? google.golang.org/grpc/xds/internal/proto/udpa/type/v1 [no test files]
ok google.golang.org/grpc/xds/internal/resolver 0.004s
```
Updates https://github.com/letsencrypt/boulder/issues/4548
Because the package versions in go.mod match what we use in Godeps.json,
there are no substantive code diffs. However, there are some tiny
differences resulting from how go mod vendors things differently than
godep:
go mod does not preserve executable permissions on shell scripts
Some packages have import lines like:
package ocsp // import "golang.org/x/crypto/ocsp"
godep used to remove the comment from these lines, but go mod vendor does not.
This introduces several indirect dependencies that we didn't have
before. This is because godep used to operate at a package level, but
go mod operates at a module (~= repository) level. So if we used a
given repository, but didn't use all of its packages, we wouldn't
previously care about the transitive dependencies of the packages we
weren't using. However, in the go mod world, once we care about the
repository, we care about all of that repository's transitive
dependencies. AFAICT this doesn't affect vendoring.
Fixes#4116
This is part of #4116 since the modules system wants higher versions of these.
golang.org/x/text -> v0.3.0
google.golang.org/grpc -> v1.20.0
google.golang.org/genproto -> master
$ go test google.golang.org/genproto/googleapis/rpc/status
? google.golang.org/genproto/googleapis/rpc/status [no test
files]
$ go test golang.org/x/text/{secure/bidirule,transform,unicode/bidi,unicode/norm}
-count=1
ok golang.org/x/text/secure/bidirule 0.016s
ok golang.org/x/text/transform 0.041s
ok golang.org/x/text/unicode/bidi 0.007s
ok golang.org/x/text/unicode/norm 1.800s
$ go test google.golang.org/grpc/{,balancer{,/base,/roundrobin},codes,connectivity,credentials,encoding,encoding/proto,grpclog,internal{,/backoff,/channelz,/envconfig,/grpcrand,/transport},keepalive,metadata,naming,peer,resolver{,/dns,/passthrough},stats,status,tap}
ok google.golang.org/grpc 22.494s
? google.golang.org/grpc/balancer [no test files]
? google.golang.org/grpc/balancer/base [no test files]
ok google.golang.org/grpc/balancer/roundrobin (cached)
ok google.golang.org/grpc/codes (cached)
? google.golang.org/grpc/connectivity [no test files]
ok google.golang.org/grpc/credentials 0.015s
? google.golang.org/grpc/encoding [no test files]
ok google.golang.org/grpc/encoding/proto 0.056s
ok google.golang.org/grpc/grpclog 0.001s
? google.golang.org/grpc/internal [no test files]
? google.golang.org/grpc/internal/backoff [no test files]
ok google.golang.org/grpc/internal/channelz 0.034s
? google.golang.org/grpc/internal/envconfig [no test files]
? google.golang.org/grpc/internal/grpcrand [no test files]
ok google.golang.org/grpc/internal/transport 81.123s
? google.golang.org/grpc/keepalive [no test files]
ok google.golang.org/grpc/metadata 0.005s
ok google.golang.org/grpc/naming 0.187s
? google.golang.org/grpc/peer [no test files]
? google.golang.org/grpc/resolver [no test files]
ok google.golang.org/grpc/resolver/dns 1.594s
? google.golang.org/grpc/resolver/passthrough [no test files]
ok google.golang.org/grpc/stats 0.036s
ok google.golang.org/grpc/status 0.002s
? google.golang.org/grpc/tap [no test files]
dependabot[bot]