Updates:
- go.opentelemetry.io/contrib/instrumentation (and subpackages) from 0.52.0 to 0.55.0
- go.opentelemetry.io/otel (and subpackages) from 1.27.0 to 1.30.0
Upstream release notes:
https://github.com/open-telemetry/opentelemetry-go-contrib/releases
Also transitively updates a few golang.org/x/ dependencies, and the grpc and protobuf
dependencies.
Updates:
github.com/weppos/publicsuffix-go from
v0.30.3-0.20240510084413-5f1d03393b3d to
v0.40.3-0.20240815124645-a8ed110559c9
golang.org/x/crypto from v0.24.0 to v0.26.0
golang.org/x/net from v0.26.0 to v0.28.0
golang.org/x/sync from v0.7.0 to v0.8.0
golang.org/x/term from v0.21.0 to v0.23.0
golang.org/x/text from v0.16.0 to v0.17.0
golang.org/x/sys from v0.21.0 to v0.23.0
It's been one month since the last PSL update.
Update:
- github.com/weppos/publicsuffix-go from
v0.30.3-0.20240411085455-21202160c2ed to
v0.30.3-0.20240510084413-5f1d03393b3d
- golang.org/x/crypto from v0.22.0 to v0.23.0 (no changes)
- golang.org/x/net from v0.24.0 to v0.25.0
- golang.org/x/term from v0.19.0 to v0.20.0 (no changes)
- golang.org/x/text from v0.14.0 to v0.15.0 (no changes)
- golang.org/x/sys from v0.19.0 to v0.20.0
Fixes https://github.com/letsencrypt/boulder/issues/7483
We last updated the PSL two months ago, and the latest officially-tagged
release of publicsuffic-go is just one month old, but the domain
prompting the latest request to update the PSL landed just a few days
ago, so update to the latest commit rather than the latest release.
Also incidentally updates x/crypto, x/net, x/term, and x/sys, but brings
in no meaningful updates for any of them.
Fixes https://github.com/letsencrypt/boulder/issues/7428
Upgrade from the old go-jose v2.6.1 to the newly minted go-jose v4.0.1.
Cleans up old code now that `jose.ParseSigned` can take a list of
supported signature algorithms.
Fixes https://github.com/letsencrypt/boulder/issues/7390
---------
Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
Upgrade to zlint v3.6.0
Two new lints are triggered in various places:
aia_contains_internal_names is ignored in integration test
configurations, and unit tests are updated to have more realistic URLs.
The w_subject_common_name_included lint needs to be ignored where we'd
ignored n_subject_common_name_included before.
Related to https://github.com/letsencrypt/boulder/issues/7261
Add a new shared config stanza which all boulder components can use to
configure their Open Telemetry tracing. This allows components to
specify where their traces should be sent, what their sampling ratio
should be, and whether or not they should respect their parent's
sampling decisions (so that web front-ends can ignore sampling info
coming from outside our infrastructure). It's likely we'll need to
evolve this configuration over time, but this is a good starting point.
Add basic Open Telemetry setup to our existing cmd.StatsAndLogging
helper, so that it gets initialized at the same time as our other
observability helpers. This sets certain default fields on all
traces/spans generated by the service. Currently these include the
service name, the service version, and information about the telemetry
SDK itself. In the future we'll likely augment this with information
about the host and process.
Finally, add instrumentation for the HTTP servers and grpc
clients/servers. This gives us a starting point of being able to monitor
Boulder, but is fairly minimal as this PR is already somewhat unwieldy:
It's really only enough to understand that everything is wired up
properly in the configuration. In subsequent work we'll enhance those
spans with more data, and add more spans for things not automatically
traced here.
Fixes https://github.com/letsencrypt/boulder/issues/6361
---------
Co-authored-by: Aaron Gable <aaron@aarongable.com>
Right now, Boulder expects to be able to connect to syslog, and panics
if it's not available. We'd like to be able to log to stdout/stderr as a
replacement for syslog.
- Add a detailed timestamp (down to microseconds, same as we collect in
prod via syslog).
- Remove the escape codes for colorizing output.
- Report the severity level numerically rather than with a letter prefix.
Add locking for stdout/stderr and syslog logs. Neither the [syslog] package
nor the [os] package document concurrency-safety, and the Go rule is: if
it's not documented to be concurrent-safe, it's not. Notably the [log.Logger]
package is documented to be concurrent-safe, and a look at its implementation
shows it uses a Mutex internally.
Remove places that use the singleton `blog.Get()`, and instead pass through
a logger from main in all the places that need it.
[syslog]: https://pkg.go.dev/log/syslog
[os]: https://pkg.go.dev/os
[log.Logger]: https://pkg.go.dev/log#Logger
Diff: https://github.com/prometheus/client_golang/compare/v1.7.1...v1.12.1
Changes:
* 1.12.1 / 2022-01-29
- [BUGFIX] Make the Go 1.17 collector concurrency-safe
- Use simpler locking in the Go 1.17 collector
- [BUGFIX] Reduce granularity of histogram buckets for Go 1.17 collector
- [ENHANCEMENT] API client: make HTTP reads more efficient
* 1.12.0 / 2022-01-19
- [CHANGE] example/random: Move flags and metrics into main()
- [FEATURE] API client: Support wal replay status api
- [FEATURE] Use the runtime/metrics package for the Go collector for 1.17+
- [ENHANCEMENT] API client: Update /api/v1/status/tsdb to include headStats
- [ENHANCEMENT] promhttp: Check validity of method and code label values
* 1.11.0 / 2021-06-07
- [CHANGE] Add new collectors package.
- [CHANGE] prometheus.NewExpvarCollector is deprecated, use collectors.NewExpvarCollector instead.
- [CHANGE] prometheus.NewGoCollector is deprecated, use collectors.NewGoCollector instead.
- [CHANGE] prometheus.NewBuildInfoCollector is deprecated, use collectors.NewBuildInfoCollector instead.
- [FEATURE] Add new collector for database/sql#DBStats.
- [FEATURE] API client: Add exemplars API support.
- [ENHANCEMENT] API client: Add newer fields to Rules API.
- [ENHANCEMENT] API client: Add missing fields to Targets API.
* 1.10.0 / 2021-03-18
- [CHANGE] Minimum required Go version is now 1.13.
- [CHANGE] API client: Add matchers to LabelNames and LabesValues.
- [FEATURE] API client: Add buildinfo call.
- [BUGFIX] Fix build on riscv64.
* 1.9.0 / 2020-12-17
- [FEATURE] NewPidFileFn helper to create process collectors for processes whose PID is read from a file.
- [BUGFIX] promhttp: Prevent endless loop in InstrumentHandler... middlewares with invalid metric or label names.
* 1.8.0 / 2020-10-15
- [CHANGE] API client: Use time.Time rather than string for timestamps in RuntimeinfoResult.
- [FEATURE] Export MetricVec to facilitate implementation of vectors of custom Metric types.
- [FEATURE] API client: Support /status/tsdb endpoint.
- [ENHANCEMENT] API client: Enable GET fallback on status code 501.
- [ENHANCEMENT] Remove Metric references after reslicing to free up more memory.
Additional transitive dependency updates:
* https://github.com/prometheus/common/compare/v0.10.0...v0.32.1
* https://github.com/prometheus/procfs/compare/v0.1.3...v0.7.3
* https://github.com/golang/appengine/compare/v1.6.5...v1.6.6
* cb27e3aa20...8632dd7979
* 0f9fa26af8...5a964db013
Add Honeycomb tracing to all Boulder components which act as
HTTP servers, gRPC servers, or gRPC clients. Add many values
which we currently emit to logs to the trace spans. Add a way to
configure the Honeycomb integration to our config files, and by
default configure all of our tests to "mute" (send nothing).
Followup changes will refine the configuration, attempt to reduce
the new dependency load, and introduce better sampling.
Part of https://github.com/letsencrypt/dev-misc-tickets/issues/218
Update the pinned version of zlint from v2.2.1 to v3.1.0.
Also update the relevant path from v2 to v3 in both go.mod
and in individual imports. Update the vendored files to match.
No changes from v2.2.1 to v3.1.0 appear to affect the lints
we directly care about (e.g. those that we explicitly ignore).
Fixes#5206
* cmd: update prometheus.NewProcessCollector args.
There's a new struct `prometheus.ProcessCollectorOpts` that is expected
to be used as the sole argument to `prometheus.NewProcessCollector`. We
don't need to specify `os.Getpid` as the `PidFn` of the struct because
the default is to assume `os.Getpid`. Similarly we don't need to set the
namespace to `""` explicitly, it is the default.
* SA: reimplement db metrics as custom collector.
The modern Prometheus golang API supports translating between legacy
metric sources on the fly with a custom collector. We can use this
approach to collect the metrics from `gorp.DbMap`'s via the `sql.DB`
type's `Stats` function and the returned `sql.DbStats` struct.
This is a cleaner solution overall (we can lose the DB metrics updating
go routine) and it avoids the need to use the now-removed `Set` method
of the `prometheus.Counter` type.
* test: Update CountHistogramSamples.
The `With` function of `prometheus.HistogramVec` types we tend to use as
the argument to `test.CountHistogramSamples` changed to return
a `prometheus.Observer`. Since we only use this function in test
contexts, and only with things that cast back to
a `prometheus.Histogram` we take that approach to fix the problem
without updating call-sites.
A unit test is included to verify that a TLS-ALPN-01 challenge to
a TLS 1.3 only server doesn't succeed when the `GODEBUG` value to
disable TLS 1.3 in `docker-compose.yml` is set. Without this env var
the test fails on the Go 1.13 build because of the new default:
```
=== RUN TestTLSALPN01TLS13
--- FAIL: TestTLSALPN01TLS13 (0.04s)
tlsalpn_test.go:531: expected problem validating TLS-ALPN-01 challenge against a TLS 1.3 only server, got nil
FAIL
FAIL github.com/letsencrypt/boulder/va 0.065s
```
With the env var set the test passes, getting the expected connection
problem reporting a tls error:
```
=== RUN TestTLSALPN01TLS13
2019/09/13 18:59:00 http: TLS handshake error from 127.0.0.1:51240: tls: client offered only unsupported versions: [303 302 301]
--- PASS: TestTLSALPN01TLS13 (0.03s)
PASS
ok github.com/letsencrypt/boulder/va 1.054s
```
Since we plan to eventually enable TLS 1.3 support and the `GODEBUG`
mechanism tested in the above test is platform-wide vs package
specific I decided it wasn't worth the time investment to write a
similar HTTP-01 unit test that verifies the TLS 1.3 behaviour on a
HTTP-01 HTTP->HTTPS redirect.
Resolves https://github.com/letsencrypt/boulder/issues/4415
Because the package versions in go.mod match what we use in Godeps.json,
there are no substantive code diffs. However, there are some tiny
differences resulting from how go mod vendors things differently than
godep:
go mod does not preserve executable permissions on shell scripts
Some packages have import lines like:
package ocsp // import "golang.org/x/crypto/ocsp"
godep used to remove the comment from these lines, but go mod vendor does not.
This introduces several indirect dependencies that we didn't have
before. This is because godep used to operate at a package level, but
go mod operates at a module (~= repository) level. So if we used a
given repository, but didn't use all of its packages, we wouldn't
previously care about the transitive dependencies of the packages we
weren't using. However, in the go mod world, once we care about the
repository, we care about all of that repository's transitive
dependencies. AFAICT this doesn't affect vendoring.
Fixes#4116
$ go test -count=1 golang.org/x/net/{bpf,context,context/ctxhttp,http/httpguts,http2,http2/hpack,idna,internal/iana,internal/socket,internal/timeseries,ipv4,ipv6,trace} golang.org/x/sys/unix golang.org/x/crypto/...
ok golang.org/x/net/bpf 0.464s
ok golang.org/x/net/context 0.064s
ok golang.org/x/net/context/ctxhttp 0.109s
ok golang.org/x/net/http/httpguts 0.008s
ok golang.org/x/net/http2 83.376s
ok golang.org/x/net/http2/hpack 0.049s
ok golang.org/x/net/idna 0.003s
? golang.org/x/net/internal/iana [no test files]
ok golang.org/x/net/internal/socket 0.003s
ok golang.org/x/net/internal/timeseries 0.017s
ok golang.org/x/net/ipv4 0.022s
ok golang.org/x/net/ipv6 0.015s
ok golang.org/x/net/trace 0.010s
ok golang.org/x/sys/unix 0.576s
ok golang.org/x/crypto/acme 4.417s
ok golang.org/x/crypto/acme/autocert 0.222s
? golang.org/x/crypto/acme/autocert/internal/acmetest [no test files]
ok golang.org/x/crypto/argon2 0.072s
ok golang.org/x/crypto/bcrypt 2.331s
ok golang.org/x/crypto/blake2b 0.041s
ok golang.org/x/crypto/blake2s 0.068s
ok golang.org/x/crypto/blowfish 0.007s
ok golang.org/x/crypto/bn256 0.355s
ok golang.org/x/crypto/cast5 3.829s
ok golang.org/x/crypto/chacha20poly1305 0.047s
ok golang.org/x/crypto/cryptobyte 0.003s
? golang.org/x/crypto/cryptobyte/asn1 [no test files]
ok golang.org/x/crypto/curve25519 0.026s
ok golang.org/x/crypto/ed25519 0.121s
? golang.org/x/crypto/ed25519/internal/edwards25519 [no test files]
ok golang.org/x/crypto/hkdf 0.030s
ok golang.org/x/crypto/internal/chacha20 0.091s
ok golang.org/x/crypto/internal/subtle 0.013s
ok golang.org/x/crypto/md4 0.001s
ok golang.org/x/crypto/nacl/auth 1.805s
ok golang.org/x/crypto/nacl/box 0.017s
ok golang.org/x/crypto/nacl/secretbox 0.016s
ok golang.org/x/crypto/nacl/sign 0.022s
ok golang.org/x/crypto/ocsp 0.029s
ok golang.org/x/crypto/openpgp 7.507s
ok golang.org/x/crypto/openpgp/armor 0.022s
ok golang.org/x/crypto/openpgp/clearsign 21.458s
ok golang.org/x/crypto/openpgp/elgamal 0.009s
? golang.org/x/crypto/openpgp/errors [no test files]
ok golang.org/x/crypto/openpgp/packet 0.227s
ok golang.org/x/crypto/openpgp/s2k 8.758s
ok golang.org/x/crypto/otr 0.396s
ok golang.org/x/crypto/pbkdf2 0.060s
ok golang.org/x/crypto/pkcs12 0.069s
ok golang.org/x/crypto/pkcs12/internal/rc2 0.003s
ok golang.org/x/crypto/poly1305 0.012s
ok golang.org/x/crypto/ripemd160 0.043s
ok golang.org/x/crypto/salsa20 0.006s
ok golang.org/x/crypto/salsa20/salsa 0.002s
ok golang.org/x/crypto/scrypt 0.626s
ok golang.org/x/crypto/sha3 0.168s
ok golang.org/x/crypto/ssh 1.290s
ok golang.org/x/crypto/ssh/agent 0.597s
ok golang.org/x/crypto/ssh/knownhosts 0.004s
ok golang.org/x/crypto/ssh/terminal 0.008s
ok golang.org/x/crypto/ssh/test 0.081s
ok golang.org/x/crypto/tea 0.002s
ok golang.org/x/crypto/twofish 0.023s
ok golang.org/x/crypto/xtea 0.009s
ok golang.org/x/crypto/xts 0.001s
Fixes#3837
```
$ go test google.golang.org/grpc/...
ok google.golang.org/grpc 24.275s
? google.golang.org/grpc/balancer [no test files]
? google.golang.org/grpc/balancer/base [no test files]
ok google.golang.org/grpc/balancer/grpclb 7.271s
? google.golang.org/grpc/balancer/grpclb/grpc_lb_v1 [no test files]
ok google.golang.org/grpc/balancer/roundrobin 0.427s
ok google.golang.org/grpc/benchmark 0.006s [no tests to run]
? google.golang.org/grpc/benchmark/benchmain [no test files]
? google.golang.org/grpc/benchmark/benchresult [no test files]
? google.golang.org/grpc/benchmark/client [no test files]
? google.golang.org/grpc/benchmark/grpc_testing [no test files]
ok google.golang.org/grpc/benchmark/latency 1.012s
ok google.golang.org/grpc/benchmark/primitives 0.036s [no tests to run]
? google.golang.org/grpc/benchmark/server [no test files]
? google.golang.org/grpc/benchmark/stats [no test files]
? google.golang.org/grpc/benchmark/worker [no test files]
? google.golang.org/grpc/binarylog/grpc_binarylog_v1 [no test files]
? google.golang.org/grpc/channelz/grpc_channelz_v1 [no test files]
ok google.golang.org/grpc/channelz/service 0.024s
ok google.golang.org/grpc/codes 0.006s
? google.golang.org/grpc/connectivity [no test files]
ok google.golang.org/grpc/credentials 0.014s
ok google.golang.org/grpc/credentials/alts 0.009s
? google.golang.org/grpc/credentials/alts/internal [no test files]
ok google.golang.org/grpc/credentials/alts/internal/authinfo 0.006s
ok google.golang.org/grpc/credentials/alts/internal/conn 0.133s
ok google.golang.org/grpc/credentials/alts/internal/handshaker 0.045s
ok google.golang.org/grpc/credentials/alts/internal/handshaker/service 0.013s
? google.golang.org/grpc/credentials/alts/internal/proto/grpc_gcp [no test files]
? google.golang.org/grpc/credentials/alts/internal/testutil [no test files]
? google.golang.org/grpc/credentials/google [no test files]
? google.golang.org/grpc/credentials/oauth [no test files]
? google.golang.org/grpc/encoding [no test files]
? google.golang.org/grpc/encoding/gzip [no test files]
ok google.golang.org/grpc/encoding/proto 0.075s
? google.golang.org/grpc/examples/helloworld/greeter_client [no test files]
? google.golang.org/grpc/examples/helloworld/greeter_server [no test files]
? google.golang.org/grpc/examples/helloworld/helloworld [no test files]
ok google.golang.org/grpc/examples/helloworld/mock_helloworld 0.011s
? google.golang.org/grpc/examples/oauth/client [no test files]
? google.golang.org/grpc/examples/oauth/server [no test files]
? google.golang.org/grpc/examples/route_guide/client [no test files]
ok google.golang.org/grpc/examples/route_guide/mock_routeguide 0.039s
? google.golang.org/grpc/examples/route_guide/routeguide [no test files]
? google.golang.org/grpc/examples/route_guide/server [no test files]
? google.golang.org/grpc/examples/rpc_errors/client [no test files]
? google.golang.org/grpc/examples/rpc_errors/server [no test files]
ok google.golang.org/grpc/grpclog 0.007s
? google.golang.org/grpc/grpclog/glogger [no test files]
ok google.golang.org/grpc/health 0.005s
? google.golang.org/grpc/health/grpc_health_v1 [no test files]
? google.golang.org/grpc/internal [no test files]
? google.golang.org/grpc/internal/backoff [no test files]
ok google.golang.org/grpc/internal/binarylog 0.006s
ok google.golang.org/grpc/internal/channelz 0.012s
? google.golang.org/grpc/internal/envconfig [no test files]
? google.golang.org/grpc/internal/grpcrand [no test files]
ok google.golang.org/grpc/internal/grpcsync 0.013s
ok google.golang.org/grpc/internal/leakcheck 4.052s
? google.golang.org/grpc/internal/syscall [no test files]
ok google.golang.org/grpc/internal/testutils 0.002s
ok google.golang.org/grpc/internal/transport 81.968s
? google.golang.org/grpc/interop [no test files]
? google.golang.org/grpc/interop/alts/client [no test files]
? google.golang.org/grpc/interop/alts/server [no test files]
? google.golang.org/grpc/interop/client [no test files]
? google.golang.org/grpc/interop/fake_grpclb [no test files]
? google.golang.org/grpc/interop/grpc_testing [no test files]
? google.golang.org/grpc/interop/http2 [no test files]
? google.golang.org/grpc/interop/server [no test files]
? google.golang.org/grpc/keepalive [no test files]
ok google.golang.org/grpc/metadata 0.006s
ok google.golang.org/grpc/naming 0.159s
? google.golang.org/grpc/peer [no test files]
ok google.golang.org/grpc/reflection 0.016s
? google.golang.org/grpc/reflection/grpc_reflection_v1alpha [no test files]
? google.golang.org/grpc/reflection/grpc_testing [no test files]
? google.golang.org/grpc/reflection/grpc_testingv3 [no test files]
? google.golang.org/grpc/resolver [no test files]
ok google.golang.org/grpc/resolver/dns 1.370s
? google.golang.org/grpc/resolver/manual [no test files]
? google.golang.org/grpc/resolver/passthrough [no test files]
ok google.golang.org/grpc/stats 0.104s
? google.golang.org/grpc/stats/grpc_testing [no test files]
ok google.golang.org/grpc/status 0.017s
? google.golang.org/grpc/stress/client [no test files]
? google.golang.org/grpc/stress/grpc_testing [no test files]
? google.golang.org/grpc/stress/metrics_client [no test files]
? google.golang.org/grpc/tap [no test files]
ok google.golang.org/grpc/test 33.866s
ok google.golang.org/grpc/test/bufconn 0.005s
? google.golang.org/grpc/test/codec_perf [no test files]
? google.golang.org/grpc/test/go_vet [no test files]
? google.golang.org/grpc/test/grpc_testing [no test files]
```
dependabot[bot]
Aaron Gable
Adin Schmahmann
Samantha Frank
Phil Porada
Matthew McPherrin
Matthew McPherrin
Jacob Hoffman-Andrews
Jacob Hoffman-Andrews
milgradesec
Daniel McCarney