This updates to current `master`,
bc7acd89f703743d050f5cd4a3b9746808e0fdae
Notably, it includes a bug-fix to error handling in the HTTP client,
which we found was hiding errors from CT logs, hindering our debugging.
That fix is
https://github.com/google/certificate-transparency-go/pull/1695
No release has been tagged since this PR merged, so using the `master`
commit.
A few mutual dependencies used by both Boulder and ct-go are updated,
including mysql, otel, and grpc.
Updates:
github.com/weppos/publicsuffix-go from
v0.30.3-0.20240510084413-5f1d03393b3d to
v0.40.3-0.20240815124645-a8ed110559c9
golang.org/x/crypto from v0.24.0 to v0.26.0
golang.org/x/net from v0.26.0 to v0.28.0
golang.org/x/sync from v0.7.0 to v0.8.0
golang.org/x/term from v0.21.0 to v0.23.0
golang.org/x/text from v0.16.0 to v0.17.0
golang.org/x/sys from v0.21.0 to v0.23.0
It's been one month since the last PSL update.
Update:
- github.com/weppos/publicsuffix-go from
v0.30.3-0.20240411085455-21202160c2ed to
v0.30.3-0.20240510084413-5f1d03393b3d
- golang.org/x/crypto from v0.22.0 to v0.23.0 (no changes)
- golang.org/x/net from v0.24.0 to v0.25.0
- golang.org/x/term from v0.19.0 to v0.20.0 (no changes)
- golang.org/x/text from v0.14.0 to v0.15.0 (no changes)
- golang.org/x/sys from v0.19.0 to v0.20.0
Fixes https://github.com/letsencrypt/boulder/issues/7483
Update all golang.org/x/ deps to their latest available version:
- https://golang.org/x/crypto from 0.7.0 to 0.8.0
- https://golang.org/x/exp from v0.0.0-20230118134722-a68e582fa157 to
v0.0.0-20230321023759-10a507213a29
- https://golang.org/x/net from 0.8.0 to 0.9.0
- https://golang.org/x/text from 0.8.0 to 0.9.0
- https://golang.org/x/mod from 0.8.0 to 0.10.0
- https://golang.org/x/tools from 0.6.0 to 0.8.0
This only affects vendored files for /x/exp, /x/net/, and /x/tools/.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
Fork the pieces of the Go standard library's crypto/x509
package which are relevant to parsing, handling, and
signing CRLs.
In our fork, fix an upstream parsing bug, hoist the reasonCode
out of the crlEntryExtensions for easier usability, and enforce
that CRL Numbers are never longer than 20 octets.
Part of #6199
Add Honeycomb tracing to all Boulder components which act as
HTTP servers, gRPC servers, or gRPC clients. Add many values
which we currently emit to logs to the trace spans. Add a way to
configure the Honeycomb integration to our config files, and by
default configure all of our tests to "mute" (send nothing).
Followup changes will refine the configuration, attempt to reduce
the new dependency load, and introduce better sampling.
Part of https://github.com/letsencrypt/dev-misc-tickets/issues/218
Update the pinned version of zlint from v2.2.1 to v3.1.0.
Also update the relevant path from v2 to v3 in both go.mod
and in individual imports. Update the vendored files to match.
No changes from v2.2.1 to v3.1.0 appear to affect the lints
we directly care about (e.g. those that we explicitly ignore).
Fixes#5206
Updates publicsuffix-go to master (983d101) since it doesn't regularly
tag releases. This also pulls in a related update to x/net. All tests
pass.
Fixes#4818
Unit tests are confirmed to pass:
```
~/go/src/golang.org/x/net$ git log --pretty=format:'%h' -n 1
2180aed
~/go/src/golang.org/x/net$ go test ./...
ok golang.org/x/net/bpf 0.494s
ok golang.org/x/net/context 0.058s
ok golang.org/x/net/context/ctxhttp 0.104s
? golang.org/x/net/dict [no test files]
ok golang.org/x/net/dns/dnsmessage 0.074s
ok golang.org/x/net/html 0.097s
ok golang.org/x/net/html/atom 0.002s
ok golang.org/x/net/html/charset 0.020s
ok golang.org/x/net/http/httpguts 0.028s
ok golang.org/x/net/http/httpproxy 0.003s
ok golang.org/x/net/http2 125.352s
ok golang.org/x/net/http2/h2c 0.015s
? golang.org/x/net/http2/h2i [no test files]
ok golang.org/x/net/http2/hpack 0.042s
ok golang.org/x/net/icmp 0.002s
ok golang.org/x/net/idna 0.012s
? golang.org/x/net/internal/iana [no test files]
ok golang.org/x/net/internal/socket 4.560s
ok golang.org/x/net/internal/socks 0.222s
ok golang.org/x/net/internal/sockstest 0.015s
ok golang.org/x/net/internal/timeseries 0.020s
ok golang.org/x/net/ipv4 0.053s
ok golang.org/x/net/ipv6 0.043s
ok golang.org/x/net/nettest 1.057s
ok golang.org/x/net/netutil 0.819s
ok golang.org/x/net/proxy 0.039s
ok golang.org/x/net/publicsuffix 0.146s
ok golang.org/x/net/trace 0.007s
ok golang.org/x/net/webdav 0.091s
ok golang.org/x/net/webdav/internal/xml 0.010s
ok golang.org/x/net/websocket 0.026s
ok golang.org/x/net/xsrftoken 0.019s
```
* cmd: update prometheus.NewProcessCollector args.
There's a new struct `prometheus.ProcessCollectorOpts` that is expected
to be used as the sole argument to `prometheus.NewProcessCollector`. We
don't need to specify `os.Getpid` as the `PidFn` of the struct because
the default is to assume `os.Getpid`. Similarly we don't need to set the
namespace to `""` explicitly, it is the default.
* SA: reimplement db metrics as custom collector.
The modern Prometheus golang API supports translating between legacy
metric sources on the fly with a custom collector. We can use this
approach to collect the metrics from `gorp.DbMap`'s via the `sql.DB`
type's `Stats` function and the returned `sql.DbStats` struct.
This is a cleaner solution overall (we can lose the DB metrics updating
go routine) and it avoids the need to use the now-removed `Set` method
of the `prometheus.Counter` type.
* test: Update CountHistogramSamples.
The `With` function of `prometheus.HistogramVec` types we tend to use as
the argument to `test.CountHistogramSamples` changed to return
a `prometheus.Observer`. Since we only use this function in test
contexts, and only with things that cast back to
a `prometheus.Histogram` we take that approach to fix the problem
without updating call-sites.
A unit test is included to verify that a TLS-ALPN-01 challenge to
a TLS 1.3 only server doesn't succeed when the `GODEBUG` value to
disable TLS 1.3 in `docker-compose.yml` is set. Without this env var
the test fails on the Go 1.13 build because of the new default:
```
=== RUN TestTLSALPN01TLS13
--- FAIL: TestTLSALPN01TLS13 (0.04s)
tlsalpn_test.go:531: expected problem validating TLS-ALPN-01 challenge against a TLS 1.3 only server, got nil
FAIL
FAIL github.com/letsencrypt/boulder/va 0.065s
```
With the env var set the test passes, getting the expected connection
problem reporting a tls error:
```
=== RUN TestTLSALPN01TLS13
2019/09/13 18:59:00 http: TLS handshake error from 127.0.0.1:51240: tls: client offered only unsupported versions: [303 302 301]
--- PASS: TestTLSALPN01TLS13 (0.03s)
PASS
ok github.com/letsencrypt/boulder/va 1.054s
```
Since we plan to eventually enable TLS 1.3 support and the `GODEBUG`
mechanism tested in the above test is platform-wide vs package
specific I decided it wasn't worth the time investment to write a
similar HTTP-01 unit test that verifies the TLS 1.3 behaviour on a
HTTP-01 HTTP->HTTPS redirect.
Resolves https://github.com/letsencrypt/boulder/issues/4415
Because the package versions in go.mod match what we use in Godeps.json,
there are no substantive code diffs. However, there are some tiny
differences resulting from how go mod vendors things differently than
godep:
go mod does not preserve executable permissions on shell scripts
Some packages have import lines like:
package ocsp // import "golang.org/x/crypto/ocsp"
godep used to remove the comment from these lines, but go mod vendor does not.
This introduces several indirect dependencies that we didn't have
before. This is because godep used to operate at a package level, but
go mod operates at a module (~= repository) level. So if we used a
given repository, but didn't use all of its packages, we wouldn't
previously care about the transitive dependencies of the packages we
weren't using. However, in the go mod world, once we care about the
repository, we care about all of that repository's transitive
dependencies. AFAICT this doesn't affect vendoring.
Fixes#4116
$ go test -count=1 golang.org/x/net/{bpf,context,context/ctxhttp,http/httpguts,http2,http2/hpack,idna,internal/iana,internal/socket,internal/timeseries,ipv4,ipv6,trace} golang.org/x/sys/unix golang.org/x/crypto/...
ok golang.org/x/net/bpf 0.464s
ok golang.org/x/net/context 0.064s
ok golang.org/x/net/context/ctxhttp 0.109s
ok golang.org/x/net/http/httpguts 0.008s
ok golang.org/x/net/http2 83.376s
ok golang.org/x/net/http2/hpack 0.049s
ok golang.org/x/net/idna 0.003s
? golang.org/x/net/internal/iana [no test files]
ok golang.org/x/net/internal/socket 0.003s
ok golang.org/x/net/internal/timeseries 0.017s
ok golang.org/x/net/ipv4 0.022s
ok golang.org/x/net/ipv6 0.015s
ok golang.org/x/net/trace 0.010s
ok golang.org/x/sys/unix 0.576s
ok golang.org/x/crypto/acme 4.417s
ok golang.org/x/crypto/acme/autocert 0.222s
? golang.org/x/crypto/acme/autocert/internal/acmetest [no test files]
ok golang.org/x/crypto/argon2 0.072s
ok golang.org/x/crypto/bcrypt 2.331s
ok golang.org/x/crypto/blake2b 0.041s
ok golang.org/x/crypto/blake2s 0.068s
ok golang.org/x/crypto/blowfish 0.007s
ok golang.org/x/crypto/bn256 0.355s
ok golang.org/x/crypto/cast5 3.829s
ok golang.org/x/crypto/chacha20poly1305 0.047s
ok golang.org/x/crypto/cryptobyte 0.003s
? golang.org/x/crypto/cryptobyte/asn1 [no test files]
ok golang.org/x/crypto/curve25519 0.026s
ok golang.org/x/crypto/ed25519 0.121s
? golang.org/x/crypto/ed25519/internal/edwards25519 [no test files]
ok golang.org/x/crypto/hkdf 0.030s
ok golang.org/x/crypto/internal/chacha20 0.091s
ok golang.org/x/crypto/internal/subtle 0.013s
ok golang.org/x/crypto/md4 0.001s
ok golang.org/x/crypto/nacl/auth 1.805s
ok golang.org/x/crypto/nacl/box 0.017s
ok golang.org/x/crypto/nacl/secretbox 0.016s
ok golang.org/x/crypto/nacl/sign 0.022s
ok golang.org/x/crypto/ocsp 0.029s
ok golang.org/x/crypto/openpgp 7.507s
ok golang.org/x/crypto/openpgp/armor 0.022s
ok golang.org/x/crypto/openpgp/clearsign 21.458s
ok golang.org/x/crypto/openpgp/elgamal 0.009s
? golang.org/x/crypto/openpgp/errors [no test files]
ok golang.org/x/crypto/openpgp/packet 0.227s
ok golang.org/x/crypto/openpgp/s2k 8.758s
ok golang.org/x/crypto/otr 0.396s
ok golang.org/x/crypto/pbkdf2 0.060s
ok golang.org/x/crypto/pkcs12 0.069s
ok golang.org/x/crypto/pkcs12/internal/rc2 0.003s
ok golang.org/x/crypto/poly1305 0.012s
ok golang.org/x/crypto/ripemd160 0.043s
ok golang.org/x/crypto/salsa20 0.006s
ok golang.org/x/crypto/salsa20/salsa 0.002s
ok golang.org/x/crypto/scrypt 0.626s
ok golang.org/x/crypto/sha3 0.168s
ok golang.org/x/crypto/ssh 1.290s
ok golang.org/x/crypto/ssh/agent 0.597s
ok golang.org/x/crypto/ssh/knownhosts 0.004s
ok golang.org/x/crypto/ssh/terminal 0.008s
ok golang.org/x/crypto/ssh/test 0.081s
ok golang.org/x/crypto/tea 0.002s
ok golang.org/x/crypto/twofish 0.023s
ok golang.org/x/crypto/xtea 0.009s
ok golang.org/x/crypto/xts 0.001s
Now that Pebble has a `pebble-challtestsrv` we can remove the `challtestrv`
package and associated command from Boulder. I switched CI to use
`pebble-challtestsrv`. Notably this means that we have to add our expected mock
data using the HTTP management interface. The Boulder-tools images are
regenerated to include the `pebble-challtestsrv` command.
Using this approach also allows separating the TLS-ALPN-01 and HTTPS HTTP-01
challenges by binding each challenge type in the `pebble-challtestsrv` to
different interfaces both using the same VA
HTTPS port. Mock DNS directs the VA to the correct interface.
The load-generator command that was previously using the `challtestsrv` package
from Boulder is updated to use a vendored copy of the new
`github.org/letsencrypt/challtestsrv` package.
Vendored dependencies change in two ways:
1) Gomock is updated to the latest release (matching what the Bouldertools image
provides)
2) A couple of new subpackages in `golang.org/x/net/` are added by way of
transitive dependency through the challtestsrv package.
Unit tests are confirmed to pass for `gomock`:
```
~/go/src/github.com/golang/mock/gomock$ git log --pretty=format:'%h' -n 1
51421b9
~/go/src/github.com/golang/mock/gomock$ go test ./...
ok github.com/golang/mock/gomock 0.002s
? github.com/golang/mock/gomock/internal/mock_matcher [no test files]
```
For `/x/net` all tests pass except two `/x/net/icmp` `TestDiag.go` test cases
that we have agreed are OK to ignore.
Resolves https://github.com/letsencrypt/boulder/issues/3962 and
https://github.com/letsencrypt/boulder/issues/3951
Switch linting library to zmap/zlint.
```
github.com/zmap/zlint$ go test ./...
ok github.com/zmap/zlint 0.190s
? github.com/zmap/zlint/cmd/zlint [no test files]
ok github.com/zmap/zlint/lints 0.216s
ok github.com/zmap/zlint/util (cached)
```
Updates `golang.org/x/net` to master (d11bb6cd).
```
$ go test ./...
ok golang.org/x/net/bpf (cached)
ok golang.org/x/net/context (cached)
ok golang.org/x/net/context/ctxhttp (cached)
? golang.org/x/net/dict [no test files]
ok golang.org/x/net/dns/dnsmessage (cached)
ok golang.org/x/net/html (cached)
ok golang.org/x/net/html/atom (cached)
ok golang.org/x/net/html/charset (cached)
ok golang.org/x/net/http/httpguts (cached)
ok golang.org/x/net/http/httpproxy (cached)
ok golang.org/x/net/http2 (cached)
? golang.org/x/net/http2/h2i [no test files]
ok golang.org/x/net/http2/hpack (cached)
ok golang.org/x/net/icmp 0.199s
ok golang.org/x/net/idna (cached)
? golang.org/x/net/internal/iana [no test files]
? golang.org/x/net/internal/nettest [no test files]
ok golang.org/x/net/internal/socket (cached)
ok golang.org/x/net/internal/socks (cached)
ok golang.org/x/net/internal/sockstest (cached)
ok golang.org/x/net/internal/timeseries (cached)
ok golang.org/x/net/ipv4 (cached)
ok golang.org/x/net/ipv6 (cached)
ok golang.org/x/net/nettest (cached)
ok golang.org/x/net/netutil (cached)
ok golang.org/x/net/proxy (cached)
ok golang.org/x/net/publicsuffix (cached)
ok golang.org/x/net/trace (cached)
ok golang.org/x/net/webdav (cached)
ok golang.org/x/net/webdav/internal/xml (cached)
ok golang.org/x/net/websocket (cached)
ok golang.org/x/net/xsrftoken (cached)
```
Fixes#3692.
Godep apparently breaks when trying to parse code that specifies build tags for versions of golang above that with which it was built (which it shouldn't be parsing in the first place). This breaks the travis tests since `golang.org/x/net/context` now contains golang 1.9 specific code. In order to get around this we temporarily disable the error check for `godep save ./...` in test.sh. Opened #2965 to revert this once Godep is fixed or we move to golang 1.9.
Requires an update to `golang.org/x/net` and adding `golang.org/x/text`.
```
[roland@niya:~/gopath/src/golang.org/x/net]$ go test ./...
ok golang.org/x/net/bpf 0.472s
ok golang.org/x/net/context 0.090s
ok golang.org/x/net/context/ctxhttp 0.161s
? golang.org/x/net/dict [no test files]
ok golang.org/x/net/dns/dnsmessage 0.044s
ok golang.org/x/net/html 0.094s
ok golang.org/x/net/html/atom 0.003s
ok golang.org/x/net/html/charset 0.027s
ok golang.org/x/net/http2 80.253s
? golang.org/x/net/http2/h2i [no test files]
ok golang.org/x/net/http2/hpack 0.064s
ok golang.org/x/net/icmp 0.026s
ok golang.org/x/net/idna 0.035s
? golang.org/x/net/internal/iana [no test files]
? golang.org/x/net/internal/nettest [no test files]
ok golang.org/x/net/internal/socket 0.005s
ok golang.org/x/net/internal/timeseries 0.024s
ok golang.org/x/net/ipv4 0.013s
ok golang.org/x/net/ipv6 0.036s
ok golang.org/x/net/lex/httplex 0.004s
ok golang.org/x/net/nettest 1.164s
ok golang.org/x/net/netutil 0.898s
ok golang.org/x/net/proxy 0.004s
ok golang.org/x/net/publicsuffix 0.202s
ok golang.org/x/net/trace 0.018s
ok golang.org/x/net/webdav 0.061s
ok golang.org/x/net/webdav/internal/xml 0.014s
ok golang.org/x/net/websocket 0.022s
ok golang.org/x/net/xsrftoken 0.025s
[roland@niya:~/gopath/src/golang.org/x/text]$ go test ./...
? golang.org/x/text [no test files]
ok golang.org/x/text/cases 0.439s
? golang.org/x/text/cmd/gotext [no test files]
ok golang.org/x/text/collate 0.038s
ok golang.org/x/text/collate/build 0.024s
? golang.org/x/text/collate/tools/colcmp [no test files]
ok golang.org/x/text/currency 2.961s
ok golang.org/x/text/encoding 0.005s
ok golang.org/x/text/encoding/charmap 0.060s
ok golang.org/x/text/encoding/htmlindex 0.005s
ok golang.org/x/text/encoding/ianaindex 0.030s
? golang.org/x/text/encoding/internal [no test files]
? golang.org/x/text/encoding/internal/enctest [no test files]
? golang.org/x/text/encoding/internal/identifier [no test files]
ok golang.org/x/text/encoding/japanese 0.098s
ok golang.org/x/text/encoding/korean 0.032s
ok golang.org/x/text/encoding/simplifiedchinese 0.100s
ok golang.org/x/text/encoding/traditionalchinese 0.012s
ok golang.org/x/text/encoding/unicode 0.013s
ok golang.org/x/text/encoding/unicode/utf32 0.071s
ok golang.org/x/text/feature/plural 0.352s
ok golang.org/x/text/internal 0.009s
ok golang.org/x/text/internal/catmsg 0.034s
ok golang.org/x/text/internal/colltab 1.817s
ok golang.org/x/text/internal/export/idna 0.040s
? golang.org/x/text/internal/format [no test files]
? golang.org/x/text/internal/gen [no test files]
ok golang.org/x/text/internal/number 0.028s
ok golang.org/x/text/internal/stringset 0.021s
ok golang.org/x/text/internal/tag 0.044s
? golang.org/x/text/internal/testtext [no test files]
ok golang.org/x/text/internal/triegen 0.357s
ok golang.org/x/text/internal/ucd 0.023s
? golang.org/x/text/internal/utf8internal [no test files]
ok golang.org/x/text/language 0.033s
ok golang.org/x/text/language/display 3.917s
ok golang.org/x/text/message 0.033s
ok golang.org/x/text/message/catalog 0.069s
ok golang.org/x/text/runes 0.039s
ok golang.org/x/text/search 0.019s
? golang.org/x/text/secure [no test files]
ok golang.org/x/text/secure/bidirule 0.032s
ok golang.org/x/text/secure/precis 0.066s
ok golang.org/x/text/transform 0.106s
? golang.org/x/text/unicode [no test files]
ok golang.org/x/text/unicode/bidi 0.026s
ok golang.org/x/text/unicode/cldr 0.114s
ok golang.org/x/text/unicode/norm 4.009s
ok golang.org/x/text/unicode/rangetable 1.516s
ok golang.org/x/text/unicode/runenames 0.011s
ok golang.org/x/text/width 0.310s
```
Fixes#2963.
Add feature flagged support for issuing for IDNs, fixes#597.
This patch expects that clients have performed valid IDN2008 encoding on any label that includes unicode characters. Invalid encodings (including non-compatible IDN2003 encoding) will be rejected. No script-mixing or script exclusion checks are performed as we assume that if a name is resolvable that it conforms to the registrar's policies on these matters and if it uses non-standard scripts in sub-domains etc that browsers should be the ones choosing how to display those names.
Required a full update of the golang.org/x/net tree to pull in golang.org/x/net/idna, all test suites pass.
When a CAA request to Unbound times out, fall back to checking CAA via Google Public DNS' HTTPS API, through multiple proxies so as to hit geographically distributed paths. All successful multipath responses must be identical in order to succeed, and at most one can fail.
Fixes#1618
Matthew McPherrin
dependabot[bot]
Aaron Gable
Adin Schmahmann
Phil Porada
Samantha Frank
Jacob Hoffman-Andrews
Roland Bracewell Shoemaker
Daniel McCarney
Daniel McCarney
Roland Bracewell Shoemaker