go1.17.9 (released 2022-04-12) includes security fixes to the crypto/elliptic and encoding/pem packages, as well as bug fixes to the linker and runtime. See the [Go 1.17.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.17.9+label%3ACherryPickApproved) on our issue tracker for details.
go1.18.1 (released 2022-04-12) includes security fixes to the crypto/elliptic, crypto/x509, and encoding/pem packages, as well as bug fixes to the compiler, linker, runtime, the go command, vet, and the bytes, crypto/x509, and go/types packages. See the [Go 1.18.1 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.18.1+label%3ACherryPickApproved) on our issue tracker for details.
First commit adding support for tooling to aid in the tracking and remediation
of incidents.
- Add new SA method `IncidentsForSerial`
- Add database models for `incident`s and `incidentCert`s
- Add protobuf type for `incident`
- Add database migrations for `incidents`, `incident_foo`, and `incident_bar`
- Give db user `sa` permissions to `incidents`, `incident_foo`, and
`incident_bar`
Part Of #5947
Update from v1.42.1 to v1.45.2. This version has both
better support for go1.18, and better automatic detection
of which go version to use (based on our go.mod file,
it will currently use go1.17).
Slightly refactor `validateTLSALPN01` to use a common function
to format the error messages it returns. This reduces code duplication
and makes the important validation logic easier to follow.
Fixes#5922
If a Redis client is configured for the SA service, OCSP responses created
during a revocation event will be written to Redis on a best effort basis.
Use the OCSP response NextUpdate time as the expiration time for the
redis entry. Change the new issuance OCSP storage to do the same.
Fixes#5888
As of the update from 1.7.1 to 1.12.1 (#5974), the
`prometheus.NewFooCollector` methods have been deprecated
and replaced by equivalent `collectors.NewFooCollector` methods.
Update the PSL from 7594db4f858a (Oct 2021) to 9a40b608a236
(March 2022). This adds approximately 165 new entries and removes
approximately 28 old entries.
Fixes#6022
Diff: https://github.com/prometheus/client_golang/compare/v1.7.1...v1.12.1
Changes:
* 1.12.1 / 2022-01-29
- [BUGFIX] Make the Go 1.17 collector concurrency-safe
- Use simpler locking in the Go 1.17 collector
- [BUGFIX] Reduce granularity of histogram buckets for Go 1.17 collector
- [ENHANCEMENT] API client: make HTTP reads more efficient
* 1.12.0 / 2022-01-19
- [CHANGE] example/random: Move flags and metrics into main()
- [FEATURE] API client: Support wal replay status api
- [FEATURE] Use the runtime/metrics package for the Go collector for 1.17+
- [ENHANCEMENT] API client: Update /api/v1/status/tsdb to include headStats
- [ENHANCEMENT] promhttp: Check validity of method and code label values
* 1.11.0 / 2021-06-07
- [CHANGE] Add new collectors package.
- [CHANGE] prometheus.NewExpvarCollector is deprecated, use collectors.NewExpvarCollector instead.
- [CHANGE] prometheus.NewGoCollector is deprecated, use collectors.NewGoCollector instead.
- [CHANGE] prometheus.NewBuildInfoCollector is deprecated, use collectors.NewBuildInfoCollector instead.
- [FEATURE] Add new collector for database/sql#DBStats.
- [FEATURE] API client: Add exemplars API support.
- [ENHANCEMENT] API client: Add newer fields to Rules API.
- [ENHANCEMENT] API client: Add missing fields to Targets API.
* 1.10.0 / 2021-03-18
- [CHANGE] Minimum required Go version is now 1.13.
- [CHANGE] API client: Add matchers to LabelNames and LabesValues.
- [FEATURE] API client: Add buildinfo call.
- [BUGFIX] Fix build on riscv64.
* 1.9.0 / 2020-12-17
- [FEATURE] NewPidFileFn helper to create process collectors for processes whose PID is read from a file.
- [BUGFIX] promhttp: Prevent endless loop in InstrumentHandler... middlewares with invalid metric or label names.
* 1.8.0 / 2020-10-15
- [CHANGE] API client: Use time.Time rather than string for timestamps in RuntimeinfoResult.
- [FEATURE] Export MetricVec to facilitate implementation of vectors of custom Metric types.
- [FEATURE] API client: Support /status/tsdb endpoint.
- [ENHANCEMENT] API client: Enable GET fallback on status code 501.
- [ENHANCEMENT] Remove Metric references after reslicing to free up more memory.
Additional transitive dependency updates:
* https://github.com/prometheus/common/compare/v0.10.0...v0.32.1
* https://github.com/prometheus/procfs/compare/v0.1.3...v0.7.3
* https://github.com/golang/appengine/compare/v1.6.5...v1.6.6
* cb27e3aa20...8632dd7979
* 0f9fa26af8...5a964db013
Simplify the WFE `RevokeCertificate` API method in three ways:
- Remove most of the logic checking if the requester is authorized to
revoke the certificate in question (based on who is making the
request, what authorizations they have, and what reason they're
requesting). That checking is now done by the RA. Instead, simply
verify that the JWS is authenticated.
- Remove the hard-to-read `authorizedToRevoke` callbacks, and make the
`revokeCertBySubscriberKey` (nee `revokeCertByKeyID`) and
`revokeCertByCertKey` (nee `revokeCertByJWK`) helpers much more
straight-line in their execution logic.
- Call the RA's new `RevokeCertByApplicant` and `RevokeCertByKey` gRPC
methods, rather than the deprecated `RevokeCertificateWithReg`.
This change, without any flag flips, should be invisible to the
end-user. It will slightly change some of our log message formats.
However, by now relying on the new RA gRPC revocation methods, this
change allows us to change our revocation policies by enabling the
`AllowDoubleRevocation` and `MozRevocationReasons` feature flags, which
affect the behavior of those new helpers.
Fixes#5936
Logging every 10 is quite noisy; instead adopt the same strategy we use
for errors, and log all of them at first, fading out to fewer of them as
we get to bigger numbers.
- Add new configuration key `throughput`, a mapping which contains all
throughput related akamai-purger settings.
- Deprecate configuration key `purgeInterval` in favor of `purgeBatchInterval` in
the new `throughput` configuration mapping.
- When no `throughput` or `purgeInterval` is provided, the purger uses optimized
default settings which offer 1.9x the throughput of current production settings.
- At startup, all throughput related settings are modeled to ensure that we
don't exceed the limits imposed on us by Akamai.
- Queue is now `[][]string`, instead of `[]string`.
- When a given queue entry is purged we know all 3 of it's URLs were purged.
- At startup we know the size of a theoretical request to purge based on the
number of queue entries included
- Raises the queue size from ~333-thousand cached OCSP responses to
1.25-million, which is roughly 6 hours of work using the optimized default
settings
- Raise `purgeInterval` in test config from 1ms, which violates API limits, to 800ms
Fixes#5984
Adds a rocsp redis client to the sa if cluster information is provided in the
sa config. If a redis cluster is configured, all new certificate OCSP
responses added with sa.AddPrecertificate will attempt to be written to
the redis cluster, but will not block or fail on errors.
Fixes: #5871
- Remove GOPATH-style path structure, which isn't needed with Go
modules.
- Remove check for existing of docker buildx builder instance, since it
was unreliable.
That causes the VA to emit ValidationRecords with the OldTLS bit set if
it observes a redirect to HTTPS that negotiates TLS < 1.2.
I've manually tested but there is not yet an integration test. I need
to make a parallel change in challtestsrv and then incorporate here.
Go releases are PGP-signed with a key from
https://www.google.com/linuxrepositories/. We can improve our confidence
in the provenance of our Go binaries by verifying that signature. This
adds a script that encapsulates the public key, the fetch, and the
verification, outputting go.tar.gz once it's verified.
So far this only adds to the release workflow in CI. It needs a little
more thought about how to organize boulder-tools so it can consume
fetch-and-verify-go.sh (which is in a different directory and therefore
not part of the input to `docker build`).
Add two new gRPC methods to the SA:
- `RevokeCertByKey` will be used when the API request was signed by the
certificate's keypair, rather than a Subscriber keypair. If the
request is for reason `keyCompromise`, it will ensure that the key is
added to the blocked keys table, and will attempt to "re-revoke" a
certificate that was already revoked for some other reason.
- `RevokeCertByApplicant` supports both the path where the original
subscriber or another account which has proven control over all of the
identifier in the certificate requests revocation via the API. It does
not allow the requested reason to be `keyCompromise`, as these
requests do not represent a demonstration of key compromise.
In addition, add a new feature flag `MozRevocationReasons` which
controls the behavior of these new methods. If the flag is not set, they
behave like they have historically (see above). If the flag is set to true,
then the new methods enforce the upcoming Mozilla policies around
revocation reasons, namely:
- Only the original Subscriber can choose the revocation reason; other
clients will get a set reason code based on the method of requesting
revocation. When the original Subscriber requests reason
`keyCompromise`, this request will be honored, but the key will not be
blocked and other certificates with that key will not also be revoked.
- Revocations signed with the certificate key will always get reason
`keyCompromise`, because we do not know who is sending the request and
therefore must assume that the use of the key in this way represents
compromise. Because these requests will always be fore reason
`keyCompromise`, they will always be added to the blocked keys table
and they will always attempt "re-revocation".
- Revocations authorized via control of all names in the cert will
always get reason `cessationOfOperation`, which is to be used when the
original Subscriber does not control all names in the certificate
anymore.
Finally, update the existing `AdministrativelyRevokeCertificate` method
to use the new helper functions shared by the two new methods.
Part of #5936
- Add a CI workflow which publishes a GitHub Release containing a Debian package
when a release tag is pushed
- Add a script, called by the CI host, that installs all of the dependencies
necessary to `make` a Debian package
- Remove the, now defunct, goreleaser config file
Fixes#5970
This requires using GODEBUG to enable a couple of thing turned off by go1.18 (TLS 1.0/1.1, SHA-1 CSRs).
Also add help for a failure mode of cross builds.
Reverts letsencrypt/boulder#5963
Turns out the tests are still flaky -- using the `grpc.WaitForReady(true)`
connection option results in sometimes seeing 9 entries added to the
purger queue, and sometimes 10 entries. Reverting because flakiness
on main should not be tolerated.
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.36.1 to 1.44.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.36.1...v1.44.0)
Also update akamai-purger integration test to avoid experimental API.
The `conn.GetState()` API is marked experimental and may change behavior
at any time. It appears to have changed between v1.36.1 and v1.44.0,
and so the akamai-purger integration tests which rely on it break.
Rather than writing our own loop which polls `conn.GetState()`, just
use the stable `WaitForReady(true)` connection option, and apply it to
all connections by setting it as a default option in the dial options.
- Make maximum queue size configurable via a new configuration key:
'MaxQueueSize'.
- Default 'MaxQueueSize' to the previous value (1M) when 'MaxQueueSize'
isn't specified.
- akamaiPurger.purge() will only place the URLs starting at the first entry of
the failed batch where a failure was encountered instead of the entire set
that was originally passed.
- Add a test to ensure that these changes are working as intended.
- Make the purge batching easier to understand with some minor changes
to variable names
- Responses whose HTTP status code is not 201 will no longer be unmarshaled
- Logs will explicitly call out if a response indicates that we've exceeded any
rate limits imposed by Akamai.
Fixes#5917
Make minor updates to our implementation of the HTTP-01 validation method based
on in-depth review of BRs Section 3.2.2.4.19 and RFC 8555 Section 8.3.
- Move the HTTP response code check above parsing the body.
- Explicitly check for 301, 302, 307, and 308 redirect codes, so that if the go
stdlib updates to allow additional redirects we don't follow suit.
- Trim additional forms of white-space from the key authorization.
Add a new gRPC method `UpdateRevokedCertificate` to the SA. This
method takes the same argument as the existing `RevokeCertificate` RPC,
but only operates on certificates that have already been revoked with a
reason other than keyCompromise (c.f. `RevokeCertificate`, which only
operates on certificates that have not been revoked).
One thing to be careful of here is that storing an updated revocation reason
should not also change the revocation date. To support this, add a new field
to the existing `RevokeCertificateRequest` that allows us to differentiate the
time at which the new OCSP response was created, and the time at which
the revocation went into effect.
Part of #5936
Adapted from https://github.com/golangci/golangci-lint-action#how-to-use.
Uses the same version we've been using in boulder-tools.
Part of #5946
Note: we will eventually want to go back to doing this in boulder-tools,
so it's easy to run the lints locally. But this is useful so we can
unblock testing on go 1.18beta2.
Light cleanup of akamai-purger and the akamai cache-client. This does not make
any material changes to logic.
- Use `errors.New` and `errors.Is` instead of a custom `ErrFatal` type and
`errors.As`
- Add whitespace to separate chunks of execution and error checking from one
another
- Use `logger.Infof` and `logger.Errorf` instead of wrapped calls to
`fmt.Sprintf`
- Remove capital letters from the beginning of error messages
- Additional comments and removal of some that are no longer accurate
Samantha