Hyrum’s Law strikes again: our Python integration tests were implicitly
relying on behavior that was changed upstream in Certbot’s ACME client
(see https://github.com/certbot/certbot/pull/10239). To ensure continued
coverage, replicate this test in our Go integration test suite.
Deprecate the IgnoreAccountContacts feature flag. This causes the SA to
never query the contact column when reading registrations from the
database, and to never write a value for the contact column when
creating a new registration.
This requires updating or disabling several tests. These tests could be
deleted now, but I felt it was more appropriate for them to be fully
deleted when their corresponding services (e.g. expiration-mailer) are
also deleted.
Fixes https://github.com/letsencrypt/boulder/issues/8176
Change most functions in `ratelimits` to use full ACMEIdentifier(s) as
arguments, instead of using their values as strings. This makes the
plumbing from other packages more consistent, and allows us to:
Rename `FQDNsToETLDsPlusOne` to `coveringIdentifiers` and handle IP
identifiers, parsing IPv6 addresses into their covering /64 prefixes for
CertificatesPerDomain[PerAccount] bucket keys.
Port improved IP/CIDR validation logic to NewRegistrationsPerIPAddress &
PerIPv6Range.
Rename `domain` parts of bucket keys to either `identValue` or
`domainOrCIDR`.
Rename other internal functions to clarify that they now handle
identifier values, not just domains.
Add the new reserved IPv6 address range from RFC 9780.
For deployability, don't (yet) rename rate limits themselves; and
because it remains the name of the database table, preserve the term
`fqdnSets`.
Fixes#8223
Part of #7311
Simplify the main logic loop to simply revoke certs as soon as they're
identified, rather than jumping through hoops to identify and
deduplicate the associated accounts and emails. Make the Mailer portion
of the config optional for deployability.
Part of https://github.com/letsencrypt/boulder/issues/8199
Go 1.24.4 is a security release containing fixes to net/http,
os.OpenFile, and x509.Certificate.Verify, all of which we use. We appear
to be unaffected by the specific vulnerabilities described, however. See
the announcement here:
https://groups.google.com/g/golang-announce/c/ufZ8WpEsA3A
The golangci-lint project has released a v2, which is noticeably faster,
splits linters and formatters into separate categories, has greatly
improved support for staticcheck, and has an incompatible config file
format. Update our boulder-tools version of golangci-lint to v2, remove
our standalone staticcheck, and update our config file to match.
Remove static IPs from services that can be reached by their service
name. Remove consulnet and redisnet, and have the services which
connected to those network connect directly to bouldernet instead.
Instruct docker-compose to only dynamically allocate IPs from the upper
half of the bouldernet subset, to avoid clashing with the few static IPs
we still specify.
We no longer need a code path to resolve reserved IP addresses during
integration tests.
Move to a public IP for the remaining tests, after #8187 did so for many
of them.
Depends on #8187
Move usage of `sa.ReverseName` to a new `sa.EncodeIssuedName`, which
detects IP addresses and exempts them from being reversed. Retain
`reverseName` as an internal helper function.
Update `id-exporter`, `reversed-hostname-checker`, and tests to use the
new function and handle IP addresses.
Part of #7311
Permit all valid identifier types in `wfe.NewOrder` and `csr.VerifyCSR`.
Permit certs with just IP address identifiers to skip
`sa.addIssuedNames`.
Check that URI SANs are empty in `csr.VerifyCSR`, which was previously
missed.
Use a real (Let's Encrypt) IP address range in integration testing, to
let challtestsrv satisfy IP address challenges.
Fixes#8192
Depends on #8154
Add a feature flag "IgnoreAccountContacts" which has two effects in the
SA:
- When a new account is created, don't insert any contacts provided; and
- When an account is retrieved, ignore any contacts already present.
This causes boulder to act as though all accounts have no associated
contacts, and is the first step towards being able to drop the contacts
from the database entirely.
Part of https://github.com/letsencrypt/boulder/issues/8176
Add `IdentifierTypes` to validation profiles' config, defaulting to DNS
if not set.
In `NewOrder`, check that the order's profile permits each identifier's
type.
Fixes#8137
Depends on #8173
Remove the ability for the issuance package to include the AIA OCSP URI
and the Must Staple (more properly known as the tlsRequest) extension in
certificates. Deprecate the "OmitOCSP" and "AllowMustStaple" profile
config keys, as they no longer have any effect. Similarly deprecate the
"OCSPURL" issuer config key, as it is no longer included in
certificates.
Update the tests to always include to CRLDP extension instead, and
remove some OCSP- or Stapling-specific test cases.
Fixes https://github.com/letsencrypt/boulder/issues/8179
Add `pa.validIP` to test IP address validity & absence from IANA
reservations.
Modify `pa.WillingToIssue` and `pa.WellFormedIdentifiers` to support IP
address identifiers.
Add a map of allowed identifier types to the `pa` config.
Part of #8137
Add an "algorithms" field to all problem documents, but tag it so it
won't be included in the serialized json unless populated. Populate it
only when the problem type is "badSignatureAlgorithm", as specified in
RFC 8555 Section 6.2.
The resulting problem document looks like this:
```json
{
"type": "urn:ietf:params:acme:error:badSignatureAlgorithm",
"detail": "Unable to validate JWS :: JWS signature header contains unsupported algorithm
\"RS512\", expected one of [RS256 ES256 ES384 ES512]",
"status": 400,
"algorithms": [
"RS256",
"ES256",
"ES384",
"ES512"
]
}
```
Fixes https://github.com/letsencrypt/boulder/issues/8155
In `ra.NewOrder`, improve safety of authz reuse logic by making it
explicit that only DNS identifiers might be wildcards. Also, now that
the conditional statements need to be more complicated, collapse them
for brevity.
In `vapb.PerformValidationRequest`, remove `DnsName`.
In `ra.PerformValidation`, pass an `Identifier` instead of a `DnsName`.
In `ra.RevokeCertByApplicant`, check that the requester controls
identifiers of all types (not just DNS).
Fixes#7995 (the RA now fully supports IP address identifiers, except
for rate limits)
Fixes#7647
Part of #8023
This updates to current `master`,
bc7acd89f703743d050f5cd4a3b9746808e0fdae
Notably, it includes a bug-fix to error handling in the HTTP client,
which we found was hiding errors from CT logs, hindering our debugging.
That fix is
https://github.com/google/certificate-transparency-go/pull/1695
No release has been tagged since this PR merged, so using the `master`
commit.
A few mutual dependencies used by both Boulder and ct-go are updated,
including mysql, otel, and grpc.
https://github.com/letsencrypt/boulder/pull/8150 updated our runtime
protobuf dependency from v1.34.1 to v1.36.5. This change does the same
for our build-time dependency, to keep them in sync.
Break validation of length and content of expected User-Agents out into
two assertion functions. Make it so that DOH and MPICFullResults can be
deprecated in either order.
Fixes#8145
Methodology:
- Copy test/config-next/* to test/config/.
- Review the diff, reverting things that should stay `next`-only.
- When in doubt, check against prod configs (e.g. for feature flags).
In the process I noticed that config for the TCP prober in `observer`
had been added to test/config but not test/config-next, so I ported it
forward (and my IDE stripped some trailing spaces in both versions).
#8109 updated CI to use 24.04 runners, now update the Docker image to
build 24.04 and CI to use it.
Build fixes:
- Unpin mariadb-client-core, 10.3 is no longer provided in 24.04 apt
repositories
- Use new pip flag --break-system-packages to comply with PEP 668, which
is now enforced in Python 3.12+
Runtime fixes:
- Start rsyslogd directly due to missing symlink (see:
https://github.com/rsyslog/rsyslog/issues/5611)
- Fix SyntaxWarning: invalid escape sequence '\w' error.
- Replace OpenSSL.crypto.load_certificate with
x509.load_pem_x509_certificate due to
d73d0ed417
Plumb the userAgent field, used to set http-01 User-Agent headers, from
va/rva configuration through to where User-Agent headers can be set for
DoH queries. Use integration tests to validate that the User-Agent is
set for http-01 challenges, dns-01 challenges over DoH, and CAA checks
over DoH.
Fixes#7963.
Add a new config field for profiles which causes the profile to omit the
AIA OCSP URI. It can only be omitted if the CRLDP extension is
configured to be included instead. Enable this flag in config-next.
When a certificate is revoked, if it does not have an AIA OCSP URI,
don't bother with an Akamai OCSP purge.
Builds on #8089
Most of the changes in this PR relate to tests. Different from #8089, I
chose to keep testing of OCSP in the config-next world. This is because
we intend to keep operating OCSP even after we have stopped including it
in new certificates. So we should test it in as many environments as
possible.
Adds a WithURLFallback option to ocsp_helper. When
`ocsp_helper.ReqDer()` is called for a certificate with no OCSP URI, it
will query the fallback URL instead. As before, if the certificate has
an OCSP URI ocsp_helper will use that. Use that for all places in the
integration tests that call ocsp_helper.
- Update the chall-test-srv-client to make DNS events and DNS01 methods
more convenient
- Add an integration test that counts DCV and CAA checks for each
validation method
Part of #7963
For explicitly sharded certificates, CRL status is read from the
`revokedCertificates` table. This table gets written at revocation time.
At re-revocation time (for key compromise), it only gets written by the
SA if the caller passes a nonzero ShardIdx to UpdateRevokedCertificate.
The RA was never passing a nonzero ShardIdx to UpdateRevokedCertificate.
- Copy
https://pkg.go.dev/github.com/letsencrypt/pebble/v2/cmd/pebble-challtestsrv
to `test/chall-test-srv`
- Rename pebble-challtestsrv to chall-test-srv, consistent with other
test server naming in Boulder
- Replace Dockerfile go install with Makefile compilation of
`chall-test-srv`
- Run chall-test-srv from `./bin/chall-test-srv`
- Bump `github.com/letsencrypt/challtestsrv` from `v1.2.1` to `v1.3.2`
in go.mod
- Update boulder-ci GitHub workflow to use `go1.24.1_2025-04-02`
Part of #7963
Replace a python integration test which relies on our
"setup_twenty_days_ago" scaffolding with a Go test that uses direct
database statements to avoid any need to do clock manipulation. The
resulting test is much more verbose, but also (in my opinion) much
clearer and significantly faster.
We already have an integration test showing that a serial does not show
up on any CRL before its certificate has been revoked, and does show up
afterwards. Extend that test to cover three new times:
- shortly before the certificate expires, when the entry must still
appear;
- shortly after the certificate expires, when the entry must still
appear; and
- significantly after the certificate expires, when the entry may be
removed.
To facilitate this, augment the s3-test-srv with a new reset endpoint,
so that the integration test can query the contents of only the
most-recently-generated set of CRLs.
I have confirmed that the new integration test fails with
https://github.com/letsencrypt/boulder/pull/8072 reverted.
Fixes https://github.com/letsencrypt/boulder/issues/8083
Delete several python revocation integration tests whose functionality
is already replicated by the go revocation integration tests. Add
support for revoking via admin-revoker to TestRevocation, and use that
to replace several more python tests.
The go versions of these tests use CRLs, rather than OCSP, to confirm
the revocation status of the certs in question. This is fine because the
purpose of these tests is to ensure that we handle revocation requests
correctly in general, not specifically via OCSP.
Part of https://github.com/letsencrypt/boulder/issues/8059
Change all of the helper methods and functions in verify.go to return an
`error` instead of a `probs.ProblemDetails`. Add a few new types to our
errors package, and support for those types in ProblemDetailsForError,
to maintain the same public-facing error types. Update the tests to
check for specific errors instead of specific problems.
This is a building block towards making the probs.ProblemDetails type
not implement the Error interface, and only be used when rendering
errors to the user (i.e. not within Boulder logic itself).
Part of https://github.com/letsencrypt/boulder/issues/4980
Remove the backwards-compatible profile hashing code. It is no longer
necessary, since all deployed profile configs now set
IncludeCRLDistributionPoints to true and set the UnsplitIssuance flag to
true. Catch up the CA and crl-updater configs to match config-next and
what is actively deployed in prod.
Part of https://github.com/letsencrypt/boulder/issues/8039
Part of https://github.com/letsencrypt/boulder/issues/8059
Add `identifier` fields, which will soon replace the `dnsName` fields,
to:
- `corepb.Authorization`
- `corepb.Order`
- `rapb.NewOrderRequest`
- `sapb.CountFQDNSetsRequest`
- `sapb.CountInvalidAuthorizationsRequest`
- `sapb.FQDNSetExistsRequest`
- `sapb.GetAuthorizationsRequest`
- `sapb.GetOrderForNamesRequest`
- `sapb.GetValidAuthorizationsRequest`
- `sapb.NewOrderRequest`
Populate these `identifier` fields in every function that creates
instances of these structs.
Use these `identifier` fields instead of `dnsName` fields (at least
preferentially) in every function that uses these structs. When crossing
component boundaries, don't assume they'll be present, for
deployability's sake.
Deployability note: Mismatched `cert-checker` and `sa` versions will be
incompatible because of a type change in the arguments to
`sa.SelectAuthzsMatchingIssuance`.
Part of #7311
Add a new custom lint which sends CRLs to PKIMetal, and configure it to
run in our integration test environment. Factor out most of the code
used to talk to the PKIMetal API so that it can be shared by the two
custom lints which do so. Add the ability to configure lints to the
CRLProfileConfig, so that zlint knows where to load the necessary custom
config from.
Update the SA to re-query the database for the updated account after
deactivating it, and return this to the RA. Update the RA to pass this
value through to the WFE. Update the WFE to return this value, rather
than locally modifying the pre-deactivation account object, if it gets
one (for deployability).
Also remove the RA's requirement that the request object specify its
current status so that the request can be trimmed down to just an ID.
This proto change is backwards-compatible because the new
DeactivateRegistrationRequest's registrationID field has the same type
(int64) and field number (1) as corepb.Registration's id field.
Part of https://github.com/letsencrypt/boulder/issues/5554
Give cert-checker the ability to load zlint configs, so that it can be
configured to talk to PKIMetal in CI and hopefully in staging/production
in the future.
Also update how cert-checker executes lints, so that it uses a real lint
registry instead of using the global registry and passing around a
dictionary of lints to filter out of the results.
Fixes https://github.com/letsencrypt/boulder/issues/7786
Replace the bpkilint container with a new bpkimetal container. Update
our custom lint which calls out to that API to speak PKIMetal's (very
similar) protocol instead. Update our zlint custom configuration to
configure this updated lint.
Fixes https://github.com/letsencrypt/boulder/issues/8009
- Plumb the "replaces" value from the WFE through to the SA via the RA
- Store validated "replaces" value for new orders in the orders table
- Reflect the stored "replaces" value to subscribers in the order object
- Reorder CertificateProfileName before Replaces/ReplacesSerial in RA
and SA protos for consistency
Fixes#8034
Return "alreadyReplaced" in addition to HTTP 409 Conflict to signal that
an order indicates that it replaces a certificate which already has a
replacement order.
Jacob Hoffman-Andrews
Samantha Frank
Aaron Gable
James Renken
Phil Porada
Matthew McPherrin