Add a `MaxExpectedReplicationLag` parameter to `bad-key-revoker`. Wait
that interval before searching for certificates to revoke.
The interval is set to only 100ms in both `test/config` and
`test/config-next` so that integration tests don't require long sleeps.
The default value within BKR is, and the production value should be,
higher.
Part of #5686
Prevent duplicate contacts in Pardot by using the upsert-by-email
endpoint.
Some background: In
https://github.com/letsencrypt/boulder/pull/7998#discussion_r1949702280
(later superseded by https://github.com/letsencrypt/boulder/pull/8016),
we discussed using this endpoint. It turns out I was wrong about Pardot
storing Prospects by email address; you can have multiple Prospects with
the same email. While subscribed newsletters won’t be sent to both
contacts, the onboarding process doesn’t deduplicate in this way and
will send to an email address each time it’s added, resulting in
duplicate onboarding messages.
This integration test was removed in the early versions of
https://github.com/letsencrypt/boulder/pull/8245, because that PR had
removed all validation of contact addresses. However, later iterations
of that PR restored (most) contact validation, so this PR restores (most
of) the TestAccountEmailError integration test.
Change the WFE to stop populating the Contact field of the
NewRegistration requests it sends to the RA. Similarly change the WFE to
ignore the Contact field of any update-account requests it receives,
thereby removing all calls to the RA's UpdateRegistrationContact method.
Hoist the RA's contact validation logic into the WFE, so that we can
still return errors to clients which are presenting grossly malformed
contact fields, and have a first layer of protection against trying to
send malformed addresses to email-exporter.
A follow-up change (after a deploy cycle) will remove the deprecated RA
and SA methods.
Part of https://github.com/letsencrypt/boulder/issues/8199
Hyrum’s Law strikes again: our Python integration tests were implicitly
relying on behavior that was changed upstream in Certbot’s ACME client
(see https://github.com/certbot/certbot/pull/10239). To ensure continued
coverage, replicate this test in our Go integration test suite.
Deprecate the IgnoreAccountContacts feature flag. This causes the SA to
never query the contact column when reading registrations from the
database, and to never write a value for the contact column when
creating a new registration.
This requires updating or disabling several tests. These tests could be
deleted now, but I felt it was more appropriate for them to be fully
deleted when their corresponding services (e.g. expiration-mailer) are
also deleted.
Fixes https://github.com/letsencrypt/boulder/issues/8176
Change most functions in `ratelimits` to use full ACMEIdentifier(s) as
arguments, instead of using their values as strings. This makes the
plumbing from other packages more consistent, and allows us to:
Rename `FQDNsToETLDsPlusOne` to `coveringIdentifiers` and handle IP
identifiers, parsing IPv6 addresses into their covering /64 prefixes for
CertificatesPerDomain[PerAccount] bucket keys.
Port improved IP/CIDR validation logic to NewRegistrationsPerIPAddress &
PerIPv6Range.
Rename `domain` parts of bucket keys to either `identValue` or
`domainOrCIDR`.
Rename other internal functions to clarify that they now handle
identifier values, not just domains.
Add the new reserved IPv6 address range from RFC 9780.
For deployability, don't (yet) rename rate limits themselves; and
because it remains the name of the database table, preserve the term
`fqdnSets`.
Fixes#8223
Part of #7311
Simplify the main logic loop to simply revoke certs as soon as they're
identified, rather than jumping through hoops to identify and
deduplicate the associated accounts and emails. Make the Mailer portion
of the config optional for deployability.
Part of https://github.com/letsencrypt/boulder/issues/8199
Go 1.24.4 is a security release containing fixes to net/http,
os.OpenFile, and x509.Certificate.Verify, all of which we use. We appear
to be unaffected by the specific vulnerabilities described, however. See
the announcement here:
https://groups.google.com/g/golang-announce/c/ufZ8WpEsA3A
The golangci-lint project has released a v2, which is noticeably faster,
splits linters and formatters into separate categories, has greatly
improved support for staticcheck, and has an incompatible config file
format. Update our boulder-tools version of golangci-lint to v2, remove
our standalone staticcheck, and update our config file to match.
Remove static IPs from services that can be reached by their service
name. Remove consulnet and redisnet, and have the services which
connected to those network connect directly to bouldernet instead.
Instruct docker-compose to only dynamically allocate IPs from the upper
half of the bouldernet subset, to avoid clashing with the few static IPs
we still specify.
We no longer need a code path to resolve reserved IP addresses during
integration tests.
Move to a public IP for the remaining tests, after #8187 did so for many
of them.
Depends on #8187
Move usage of `sa.ReverseName` to a new `sa.EncodeIssuedName`, which
detects IP addresses and exempts them from being reversed. Retain
`reverseName` as an internal helper function.
Update `id-exporter`, `reversed-hostname-checker`, and tests to use the
new function and handle IP addresses.
Part of #7311
Permit all valid identifier types in `wfe.NewOrder` and `csr.VerifyCSR`.
Permit certs with just IP address identifiers to skip
`sa.addIssuedNames`.
Check that URI SANs are empty in `csr.VerifyCSR`, which was previously
missed.
Use a real (Let's Encrypt) IP address range in integration testing, to
let challtestsrv satisfy IP address challenges.
Fixes#8192
Depends on #8154
Add a feature flag "IgnoreAccountContacts" which has two effects in the
SA:
- When a new account is created, don't insert any contacts provided; and
- When an account is retrieved, ignore any contacts already present.
This causes boulder to act as though all accounts have no associated
contacts, and is the first step towards being able to drop the contacts
from the database entirely.
Part of https://github.com/letsencrypt/boulder/issues/8176
Add `IdentifierTypes` to validation profiles' config, defaulting to DNS
if not set.
In `NewOrder`, check that the order's profile permits each identifier's
type.
Fixes#8137
Depends on #8173
Remove the ability for the issuance package to include the AIA OCSP URI
and the Must Staple (more properly known as the tlsRequest) extension in
certificates. Deprecate the "OmitOCSP" and "AllowMustStaple" profile
config keys, as they no longer have any effect. Similarly deprecate the
"OCSPURL" issuer config key, as it is no longer included in
certificates.
Update the tests to always include to CRLDP extension instead, and
remove some OCSP- or Stapling-specific test cases.
Fixes https://github.com/letsencrypt/boulder/issues/8179
Add `pa.validIP` to test IP address validity & absence from IANA
reservations.
Modify `pa.WillingToIssue` and `pa.WellFormedIdentifiers` to support IP
address identifiers.
Add a map of allowed identifier types to the `pa` config.
Part of #8137
Add an "algorithms" field to all problem documents, but tag it so it
won't be included in the serialized json unless populated. Populate it
only when the problem type is "badSignatureAlgorithm", as specified in
RFC 8555 Section 6.2.
The resulting problem document looks like this:
```json
{
"type": "urn:ietf:params:acme:error:badSignatureAlgorithm",
"detail": "Unable to validate JWS :: JWS signature header contains unsupported algorithm
\"RS512\", expected one of [RS256 ES256 ES384 ES512]",
"status": 400,
"algorithms": [
"RS256",
"ES256",
"ES384",
"ES512"
]
}
```
Fixes https://github.com/letsencrypt/boulder/issues/8155
In `ra.NewOrder`, improve safety of authz reuse logic by making it
explicit that only DNS identifiers might be wildcards. Also, now that
the conditional statements need to be more complicated, collapse them
for brevity.
In `vapb.PerformValidationRequest`, remove `DnsName`.
In `ra.PerformValidation`, pass an `Identifier` instead of a `DnsName`.
In `ra.RevokeCertByApplicant`, check that the requester controls
identifiers of all types (not just DNS).
Fixes#7995 (the RA now fully supports IP address identifiers, except
for rate limits)
Fixes#7647
Part of #8023
This updates to current `master`,
bc7acd89f703743d050f5cd4a3b9746808e0fdae
Notably, it includes a bug-fix to error handling in the HTTP client,
which we found was hiding errors from CT logs, hindering our debugging.
That fix is
https://github.com/google/certificate-transparency-go/pull/1695
No release has been tagged since this PR merged, so using the `master`
commit.
A few mutual dependencies used by both Boulder and ct-go are updated,
including mysql, otel, and grpc.
https://github.com/letsencrypt/boulder/pull/8150 updated our runtime
protobuf dependency from v1.34.1 to v1.36.5. This change does the same
for our build-time dependency, to keep them in sync.
Break validation of length and content of expected User-Agents out into
two assertion functions. Make it so that DOH and MPICFullResults can be
deprecated in either order.
Fixes#8145
Methodology:
- Copy test/config-next/* to test/config/.
- Review the diff, reverting things that should stay `next`-only.
- When in doubt, check against prod configs (e.g. for feature flags).
In the process I noticed that config for the TCP prober in `observer`
had been added to test/config but not test/config-next, so I ported it
forward (and my IDE stripped some trailing spaces in both versions).
#8109 updated CI to use 24.04 runners, now update the Docker image to
build 24.04 and CI to use it.
Build fixes:
- Unpin mariadb-client-core, 10.3 is no longer provided in 24.04 apt
repositories
- Use new pip flag --break-system-packages to comply with PEP 668, which
is now enforced in Python 3.12+
Runtime fixes:
- Start rsyslogd directly due to missing symlink (see:
https://github.com/rsyslog/rsyslog/issues/5611)
- Fix SyntaxWarning: invalid escape sequence '\w' error.
- Replace OpenSSL.crypto.load_certificate with
x509.load_pem_x509_certificate due to
d73d0ed417
Plumb the userAgent field, used to set http-01 User-Agent headers, from
va/rva configuration through to where User-Agent headers can be set for
DoH queries. Use integration tests to validate that the User-Agent is
set for http-01 challenges, dns-01 challenges over DoH, and CAA checks
over DoH.
Fixes#7963.
Add a new config field for profiles which causes the profile to omit the
AIA OCSP URI. It can only be omitted if the CRLDP extension is
configured to be included instead. Enable this flag in config-next.
When a certificate is revoked, if it does not have an AIA OCSP URI,
don't bother with an Akamai OCSP purge.
Builds on #8089
Most of the changes in this PR relate to tests. Different from #8089, I
chose to keep testing of OCSP in the config-next world. This is because
we intend to keep operating OCSP even after we have stopped including it
in new certificates. So we should test it in as many environments as
possible.
Adds a WithURLFallback option to ocsp_helper. When
`ocsp_helper.ReqDer()` is called for a certificate with no OCSP URI, it
will query the fallback URL instead. As before, if the certificate has
an OCSP URI ocsp_helper will use that. Use that for all places in the
integration tests that call ocsp_helper.
- Update the chall-test-srv-client to make DNS events and DNS01 methods
more convenient
- Add an integration test that counts DCV and CAA checks for each
validation method
Part of #7963
For explicitly sharded certificates, CRL status is read from the
`revokedCertificates` table. This table gets written at revocation time.
At re-revocation time (for key compromise), it only gets written by the
SA if the caller passes a nonzero ShardIdx to UpdateRevokedCertificate.
The RA was never passing a nonzero ShardIdx to UpdateRevokedCertificate.
- Copy
https://pkg.go.dev/github.com/letsencrypt/pebble/v2/cmd/pebble-challtestsrv
to `test/chall-test-srv`
- Rename pebble-challtestsrv to chall-test-srv, consistent with other
test server naming in Boulder
- Replace Dockerfile go install with Makefile compilation of
`chall-test-srv`
- Run chall-test-srv from `./bin/chall-test-srv`
- Bump `github.com/letsencrypt/challtestsrv` from `v1.2.1` to `v1.3.2`
in go.mod
- Update boulder-ci GitHub workflow to use `go1.24.1_2025-04-02`
Part of #7963
Replace a python integration test which relies on our
"setup_twenty_days_ago" scaffolding with a Go test that uses direct
database statements to avoid any need to do clock manipulation. The
resulting test is much more verbose, but also (in my opinion) much
clearer and significantly faster.
We already have an integration test showing that a serial does not show
up on any CRL before its certificate has been revoked, and does show up
afterwards. Extend that test to cover three new times:
- shortly before the certificate expires, when the entry must still
appear;
- shortly after the certificate expires, when the entry must still
appear; and
- significantly after the certificate expires, when the entry may be
removed.
To facilitate this, augment the s3-test-srv with a new reset endpoint,
so that the integration test can query the contents of only the
most-recently-generated set of CRLs.
I have confirmed that the new integration test fails with
https://github.com/letsencrypt/boulder/pull/8072 reverted.
Fixes https://github.com/letsencrypt/boulder/issues/8083
James Renken
Jacob Hoffman-Andrews
Samantha Frank
Aaron Gable
Phil Porada
Matthew McPherrin