Change the WFE to stop populating the Contact field of the
NewRegistration requests it sends to the RA. Similarly change the WFE to
ignore the Contact field of any update-account requests it receives,
thereby removing all calls to the RA's UpdateRegistrationContact method.
Hoist the RA's contact validation logic into the WFE, so that we can
still return errors to clients which are presenting grossly malformed
contact fields, and have a first layer of protection against trying to
send malformed addresses to email-exporter.
A follow-up change (after a deploy cycle) will remove the deprecated RA
and SA methods.
Part of https://github.com/letsencrypt/boulder/issues/8199
Move `policy.IsReservedIP` to `iana.IsReservedAddr`.
Move `policy.IsReservedPrefix` to `iana.IsReservedPrefix`.
Embed & parse IANA's special-purpose address registries for IPv4 and
IPv6 in their original CSV format.
Fixes#8080
The ProtoText printing of this structure prints the binary string as
escaped
utf8 text, which is essentially gibberish for my processes.
---------
Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
Currently, we check the cache only immediately before attempting to send
an email address. However, we only reach that point if the rate limiter
(used to respect the daily API quota) permits it. As a result, around
40% of sends are wasted on email addresses that are ultimately skipped
due to cache hits.
Replace the pre-send cache `Seen` check with an atomic `StoreIfAbsent`
executed before the `limiter.Wait()` so that limiter tokens are consumed
only for email addresses that actually need sending. Skip the
`limiter.Wait()` on cache hits, remove cache entries only when a send
fails, and increment metrics only on successful sends.
- Configure all gRPC clients to check the overall serving status of each
endpoint via the `grpc_health_v1` service.
- Configure all gRPC servers to expose the `grpc_health_v1` service to
any client permitted to access one of the server’s services.
- Modify long-running, deep health checks to set and transition the
overall (empty string) health status of the gRPC server in addition to
the specific service they were configured for.
Fixes#8227
Deprecate the IgnoreAccountContacts feature flag. This causes the SA to
never query the contact column when reading registrations from the
database, and to never write a value for the contact column when
creating a new registration.
This requires updating or disabling several tests. These tests could be
deleted now, but I felt it was more appropriate for them to be fully
deleted when their corresponding services (e.g. expiration-mailer) are
also deleted.
Fixes https://github.com/letsencrypt/boulder/issues/8176
Remove `ToDNSSlice`, `FromProtoWithDefault`, and
`FromProtoSliceWithDefault` now that all their callers are gone. All
protobufs but one have migrated from DnsNames to Identifiers.
Remove TODOs for the exception, `ValidationRecord`, where an identifier
type isn't appropriate and it really only needs a string.
Rename `corepb.ValidationRecord.DnsName` to `Hostname` for clarity, to
match the corresponding PB's field name.
Improve various comments and docs re: IP address identifiers.
Depends on #8221 (which removes the last callers)
Fixes#8023
Simplify the main logic loop to simply revoke certs as soon as they're
identified, rather than jumping through hoops to identify and
deduplicate the associated accounts and emails. Make the Mailer portion
of the config optional for deployability.
Part of https://github.com/letsencrypt/boulder/issues/8199
The golangci-lint project has released a v2, which is noticeably faster,
splits linters and formatters into separate categories, has greatly
improved support for staticcheck, and has an incompatible config file
format. Update our boulder-tools version of golangci-lint to v2, remove
our standalone staticcheck, and update our config file to match.
This partially reverts https://github.com/letsencrypt/boulder/pull/8203,
which was landed as commit dea81c7381.
It leaves all of the boulder integration test environment changes in
place, while restoring the DNSAllowLoopbackAddresses config key and its
ability to influence the VA's behavior.
Remove static IPs from services that can be reached by their service
name. Remove consulnet and redisnet, and have the services which
connected to those network connect directly to bouldernet instead.
Instruct docker-compose to only dynamically allocate IPs from the upper
half of the bouldernet subset, to avoid clashing with the few static IPs
we still specify.
We no longer need a code path to resolve reserved IP addresses during
integration tests.
Move to a public IP for the remaining tests, after #8187 did so for many
of them.
Depends on #8187
Move usage of `sa.ReverseName` to a new `sa.EncodeIssuedName`, which
detects IP addresses and exempts them from being reversed. Retain
`reverseName` as an internal helper function.
Update `id-exporter`, `reversed-hostname-checker`, and tests to use the
new function and handle IP addresses.
Part of #7311
Move `IsReservedIP` and its supporting vars from `bdns` to `policy`.
Rewrite `IsReservedIP` to:
* Use `netip` because `netip.Prefix` can be used as a map key, allowing
us to define prefix lists more elegantly. This will enable future work
to import prefix lists from IANA's primary source data.
* Return an error including the reserved network's name.
Refactor `IsReservedIP` tests to be table-based.
Fixes#8040
It appears that, in the past, we wanted id-exporter's "tell me all the
accounts with unexpired certificates" functionality to limit itself to
account that have contact info. The reasons for this limitation are
unclear, and are quickly becoming obsolete as we remove contact info
from the registrations table.
Remove this layer of filtering, so that id-exporter will retrieve all
accounts with active certificates, and not care whether the contact
column exists or not.
Part of https://github.com/letsencrypt/boulder/issues/8199
In cert-checker, inspect both the DNS Names and the IP Addresses
contained within the certificate being examined. Also add a check that
no other kinds of SANs exist in the certificate.
Fixes https://github.com/letsencrypt/boulder/issues/8183
Add `pa.validIP` to test IP address validity & absence from IANA
reservations.
Modify `pa.WillingToIssue` and `pa.WellFormedIdentifiers` to support IP
address identifiers.
Add a map of allowed identifier types to the `pa` config.
Part of #8137
This field is unused. This has been configured in the CTLogs field for
years.
The field has been a no-op since #6485 and was removed from Let's
Encrypt prod configuration in 2022.
This follows the system we've used for other types, where the SA has a
model type that is converted to a proto message for use outside the SA.
Part of #8112.
Plumb the userAgent field, used to set http-01 User-Agent headers, from
va/rva configuration through to where User-Agent headers can be set for
DoH queries. Use integration tests to validate that the User-Agent is
set for http-01 challenges, dns-01 challenges over DoH, and CAA checks
over DoH.
Fixes#7963.
Remove the backwards-compatible profile hashing code. It is no longer
necessary, since all deployed profile configs now set
IncludeCRLDistributionPoints to true and set the UnsplitIssuance flag to
true. Catch up the CA and crl-updater configs to match config-next and
what is actively deployed in prod.
Part of https://github.com/letsencrypt/boulder/issues/8039
Part of https://github.com/letsencrypt/boulder/issues/8059
Add `identifier` fields, which will soon replace the `dnsName` fields,
to:
- `corepb.Authorization`
- `corepb.Order`
- `rapb.NewOrderRequest`
- `sapb.CountFQDNSetsRequest`
- `sapb.CountInvalidAuthorizationsRequest`
- `sapb.FQDNSetExistsRequest`
- `sapb.GetAuthorizationsRequest`
- `sapb.GetOrderForNamesRequest`
- `sapb.GetValidAuthorizationsRequest`
- `sapb.NewOrderRequest`
Populate these `identifier` fields in every function that creates
instances of these structs.
Use these `identifier` fields instead of `dnsName` fields (at least
preferentially) in every function that uses these structs. When crossing
component boundaries, don't assume they'll be present, for
deployability's sake.
Deployability note: Mismatched `cert-checker` and `sa` versions will be
incompatible because of a type change in the arguments to
`sa.SelectAuthzsMatchingIssuance`.
Part of #7311
Give cert-checker the ability to load zlint configs, so that it can be
configured to talk to PKIMetal in CI and hopefully in staging/production
in the future.
Also update how cert-checker executes lints, so that it uses a real lint
registry instead of using the global registry and passing around a
dictionary of lints to filter out of the results.
Fixes https://github.com/letsencrypt/boulder/issues/7786
Populate the new x509.Certificate.Policies field everywhere we currently populate the x509.Certificate.PolicyIdentifiers field. This allows Go to use whichever field it prefers (go1.23 prefers PolicyIdentifiers, go1.24 prefers Policies) as the source of truth when serializing a certificate.
Part of https://github.com/letsencrypt/boulder/issues/7148
Add a new boulder service, email-exporter, which uses the Pardot API
client added in #8016 and the email.Exporter gRPC service added in
#8017.
Add pardot-test-srv, a test-only service for mocking communication with
Salesforce OAuth and Pardot APIs in non-production environments. Since
Salesforce does not provide Pardot functionality in developer sandboxes,
pardot-test-srv must run in all non-production environments (e.g.,
sre-development and staging).
Integrate the email-exporter service with the WFE and modify
WFE.NewAccount and WFE.UpdateAccount to submit valid email contacts.
Ensure integration tests verify that contacts eventually reach
pardot-test-srv.
Update configuration where necessary to:
- Build pardot-test-srv as a standalone binary.
- Bring up pardot-test-srv and cmd/email-exporter for integration
testing.
- Integrate WFE with cmd/email-exporter when running test/config-next.
Closes#7966
Add an `identifier` field to the `va.PerformValidationRequest` proto, which will soon replace its `dnsName` field.
Accept and prefer the `identifier` field in every VA function that uses this struct. Don't (yet) assume it will be present.
Throughout the VA, accept and handle the IP address identifier type. Handling is similar to DNS names, except that `getAddrs` is not called, and consider that:
- IPs are represented in a different field in the `x509.Certificate` struct.
- IPs must be presented as reverse DNS (`.arpa`) names in SNI for [TLS-ALPN-01 challenge requests](https://datatracker.ietf.org/doc/html/rfc8738#name-tls-with-application-layer-).
- IPv6 addresses are enclosed in square brackets when composing or parsing URLs.
For HTTP-01 challenges, accept redirects to bare IP addresses, which were previously rejected.
Fixes#2706
Part of #7311
Replace DCV and CAA checks (PerformValidation and IsCAAValid) in
va/va.go and va/caa.go with their MPIC compliant counterparts (DoDCV and
DoCAA) in va/vampic.go. Deprecate EnforceMultiCAA and EnforceMPIC and
default code paths as though they are both true. Require that RIR and
Perspective be set for primary and remote VAs.
Fixes#7965Fixes#7819
Add MaxNames to the set of things that can be configured on a
per-profile basis. Remove all references to the RA's global maxNames,
replacing them with reference's to the current profile's maxNames. Add
code to the RA's main() to copy a globally-configured MaxNames into each
profile, for deployability.
Also remove any understanding of MaxNames from the WFE, as it is
redundant with the RA and is not configured in staging or prod. Instead,
hardcode the upper limit of 100 into the ratelimit package itself.
Fixes https://github.com/letsencrypt/boulder/issues/7993
Add a new RPC to the CA: `IssueCertificate` covers issuance of both the
precertificate and the final certificate. In between, it calls out to
the RA's new method `GetSCTs`.
The RA calls the new `CA.IssueCertificate` if the `UnsplitIssuance`
feature flag is true.
The RA had a metric that counted certificates by profile name and hash.
Since the RA doesn't receive a profile hash in the new flow, simply
record the total number of issuances.
Fixes https://github.com/letsencrypt/boulder/issues/7983
Update from go1.23.1 to go1.23.6 for our primary CI and release builds.
This brings in a few security fixes that aren't directly relevant to us.
Add go1.24.0 to our matrix of CI and release versions, to prepare for
switching to this next major version in prod.
These paths receive (literally) zero traffic, and they require the WFE
to duplicate the RA's authorization lifetime configuration. Since that
configuration is now per-profile, the WFE can no longer easily replicate
it, and the resulting staleness calculations will be wrong. Remove the
duplicated configuration, remove the unused endpoints that rely on it,
and remove the staleness-checking code which supported those endpoints.
Leave the non-ACME /get/ endpoint for certificates in place, because
checking staleness for those does not require any additional
configuration, and having a non-ACME serial-based API for certificates
is a good thing.
Fixes https://github.com/letsencrypt/boulder/issues/8007
The crl-storer passes along Cache-Control and Expires from the
crl-updater (because the crl-updater knows the UpdatePeriod).
The crl-updater calculates the Expires header based on when it expects
to update the CRL, plus a margin of error.
Fixes#8004
Aaron Gable
James Renken
Matthew McPherrin
Samantha Frank
Jacob Hoffman-Andrews
dependabot[bot]
Eng Zer Jun