Change most functions in `ratelimits` to use full ACMEIdentifier(s) as
arguments, instead of using their values as strings. This makes the
plumbing from other packages more consistent, and allows us to:
Rename `FQDNsToETLDsPlusOne` to `coveringIdentifiers` and handle IP
identifiers, parsing IPv6 addresses into their covering /64 prefixes for
CertificatesPerDomain[PerAccount] bucket keys.
Port improved IP/CIDR validation logic to NewRegistrationsPerIPAddress &
PerIPv6Range.
Rename `domain` parts of bucket keys to either `identValue` or
`domainOrCIDR`.
Rename other internal functions to clarify that they now handle
identifier values, not just domains.
Add the new reserved IPv6 address range from RFC 9780.
For deployability, don't (yet) rename rate limits themselves; and
because it remains the name of the database table, preserve the term
`fqdnSets`.
Fixes#8223
Part of #7311
All of the identifiers being passed into the bucket construction helpers
have already passed through policy.WellFormedIdentifiers in the WFE. We
can trust that function, and our own ability to construct bucket keys,
to reduce the amount of revalidation we do before sending bucket keys to
redis.
The validateIdForName function is still used to validate override bucket
keys loaded from yaml.
Add `IdentifierTypes` to validation profiles' config, defaulting to DNS
if not set.
In `NewOrder`, check that the order's profile permits each identifier's
type.
Fixes#8137
Depends on #8173
Remove the ability for the issuance package to include the AIA OCSP URI
and the Must Staple (more properly known as the tlsRequest) extension in
certificates. Deprecate the "OmitOCSP" and "AllowMustStaple" profile
config keys, as they no longer have any effect. Similarly deprecate the
"OCSPURL" issuer config key, as it is no longer included in
certificates.
Update the tests to always include to CRLDP extension instead, and
remove some OCSP- or Stapling-specific test cases.
Fixes https://github.com/letsencrypt/boulder/issues/8179
In `vapb.IsCAAValidRequest`, even though CAA is only for DNS names,
deprecate `Domain` in favour of `Identifier` for consistency.
In `va.DoCAA`, reject attempts to validate CAA for non-DNS identifiers.
Rename `identifier` to `ident` inside some VA functions, also for
consistency.
In `ra.checkDCVAndCAA` & `ra.checkAuthorizationsCAA`, bypass CAA checks
for IP address identifiers.
Part of #7995
In `ra.NewOrder`, improve safety of authz reuse logic by making it
explicit that only DNS identifiers might be wildcards. Also, now that
the conditional statements need to be more complicated, collapse them
for brevity.
In `vapb.PerformValidationRequest`, remove `DnsName`.
In `ra.PerformValidation`, pass an `Identifier` instead of a `DnsName`.
In `ra.RevokeCertByApplicant`, check that the requester controls
identifiers of all types (not just DNS).
Fixes#7995 (the RA now fully supports IP address identifiers, except
for rate limits)
Fixes#7647
Part of #8023
Refactor `GetAuthorizations2`, `GetValidAuthorizations2` and
`GetValidOrderAuthorizations2` to support non-DNS identifier types.
Remove the deprecated `DnsNames` field from the
`GetAuthorizationsRequest` and `GetValidAuthorizationsRequest` structs.
All users of these structs use `Identifier` instead.
Fixes#7922
Part of #7311
Add a new config field for profiles which causes the profile to omit the
AIA OCSP URI. It can only be omitted if the CRLDP extension is
configured to be included instead. Enable this flag in config-next.
When a certificate is revoked, if it does not have an AIA OCSP URI,
don't bother with an Akamai OCSP purge.
Builds on #8089
Most of the changes in this PR relate to tests. Different from #8089, I
chose to keep testing of OCSP in the config-next world. This is because
we intend to keep operating OCSP even after we have stopped including it
in new certificates. So we should test it in as many environments as
possible.
Adds a WithURLFallback option to ocsp_helper. When
`ocsp_helper.ReqDer()` is called for a certificate with no OCSP URI, it
will query the fallback URL instead. As before, if the certificate has
an OCSP URI ocsp_helper will use that. Use that for all places in the
integration tests that call ocsp_helper.
Remove the deprecated `DnsNames` field from the `corepb.Order` proto
message. All users of this struct use `Identifiers` instead.
This unblocks future changes that will require `Order` users to handle
different identifier types.
Part of #7311
Remove the deprecated `DnsNames` field from the `CountFQDNSetsRequest`,
`FQDNSetExistsRequest`, and `GetOrderForNamesRequest` structs. All users
of these structs use `Identifier` instead.
Part of #7311
Remove the deprecated `DnsName` field from the core `Authorization`
struct. All users of this struct use `Identifier` instead.
This unblocks future changes that will require `Authorization` users to
handle different identifier types.
Part of #7311
For explicitly sharded certificates, CRL status is read from the
`revokedCertificates` table. This table gets written at revocation time.
At re-revocation time (for key compromise), it only gets written by the
SA if the caller passes a nonzero ShardIdx to UpdateRevokedCertificate.
The RA was never passing a nonzero ShardIdx to UpdateRevokedCertificate.
Add `identifier` fields, which will soon replace the `dnsName` fields,
to:
- `corepb.Authorization`
- `corepb.Order`
- `rapb.NewOrderRequest`
- `sapb.CountFQDNSetsRequest`
- `sapb.CountInvalidAuthorizationsRequest`
- `sapb.FQDNSetExistsRequest`
- `sapb.GetAuthorizationsRequest`
- `sapb.GetOrderForNamesRequest`
- `sapb.GetValidAuthorizationsRequest`
- `sapb.NewOrderRequest`
Populate these `identifier` fields in every function that creates
instances of these structs.
Use these `identifier` fields instead of `dnsName` fields (at least
preferentially) in every function that uses these structs. When crossing
component boundaries, don't assume they'll be present, for
deployability's sake.
Deployability note: Mismatched `cert-checker` and `sa` versions will be
incompatible because of a type change in the arguments to
`sa.SelectAuthzsMatchingIssuance`.
Part of #7311
Rename `FromDNSNames` to `NewDNSSlice`, since it's exactly `NewDNS`
except for slices.
Rename `AsProto` to use the "To" prefix, since it's the opposite of
"From".
Add a named type `ACMEIdentifiers` so that we can add methods to slices.
We will have a lot of slice handling code coming up, which this will
make more elegant and readable.
Add a comment to explain naming conventions in the `identifier` package.
Part of #7311
Alternative to #8068
Update the SA to re-query the database for the updated account after
deactivating it, and return this to the RA. Update the RA to pass this
value through to the WFE. Update the WFE to return this value, rather
than locally modifying the pre-deactivation account object, if it gets
one (for deployability).
Also remove the RA's requirement that the request object specify its
current status so that the request can be trimmed down to just an ID.
This proto change is backwards-compatible because the new
DeactivateRegistrationRequest's registrationID field has the same type
(int64) and field number (1) as corepb.Registration's id field.
Part of https://github.com/letsencrypt/boulder/issues/5554
- Plumb the "replaces" value from the WFE through to the SA via the RA
- Store validated "replaces" value for new orders in the orders table
- Reflect the stored "replaces" value to subscribers in the order object
- Reorder CertificateProfileName before Replaces/ReplacesSerial in RA
and SA protos for consistency
Fixes#8034
Remove some RA tests that were checking for errors specific to the split
issuance flow. Make one of the tests test GetSCTs directly, which makes
for a much nicer test!
Replace DCV and CAA checks (PerformValidation and IsCAAValid) in
va/va.go and va/caa.go with their MPIC compliant counterparts (DoDCV and
DoCAA) in va/vampic.go. Deprecate EnforceMultiCAA and EnforceMPIC and
default code paths as though they are both true. Require that RIR and
Perspective be set for primary and remote VAs.
Fixes#7965Fixes#7819
Add MaxNames to the set of things that can be configured on a
per-profile basis. Remove all references to the RA's global maxNames,
replacing them with reference's to the current profile's maxNames. Add
code to the RA's main() to copy a globally-configured MaxNames into each
profile, for deployability.
Also remove any understanding of MaxNames from the WFE, as it is
redundant with the RA and is not configured in staging or prod. Instead,
hardcode the upper limit of 100 into the ratelimit package itself.
Fixes https://github.com/letsencrypt/boulder/issues/7993
Add a new RPC to the CA: `IssueCertificate` covers issuance of both the
precertificate and the final certificate. In between, it calls out to
the RA's new method `GetSCTs`.
The RA calls the new `CA.IssueCertificate` if the `UnsplitIssuance`
feature flag is true.
The RA had a metric that counted certificates by profile name and hash.
Since the RA doesn't receive a profile hash in the new flow, simply
record the total number of issuances.
Fixes https://github.com/letsencrypt/boulder/issues/7983
Update from go1.23.1 to go1.23.6 for our primary CI and release builds.
This brings in a few security fixes that aren't directly relevant to us.
Add go1.24.0 to our matrix of CI and release versions, to prepare for
switching to this next major version in prod.
Remove the RA's deprecated top-level config keys which used to control
order and authz lifetimes. Make the new profile-based config keys which
replaced them required.
Since configuring a profile and default profile name is now mandatory,
always supply a profile name to the CA when requesting issuance.
Fixes https://github.com/letsencrypt/boulder/issues/7986
Continue to use a 24-hour cutoff for authzs with "long" lifetimes, so
that our behavior is unchanged for authzs created with no profile
specified. Use a 1-hour cutoff for authzs with "short" (less than
24-hour) lifetimes, so that we can reuse authzs created with modern
profiles. Use linear interpolation between those values.
Fixes https://github.com/letsencrypt/boulder/issues/7994
If validation profiles haven't been explicitly configured, use the
default profile for all incoming requests regardless of which profile
they specify.
Fixes https://github.com/letsencrypt/boulder/issues/7605
Add three new fields to the ra.ValidationProfile structure, representing
the profile's pending authorization lifetime (used to assign an
expiration when a new authz is created), valid authorization lifetime
(used to assign an expiration when an authz is successfully validated),
and order lifetime (used to assign an expiration when a new order is
created). Remove the prior top-level fields which controlled these
values across all orders.
Add a "defaultProfileName" field to the RA as well, to facilitate
looking up a default set of lifetimes when the order doesn't specify a
profile. If this default name is explicitly configured, always provide
it to the CA when requesting issuance, so we don't have to duplicate the
default between the two services.
Modify the RA's config struct in a corresponding way: add three new
fields to the ValidationProfiles structure, and deprecate the three old
top-level fields. Also upgrade the ra.NewValidationProfile constructor
to handle these new fields, including doing validation on their values.
Fixes https://github.com/letsencrypt/boulder/issues/7605
In the RA, inspect the profile of all authorizations returned when
looking for authz reuse, and refuse to reuse any whose profile doesn't
match the requested profile of the current NewOrder request.
Fixes https://github.com/letsencrypt/boulder/issues/7949
Add support in the RA for an allow list of accounts permitted to request
certificates containing the OCSP Must-Staple extension. If no allow list
is configured, all accounts are permitted. When a list is provided,
Finalize requests with Must-Staple are rejected unless the account is on
the list, and metrics are updated to track allowed and denied requests.
Fixes#7914
In RA.RevokedCertificate, if the certificate being revoked has a
crlDistributionPoints extension, parse the URL and pass the appropriate
shard to the SA.
This required some changes to the `admin` tool. When a malformed
certificate is revoked, we don't have a parsed copy of the certificate
to extract a CRL URL from. So, specifically when a malformed certificate
is being revoked, allow specifying a CRL shard. Because different
certificates will have different shards, require one-at-a-time
revocation for malformed certificates.
To support that refactoring, move the serial-cleaning functionality
earlier in the `admin` tool's flow.
Also, split out one of the cases handled by the `revokeCertificate`
helper in the RA. For admin malformed revocations, we need to accept a
human-specified ShardIdx, so call the SA directly in that case (and skip
stat increment since admin revocations aren't useful for metrics). This
allows `revokeCertificate` to be a more helpful helper, by extracting
serial, issuer ID, and CRL shard automatically from an
`*x509.Certificate`.
Note: we don't yet issue certificates with the crlDistributionPoints
extension, so this code will not be active until we start doing so.
Part of #7094.
Replace the non-MPIC-compliant IsCAAValid VA method with the correct
MPIC-compliant DoCAA VA method when the EnforceMPIC feature is enabled.
This fixes the mistake introduced in #7870.
Part of #7920
There will be a followup removing the remaining places that set
`contactsPresent`.
---------
Co-authored-by: Jacob Hoffman-Andrews <jsha+github@letsencrypt.org>
This is the final stage of #5554: removing the old, combined
`UpdateRegistration` flow, which has been replaced by
`UpdateRegistrationContact` and `UpdateRegistrationKey`. Those new
functions have their own tests.
The RA's `UpdateRegistration` function no longer has any callers (as of
#7827's deployment), so it is safely deployable to remove it from the SA
too, and its request from gRPC.
Fixes#5554
---------
Co-authored-by: Jacob Hoffman-Andrews <jsha+github@letsencrypt.org>
Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
In the WFE, store the User-Agent in a `context.Context` object. In our
gRPC interceptors, pass that field in a Metadata header, and re-add it
to `Context` on the server side.
Add a test in the gRPC interceptors that User-Agent is properly
propagated.
Note: this adds a new `setup()` function for the gRPC tests that is
currently only used by the new test. I'll upload another PR shortly that
expands the use of that function to more tests.
Fixes https://github.com/letsencrypt/boulder/issues/7792
The admin-revoker tool is dead. Long live the admin tool.
There's a number places that still reference admin-revoker, including
Boulder's ipki and the revocation source in the database which are still
used, even if the tool is gone. But nothing actually using the tool.
Replace looking for AlreadyRevoked (which is never returned by the
underlying SA method) with the correct NotFound. Also add a comment
documenting why this behavior exists.
Fixes https://github.com/letsencrypt/boulder/issues/3995
Remove code using `certificatesPerName` & `newOrdersRL` tables.
Deprecate `DisableLegacyLimitWrites` & `UseKvLimitsForNewOrder` flags.
Remove legacy `ratelimit` package.
Delete these RA test cases:
- `TestAuthzFailedRateLimitingNewOrder` (rl:
`FailedAuthorizationsPerDomainPerAccount`)
- `TestCheckCertificatesPerNameLimit` (rl: `CertificatesPerDomain`)
- `TestCheckExactCertificateLimit` (rl: `CertificatesPerFQDNSet`)
- `TestExactPublicSuffixCertLimit` (rl: `CertificatesPerDomain`)
Rate limits in NewOrder are now enforced by the WFE, starting here:
5a9b4c4b18/wfe2/wfe.go (L781)
We collect a batch of transactions to check limits, check them all at
once, go through and find which one(s) failed, and serve the failure
with the Retry-After that's furthest in the future. All this code
doesn't really need to be tested again; what needs to be tested is that
we're returning the correct failure. That code is
`NewOrderLimitTransactions`, and the `ratelimits` package's tests cover
this.
The public suffix handling behavior is tested by
`TestFQDNsToETLDsPlusOne`:
5a9b4c4b18/ratelimits/utilities_test.go (L9)
Some other RA rate limit tests were deleted earlier, in #7869.
Part of #7671.
And in the RA, log the notBefore of the previous issuance.
To make this happen, I had to hoist the "check for previous certificate"
up a level into `issueCertificateOuter`. That meant I also had to hoist
the "split off a WithoutCancel context" logic all the way up to
`FinalizeOrder`.
The RA's DeactivateAccount method expects the account provided to it by
the WFE to still have status Valid. The new WFE deactivation code was
hardcoding the status to Deactivated. Fix the WFE to pass the account's
current status instead.
Add an integration test to confirm both the breakage and the fix. Also
leave behind some TODOs to simplify this codepath further, and not
require the status to be provided at all.
Part of #5554
Mostly we refer consistently to token bucket, but these two places (one
of which is soon to be removed) still had the "leaky" terminology, which
is potentially confusing.
Today, we have VA.PerformValidation, a method called by the RA at
challenge time to perform DCV and check CAA. We also have VA.IsCAAValid,
a method invoked by the RA at finalize time when a CAA re-check is
necessary. Both of these methods can be executed on remote VA
perspectives by calling the generic VA.performRemoteValidation.
This change splits VA.PerformValidation into VA.DoDCV and VA.DoCAA,
which are both called on remote VA perspectives by calling the generic
VA.doRemoteOperation. VA.DoDCV, VA.DoCAA, and VA.doRemoteOperation
fulfill the requirements of SC-067 V3: Require Multi-Perspective
Issuance Corroboration by:
- Requiring at least three distinct perspectives, as outlined in the
"Phased Implementation Timeline" in BRs section 3.2.2.9 ("Effective
March 15, 2025").
- Ensuring that the number of non-corroborating (failing) perspectives
remains below the threshold defined by the "Table: Quorum Requirements"
in BRs section 3.2.2.9.
- Ensuring that corroborating (passing) perspectives reside in at least
2 distinct Regional Internet Registries (RIRs) per the "Phased
Implementation Timeline" in BRs section 3.2.2.9 ("Effective March 15,
2026").
- Including an MPIC summary consisting of: passing perspectives, failing
perspectives, passing RIRs, and a quorum met for issuance (e.g., 2/3 or
3/3) in each validation audit log event, per BRs Section 5.4.1,
Requirement 2.8.
When the new SeparateDCVAndCAAChecks feature flag is enabled on the RA,
calls to VA.IsCAAValid (during finalization) and VA.PerformValidation
(during challenge) are replaced with calls to VA.DoCAA and a sequence of
VA.DoDCV followed by VA.DoCAA, respectively.
Fixes#7612Fixes#7614Fixes#7615Fixes#7616
James Renken
Aaron Gable
Samantha Frank
Jacob Hoffman-Andrews
Matthew McPherrin