Add mustTime and mustTimestamp, each of which parses a time in a simple
format and panics if it cannot be parsed.
Also, make the intent of each check in the GetRevokedCerts tests a
little clearer by starting with a basicRequest, and then defining
variations in terms of that request.
Fix the "different issuer" case in `TestGetRevokedCerts`, which was not
actually setting a different issuer.
Remove the gates for the paused and revokedCertificates tables, which
are now live and in `config`. Refine the documentation for the
orderModelv2 migration.
Also move the ShutdownStopTimeout stanza next to timeout, and make the
comment the same across the multiple components. In the future we may
want to factor out some of the common config fields into a struct that
can be embedded.
Change the type of the orderModelv2 CertificateProfileName field to be a
pointer to a string, reflecting the fact that the underlying database
column is nullable. Add tests to ensure that order rows inserted with
either order model can be read using the other model.
Fixes https://github.com/letsencrypt/boulder/issues/7906
We recently started getting these errors in CI:
```
ERROR: Error installing fpm:
The last version of rchardet (~> 1.8) to support your Ruby & RubyGems was 1.8.0. Try installing it with `gem install rchardet -v 1.8.0` and then running the current command again
rchardet requires Ruby version >= 3.0.0. The current ruby version is 2.7.0.0.
```
Installing specific versions of dependencies fixes it.
And in the RA, log the notBefore of the previous issuance.
To make this happen, I had to hoist the "check for previous certificate"
up a level into `issueCertificateOuter`. That meant I also had to hoist
the "split off a WithoutCancel context" logic all the way up to
`FinalizeOrder`.
This reverts commit 242d746040 (#7796)
We want to make this change, but it carries some risk that we'd prefer
not to take over the holiday. We'd also like to keep `main` in a state
where it would be reasonable to deploy (even if, in practice, any
over-the-holiday deploy would be a hotfix, not a direct tag from
`main`).
Boulder switched from multiple binaries to one by having symlinks for
the old binaries, but we invoke boulder via subcommands now. This drops
support for running via symlinks in Boulder, and drops them from the
build process.
This does explicitly list out the four binaries in the makefile, which I
think explicitly listing them is fine given that we rarely add them.
This also avoids needing to duplicate mentioning the special ct-test-srv
in the deb/tar rules. We could probably just look at what's in `bin/`
after `go install ./...`, but I didn't want to get too into makefile
changes.
We haven't used the symlinked versions of commands for a while, and can
drop them from builds.
This also drops the .rpm builds, which we also haven't used in a long
time.
This removes the `handlerPath` parameter to various calls, which was
used solely to distinguish the `-v3`-style paths from the `WithAccount`
paths.
Also, this removes `WithAccount` from all names that had it. The fact
that these URLS include an account ID is now implicit.
This allows us to replace FQDNSetExists with a call to
FQDNSetTimestampsForWindow, with Limit set to 1. That, in turn, will
allow us to log the time since issuance of the most recent certificate
with a given FQDNSet.
Add a new `ratelimits.NewTransactionBuilderWithLimits` constructor which
takes pre-populated rate limit data, instead of filenames for reading it
off disk.
Use this new constructor to change rate limits during RA tests, instead
of using extra `testdata` files.
Fix ARI renewals' exception from rate limits: consider `isARIRenewal` as
part of the `isRenewal` arg to `checkNewOrderLimits`.
Remove obsolete RA tests for rate limits that are now only checked in
the WFE.
Update remaining new order rate limit tests from deprecated `ratelimit`s
to new Redis `ratelimits`.
This brings in several new and useful lints. It also brings in one CABF
BR lint which we have to ignore in our default profile which includes
the Subject Key Identifier extension:
"w_ext_subject_key_identifier_not_recommended_subscriber". In our modern
profile which omits several fields, we have to ignore the opposite
RFC5280 lint "w_ext_subject_key_identifier_missing_sub_cert".
Release notes: https://github.com/zmap/zlint/releases/tag/v3.6.4
Changelog: https://github.com/zmap/zlint/compare/v3.6.0...v3.6.4
Note that the majority of the ~400 file changes are merely copyright
date changes.
The corresponding production config changes tracked in IN-10466 are
complete.
The RA's DeactivateAccount method expects the account provided to it by
the WFE to still have status Valid. The new WFE deactivation code was
hardcoding the status to Deactivated. Fix the WFE to pass the account's
current status instead.
Add an integration test to confirm both the breakage and the fix. Also
leave behind some TODOs to simplify this codepath further, and not
require the status to be provided at all.
Part of #5554
#7782 fixed an issue where concurrent requests to the same existing
bucket ignored all but one rate limit spend. However, concurrent
requests to the same empty bucket can still cause multiple
initializations that skip all but one spend. Use BatchSetNotExisting
(SETNX in Redis) to detect this scenario and then fall back to
BatchIncrement (INCRBY in Redis).
#7782 sets the TTL (Time-To-Live) of incremented buckets to the maximum
possible burst for the applied limit. Because this TTL doesn’t match the
TAT, these buckets can become "stale," holding a TAT in the past.
Incrementing these stale buckets by cost * emissionInterval leaves the
new TAT behind the current time, allowing clients who are sometimes idle
to gain extra burst capacity. Instead, use BatchSet (SET in Redis) to
overwrite the TAT to now + cost * emissionInterval. Though this
introduces a similar race condition as empty buckets, it’s less harmful
than granting extra burst capacity.
webpki.go was discarding stdout when "make build" failed. We can make it
print stdout in that context, but it's more straightforward to run "make
build" from the shell script that calls webpki.go, where its stdout will
naturally be emitted.
Inspired by a recent CI run where there was a straightforward build
failure in some of Boulder's code, but it was masked by an error running
webpki.go in the `bsetup` container.
Mostly we refer consistently to token bucket, but these two places (one
of which is soon to be removed) still had the "leaky" terminology, which
is potentially confusing.
Remove `debugAddr` from the `admin` tool, which doesn't use it - or need
it, now that `newStatsRegistry` via `StatsAndLogging` doesn't require
it.
Remove `debugAddr` from `config-next/sfe.json`, as we usually set it on
the CLI instead.
Fixes#7838
Today, we have VA.PerformValidation, a method called by the RA at
challenge time to perform DCV and check CAA. We also have VA.IsCAAValid,
a method invoked by the RA at finalize time when a CAA re-check is
necessary. Both of these methods can be executed on remote VA
perspectives by calling the generic VA.performRemoteValidation.
This change splits VA.PerformValidation into VA.DoDCV and VA.DoCAA,
which are both called on remote VA perspectives by calling the generic
VA.doRemoteOperation. VA.DoDCV, VA.DoCAA, and VA.doRemoteOperation
fulfill the requirements of SC-067 V3: Require Multi-Perspective
Issuance Corroboration by:
- Requiring at least three distinct perspectives, as outlined in the
"Phased Implementation Timeline" in BRs section 3.2.2.9 ("Effective
March 15, 2025").
- Ensuring that the number of non-corroborating (failing) perspectives
remains below the threshold defined by the "Table: Quorum Requirements"
in BRs section 3.2.2.9.
- Ensuring that corroborating (passing) perspectives reside in at least
2 distinct Regional Internet Registries (RIRs) per the "Phased
Implementation Timeline" in BRs section 3.2.2.9 ("Effective March 15,
2026").
- Including an MPIC summary consisting of: passing perspectives, failing
perspectives, passing RIRs, and a quorum met for issuance (e.g., 2/3 or
3/3) in each validation audit log event, per BRs Section 5.4.1,
Requirement 2.8.
When the new SeparateDCVAndCAAChecks feature flag is enabled on the RA,
calls to VA.IsCAAValid (during finalization) and VA.PerformValidation
(during challenge) are replaced with calls to VA.DoCAA and a sequence of
VA.DoDCV followed by VA.DoCAA, respectively.
Fixes#7612Fixes#7614Fixes#7615Fixes#7616
Early drafts of the ACME spec said that clients should retrieve their existing account information by POSTing the empty JSON object `{}` to their account URL. This instruction was removed in the final version of ACME, replaced by the concept of POST-as-GET, which uses a wholly empty body to accomplish the same goal. However, Boulder has continued to incidentally support this behavior: when we receive an empty JSON object, our `updateAccount` code in the RA applies their desired diff (none) on top of their current account, writes it back to the database, and returns the updated account object...which hasn't actually changed. This behavior is also half-tested by `TestEmptyAccount`, but that test is actually testing that the MockRA implements the same behavior as the real RA; it's not truly testing the WFE's behavior.
This PR changes the WFE to explicitly treat receiving the empty JSON object as a request to retrieve the account data unchanged, rather than implicitly relying on internal details of the RA's account-update logic, which are expected to change in #7827.
---------
Co-authored-by: Jacob Hoffman-Andrews <jsha+github@letsencrypt.org>
Github is currently rolling out ubuntu-latest as ubuntu-24.04. Manage
that switch explicitly by running most jobs on 24.04
https://github.com/actions/runner-images/issues/10636
This keeps the release on 20.04 to ensure released binaries can run on
older operating systems (because of CGO/glibc versions)
All 4 usages of the `maps.Keys` function from the
`golang.org/x/exp/maps` package can be refactored to a simpler
alternative. If we need it in the future, it is available in the
standard library since Go 1.23.
- Ensure the Perspective and RIR reported by each remoteVA in the
*vapb.ValidationResult returned by VA.PerformValidation, matches the
expected local configuration when that configuration is present.
- Correct "AfriNIC" to "AFRINIC", everywhere.
Part of https://github.com/letsencrypt/boulder/issues/7819
Pending authz reuse is a nice-to-have feature because it allows us to
create fewer rows in the authz database table when creating new orders.
However, stats show that less than 2% of authorizations that we attach
to new orders are reused pending authzs. And as we move towards using a
more streamlined database schema to store our orders, authorizations,
and validation attempts, disabling pending authz reuse will greatly
simplify our database schema and code.
CPS Compliance Review: our CPS does not speak to whether or not we reuse
pending authorizations for new orders.
IN-10859 tracks enabling this flag in prod
Part of https://github.com/letsencrypt/boulder/issues/7715
For batch operations, include the operation and the number of keys in
the error message. This should help diagnose whether we are getting `i/o
timeout` errors disproportionately for larger requests, or for certain
operations.
Also, make the ignored errors part of the overall WFE request logs,
which allows us to get additional context, like whether certain
requesters or domain names are getting disproportionately many errors.
Related to #7846.
Greatly simplify the two RA unit tests covering failed validations and
account+identifier pausing. Most importantly, directly manipulate the
ratelimit backing store during test setup, to avoid having to "perform"
extra validations.
Fixes https://github.com/letsencrypt/boulder/issues/7812
The purpose of these RA and WFE unit tests is to test how they deal with
certain rate limit conditions, not to test talking to an actual redis
instance. Streamline the tests by having them talk to an in-memory rate
limits store, rather than a redis-backed one.
This reduces the number of validations that get left indefinitely in
"pending" state.
Rename `DrainFinalize()` to `Drain()` to indicate that it now covers
more cases than just finalize.
- Remove undeployed feature flag MultiCAAFullResults
- Perform local CAA checks prior to initiating remote checks, instead of
starting remote checks and proceeding to perform local checks.
- Remove VA.IsCAAValid specific remote validation logic, use
VA.performRemoteOperation instead
- Refactor va.logRemoteResults to be easier to test and omit the RVA
problem
- Drive-by fix: Calculate logEvent.Latency with va.clk.Since() instead
of time.Since() like everything else in VA.performRemoteOperation
- Make performRemoteValidation a more generic function that returns a
new remoteResult interface
- Modify the return value of IsCAAValid and PerformValidation to satisfy
the remoteResult interface
- Include compile time checks and tests that pass an arbitrary operation
- Make the primary VA aware of the expected Perspective and RIR of each
remote VA.
- All Perspectives should be unique, have the primary VA check for
duplicate Perspectives at startup.
- Update test setup functions to ensure that each remote VA client and
corresponding inmem impl have a matching perspective and RIR.
Part of #7819
When a client deactivates a pending authorization, count that towards
their FailedAuthorizationsPerDomainPerAccount and
FailedAuthorizationsForPausingPerDomainPerAccount rate limits. This
should help curb the few clients which constantly create new orders and
authzs, deactivate those pending authzs preventing reuse of them or
their orders, and then rinse and repeat.
Fixes https://github.com/letsencrypt/boulder/issues/7834
Now that the paths with an account (and no `-v3`) are the default,
rename the old-style path constants and handlers to reflect that they
are deprecated.
Part of #7683.
Jacob Hoffman-Andrews