To achieve this without breaking hashes of deployed configs, create a
ProfileConfigNew containing the new field (and removing some deprecated
fields).
Move the CA's profile-hashing logic into the `issuance` package, and
gate it on the presence of IncludeCRLDistributionPoints. If that field
is false (the default), create an instance of the old `ProfileConfig`
with the appropriate values and encode/hash that instead.
Note: the IncludeCRLDistributionPoints field does not yet control any
behavior. That will be part of #7974.
Part of #7094
Improve the test of the `ra.validationProfiles` field by providing a
constructed `map[string]*ValidationProfile` instead of constructing one
inside the test. Much like how this data is provided, or `nil`, in calls
to `ra.NewRegistrationAuthorityImpl()`.
Add "certificateProfileName" to the model used to insert new authz2 rows
and to the list of column names read when retrieving rows from the
authz2 table. Add support for this column to the functions which convert
to and from authz2 model types.
Add support for the profile field to core types so that it can be
returned by the SA.
Fixes https://github.com/letsencrypt/boulder/issues/7955
Add support in the RA for an allow list of accounts permitted to request
certificates containing the OCSP Must-Staple extension. If no allow list
is configured, all accounts are permitted. When a list is provided,
Finalize requests with Must-Staple are rejected unless the account is on
the list, and metrics are updated to track allowed and denied requests.
Fixes#7914
We've been seeing some flaky integration tests where issuance fails. The
integration test only has access to the generic user-facing error. The
real error is available as `InternalError` in the WFE logs, but we need
a higher log level to see it.
Add querying by explicit shard (SA.GetRevokedCertsByShard) in addition
to querying by temporal shard (SA.GetRevokedCerts).
Merge results from both kinds of shard. De-duplicate by serial within a
shard, because the same certificate could wind up in a temporal shard
that matches its explicit shard.
When de-duplicating, validate that revocation reasons are the same or
(very unlikely) represent a re-revocation based on demonstrating key
compromise. This can happen because the two different SA queries occur
at slightly different times.
Add unit testing that CRL entries make it through the whole pipeline
from SA, to CA, to uploader.
Rename some types in the unittest to be more accessible.
Tweak a comment in SA.UpdateRevokedCertificate to make it clear that
status _and_ reason are critical for re-revocation.
Note: This GetRevokedCertsByShard code path will always return zero
certificates right now, because nothing is writing to the
`revokedCertificates` table. Writing to that table is gated on
certificates having CRL URLs in them, which is not yet implemented (and
will be config-gated).
Part of #7094
- Fix 'requesteed' -> 'requested' in errors/errors.go
- Fix 'paylod' -> 'payload' in docs/acme-divergences.md
These changes address typos identified by the linter.
In revocation_test.go, fetch all CRLs, and look for revoked certificates
on both CRLs and OCSP.
Make s3-test-srv listen on all interfaces, so the CRL URLs in the CA
config work.
Add IssuerNameIDs to the CRL URLs in ca.json, to match how those CRLs
are uploaded to S3.
Make TestRevocation parallel. Speedup from ~60s to ~3s.
Increase ocsp-responder's allowed parallelism to account for parallel
test. Also, add "maxInflightSignings" to config/ since it's in prod.
"maxSigningWaiters" is not yet in prod, so don't move that field.
Add a mutex around running crl-updater, and decrease the log level so
errors stand out more when they happen.
The SA had some logic (not yet in use) to return revoked certificates
either by temporal sharding (if `req.ShardIdx` is zero) or by explicit
sharding (if `req.ShardIdx` is nonzero).
This PR splits the function into two. The existing `GetRevokedCerts`
always does temporal sharding. The new `GetRevokedCertsByShard` always
does explicit sharding. Eventually only `GetRevokedCertsByShard` will be
necessary. This change was discussed in
https://github.com/letsencrypt/boulder/issues/7094#issuecomment-2587940962
and is a precursor to having the crl-updater call both methods, so we
can merge the results when generating CRLs.
In RA.RevokedCertificate, if the certificate being revoked has a
crlDistributionPoints extension, parse the URL and pass the appropriate
shard to the SA.
This required some changes to the `admin` tool. When a malformed
certificate is revoked, we don't have a parsed copy of the certificate
to extract a CRL URL from. So, specifically when a malformed certificate
is being revoked, allow specifying a CRL shard. Because different
certificates will have different shards, require one-at-a-time
revocation for malformed certificates.
To support that refactoring, move the serial-cleaning functionality
earlier in the `admin` tool's flow.
Also, split out one of the cases handled by the `revokeCertificate`
helper in the RA. For admin malformed revocations, we need to accept a
human-specified ShardIdx, so call the SA directly in that case (and skip
stat increment since admin revocations aren't useful for metrics). This
allows `revokeCertificate` to be a more helpful helper, by extracting
serial, issuer ID, and CRL shard automatically from an
`*x509.Certificate`.
Note: we don't yet issue certificates with the crlDistributionPoints
extension, so this code will not be active until we start doing so.
Part of #7094.
Replace the non-MPIC-compliant IsCAAValid VA method with the correct
MPIC-compliant DoCAA VA method when the EnforceMPIC feature is enabled.
This fixes the mistake introduced in #7870.
Use MariaDB's "instant add column" feature to add a new
certificateProfileName column to the existing authz2 table. This column
is nullable to reflect the fact that profiles are optional, and to
mirror the similarly-added column on the orders table.
This change is standalone, with no code reading or writing this field,
so that it can be deployed to production and a follow-up change can
begin reading and writing the field all at once with no deployability
concerns.
Part of https://github.com/letsencrypt/boulder/issues/7955
This uses a pattern that is new to our tests. setup accepts a variadic
list of options, and uses a type switch to make use of those options
during setup. This allows us to pass setup only the options that are
relevant to any given test case, leaving the rest to sensible defaults.
Remove the singular Profile field from the CA config, as it has been
replaced by the plural CertProfiles key. Remove the Expiry, Backdate,
LintConfig, and IgnoredLints keys from the top-level CA config, as they
are now also configured on a per-profile basis. Remove the LifespanCRL
key from the CA config, as it is now configured within the CRLProfile.
For all of the above, remove transitional fallbacks from within
//ca/main.go.
These config changes were deployed to production in IN-10568, IN-10506,
and IN-10045.
Fixes https://github.com/letsencrypt/boulder/issues/7414
Fixes https://github.com/letsencrypt/boulder/issues/7159
Part of #7920
There will be a followup removing the remaining places that set
`contactsPresent`.
---------
Co-authored-by: Jacob Hoffman-Andrews <jsha+github@letsencrypt.org>
This is the final stage of #5554: removing the old, combined
`UpdateRegistration` flow, which has been replaced by
`UpdateRegistrationContact` and `UpdateRegistrationKey`. Those new
functions have their own tests.
The RA's `UpdateRegistration` function no longer has any callers (as of
#7827's deployment), so it is safely deployable to remove it from the SA
too, and its request from gRPC.
Fixes#5554
---------
Co-authored-by: Jacob Hoffman-Andrews <jsha+github@letsencrypt.org>
Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
In the WFE, store the User-Agent in a `context.Context` object. In our
gRPC interceptors, pass that field in a Metadata header, and re-add it
to `Context` on the server side.
Add a test in the gRPC interceptors that User-Agent is properly
propagated.
Note: this adds a new `setup()` function for the gRPC tests that is
currently only used by the new test. I'll upload another PR shortly that
expands the use of that function to more tests.
Fixes https://github.com/letsencrypt/boulder/issues/7792
The admin-revoker tool is dead. Long live the admin tool.
There's a number places that still reference admin-revoker, including
Boulder's ipki and the revocation source in the database which are still
used, even if the tool is gone. But nothing actually using the tool.
The initialIP column has been defaulted to 0.0.0.0 since #7760. Remove
this field from the all structs while leaving the schema itself intact.
Part of #7917
This is required now that we're going to issue certificates with only
the server EKU.
Fixes#7938
---------
Co-authored-by: James Renken <jprenken@users.noreply.github.com>
Replace looking for AlreadyRevoked (which is never returned by the
underlying SA method) with the correct NotFound. Also add a comment
documenting why this behavior exists.
Fixes https://github.com/letsencrypt/boulder/issues/3995
The schema tool used to parse log_list_schema.json doesn't work well
with the updated schema. This is going to be required to support
static-ct-api logs from current Chrome log lists.
Instead, use the loglist3 package inside the certificate-transparency-go
project, which Boulder already uses for CT submission otherwise.
As well, the Log IDs and keys returned from loglist3 have already been
base64 decoded, so this re-encodes them to minimize the impact on the
rest of the codebase and keep this change small.
The test log_list.json file needed to be made a bit more realistic for
loglist3 to parse without base64 or date parsing errors.
Remove code using `certificatesPerName` & `newOrdersRL` tables.
Deprecate `DisableLegacyLimitWrites` & `UseKvLimitsForNewOrder` flags.
Remove legacy `ratelimit` package.
Delete these RA test cases:
- `TestAuthzFailedRateLimitingNewOrder` (rl:
`FailedAuthorizationsPerDomainPerAccount`)
- `TestCheckCertificatesPerNameLimit` (rl: `CertificatesPerDomain`)
- `TestCheckExactCertificateLimit` (rl: `CertificatesPerFQDNSet`)
- `TestExactPublicSuffixCertLimit` (rl: `CertificatesPerDomain`)
Rate limits in NewOrder are now enforced by the WFE, starting here:
5a9b4c4b18/wfe2/wfe.go (L781)
We collect a batch of transactions to check limits, check them all at
once, go through and find which one(s) failed, and serve the failure
with the Retry-After that's furthest in the future. All this code
doesn't really need to be tested again; what needs to be tested is that
we're returning the correct failure. That code is
`NewOrderLimitTransactions`, and the `ratelimits` package's tests cover
this.
The public suffix handling behavior is tested by
`TestFQDNsToETLDsPlusOne`:
5a9b4c4b18/ratelimits/utilities_test.go (L9)
Some other RA rate limit tests were deleted earlier, in #7869.
Part of #7671.
Feedback from SRE was to just go straight to the new packaging.
Also, fix the Architecture field of the .deb to be amd64 (Debian
requires this specific value), and check that we are building on x86_64
OR amd64.
Add mustTime and mustTimestamp, each of which parses a time in a simple
format and panics if it cannot be parsed.
Also, make the intent of each check in the GetRevokedCerts tests a
little clearer by starting with a basicRequest, and then defining
variations in terms of that request.
Fix the "different issuer" case in `TestGetRevokedCerts`, which was not
actually setting a different issuer.
Remove the gates for the paused and revokedCertificates tables, which
are now live and in `config`. Refine the documentation for the
orderModelv2 migration.
Also move the ShutdownStopTimeout stanza next to timeout, and make the
comment the same across the multiple components. In the future we may
want to factor out some of the common config fields into a struct that
can be embedded.
Change the type of the orderModelv2 CertificateProfileName field to be a
pointer to a string, reflecting the fact that the underlying database
column is nullable. Add tests to ensure that order rows inserted with
either order model can be read using the other model.
Fixes https://github.com/letsencrypt/boulder/issues/7906
We recently started getting these errors in CI:
```
ERROR: Error installing fpm:
The last version of rchardet (~> 1.8) to support your Ruby & RubyGems was 1.8.0. Try installing it with `gem install rchardet -v 1.8.0` and then running the current command again
rchardet requires Ruby version >= 3.0.0. The current ruby version is 2.7.0.0.
```
Installing specific versions of dependencies fixes it.
And in the RA, log the notBefore of the previous issuance.
To make this happen, I had to hoist the "check for previous certificate"
up a level into `issueCertificateOuter`. That meant I also had to hoist
the "split off a WithoutCancel context" logic all the way up to
`FinalizeOrder`.
This reverts commit 242d746040 (#7796)
We want to make this change, but it carries some risk that we'd prefer
not to take over the holiday. We'd also like to keep `main` in a state
where it would be reasonable to deploy (even if, in practice, any
over-the-holiday deploy would be a hotfix, not a direct tag from
`main`).
Jacob Hoffman-Andrews
Samantha Frank
Aaron Gable
dependabot[bot]
James Renken
Shiloh Heurich
Matthew McPherrin