Commit Graph

70 Commits

Author SHA1 Message Date
James Renken 7214b285e4
identifier: Remove helper funcs from PB identifiers migration (#8236)
Remove `ToDNSSlice`, `FromProtoWithDefault`, and
`FromProtoSliceWithDefault` now that all their callers are gone. All
protobufs but one have migrated from DnsNames to Identifiers.

Remove TODOs for the exception, `ValidationRecord`, where an identifier
type isn't appropriate and it really only needs a string.

Rename `corepb.ValidationRecord.DnsName` to `Hostname` for clarity, to
match the corresponding PB's field name.

Improve various comments and docs re: IP address identifiers.

Depends on #8221 (which removes the last callers)
Fixes #8023
2025-06-13 12:55:32 -07:00
James Renken 9b9ed86c10
sa: Encode IP identifiers for issuedNames (#8210)
Move usage of `sa.ReverseName` to a new `sa.EncodeIssuedName`, which
detects IP addresses and exempts them from being reversed. Retain
`reverseName` as an internal helper function.

Update `id-exporter`, `reversed-hostname-checker`, and tests to use the
new function and handle IP addresses.

Part of #7311
2025-05-27 14:55:19 -07:00
Jacob Hoffman-Andrews 388c68cb49
sa: use internal certificateStatusModel instead of core.CertificateStatus (#8159)
Part of https://github.com/letsencrypt/boulder/issues/8112
2025-05-12 14:53:08 -07:00
Samantha Frank e625ff3534
sa: Store and manage rate limit overrides in the database (#8142)
Add support for managing and querying rate limit overrides in the
database.
- Add `sa.AddRateLimitOverride` to insert or update a rate limit
override. This will be used during Rate Limit Override Portal to commit
approved overrides to the database.
- Add `sa.DisableRateLimitOverride` and `sa.EnableRateLimitOverride` to
toggle override state. These will be used by the `admin` tool.
- Add `sa.GetRateLimitOverride` to retrieve a single override by limit
enum and bucket key. This will be used by the Rate Limit Portal to
prevent duplicate or downgrade requests but allow upgrade requests.
- Add `sa.GetEnabledRateLimitOverrides` to stream all currently enabled
overrides. This will be used by the rate limit consumers (`wfe` and
`ra`) to refresh the overrides in-memory.
- Implement test coverage for all new methods.
2025-05-08 14:50:30 -04:00
Aaron Gable 0038149c79
Fix profile comparison when looking for authzs to reuse (#8144)
Previously, if the request asked for a profile, we were comparing the
address of that requested profile to the address of the profile field of
the found authz. Obviously these addresses were never the same. Instead,
compare the actual values, with an added nil check for safety.

This fixes a bug reported on the community forum. The updated test fails
without the accompanying code change.
2025-04-25 15:24:50 -07:00
Jacob Hoffman-Andrews 967d722cf4
sa: use internal certificateModel (#8130)
This follows the system we've used for other types, where the SA has a
model type that is converted to a proto message for use outside the SA.

Part of #8112.
2025-04-21 13:48:29 -07:00
James Renken 722f7c5318
sa: Support new identifier types in authz getting funcs (#8104)
Refactor `GetAuthorizations2`, `GetValidAuthorizations2` and
`GetValidOrderAuthorizations2` to support non-DNS identifier types.

Remove the deprecated `DnsNames` field from the
`GetAuthorizationsRequest` and `GetValidAuthorizationsRequest` structs.
All users of these structs use `Identifier` instead.

Fixes #7922
Part of #7311
2025-04-10 10:57:17 -07:00
James Renken ff9e59d70b
core: Remove DnsNames from Order (#8108)
Remove the deprecated `DnsNames` field from the `corepb.Order` proto
message. All users of this struct use `Identifiers` instead.

This unblocks future changes that will require `Order` users to handle
different identifier types.

Part of #7311
2025-04-08 15:17:18 -07:00
James Renken 9b53c3455b
sa: Remove DnsNames from more request protos (#8105)
Remove the deprecated `DnsNames` field from the `CountFQDNSetsRequest`,
`FQDNSetExistsRequest`, and `GetOrderForNamesRequest` structs. All users
of these structs use `Identifier` instead.

Part of #7311
2025-04-08 13:38:03 -07:00
James Renken 38a7197909
sa: Support IP identifiers in CountInvalidAuthorizations2 (#8098)
Remove the deprecated `DnsName` field from the
`CountInvalidAuthorizationsRequest` struct. All users of this struct use
`Identifier` instead.

Part of #7311
2025-04-08 10:15:08 -07:00
James Renken 3f879ed0b4
Add Identifiers to Authorization & Order structs (#7961)
Add `identifier` fields, which will soon replace the `dnsName` fields,
to:
- `corepb.Authorization`
- `corepb.Order`
- `rapb.NewOrderRequest`
- `sapb.CountFQDNSetsRequest`
- `sapb.CountInvalidAuthorizationsRequest`
- `sapb.FQDNSetExistsRequest`
- `sapb.GetAuthorizationsRequest`
- `sapb.GetOrderForNamesRequest`
- `sapb.GetValidAuthorizationsRequest`
- `sapb.NewOrderRequest`

Populate these `identifier` fields in every function that creates
instances of these structs.

Use these `identifier` fields instead of `dnsName` fields (at least
preferentially) in every function that uses these structs. When crossing
component boundaries, don't assume they'll be present, for
deployability's sake.

Deployability note: Mismatched `cert-checker` and `sa` versions will be
incompatible because of a type change in the arguments to
`sa.SelectAuthzsMatchingIssuance`.

Part of #7311
2025-03-26 10:30:24 -07:00
James Renken edc3c7fa6d
Shorten "identifier(s)" in variable names & function arguments (#8066)
For consistency, and to prevent confusion with the `identifier` package,
use "ident(s)" instead.

Part of #7311
2025-03-14 10:59:38 -07:00
Aaron Gable 212a66ab49
Update go versions in CI and release (#7971)
Update from go1.23.1 to go1.23.6 for our primary CI and release builds.
This brings in a few security fixes that aren't directly relevant to us.

Add go1.24.0 to our matrix of CI and release versions, to prepare for
switching to this next major version in prod.
2025-02-19 14:37:01 -08:00
Aaron Gable 6695895f8b
RA: Don't reuse authzs with mismatched profiles (#7967)
In the RA, inspect the profile of all authorizations returned when
looking for authz reuse, and refuse to reuse any whose profile doesn't
match the requested profile of the current NewOrder request.

Fixes https://github.com/letsencrypt/boulder/issues/7949
2025-02-03 16:47:35 -05:00
Jacob Hoffman-Andrews 02af55293e
sa: add GetRevokedCertsByShard (#7946)
The SA had some logic (not yet in use) to return revoked certificates
either by temporal sharding (if `req.ShardIdx` is zero) or by explicit
sharding (if `req.ShardIdx` is nonzero).

This PR splits the function into two. The existing `GetRevokedCerts`
always does temporal sharding. The new `GetRevokedCertsByShard` always
does explicit sharding. Eventually only `GetRevokedCertsByShard` will be
necessary. This change was discussed in
https://github.com/letsencrypt/boulder/issues/7094#issuecomment-2587940962
and is a precursor to having the crl-updater call both methods, so we
can merge the results when generating CRLs.
2025-01-22 09:46:57 -08:00
Aaron Gable 6b1e7f04e8
SA: Clean up pre-profile order schema and feature flag (#7953)
Deprecate the MultipleCertificateProfiles feature flag, which has been
enabled in both Staging and Prod. Delete all code protected by that flag
being false, namely the orderModelv1 type and its support code. Update
the config schema to match the config-next schema.

Fixes https://github.com/letsencrypt/boulder/issues/7324
Fixes https://github.com/letsencrypt/boulder/issues/7408
2025-01-17 17:15:01 -08:00
James Renken e4668b4ca7
Deprecate DisableLegacyLimitWrites & UseKvLimitsForNewOrder flags; remove code using certificatesPerName & newOrdersRL tables (#7858)
Remove code using `certificatesPerName` & `newOrdersRL` tables.

Deprecate `DisableLegacyLimitWrites` & `UseKvLimitsForNewOrder` flags.

Remove legacy `ratelimit` package.

Delete these RA test cases:

- `TestAuthzFailedRateLimitingNewOrder` (rl:
`FailedAuthorizationsPerDomainPerAccount`)
- `TestCheckCertificatesPerNameLimit` (rl: `CertificatesPerDomain`)
- `TestCheckExactCertificateLimit` (rl: `CertificatesPerFQDNSet`)
- `TestExactPublicSuffixCertLimit` (rl: `CertificatesPerDomain`)

Rate limits in NewOrder are now enforced by the WFE, starting here:
5a9b4c4b18/wfe2/wfe.go (L781)

We collect a batch of transactions to check limits, check them all at
once, go through and find which one(s) failed, and serve the failure
with the Retry-After that's furthest in the future. All this code
doesn't really need to be tested again; what needs to be tested is that
we're returning the correct failure. That code is
`NewOrderLimitTransactions`, and the `ratelimits` package's tests cover
this.

The public suffix handling behavior is tested by
`TestFQDNsToETLDsPlusOne`:
5a9b4c4b18/ratelimits/utilities_test.go (L9)

Some other RA rate limit tests were deleted earlier, in #7869.

Part of #7671.
2025-01-10 12:50:57 -08:00
Jacob Hoffman-Andrews 635f43266a
use core.IsAnyNilOrZero more places (#7925)
There were a bunch of places that had `TODO(#7153)`; that issue is now
closed, so let's tidy up.
2025-01-07 15:48:47 -08:00
Jacob Hoffman-Andrews d42865c187
sa: add Limit field to CountFQDNSetsRequest (#7887)
This allows us to replace FQDNSetExists with a call to
FQDNSetTimestampsForWindow, with Limit set to 1. That, in turn, will
allow us to log the time since issuance of the most recent certificate
with a given FQDNSet.
2024-12-19 10:11:50 -08:00
Jacob Hoffman-Andrews 1f9f2bccf5
sa: remove CountFQDNSetTimestamps (#7883)
This was superseded in #6220 by FQDNTimestampsForWindow and is no longer
called.
2024-12-16 12:24:01 -08:00
Aaron Gable 95e5f87f9e
Add feature flag to disable pending authz reuse (#7836)
Pending authz reuse is a nice-to-have feature because it allows us to
create fewer rows in the authz database table when creating new orders.
However, stats show that less than 2% of authorizations that we attach
to new orders are reused pending authzs. And as we move towards using a
more streamlined database schema to store our orders, authorizations,
and validation attempts, disabling pending authz reuse will greatly
simplify our database schema and code.

CPS Compliance Review: our CPS does not speak to whether or not we reuse
pending authorizations for new orders.
IN-10859 tracks enabling this flag in prod

Part of https://github.com/letsencrypt/boulder/issues/7715
2024-12-05 16:14:57 -08:00
Samantha Frank a8cdaf8989
ratelimit: Remove legacy registrations per IP implementation (#7760)
Part of #7671
2024-11-19 18:39:21 -05:00
Aaron Gable dad9e08606
Lay the groundwork for supporting IP identifiers (#7692)
Clean up how we handle identifiers throughout the Boulder codebase by
- moving the Identifier protobuf message definition from sa.proto to
core.proto;
- adding support for IP identifier to the "identifier" package;
- renaming the "identifier" package's exported names to be clearer; and
- ensuring we use the identifier package's helper functions everywhere
we can.

This will make future work to actually respect identifier types (such as
in Authorization and Order protobuf messages) simpler and easier to
review.

Part of https://github.com/letsencrypt/boulder/issues/7311
2024-08-30 11:40:38 -07:00
Aaron Gable 4482818a4f
Unify sa.GetAuthorizations2 and sa.GetValidAuthorizations2 (#7663)
These two methods were already nearly identical, their only meaningful
differences being the set of statuses they select for in their database
query, and the condition on which they prioritize entries in their
output.

Update them to have the exact same structure and logic. Most
meaningfully, update GetValidAuthorizations2 to hint the database to use
the same index which GetAuthorizations2 already hints.
2024-08-19 10:36:22 -07:00
Aaron Gable ced0117f6e
Remove deprecated sapb.Authorizations.Authz "map" (#7658)
This field was deprecated in
https://github.com/letsencrypt/boulder/pull/7646 and the last uses of it
were removed in https://github.com/letsencrypt/boulder/pull/7650.
2024-08-15 15:46:46 -07:00
Aaron Gable 46859a22d9
Use consistent naming for dnsName gRPC fields (#7654)
Find all gRPC fields which represent DNS Names -- sometimes called
"identifier", "hostname", "domain", "identifierValue", or other things
-- and unify their naming. This naming makes it very clear that these
values are strings which may be included in the SAN extension of a
certificate with type dnsName.

As we move towards issuing IP Address certificates, all of these fields
will need to be replaced by fields which carry both an identifier type
and value, not just a single name. This unified naming makes it very
clear which messages and methods need to be updated to support
non-dnsName identifiers.

Part of https://github.com/letsencrypt/boulder/issues/7647
2024-08-12 14:32:55 -07:00
Aaron Gable 28f09341b9
Simplify GetValidOrderAuthorizations2 (#7646)
Simplify SA.GetValidOrderAuthorizations2 so that it no longer conditions
the query on the status, expiry, or registration ID of the authorization
rows. This gives the query much better performance, because it no longer
tries to use an overly-large index, and fall back to large row-scans
when the query planner decides the index is too large.

While we're here, also improve the return type of
GetValidOrderAuthorizations2, so that instead of returning a map of
names to authorizations, it simply returns a list of authzs. This both
reduces the size of the gRPC message (once the old map is fully
removed), and improves its correctness because we cannot count on names
to be unique across multiple identifier types.

Finally, improve the RA code which calls SA.GetValidOrderAuthorizations2
to handle this improved return type, to make fewer assumptions about
identifier types, and to separate static authorization-checking from CAA
rechecking.

Fixes https://github.com/letsencrypt/boulder/issues/7645
2024-08-08 10:40:40 -07:00
Aaron Gable c9132baa37
Delete sa.GetPendingAuthorization2 (#7648)
This method's last caller was removed in
https://github.com/letsencrypt/boulder/pull/5862, when the ACMEv1
NewAuthorization code path was deleted. It has been dead code ever
since.
2024-08-07 09:33:37 -07:00
Aaron Gable 98a4bc01ea
Rename 'now' to 'validUntil' in GetAuthz requests (#7631)
The name "now" was always misleading, because we never set the value to
be the actual current time, we always set it to be some time in the
future to avoid returning authzs which expire in the very near future.
Changing the name to "validUntil" matches the current naming in
GetPendingAuthorizationRequest.
2024-07-25 10:52:34 -07:00
Jacob Hoffman-Andrews 3baea4356f
Revert "sa: truncate all timestamps to seconds (#7519)" (#7559)
This reverts commit 2b5b6239a4.

Following up on #7556, after we made a more systematic change to use
borp's TypeConverter, we no longer need to manually truncate timestamps.
2024-06-26 17:25:05 -07:00
Samantha 594cb1332f
SA: Implement schema and methods for (account, hostname) pausing (#7490)
Add the storage implementation for our new (account, hostname) pair
pausing feature.

- Add schema and model for for the new paused table
- Add SA service methods for interacting with the paused table

Part of #7406
Part of #7475
2024-06-17 10:18:10 -04:00
Jacob Hoffman-Andrews 2b5b6239a4
sa: truncate all timestamps to seconds (#7519)
As described in #7075, go-sql-driver/mysql v1.5.0 truncates timestamps
to microseconds, while v1.6.0 and above does not. That means upon
upgrading to v1.6.0, timestamps are written to the database with a
resolution of nanoseconds, and SELECT statements also use a resolution
of nanoseconds. We believe this is the cause of performance problems
we observed when upgrading to v1.6.0 and above.

To fix that, apply rounding in the application code. Rather than
just rounding to microseconds, round to seconds since that is the
resolution we care about.  Using seconds rather than microseconds
may also allow some of our indexes to grow more slowly over time.

Note: this omits truncating some timestamps in CRL shard calculations,
since truncating those resulted in test failures that I'll follow up
on separately.
2024-06-12 15:00:24 -07:00
Aaron Gable b92581d620
Better compile-time type checking for gRPC server implementations (#7504)
Replaced our embeds of foopb.UnimplementedFooServer with
foopb.UnsafeFooServer. Per the grpc-go docs this reduces the "forwards
compatibility" of our implementations, but that is only a concern for
codebases that are implementing gRPC interfaces maintained by third
parties, and which want to be able to update those third-party
dependencies without updating their own implementations in lockstep.
Because we update our protos and our implementations simultaneously, we
can remove this safety net to replace runtime type checking with
compile-time type checking.

However, that replacement is not enough, because we never pass our
implementation objects to a function which asserts that they match a
specific interface. So this PR also replaces our reflect-based unittests
with idiomatic interface assertions. I do not view this as a perfect
solution, as it relies on people implementing new gRPC servers to add
this line, but it is no worse than the status quo which relied on people
adding the "TestImplementation" test.

Fixes https://github.com/letsencrypt/boulder/issues/7497
2024-05-28 09:26:29 -07:00
Aaron Gable 89213f9214
Use generic types for gRPC stream implementations (#7501)
Update the version of protoc-gen-go-grpc that we use to generate Go gRPC
code from our proto files, and update the versions of other gRPC tools
and libraries that we use to match. Turn on the new
`use_generic_streams` code generation flag to change how
protoc-gen-go-grpc generates implementations of our streaming methods,
from creating a wholly independent implementation for every stream to
using shared generic implementations.

Take advantage of this code-sharing to remove our SA "wrapper" methods,
now that they have truly the same signature as the SARO methods which
they wrap. Also remove all references to the old-style stream names
(e.g. foopb.FooService_BarMethodClient) and replace them with the new
underlying generic names, for the sake of consistency. Finally, also
remove a few custom stream test mocks, replacing them with the generic
mocks.ServerStreamClient.

Note that this PR does not change the names in //mocks/sa.go, to avoid
conflicts with work happening in the pursuit of
https://github.com/letsencrypt/boulder/issues/7476. Note also that this
PR updates the version of protoc-gen-go-grpc that we use to a specific
commit. This is because, although a new release of grpc-go itself has
been cut, the codegen binary is a separate Go module with its own
releases, and it hasn't had a new release cut yet. Tracking for that is
in https://github.com/grpc/grpc-go/issues/7030.
2024-05-24 13:54:25 -07:00
Aaron Gable 6063430aed
admin: fail if any error is encountered during parallel processing (#7466)
While we don't want to halt the admin tool in the midst of its parallel
processing, we can keep track of whether it has encountered any errors
and raise one meta-error at the end of its execution. This will prevent
the top-level admin code from claiming that execution succeeded, and
ensure operators notice any previously-logged errors.

As part of this, fix the SA's GetLintPrecertificate wrapper to actually
call the SARO's GetLintPrecertificate, instead of incorrectly calling
the SARO's GetCertificate.

Fixes https://github.com/letsencrypt/boulder/issues/7460
2024-05-01 13:57:32 -07:00
Aaron Gable db77952e87
RA: fix GetSerialsByKey and GetSerialsByAccount (#7465)
Correctly explode the params slice with Go's "..." notation so that
gorp/go-sql-driver correctly receives each element of the params slice,
rather than receiving the slice as a whole. Also use the SA's clock,
rather than the DB's, to control which certs are selected -- in
deployments this wouldn't make a difference but in test those clocks can
be very different.

Add two unit tests to ensure this query does not regress, and create a
generic fake gRPC server stream for use in several SA tests including
the new ones.

Fixes https://github.com/letsencrypt/boulder/issues/7460
2024-05-01 13:57:25 -07:00
Aaron Gable e05d47a10a
Replace explicit int loops with range-over-int (#7434)
This adopts modern Go syntax to reduce the chance of off-by-one errors
and remove unnecessary loop variable declarations.

Fixes https://github.com/letsencrypt/boulder/issues/7227
2024-04-22 10:34:51 -07:00
Jacob Hoffman-Andrews 89b07f4543
sa: get order names from authorizations (#7435)
This removes the only place we query the requestedNames table, which
allows us to get rid of it in a subsequent PR (once this one is merged
and deployed).

Part of https://github.com/letsencrypt/boulder/issues/7432
2024-04-18 14:00:53 -07:00
Aaron Gable 5c97f994bb
SA: Remove unused PreviousCertificateExists method (#7439)
This method has had no callers since the removal of ACMEv1.
2024-04-18 12:29:35 -04:00
Phil Porada 5f616ccdb9
Upgrade go-jose from v2.6.1 to v.4.0.1 (#7345)
Upgrade from the old go-jose v2.6.1 to the newly minted go-jose v4.0.1. 
Cleans up old code now that `jose.ParseSigned` can take a list of
supported signature algorithms.

Fixes https://github.com/letsencrypt/boulder/issues/7390

---------

Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
2024-04-02 17:49:51 -04:00
Phil Porada 8556eaedca
SA: store and return certificate profile name (#7352)
Adds `certificateProfileName` to the `orders` database table. The
[maximum
length](https://github.com/letsencrypt/boulder/pull/7325/files#diff-a64a0af7cbf484da8e6d08d3eefdeef9314c5d9888233f0adcecd21b800102acR35)
of a profile name matches the `//issuance` package.

Adds a `MultipleCertificateProfiles` feature flag that, when enabled,
will store the certificate profile name from a `NewOrderRequest`. The
certificate profile name is allowed to be empty and the database will
treat that row as [NULL](https://mariadb.com/kb/en/null-values/). When
the SA retrieves this potentially NULL row, it will be cast as the
golang string zero value `""`.

SRE ticket IN-10145 has been filed to perform the database migration and
enable the new feature flag. The migration must be performed before
enabling the feature flag.

Part of https://github.com/letsencrypt/boulder/issues/7324
2024-03-20 13:08:31 -04:00
Aaron Gable 7f04092e72
Simplify streaming rows from the database (#7372)
Create a new method on the gorm rows object which runs a small closure
for every row retrieved from the database. Use this new method to remove
20 lines of boilerplate from five different SA methods and rocsp-tool.
2024-03-19 08:39:00 -07:00
Aaron Gable 6710ebe4cd
admin: use SA to get serials by account and by SPKI hash (#7369)
Add two new methods to the SA, GetSerialsByKey and GetSerialsByAccount,
which use the same query as the admin tool has previously used to get
serials matching a given SPKI hash or a given registration ID. These two
new gRPC methods read the database row-by-row and produce streams of
results to keep SA memory usage low.

Use these methods in the admin tool so it no longer needs a direct
database connection for these actions.

Part of https://github.com/letsencrypt/boulder/issues/7350
2024-03-11 13:25:59 -07:00
Samantha f10abd27eb
SA/ARI: Add method of tracking certificate replacement (#7284)
Part of #6732
Part of #7038
2024-02-08 14:19:29 -05:00
Aaron Gable c305acfd97
SA: Add GetLintPrecertificate gRPC method (#7274)
Add a new "GetLintPrecertificate" method to the SA's gRPC service. This
acts identically to the existing "GetCertificate", but returns the
linting precertificate created just prior to the actual precertificate
instead. This is useful for revocation, where we need to be able to act
on a serial even if the corresponding (pre)certificate was never issued
or never saved to the database.

Part of https://github.com/letsencrypt/boulder/issues/7135
2024-01-23 14:01:28 -08:00
Phil Porada 6925fad324
Finish migration from int64 timestamps to timestamppb (#7142)
This is a cleanup PR finishing the migration from int64 timestamps to
protobuf `*timestamppb.Timestamps` by removing all usage of the old
int64 fields. In the previous PR
https://github.com/letsencrypt/boulder/pull/7121 all fields were
switched to read from the protobuf timestamppb fields.

Adds a new case to `core.IsAnyNilOrZero` to check various properties of
a `*timestamppb.Timestamp` reducing the visual complexity for receivers.

Fixes https://github.com/letsencrypt/boulder/issues/7060
2023-11-27 13:37:31 -08:00
Phil Porada 279a4d539d
Read from durationpb instead of int64 durations (#7146)
Switch to reading grpc duration values from the new durationpb protofbuf
fields, completely ignoring the old int64 fields.

Part 2 of 3 for https://github.com/letsencrypt/boulder/issues/7097
2023-11-13 12:23:46 -05:00
Phil Porada b8b105453a
Rename protobuf duration fields to <fieldname>NS and populate new duration fields (#7115)
* Renames all of int64 as a time.Duration fields to `<fieldname>NS` to
indicate they are Unix nanoseconds.
* Adds new `google.protobuf.Duration` fields to each .proto file where
we previously had been using an int64 field to populate a time.Duration.
* Updates relevant gRPC messages.

Part 1 of 3 for https://github.com/letsencrypt/boulder/issues/7097
2023-10-26 10:46:03 -04:00
Phil Porada a5c2772004
Add and populate new protobuf Timestamp fields (#7070)
* Adds new `google.protobuf.Timestamp` fields to each .proto file where
we had been using `int64` fields as a timestamp.
* Updates relevant gRPC messages to populate the new
`google.protobuf.Timestamp` fields in addition to the old `int64`
timestamp fields.
* Added tests for each `<x>ToPB` and `PBto<x>` functions to ensure that
new fields passed into a gRPC message arrive as intended.
* Removed an unused error return from `PBToCert` and `PBToCertStatus`
and cleaned up each call site.

Built on-top of https://github.com/letsencrypt/boulder/pull/7069
Part 2 of 4 related to
https://github.com/letsencrypt/boulder/issues/7060
2023-10-11 12:12:12 -04:00
Aaron Gable bab048d221
SA: Add and use revokedCertificates table (#7095)
Add a new "revokedCertificates" table to the database schema. This table
is similar to the existing "certificateStatus" table in many ways, but
the idea is that it will only have rows added to it when certificates
are revoked, not when they're issued. Thus, it will grow many orders of
magnitude slower than the certificateStatus table does. Eventually, it
will replace that table entirely.

The one column that revokedCertificates adds is the new "ShardIdx"
column, which is the CRL shard in which the revoked certificate will
appear. This way we can assign certificates to CRL shards at the time
they are revoked, and guarantee that they will never move to a different
shard even if we change the number of shards we produce. This will
eventually allow us to put CRL URLs directly into our certificates,
replacing OCSP URLs.

Add new logic to the SA's RevokeCertificate and UpdateRevokedCertificate
methods to handle this new table. If these methods receive a request
which specifies a CRL shard (our CRL shards are 1-indexed, so shard 0
does not exist), then they will ensure that the new revocation status is
written into both the certificateStatus and revokedCertificates tables.
This logic will not function until the RA is updated to take advantage
of it, so it is not a risk for it to appear in Boulder before the new
table has been created.

Also add new logic to the SA's GetRevokedCertificates method. Similar to
the above, this reads from the new table if the ShardIdx field is
supplied in the request message. This code will not operate until the
crl-updater is updated to include this field. We will not perform this
update for a minimum of 100 days after this code is deployed, to ensure
that all unexpired revoked certificates are present in the
revokedCertificates table.

Part of https://github.com/letsencrypt/boulder/issues/7094
2023-10-02 10:21:14 -07:00