Permit all valid identifier types in `wfe.NewOrder` and `csr.VerifyCSR`.
Permit certs with just IP address identifiers to skip
`sa.addIssuedNames`.
Check that URI SANs are empty in `csr.VerifyCSR`, which was previously
missed.
Use a real (Let's Encrypt) IP address range in integration testing, to
let challtestsrv satisfy IP address challenges.
Fixes#8192
Depends on #8154
Remove the ability for the issuance package to include the AIA OCSP URI
and the Must Staple (more properly known as the tlsRequest) extension in
certificates. Deprecate the "OmitOCSP" and "AllowMustStaple" profile
config keys, as they no longer have any effect. Similarly deprecate the
"OCSPURL" issuer config key, as it is no longer included in
certificates.
Update the tests to always include to CRLDP extension instead, and
remove some OCSP- or Stapling-specific test cases.
Fixes https://github.com/letsencrypt/boulder/issues/8179
Add `pa.validIP` to test IP address validity & absence from IANA
reservations.
Modify `pa.WillingToIssue` and `pa.WellFormedIdentifiers` to support IP
address identifiers.
Add a map of allowed identifier types to the `pa` config.
Part of #8137
https://github.com/letsencrypt/boulder/pull/8150 updated our runtime
protobuf dependency from v1.34.1 to v1.36.5. This change does the same
for our build-time dependency, to keep them in sync.
Refactor `ca.issuePrecertificateInner` away from the old `NamesFromCSR`
logic, and to our `identifier` functions.
Add `identifier.ToValues` to provide slices of identifier values, split
up by type.
Fixes#8135
Part of #7311
Instead, simply return DER bytes from `issuePrecertificate`, and accept
regular parameters to `issueCertificateForPrecertificate` (instead of a
proto message).
Also, move the lookup of the certificate profile up to
`IssueCertificate`, and pass the selected `*certProfileWithId` to both
`issuePrecertificate` and `issueCertificateForPrecertificate`.
Also, change `issueCertificateForPrecertificate` to just return DER, not
a `*corepb.Certificate` (of which most fields were already being
ignored).
These methods are still preserved as-is for now, and still take proto
messages as arguments. But they are not exported as RPCs. Refactoring
the arguments will be a followup PR.
Part of #8039
Remove the backwards-compatible profile hashing code. It is no longer
necessary, since all deployed profile configs now set
IncludeCRLDistributionPoints to true and set the UnsplitIssuance flag to
true. Catch up the CA and crl-updater configs to match config-next and
what is actively deployed in prod.
Part of https://github.com/letsencrypt/boulder/issues/8039
Part of https://github.com/letsencrypt/boulder/issues/8059
Add a new RPC to the CA: `IssueCertificate` covers issuance of both the
precertificate and the final certificate. In between, it calls out to
the RA's new method `GetSCTs`.
The RA calls the new `CA.IssueCertificate` if the `UnsplitIssuance`
feature flag is true.
The RA had a metric that counted certificates by profile name and hash.
Since the RA doesn't receive a profile hash in the new flow, simply
record the total number of issuances.
Fixes https://github.com/letsencrypt/boulder/issues/7983
To achieve this without breaking hashes of deployed configs, create a
ProfileConfigNew containing the new field (and removing some deprecated
fields).
Move the CA's profile-hashing logic into the `issuance` package, and
gate it on the presence of IncludeCRLDistributionPoints. If that field
is false (the default), create an instance of the old `ProfileConfig`
with the appropriate values and encode/hash that instead.
Note: the IncludeCRLDistributionPoints field does not yet control any
behavior. That will be part of #7974.
Part of #7094
This makes the log events easier to parse, and makes it easier to
consistently use the correct fields from the issuance request.
Also, reduce the number of fields that are logged on error events.
Logging just the serial and the error in most cases should suffice to
cross-reference the error with the item that we attempted to sign.
One downside is that this increases the total log size (10kB above, vs
7kB from a similar production issuance) due in part to more repetition.
For example, both the "signing cert" and "signing cert success" log
lines include the full precert DER.
Note that our long-term plan for more structured logs is to have a
unique event id to join logs on, which can avoid this repetition. But
since we don't currently have convenient ways to do that join, some
duplication (as we currently have in the logs) seems reasonable.
The naming of our `precertificates` table (now used to store linting
certificates) is definitely confusing, so add some more comments in
various places explaining. See #6807.
Previously, we logged csr.DNSNames along with the precertificate
issuance events, but this was potentially misleading. For instance, if a
CSR contained only a CN and no SANs, this set would be empty.
Instead, log the uniquified, lowercased, sorted SANs that are actually
issued.
Also, emit precert=[] in issuePrecertificateInner, which is consistent
with the field name used when the final certificate is issued.
Note: this definitely underscores why it would be better to log these
things in a more structured way, for instance logging the
`issuance.IssuanceRequest` encoded as JSON, but I wanted to get these
quick fixes out of the way first.
Add a new SerialPrefixHex field to the CA's config, which takes a
two-character hexadecimal string to use as the serial prefix. This
matches the way that the OCSP Responder's acceptable serial prefixes are
configured, and is easier for human operators to configure than raw
integers.
At the same time, change the type of the CA's internal serial prefix
from `int` to `byte`, using the type system to enforce its 8-bit length.
Fixes#7213
This adds custom spans around one of the more important parts of the
issuance stack - actually signing the (pre)certificates. We only have
automatic tracing right now, so this is just a small step towards adding
more customization there.
One specific note: I didn't include the regID in the span attributes,
though it is in the nearby log lines. I think that's something we likely
want to handle holistically (eg, via baggage propagation) rather than
one-off in manual spans like this.
Replace all of Boulder's usage of the Go stdlib "math/rand" package with
the newer "math/rand/v2" package which first became available in go1.22.
This package has an improved API and faster performance across the
board.
See https://go.dev/blog/randv2 and https://go.dev/blog/chacha8rand for
details.
Move the two lint-configuration keys, LintConfig and IgnoreLints, from
the top-level CA.Issuance config stanza into each individual
CA.Issuance.CertProfiles stanza. This allows us to have
differently-configured lints for different profiles, to ensure that our
linting regime is as strict as possible.
Without this change, it would be necessary for us to ignore both the
"common name included" and the "no subject key id" lints at the
top-level, when in fact each of those warnings only triggers on one of
our two profiles.
Fixes https://github.com/letsencrypt/boulder/issues/7635
Add three new keys to the CA's ProfileConfig:
- OmitKeyEncipherment causes the keyEncipherment Key Usage to be omitted
from certificates with RSA public keys. We currently include it for
backwards compatibility with TLS 1.1 servers that don't support modern
cipher suites, but this KU is completely useless as of TLS 1.3.
- OmitClientAuth causes the tlsClientAuthentication Extended Key Usage
to be omitted from all certificates. We currently include it to support
any subscribers who may be relying on it, but Root Programs are moving
towards single-purpose hierarchies and its inclusion is being
discouraged.
- OmitSKID causes the Subject Key Identifier extension to be omitted
from all certificates. We currently include this extension because it is
recommended by RFC 5280, but it serves little to no practical purpose
and consumes a large number of bytes, so it is now NOT RECOMMENDED by
the Baseline Requirements.
Make substantive changes to issuer.requestValid and issuer.Prepare to
implement the desired behavior for each of these options. Make a very
slight change to ra.matchesCSR to generally allow for serverAuth-only
EKUs. Improve the unit tests of both the //ca and //issuance packages to
cover the new behavior.
Part of https://github.com/letsencrypt/boulder/issues/7610
One of our goals with profiles is to allow different profiles to have
different validity periods. While the profiles already had the ability
to enforce different maximum backdates and validities, the CA still had
separate global configuration for what the backdate and validity period
should actually be.
Move the computation of the notBefore and notAfter timestamps into the
issuance package, so that it can be based on the profile's configured
backdate and validity durations. Deprecate the global "backdate" and
"expiry" config fields, as they are no longer used. Finally, add more
validation for the profile's backdate and validity.
Part of https://github.com/letsencrypt/boulder/issues/7610
Add a new profile config key named "OmitCommonName" which, if set to
`true`, causes the issuance package to exclude the CN from the resulting
certificate even if the initiating IssuanceRequest specified one.
Deprecate the old "AllowCommonName" config key, so that it no longer has
any effect, rather than causing the issuance package to fully reject
IssuanceRequests containing a CN.
This allows for more graceful variation between profiles, since we know
that excluding the Common Name is always safe.
Part of https://github.com/letsencrypt/boulder/issues/7610
`ECDSAForAll` feature is now enabled by default (due to it not being
referenced in any issuance path) and as a result the `ECDSAAllowlist`
has been deleted.
Fixes https://github.com/letsencrypt/boulder/issues/7535
Change how goodkey.KeyPolicy keeps track of allowed RSA and ECDSA key
sizes, to make it slightly more flexible while still retaining the very
locked-down allowlist of only 6 acceptable key sizes (RSA 2048, 3076,
and 4092, and ECDSA P256, P384, and P521). Add a new constructor which
takes in a collection of allowed key sizes, so that users of the goodkey
package can customize which keys they accept. Rename the existing
constructor to make it clear that it uses hardcoded default values.
With these new constructors available, make all of the goodkey.KeyPolicy
member fields private, so that a KeyPolicy can only be built via these
constructors.
In the process of writing
https://github.com/letsencrypt/boulder/pull/7533 I discovered that the
method for detecting pkcs11.Error errors is broken: it attempts to
unwrap the returned error into a pointer-to-a-pointer type, which
doesn't work because only `pkcs11.Error` implements the Error interface,
while `*pkcs11.Error` does not.
Add a test which shows that the current noteSignError implementation is
broken. Then fix noteSignError and the two locations which duplicate
that code by removing the extra layer of indirection. And since the same
code exists in three locations, refactor how the caImpl, ocspImpl, and
crlImpl share metrics so that it only has to exist in one place.
A minimal reproduction case of this type of breakage can be seen here:
https://go.dev/play/p/qCLDQ1SFiWu
Add the "signatureCount" and "signErrorCount" metrics, which are already
incremented by the certificateAuthorityImpl and ocspImpl after all
signing operations, to the crlImpl.
Note that in the process of writing this PR I discovered that the method
for determining whether to increment the signErrorCount metric is
broken. Rather than diverge the crlImpl's version of that code from the
identical code in the other two files, I have duplicated the broken code
and will fix it in all three places in a follow-up.
Fixes https://github.com/letsencrypt/boulder/issues/7532
In https://github.com/letsencrypt/boulder/pull/7005 several safety
checks were added to the `ceremony` tool:
This change extracts the `RawSubject` to `RawIssuer` DER byte comparison
into the `//linter` package proper so that it can serve both `//ca` and
`//cmd/ceremony`.
Adds a helper function `verifyTBSCertificateDeterminism` to `//ca`
similar to an existing check in `//cmd/ceremony`. This code is not
shared because we want `//cmd/ceremony` to largely stand alone from
boulder proper. The helper performs a byte comparison on the
`RawTBSCertificate` DER bytes for a given linting certificate and leaf
certificate. The goal is to verify that `x509.CreateCertificate` was
deterministic and produced identical DER bytes after each signing
operation.
Fixes https://github.com/letsencrypt/boulder/issues/6965
Replaced our embeds of foopb.UnimplementedFooServer with
foopb.UnsafeFooServer. Per the grpc-go docs this reduces the "forwards
compatibility" of our implementations, but that is only a concern for
codebases that are implementing gRPC interfaces maintained by third
parties, and which want to be able to update those third-party
dependencies without updating their own implementations in lockstep.
Because we update our protos and our implementations simultaneously, we
can remove this safety net to replace runtime type checking with
compile-time type checking.
However, that replacement is not enough, because we never pass our
implementation objects to a function which asserts that they match a
specific interface. So this PR also replaces our reflect-based unittests
with idiomatic interface assertions. I do not view this as a perfect
solution, as it relies on people implementing new gRPC servers to add
this line, but it is no worse than the status quo which relied on people
adding the "TestImplementation" test.
Fixes https://github.com/letsencrypt/boulder/issues/7497
Update the version of protoc-gen-go-grpc that we use to generate Go gRPC
code from our proto files, and update the versions of other gRPC tools
and libraries that we use to match. Turn on the new
`use_generic_streams` code generation flag to change how
protoc-gen-go-grpc generates implementations of our streaming methods,
from creating a wholly independent implementation for every stream to
using shared generic implementations.
Take advantage of this code-sharing to remove our SA "wrapper" methods,
now that they have truly the same signature as the SARO methods which
they wrap. Also remove all references to the old-style stream names
(e.g. foopb.FooService_BarMethodClient) and replace them with the new
underlying generic names, for the sake of consistency. Finally, also
remove a few custom stream test mocks, replacing them with the generic
mocks.ServerStreamClient.
Note that this PR does not change the names in //mocks/sa.go, to avoid
conflicts with work happening in the pursuit of
https://github.com/letsencrypt/boulder/issues/7476. Note also that this
PR updates the version of protoc-gen-go-grpc that we use to a specific
commit. This is because, although a new release of grpc-go itself has
been cut, the codegen binary is a separate Go module with its own
releases, and it hasn't had a new release cut yet. Tracking for that is
in https://github.com/grpc/grpc-go/issues/7030.
The summary here is:
- Move test/cert-ceremonies to test/certs
- Move .hierarchy (generated by the above) to test/certs/webpki
- Remove our mapping of .hierarchy to /hierarchy inside docker
- Move test/grpc-creds to test/certs/ipki
- Unify the generation of both test/certs/webpki and test/certs/ipki
into a single script at test/certs/generate.sh
- Make that script the entrypoint of a new docker compose service
- Have t.sh and tn.sh invoke that service to ensure keys and certs are
created before tests run
No production changes are necessary, the config changes here are just
for testing purposes.
Part of https://github.com/letsencrypt/boulder/issues/7476
Also update the CA and RA doccomments to link to it and describe the
roles of key functions a little better.
Remove outdated reference to generating OCSP at issuance time.
Remove the CA's global "crldpBase" config item, and the code which used
it to produce a IDP URI in our CRLs if it was configured.
This config item has been replaced by per-issuer crlURLBase configs
instead, because we have switched our CRL URL format from
"commonURL/issuerID/shard.crl" to "issuerURL/shard.crl" in anticipation
of including these URLs directly in our end-entity certs.
IN-10046 tracked the corresponding change in prod
Add a new "LintConfig" item to the CA's config, which can point to a
zlint configuration toml file. This allows lints to be configured, e.g.
to control the number of rounds of factorization performed by the Fermat
factorization lint.
Leverage this new config to create a new custom zlint which calls out to
a configured pkilint API endpoint. In config-next integration tests,
configure the lint to point at a new pkilint docker container.
This approach has three nice forward-looking features: we now have the
ability to configure any of our lints; it's easy to expand this
mechanism to lint CRLs when the pkilint API has support for that; and
it's easy to enable this new lint if we decide to stand up a pkilint
container in our production environment.
No production configuration changes are necessary at this time.
Fixes https://github.com/letsencrypt/boulder/issues/7430
[Previously](https://github.com/letsencrypt/boulder/pull/7438) I had
made a change to run all CA tests in parallel, which was great, but I
failed to account for several table driven tests. By rebinding the
subtest's iterator to the lexical scope, each subtest can now run in
parallel.
Partial revert of https://github.com/letsencrypt/boulder/pull/7434
The purpose of this loop is to chunk the CRL into thousand-byte pieces.
To that end, it iterated across the bytes of the CRL in steps of size
1000. When this loop was replaced with a range loop, that step size was
not preserved.
Our tests do not usually produce any CRLs that exceed 1k bytes, so the
test CRLs fit within a single chunk, and this bug was not detected.
A search of the diff in the previous change does not show any other
instances where a step size other than 1 was being used.
* Adds `CertProfileName` to the CAs `capb.IssuePrecertificateResponse`
so the RA can receive the CAs configured default profile name for audit
logging/metrics. This is useful for when the RA sends an empty string as
the profile name to the CA, but we want to know exactly what the profile
name chosen by the CA was, rather than just relying on comparing hashes
between CA and RA audit logs.
* Adds the profile name and hash to RA audit logs emitted after a
successful issuance.
* Adds new labels to the existing `new_certificates` metric exported by
the RA.
```
# HELP new_certificates A counter of new certificates including the certificate profile name and hexadecimal certificate profile hash
# TYPE new_certificates counter
new_certificates{profileHash="de4c8c8866ed46b1d4af0d79e6b7ecf2d1ea625e26adcbbd3979ececd8fbd05a",profileName="defaultBoulderCertificateProfile"} 2
```
Fixes https://github.com/letsencrypt/boulder/issues/7421
Replace the CA's "useForRSA" and "useForECDSA" config keys with a single
"active" boolean. When the CA starts up, all active RSA issuers will be
used to issue precerts with RSA pubkeys, and all ECDSA issuers will be
used to issue precerts with ECDSA pubkeys (if the ECDSAForAll flag is
true; otherwise just those that are on the allow-list). All "inactive"
issuers can still issue OCSP responses, CRLs, and (notably) final
certificates.
Instead of using the "useForRSA" and "useForECDSA" flags, plus implicit
config ordering, to determine which issuer to use to handle a given
issuance, simply use the issuer's public key algorithm to determine
which issuances it should be handling. All implicit ordering
considerations are removed, because the "active" certificates now just
form a pool that is sampled from randomly.
To facilitate this, update some unit and integration tests to be more
flexible and try multiple potential issuing intermediates, particularly
when constructing OCSP requests.
For this change to be safe to deploy with no user-visible behavior
changes, the CA configs must contain:
- Exactly one RSA-keyed intermediate with "useForRSALeaves" set to true;
and
- Exactly one ECDSA-keyed intermediate with "useForECDSALeaves" set to
true.
If the configs contain more than one intermediate meeting one of the
bullets above, then randomized issuance will begin immediately.
Fixes https://github.com/letsencrypt/boulder/issues/7291
Fixes https://github.com/letsencrypt/boulder/issues/7290
The CA tests don't share any state and create their own individual CA
implementations. We can safely run these tests in parallel within the CA
package to shave at least a second off of unit test runs at the expense
of additional CPU and memory usage.
When https://github.com/letsencrypt/boulder/pull/7325 was deployed to
staging, the CA threw "Incomplete cert for precertificate request"
errors. Even though the RA was forwarding the CertProfileHash in all
IssueCertificateForPrecertificate requests to updated CA instances, it
can't do that if the IssuePrecertificate request was handled by a
non-updated CA instance that didn't yet know to return the hash.
This PR should be landed, tagged with a release, and then immediately
reverted for inclusion in the next release.
Part of https://github.com/letsencrypt/boulder/issues/6966
This change introduces a new config key `certProfiles` which contains a
map of `profiles`. Only one of `profile` or `certProfiles` should be
used, because configuring both will result in the CA erroring and
shutting down. Further, the singular `profile` is now
[deprecated](https://github.com/letsencrypt/boulder/issues/7414).
The CA pre-computes several maps at startup;
* A human-readable name to a `*issuance.Profile` which is referred to as
"name".
* A SHA-256 sum over the entire contents of the given profile to the
`*issuance.Profile`. We'll refer to this as "hash".
Internally, CA methods no longer pass an `*issuance.Profile`, instead
they pass a structure containing maps of certificate profile
identifiers. To determine the default profile used by the CA, a new
config field `defaultCertificateProfileName` has been added to the
Issuance struct. Absence of `defaultCertificateProfileName` will cause
the CA to use the default value of `defaultBoulderCertificateProfile`
such as for the the deprecated `profile`. The key for each given
certificate profile will be used as the "name". Duplicate names or
hashes will cause the CA to error during initialization and shutdown.
When the RA calls `ra.CA.IssuePrecertificate`, it will pass an arbitrary
certificate profile name to the CA triggering the CA to lookup if the
name exists in its internal mapping. The RA maintains no state or
knowledge of configured certificate profiles and relies on the CA to
provide this information. If the name exists in the CA's map, it will
return the hash along with the precertificate bytes in a
`capb.IssuePrecertificateResponse`. The RA will then call
`ra.CA.IssueCertificateForPrecertificate` with that same hash. The CA
will lookup the hash to determine if it exists in its map, and if so
will continue on with certificate issuance.
Precertificate and certificate issuance audit logs will now include the
certificate profile name and hex representation of the hash that they
were issued with.
Fixes https://github.com/letsencrypt/boulder/issues/6966
There are no required config or SQL changes.
Adds a new `certProfileName` message to the
`CA.IssueCertificateRequest`. This field contains a human-readable
"name" set by the
[WFE2](https://github.com/letsencrypt/boulder/issues/7332), and in turn
the RA. At the time of precertificate issuance, the receiving CA will
determine if it is capable of fulfilling the `ra.CA.IssuePrecertificate`
request for the given `certProfileName`. If the name is found in the
CA's map, the CA will return a `capb.IssuePrecertificateResponse`
message with a populated `certProfileHash` field back to the RA. When
that RA calls `ra.CA.IssueCertificateForPrecertificate`, it will send
that same `certProfileHash` message to a CA which must ensure it
contains a certificate profile matching the provided hash. If the hash
in found in the CA's map a final certificate issuance attempt will
proceed. This is done to prevent certificate profile changes in the
duration between requests from causing a mismatch between precerticate
and final certificate.
Part of https://github.com/letsencrypt/boulder/issues/7309
Part of https://github.com/letsencrypt/boulder/issues/6966
James Renken
Aaron Gable
Jacob Hoffman-Andrews
Matthew McPherrin
Phil Porada