- Add `Perspective` and `RIR` fields to the remote-va configuration
- Configure RVA ValidationAuthorityImpl instances with the contents of
the JSON configuration
- Configure VA ValidationAuthorityImpl instances with the constant
`va.PrimaryPerspective`
- Log `Perspective` for non-Primary Perspectives, per the MPIC
requirements in section 5.4.1 (2) vii of the BRs. Also log the RIR for
posterity.
- Introduce `ValidationResult` RPC fields `Perspective` and `Rir`, which
are not currently used but will be required for corroboration in #7616
Fixes https://github.com/letsencrypt/boulder/issues/7613
Part of https://github.com/letsencrypt/boulder/issues/7615
Part of https://github.com/letsencrypt/boulder/issues/7616
Add a new SerialPrefixHex field to the CA's config, which takes a
two-character hexadecimal string to use as the serial prefix. This
matches the way that the OCSP Responder's acceptable serial prefixes are
configured, and is easier for human operators to configure than raw
integers.
At the same time, change the type of the CA's internal serial prefix
from `int` to `byte`, using the type system to enforce its 8-bit length.
Fixes#7213
OpenTelemetry has "semantic conventions" which are versioned
independently of the software package, as it describes the semantics of
the resources being produced. Previously, we'd combined
`resource.Default()` using the `Merge` function with our own resources.
Merge, however, doesn't handle merging resources with different semantic
conventions. This means that every dependabot PR that bumps otel will
break when the `resources.Default` has a new version.
That doesn't seem worth it for the default resources, so just provide
our own resources which have everything we care about. I've added the
PID which we didn't have before but will be interesting. We will lose
the SDK's version, but I don't think that matters.
For more discussion on this topic, see
https://github.com/open-telemetry/opentelemetry-go/issues/3769
Clean up how we handle identifiers throughout the Boulder codebase by
- moving the Identifier protobuf message definition from sa.proto to
core.proto;
- adding support for IP identifier to the "identifier" package;
- renaming the "identifier" package's exported names to be clearer; and
- ensuring we use the identifier package's helper functions everywhere
we can.
This will make future work to actually respect identifier types (such as
in Authorization and Order protobuf messages) simpler and easier to
review.
Part of https://github.com/letsencrypt/boulder/issues/7311
Have our gRPC server interceptor check for excessive clock skew between
its own clock and gRPC client clocks. Do this by taking advantage of the
client request timestamp that most clients already supply for the
purpose of measuring cross-service latency. If the included timestamp is
more than 10 minutes from the gRPC server's local time, immediately
error out.
To keep the integration tests -- which heavily rely on clock
manipulation -- working, use build tags to disable this behavior during
integration testing.
Fixes https://github.com/letsencrypt/boulder/issues/7684
Begin testing on go1.23. To facilitate this, also update /x/net,
golangci-lint, staticcheck, and pebble-challtestsrv to versions which
support go1.23. As a result of these updates, also fix a handful of new
lint findings, mostly regarding passing non-static (i.e. potentially
user-controlled) format strings into Sprintf-style functions.
Additionally, delete one VA unittest that was duplicating the checks
performed by a different VA unittest, but with a context timeout bug
that caused it to break when go1.23 subtly changed DialContext behavior.
Implements tooling in `admin` that allows an operator to
administratively pause account/identifier pairs and unpause
whole accounts. This functionality mirrors the self-service
capabilities of the SFE, so that we can administratively intervene
in the pausing and unpausing process.
The new `pause-identifier` subcommand accepts a single form
of input, specified by the `-batch-file` flag. This expects a CSV
where each row is an accountID, identifierType, identifierValue
triple.
The new `unpause-account` subcommand accepts either a single
account ID with the `-account` flag, or a text file containing a list
of account IDs with the `-batch-file` flag.
Relates to https://github.com/letsencrypt/boulder/issues/7406
Fixes https://github.com/letsencrypt/boulder/issues/7618
Find all gRPC fields which represent DNS Names -- sometimes called
"identifier", "hostname", "domain", "identifierValue", or other things
-- and unify their naming. This naming makes it very clear that these
values are strings which may be included in the SAN extension of a
certificate with type dnsName.
As we move towards issuing IP Address certificates, all of these fields
will need to be replaced by fields which carry both an identifier type
and value, not just a single name. This unified naming makes it very
clear which messages and methods need to be updated to support
non-dnsName identifiers.
Part of https://github.com/letsencrypt/boulder/issues/7647
Replace all of Boulder's usage of the Go stdlib "math/rand" package with
the newer "math/rand/v2" package which first became available in go1.22.
This package has an improved API and faster performance across the
board.
See https://go.dev/blog/randv2 and https://go.dev/blog/chacha8rand for
details.
Move the two lint-configuration keys, LintConfig and IgnoreLints, from
the top-level CA.Issuance config stanza into each individual
CA.Issuance.CertProfiles stanza. This allows us to have
differently-configured lints for different profiles, to ensure that our
linting regime is as strict as possible.
Without this change, it would be necessary for us to ignore both the
"common name included" and the "no subject key id" lints at the
top-level, when in fact each of those warnings only triggers on one of
our two profiles.
Fixes https://github.com/letsencrypt/boulder/issues/7635
One of our goals with profiles is to allow different profiles to have
different validity periods. While the profiles already had the ability
to enforce different maximum backdates and validities, the CA still had
separate global configuration for what the backdate and validity period
should actually be.
Move the computation of the notBefore and notAfter timestamps into the
issuance package, so that it can be based on the profile's configured
backdate and validity durations. Deprecate the global "backdate" and
"expiry" config fields, as they are no longer used. Finally, add more
validation for the profile's backdate and validity.
Part of https://github.com/letsencrypt/boulder/issues/7610
Change the way profiles are configured at the WFE to allow them to be
accompanied by descriptive strings. Augment the construction of the
directory resource's "meta" sub-object to include these profile names
and descriptions.
This config swap is safe, since no Boulder WFE instance is configured
with `CertificateProfileNames` yet.
Fixes https://github.com/letsencrypt/boulder/issues/7602
This change guarantees compliance with CA/BF Ballot SC-073 "Compromised
and Weak Keys", which requires that at least 100 rounds of Fermat
Factorization be attempted:
> Section 6.1.1.3 Subscriber Key Pair Generation
> The CA SHALL reject a certificate request if... The Public Key
corresponds to an industry-demonstrated weak Private Key. For requests
submitted on or after November 15, 2024,... In the case of Close Primes
vulnerability (https://fermatattack.secvuln.info/), the CA SHALL reject
weak keys which can be factored within 100 rounds using Fermat’s
factorization method.
We choose 110 rounds to ensure a margin above and beyond the requirements.
Fixes https://github.com/letsencrypt/boulder/issues/7558
Adds a new boulder component named `sfe` aka the Self-service FrontEnd
which is dedicated to non-ACME related Subscriber functions. This change
implements one such function which is a web interface and handlers for
account unpausing.
When paused, an ACME client receives a log line URL with a JWT parameter
from the WFE. For the observant Subscriber, manually clicking the link
opens their web browser and displays a page with a pre-filled HTML form.
Upon clicking the form button, the SFE sends an HTTP POST back to itself
and either validates the JWT and issues an RA gRPC request to unpause
the account, or returns an HTML error page.
The SFE and WFE should share a 32 byte seed value e.g. the output of
`openssl rand -hex 16` which will be used as a go-jose symmetric signer
using the HS256 algorithm. The SFE will check various [RFC
7519](https://datatracker.ietf.org/doc/html/rfc7519) claims on the JWT
such as the `iss`, `aud`, `nbf`, `exp`, `iat`, and a custom `apiVersion`
claim.
The SFE should not yet be relied upon or deployed to staging/production
environments. It is very much a work in progress, but this change is big
enough as-is.
Related to https://github.com/letsencrypt/boulder/issues/7406
Part of https://github.com/letsencrypt/boulder/issues/7499
This reverts commit 2b5b6239a4.
Following up on #7556, after we made a more systematic change to use
borp's TypeConverter, we no longer need to manually truncate timestamps.
`ECDSAForAll` feature is now enabled by default (due to it not being
referenced in any issuance path) and as a result the `ECDSAAllowlist`
has been deleted.
Fixes https://github.com/letsencrypt/boulder/issues/7535
Change how goodkey.KeyPolicy keeps track of allowed RSA and ECDSA key
sizes, to make it slightly more flexible while still retaining the very
locked-down allowlist of only 6 acceptable key sizes (RSA 2048, 3076,
and 4092, and ECDSA P256, P384, and P521). Add a new constructor which
takes in a collection of allowed key sizes, so that users of the goodkey
package can customize which keys they accept. Rename the existing
constructor to make it clear that it uses hardcoded default values.
With these new constructors available, make all of the goodkey.KeyPolicy
member fields private, so that a KeyPolicy can only be built via these
constructors.
As described in #7075, go-sql-driver/mysql v1.5.0 truncates timestamps
to microseconds, while v1.6.0 and above does not. That means upon
upgrading to v1.6.0, timestamps are written to the database with a
resolution of nanoseconds, and SELECT statements also use a resolution
of nanoseconds. We believe this is the cause of performance problems
we observed when upgrading to v1.6.0 and above.
To fix that, apply rounding in the application code. Rather than
just rounding to microseconds, round to seconds since that is the
resolution we care about. Using seconds rather than microseconds
may also allow some of our indexes to grow more slowly over time.
Note: this omits truncating some timestamps in CRL shard calculations,
since truncating those resulted in test failures that I'll follow up
on separately.
In the process of writing
https://github.com/letsencrypt/boulder/pull/7533 I discovered that the
method for detecting pkcs11.Error errors is broken: it attempts to
unwrap the returned error into a pointer-to-a-pointer type, which
doesn't work because only `pkcs11.Error` implements the Error interface,
while `*pkcs11.Error` does not.
Add a test which shows that the current noteSignError implementation is
broken. Then fix noteSignError and the two locations which duplicate
that code by removing the extra layer of indirection. And since the same
code exists in three locations, refactor how the caImpl, ocspImpl, and
crlImpl share metrics so that it only has to exist in one place.
A minimal reproduction case of this type of breakage can be seen here:
https://go.dev/play/p/qCLDQ1SFiWu
Add the "signatureCount" and "signErrorCount" metrics, which are already
incremented by the certificateAuthorityImpl and ocspImpl after all
signing operations, to the crlImpl.
Note that in the process of writing this PR I discovered that the method
for determining whether to increment the signErrorCount metric is
broken. Rather than diverge the crlImpl's version of that code from the
identical code in the other two files, I have duplicated the broken code
and will fix it in all three places in a follow-up.
Fixes https://github.com/letsencrypt/boulder/issues/7532
In https://github.com/letsencrypt/boulder/pull/7005 several safety
checks were added to the `ceremony` tool:
This change extracts the `RawSubject` to `RawIssuer` DER byte comparison
into the `//linter` package proper so that it can serve both `//ca` and
`//cmd/ceremony`.
Adds a helper function `verifyTBSCertificateDeterminism` to `//ca`
similar to an existing check in `//cmd/ceremony`. This code is not
shared because we want `//cmd/ceremony` to largely stand alone from
boulder proper. The helper performs a byte comparison on the
`RawTBSCertificate` DER bytes for a given linting certificate and leaf
certificate. The goal is to verify that `x509.CreateCertificate` was
deterministic and produced identical DER bytes after each signing
operation.
Fixes https://github.com/letsencrypt/boulder/issues/6965
Directly update:
- go.opentelemetry.io/otel/* from v1.26.0 to v1.27.0
- go.opentelemetry.io/contrib/* from v0.51.0 to v0.52.0
Indirectly update:
- google.golang.org/protobuf from v1.33.0 to v1.34.0
This update breaks some of our existing otel grpc interceptors, but in
return allows us to use the newer grpc StatsHandler mechanism, while
still filtering out health-check requests.
Fixes https://github.com/letsencrypt/boulder/issues/7235
Replaced our embeds of foopb.UnimplementedFooServer with
foopb.UnsafeFooServer. Per the grpc-go docs this reduces the "forwards
compatibility" of our implementations, but that is only a concern for
codebases that are implementing gRPC interfaces maintained by third
parties, and which want to be able to update those third-party
dependencies without updating their own implementations in lockstep.
Because we update our protos and our implementations simultaneously, we
can remove this safety net to replace runtime type checking with
compile-time type checking.
However, that replacement is not enough, because we never pass our
implementation objects to a function which asserts that they match a
specific interface. So this PR also replaces our reflect-based unittests
with idiomatic interface assertions. I do not view this as a perfect
solution, as it relies on people implementing new gRPC servers to add
this line, but it is no worse than the status quo which relied on people
adding the "TestImplementation" test.
Fixes https://github.com/letsencrypt/boulder/issues/7497
Update the version of protoc-gen-go-grpc that we use to generate Go gRPC
code from our proto files, and update the versions of other gRPC tools
and libraries that we use to match. Turn on the new
`use_generic_streams` code generation flag to change how
protoc-gen-go-grpc generates implementations of our streaming methods,
from creating a wholly independent implementation for every stream to
using shared generic implementations.
Take advantage of this code-sharing to remove our SA "wrapper" methods,
now that they have truly the same signature as the SARO methods which
they wrap. Also remove all references to the old-style stream names
(e.g. foopb.FooService_BarMethodClient) and replace them with the new
underlying generic names, for the sake of consistency. Finally, also
remove a few custom stream test mocks, replacing them with the generic
mocks.ServerStreamClient.
Note that this PR does not change the names in //mocks/sa.go, to avoid
conflicts with work happening in the pursuit of
https://github.com/letsencrypt/boulder/issues/7476. Note also that this
PR updates the version of protoc-gen-go-grpc that we use to a specific
commit. This is because, although a new release of grpc-go itself has
been cut, the codegen binary is a separate Go module with its own
releases, and it hasn't had a new release cut yet. Tracking for that is
in https://github.com/grpc/grpc-go/issues/7030.
Give akamai-purger a new "Throughput.TotalInstances" config value, to
inform it how many instances of itself are competing for akamai rate
limit quote. Combine the `useOptimizedDefaults` and `validate` functions
into a single `optimizeAndValidate` function which sets default values
according to the number of active instances, and confirms that the
results still fall within the rate limits.
Fixes https://github.com/letsencrypt/boulder/issues/7487
Replace "mocks.StorageAuthority" with "sapb.StorageAuthorityClient" in
our test mocks. The improves them by removing implementations of the
methods the tests don't actually need, instead of inheriting lots of
extraneous methods from the huge and cumbersome mocks.StorageAuthority.
This reduces our usage of mocks.StorageAuthority to only the WFE tests
(which create one in the frequently-used setup() function), which will
make refactoring those mocks in the pursuit of
https://github.com/letsencrypt/boulder/issues/7476 much easier.
Part of https://github.com/letsencrypt/boulder/issues/7476
Remove the redis-tls, wfe-tls, and mail-test-srv keys which were
generated by minica and then checked in to the repo. All three are
replaced by the dynamically-generated ipki directory.
Part of https://github.com/letsencrypt/boulder/issues/7476
The summary here is:
- Move test/cert-ceremonies to test/certs
- Move .hierarchy (generated by the above) to test/certs/webpki
- Remove our mapping of .hierarchy to /hierarchy inside docker
- Move test/grpc-creds to test/certs/ipki
- Unify the generation of both test/certs/webpki and test/certs/ipki
into a single script at test/certs/generate.sh
- Make that script the entrypoint of a new docker compose service
- Have t.sh and tn.sh invoke that service to ensure keys and certs are
created before tests run
No production changes are necessary, the config changes here are just
for testing purposes.
Part of https://github.com/letsencrypt/boulder/issues/7476
* Adds a `VerifyGRPCClientCertIfGiven` boolean to the `remoteva` config
that cause the RVA server to use the less strict
`tls.VerifyClientCertIfGiven` for use with an Amazon Web Services
Application Load Balancer (ALB) between the `boulder-va` and `remoteva`.
See https://github.com/letsencrypt/boulder/issues/7386.
Part of https://github.com/letsencrypt/boulder/issues/5294
---------
Co-authored-by: Samantha <hello@entropy.cat>
* Adds a new `remoteva` binary that takes a distinct configuration from
the existing `boulder-va`
* Removed the `boulder-remoteva` name registration from `boulder-va`.
* Existing users of `boulder-remoteva` must either
1. laterally migrate to `boulder-va` which uses that same config, or
2. switch to using `remoteva` with a new config.
Part of https://github.com/letsencrypt/boulder/issues/5294
Remove the CA's global "crldpBase" config item, and the code which used
it to produce a IDP URI in our CRLs if it was configured.
This config item has been replaced by per-issuer crlURLBase configs
instead, because we have switched our CRL URL format from
"commonURL/issuerID/shard.crl" to "issuerURL/shard.crl" in anticipation
of including these URLs directly in our end-entity certs.
IN-10046 tracked the corresponding change in prod
While we don't want to halt the admin tool in the midst of its parallel
processing, we can keep track of whether it has encountered any errors
and raise one meta-error at the end of its execution. This will prevent
the top-level admin code from claiming that execution succeeded, and
ensure operators notice any previously-logged errors.
As part of this, fix the SA's GetLintPrecertificate wrapper to actually
call the SARO's GetLintPrecertificate, instead of incorrectly calling
the SARO's GetCertificate.
Fixes https://github.com/letsencrypt/boulder/issues/7460
Set the minimum TLS version used for communication with gRPC, Redis,
and Unbound to 1.3. Also remove deprecated `SecurityVersion` setting in
`clientTransportCredentials` and `serverTransportCredentials`, as
grpc-go now uses the settings provided by the `tls.Config`.
The http-01 and tls-alpn-01challenges are not affected:
- 939ac1be8f/va/http.go (L140-L157)
- 939ac1be8f/va/tlsalpn.go (L213-L217)
Add a new "LintConfig" item to the CA's config, which can point to a
zlint configuration toml file. This allows lints to be configured, e.g.
to control the number of rounds of factorization performed by the Fermat
factorization lint.
Leverage this new config to create a new custom zlint which calls out to
a configured pkilint API endpoint. In config-next integration tests,
configure the lint to point at a new pkilint docker container.
This approach has three nice forward-looking features: we now have the
ability to configure any of our lints; it's easy to expand this
mechanism to lint CRLs when the pkilint API has support for that; and
it's easy to enable this new lint if we decide to stand up a pkilint
container in our production environment.
No production configuration changes are necessary at this time.
Fixes https://github.com/letsencrypt/boulder/issues/7430
Previously, cert-checker largely ignored its own config file and always
initialized its logger to log to syslog. This makes cert-checker
initialize its logger in the same way as all other boulder components.
Add a new "-cert-file" input mode to both `admin revoke-cert` and `admin
block-key` which operates on the serial or pubkey found in the
PEM-encoded certificate in the supplied file.
Fixes https://github.com/letsencrypt/boulder/issues/7267
We first introduced caa-log-checker as a remediation item in the wake of
https://bugzilla.mozilla.org/show_bug.cgi?id=1619047. Since that time,
we have upgraded to go1.22, which completely remoes the class of bug
which led to that incident (https://tip.golang.org/doc/go1.22#language).
Throughout its life, caa-log-checker was an operational burden, and was
at best a post-hoc check to detect issues after they had already
occurred. Therefore, we no longer run it in our production environment,
and it can be removed from the Boulder source.
This change introduces a new config key `certProfiles` which contains a
map of `profiles`. Only one of `profile` or `certProfiles` should be
used, because configuring both will result in the CA erroring and
shutting down. Further, the singular `profile` is now
[deprecated](https://github.com/letsencrypt/boulder/issues/7414).
The CA pre-computes several maps at startup;
* A human-readable name to a `*issuance.Profile` which is referred to as
"name".
* A SHA-256 sum over the entire contents of the given profile to the
`*issuance.Profile`. We'll refer to this as "hash".
Internally, CA methods no longer pass an `*issuance.Profile`, instead
they pass a structure containing maps of certificate profile
identifiers. To determine the default profile used by the CA, a new
config field `defaultCertificateProfileName` has been added to the
Issuance struct. Absence of `defaultCertificateProfileName` will cause
the CA to use the default value of `defaultBoulderCertificateProfile`
such as for the the deprecated `profile`. The key for each given
certificate profile will be used as the "name". Duplicate names or
hashes will cause the CA to error during initialization and shutdown.
When the RA calls `ra.CA.IssuePrecertificate`, it will pass an arbitrary
certificate profile name to the CA triggering the CA to lookup if the
name exists in its internal mapping. The RA maintains no state or
knowledge of configured certificate profiles and relies on the CA to
provide this information. If the name exists in the CA's map, it will
return the hash along with the precertificate bytes in a
`capb.IssuePrecertificateResponse`. The RA will then call
`ra.CA.IssueCertificateForPrecertificate` with that same hash. The CA
will lookup the hash to determine if it exists in its map, and if so
will continue on with certificate issuance.
Precertificate and certificate issuance audit logs will now include the
certificate profile name and hex representation of the hash that they
were issued with.
Fixes https://github.com/letsencrypt/boulder/issues/6966
There are no required config or SQL changes.
Give the admin tool a slightly more structured subcommand system:
subcommands are now represented by structs (rather than just methods on
the admin object) which satisfy a particular interface. This interface
separates flag declaration from execution, so that we can have greater
flexibility with regard to command line parsing. This allows the two
top-level flags (-config and -dry-run) to appear anywhere in the command
line (not only before the subcommand name), and allows the -h/-help flag
to print usage information even when other critical flags (like -config)
are missing.
Fixes https://github.com/letsencrypt/boulder/issues/7393
Fixes https://github.com/letsencrypt/boulder/issues/7359
Add a new input method flag to `admin block-key` which processes a file
containing one hexadecimal-encoded SPKI hash on each line. To facilitate
this, restructure the block-key subcommand's execution to more closely
resemble the revoke-cert subcommand, with a parallelism flag and the
ability to run many workers at the same time.
Part of https://github.com/letsencrypt/boulder/issues/7267
- Parse and validate the `profile` field in `newOrder` requests.
- Pass the `profile` field from `newOrder` calls to the resulting
`RA.NewOrder` call.
- When the client requests a specific profile, ensure that the profile
field is populated in the order returned.
Fixes#7332
Part of #7309
Create a new method on the gorm rows object which runs a small closure
for every row retrieved from the database. Use this new method to remove
20 lines of boilerplate from five different SA methods and rocsp-tool.
When a serial is passed in, all extraneous characters that are not
alphanumeric are stripped. The result is checked against
[[core.ValidSerial](9b05c38eb3/core/util.go (L170))]
to ensure that it is a valid hex of 32 or 36 characters and then passed
to the rest of boulder. If the stripped serial is not a valid serial, an
error is thrown and revocation does not proceed.
Use two existing SA methods, KeyBlocked and GetSerialsByKey, to replace
the direct database access previously used by the blockSPKIHash method.
This is less efficient than before -- it now streams the whole set of
affected serials rather than just counting them -- but doing so prevents
us from needing an additional SA method just for counting.
Also update the default mock StorageAuthority and
StorageAuthorityReadOnly provided by the mocks package to return actual
stream objects (which stream zero results) instead of nil, so that tests
can attempt to read from the resulting stream without getting a nil
pointer exception.
Part of https://github.com/letsencrypt/boulder/issues/7350
Add two new methods to the SA, GetSerialsByKey and GetSerialsByAccount,
which use the same query as the admin tool has previously used to get
serials matching a given SPKI hash or a given registration ID. These two
new gRPC methods read the database row-by-row and produce streams of
results to keep SA memory usage low.
Use these methods in the admin tool so it no longer needs a direct
database connection for these actions.
Part of https://github.com/letsencrypt/boulder/issues/7350
Before this PR, this is what example output of using the admin tool
would look like for an example command (in this case, certificate
revocation):
```
$ sudo /vagrant/admin -config /etc/boulder/config/admin.json revoke-cert -serial 2a13699017c283dea4d6ac5ac6d40caa3321
20:22:05.682795 6 admin qK_A7gU Debug server listening on :8000
20:22:05.682908 6 admin ts7-5Ag Versions: admin=(Unspecified Unspecified) Golang=(go1.21.8) BuildHost=(Unspecified)
20:22:05.689743 6 admin 6MX16g0 Found 1 certificates to revoke
20:22:05.691842 6 admin 1b3FsgU dry-run: &proto.AdministrativelyRevokeCertificateRequest{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Cert:[]uint8(nil), Serial:"2a13699017c283dea4d6ac5ac6d40caa3321", Code:0, AdminName:"root", SkipBlockKey:false, Malformed:false}
20:22:05.691901 6 admin 7tT7rAY Dry run complete. Pass -dry-run=false to mutate the database.
```
after this change the output looks like this:
```
$ sudo /vagrant/admin -config /etc/boulder/config/admin.json revoke-cert -serial 2a13699017c283dea4d6ac5ac6d40caa3321
21:22:13.769728 6 admin qK_A7gU Debug server listening on :8000
21:22:13.770156 6 admin ts7-5Ag Versions: admin=(Unspecified Unspecified) Golang=(go1.21.8) BuildHost=(Unspecified)
21:22:13.779291 6 admin xNuU_gY [AUDIT] admin tool executing a dry-run with the following arguments: revoke-cert -serial 2a13699017c283dea4d6ac5ac6d40caa3321
21:22:13.779534 6 admin 6MX16g0 Found 1 certificates to revoke
21:22:13.784524 6 admin yvHv9AM dry-run: "serial:\"2a13699017c283dea4d6ac5ac6d40caa3321\" adminName:\"root\""
21:22:13.786379 6 admin nKfNswk [AUDIT] admin tool has successfully completed executing a dry-run with the following arguments: revoke-cert -serial 2a13699017c283dea4d6ac5ac6d40caa3321
21:22:13.786951 6 admin 7tT7rAY Dry run complete. Pass -dry-run=false to mutate the database.
```
and with `-dry-run=false`:
```
$ sudo /vagrant/admin -config /etc/boulder/config/admin.json -dry-run=false revoke-cert -serial 2a13699017c283dea4d6ac5ac6d40caa3321
21:23:19.080073 6 admin qK_A7gU Debug server listening on :8000
21:23:19.080510 6 admin ts7-5Ag Versions: admin=(Unspecified Unspecified) Golang=(go1.21.8) BuildHost=(Unspecified)
21:23:19.089588 6 admin iKnckQ0 [AUDIT] admin tool executing with the following arguments: revoke-cert -serial 2a13699017c283dea4d6ac5ac6d40caa3321
21:23:19.089625 6 admin 6MX16g0 Found 1 certificates to revoke
21:23:19.169317 6 admin 9oyv3QY [AUDIT] admin tool has successfully completed executing with the following arguments: revoke-cert -serial 2a13699017c283dea4d6ac5ac6d40caa3321
```
Fixes#7358
Change crl-storer to only require that 1 of the IssuingDistributionPoint
URIs remain consistent between consecutive CRLs in the same sequence.
This allows us to add and remove IDP URIs, so we can change our IDP
scheme over time.
To facilitate this, also move all code which builds or parses IDP
extensions into a single place, so that we don't have to have multiple
definitions of the same types and similar code in many places.
Fixes https://github.com/letsencrypt/boulder/issues/7340
Part of https://github.com/letsencrypt/boulder/issues/7296
Adds post-issuance zlint linting to the `rootCeremony`,
`intermediateCeremony`, and `crossCertCeremony` ceremonies. It calls
zlint directly rather than using the existing
`issueLintCertAndPerformLinting` because the throwaway linting key pair
is unnecessary at this point.
Fixes https://github.com/letsencrypt/boulder/issues/7354
De-duplicate the code that has been replaced by `admin`, and cause all
of its subcommands to print helpful messages indicating the
corresponding `admin` command to run instead.
makes linter happy: not sure why 7 year old typo starts to hit by linter
nowdays though
not sure why github CI can't catch this but running t.sh locally marks
this as typo: (and it is)
We run the RVAs in AWS, where we don't have all the same service
discovery infrastructure we do for the primary VAs and the rest of
Boulder. The solution for populating SRV records we have today hasn't
been reliable, so we'd like to experiment with bringing up RVAs paired
1:1 with a local DNS resolver. This brings back some of the previous
static DNS resolver configuration, though it's not a clean revert
because other configuration has changed in the meantime
Move the CRL issuance logic -- building an x509.RevocationList template,
populating it with correctly-built extensions, linting it, and actually
signing it -- out of the //ca package and into the //issuance package.
This means that the CA's CRL code no longer needs to be able to reach
inside the issuance package to access its issuers and certificates (and
those fields will be able to be made private after the same is done for
OCSP issuance).
Additionally, improve the configuration of CRL issuance, create
additional checks on CRL's ThisUpdate and NextUpdate fields, and make it
possible for a CRL to contain two IssuingDistributionPoint URIs so that
we can migrate to shorter addresses.
IN-10045 tracks the corresponding production changes.
Fixes https://github.com/letsencrypt/boulder/issues/7159
Part of https://github.com/letsencrypt/boulder/issues/7296
Part of https://github.com/letsencrypt/boulder/issues/7294
Part of https://github.com/letsencrypt/boulder/issues/7094
Part of https://github.com/letsencrypt/boulder/issues/7100
Create a new administration tool "bin/admin" as a successor to and
replacement of "admin-revoker".
This new tool supports all the same fundamental capabilities as the old
admin-revoker, including:
- Revoking by serial, by batch of serials, by incident table, and by
private key
- Blocking a key to let bad-key-revoker take care of revocation
- Clearing email addresses from all accounts that use them
Improvements over the old admin-revoker include:
- All commands run in "dry-run" mode by default, to prevent accidental
executions
- All revocation mechanisms allow setting the revocation reason,
skipping blocking the key, indicating that the certificate is malformed,
and controlling the number of parallel workers conducting revocation
- All revocation mechanisms do not parse the cert in question, leaving
that to the RA
- Autogenerated usage information for all subcommands
- A much more modular structure to simplify adding more capabilities in
the future
- Significantly simplified tests with smaller mocks
The new tool has analogues of all of admin-revokers unit tests, and all
integration tests have been updated to use the new tool instead. A
future PR will remove admin-revoker, once we're sure SRE has had time to
update all of their playbooks.
Fixes https://github.com/letsencrypt/boulder/issues/7135
Fixes https://github.com/letsencrypt/boulder/issues/7269
Fixes https://github.com/letsencrypt/boulder/issues/7268
Fixes https://github.com/letsencrypt/boulder/issues/6927
Part of https://github.com/letsencrypt/boulder/issues/6840
Remove the Profile field from issuance.Issuer, to reflect the fact that
profiles are in fact independent pieces of configuration which can be
shared across (and are configured independently of) multiple issuers.
Move the IssuerURL, OCSPUrl, and CRLURL fields from issuance.Profile to
issuance.Issuer, since they reflect fundamental attributes of the
issuer, rather than attributes of a particular profile. This also
reflects the location at which those values are configured, in
issuance.IssuerConfig.
All other changes are fallout from the above: adding a Profile argument
to various methods in the issuance and linting packages, adding a
profile field to the caImpl struct, etc. This change paves the way for
two future changes: moving OCSP and CRL creation into the issuance
package, and supporting multiple simultaneous profiles that the CA can
select between.
Part of https://github.com/letsencrypt/boulder/issues/7159
Part of https://github.com/letsencrypt/boulder/issues/6316
Part of https://github.com/letsencrypt/boulder/issues/6966
Part of #7245.
There are still a few places that use 10.88.88.88 that will be harder to
remove. In particular, some of the Python integration tests start up
their own HTTP servers that differ from challtestsrv in some important
way (like timing out requests). Because challtestsrv already binds to
10.77.77.77:80, those test servers need a different IP address to bind
to. We can probably solve that but I'll leave it for another PR.
Previously, `va.IsCAAValid` would only check CAA records from the
primary VA during initial domain control validation, completely ignoring
any configured RVAs. The upcoming
[MPIC](https://github.com/ryancdickson/staging/pull/8) ballot will
require that it be done from multiple perspectives. With the currently
deployed [Multi-Perspective
Validation](https://letsencrypt.org/2020/02/19/multi-perspective-validation.html)
in staging and production, this change brings us in line with the
[proposed phase
3](https://github.com/ryancdickson/staging/pull/8/files#r1368708684).
This change reuses the existing
[MaxRemoteValidationFailures](21fc191273/cmd/boulder-va/main.go (L35))
variable for the required non-corroboration quorum.
> Phase 3: June 15, 2025 - December 14, 2025 ("CAs MUST implement MPIC
in blocking mode*"):
>
> MUST implement MPIC? Yes
> Required quorum?: Minimally, 2 remote perspectives must be used. If
using less than 6 remote perspectives, 1 non-corroboration is allowed.
If using 6 or more remote perspectives, 2 non-corroborations are
allowed.
> MUST block issuance if quorum is not met: Yes.
> Geographic diversity requirements?: Perspectives must be 500km from 1)
the primary perspective and 2) all other perspectives used in the
quorum.
>
> * Note: "Blocking Mode" is a nickname. As opposed to "monitoring mode"
(described in the last milestone), CAs MUST NOT issue a certificate if
quorum requirements are not met from this point forward.
Adds new VA feature flags:
* `EnforceMultiCAA` instructs a primary VA to command each of its
configured RVAs to perform a CAA recheck.
* `MultiCAAFullResults` causes the primary VA to block waiting for all
RVA CAA recheck results to arrive.
Renamed `va.logRemoteValidationDifferentials` to
`va.logRemoteDifferentials` because it can handle initial domain control
validations and CAA rechecking with minimal editing.
Part of https://github.com/letsencrypt/boulder/issues/7061
These names corresponded to single instances of a service, and were
primarily used for (a) specifying which interface to bind a gRPC port on
and (b) allowing `health-checker` to check individual instances rather
than a service as a whole.
For (a), change the `--grpc-addr` flags to bind to "all interfaces." For
(b), provide a specific IP address and port for health checking. This
required adding a `--hostOverride` flag for `health-checker` because the
service certificates contain hostname SANs, not IP address SANs.
Clarify the situation with nonce services a little bit. Previously we
had one nonce "service" in Consul and got nonces from that (i.e.
randomly between the two nonce-service instances). Now we have two nonce
services in consul, representing multiple datacenters, and one of them
is explicitly configured as the "get" service, while both are configured
as the "redeem" service.
Part of #7245.
Note this change does not yet get rid of the rednet/bluenet distinction,
nor does it get rid of all use of 10.88.88.88. That will be a followup
change.
Rename "IssuerNameID" to just "NameID". Similarly rename the standalone
functions which compute it to better describe their function. Add a
.NameID() directly to issuance.Issuer, so that callers in other packages
don't have to directly access the .Cert member of an Issuer. Finally,
rearrange the code in issuance.go to be sensibly grouped as concerning
NameIDs, Certificates, or Issuers, rather than all mixed up between the
three.
Fixes https://github.com/letsencrypt/boulder/issues/5152
Enable the atomicalign, deepequalerrors, findcall, nilness,
reflectvaluecompare, sortslice, timeformat, and unusedwrite go vet
analyzers, which golangci-lint does not enable by default. Additionally,
enable new go vet analyzers by default as they become available.
The fieldalignment and shadow analyzers remain disabled because they
report so many errors that they should be fixed in a separate PR.
Note that the nilness analyzer appears to have found one very real bug
in tlsalpn.go.
Upgrade to zlint v3.6.0
Two new lints are triggered in various places:
aia_contains_internal_names is ignored in integration test
configurations, and unit tests are updated to have more realistic URLs.
The w_subject_common_name_included lint needs to be ignored where we'd
ignored n_subject_common_name_included before.
Related to https://github.com/letsencrypt/boulder/issues/7261
Revamp WillingToIssueWildcards to WillingToIssue. Remove the need for
identifier.ACMEIdentifiers in the WillingToIssue(Wildcards) method.
Previously, before invoking this method, a slice of identifiers was
created by looping over each dnsName. However, these identifiers were
solely used in error messages.
Segment the validation process into distinct parts for domain
validation, wildcard validation, and exact blocklist checks. This
approach eliminates the necessity of substituting *. with x. in wildcard
domains.
Introduce a new helper, ValidDomain. It checks that a domain is valid
and that it doesn't contain any invalid wildcard characters.
Functionality from the previous ValidDomain is preserved in
ValidNonWildcardDomain.
Fixes#3323
Use policy.ValidEmail to vet email addresses before sending expiration
notifications to them. This same check is performed by notify-mailer,
and it helps reduce the number of invalid addresses we attempt to send
to and the number of email bounces we generate.
Additionally, mark certificates as having had a nag email sent if there
are no valid addresses for us to send to, so that we don't constantly
retry them.
Fixes https://github.com/letsencrypt/boulder/issues/5372
Change the max value of the CA's `SerialPrefix` config value from 255 (a
byte of all 1s) to 127 (a byte of one 0 followed by seven 1s). This
prevents the serial prefix from ever beginning with a 1.
This is important because serials are interpreted as signed
(twos-complement) integers, and are required to be positive -- a serial
whose first bit is 1 is considered to be negative and therefore in
violation of RFC 5280. The go stdlib fixes this for us by prepending a
zero byte to any serial that begins with a 1 bit, but we'd prefer all
our serials to be the same length.
Corresponding config change was completed in IN-9880.
Replace the current three-piece setup (enum of feature variables, map of
feature vars to default values, and autogenerated bidirectional maps of
feature variables to and from strings) with a much simpler one-piece
setup: a single struct with one boolean-typed field per feature. This
preserves the overall structure of the package -- a single global
feature set protected by a mutex, and Set, Reset, and Enabled methods --
although the exact function signatures have all changed somewhat.
The executable config format remains the same, so no deployment changes
are necessary. This change does deprecate the AllowUnrecognizedFeatures
feature, as we cannot tell the json config parser to ignore unknown
field names, but that flag is set to False in all of our deployment
environments already.
Fixes https://github.com/letsencrypt/boulder/issues/6802
Fixes https://github.com/letsencrypt/boulder/issues/5229
Many services already have --addr and/or --debug-addr flags.
However, it wasn't universal, so this PR adds flags to commands where
they're not currently present.
This makes it easier to use a shared config file but listen on different
ports, for running multiple instances on a single host.
The config options are made optional as well, and removed from
config-next/.
- Move default and override limits, and associated methods, out of the
Limiter to new limitRegistry struct, embedded in a new public
TransactionBuilder.
- Export Transaction and add corresponding Transaction constructor
methods for each limit Name, making Limiter and TransactionBuilder the
API for interacting with the ratelimits package.
- Implement batched Spends and Refunds on the Limiter, the new methods
accept a slice of Transactions.
- Add new boolean fields check and spend to Transaction to support more
complicated cases that can arise in batches:
1. the InvalidAuthorizations limit is checked at New Order time in a
batch with many other limits, but should only be spent when an
Authorization is first considered invalid.
2. the CertificatesPerDomain limit is overridden by
CertficatesPerDomainPerAccount, when this is the case, spends of the
CertificatesPerDomain limit should be "best-effort" but NOT deny the
request if capacity is lacking.
- Modify the existing Spend/Refund methods to support
Transaction.check/spend and 0 cost Transactions.
- Make bucketId private and add a constructor for each bucket key format
supported by ratelimits.
- Move domainsForRateLimiting() from the ra.go to ratelimits. This
avoids a circular import issue in ra.go.
Part of #5545
- Adds a feature flag to gate rollout for SHA256 Subject Key Identifiers
for end-entity certificates.
- The ceremony tool will now use the RFC 7093 section 2 option 1 method
for generating Subject Key Identifiers for future root CA, intermediate
CA, and cross-sign ceremonies.
- - - -
[RFC 7093 section 2 option
1](https://datatracker.ietf.org/doc/html/rfc7093#section-2) provides a
method for generating a truncated SHA256 hash for the Subject Key
Identifier field in accordance with Baseline Requirement [section
7.1.2.11.4 Subject Key
Identifier](90a98dc7c1/docs/BR.md (712114-subject-key-identifier)).
> [RFC5280] specifies two examples for generating key identifiers from
> public keys. Four additional mechanisms are as follows:
>
> 1) The keyIdentifier is composed of the leftmost 160-bits of the
> SHA-256 hash of the value of the BIT STRING subjectPublicKey
> (excluding the tag, length, and number of unused bits).
The related [RFC 5280 section
4.2.1.2](https://datatracker.ietf.org/doc/html/rfc5280#section-4.2.1.2)
states:
> For CA certificates, subject key identifiers SHOULD be derived from
> the public key or a method that generates unique values. Two common
> methods for generating key identifiers from the public key are:
> ...
> Other methods of generating unique numbers are also acceptable.
When running in manual mode, the `configFile` variable will take the
zero value of `""` while `manualConfigFile` will be provided on the CLI
by the operator. A startup check incorrectly dereferences `configFile`;
but correctly determines that it is the zero value `""`, outputs the
help text, and exits never allowing manual mode to perform work.
Fixes https://github.com/letsencrypt/boulder/issues/7176
This is a cleanup PR finishing the migration from int64 timestamps to
protobuf `*timestamppb.Timestamps` by removing all usage of the old
int64 fields. In the previous PR
https://github.com/letsencrypt/boulder/pull/7121 all fields were
switched to read from the protobuf timestamppb fields.
Adds a new case to `core.IsAnyNilOrZero` to check various properties of
a `*timestamppb.Timestamp` reducing the visual complexity for receivers.
Fixes https://github.com/letsencrypt/boulder/issues/7060
Samantha Frank