@jsha suggested I re-implement a PR against Pebble regarding Authorization
reuse into Boulder (see https://github.com/letsencrypt/pebble/pull/325).
This is an initial attempt. I opted to handle this by creating a new file for
"Implementation Details" that are RFC conformant and are known to have
confused client developers.
In #5103 I tweaked the help to allow customizing where the output goes,
but I didn't ensure that the output field was always set. Also, I forgot
to expand the `...v` parameter when passing it to Fprintf.
This adds a configurable output parameter for ocsp_helper, which
defaults to stdout. This allows suppressing the stdout output when using
ocsp_helper in integration tests. That output was making it hard to
see details about failing tests.
The IssuerID shouldn't ever be 0 (it should always be NULL/nil or
an actual value), but we recently had an incident in which it was
being set to 0 instead of NULL. This ensures that functionality
will continue as intended even in the face of that circumstance.
Fixes#5098
This change adds `req.IssuerID` to the set of fields that the SA's
`AddPrecertificate` method requires be non-zero.
As a result, this also updates many tests, both unit and integration,
to ensure that they supply a value (usually just 1) for that field. The
most complex part of the test changes is a slight refactoring to the
orphan-finder code, which makes it easier to reason about the
separation between log line parsing and building and sending the
request.
Based on #5096Fixes#5097
This change reorganizes the document to have all changes
noted under their respective section headings, updates estimated
resolution dates on long-standing divergences, and updates all URLs
to reference the final RFC 8555 instead of various drafts.
In addition, it adds a note that we do not accept the (optional)
`notBefore` and `notAfter` fields of a `newOrder` request.
The previous implementation of `IsAnyNilOrZero` did not in fact work,
and its tests did not catch this fact. Within the numeric clause, the
compiler would only instantiate the comparison literal 0 to be one
of the eight possible types. Comparisons against any of the other
seven types would always be false, no matter what value that type
held.
The tests did not catch this because they only tested two literal
values: `0` and `-12.345`, both of which can be `float64`s.
This change updates the utility function to use the `reflect` package,
to ensure that it works correctly. It also updates the test to test
multiple different kinds of numeric values, and removes the code
for handling pointer-to- types, as all of our proto2 code has been
removed.
Finally, it updates the SA wrapper's `RevokeCertificate` method to
correctly not require that `req.Reason` be non-zero: this field can
and often is zero, as that value represents `Unspecified`.
Using the reflect package is a conscious tradeoff. It will be slower
than manually writing out every single case, but it will also be less
prone to error.
Part of #5097
When the CA loads new issuers (both their certificates and their
private keys), it performs a variety of sanity checks, such as
ensuring that the profile's signature algorithm matches the key
type.
With this change, we also check that the issuer's certificate has
the appropriate key usage bits set:
`certSign`, if it is going to be issuing end-entity certs; and
`digitalSignature`, because it will be signing OCSP responses for
previously-issued certificates.
Fixes#5068
One of the log lines describes the most frequent address corresponding
to a number of accounts, but it actually corresponds to a number of
lines in the input CSV.
Also, now that we escape newlines in log output, the dryRunMailer's
output looks messed up. Split the message body into lines and emit one
log message per line.
Generated keys have the same label on both the private and public key
objects. When looking up keys for signing, the label is used to find the
public key.
The CA is the only service which still defines its json config format
in the package itself, rather than in its corresponding boulder-ca cmd
package. This has allowed the CA's constructor interface to hide
arbitrary complexity inside its first argument, the whole config blob.
This change moves the CA's config to boulder-ca/main.go, to match
the other Boulder components. In the process, it makes a host of
other improvements:
It refactors the issuance package to have a cleaner configuration
interface. It also separates the config into a high-level profile (which
applies equally to all issuers), and issuer-level profiles (which apply
only to a single issuer). This does involve some code duplication,
but that will be removed when CFSSL goes away.
It adds helper functions to the issuance package to make it easier
to construct a new issuer, and takes advantage of these in the
boulder-ca package. As a result, the CA now receives fully-formed
Issuers at construction time, rather than constructing them from
nearly-complete configs during its own initialization.
It adds a Linter struct to the lint package, so that an issuer can
simply carry around a Linter, rather than a separate lint signing
key and registry of lints to run.
It makes CFSSL-specific code more clearly marked as such,
making future removal easier and cleaner.
Fixes#5070Fixes#5076
We define a "signer" to be a private key, or something that satisfies the
crypto.Signer interface. We define an "issuer" to be an object which has
both a signer (so it can sign things) and a certificate (so that the things
it signs can have appropriate issuer fields set).
As a result, this change:
- moves the new "signer" library to be called "issuance" instead
- renames several "signers" to instead be "issuers", as defined above
- renames several "issuers" to instead be "certs", to reduce confusion more
There are some further cleanups which could be made, but most of them
will be made irrelevant by the removal of the CFSSL code, so I'm leaving
them be for now.
This just changes the `loadCFSSLIssuers` signature to more closely match
the `loadBoulderIssuers` signature (it didn't need access to the whole config
object), and standardize our json on lowercase string keys.
There are two code paths which end up calling the SA's AddPrecertificate
RPC: one "normal" path from inside the CA's IssuePrecertificate
method, and one "exceptional" path from inside the orphan-integration
path which only gets executed if the initial attempt to store the cert
failed for some reason.
The first of these paths always sets the IssuerID in the RPC's
AddCertificateRequest. The latter of these paths never does.
Historically, this has been fine, as the SA has stored NULL in the
certificateStatus table when given a nil IssuerID, which matches the
behavior prior to the StoreIssuerInfo flag (when the IssuerID was
always nil, and cert status was tracked by full cert DER instead).
However, with the switch to proto3, the SA now interprets a nil
IssuerID as a 0, and stores that value in the table instead. The
ocsp-updater code which consumes rows of the certificateStatus table
is not able to properly handle IssuerIDs of 0, leading to fun times.
This change ensures that the orphan handling code also sets the
IssuerID when asking the SA to create a new certificateStatus row,
so we should stop producing new rows with a 0 IssuerID.
These functions sign a random nonce with a newly-created issuance key,
in order to verify that the key was correctly generated and its public
component was correctly extracted. In general we can trust that keys are
correctly created by the HSM, and unit and integration tests can check
that we are correctly extracting public keys.
Removing these avoids the possibility of signing something that could be
construed as a "certificate, but malformed."
Since we generate an intermediate on each integration test run, this
speeds things up by a few seconds. It also makes generation of our
linting keys on CA startup faster.
In #4992, we refactored NewSigner to look keys up by public key and by
label. However, we didn't correctly incorporate the label check into
the new code. This fixes that and adds a test.
The ca's configuration already has support for containing multiple
issuers. However, when it comes time to actually sign a (pre)cert,
it always uses the defaultIssuer.
This change has the ca instead choose which issuer to use based
on the PublicKeyAlgorithm requested in the CSR (or, for final cert
issuances, based on the PublicKeyAlgorithm in the precert).
This will allow us to use our RSA issuers to sign certificates for
users who aren't ready to switch to ECDSA, while immediately switching
to our new ECDSA chain for subscribers who want to use it.
Fixed#5027
It's not vital that this row be strongly consistent with the other
updates. And updating it inside the transaction means we hold a lock on
this row while doing a bunch of other expensive inserts, which is likely
creating lock contention.
When issuing a new root or intermediate cert, we should take every
precaution possible to ensure that these certs are well-formed.
This change introduces a new step prior to issuing and writing a new CA
cert. We generate a new disposable private key based on the type of the
key being used in the real ceremony, then use this key to sign a fake
certificate for the sole purpose of linting. We then pass this through
the full suite of zlint's checks before proceeding with the actual
issuance.
Since this code path is largely similar to the pre-issuance linting done
by the new boulder signer tool, this change also factors it out into a
small, single-purpose `lint` package.
Fixes#5051
This brings in the following changes to zlint:
https://github.com/zmap/zlint/compare/v2.1.0...9ab0643
Importantly, this prevents the cert lifetime lint from triggering on
CA certs, and removes the OCSP url requirement lint entirely.
In preparation for moving to proto3 for the core/proto types
(Registration, Order, Authorization, Challenge, etc), this removes
checks that will fail when a proto2 client or server receives a message
from a proto3 client or server. Since proto3 encodes fields that have
their zero value as being absent (i.e. nil, in Go), we treat nil the
same as having a zero value.
In the process this change introduces additional checks, verifying
certain fields which should never have a zero value.
This involves factoring out registrationValid into registrationValid and
newRegistrationValid, since new registration requests lack some fields that
already-created registrations should always have.
Similarly, UpdateRegistration is changed to not verify that
`request.Update` is valid, since an update to registration object is not
a complete registration object - it may only update one field.
Previously this was a NotFound error, but since we now update the
certificateStatus table synchronously on issuance and revocation, we
expect to always get a successful response; if we get an error, that's
a ServerInternal error.
These were used during the transition to authzv2. The SA side of these
RPCs already ignores these booleans. This is just cleaning up the
protobufs and call sites.
As part of #5050, I'm updating some of the code in grpc/pb-marshaling.go
to move from nil checks to zero checks. In the process I'm introducing some
new zero checks, on things like challenge type, status, and token. This is
shaking out some places where our mocks have taken shortcuts by not
creating a "full" object including all fields that are normally present.
This PR updates our mocks and tests to provide more realistic objects in
all the places that broke when introducing those zero checks.
One slightly surprising / interesting thing: Since core types like
Order and Registration are still proto2 and have pointer fields,
there are actually some places in this PR where I had to add
a `*` rather than delete an `&`, because I was taking a pointer
field from one of those core types and passing it as a field in
an SA RPC request.
Fixes#5037.
Since we now sync caaChecks logs daily instead of continuously,
caa-log-checker can no longer assume that the validation logs it is
checking cover the exact same span of time as the issuance logs. This
commit adds -earliest and -latest parameters so that the script
that drives this tool can restrict verification to a timespan where we
know the data is valid.
Also adds a -debug flag to caa-log-checker to enable debug logs. At the
moment this makes the tool write to stderr how many issuance messages
were evaluated and how many were skipped due to -earliest and
-latest parameters.
Updates the Registration Authority to use proto3 for its
RPC methods. This turns out to be a fairly minimal change,
as many of the RA's request and response messages are
defined in core.proto, and are therefore still proto2.
Fixes#4955
Any field which can be zero must be allowed to be nil,
so that a proto2 server receiving requests from a proto3
client is willing to process messages with zero-value fields
encoded as missing.
Part of #4955
As part of the migration to proto3, any fields in requests that may be
zero should also be allowed to be nil. That's because proto3 will
represent those fields as absent when they have their zero value.
This is based on a manual review of the wrappers for the SA, plus
a pair of integration test runs. For the integration test runs I took these
steps:
1. Copy sa/proto to sa/proto2
2. Change sa/proto to use proto3 and regenerate.
3. In sa/*.go and cmd/boulder-sa/main.go, update the imports to use the
proto2 version.
4. Split grpc/sa-wrappers.go into sa-server-wrappers.go and sa-wrappers.go
(containing the client code)
5. In sa-server-wrappers.go, change the import to use sa/proto2.
6. In sa-server-wrappers.go, make a local copy of the core.StorageAuthority
interface that uses the sa/proto2 types. This was necessary as
a temporary kludge because of how the server wrapper internally
uses the core.StorageAuthority interface.
7. Fix all the pointer-vs-value build errors in every other package.
8. Run integration tests.
I also performed those steps with proto2 and proto3 swapped, to confirm the
behavior when a proto2 client talks to a proto3 SA.
Jonathan Vanasco