Add a new config field for profiles which causes the profile to omit the
AIA OCSP URI. It can only be omitted if the CRLDP extension is
configured to be included instead. Enable this flag in config-next.
When a certificate is revoked, if it does not have an AIA OCSP URI,
don't bother with an Akamai OCSP purge.
Builds on #8089
Most of the changes in this PR relate to tests. Different from #8089, I
chose to keep testing of OCSP in the config-next world. This is because
we intend to keep operating OCSP even after we have stopped including it
in new certificates. So we should test it in as many environments as
possible.
Adds a WithURLFallback option to ocsp_helper. When
`ocsp_helper.ReqDer()` is called for a certificate with no OCSP URI, it
will query the fallback URL instead. As before, if the certificate has
an OCSP URI ocsp_helper will use that. Use that for all places in the
integration tests that call ocsp_helper.
Update the hierarchy which the integration tests auto-generate inside
the ./hierarchy folder to include three intermediates of each key type,
two to be actively loaded and one to be held in reserve. To facilitate
this:
- Update the generation script to loop, rather than hard-coding each
intermediate we want
- Improve the filenames of the generated hierarchy to be more readable
- Replace the WFE's AIA endpoint with a thin aia-test-srv so that we
don't have to have NameIDs hardcoded in our ca.json configs
Having this new hierarchy will make it easier for our integration tests
to validate that new features like "unpredictable issuance" are working
correctly.
Part of https://github.com/letsencrypt/boulder/issues/729
Run staticcheck as a standalone binary rather than as a library via
golangci-lint. From the golangci-lint help out,
> staticcheck (megacheck): It's a set of rules from staticcheck. It's
not the same thing as the staticcheck binary. The author of staticcheck
doesn't support or approve the use of staticcheck as a library inside
golangci-lint.
We decided to disable ST1000 which warns about incorrect or missing
package comments.
For SA4011, I chose to change the semantics[1] of the for loop rather
than ignoring the SA4011 lint for that line.
Fixes https://github.com/letsencrypt/boulder/issues/6988
1. https://go.dev/ref/spec#Continue_statements
In order to get rid of the orphan queue, we want to make sure that
before we sign a precertificate, we have enough data in the database
that we can fulfill our revocation-checking obligations even if storing
that precertificate in the database fails. That means:
- We should have a row in the certificateStatus table for the serial.
- But we should not serve "good" for that serial until we are positive
the precertificate was issued (BRs 4.9.10).
- We should have a record in the live DB of the proposed certificate's
public key, so the bad-key-revoker can mark it revoked.
- We should have a record in the live DB of the proposed certificate's
names, so it can be revoked if we are required to revoke based on names.
The SA.AddPrecertificate method already achieves these goals for
precertificates by writing to the various metadata tables. This PR
repurposes the SA.AddPrecertificate method to write "proposed
precertificates" instead.
We already create a linting certificate before the precertificate, and
that linting certificate is identical to the precertificate that will be
issued except for the private key used to sign it (and the AKID). So for
instance it contains the right pubkey and SANs, and the Issuer name is
the same as the Issuer name that will be used. So we'll use the linting
certificate as the "proposed precertificate" and store it to the DB,
along with appropriate metadata.
In the new code path, rather than writing "good" for the new
certificateStatus row, we write a new, fake OCSP status string "wait".
This will cause us to return internalServerError to OCSP requests for
that serial (but we won't get such requests because the serial has not
yet been published). After we finish precertificate issuance, we update
the status to "good" with SA.SetCertificateStatusReady.
Part of #6665
This requires setting --issuer-file and --url, but it allows (for
instance) collecting a big pile of serial numbers for a known issuer,
rather than having to keep whole certificates.
The iotuil package has been deprecated since go1.16; the various
functions it provided now exist in the os and io packages. Replace all
instances of ioutil with either io or os, as appropriate.
Add a new filter to mail-test-srv, allowing test processes to query
for messages sent from a specific address, not just ones sent to
a specific address. This fixes a race condition in the revocation
integration tests where the number of messages sent to a cert's
contact address would be higher than expected because expiration
mailer sent a message while the test was running. Also reduce
bad-key-revoker's maximum backoff to 2 seconds to ensure that
it continues to run frequently during the integration tests, despite
usually not having any work to do.
While we're here, also improve the comments on various revocation
integration tests, remove some unnecessary cruft, and split the tests
out to explicitly test functionality with the MozRevocationReasons
flag both enabled and disabled. Also, change ocsp_helper's default
output from os.Stdout to ioutil.Discard to prevent hundreds of lines
of log spam when the integration tests fail during a test that uses
that library.
Fixes#6248
Simplify the WFE `RevokeCertificate` API method in three ways:
- Remove most of the logic checking if the requester is authorized to
revoke the certificate in question (based on who is making the
request, what authorizations they have, and what reason they're
requesting). That checking is now done by the RA. Instead, simply
verify that the JWS is authenticated.
- Remove the hard-to-read `authorizedToRevoke` callbacks, and make the
`revokeCertBySubscriberKey` (nee `revokeCertByKeyID`) and
`revokeCertByCertKey` (nee `revokeCertByJWK`) helpers much more
straight-line in their execution logic.
- Call the RA's new `RevokeCertByApplicant` and `RevokeCertByKey` gRPC
methods, rather than the deprecated `RevokeCertificateWithReg`.
This change, without any flag flips, should be invisible to the
end-user. It will slightly change some of our log message formats.
However, by now relying on the new RA gRPC revocation methods, this
change allows us to change our revocation policies by enabling the
`AllowDoubleRevocation` and `MozRevocationReasons` feature flags, which
affect the behavior of those new helpers.
Fixes#5936
This is a sort of proof of concept of the Redis interaction, which will
evolve into a tool for inspection and manual repair of missing entries,
if we find ourselves needing to do that.
The important bits here are rocsp/rocsp.go and
cmd/rocsp-tool/main.go. Also, the newly-vendored Redis client.
Add a unit test and an integration test that both exercise the new
experimental ACME Renewal Info endpoint. These tests do not
yet validate the contents of the response, just that the appropriate
HTTP response code is returned, but they will be developed as the
code under test evolves.
Fixes#5674
The ocsp-responder takes a path to a certificate file as one of
its config values. It uses this path as one of the inputs when
constructing its DBSource, the object responsible for querying
the database for pregenerated OCSP responses to fulfill requests.
However, this certificate file is not necessary to query the
database; rather, it only acts as a filter: OCSP requests whose
IssuerKeyHash do not match the hash of the loaded certificate are
rejected outright, without querying the DB. In addition, there is
currently only support for a single certificate file in the config.
This change adds support for multiple issuer certificate files in
the config, and refactors the pre-database filtering of bad OCSP
requests into a helper object dedicated solely to that purpose.
Fixes#5119
In #5103 I tweaked the help to allow customizing where the output goes,
but I didn't ensure that the output field was always set. Also, I forgot
to expand the `...v` parameter when passing it to Fprintf.
This adds a configurable output parameter for ocsp_helper, which
defaults to stdout. This allows suppressing the stdout output when using
ocsp_helper in integration tests. That output was making it hard to
see details about failing tests.
Previously, if ocspchecker encountered a cert whose OCSP response
it didn't like, it would print the first reason it didn't like it and then
move on to the next cert provided on the command line. This
behavior both obscures the cert in question (by not printing
details of its OCSP response) and the error in question (by only
printing the first encountered error, instead of all errors).
This change causes ocspchecker to both print the details of the
OCSP response whether it likes the response or not, and to
accumulate all errors it encounters while validating that response.
This should increase interactive usability, as requested in #4901.
This changes the default behavior of the `-expectStatus` flag of
the ocspchecker and ocsp_forever binaries (as well as any other
consumer of the ocsp/helper.go library).
Previously, the flag had a default value of 0, meaning that all
OCSP responses were expected to have a status of "Good" (i.e. not
"Revoked"); having any other status is an error which would cause
the tool to print different information and have a non-zero exit code.
This flag now has a default value of -1, meaning that no enforcement
of the OCSP status should happen. All OCSP responses, whether Good,
Revoked, or otherwise, will simply be logged for informational
purposes and the tool will exit successfully.
This is a backwards-incompatible change, and users which rely on
the default behavior of this flag should update themselves.
Adds a new -expect-reason flag to the checkocsp binary to allow for
verifying the revocation reason of the certificate(s) in question.
This flag has a default value of -1, meaning that no particular
revocation reason will be expected or enforced.
Also updates the -expect-status flag to have the same default (-1) and
behavior, so that when the tool is run interactively it can simply
print the revocation status of each certificate.
Finally, refactors the way the ocsp/helper library declares flags and
accesses their values. This unifies the interface and makes it easy to
extend to allow tests to modify parameters other than expectStatus when
desired.
Fixes#4885
When downloading a pkcs7 file to parse for issuer, some servers
appear to deliver application/pkcs7-mime Content-Type and others
application/x-pkcs-mime. This patch changes the conditional
to look for both to invoke parseCMS().
Some components, particularly net/http, occasionally output log lines
via log.Print. We'd like to capture these and send them to rsyslog so
all our log data goes to the same place, and so that we can attach log
line checksums to them.
This uses log.SetOutput to change the log output to an io.Pipe,
then consumes that buffer line-by-line in a goroutine and sends it to
our rsyslog logger.
This seems to tickle an unrelated race condition in test/ocsp/helper.go,
so I fixed that too.
Also filters out a noisy and unimportant error from the grpcLog handler.
Fixes#4664Fixes#4628
When the `features.PrecertificateRevocation` feature flag is enabled the WFE2
will allow revoking certificates for a submitted precertificate. The legacy WFE1
behaviour remains unchanged (as before (pre)certificates issued through the V1
API will be revocable with the V2 API).
Previously the WFE2 vetted the certificate from the revocation request by
looking up a final certificate by the serial number in the requested
certificate, and then doing a byte for byte comparison between the stored and
requested certificate.
Rather than adjust this logic to handle looking up and comparing stored
precertificates against requested precertificates (requiring new RPCs and an
additional round-trip) we choose to instead check the signature on the requested
certificate or precertificate and consider it valid for revocation if the
signature validates with one of the WFE2's known issuers. We trust the integrity
of our own signatures.
An integration test that performs a revocation of a precertificate (in this case
one that never had a final certificate issued due to SCT embedded errors) with
all of the available authentication mechanisms is included.
Resolves https://github.com/letsencrypt/boulder/issues/4414
This test adds support in ct-test-srv for rejecting precertificates by
hostname, in order to artificially generate a condition where a
precertificate is issued but no final certificate can be issued. Right
now the final check in the test is temporarily disabled until the
feature is fixed.
Also, as our first Go-based integration test, this pulls in the
eggsampler/acme Go client, and adds some suport in integration-test.py.
This also refactors ct-test-srv slightly to use a ServeMux, and fixes
a couple of cases of not returning immediately on error.
* in boulder-ra we connected to the publisher and created a publisher gRPC client twice for no apparent reason
* in the SA we ignored errors from `getChallenges` in `GetAuthorizations` which could result in a nil challenge being returned in an authorization
Prior to this commit the `httpResp` result of `sendHTTPRequest` was
examined even in the case where `sendHTTPRequest` returns a non-nil
error. This can cause a nil panic since the `httpResp` may be `nil` when
the error is not. This commit returns an error from `Req()` immediately
when `sendHTTPRequest` returns one.
These are monitoring tools, originally from
https://github.com/jsha/go/tree/master/ocsp. We'd like to formalize their role
in monitoring Boulder, so I'm adding them to the Boulder repo and getting them
reviewed.
Jacob Hoffman-Andrews
Aaron Gable
Phil Porada
Andrew Gabbitas
Jacob Hoffman-Andrews
Daniel McCarney
Roland Bracewell Shoemaker
Daniel McCarney