Create a new type `db.MappedSelector` which exposes a new
`Query` method. This method behaves similar to gorp's
`SelectFoo` methods, in that it uses the desired result type to
look up the correct table to query and uses reflection to map
the table columns to the struct fields. It behaves similarly to
the stdlib's `sql.Query` in that it returns a `Rows` object which
can be iterated over to get one row of results at a time. And it
improves both of those by using generics, rather than `interface{}`,
to provide a nicely-typed calling interface.
Use this new type to simplify the existing streaming query in
`SerialsForIncident`. Similarly use the new type to simplify
rocsp-tool's and ocsp-updater's streams of `CertStatusMetadata`.
This new type will also be used by the crl-updater's upcoming
`GetRevokedCerts` streaming query.
Fixes#6173
When we get a primary result, but the primary result is too stale
(>60h), try to serve from the secondary instead. This allows us to
smoothly transition into the "ocsp-updater no longer updates MariaDB"
regime.
If the primary result is too stale, and the secondary returns an
error, go ahead and serve the primary result. This could happen if
ocsp-updater falls behind. We would still like to try and serve an OCSP
response from the primary at until its actual expiration date
(168h currently). This preserves our current graceful failure mechanism
when ocsp-updater is not running for many hours.
Also, in the case where secondary returns first but primary has a
fresher response, serve the primary's response. This ensures we always
serve the freshest available, for instance if Redis stops getting
updates for a while.
Related to #6079
In multiSource, if the primary source returned before the secondary
source, we would leak the goroutine waiting on results from the secondary
source. This was because that goroutine was writing to a channel that
no longer had any readers.
Fix that by making the channel in getResponse buffered, so getResponse
can always write to its channel, regardless of whether there are readers.
I've confirmed that the unittest fails when run without the fix.
These new linters are almost all part of golangci-lint's collection
of default linters, that would all be running if we weren't setting
`disable-all: true`. By adding them, we now have parity with the
default configuration, as well as the additional linters we like.
Adds the following linters:
* unconvert
* deadcode
* structcheck
* typecheck
* varcheck
* wastedassign
Adds a rocsp redis client to the sa if cluster information is provided in the
sa config. If a redis cluster is configured, all new certificate OCSP
responses added with sa.AddPrecertificate will attempt to be written to
the redis cluster, but will not block or fail on errors.
Fixes: #5871
Taking a config object as an argument to a constructor is an anti-
pattern we avoid elsewhere. Resolve the TODO previously put in
place when this first written by instead increasing the number of
arguments `New()` takes and pulling those values out of the config
ahead of time.
Fixes#5904
Completely refactor the way we organize our code related to OCSP.
- Move it all into one `//ocsp/` package, rather than having multiple
top-level packages.
- Merge the OCSP updater's config sub-package with its parent
(since it isn't necessary to break it out to avoid cyclic imports).
- Remove all `Source` logic from ocsp-responder's `main.go`, because
it was difficult to mentally trace the control flow there.
- Replace that logic with a set of composable `Source`s in the
`//ocsp/responder/` package, each of which is good at just one thing.
- Update the way the filters work to make sure that the request's
`IssuerKeyHash` and the response's `ResponderName` can both
be derived from the same issuer certificate, ensuring that the req and
resp are correctly matched.
- Split the metrics into a separate metric for each `Source`, so we can
tell what all of them are doing, not just aggregate behavior.
- Split the tests into individual files for each `Source`, and update them
for the new public interfaces.
Add `stylecheck` to our list of lints, since it got separated out from
`staticcheck`. Fix the way we configure both to be clearer and not
rely on regexes.
Additionally fix a number of easy-to-change `staticcheck` and
`stylecheck` violations, allowing us to reduce our number of ignored
checks.
Part of #5681
The `Source` interface in ocsp-responder defines a `Response` function.
Add a context to the function signature so that ocsp lookups can be more
tracable and cancelable. This is also a precursor to having cancelable
parallel lookups to multiple sources.
Add Honeycomb tracing to all Boulder components which act as
HTTP servers, gRPC servers, or gRPC clients. Add many values
which we currently emit to logs to the trace spans. Add a way to
configure the Honeycomb integration to our config files, and by
default configure all of our tests to "mute" (send nothing).
Followup changes will refine the configuration, attempt to reduce
the new dependency load, and introduce better sampling.
Part of https://github.com/letsencrypt/dev-misc-tickets/issues/218
Update all of our tests to use `AssertMetricWithLabelsEquals`
instead of combinations of the older `CountFoo` helpers with
simple asserts. This coalesces all of our prometheus inspection
logic into a single function, allowing the deletion of four separate
helper functions.
There are various technical requirements on the maximum age of
an OCSP response. Although ocsp-updater has mechanisms to ensure
that all certificates have responses which are sufficiently recent,
there is the possibility of a bug which results in some OCSP
responses escaping its notice.
This change adds a historgram metric to ocsp-responder which collects
the ages (i.e. now minus the `thisUpdate` timestamp) of the OCSP
respones which it serves. The histogram has equal buckets in 12-hour
increments. During normal operation, the first 7 such buckets
(representing ages 0 to 3.5 days) should have roughly equal counts,
while the latter 7 buckets (3.5 to 7 days) should be empty.
Fixes#5080
The ocsp-responder takes a path to a certificate file as one of
its config values. It uses this path as one of the inputs when
constructing its DBSource, the object responsible for querying
the database for pregenerated OCSP responses to fulfill requests.
However, this certificate file is not necessary to query the
database; rather, it only acts as a filter: OCSP requests whose
IssuerKeyHash do not match the hash of the loaded certificate are
rejected outright, without querying the DB. In addition, there is
currently only support for a single certificate file in the config.
This change adds support for multiple issuer certificate files in
the config, and refactors the pre-database filtering of bad OCSP
requests into a helper object dedicated solely to that purpose.
Fixes#5119
Integrates the cfssl/ocsp responder code directly into boulder. I've tried to
pare down the existing code to only the bits we actually use and have removed
some generic interfaces in places in favor of directly using our boulder
specific interfaces.
Fixes#4427.
Aaron Gable