Bumps
[github.com/prometheus/client_model](https://github.com/prometheus/client_model)
from 0.2.0 to 0.3.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="63fb9822ca"><code>63fb982</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/prometheus/client_model/issues/63">#63</a>
from prometheus/sparsehistogram</li>
<li><a
href="fdb567dcc1"><code>fdb567d</code></a>
Add note about native histograms to README</li>
<li><a
href="7f720d2282"><code>7f720d2</code></a>
Add note about experimental state of native histograms</li>
<li><a
href="1f8dcad122"><code>1f8dcad</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/prometheus/client_model/issues/59">#59</a>
from prometheus/beorn7/histogram</li>
<li><a
href="a7ff7138f2"><code>a7ff713</code></a>
Flatten the buckets of native histograms</li>
<li><a
href="421ad2b045"><code>421ad2b</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/prometheus/client_model/issues/58">#58</a>
from prometheus/beorn7/histogram</li>
<li><a
href="0da3265134"><code>0da3265</code></a>
Explain Span layout better</li>
<li><a
href="8171e83b1d"><code>8171e83</code></a>
Add float histograms and gauge histograms to proto spec</li>
<li><a
href="408689db4e"><code>408689d</code></a>
Merge branch 'master' into sparsehistogram</li>
<li><a
href="5c16fa2528"><code>5c16fa2</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/prometheus/client_model/issues/57">#57</a>
from prometheus/repo_sync</li>
<li>Additional commits viewable in <a
href="https://github.com/prometheus/client_model/compare/v0.2.0...v0.3.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Add two new config keys to the crl-updater:
* shardWidth, which controls the width of the chunks that we divide all
of time into, with a default value of "16h" (approximately the same as
today's shard width derived from 128 shards covering 90 days); and
* lookbackPeriod, which controls the amount of already-expired
certificates that should be included in our CRLs to ensure that even
certificates which are revoked immediately before they expire still show
up in aborts least one CRL, with a default value of "24h" (approximately
the same as today's lookback period derived from our run frequency of
6h).
Use these two new values to change the way CRL shards are computed.
Previously, we would compute the total time we care about based on the
configured certificate lifetime (to determine how far forward to look)
and the configured update period (to determine how far back to look),
and then divide that time evenly by the number of shards. However, this
method had two fatal flaws. First, if the certificate lifetime is
configured incorrectly, then the CRL updater will fail to query the
database for some certs that should be included in the CRLs. Second, if
the update period is changed, this would change the lookback period,
which in turn would change the shard width, causing all CRL entries to
suddenly change which shard they're in.
Instead, first compute all chunk locations based only on the shard width
and number of shards. Then determine which chunks we need to care about
based on the configured lookback period and by querying the database for
the farthest-future expiration, to ensure we cover all extant
certificates. This may mean that more than one chunk of time will get
mapped to a single shard, but that's okay -- each chunk will remain
mapped to the same shard for the whole time we care about it.
Fixes#6438Fixes#6440
Collapse most of our boilerplate gRPC creation steps (in particular,
creating default metrics, making the server and listener, registering
the server, creating and registering the health service, filtering
shutdown errors from the output, and gracefully stopping) into a single
function in the existing bgrpc package. This allows all but one of our
server main functions to drop their calls to NewServer and
NewServerMetrics.
To enable this, create a new helper type and method in the bgrpc
package. Conceptually, this could be just a new function, but it must be
attached to a new type so that it can be generic over the type of gRPC
server being created. (Unfortunately, the grpc.RegisterFooServer methods
do not accept an interface type for their second argument).
The only main function which is not updated is the boulder-va, which is
a special case because it creates multiple gRPC servers but (unlike the
CA) serves them all on the same port with the same server and listener.
Part of #6452
Add the Issuing Distribution Point extension to all of our end-entity
CRLs. The extension contains the Distribution Point, the URL from
which this CRL is meant to be downloaded. Because our CRLs are
sharded, this URL prevents an on-path attacker from substituting a
different shard than the client expected in order to hide a revocation.
The extension also contains the OnlyContainsUserCerts boolean,
because our CRLs only contain end-entity certificates.
The Distribution Point url is constructed from a configurable base URI,
the issuer's NameID, the shard index, and the suffix ".crl". The base
URI must use the "http://" scheme and must not end with a slash.
openssl displays the IDP extension as:
```
X509v3 Issuing Distribution Point: critical
Full Name:
URI:http://c.boulder.test/66283756913588288/0.crl Only User Certificates
```
Fixes#6410
Rather than simply refusing to write OCSP Response bytes to the
database (which is what ROCSP Stage 6 did), Stage 7 refuses to
even generate those bytes in the first place. We obviously can't
disable OCSP Response generation in the CA, since it still needs to
be usable by the ocsp-responder's live-signing path, so instead we
disable it in all of the non-live-signing codepaths (orphan finder,
issue precertificate, revoke certificate, and re-revoke certificate)
which have previously called GenerateOCSP.
Part of #6285
Clean up several spots where we were behaving differently on
go1.18 and go1.19, now that we're using go1.19 everywhere. Also
re-enable the lint and generate tests, and fix the various places where
the two versions disagreed on how comments should be formatted.
Also clean up the OldTLS codepaths, now that both go1.19 and our
own feature flags have forbidden TLS < 1.2 everywhere.
Fixes#6011
When the RA is generating OCSP (as part of new issuance, revocation,
or when its own GenerateOCSP method is called by the ocsp-responder)
have it use the CA's dedicated OCSPGenerator service, rather than
calling the method exposed by the CA's catch-all CertificateAuthority
service. To facilitate this, add a new GRPCClientConfig stanza to the
RA.
This change will allow us to remove the GenerateOCSP and GenerateCRL
methods from the catch-all CertificateAuthority service, allowing us to
independently control which kinds of objects the CA is willing to sign
by turning off individual service interfaces. The RA's new config stanza
will need to be populated in prod before further changes are possible.
Fixes#6451
These flags are set in both staging and prod. Deprecate them, make
all code gated behind them the only path, and delete code (multi_source)
which was only accessible when these flags were not set.
Part of #6285
Thing brings in a number of new lints, including those which check for
correct encoding of the KeyUsage bitstring, which has led to incidents
for a number of CAs recently.
This method simply returns the greatest notAfter timestamp in
the certificateStatus table. This will be used by the crl-updater
to ensure that it includes all unexpired certificates in its CRLs,
rather than only those which happen to fall within its configured
bounds.
Part of #6438
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.3.7 to 0.3.8.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.3.7...v0.3.8)
This also transitively updates x/tools and x/sync, which is good because those
are unversioned packages which are otherwise ignored by dependabot.
Note that we are not affected by the vulnerability which prompted the release
of version 0.3.8; the affected files are in the language subpackage which we
do not use or vendor.
Add a new configuration key to the CA which allows us to
specify the "base URL" for our CRLs. This will be necessary
before including an Issuing Distribution Point extension in our
CRLs, or a CRL Distribution Point in our certificates.
Part of #6410
Boulder builds a single binary which is symlinked to the different binary names, which are included in its releases.
However, requiring symlinks isn't always convenient.
This change makes the base `boulder` command usable as any of the other binary names. If the binary is invoked as boulder, runs the second argument as the command name. It shifts off the `boulder` from os.Args so that all the existing argument parsing can remain unchanged.
This uses the subcommand versions in integration tests, which I think is important to verify this change works, however we can debate whether or not that should be merged, since we're using the symlink method in production, that's what we want to test.
Issue #6362 suggests we want to move to a more fully-featured command-line parsing library that has proper subcommand support. This fixes one fragment of that, by providing subcommands, but is definitely nowhere near as nice as it could be with a more fully fleshed out library. Thus this change takes a minimal-touch approach to this change, since we know a larger refactoring is coming.
- Add a new field, `RetryAfter` to `BoulderError`s
- Add logic to wrap/unwrap the value of the `RetryAfter` field to our gRPC error
interceptor
- Plumb `RetryAfter` for `DuplicateCertificateError` emitted by RA to the WFE
client response header
Part of #6256
Add output listing the number of CRLs inspected, their total size in
bytes, and the total number of revocation entries seen. Also print the
oldest thisUpdate timestamp observed.
Allow the -issuer flag to take the special value "-", which causes no
issuer file to be loaded and no CRL signatures to be validated. Also
improve the error message if no -issuer flag is provided at all.
Fixes#6409
- Add a new gRPC client config field which overrides the dNSName checked in the
certificate presented by the gRPC server.
- Revert all test gRPC credentials to `<service>.boulder`
- Revert all ClientNames in gRPC server configs to `<service>.boulder`
- Set all gRPC clients in `test/config` to use `serverAddress` + `hostOverride`
- Set all gRPC clients in `test/config-next` to use `srvLookup` + `hostOverride`
- Rename incorrect SRV record for `ca` with port `9096` to `ca-ocsp`
- Rename incorrect SRV record for `ca` with port `9106` to `ca-crl`
Resolves#6424
Emit a metric (`version`) with a constant value of '1' labeled by the short
commit-id (`buildId`), build timestamp in RFC3339 format (`buildTime`), and Go
release tag like 'go1.3' (`goVersion`) from which the Boulder binary was built.
Resolves#6405
While downloading all of the CRLs in a JSON Array of Partitioned
CRLs, keep track of every serial seen so far, and report any dupes
as errors. The same serial should never appear in two different
shards of the same CRL.
Currently, we refuse to error out when checking the error from
S3, to ensure that we can update the metrics appropriately. This
requires us to use an unconventional error-checking structure,
and to check the error again when it comes time to return.
Instead, move the metrics above the error check, and then make
the error check a more traditional structure.
Add a -ageLimit flag to crl-checker, with a default value of 7d,
which causes it to log an error if any of the fetched CRLs have
a thisUpdate timestamp older than the given value.
Change the anchor time used to stabilize shard boundaries from
the zero time (time.Time{}) to the notBefore date of Let's Encrypt's
first self-signed root certificate.
This prevents us from attempting to compute durations that are
greater than 290 years, the maximum representable duration in
Go. Previously, using the zero time was causing our durations to
all be equal to the maximum duration, removing the utility of the
anchor time and causing our shard boundaries to drift with time.
Explicitly inform go vet about the names of our logging methods
which should be checked in the same way as fmt.Printf is. Although
go vet can often find such functions on its own, it can't find these
ones because log.Logger is an interface, not a struct.
In addition, fix several format string mistakes caught by go vet.
Previously, we would stream CRL Entries directly from the SA's response
stream into the CA's request stream, and similarly directly stream bytes
from the CA's response stream into the Storer's request stream.
Since we're seeing odd errors and inconsistencies in our gRPC streaming
metrics, simplify these to only conduct one stream at a time. This will
make our streaming and error semantics much simpler, at the cost of
memory usage in the updater.
Fix instances where an error check was conditioned on something
other than the traditional `err`, such as `myStruct.err`, but then the
error being logged was the `err` from elsewhere in the function.
This uncovered a bug! The stdout logger was truncating the microseconds part
of its timestamp if the last digit was zero. Fixed that. Also coerced the
stdout logger to use UTC.
To run the checker over our integration test logs, I changed t.sh to use
an explicit name for the container that runs boulder during the tests,
and pulled logs from that container after the tests.
Previously, the reloader relied on user code to log errors and changes.
But this led to gaps (for instance the RA not logging reloads of the rate
limit file). Instead, make the reloader responsible for this.
This allowed removing the error callback, at the minor cost of removing
the status gauge for the ECDSA allowlist.
- Fork the default `dns` resolver from `go-grpc` to add backend discovery via
DNS SRV resource records.
- Add new fields for SRV based discovery to `cmd.GRPCClientConfig`
- Add new (optional) field `DNSAuthority` for specifying custom DNS server to
`cmd.GRPCClientConfig`
- Add a utility method to `cmd.GRPCClientConfig` to simplify target URI and host
construction. With three schemes and `DNSAuthority` it makes more sense to
handle all of this parsing and construction outside of the RPC client
constructor.
Resolves#6111
I got some URLs wrong when using this tool, which resulted in confusing errors
as the error bodies tried to get parsed as a CRL. As well, the errors had
a stray \n at the end which I've also removed. The logger interface escapes
newlines, and already includes one at the end of the line.
- Add a dedicated Consul container
- Replace `sd-test-srv` with Consul
- Add documentation for configuring Consul
- Re-issue all gRPC credentials for `<service-name>.service.consul`
Part of #6111
According to the documentation for certificateRequestEvent:
// CommonName is the subject common name from the issued cert
CommonName string `json:",omitempty"`
// Names are the DNS SAN entries from the issued cert
Names []string `json:",omitempty"`
We were erroneously setting these based on the CSR, not the issued
certificate.
- Add subcommand `incident-table-revoke` to `admin-revoker`
- Implement streaming RPC adapter for `SerialsForIncident()` in `/test/inmem/sa`
- Refactor the `admin-revoker` tests to use shared setup functions and methods
Resolves#6332
This PR is a follow-up to #6277 and #6290 to add a new prober type to
boulder-observer for monitoring CRLs, making use of the new prober-specific
metrics capability to define the following new metrics:
- `obs_crl_this_update` the Unix timestamp of the CRL's thisUpdate value
- `obs_crl_next_update` the Unix timestamp of the CRL's nextUpdate value
- `obs_crl_revoked_cert_count` the number of certificates listed in the CRL
**Configuration:** Each defined CRL monitor takes a single configuration option,
a URL that specifies the location of the CRL to monitor.
**Metrics:** The three CRL-specific metrics described above are only published
at /metrics if at least one valid monitor is defined in the config.yml. The
metrics have a single label `url` that is set to the URL configured for the
monitor
Add crl-checker, a simple tool which downloads, parses, lints,
and validates signatures on a list of CRLs. It takes its input in
the form of a JSON Array of Sharded CRL URLs, the exact
same format as we will be disclosing in CCADB.
We can add additional checks -- such as ensuring that a set of
known-revoked serials are present, or checking that all of the
downloaded CRLs are "recent enough" -- over time.
Modify the way errors are handled in crl-updater:
- Rather than having each method in the tick, tickIssuer, tickShard
chain concatenate all errors from its children, simply have them
summarize the errors. This results in much shorter error messages.
- Rather than having each method log its own errors as it returns, have
each caller responsible for logging the errors it receives from its
children.
In addition, add tests for tick, tickIssuer, and tickShard which cover
their simple errors paths, where one of the gRPC requests to the SA, CA,
or CRLStorer encounters an error. These tests let us ensure that errors
are being properly propagated upwards through the layers of indirection
and goroutines in the three methods under test, and that the appropriate
metrics are being incremented and log messages are being printed.
Fixes#6375
dependabot[bot]
Aaron Gable
David
Jacob Hoffman-Andrews
Matthew McPherrin
Samantha
Preston Locke