Bring in the release docs from the boulder-release-process repo,
so that they're adjacent to all of our other docs. This allows us to
delete that repo. Also update references to that repo to instead point
to the new doc here.
Also make minor organization updates to other docs to keep the root
of this repository clean.
@jsha suggested I re-implement a PR against Pebble regarding Authorization
reuse into Boulder (see https://github.com/letsencrypt/pebble/pull/325).
This is an initial attempt. I opted to handle this by creating a new file for
"Implementation Details" that are RFC conformant and are known to have
confused client developers.
This change reorganizes the document to have all changes
noted under their respective section headings, updates estimated
resolution dates on long-standing divergences, and updates all URLs
to reference the final RFC 8555 instead of various drafts.
In addition, it adds a note that we do not accept the (optional)
`notBefore` and `notAfter` fields of a `newOrder` request.
This ended up taking a lot more work than I expected. In order to make the implementation more robust a bunch of stuff we previously relied on has been ripped out in order to reduce unnecessary complexity (I think I insisted on a bunch of this in the first place, so glad I can kill it now).
In particular this change:
* Removes bhsm and pkcs11-proxy: softhsm and pkcs11-proxy don't play well together, and any softhsm manipulation would need to happen on bhsm, then require a restart of pkcs11-proxy to pull in the on-disk changes. This makes manipulating softhsm from the boulder container extremely difficult, and because of the need to initialize new on each run (described below) we need direct access to the softhsm2 tools since pkcs11-tool cannot do slot initialization operations over the wire. I originally argued for bhsm as a way to mimic a network attached HSM, mainly so that we could do network level fault testing. In reality we've never actually done this, and the extra complexity is not really realistic for a handful of reasons. It seems better to just rip it out and operate directly on a local softhsm instance (the other option would be to use pkcs11-proxy locally, but this still would require manually restarting the proxy whenever softhsm2-util was used, and wouldn't really offer any realistic benefit).
* Initializes the softhsm slots on each integration test run, rather than when creating the docker image (this is necessary to prevent churn in test/cert-ceremonies/generate.go, which would need to be updated to reflect the new slot IDs each time a new boulder-tools image was created since slot IDs are randomly generated)
* Installs softhsm from source so that we can use a more up to date version (2.5.0 vs. 2.2.0 which is in the debian repo)
* Generates the root and intermediate private keys in softhsm and writes out the root and intermediate public keys to /tmp for use in integration tests (the existing test-{ca,root} certs are kept in test/ because they are used in a whole bunch of unit tests. At some point these should probably be renamed/moved to be more representative of what they are used for, but that is left for a follow-up in order to keep the churn in this PR as related to the ceremony work as possible)
Another follow-up item here is that we should really be zeroing out the database at the start of each integration test run, since certain things like certificates and ocsp responses will be signed by a key/issuer that is no longer is use/doesn't match the current key/issuer.
Fixes#4832.
We can remove some divergences since in the V2 API:
* We use the correct error namespace.
* We use the correct `Content-Type` for Certificate requests.
* We use the correct `url` field in Challenge objects.
* We use the correct HTTP status code for Challenge POST requests.
* Remove the challenge whitelist
* Reduce the signature for ChallengesFor and ChallengeTypeEnabled
* Some unit tests in the VA were changed from testing TLS-SNI to testing the same behavior
in TLS-ALPN, when that behavior wasn't already tested. For instance timeouts during connect
are now tested.
Fixes#4109
This is primarily based on @rolandshoemaker's `readme-updates` branch and the closed PR: #3195
The README is restructured to be user-driven. Information that is strictly contributor focused (e.g. dep upgrades) is moved to CONTRIBUTING.md. The review feedback from #3195 from @jsha was all about text that we removed entirely in 29cdd78155
The README now links to the production deployment guide in the wiki and describes our general position on using Boulder.
The CONTRIBUTING.md guide links to the various docs/ pages and was updated.
The docs/acme-divergences page is updated for draft-15.
The DESIGN.md document is moved from the root of the repo into docs/. It has been updated to cover ACMEv2 and precertificates/SCT embedding.
Resolves https://github.com/letsencrypt/boulder/issues/3850
A frequent point of confusion is which ACME draft Boulder implements. Often people imagine (sensibly!) that there is one draft they can reference to understand Boulder.
This commit updates the divergences doc to clarify that it should be used to compare Boulder to whatever the most current ACME draft is and that Boulder doesn't implement a specific draft. This commit also adds a reference to what ACME v1 is and a link to the ACME v2 blog post.
Small references are also added to the "applications" concept from prev. drafts. Otherwise folks that land on older ACME drafts may wonder why the divergences doc doesn't mention "applications", a concept that was renamed to "orders" in subsequent drafts. We do document divergences for "orders" and attention should be directed there.
The IETF working group has published a [draft-06](https://tools.ietf.org/html/draft-ietf-acme-acme-07) and [draft-07 revision of ACME](https://tools.ietf.org/html/draft-ietf-acme-acme-07). This PR updates the Boulder `docs/acme-divergences.md` documentation for both drafts. Primarily this meant updating section numbers and links.
Notable updates:
* Added "index" directory Link divergence
* Removed divergence for "existing" field of authorizations - this was removed from the spec so it isn't a divergence anymore \o/
* Added divergence for the Boulder certificates endpoint not respecting client `Accept` headers and using the `application/pkix-cert` content type in responses vs `application/pem-certificate-chain`
* Added divergence for `unsupportedContact` and `accountDoesNotExist` errors.
* Added divergence for the `only-return-existing` field.
* Added divergence for retrying challenges
* Removed "meta" directory divergence since Boulder supports this now
Resolves#2825
This removes the config and code to output to statsd.
- Change `cmd.StatsAndLogging` to output a `Scope`, not a `Statter`.
- Remove the prefixing of component name (e.g. "VA") in front of stats; this was stripped by `autoProm` but now no longer needs to be.
- Delete vendored statsd client.
- Delete `MockStatter` (generated by gomock) and `mocks.Statter` (hand generated) in favor of mocking `metrics.Scope`, which is the interface we now use everywhere.
- Remove a few unused methods on `metrics.Scope`, and update its generated mock.
- Refactor `autoProm` and add `autoRegisterer`, which can be included in a `metrics.Scope`, avoiding global state. `autoProm` now registers everything with the `prometheus.Registerer` it is given.
- Change va_test.go's `setup()` to not return a stats object; instead the individual tests that care about stats override `va.stats` directly.
Fixes#2639, #2733.
Deletes github.com/streadway/amqp and the various RabbitMQ setup tools etc. Changes how listenbuddy is used to proxy all of the gRPC client -> server connections so we test reconnection logic.
+49 -8,221 😁Fixes#2640 and #2562.
This patch removes all usages of the `core.XXXError` and almost all usages of `probs` outside of the WFE and VA and replaces them with a unified internal error type. Since the VA uses `probs.ProblemDetails` quite extensively in challenges, and currently stores them in the DB I've saved this change for another change (it'll also require a migration). Since `ProblemDetails` should only ever be exposed to end-users all of its related logic should be moved into the `WFE` but since it still needs to be exposed to the VA and SA I've left it in place for now.
The new internal `errors` package offers the same convenience functions as `probs` does as well as a new simpler type testing method. A few small changes have also been made to error messages, mainly adding the library and function name to internal server errors for easier debugging (i.e. where a number of functions return the exact same errors and there is no other way to distinguish which method threw the error).
Also adds proper encoding of internal errors transferred over gRPC (the current encoding scheme is kept for `core` and `probs` errors since it'll be ideally be removed after we deploy this and follow-up changes) using `grpc/metadata` instead of the gRPC status codes.
Fixes#2507. Updates #2254 and #2505.
Prefer up over start to allow prometheus container to find boulder.
Use ocspMinTimeToExpiry: 0h trick instead of updating DB manually.
Offer command to fill DB.
Offer Prometheus link to throughput graph.
We have a number of stats already expressed using the statsd interface. During
the switchover period to direct Prometheus collection, we'd like to make those
stats available both ways. This change automatically exports any stats exported
using the statsd interface via Prometheus as well.
This is a little tricky because Prometheus expects all stats to by registered
exactly once. Prometheus does offer a mechanism to gracefully recover from
registering a stat more than once by handling a certain error, but it is not
safe for concurrent access. So I added a concurrency-safe wrapper that creates
Prometheus stats on demand and memoizes them.
In the process, made a few small required side changes:
- Clean "/" from method names in the gRPC interceptors. They are allowed in
statsd but not in Prometheus.
- Replace "127.0.0.1" with "boulder" as the name of our testing CT log.
Prometheus stats can't start with a number.
- Remove ":" from the CT-log stat names emitted by Publisher. Prometheus stats
can't include it.
- Remove a stray "RA" in front of some rate limit stats, since it was
duplicative (we were emitting "RA.RA..." before).
Note that this means two stat groups in particular are duplicated:
- Gostats* is duplicated with the default process-level stats exported by the
Prometheus library.
- gRPCClient* are duplicated by the stats generated by the go-grpc-prometheus
package.
When writing dashboards and alerts in the Prometheus world, we should be careful
to avoid these two categories, as they will disappear eventually. As a general
rule, if a stat is available with an all-lowercase name, choose that one, as it
is probably the Prometheus-native version.
In the long run we will want to create most stats using the native Prometheus
stat interface, since it allows us to use add labels to metrics, which is very
useful. For instance, currently our DNS stats distinguish types of queries by
appending the type to the stat name. This would be more natural as a label in
Prometheus.
Issue #2365 reported two places where we had divergences from ACME-04 in Boulder's implementation that were not reflected in the divergences doc. This PR documents:
1. That Boulder checks the `resource` field from the protected JWS header instead of the `url` field as described in Section 5.4.1
2. That Boulder uses a response with HTTP status code 409 (Conflict) when returning a Location header for an existing reg while Section 6.3 describes using HTTP status code 200 for this purpose.
This resolves#2365.
The current ACME specification allows certificates to be revoked by a account key for an account that holds valid authorizations for every name in the certificate to be revoked. This PR adds a branch to the existing wfe.RevokeCertificate method which checks if the account key holds the required authorizations if it isn't the key for the issuing account or the certificate key.
Fixes#2318.
Fixes#503.
Functionality is gated by the feature flag `AllowKeyRollover`. Since this functionality is only specified in ACME draft-03 and we mostly implement the draft-02 style this takes some liberties in the implementation, which are described in the updated divergences doc. The `key-change` resource is used to side-step draft-03 `url` requirement.
This PR adds a divergence to the acme-divergence doc for Section 6.6 "Certificate Revocation". Boulder does not currently support authorizing a revocation request using an arbitrary account key that is authorized for the same domains as in the certificate.
Aaron Gable
Jacob Hoffman-Andrews
Jacob Hoffman-Andrews
Jonathan Vanasco
Dennis Benzinger | SAP Hybris
Roland Bracewell Shoemaker
Daniel McCarney
Daniel McCarney
Tom Delmas
Roland Bracewell Shoemaker
Sophie Herold
Blake Griffith