62 lines
3.9 KiB
Markdown
62 lines
3.9 KiB
Markdown
# Multi-VA implementation
|
|
|
|
Boulder supports a multi-perspective validation feature intended to increase
|
|
resilience against local network hijacks and BGP attacks. It is currently
|
|
[deployed in a production
|
|
capacity](https://letsencrypt.org/2020/02/19/multi-perspective-validation.html)
|
|
by Let's Encrypt.
|
|
|
|
If you follow the [Development Instructions](https://github.com/letsencrypt/boulder#development)
|
|
to set up a Boulder environment in Docker and then change your `docker-compose.yml`'s
|
|
`BOULDER_CONFIG_DIR` to `test/config-next` instead of `test/config` you'll have
|
|
a Boulder environment configured with two primary VA instances (validation
|
|
requests are load balanced across the two) and two remote VA instances (each
|
|
primary VA will ask both remote VAs to perform matching validations for each
|
|
primary validation). Of course this is a development environment so both the
|
|
primary and remote VAs are all running on one host.
|
|
|
|
The primary and remote VAs are both the same piece of software, the `boulder-va`
|
|
service ([cmd here](https://github.com/letsencrypt/boulder/tree/main/cmd/boulder-va),
|
|
[package here](https://github.com/letsencrypt/boulder/tree/main/va)).
|
|
The boulder-ra uses [the same RPC interface](https://github.com/letsencrypt/boulder/blob/ea231adc36746cce97f860e818c2cdf92f060543/va/proto/va.proto#L8-L10)
|
|
to ask for a primary validation as the primary VA uses to ask a remote VA for a
|
|
confirmation validation.
|
|
|
|
Primary VA instances know they are a primary based on the presence of the
|
|
`"remoteVAs"` configuration element. If present it specifies gRPC service
|
|
addresses for other VA instances to use as remotes. There's also a handful of
|
|
feature flags that control how the primary VAs handle the remote VAs.
|
|
|
|
In the development environment with `config-next` the two primary VAs are `va1.service.consul:9092` and
|
|
`va2.service.consul:9092` and use
|
|
[`test/config-next/va.json`](https://github.com/letsencrypt/boulder/blob/ea231adc36746cce97f860e818c2cdf92f060543/test/config-next/va.json)
|
|
as their configuration. This config file specifies two `"remoteVA"s`,
|
|
`rva1.service.consul:9097` and `va2.service.consul:9098` and enforces
|
|
[that a maximum of 1 of the 2 remote VAs disagree](https://github.com/letsencrypt/boulder/blob/ea231adc36746cce97f860e818c2cdf92f060543/test/config-next/va.json#L44)
|
|
with the primary VA for all validations. The remote VA instances use
|
|
[`test/config-next/va-remote-a.json`](https://github.com/letsencrypt/boulder/blob/ea231adc36746cce97f860e818c2cdf92f060543/test/config-next/va-remote-a.json)
|
|
and
|
|
[`test/config-next/va-remote-b.json`](https://github.com/letsencrypt/boulder/blob/ea231adc36746cce97f860e818c2cdf92f060543/test/config-next/va-remote-b.json)
|
|
as their config files.
|
|
|
|
There are two feature flags that control whether multi-VA takes effect:
|
|
MultiVAFullResults and EnforceMultiVA. If MultiVAFullResults is enabled
|
|
then each primary validation will also send out remote validation requests, and
|
|
wait for all the results to come in, so we can log the results for analysis. If
|
|
EnforceMultiVA is enabled, we require that almost all remote validation requests
|
|
succeed. The primary VA's "maxRemoteValidationFailures" config field specifies
|
|
how many remote VAs can fail before the primary VA considers overall validation
|
|
a failure. It should be strictly less than the number of remote VAs.
|
|
|
|
Validation is also controlled by the "multiVAPolicyFile" config field on the
|
|
primary VA. This specifies a file that can contain temporary overrides for
|
|
domains or accounts that fail under multi-va. Over time those temporary
|
|
overrides will be removed.
|
|
|
|
There are some integration tests that test this end to end. The most relevant is
|
|
probably
|
|
[`test_http_multiva_threshold_fail`](https://github.com/letsencrypt/boulder/blob/ea231adc36746cce97f860e818c2cdf92f060543/test/v2_integration.py#L876-L908).
|
|
It tests that a HTTP-01 challenge made to a webserver that only gives the
|
|
correct key authorization to the primary VA and not the remotes will fail the
|
|
multi-perspective validation.
|