For the upcoming SCT embedding changes, it will be useful to have a CT test server that blocks for nontrivial amounts of time before responding. This change introduces a config file for `ct-test-srv` that can be used to set up multiple "personalities" on various ports. Each personality can have a "latencySchedule" that determines how long it will sleep before servicing responding to a submission.
This change also introduces two new "personalities" on :4510 and :4511, plus configures CTLogGroups in the RA. Having four CT log personalities allows us to simulate two nontrivial log groups.
Note: This triggers Publisher to emit audit errors on timed-out submissions. We may want to make Publisher not treat those as errors, and instead only log an error if a whole log group fails.
Prior to this commit a logical error in the RA's `NewOrder` caused
a safety check that prevents authorization reuse with a non-wildcard
authz for a wildcard name to not work. This commit adds a test for the
condition that the safety check is designed for and fixes the logical
error. Prior to fixing the logical error the test fails. With the
corrected safety check the test passes.
Prior to this commit when building up the authorizations for a new-order
request we looked for any unexpired pending/valid authorizations owned
by the account and used them for the order. This allows a client to use
the V1 new-authz endpoint in combination with the V2 new-order endpoint
and we do not want to support this behaviour. All V2 authorizations
should be sourced from other V2 orders. This commit implements a new
parameter for the SA's getAuthorizations function that allows filtering
out legacy V1 authorizations by doing a JOIN on the order to
authorizations join table.
Resolves#3328
This commit resolves the case where an error during finalization occurs.
Prior to this commit if an error (expected or otherwise) occurred after
setting an order to status processing at the start of order
finalization the order would be stuck processing forever.
The SA now has a `SetOrderError` RPC that can be used by the RA to
persist an error onto an order. The order status calculation can use
this error to decide if the order is invalid. The WFE is updated to
write the error to the order JSON when displaying the order information.
Prior to this commit the order protobuf had the error field as
a `[]byte`. It doesn't seem like this is the right decision, we have
a specific protobuf type for ProblemDetails and so this commit switches
the error field to use it. The conversion to/from `[]byte` is done with
the model by the SA.
An integration test is included that prior to this commit left an order
in a stuck processing state. With this commit the integration test
passes as expected.
Resolves https://github.com/letsencrypt/boulder/issues/3403
The SA RPC previously called `GetOrderAuthorizations` only returns
**valid, unexpired** authorizations. This commit updates the name to
emphasize that it only returns valid order authzs.
Previously, all errors were treated as Not Found, but we actually want
to treat database errors differently; for instance, by not caching them,
and by setting tighter alerting thresholds for them.
Fixes#3419.
This PR is a rework of what was originally https://github.com/letsencrypt/boulder/pull/3382, integrating the design feedback proposed by @jsha: https://github.com/letsencrypt/boulder/pull/3382#issuecomment-359912549
This PR removes the stored Order status field and replaces it with a value that is calculated on-the-fly by the SA when fetching an order, based on the order's associated authorizations.
In summary (and order of precedence):
* If any of the order's authorizations are invalid, the order is invalid.
* If any of the order's authorizations are deactivated, the order is deactivated.
* If any of the order's authorizations are pending, the order is pending.
* If all of the order's authorizations are valid, and there is a certificate serial, the order is valid.
* If all of the order's authorizations are valid, and we have began processing, but there is no certificate serial, the order is processing.
* If all of the order's authorizations are valid, and we haven't processing, then the order is pending waiting a finalization request.
This avoids having to explicitly update the order status when an associated authorization changes status.
The RA's implementation of new-order is updated to only reuse an existing order if the calculated status is pending. This avoids giving back invalid or deactivated orders to clients.
Resolves#3333
Update CFSSL to get upstream ocsp changes required to minimize log
volume.
Confirmed that unit tests pass:
```
$ git rev-parse HEAD
ed5223a490ece4d66899bbb292e3e46c0677cb86
$> go test ./...
ok github.com/cloudflare/cfssl/api 0.009s
ok github.com/cloudflare/cfssl/api/bundle 0.811s
ok github.com/cloudflare/cfssl/api/certadd 6.735s
? github.com/cloudflare/cfssl/api/certinfo [no test files]
ok github.com/cloudflare/cfssl/api/client 0.069s
ok github.com/cloudflare/cfssl/api/crl 0.103s
ok github.com/cloudflare/cfssl/api/gencrl 0.008s
ok github.com/cloudflare/cfssl/api/generator 0.051s
ok github.com/cloudflare/cfssl/api/info 0.027s
ok github.com/cloudflare/cfssl/api/initca 0.022s
ok github.com/cloudflare/cfssl/api/ocsp 0.026s
ok github.com/cloudflare/cfssl/api/revoke 0.614s
ok github.com/cloudflare/cfssl/api/scan 51.888s
ok github.com/cloudflare/cfssl/api/sign 0.329s
ok github.com/cloudflare/cfssl/api/signhandler 0.056s
ok github.com/cloudflare/cfssl/auth 0.002s
ok github.com/cloudflare/cfssl/bundler 7.864s
? github.com/cloudflare/cfssl/certdb [no test files]
ok github.com/cloudflare/cfssl/certdb/dbconf 0.003s
ok github.com/cloudflare/cfssl/certdb/ocspstapling 1.103s
ok github.com/cloudflare/cfssl/certdb/sql 0.369s
? github.com/cloudflare/cfssl/certdb/testdb [no test files]
? github.com/cloudflare/cfssl/certinfo [no test files]
ok github.com/cloudflare/cfssl/cli 0.003s
ok github.com/cloudflare/cfssl/cli/bundle 0.003s [no tests to run]
? github.com/cloudflare/cfssl/cli/certinfo [no test files]
ok github.com/cloudflare/cfssl/cli/crl 0.061s
ok github.com/cloudflare/cfssl/cli/gencert 1.518s
ok github.com/cloudflare/cfssl/cli/gencrl 0.011s
ok github.com/cloudflare/cfssl/cli/gencsr 0.010s
ok github.com/cloudflare/cfssl/cli/genkey 0.583s
? github.com/cloudflare/cfssl/cli/info [no test files]
? github.com/cloudflare/cfssl/cli/ocspdump [no test files]
ok github.com/cloudflare/cfssl/cli/ocsprefresh 0.068s
? github.com/cloudflare/cfssl/cli/ocspserve [no test files]
? github.com/cloudflare/cfssl/cli/ocspsign [no test files]
? github.com/cloudflare/cfssl/cli/printdefault [no test files]
ok github.com/cloudflare/cfssl/cli/revoke 0.092s
ok github.com/cloudflare/cfssl/cli/scan 0.003s
ok github.com/cloudflare/cfssl/cli/selfsign 0.648s
ok github.com/cloudflare/cfssl/cli/serve 0.016s
ok github.com/cloudflare/cfssl/cli/sign 0.041s
ok github.com/cloudflare/cfssl/cli/version 0.003s
ok github.com/cloudflare/cfssl/cmd/cfssl 0.005s [no tests to run]
? github.com/cloudflare/cfssl/cmd/cfssl-bundle [no test files]
? github.com/cloudflare/cfssl/cmd/cfssl-certinfo [no test files]
? github.com/cloudflare/cfssl/cmd/cfssl-newkey [no test files]
? github.com/cloudflare/cfssl/cmd/cfssl-scan [no test files]
ok github.com/cloudflare/cfssl/cmd/cfssljson 0.012s
ok github.com/cloudflare/cfssl/cmd/mkbundle 0.011s [no tests
to run]
? github.com/cloudflare/cfssl/cmd/multirootca [no test files]
ok github.com/cloudflare/cfssl/config 0.004s
ok github.com/cloudflare/cfssl/crl 0.013s
? github.com/cloudflare/cfssl/crypto [no test files]
? github.com/cloudflare/cfssl/crypto/pkcs7 [no test files]
ok github.com/cloudflare/cfssl/csr 4.836s
ok github.com/cloudflare/cfssl/errors 0.004s
ok github.com/cloudflare/cfssl/helpers 0.037s
? github.com/cloudflare/cfssl/helpers/derhelpers [no test files]
ok github.com/cloudflare/cfssl/helpers/testsuite 4.830s
? github.com/cloudflare/cfssl/info [no test files]
ok github.com/cloudflare/cfssl/initca 17.794s
ok github.com/cloudflare/cfssl/log 0.002s
ok github.com/cloudflare/cfssl/multiroot/config 0.022s
ok github.com/cloudflare/cfssl/ocsp 0.119s
? github.com/cloudflare/cfssl/ocsp/config [no test files]
? github.com/cloudflare/cfssl/ocsp/universal [no test files]
ok github.com/cloudflare/cfssl/revoke 2.172s
ok github.com/cloudflare/cfssl/scan 0.003s
? github.com/cloudflare/cfssl/scan/vendor/crypto [no test files]
? github.com/cloudflare/cfssl/scan/vendor/crypto/md5 [no test
files]
? github.com/cloudflare/cfssl/scan/vendor/crypto/rsa [no test
files]
? github.com/cloudflare/cfssl/scan/vendor/crypto/sha1 [no test
files]
? github.com/cloudflare/cfssl/scan/vendor/crypto/sha256 [no test
files]
? github.com/cloudflare/cfssl/scan/vendor/crypto/sha512 [no test
files]
? github.com/cloudflare/cfssl/scan/vendor/crypto/tls [no test
files]
ok github.com/cloudflare/cfssl/selfsign 0.011s
ok github.com/cloudflare/cfssl/signer 0.003s
ok github.com/cloudflare/cfssl/signer/local 0.419s
ok github.com/cloudflare/cfssl/signer/remote 0.341s
ok github.com/cloudflare/cfssl/signer/universal 0.262s
ok github.com/cloudflare/cfssl/transport 0.017s
? github.com/cloudflare/cfssl/transport/ca [no test files]
ok github.com/cloudflare/cfssl/transport/ca/localca 0.020s
ok github.com/cloudflare/cfssl/transport/core 0.021s
? github.com/cloudflare/cfssl/transport/example/exlib [no test
files]
? github.com/cloudflare/cfssl/transport/example/maclient [no test
files]
? github.com/cloudflare/cfssl/transport/example/maserver [no test
files]
ok github.com/cloudflare/cfssl/transport/kp 0.021s
? github.com/cloudflare/cfssl/transport/roots [no test files]
? github.com/cloudflare/cfssl/transport/roots/system [no test
files]
ok github.com/cloudflare/cfssl/ubiquity 0.012s
ok github.com/cloudflare/cfssl/whitelist 0.086s
? github.com/cloudflare/cfssl/whitelist/example [no test files]
```
Fixes#3368.
Basically just adds a `csr.VerifyCSR` call in `ra.FinalizeOrder` that mirrors what we have in `ra.NewCertificate`, this moves the CN to SAN as expected if included.
This commit implements a mapping from certificate AIA Issuer URL to PEM
encoded certificate chain. GET's to the V2 Certificate endpoint will
return a full PEM encoded certificate chain in addition to the leaf cert
using the AIA issuer URL of the leaf cert and the configured mapping.
The boulder-wfe2 command builds the chain mapping by reading the
"wfe" config section's 'certificateChains" field, specifying a list
of file paths to PEM certificates for each AIA issuer URL. At startup
the PEM file contents are ready, verified and separated by a newline.
The resulting populated AIA issuer URL -> PEM cert chain mapping is
given to the WFE for use with the Certificate endpoint.
Resolves#3291
The WFE2 doesn't check any of the feature flags that are configured in
the `test/config/wfe2.json` and `test/config-next/wfe2.json` config
files - we default to acting as if all new features are enabled for the
V2 work. This commit removes the flags from the config to avoid
confusion or expectations that changing the config will disable the
features.
This commit removes `CertCacheDuration`, `CertNoCacheExpirationWindow`,
`IndexCacheDuration` and `IssuerCacheDuration`. These were read from
config values that weren't set in config/config-next into WFE struct
fields that were never referenced in any code.
Per
https://community.letsencrypt.org/t/2018-01-11-update-regarding-acme-tls-sni-and-shared-hosting-infrastructure/50188/3,
we are planning to treat prior issuance by an account as reason to whitelist
that account for reissuance via TLS-SNI. By extension, reusing validations that
occurred prior to disclosure of the TLS-SNI issue is reasonably safe, so this
change removes the issuance-time check for whether a challenge has been
disabled. This saves us significant complexity and database load in implementing
TLSSNIRevalidation (https://github.com/letsencrypt/boulder/pull/3361), since
ChallengeTypeEnabled returns false, so we'd have to plumb through data about
whether an issuance was based on a revalidation. Instead, we can safely delete
this code.
Note that "EnforceChallengeDisable" is implemented in three places: new-authz,
validation time, and issuance time. We're keeping it in place at new-authz for
now because it's intertwined with the account whitelisting code. We're keeping
it in place at validation time, because there's a small chance that someone
could have created a pending authz for a domain they don't control before the
TLS-SNI issue was announced, and that authz could still be pending, and they
could find out that that domain is hosted on a vulnerable provider, and use the
vulnerability now that they know about it. A tiny chance, but may as well be
careful.
This change adds a feature flag, TLSSNIRevalidation. When it is enabled, Boulder
will create new authorization objects with TLS-SNI challenges if the requesting
account has issued a certificate with the relevant domain name, and was the most
recent account to do so*. This setting overrides the configured list of
challenges in the PolicyAuthority, so even if TLS-SNI is disabled in general, it
will be enabled for revalidation.
Note that this interacts with EnforceChallengeDisable. Because
EnforceChallengeDisable causes additional checked at validation time and at
issuance time, we need to update those two places as well. We'll send a
follow-up PR with that.
*We chose to make this work only for the most recent account to issue, even if
there were overlapping certificates, because it significantly simplifies the
database access patterns and should work for 95+% of cases.
Note that this change will let an account revalidate and reissue for a domain
even if the previous issuance on that account used http-01 or dns-01. This also
simplifies implementation, and fits within the intent of the mitigation plan: If
someone previously issued for a domain using http-01, we have high confidence
that they are actually the owner, and they are not going to "steal" the domain
from themselves using tls-sni-01.
Also note: This change also doesn't work properly with ReusePendingAuthz: true.
Specifically, if you attempted issuance in the last couple days and failed
because there was no tls-sni challenge, you'll still have an http-01 challenge
lying around, and we'll reuse that; then your client will fail due to lack of
tls-sni challenge again.
This change was joint work between @rolandshoemaker and @jsha.
This updates the PA component to allow authorization challenge types that are globally disabled if the account ID owning the authorization is on a configured whitelist for that challenge type.
This patch does three things:
* Prevent the use of a authorization for issuance that was validated using a disabled challenge type
* Don't reuse a authorization that was validated using a disabled challenge type
* Don't allow validation using a disabled validation type
And adds tests for all three cases.
It also factors out the challenge-fetching code common to several SA functions into
a new getChallenges function, and adds a call to getChallenges as part of getAuthorizations.
Jacob Hoffman-Andrews