Prior to this commit the `Rollback` function always wrapped the provided
error in a `sa.RollbackError`. This makes it difficult for callers to
test the type of the original error. This commit updates the `Rollback` function to only
return a `sa.RollbackError` when the call to `tx.Rollback()` produces an error.
This is a followup from https://github.com/letsencrypt/boulder/pull/3017, in
which we identified a data race caused by the use of named returns. This also
reverts the change from that PR, which was only a surface level fix.
Fixes#3019.
Travis only allows us 5 simultaneous build jobs, so going from 6 to 5 jobs per
build should reduce the wall time required to get a CI result on any given
branch.
Fixes#2889.
VA now implements two gRPC services: VA and CAA. These both run on the same port, but this allows implementation of the IsCAAValid RPC to skip using the gRPC wrappers, and makes it easier to potentially separate the service into its own package in the future.
RA.NewCertificate now checks the expiration times of authorizations, and will call out to VA to recheck CAA for those authorizations that were not validated recently enough.
va.go is quite a large file. This splits out the CAA-related code and tests into its own file for simplicity. This is a simple move; no code has been changed, and there is no package split.
They used to be a millisecond, which remarkably worked most of the time.
However, some fraction of DNS requests would fail and need to be retried. Even
successful integration test runs had a number of such failures, but retries
generally saved them. However, sometimes all of the retries for a given lookup
would fail, leading to a failure of the overall lookup. This typically
manifested as an error looking up CAA, because our integration tests look up CAA
much more frequently than other record types.
This appears to fix our integration test flakiness.
The VA test had a global:
`var ident = core.AcmeIdentifier{Type: core.IdentifierDNS, Value: "localhost"}`
Evidently this was meant as a convenience to avoid having to retype this common value, but it wound up being mutated independently by different tests. This PR replaces it with a convenience function `dnsi()` that generates a DNS-type identifier with the given hostname. Makes the VA test much more reliable locally.
The ACME specification no longer describes "registrations" since this is
a fairly overloaded term. Instead the term used is "account". This
commit updates the WFE2 & tests throughout to replace occurrences of
"reg" and "registration" to use "acct" and "account".
NOTE: This change is strictly limited to the wfe2 package. E.g. the
RA/SA and many core objects still refer to registrations.
Resolves#2986
This commit updates the `github.com/google/safebrowsing` dependency to
commit f387af, the tip of master at the time of writing.
Unit tests were confirmed to pass per CONTRIBUTING.md:
```
$ go test ./...
ok github.com/google/safebrowsing 2.500s
? github.com/google/safebrowsing/cmd/sblookup [no test files]
? github.com/google/safebrowsing/cmd/sbserver [no test files]
? github.com/google/safebrowsing/cmd/sbserver/statik [no test files]
? github.com/google/safebrowsing/internal/safebrowsing_proto [no test files]
```
This commit implements certificate revocation for the WFE2. This
endpoint differs from others in that it supports *both* traditional key
ID based JWS request authentication in addition to embedded JWK based
JWS request authentication. The first is considered authenticated to
revoke a certificate if the signer account has valid authorizations for
all of the names in the certificate. The second is considered
authenticated if the embedded JWK that signs the request has the same
public key as the certificate being revoked.
Per #3001 we should not be adding new StatsD code for metrics anymore.
This commit updates all of the WFE2 to use 1st class Prometheus stats.
Unit tests are updated accordingly.
I have broken the error stats into two counts:
1. httpErrorCount for all of the http layer client request errors (e.g.
no POST body, no content-length)
2. joseErrorCount, for all of the JOSE layer client request errors (e.g.
malformed JWS, broken signature, invalid JWK)
This commit also removes the stubbed out `TestValidKeyRollover` function
from `wfe2/verify_test.go`. This was committed accidentally and the same
functionality is covered by the `wfe2/wfe_test.go` `TestKeyRollover`
function.
In ca/certificate-authority.go, we previously had a block list of signature algorithms we will not accept for CSRs. This commit switches to an allowed list of signature algorithms that we will accept.
RFC 7515 section 7.2.1 "General JWS JSON Serialization Syntax" describes
an optional "signatures" field that contains an array of JSON objects,
each representing a signature or MAC. ACME only uses the mandatory
"signature" field that contains the BASE64URL of a signature.
We previously checked that the parsed JWS had only one signature and
rejected accordingly but in order to be safe and ensure that nothing is
read from this "signatures" array when we intended to be using the
"signature" field this commit updates the check to explicitly reject the
"signatures" field prior to parsing with go-jose similar to how the
unprotected header is handled.
Stub out IssueCertificateForPrecertificate() enough so that we can continue with the PRs that implement & test it in parallel with PRs that implement and test the calling side (via mock implementations of the CA side).
This is the second half of a clean-up of the WFE2 unit tests that were copied over from the original WFE implementation.
This PR covers TestChallenge, TestAuthorization, and TestGetCertificate.
Resolves#2928
This is required by RFC 5890, which is not explicitly required by the BRs _at the moment_ but prepares us for a world where RFC 5280 or the BRs are updated to refer to the most recent IDNA RFC (and is general best practice).
Fixes#2885.
This commit updates the VA's `IsSafeDomain` RPC to treat errors from the
Google Safe Browsing client as a positive response. Subsequently the VA
will only block authz creation in the case that the GSB API returns
a true negative (e.g. confirms an unsafe domain). If the database is in
an inconsistent state due to an API outage we will allow the authz to be
created.
This is the first half of a clean-up of the WFE2 unit tests that were copied over from the original WFE implementation.
I will file a follow-up pt2 PR for TestChallenge, TestAuthorization, and TestGetCert which I think are the remaining tests that could use a 🛁.
The one non-test commit changed the WFE2 index to return a problem when the method is unsupported similar to the other API endpoints. This might be inappropriate since normally the index returns XHTML and not JSON. It made testing easier but I'm open to switching back to returning a "" body and special casing the index test.
Note to reviewers: The main diff is hidden by GH by default and needs to be expanded.
Updates #2928
This commit updates the publicsuffix-go dependency to
6787cd3b348b18fab6371264ae5392cd8eca1723 the tip of master at the time
of writing.
The unit tests were verified to pass:
```
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.006s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.024s
```
RFC 5280 incorporates RFC 3490 by reference. RFC 3490 requires using the KC normalization form, which therefore is required by the BRs. There was some confusion in #2964 as RFC 3490 was obsoleted by RFC 5890 but since the BRs simply reference RFC 5280 this doesn't matter. This PR fixes the confusion.
Godep apparently breaks when trying to parse code that specifies build tags for versions of golang above that with which it was built (which it shouldn't be parsing in the first place). This breaks the travis tests since `golang.org/x/net/context` now contains golang 1.9 specific code. In order to get around this we temporarily disable the error check for `godep save ./...` in test.sh. Opened #2965 to revert this once Godep is fixed or we move to golang 1.9.
Requires an update to `golang.org/x/net` and adding `golang.org/x/text`.
```
[roland@niya:~/gopath/src/golang.org/x/net]$ go test ./...
ok golang.org/x/net/bpf 0.472s
ok golang.org/x/net/context 0.090s
ok golang.org/x/net/context/ctxhttp 0.161s
? golang.org/x/net/dict [no test files]
ok golang.org/x/net/dns/dnsmessage 0.044s
ok golang.org/x/net/html 0.094s
ok golang.org/x/net/html/atom 0.003s
ok golang.org/x/net/html/charset 0.027s
ok golang.org/x/net/http2 80.253s
? golang.org/x/net/http2/h2i [no test files]
ok golang.org/x/net/http2/hpack 0.064s
ok golang.org/x/net/icmp 0.026s
ok golang.org/x/net/idna 0.035s
? golang.org/x/net/internal/iana [no test files]
? golang.org/x/net/internal/nettest [no test files]
ok golang.org/x/net/internal/socket 0.005s
ok golang.org/x/net/internal/timeseries 0.024s
ok golang.org/x/net/ipv4 0.013s
ok golang.org/x/net/ipv6 0.036s
ok golang.org/x/net/lex/httplex 0.004s
ok golang.org/x/net/nettest 1.164s
ok golang.org/x/net/netutil 0.898s
ok golang.org/x/net/proxy 0.004s
ok golang.org/x/net/publicsuffix 0.202s
ok golang.org/x/net/trace 0.018s
ok golang.org/x/net/webdav 0.061s
ok golang.org/x/net/webdav/internal/xml 0.014s
ok golang.org/x/net/websocket 0.022s
ok golang.org/x/net/xsrftoken 0.025s
[roland@niya:~/gopath/src/golang.org/x/text]$ go test ./...
? golang.org/x/text [no test files]
ok golang.org/x/text/cases 0.439s
? golang.org/x/text/cmd/gotext [no test files]
ok golang.org/x/text/collate 0.038s
ok golang.org/x/text/collate/build 0.024s
? golang.org/x/text/collate/tools/colcmp [no test files]
ok golang.org/x/text/currency 2.961s
ok golang.org/x/text/encoding 0.005s
ok golang.org/x/text/encoding/charmap 0.060s
ok golang.org/x/text/encoding/htmlindex 0.005s
ok golang.org/x/text/encoding/ianaindex 0.030s
? golang.org/x/text/encoding/internal [no test files]
? golang.org/x/text/encoding/internal/enctest [no test files]
? golang.org/x/text/encoding/internal/identifier [no test files]
ok golang.org/x/text/encoding/japanese 0.098s
ok golang.org/x/text/encoding/korean 0.032s
ok golang.org/x/text/encoding/simplifiedchinese 0.100s
ok golang.org/x/text/encoding/traditionalchinese 0.012s
ok golang.org/x/text/encoding/unicode 0.013s
ok golang.org/x/text/encoding/unicode/utf32 0.071s
ok golang.org/x/text/feature/plural 0.352s
ok golang.org/x/text/internal 0.009s
ok golang.org/x/text/internal/catmsg 0.034s
ok golang.org/x/text/internal/colltab 1.817s
ok golang.org/x/text/internal/export/idna 0.040s
? golang.org/x/text/internal/format [no test files]
? golang.org/x/text/internal/gen [no test files]
ok golang.org/x/text/internal/number 0.028s
ok golang.org/x/text/internal/stringset 0.021s
ok golang.org/x/text/internal/tag 0.044s
? golang.org/x/text/internal/testtext [no test files]
ok golang.org/x/text/internal/triegen 0.357s
ok golang.org/x/text/internal/ucd 0.023s
? golang.org/x/text/internal/utf8internal [no test files]
ok golang.org/x/text/language 0.033s
ok golang.org/x/text/language/display 3.917s
ok golang.org/x/text/message 0.033s
ok golang.org/x/text/message/catalog 0.069s
ok golang.org/x/text/runes 0.039s
ok golang.org/x/text/search 0.019s
? golang.org/x/text/secure [no test files]
ok golang.org/x/text/secure/bidirule 0.032s
ok golang.org/x/text/secure/precis 0.066s
ok golang.org/x/text/transform 0.106s
? golang.org/x/text/unicode [no test files]
ok golang.org/x/text/unicode/bidi 0.026s
ok golang.org/x/text/unicode/cldr 0.114s
ok golang.org/x/text/unicode/norm 4.009s
ok golang.org/x/text/unicode/rangetable 1.516s
ok golang.org/x/text/unicode/runenames 0.011s
ok golang.org/x/text/width 0.310s
```
Fixes#2963.
The notBefore date in certificates is set based on the current system time,
not based on ca.clk. Work around that problem in the issuance tests by
syncing the test ca.clk with the system time. This doesn't affect any
current tests but is required for upcoming tests to work correctly.
* CA: Stub IssuePrecertificate gPRC method.
* CA: Implement IssuePrecertificate.
* CA: Test Precertificate flow in TestIssueCertificate().
move verification of certificate storage
IssuePrecertificate tests
Add CT precertificate poison extension to CFSSL whitelist.
CFSSL won't allow us to add an extension to a certificate unless that
certificate is in the whitelist.
According to its documentation, "Extensions requested in the CSR are
ignored, except for those processed by ParseCertificateRequest (mainly
subjectAltName)." Still, at least we need to add tests to make sure a
poison extension in a CSR isn't copied into the final certificate.
This allows us to avoid making invasive changes to CFSSL.
* CA: Test precertificate issuance in TestInvalidCSRs().
* CA: Only support IssuePrecertificate() if it is explicitly enabled.
* CA: Test that we produce CT poison extensions in the valid form.
The poison extension must be critical in order to work correctly. It probably wouldn't
matter as much what the value is, but the spec requires the value to be ASN.1 NULL, so
verify that it is.
This PR reworks the original WFE2 JWS post validation code (primarily
from `verifyPOST()` in WFE1) to use the new "ACME v2" style of JWS verification.
For most endpoints this means switching to a style where the JWS does
*not* contain an embedded JWK and instead contains a Key ID that is used
to lookup the JWK to verify the JWS from the database. For some special
endpoints (e.g. new-reg) there is a self-authenticated JWS style that
uses the old method of embedding a JWK instead of using a Key ID
(because no account to reference by ID exists yet).
The JWS validation now lives in `wfe2/verify.go` to keep the main WFEv2
code cleaner. Compared to `verifyPOST` there has been substantial work
done to create smaller easier to test functions instead of one big
validation function. The existing WFE unit tests that were copied to the
WFE2 are largely left as they were (e.g. cruddy) and updated as
minimally as possible to support the new request validation. All tests
for new code were written in a cleaner subtest style. Cleaning up the
existing tests will be follow-up work (See https://github.com/letsencrypt/boulder/issues/2928).
Since the POST validation for the key-change and revocation endpoints
requires special care they were left out of the WFE2 implementation for now
and will return a "not implemented" error if called.
_Note to reviewers_: this is a large diff to `wfe2/wfe.go` and `wfe2/verify.go`
that Github will hide by default. You will need to click to view the diffs.
Resolves https://github.com/letsencrypt/boulder/issues/2858
We are planning to have the CA expose three operations:
IssueCertificate =
generateNotAfterAndSerialNumber . issueCertificateOrPrecertificate("cert")
IssuePrecertificate =
generateNotAfterAndSerialNumber . issueCertificateOrPrecertificate("precert")
IssueCertificateForPrecertificate =
checkNotAfterAndSerialNumber(precert) . issueCertificateOrPrecertificate("cert")
("." denotes function composition. checkNotAfterAndSerialNumber does
not exist yet, but will be symmetric with generateNotAfterAndSerialNumber)
Prior to this commit the `httpResp` result of `sendHTTPRequest` was
examined even in the case where `sendHTTPRequest` returns a non-nil
error. This can cause a nil panic since the `httpResp` may be `nil` when
the error is not. This commit returns an error from `Req()` immediately
when `sendHTTPRequest` returns one.
Daniel