Commit Graph

4978 Commits

Author SHA1 Message Date
Jacob Hoffman-Andrews 5254844ba2
Make TestValidAuthzExpires non-flaky. (#4778)
Previously, the test called `.Round(time.Minute)` on the expected
and actual expiration times, intending to perform an "approximately
equal" function.

However, when the expected and actual times differed by a second, but
they happened to fall on opposite sides of a rounding interval (i.e. 30
seconds into a minute), they would be rounded in opposite directions,
resulting in a conclusion that they were not equal.

This change instead defines an acceptable range of plus or minus a
minute for the expiration time, and checks that the actual expiration
time is in that interval.
2020-04-15 12:54:53 -07:00
Jacob Hoffman-Andrews 36c1f1ab2d
Deprecate some feature flags (#4771)
Deprecate some feature flags.

These are all enabled in production.
2020-04-13 15:49:55 -07:00
Jacob Hoffman-Andrews db1a1a8595
Set memlock ulimit to unlimited. (#4770)
This works around a kernel bug that is tickled by Go 1.14:

https://github.com/golang/go/issues/37436
2020-04-13 15:49:38 -07:00
Roland Bracewell Shoemaker 7cc5f64a48
Reject RSA keys with non-standard exponent (#4769)
Only allow the RSA exponent 65537, which is the defacto standard (only 3 unexpired certificates issued by Let's Encrypt use a different exponent).
2020-04-13 15:29:32 -07:00
Daniel McCarney 005cd54eef
docs: update multi-va.md for prod deploy (#4767)
Quick update to docs/multi-va.md to reflect that the feature has been promoted out of the test deployment stage into a production feature.
2020-04-09 15:49:51 -07:00
Jacob Hoffman-Andrews 9fbab1a1cd
Switch to golangci-lint for our lints. (#4766)
This makes it easier to configure additional linters, and provides us an
easy command to run locally.

The initial set of linters reflects those we are already running:

govet gofmt ineffassign errcheck misspell staticcheck

Note that misspell is in addition to the Python codespell package.

Since the invocation of these linters from golangci-lint is slightly
different from how we currently invoke them, there are some new
findings. This PR won't pass tests until #4763, #4764, and #4765 are
merged.

Incidentally, rename strat -> strategy to appeal misspell.
2020-04-09 09:42:24 -07:00
Jacob Hoffman-Andrews 0db7d9ff89
Block keys using hex(sha256(spki)). (#4745)
In addition to base64(sha256(spki)).

As part of that, change KeyDigest to return [32]byte, and add KeyDigestB64 which provides the base64-encoded output that KeyDigest used to provide. Also update all call sites.
2020-04-09 09:41:33 -07:00
Roland Bracewell Shoemaker 324d92d7c5
goodkey cleanups (#4754)
Fixes #4748 and fixes #4740.
2020-04-08 17:57:23 -07:00
Jacob Hoffman-Andrews 84a51ecce0
Add error checking in load-generator and ocsp_forever (#4765)
Found by golangci-lint's errcheck invocation.
2020-04-08 17:26:42 -07:00
Jacob Hoffman-Andrews 0a0a776882
Fix some misspellings. (#4764)
Found by golangci-lint's `misspell` linter.

`strat` was a false positive vs `start` but I figured it's easier to
just change the name.
2020-04-08 17:26:10 -07:00
Jacob Hoffman-Andrews 72deb5b798
gofmt code with -s (simplify) flag (#4763)
Found by golangci-lint's `gofmt` linter.
2020-04-08 17:25:35 -07:00
Jacob Hoffman-Andrews aa9ec70ef7
Remove deprecated HeaderMap reference. (#4762)
ResponseRecorder's HeaderMap field is deprecated:
https://godoc.org/net/http/httptest#ResponseRecorder

partial staticcheck cleanup: https://staticcheck.io/docs/checks#SA1019
2020-04-08 17:24:44 -07:00
Jacob Hoffman-Andrews 75024c3ec1
Replace clock.Default() with clock.New() (#4761)
clock.Default is deprecated:
https://godoc.org/github.com/jmhodges/clock#Default
2020-04-08 17:23:43 -07:00
Jacob Hoffman-Andrews b567adb9f5
Switch to godoc-style comments. (#4760)
* Switch to godoc-style comments.

staticcheck cleanup: https://staticcheck.io/docs/checks#ST1021

* Revert spurious diff.
2020-04-08 17:22:40 -07:00
Jacob Hoffman-Andrews 4df94d5215
Use responseWriter.Body.Bytes() (#4759)
staticcheck cleanup: https://staticcheck.io/docs/checks#S1030
2020-04-08 17:21:50 -07:00
Jacob Hoffman-Andrews 0e9ac0c638
Use bytes.Equal instead of bytes.Compare == 0 (#4758)
staticcheck cleanup: https://staticcheck.io/docs/checks#S1004
2020-04-08 17:20:56 -07:00
Jacob Hoffman-Andrews de8855f15b
Use sort.Strings instead of sort.Sort (#4757)
staticcheck cleanup: https://staticcheck.io/docs/checks#S1032
2020-04-08 17:20:20 -07:00
Jacob Hoffman-Andrews 8d99ada705
Change order of "yoda conditions" (1 != foo) (#4756)
staticcheck cleanup: https://staticcheck.io/docs/checks#ST1017
2020-04-08 17:19:56 -07:00
Jacob Hoffman-Andrews cdb0bddbd8
Prefix error names with "Err" (#4755)
Staticcheck cleanup: https://staticcheck.io/docs/checks#ST1012
2020-04-08 17:19:35 -07:00
Roland Bracewell Shoemaker fb0d2ffaa8
Store key hash when adding precertificate, add backfill tool (#4753)
Fixes #4749
2020-04-08 13:53:19 -07:00
Roland Bracewell Shoemaker 36b9c95324
Fix link to test PKI info (#4750)
Fixes #4723
2020-04-08 10:27:32 -07:00
Jacob Hoffman-Andrews 1278679afb
Handle mismatched URLs in key rollover. (#4752)
Fixes #4751
2020-04-07 19:21:02 -07:00
Jacob Hoffman-Andrews aad43e4688
Fix entrypoint.sh / docker-compose up. (#4747)
We no longer use virtualenv; we just install our Python dependencies
globally.
2020-04-07 11:35:42 -07:00
Jacob Hoffman-Andrews 5af7541c85
Improve output when Go integration tests fail. (#4734)
Right now we show output like:

Traceback (most recent call last):
File "test/integration-test.py", line 60, in run_go_tests
subprocess.check_call(cmdLine, shell=False, stderr=subprocess.STDOUT)
File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['go', 'test', '-tags', 'integration', '-count=1', '-race', './test/integration']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test/integration-test.py", line 414, in
main()
File "test/integration-test.py", line 293, in main
run_go_tests(args.test_case_filter)
File "test/integration-test.py", line 62, in run_go_tests
raise(Exception("%s. Output:\n%s" % (e, e.output)))
Exception: Command '['go', 'test', '-tags', 'integration', '-count=1', '-race', './test/integration']' returned non-zero exit status 1. Output:
None

This change removes the try / raise clauses that were causing this
double exception logging. The original purpose of these clauses was to
make sure we logged output on failure. To continue to fulfill that
purpose, I switched the run function to use check_call instead of
check_output. check_output captures the stdout; check_call emits it to
the caller's stdout as normal, so we still see the output.

I also changed the two cases that actually wanted to process output so
they use check_output directly.
2020-04-06 17:42:40 -07:00
Jacob Hoffman-Andrews 48942db1a9
Add staticcheck to CI (#4731)
This adds staticcheck to our "lints" CI, with a list of excluded checks. Some of these are checks that we don't care about much (like error string capitalization). Others are nice to fix (possible nil pointer dereferences in _test.go files), but we'd like to land the automated checking first to catch any new issues, then later winnow down the list.

This builds on #4726, #4725, and #4722, which addressed some of the categories of findings from staticcheck.
2020-04-06 17:37:19 -07:00
Jacob Hoffman-Andrews 28dbbce8da
Make boulder-tools image smaller. (#4732)
Instead of installing Certbot from the repo, install the python-acme
library (the only piece we need) from the apt repository. This also
allows us to skip installing build dependencies for Certbot.

Uninstall cmake after building.

Clean the various Go caches.

Move codespell and acme into requirements.txt. Don't use virtualenv anymore.

This reduces image size from 1.4 GB to 1.0 GB.

Incidentally, move the Go install to its own phase in the Dockerfile.
This will give it its own image layer, making rebuilds faster.
2020-04-06 16:53:42 -07:00
Jacob Hoffman-Andrews 1f0d8c1e4f
Clean up .travis.yml (#4735)
We have a comment in .travis.yml indicating that Travis' version of docker-compose is too old for us, but that's no longer true, so we can simplify things and speed up by using the stock installed one.

The list of custom hostnames in .travis.yml that all point to localhost are no longer used. Instead we use a custom DNS server within our docker-compose setup. So this PR removes those hostnames.

Remove staging and release as special branches to have tests run. We no longer use branches as part of our release process.
2020-04-06 13:51:57 -07:00
Roland Bracewell Shoemaker 9af9de0541
Fix incorrect key name in ceremony docs (#4742) 2020-04-06 13:43:32 -07:00
Jacob Hoffman-Andrews 7e926ca4c6
Remove travis-before-install.sh. (#4733)
The comment in that script, that the master branch isn't present in
Travis builds, is no longer accurate. So we don't need this workaround
anymore.
2020-04-06 11:15:43 -07:00
Jacob Hoffman-Andrews df7ffbe206
Update .travis.yml in response to Travis' validation tool. (#4720)
In our builds, under "View config", Travis was showing two warnings and
4 info-level reports:

root: deprecated key sudo (The key `sudo` has no effect anymore.)
root: deprecated key sudo (The key `sudo` has no effect anymore.)
root: missing dist, using the default xenial
root: missing os, using the default linux
root: key matrix is an alias for jobs, using jobs
root: key matrix is an alias for jobs, using jobs

I believe this should fix all of these. I've selected xenial as the dist
for now because that's what we've been defaulting to, but we can switch
to bionic, which is newer (though it doesn't make much difference since
we're running most things in Docker).
2020-04-06 10:27:46 -07:00
Jacob Hoffman-Andrews 995e25e8c3
Start testing Go 1.14.1 in Travis. (#4718)
As part of the process, pin specific versions of protoc-gen-go, mockgen,
and goveralls. Protoc-gen-go recently released a version that was incompatible
with our current version of gRPC. Mockgen has a version that was generating
spurious diffs in our generate test phase, and goveralls recently added
some code that calls git branch --format=..., which breaks on the version of
git in our Docker image.

Pinning versions required forcing go get into module-aware mode, since the
old-style go get doesn't understand versions.
2020-04-06 09:18:00 -07:00
Jacob Hoffman-Andrews f26228703e
Temporarily remove crl package. (#4725)
This is currently unused, and was a copy of the Go upstream package
anyhow. We will either re-copy when needed or wait for Go 1.15, which
will include the updates we need.
2020-03-30 10:55:38 -07:00
Jacob Hoffman-Andrews bef02e782a
Fix nits found by staticcheck (#4726)
Part of #4700
2020-03-30 10:20:20 -07:00
Roland Bracewell Shoemaker 81bb4047d5
Ceremony monolith (#4678)
Merges gen-ca and gen-key into a single tool that can be used to complete a key/certificate generation ceremony. The driving idea here is that instead of having to write out multiple long commands in a specific order in order to complete a ceremony a configuration file is fed to a single binary. This config file contains all of the information needed to complete the ceremony, and can be easily tested outside of the secure environment before hand without fear of later typing a command/flag incorrectly etc.

The tooling works against the test hardware I have (there are minimal changes to the actual PKCS#11 code behind the scenes). Specific attention should be given to the documentation, and the general UX of the tool.

Fixes #4639 and fixes #4667.
2020-03-27 13:54:56 -07:00
Jacob Hoffman-Andrews 3a1a08a10b
Remove unused code. (#4722)
Found by staticcheck.
2020-03-27 11:55:42 -07:00
Jacob Hoffman-Andrews 3b54070a78
Deflake GetValidAuthorizations2 unittest. (#4721)
The test would sometimes fail based on randomized ordering of map
elements.
2020-03-27 11:55:29 -07:00
Jacob Hoffman-Andrews 9e2e08ece6
Update cfssl to latest. (#4719)
This pulls in an upgrade to zlint 2.0.0.
2020-03-26 10:11:05 -07:00
alexzorin 93cb918ce4
wfe: implement alternate certificate chains (#4714)
Closes #4567.

Enabled in `config-next`.

This PR cross-signs the existing issuers (`test-ca-cross.pem`, `test-ca2-cross.pem`) with a new root (`test-root2.key`, `test-root2.pem` = *c2ckling cryptogr2pher f2ke ROOT*).

The cross-signed issuers are referenced in wfe2's configuration, beside the existing `certificateChains` key:

```json
    "certificateChains": {
      "http://boulder:4430/acme/issuer-cert": [ "test/test-ca2.pem" ],
      "http://127.0.0.1:4000/acme/issuer-cert": [ "test/test-ca2.pem" ]
    },
    "alternateCertificateChains": {
      "http://boulder:4430/acme/issuer-cert": [ "test/test-ca2-cross.pem" ],
      "http://127.0.0.1:4000/acme/issuer-cert": [ "test/test-ca2-cross.pem" ]
    },
```

When this key is populated, the WFE will send links for all alternate certificate chains available for the current end-entity certificate (except for the chain sent in the current response):

    Link: <http://localhost:4001/acme/cert/ff5d3d84e777fc91ae3afb7cbc1d2c7735e0/1>;rel="alternate"

For backwards-compatibility, not specifying a chain is the same as specifying `0`: `/acme/cert/{serial} == /acme/cert/{serial}/0` and `0` always refers to the default certificate chain for that issuer (i.e. the value of `certificateChains[aiaIssuerURL]`).
2020-03-24 12:43:26 -07:00
Jacob Hoffman-Andrews e92c5c66ea
Remove unused code in SA. (#4710)
Spotted by staticcheck.
2020-03-18 13:17:56 -07:00
Jacob Hoffman-Andrews 6c1af624d8
Add unittest for authzModelMapToPB. (#4705)
Fixes #4696
2020-03-18 11:50:57 -07:00
Jacob Hoffman-Andrews 9e57fd477f
Improve GetValidOrderAuthorizations2 unittest. (#4706)
Fixes #4697
2020-03-17 12:16:27 -07:00
vairakkumar-vairaperumal b30a1e9595
Log message typo in entrypoint-netaccess.sh (#4708)
Fixes #4707.
2020-03-17 11:31:18 -07:00
alexzorin 0dd8f41c1d
ra: forbid mailto contacts that contain hfields (#4694)
https://tools.ietf.org/html/rfc8555#section-7.3

   Clients MUST NOT
   provide a "mailto" URL in the "contact" field that contains "hfields"
   [RFC6068] or more than one "addr-spec" in the "to" component.  If a
   server encounters a "mailto" contact URL that does not meet these
   criteria, then it SHOULD reject it as invalid.
2020-03-11 17:15:23 -07:00
Jacob Hoffman-Andrews 2bf12b93e1
Rename "authz2" types to just "authz" (#4701)
This cleans up after the authzv2 migration and makes names a little
easier to read, since there is no longer a v1/v2 distinction. This
leaves the names of tables the same since they would require a migration
to change.
2020-03-10 16:40:46 -07:00
Roland Bracewell Shoemaker 9232f0e9a4
Fix admin-revoker argument check (#4693)
Fixes a check for the number of arguments that was incorrect in #4692.
2020-03-03 13:22:46 -08:00
Roland Bracewell Shoemaker b58a28031f
Add admin-revoker parallel batched revocation mode (#4692)
* Add admin-revoker parallel batched revocation mode

* revert go.mod + go.sum changes
2020-03-03 09:55:59 -08:00
Jacob Hoffman-Andrews b58e5453e8
Fix output of logDNSError. (#4691)
The message had hostname and queryType backwards.
2020-03-02 08:39:21 -08:00
Jacob Hoffman-Andrews aa01f4d0eb
Pass authzModel by value, not reference (#4690) 2020-02-28 20:07:12 -08:00
Roland Bracewell Shoemaker 542cb6d2e0
Make orders with deact authz invalid (#4687)
Fixes #4685.
2020-02-27 11:22:03 -08:00
Roland Bracewell Shoemaker 4184dc3fc9
Cleanup issuePrecertificateInner (#4689)
Removes an unnecessary issuePrecertificateInner argument that was always the same thing.
2020-02-27 09:43:59 -08:00