Previously our executables would all try to declare the boulder exchange on
startup, which may have been leading to some race conditions in Travis. Also,
the Activity Monitor would try to bind a queue to the exchange at startup.
In prod both of these tasks are taken care of administratively, so including
them in the app code was adding unnecessary complexity. It also may have been
part of an issue causing Activity Monitor to fail to start up recently.
Also, turn the Activity Monitor into an RPC service, which gets it reconnects
for free, and add it to startservers.py.
We're getting many more spurious "Can't connect to WFE" errors in
TravisCI. So, we add the WFE's main port to the port liveness check in
amqp-integration-test.py.
Fixes#1099
I think even the ldflags that did not change between subsequent invocations of
./start.py, e.g. BUILD_HOST_VAR, were different between ./start.py and
`go test ./...`, which would cause test runs to be unnecessarily slow.
Open question: To keep local developer builds fast, maybe we should enable race
detection only in Travis? Otherwise, `go test ./...` runs with one set of
ldflags, and then `ampq-integration-test.py` runs with a different set, which I
think makes both of them slower.
also moves the first OCSP responses generation from the CA to the OCSP updater. This patch lays the
ground work for moving CT submission and adding CT backfill to the OCSP updater.
Make `make` aware of output files so it doesn't always have to rebuild. Also
make it use `go install`, which is faster than building files individually.
Now that make is faster, use it in startservers.py to consolidate building
logic. This also has the handy side-effect that ./start.py exposes useful build
information through /build, whereas before only the .rpm packaged version did.
Additionally, this allows us to remove `make` from the Travis matrix, since we
are running `make` as part of the integration test. This means each PR only
triggers two Travis builds instead of one, which means we will get results from
Travis faster.
Also, change the Travis matrix logic to be a list of actions to run, rather than
a list of actions to skip. That fixes
https://github.com/letsencrypt/boulder/issues/817.
Enumerate specific sections of test.sh to run, rather than sections to skip.
Note: ./start.py now installs into ./bin/ instead of $GOPATH/bin.
Only set up GitHub secret file (for PR status reporting) when available, and
decrypt it into /tmp rather than $HOME, to avoid accidentally caching it once
Travis' caching features are available.
Clone letsencrypt repo into $HOME instead of $TMP, to make it possible to cache
eventually.
Remove unused `mysql` dependency in Travis.
Override default Travis install command to prevent it from adding
Godeps/_workspace to GOPATH. When that happens, it hides failures that should
arise from importing non-vendorized paths.
Adds a new service, Publisher, which exists to submit issued certificates to various Certificate Transparency logs. Once submitted the Publisher will also parse and store the returned SCT (Signed Certificate Timestamp) receipts that are used to prove inclusion in a specific log in the SA database. A SA migration adds the new SCT receipt table.
The Publisher only exposes one method, SubmitToCT, which is called in a goroutine by ca.IssueCertificate as to not block any other issuance operations. This method will iterate through all of the configured logs attempting to submit the certificate, and any required intermediate certificates, to them. If a submission to a log fails it will be retried the pre-configured number of times and will either use a back-off set in a Retry-After header or a pre-configured back-off between submission attempts.
This changeset is the first of a number of changes ending with serving SCT receipts in OCSP responses and purposefully leaves out the following pieces for follow-up PRs.
* A fake CT server for integration testing
* A external tool to search the database for certificates lacking a full set of SCT receipts
* A method to construct X.509 v3 extensions containing receipts for the OCSP responder
* Returned SCT signature verification (beyond just checking that the signature is of the correct type so we aren't just serving arbitrary binary blobs to clients)
Resolves#95.
Changes this to use just communicate(), not the subprocess.PIPE stuff (which
apparently can do Weird Things)
Also rename the install variable to cmd in the install function
This eases the CPU and thread requirements of our tests (by forking
less, not doing everything at once). It should also speed up the tests
by avoiding certain repetitive work.
Updates https://github.com/letsencrypt/letsencrypt/issues/712
Run builds in parallell as well as starting servers in parallel.
Wait for the servers to come up, so tests don't start running too early.
Enable race detection only for the integration test, not for start.py.
Previously I'd suggested it should always be on, but after running with it for a
while I'm convinced it's too slow for start.py (but still very valuable for
integration tests!).
Jacob Hoffman-Andrews
Jeff Hodges
Roland Shoemaker
Tom Clegg