The summary here is:
- Move test/cert-ceremonies to test/certs
- Move .hierarchy (generated by the above) to test/certs/webpki
- Remove our mapping of .hierarchy to /hierarchy inside docker
- Move test/grpc-creds to test/certs/ipki
- Unify the generation of both test/certs/webpki and test/certs/ipki
into a single script at test/certs/generate.sh
- Make that script the entrypoint of a new docker compose service
- Have t.sh and tn.sh invoke that service to ensure keys and certs are
created before tests run
No production changes are necessary, the config changes here are just
for testing purposes.
Part of https://github.com/letsencrypt/boulder/issues/7476
Previously, `starservers.start()` would implicitly build the binaries.
This separates the `startservers.install()` step as a separate one that
must happen first. This is useful because it allows us to ensure the
`ceremony` tool has been built before we run `setupHierarchy`.
Also, add a `-s` flag to `curl` when checking whether start.py resulted
in a successful startup. This reduces the amount of log spam when it
failed to come up.
RabbitMQ was removed in favor of gRPC in 2016. While there have been
many commits to remove the AMQP-related code, this last mention in
start.py persisted.
Modified the Dockerfile to build using Debian Buster, an upgrade from
Debian Stretch. The default Python 3 version for Stretch is 3.5.x which
is soon to de deprecated by Python-cryptography a dependency we rely on
for our integration test suite. The default Python 3 version for Debian
Buster is 3.7.x
In the .travis.yml file we are instructing travis to provision Xenial
instances and install two versions of Go. This change bumps Xenial
(16.04) -> Focal (20.04) and removes the installation of the two Go
versions; all of our testing happens inside of a docker container so
having Go installed on the Docker parent isn't necessary.
In the docker-compose.yml file we configure which docker image to pull
from Dockerhub, I've updated these to reflect the Debian Buster images
already built and pushed.
Modified build.sh to install mariadb-client-core 10.3, there is no 10.1
install candidate for Debian Buster and release notes for 10.2 and 10.3
indicate that these were both security releases.
Modified test.sh to use python3 instead of system python (usually 2.7)
for test/grafana/lints.py
Fixes#5180
Once we de-dupe startservers.install calls we may want to move
setupHierarchy there, but for now we need to call it from
integration-test.py and start.py.
Fixes#4842
Python 2 is over in 1 month 4 days: https://pythonclock.org/
This rolls forward most of the changes in #4313.
The original change was rolled back in #4323 because it
broke `docker-compose up`. This change fixes those original issues by
(a) making sure `requests` is installed and (b) sourcing a virtualenv
containing the `requests` module before running start.py.
Other notable changes in this:
- Certbot has changed the developer instructions to install specific packages
rather than rely on `letsencrypt-auto --os-packages-only`, so we follow suit.
- Python3 now has a `bytes` type that is used in some places that used to
provide `str`, and all `str` are now Unicode. That means going from `bytes` to
`str` and back requires explicit `.decode()` and `.encode()`.
- Moved from urllib2 to requests in many places.
* Make restarting boulder in docker nicer.
Handle SIGTERM in startservers.py.
Forcibly remove rsyslog pid to avoid error.
* Add explanatory comment.
* Send SIGTERM instead of kill.
* Further improvements.
- Handle SIGINT too.
- Use unbuffered mode for Python so the print statements (like "all servers
running") get printed right away rather than at shutdown
- Squelch an unnecessary OSError about interrupting the wait() call.
Run builds in parallell as well as starting servers in parallel.
Wait for the servers to come up, so tests don't start running too early.
Enable race detection only for the integration test, not for start.py.
Previously I'd suggested it should always be on, but after running with it for a
while I'm convinced it's too slow for start.py (but still very valuable for
integration tests!).
This allows us to use the same PKCS#11 key for both cert signing and OCSP
signing, and simplifies config and startup.
This also starts building with -tags pkcs11 in all scripts, which is required
now that the CA can choose between pkcs11 and non-pkcs11.
In order to successfully issue using a pkcs11 key, you'll need to run a version
of Go built off the master branch. The released versions are missing this
commit:
fe40cdd756,
which is necessary for PKCS#11 signing.
Aaron Gable
Phil Porada
Jacob Hoffman-Andrews
J.C. Jones
Samantha
Roland Bracewell Shoemaker
alexzorin
Tom Clegg
Jeff Hodges