Go 1.20.1 is now deployed everywhere. Removing go 1.19 from CI will
allow us to begin adopting various go 1.20-only features that we want,
such as the new crypto/ecdh package.
Only build arm64 images for one version of Go.
Split build.sh into two scripts: build.sh (which installs apt and
Python) and install-go.sh (which installs a specific Go version and Go
dependencies). This allows reusing a cached layer for the build.sh step
across multiple Go versions.
Remove installation of fpm from build.sh. This is no longer needed since
#6669 and allows us to get rid of `rpm`, `ruby`, and `ruby-dev`.
Remove apt dependency on pkg-config, libtool, autoconf, and automake.
These were introduced in
https://github.com/letsencrypt/boulder/pull/4832 but aren't needed
anymore because we don't build softhsm2 ourselves (we get it from apt).
Remove apt dependency on cmake, libssl-dev, and openssl. I'm not totally
sure what these were needed for but they're not needed anymore.
Running this locally on my laptop for our current 3 GO_CI_VERSIONS and 1
GO_DEV_VERSION takes 23 minutes of wall time, dominated by the cross
build for arm64.
The code path is now adequately tested in CI with try-release.yml. This
means it will no longer be automatically tested locally with `./t.sh`,
but it can be manually tested locally with `./tools/make-assets.sh`.
Also, to ensure CI has similar coverage to the old make-artifacts phase,
change make-deb.sh to make-assets.sh, and have it make all of rpm, deb,
and tar.
Change release.yml so it uploads the .tar.gz as well as the .deb.
Add go1.20 as a new version to run tests on, and to build release
artifacts from. Fix one test which was failing because it was
accidentally relying on consistent (i.e. unseeded) non-cryptographic
random number generation, which go1.20 now automatically seeds at import
time.
Update the version of golangci-lint used in our docker containers to the
new version that has go1.20 support. Remove a number of nolint comments
that were required due to an old version of the gosec linter.
This uncovered a bug! The stdout logger was truncating the microseconds part
of its timestamp if the last digit was zero. Fixed that. Also coerced the
stdout logger to use UTC.
To run the checker over our integration test logs, I changed t.sh to use
an explicit name for the container that runs boulder during the tests,
and pulled logs from that container after the tests.
When rsyslog receives multiple identical log lines in a row, it can
collapse those lines into a single instance of the log line and a
follow-up line saying "message repeated X times". However, that
rsyslog-generated line does not contain our log line checksum, so it
immediately causes log-validator to complain about the line. In
addition, the rsyslog docs themselves state that this feature is a
misfeature and should never be turned on. Despite this, Ubuntu turns the
feature on by default when the rsyslog package is installed from apt.
Add an additional command to our dockerfile which overwrites Ubuntu's
default setting to disable this misfeature, and update our test
environment to use the new docker image.
Fixes#6252
Run the Boulder unit and integration tests with go1.19.
In addition, make a few small changes to allow both sets of
tests to run side-by-side. Mark a few tests, including our lints
and generate checks, as go1.18-only. Reformat a few doc
comments, particularly lists, to abide by go1.19's stricter gofmt.
Causes #6275
This reverts commit 5fe5859c38.
Per #5973:
> we will eventually want to go back to doing this in boulder-tools, so it's easy
> to run the lints locally. But this is useful so we can unblock testing on go 1.18beta2.
Update:
- golangci-lint from v1.42.1 to v1.46.2
- protoc from v3.15.6 to v3.20.1
- protoc-gen-go from v1.26.0 to v1.28.0
- protoc-gen-go-grpc from v1.1.0 to v1.2.0
- fpm from v1.14.0 to v1.14.2
Also remove a reference to go1.17.9 from one last place.
This does result in updating all of our generated .pb.go files, but only
to update the version number embedded in each file's header.
Fixes#6123
go1.17.9 (released 2022-04-12) includes security fixes to the crypto/elliptic and encoding/pem packages, as well as bug fixes to the linker and runtime. See the [Go 1.17.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.17.9+label%3ACherryPickApproved) on our issue tracker for details.
go1.18.1 (released 2022-04-12) includes security fixes to the crypto/elliptic, crypto/x509, and encoding/pem packages, as well as bug fixes to the compiler, linker, runtime, the go command, vet, and the bytes, crypto/x509, and go/types packages. See the [Go 1.18.1 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.18.1+label%3ACherryPickApproved) on our issue tracker for details.
- Remove GOPATH-style path structure, which isn't needed with Go
modules.
- Remove check for existing of docker buildx builder instance, since it
was unreliable.
This requires using GODEBUG to enable a couple of thing turned off by go1.18 (TLS 1.0/1.1, SHA-1 CSRs).
Also add help for a failure mode of cross builds.
Adapted from https://github.com/golangci/golangci-lint-action#how-to-use.
Uses the same version we've been using in boulder-tools.
Part of #5946
Note: we will eventually want to go back to doing this in boulder-tools,
so it's easy to run the lints locally. But this is useful so we can
unblock testing on go 1.18beta2.
Build a new docker container for the new Go 1.17.5 security release,
which includes a fix for the `net/http` package. Update our CI to run
tests on both our current and the new go versions.
Remove goreleaser since we're not currently using it.
Remove the script that moves PRs from master to main because it hasn't
been needed in practice.
Set persist-credentials: true for Boulder CI.
Right now when looking at a list of Boulder CI test results, they all
say:
boulder_ci_tests (go_1.17_2021-...
Which is not very informative as to which type of test failed. This
shortens the test name to "ci", and also changes the invoked command so
more of it fits on the screen. That involves adding two new scripts,
t.sh and tn.sh, which each run `docker-compose run ... test.sh`. tn.sh
runs it with the appropriate flags to use config-next.
Add go1.17beta1 docker images to the set of things we build,
and integrate go1.17beta1 into the set of environments CI runs.
Fix one test which breaks due to an underlying refactoring in
the `crypto/x509` stdlib package. Fix one other test which breaks
due to new guarantees in the stdlib's TLS ALPN implementation.
Also removes go1.16.5 from CI so we're only running 2 versions.
Fixes#5480
Currently we only run pre-submit CI for PRs which are against
our main branch or release branches. This means that "stacked"
PRs (a chain of PRs which all build on each other, and which will
need to be submitted in order) don't get any CI runs for PRs beyond
the first.
We don't expect large amounts of PR spam running us out of free
GH Actions credits, so just run CI for all PRs. Continue to run post-
submit CI only for commits to main and release branches so that we
don't prematurely run it on not-yet-PR-ready branches.
In normal operation, Boulder does not have release branches, only
release tags. However, when we need to add hotfix commits on top of an
old release, we create a release branch, merge the commits there, and
then produce a new tag pointing at the tip of that branch. These release
branches are documented[1] to be named `refs/heads/release-branch-*`.
Therefore, we should run CI for PRs targeting, and new commits on, those
release branches.
[1] https://github.com/letsencrypt/boulder-release-process#when-main-is-dirty
protoc now generates grpc code in a separate file from protobuf code.
Also, grpc servers are now required to embed an "unimplemented"
interface from the generated .pb.go file, which provides forward
compatibility.
Update the generate.go files since the invocation for protoc has changed
with the split into .pb.org and _grpc.pb.go.
Fixes#5368
- Remove `goveralls`, `gover`, and `cover` from `build.sh`.
- Remove `--coverage` option from `test.sh`.
- Update Docker image in `docker-compose.yml` and
`.github/workflows/boulder-ci.yml`
Fixes#5357
- Remove `.travis.yml`
- Remove references to Travis in `test.sh`
- Update documentation in `test/boulder-tools/README.md`, `README.MD`,
and `CONTRIBUTING.MD`
- Update comments in `.github/workflows/boulder-ci.yml`
Fixes#5329
Docker container should load the appropriate schema (`sa/_db` or
`sa/_db-next`) for the given configuration.
- Add `docker-compose.next.yml` docker-compose overrides
- Detect when to apply `sa/_db-next/migrations`
- Detect mismatch between `goose dbversion` and the latest migration
- Symlink `promoted` schema back to `sa/_db-next/migrations`
- Add tooling to consistently promote/demote schema migrations
Fixes#5300
- Add 1.16.1 to the GitHub CI test matrix
- Fix tlsalpn tests for go 1.16.1 but maintain compatibility with 1.15.x
- Fix integration tests.
Fix: #5301Fix: #5316
- Add GitHub actions workflow for Boulder CI tests in parity with Travis
CI except the coverage test.
- Change boulder-tools docker image to push to a static docker repo
instead of creating a new one each time. Use docker version tags and git
hash to identify go versions in the repo.
- Change docker-compose to pull from the static boulder-tools repo. This
breaks using the TRAVIS_GO_VERSION env variable to pull the docker image, but
the default will still work with intent of decommissioning Travis-CI for
GitHub CI.
Fix: #5289
Phil Porada