This version includes a fix that seems relevant to us:
> The HTTP/1 client did not fully validate the contents of the Host
header. A maliciously crafted Host header could inject additional
headers or entire requests. The HTTP/1 client now refuses to send
requests containing an invalid Request.Host or Request.URL.Host value.
>
> Thanks to Bartek Nowotarski for reporting this issue.
>
> Includes security fixes for CVE-2023-29406 and Go issue
https://go.dev/issue/60374
Add go1.21rc2 to the matrix of go versions we test against.
Add a new step to our CI workflows (boulder-ci, try-release, and
release) which sets the "GOEXPERIMENT=loopvar" environment variable if
we're running go1.21. This experiment makes it so that loop variables
are scoped only to their single loop iteration, rather than to the whole
loop. This prevents bugs such as our CAA Rechecking incident
(https://bugzilla.mozilla.org/show_bug.cgi?id=1619047). Also add a line
to our docker setup to propagate this environment variable into the
container, where it can affect builds.
Finally, fix one TLS-ALPN-01 test to have the fake subscriber server
actually willing to negotiate the acme-tls/1 protocol, so that the ACME
server's tls client actually waits to (fail to) get the certificate,
instead of dying immediately. This fix is related to the upgrade to
go1.21, not the loopvar experiment.
Fixes https://github.com/letsencrypt/boulder/issues/6950
Go 1.20.1 is now deployed everywhere. Removing go 1.19 from CI will
allow us to begin adopting various go 1.20-only features that we want,
such as the new crypto/ecdh package.
Only build arm64 images for one version of Go.
Split build.sh into two scripts: build.sh (which installs apt and
Python) and install-go.sh (which installs a specific Go version and Go
dependencies). This allows reusing a cached layer for the build.sh step
across multiple Go versions.
Remove installation of fpm from build.sh. This is no longer needed since
#6669 and allows us to get rid of `rpm`, `ruby`, and `ruby-dev`.
Remove apt dependency on pkg-config, libtool, autoconf, and automake.
These were introduced in
https://github.com/letsencrypt/boulder/pull/4832 but aren't needed
anymore because we don't build softhsm2 ourselves (we get it from apt).
Remove apt dependency on cmake, libssl-dev, and openssl. I'm not totally
sure what these were needed for but they're not needed anymore.
Running this locally on my laptop for our current 3 GO_CI_VERSIONS and 1
GO_DEV_VERSION takes 23 minutes of wall time, dominated by the cross
build for arm64.
The code path is now adequately tested in CI with try-release.yml. This
means it will no longer be automatically tested locally with `./t.sh`,
but it can be manually tested locally with `./tools/make-assets.sh`.
Also, to ensure CI has similar coverage to the old make-artifacts phase,
change make-deb.sh to make-assets.sh, and have it make all of rpm, deb,
and tar.
Change release.yml so it uploads the .tar.gz as well as the .deb.
Split creating a release and uploading build assets to that release into
two separate steps. This allows the release creation step to have the
"continue on error" flag set, so that whichever release job completes
first can create the release, while the slower one will fail, move on,
and still successfully upload its files.
Add go1.20 as a new version to run tests on, and to build release
artifacts from. Fix one test which was failing because it was
accidentally relying on consistent (i.e. unseeded) non-cryptographic
random number generation, which go1.20 now automatically seeds at import
time.
Update the version of golangci-lint used in our docker containers to the
new version that has go1.20 support. Remove a number of nolint comments
that were required due to an old version of the gosec linter.
This uncovered a bug! The stdout logger was truncating the microseconds part
of its timestamp if the last digit was zero. Fixed that. Also coerced the
stdout logger to use UTC.
To run the checker over our integration test logs, I changed t.sh to use
an explicit name for the container that runs boulder during the tests,
and pulled logs from that container after the tests.
When rsyslog receives multiple identical log lines in a row, it can
collapse those lines into a single instance of the log line and a
follow-up line saying "message repeated X times". However, that
rsyslog-generated line does not contain our log line checksum, so it
immediately causes log-validator to complain about the line. In
addition, the rsyslog docs themselves state that this feature is a
misfeature and should never be turned on. Despite this, Ubuntu turns the
feature on by default when the rsyslog package is installed from apt.
Add an additional command to our dockerfile which overwrites Ubuntu's
default setting to disable this misfeature, and update our test
environment to use the new docker image.
Fixes#6252
Run the Boulder unit and integration tests with go1.19.
In addition, make a few small changes to allow both sets of
tests to run side-by-side. Mark a few tests, including our lints
and generate checks, as go1.18-only. Reformat a few doc
comments, particularly lists, to abide by go1.19's stricter gofmt.
Causes #6275
It turns out that `github.event.repository.default_branch` doesn't
exist when the action is triggered by cron (though it does exist
when the action is triggered manually, which I would expect to
be nearly identical...). So instead add a step to acquire the default
branch name from the API and use that as the target branch instead.
Create a GitHub Actions workflow that creates a PR to update
the publicsuffix-go dependency. This workflow runs monthly,
or whenever we choose to trigger it manually.
Fixes#5796
This reverts commit 5fe5859c38.
Per #5973:
> we will eventually want to go back to doing this in boulder-tools, so it's easy
> to run the lints locally. But this is useful so we can unblock testing on go 1.18beta2.
Update:
- golangci-lint from v1.42.1 to v1.46.2
- protoc from v3.15.6 to v3.20.1
- protoc-gen-go from v1.26.0 to v1.28.0
- protoc-gen-go-grpc from v1.1.0 to v1.2.0
- fpm from v1.14.0 to v1.14.2
Also remove a reference to go1.17.9 from one last place.
This does result in updating all of our generated .pb.go files, but only
to update the version number embedded in each file's header.
Fixes#6123
Generate .deb packages for all currently configured Go versions
(usually the current and upcoming versions that we use in prod), rather
than just the one default version. Also ensure that the uploaded
artifacts have 8-character short hashes in their names.
Unfortunately this does require updating Go versions in one additional
place (the release.yml file), since we are no longer parsing it out of the
docker-compose.yml. This is unavoidable without hacks that I consider
to be even uglier than the repetition.
Fixes#6075Fixes#6084
This allows us to narrow permissions by only granting write privileges
to the upload portion of the job (which doesn't run any code from our
repo). It also allows us to verify that the release build works on every
commit, while only generating releases on actual release tags.
go1.17.9 (released 2022-04-12) includes security fixes to the crypto/elliptic and encoding/pem packages, as well as bug fixes to the linker and runtime. See the [Go 1.17.9 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.17.9+label%3ACherryPickApproved) on our issue tracker for details.
go1.18.1 (released 2022-04-12) includes security fixes to the crypto/elliptic, crypto/x509, and encoding/pem packages, as well as bug fixes to the compiler, linker, runtime, the go command, vet, and the bytes, crypto/x509, and go/types packages. See the [Go 1.18.1 milestone](https://github.com/golang/go/issues?q=milestone%3AGo1.18.1+label%3ACherryPickApproved) on our issue tracker for details.
Update from v1.42.1 to v1.45.2. This version has both
better support for go1.18, and better automatic detection
of which go version to use (based on our go.mod file,
it will currently use go1.17).
- Remove GOPATH-style path structure, which isn't needed with Go
modules.
- Remove check for existing of docker buildx builder instance, since it
was unreliable.
- Add a CI workflow which publishes a GitHub Release containing a Debian package
when a release tag is pushed
- Add a script, called by the CI host, that installs all of the dependencies
necessary to `make` a Debian package
- Remove the, now defunct, goreleaser config file
Fixes#5970
This requires using GODEBUG to enable a couple of thing turned off by go1.18 (TLS 1.0/1.1, SHA-1 CSRs).
Also add help for a failure mode of cross builds.
Adapted from https://github.com/golangci/golangci-lint-action#how-to-use.
Uses the same version we've been using in boulder-tools.
Part of #5946
Note: we will eventually want to go back to doing this in boulder-tools,
so it's easy to run the lints locally. But this is useful so we can
unblock testing on go 1.18beta2.
Build a new docker container for the new Go 1.17.5 security release,
which includes a fix for the `net/http` package. Update our CI to run
tests on both our current and the new go versions.
Remove goreleaser since we're not currently using it.
Remove the script that moves PRs from master to main because it hasn't
been needed in practice.
Set persist-credentials: true for Boulder CI.
Right now when looking at a list of Boulder CI test results, they all
say:
boulder_ci_tests (go_1.17_2021-...
Which is not very informative as to which type of test failed. This
shortens the test name to "ci", and also changes the invoked command so
more of it fits on the screen. That involves adding two new scripts,
t.sh and tn.sh, which each run `docker-compose run ... test.sh`. tn.sh
runs it with the appropriate flags to use config-next.
Add go1.17beta1 docker images to the set of things we build,
and integrate go1.17beta1 into the set of environments CI runs.
Fix one test which breaks due to an underlying refactoring in
the `crypto/x509` stdlib package. Fix one other test which breaks
due to new guarantees in the stdlib's TLS ALPN implementation.
Also removes go1.16.5 from CI so we're only running 2 versions.
Fixes#5480
Currently we only run pre-submit CI for PRs which are against
our main branch or release branches. This means that "stacked"
PRs (a chain of PRs which all build on each other, and which will
need to be submitted in order) don't get any CI runs for PRs beyond
the first.
We don't expect large amounts of PR spam running us out of free
GH Actions credits, so just run CI for all PRs. Continue to run post-
submit CI only for commits to main and release branches so that we
don't prematurely run it on not-yet-PR-ready branches.
In normal operation, Boulder does not have release branches, only
release tags. However, when we need to add hotfix commits on top of an
old release, we create a release branch, merge the commits there, and
then produce a new tag pointing at the tip of that branch. These release
branches are documented[1] to be named `refs/heads/release-branch-*`.
Therefore, we should run CI for PRs targeting, and new commits on, those
release branches.
[1] https://github.com/letsencrypt/boulder-release-process#when-main-is-dirty
protoc now generates grpc code in a separate file from protobuf code.
Also, grpc servers are now required to embed an "unimplemented"
interface from the generated .pb.go file, which provides forward
compatibility.
Update the generate.go files since the invocation for protoc has changed
with the split into .pb.org and _grpc.pb.go.
Fixes#5368
- Remove `goveralls`, `gover`, and `cover` from `build.sh`.
- Remove `--coverage` option from `test.sh`.
- Update Docker image in `docker-compose.yml` and
`.github/workflows/boulder-ci.yml`
Fixes#5357
- Remove `.travis.yml`
- Remove references to Travis in `test.sh`
- Update documentation in `test/boulder-tools/README.md`, `README.MD`,
and `CONTRIBUTING.MD`
- Update comments in `.github/workflows/boulder-ci.yml`
Fixes#5329
Docker container should load the appropriate schema (`sa/_db` or
`sa/_db-next`) for the given configuration.
- Add `docker-compose.next.yml` docker-compose overrides
- Detect when to apply `sa/_db-next/migrations`
- Detect mismatch between `goose dbversion` and the latest migration
- Symlink `promoted` schema back to `sa/_db-next/migrations`
- Add tooling to consistently promote/demote schema migrations
Fixes#5300
- Add 1.16.1 to the GitHub CI test matrix
- Fix tlsalpn tests for go 1.16.1 but maintain compatibility with 1.15.x
- Fix integration tests.
Fix: #5301Fix: #5316
Samantha