We have upgraded to go1.24.1 in production, and no longer need to test
go1.23.x. Updating the version in our go.mod also allows us to begin
using x509.Certificate.Policies instead of .PolicyIdentifiers.
Update from go1.23.1 to go1.23.6 for our primary CI and release builds.
This brings in a few security fixes that aren't directly relevant to us.
Add go1.24.0 to our matrix of CI and release versions, to prepare for
switching to this next major version in prod.
Github is currently rolling out ubuntu-latest as ubuntu-24.04. Manage
that switch explicitly by running most jobs on 24.04
https://github.com/actions/runner-images/issues/10636
This keeps the release on 20.04 to ensure released binaries can run on
older operating systems (because of CGO/glibc versions)
Have the release action write the checksums to a file whose name matches
the name of the boulder deb and rpm files whose checksums it contains.
This prevents name-collision errors when we are running multiple Go
versions side-by-side in CI.
To demonstrate this, augment the try-release action to write the
checksums to the same file, print the list of files it would upload, and
print the contents of the checksums file. Also allow the try-release
workflow to be triggered manually.
Begin testing on go1.23. To facilitate this, also update /x/net,
golangci-lint, staticcheck, and pebble-challtestsrv to versions which
support go1.23. As a result of these updates, also fix a handful of new
lint findings, mostly regarding passing non-static (i.e. potentially
user-controlled) format strings into Sprintf-style functions.
Additionally, delete one VA unittest that was duplicating the checks
performed by a different VA unittest, but with a context timeout bug
that caused it to break when go1.23 subtly changed DialContext behavior.
Running this workflow on both `ready_for_review` and `review_requested`
was causing duplicate comments to show up when PRs were moved out of
draft mode. This is because moving out of draft mode would both trigger
a workflow run and automatically request review, which in turn would
trigger a second workflow run, and the two runs were triggered in such
quick succession that our commentMarker detection wouldn't work.
This will cause dependabot to open one PR that updates dependencies
matching these patterns in a single PR. For both AWS and otel, we depend
on multiple packages from them, which generally are good to bump
together.
Update the version of protoc-gen-go-grpc that we use to generate Go gRPC
code from our proto files, and update the versions of other gRPC tools
and libraries that we use to match. Turn on the new
`use_generic_streams` code generation flag to change how
protoc-gen-go-grpc generates implementations of our streaming methods,
from creating a wholly independent implementation for every stream to
using shared generic implementations.
Take advantage of this code-sharing to remove our SA "wrapper" methods,
now that they have truly the same signature as the SARO methods which
they wrap. Also remove all references to the old-style stream names
(e.g. foopb.FooService_BarMethodClient) and replace them with the new
underlying generic names, for the sake of consistency. Finally, also
remove a few custom stream test mocks, replacing them with the generic
mocks.ServerStreamClient.
Note that this PR does not change the names in //mocks/sa.go, to avoid
conflicts with work happening in the pursuit of
https://github.com/letsencrypt/boulder/issues/7476. Note also that this
PR updates the version of protoc-gen-go-grpc that we use to a specific
commit. This is because, although a new release of grpc-go itself has
been cut, the codegen binary is a separate Go module with its own
releases, and it hasn't had a new release cut yet. Tracking for that is
in https://github.com/grpc/grpc-go/issues/7030.
We first introduced caa-log-checker as a remediation item in the wake of
https://bugzilla.mozilla.org/show_bug.cgi?id=1619047. Since that time,
we have upgraded to go1.22, which completely remoes the class of bug
which led to that incident (https://tip.golang.org/doc/go1.22#language).
Throughout its life, caa-log-checker was an operational burden, and was
at best a post-hoc check to detect issues after they had already
occurred. Therefore, we no longer run it in our production environment,
and it can be removed from the Boulder source.
Rather than running the "check-changes" action immediately when a PR is
opened, wait for the PR to be ready for review (either by being taken
out of draft mode, or by having reviewers added). This way it's possible
to create a draft PR, then create a corresponding deployment ticket
which references the PR, add that ticket to the PR description, and mark
the PR ready for review, thereby pre-empting the bot so it doesn't have
to leave a comment.
We had disabled our lints on go1.22 because golangci-lint and
staticcheck didn't work with some of its updates. Re-enable them, and
fix the things which the updated linters catch now.
Fixes https://github.com/letsencrypt/boulder/issues/7229
We have moved entirely to go1.22 in prod. This also allows us to remove
setting loopvar from our CI tasks, since it is the default behavior as
of go1.22.
Always run on the latest Go version. This reduces busywork / failed CI
builds when Go publishes a security release, since we have other
processes for making sure we get notice of those security releases.
This may still fail for a little while after the Go release is
published, while we wait for GitHub to update the Go version used in CI
as "latest", but it will be a little faster and require less manual
work.
Allow github workflow configuration change checker to also check for
changes to SQL schema and user grants. This is an incredibly simple
version so the PR author is in charge of determining what tickets should
be created.
Replace the python "codespell" tool with the rust "typos" tool.
To accomplish this, add a new rust-based step to the boulder-tools
docker build process, with some complexity to handle builds on
multiple developer architectures.
Co-authored-by: Viktor Szépe <viktor@szepe.net>
Replace the current three-piece setup (enum of feature variables, map of
feature vars to default values, and autogenerated bidirectional maps of
feature variables to and from strings) with a much simpler one-piece
setup: a single struct with one boolean-typed field per feature. This
preserves the overall structure of the package -- a single global
feature set protected by a mutex, and Set, Reset, and Enabled methods --
although the exact function signatures have all changed somewhat.
The executable config format remains the same, so no deployment changes
are necessary. This change does deprecate the AllowUnrecognizedFeatures
feature, as we cannot tell the json config parser to ignore unknown
field names, but that flag is set to False in all of our deployment
environments already.
Fixes https://github.com/letsencrypt/boulder/issues/6802
Fixes https://github.com/letsencrypt/boulder/issues/5229
Previously we made these a single `RUN` step in the Dockerfile to reduce
the size of the final image. Docker pulls all the dependent layers for
an image, which means that even if you delete intermediate build files
in a later `RUN` step, they still contribute to the overall download
size. You can work around that by deleting the intermediate files within
a single `RUN` step.
However, that has downsides: changing one Go dependency meant
downloading Go and all the other dependencies again. By moving these
back into `RUN` steps we get incremental builds, which are nice. And by
adding the builder pattern (`FROM ... AS godeps`), we can avoid having
intermediate files contribute to the overall image size.
Having govulncheck prevent a PR from merging means that circumstances
entirely outside our control can grind Boulder development to a halt
until they are addressed. When the vulnerability is within Go itself, it
prevents PRs from being merged until we do a production deploy, because
we want our CI to always match what is in production. This is too
strict.
This PR removes govulncheck from the set of jobs depended upon by our
Boulder CI Test Matrix meta-job. It also adds vendorcheck, which was
accidentally omitted in #7123.
Remove the "netaccess" container from the docker-compose dev
environment.
It isn't needed during a regular 'docker compose up' developer
environment, and only really serves as a way to use the same tools image
in CI. Two checks run during CI are the govulncheck and verifying go mod
tidy / go vendor. Neither of these checks require anything from the
custom image other than Golang itself, which can be provided directly
from the CI environment.
If a developer is working inside the existing containers, they can still
run `go mod tidy; go mod vendor` themselves, which is a standard Golang
workflow and thus is simpler than using the netaccess image via docker
compose.
Run staticcheck as a standalone binary rather than as a library via
golangci-lint. From the golangci-lint help out,
> staticcheck (megacheck): It's a set of rules from staticcheck. It's
not the same thing as the staticcheck binary. The author of staticcheck
doesn't support or approve the use of staticcheck as a library inside
golangci-lint.
We decided to disable ST1000 which warns about incorrect or missing
package comments.
For SA4011, I chose to change the semantics[1] of the for loop rather
than ignoring the SA4011 lint for that line.
Fixes https://github.com/letsencrypt/boulder/issues/6988
1. https://go.dev/ref/spec#Continue_statements
These workflows relied on old-style Personal Access Tokens. They've been
disabled for a while, and we have no intention of re-enabling them. The
add-to-project workflow in particular has been fully replaced by
Projects-native features.
Aaron Gable