Remove the `MandatoryPOSTasGET` flag from the WFE2.
Update the ACMEv2 divergence doc to note that neither staging nor
production use MandatoryPOSTasGET.
Fixes#6582.
Go 1.20.1 is now deployed everywhere. Removing go 1.19 from CI will
allow us to begin adopting various go 1.20-only features that we want,
such as the new crypto/ecdh package.
Add the GRPCStatus method to our BoulderError type, so that the gRPC
server code can automatically set an appropriate Status on all gRPC
responses, based on the kind of error that we return. We still serialize
the whole BoulderError type and details into the response metadata, so
that it can be rehydrated on the client side, but this allows the
gRPC-native Status to be something other than Unknown. As part of this
change, have our custom error serialization code stop manually setting
the gRPC status code to codes.Unknown.
This change allows the default gRPC prometheus metrics to more
accurately report the kinds of errors our gRPC requests experience, and
may allow us to more elegantly transition to using grpc.Status errors in
other places where they're relevant and useful.
Bumps
[github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2)
from 1.18.9 to 1.18.12.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="3f28b5134e"><code>3f28b51</code></a>
Release 2023-02-03</li>
<li><a
href="6e8d17fd21"><code>6e8d17f</code></a>
Regenerated Clients</li>
<li><a
href="60dbdbb0da"><code>60dbdbb</code></a>
Update endpoints model</li>
<li><a
href="212910ac25"><code>212910a</code></a>
Update API model</li>
<li><a
href="eb8cb66b44"><code>eb8cb66</code></a>
Upgrade smithy to 1.27.2, correct query empty list serialization</li>
<li><a
href="24db9f5f6e"><code>24db9f5</code></a>
Update processcreds.CredentialProcessResponse visibility to public (<a
href="https://github-redirect.dependabot.com/aws/aws-sdk-go-v2/issues/1921">#1921</a>)</li>
<li><a
href="bd3003e29f"><code>bd3003e</code></a>
dependency: upgrade smithy to 1.27.2 and correct query empty list
serialization</li>
<li><a
href="0d94f223e8"><code>0d94f22</code></a>
Release 2023-02-02</li>
<li><a
href="2eec85ed13"><code>2eec85e</code></a>
Regenerated Clients</li>
<li><a
href="4ca6e32eed"><code>4ca6e32</code></a>
Update endpoints model</li>
<li>Additional commits viewable in <a
href="https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.9...config/v1.18.12">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Update the docker-compose.yml container build timestamp when running
tag_and_upload.sh. Does not currently handle updating the Go version in
the container tag.
Only build arm64 images for one version of Go.
Split build.sh into two scripts: build.sh (which installs apt and
Python) and install-go.sh (which installs a specific Go version and Go
dependencies). This allows reusing a cached layer for the build.sh step
across multiple Go versions.
Remove installation of fpm from build.sh. This is no longer needed since
#6669 and allows us to get rid of `rpm`, `ruby`, and `ruby-dev`.
Remove apt dependency on pkg-config, libtool, autoconf, and automake.
These were introduced in
https://github.com/letsencrypt/boulder/pull/4832 but aren't needed
anymore because we don't build softhsm2 ourselves (we get it from apt).
Remove apt dependency on cmake, libssl-dev, and openssl. I'm not totally
sure what these were needed for but they're not needed anymore.
Running this locally on my laptop for our current 3 GO_CI_VERSIONS and 1
GO_DEV_VERSION takes 23 minutes of wall time, dominated by the cross
build for arm64.
Remove `example.com` domain name, which was used by the deleted OldTLS
tests.
Remove GODEBUG=x509sha1=1.
Add a longer comment for the Consul DNS fallback in docker-compose.yml.
Use the "dnsAuthority" field for all gRPC clients in config-next,
instead of implicitly relying on the system DNS. This matches what we do
in prod.
Make "dnsAuthority" field of GRPCClientConfig mandatory whenever
SRVLookup or SRVLookups is used.
Make test/config/ocsp-responder.json use ServerAddress instead of
SRVLookup, like the rest of test/config.
The code path is now adequately tested in CI with try-release.yml. This
means it will no longer be automatically tested locally with `./t.sh`,
but it can be manually tested locally with `./tools/make-assets.sh`.
Also, to ensure CI has similar coverage to the old make-artifacts phase,
change make-deb.sh to make-assets.sh, and have it make all of rpm, deb,
and tar.
Change release.yml so it uploads the .tar.gz as well as the .deb.
Add a new time.Duration field, LagFactor, to both the SA's config struct
and the read-only SA's implementation struct. In the GetRegistration,
GetOrder, and GetAuthorization2 methods, if the database select returned
a NoRows error and a lagFactor duration is configured, then sleep for
lagFactor seconds and retry the select.
This allows us to compensate for the replication lag between our primary
write database and our read-only replica databases. Sometimes clients
will fire requests in rapid succession (such as creating a new order,
then immediately querying the authorizations associated with that
order), and the subsequent requests will fail because they are directed
to read replicas which are lagging behind the primary. Adding this
simple sleep-and-retry will let us mitigate many of these failures,
without adding too much complexity.
Fixes#6593
This reverts commit fdfea0d469.
With a Go security release out this week we prefer to do a single
release on the new Go version rather than trying to deploy the new
go-sql-driver version.
Split creating a release and uploading build assets to that release into
two separate steps. This allows the release creation step to have the
"continue on error" flag set, so that whichever release job completes
first can create the release, while the slower one will fail, move on,
and still successfully upload its files.
- Log counts of Authzs where CAA was rechecked/reused.
- Move the CAA recheck duration to a single variable in the RA.
- Add new method `InfoObject` to our logger.
Fixes#6560
Part of #6623
Also remove CSRDNSNames, CSRIPAddresses and CSREmailAddresses.
And add a new log field "DNSNames", for use in new-order, finalize, and
revoke requests.
Add a "RevocationReason" field in the "Extra" section for revoke
requests.
`ReuseValidAuthz` was introduced
here [1] and enabled in staging and production configs on 2016-07-13.
There was a brief stint during the TLS-SNI-01 challenge type removal where
SRE disabled it. However, time has finally come to remove this configuration
option. Issue #6623 will determine the feasibility of shorter authz
lifetimes and potentially the removal of authz reuse.
This change is broken up into two parts to allow SRE to safely remove
the flag from staging and production configs. We'll merge this PR, SRE
will deploy boulder and the config change, then we'll finish removing
`ReuseValidAuthz` configuration from the codebase.
[1] boulder commit 9abc212448
Part 1 of 2 for fixing #2734.
When the ocsp-responder queries the database for a certificate status,
we want to return a 404 if we don't find a certificate status row for
the serial in question. This is because we often receive requests for
serials which we never issued, and for very old (expired) serials whose
status data we may have purged from the database.
Previously, we did this by checking whether the error returned by the
database was the "ErrNoRows" used by Go's SQL library. However, when the
ocsp-responder uses the SA to get this information, rather than querying
the database directly, the SA's gRPC service returns berrors.NotFound
instead. The code was not checking for this error, and therefore turned
some requests that should have been 404s into 500s.
Check for both kinds of "not found" error, and return a 404 for both.
Add tests to ensure that we return responder.ErrNotFound in both cases.
Give ARI improved error messages when no request path is specified and
when parsing of the request path blob fails.
Also, add a tool which can be used to quickly generate ARI requests and
print their results, to make manual spot-checking easier.
Fixes#6629
Use SHA256 instead of SHA1 to compute the Subject Key Identifier when
producing new CA certs with the ceremony tool.
This change is safe, as the issuance package ensures that we directly
copy the Issuing CA's Subject Key Identifier value into the Authority
Key Identifier field of the template certificate we pass to
x509.CreateCertificate. Thus, all end-entity certs issued from a CA with
a SHA256 Subject Key Identifier will have the correct value for their
Authority Key Identifier, regardless of the method used to initially
compute that value.
This change adds 12 bytes to each self-signed Root CA certificate issued
with this change, 24 bytes to each Subordinate CA certificate (whether a
normal intermediate or a cross-signed root), and 12 bytes to each
end-entity issued from those intermediates.
Fixes#6630
Assign nonce prefixes for each nonce-service by taking the first eight
characters of the the base64url encoded HMAC-SHA256 hash of the RPC
listening address using a provided key. The provided key must be same
across all boulder-wfe and nonce-service instances.
- Add a custom `grpc-go` load balancer implementation (`nonce`) which
can route nonce redemption RPC messages by matching the prefix to the
derived prefix of the nonce-service instance which created it.
- Modify the RPC client constructor to allow the operator to override
the default load balancer implementation (`round_robin`).
- Modify the `srv` RPC resolver to accept a comma separated list of
targets to be resolved.
- Remove unused nonce-service `-prefix` flag.
Fixes#6404
This includes two feature flags: one that controls turning on the extra
database queries, and one that causes cert-checker to fail on missing
validations. If the second flag isn't turned on, it will just emit error
log lines. This will help us find any edge conditions we need to deal
with before making the new code trigger alerts.
Fixes#6562
A `core.Authorization` object has lots of fields (e.g. `status`,
`attempted`, `attemptedAt`) which are not relevant to a
newly-created authorization: a brand new authz can only be in
the "pending" state, cannot have been attempted already or have
been validated.
Fix a nil pointer dereference in `sa.NewOrderAndAuthzs` if a
`req *sapb.NewOrderAndAuthzsRequest` is passed into the
function with an inner nil `req.NewOrder`.
Add new tests.
- TestNewOrderAndAuthzs_MissingInnerOrder
- Checks that
the nil pointer dereference no longer occurs
- TestNewOrderAndAuthzs_NewAuthzExpectedFields
- Checks that the `Attempted`, `AttemptedAt`, `ValidationRecords`,
and `ValidationErrors` fields for a brand new authz in the
`pending` state are correctly defaulted to `nil` in
`sa.NewOrderAndAuthzs`.
Add a new test assertion `AssertBoxedNil` that returns true for the
existence of a "boxed nil" - a nil value wrapped in a non-nil interface
type.
Fixes#6535
---------
Co-authored-by: Samantha <hello@entropy.cat>
Add go1.20 as a new version to run tests on, and to build release
artifacts from. Fix one test which was failing because it was
accidentally relying on consistent (i.e. unseeded) non-cryptographic
random number generation, which go1.20 now automatically seeds at import
time.
Update the version of golangci-lint used in our docker containers to the
new version that has go1.20 support. Remove a number of nolint comments
that were required due to an old version of the gosec linter.
For consistency, put the error field at the end of unstructured log
lines to make them more ... structured.
Adds the `issuerID` field to "orphaning certificate" log line in the CA
to match the "orphaning precertificate" log line.
Fixes broken tests as a result of the CA and bdns log line change.
Fixes#5457
Remove the PortConfig field from both the VA's config struct and from
the NewValidationAuthorityImpl constructor. This config item is no
longer used anywhere, and removing this prevents us from accidentally
overriding the "Authorized Ports" (80 and 443) which are required by the
Baseline Requirements.
Unit tests are still able to override the httpPort and tlsPort fields of
the ValidationAuthorityImpl.
Fixes#3940
Add validation of input parameters as unquoted MariaDB identifiers, and
document the regex that does it.
Accept a narrower interface (Queryer) for `Insert()`.
Take a list of fields rather than a string containing multiple fields,
to make validation simpler. Rename retCol to returningColumn.
Document safety properties and requirements.
If the input key asserts that it is a PKCS#1 key ("BEGIN RSA PRIVATE
KEY"), we can safely return the error from parsing as PKCS1, rather than
trying all key types and returning a generic error.
Neither our testing, staging, nor production configs use the
DBConfig.DBConnect config value. Remove it.
To connect to a database, you have to provide a connection URL. These
URLs often contain sensitive information such as DB usernames and
passwords, so we don't store them directly in our configs -- instead, we
store paths to files which contain these strings, and provision those
files via a separate mechanism. We maintained the ability to provide a
URL directly in the config for the sake of easy testing, but have not
used it for that purpose for some time now.
Add an integration test which verifies that we reject finalize requests
with CSRs containing a fermat-factorizable public key.
Originally this change was also going to remove our Fermat factorization
implementation from good_key.go, and simply rely on the similar check in
zlint's e_rsa_fermat_factorization check. However, while relying solely
on the lint works, it causes us to block such requests with a 500
serverInternal error, because we consider failing lints to be our fault.
This would be a regression from the current status quo, where such
requests are rejected with a 400 badCSR error and details of the
factorization, so we are leaving our goodkey checks in place.
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.4.0 to
0.6.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ec5565b1b7"><code>ec5565b</code></a>
README.md: update documentation of module versioning</li>
<li><a
href="c8236a6712"><code>c8236a6</code></a>
unicode/bidi: remove unused global</li>
<li><a
href="ada7473102"><code>ada7473</code></a>
all: remove redundant type conversion</li>
<li>See full diff in <a
href="https://github.com/golang/text/compare/v0.4.0...v0.6.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
In #6293, we gave the ocsp-responder the ability to use a gRPC
connection to the SA to get status information for certificates, rather
than using a database connection directly. However, that change
neglected to make the database connection configuration optional: an
ocsp-responder with an SA gRPC client configured would never use its
database connection, but if it wasn't configured it would refuse to
start. Fix this oversight by making the DBConfig stanza optional.
From the go-jose/go-jose v2.6.1 merge commit:
> The square/go-jose repo had one commit "Fix EC thumbprint template"
> which existed on the v2.6.0 tag but not on the v2 branch. This means
that
> it was missed in the recent PR which merged square's v2 branch into
this
> repository's v2 branch. This also means that the current v2.6.0 tag in
this
> repo points to a commit which is not on any branch in this repo.
Now that Aaron has pushed a go-jose/go-jose v2.6.1 tag, we can upgrade
to that and remove the indirect reference to square/go-jose v2.6.0.
Related to https://github.com/letsencrypt/boulder/issues/6573
Simplify the control flow of the FinalizeOrder handler to make it easier
to read and reason about:
- Move all validation to before we set the order to Processing, and put
it all in a single helper funcion.
- Move almost all logEvent/trace handling directly into FinalizeOrder so
it cannot be missed.
- Flatten issueCertificate and issueCertificateInner into a single
helper function, now that they're no longer being called from both
ACMEv1 and v2 entry points.
- Other minor cleanups, such as making SolvedBy not return a pointer and
making matchesCSR private.
This paves the way for making both issueCertificateInner and failOrder
asynchronous, which we plan to do in the near future.
Part of #6575
Phil Porada