Neither our testing, staging, nor production configs use the
DBConfig.DBConnect config value. Remove it.
To connect to a database, you have to provide a connection URL. These
URLs often contain sensitive information such as DB usernames and
passwords, so we don't store them directly in our configs -- instead, we
store paths to files which contain these strings, and provision those
files via a separate mechanism. We maintained the ability to provide a
URL directly in the config for the sake of easy testing, but have not
used it for that purpose for some time now.
Add an integration test which verifies that we reject finalize requests
with CSRs containing a fermat-factorizable public key.
Originally this change was also going to remove our Fermat factorization
implementation from good_key.go, and simply rely on the similar check in
zlint's e_rsa_fermat_factorization check. However, while relying solely
on the lint works, it causes us to block such requests with a 500
serverInternal error, because we consider failing lints to be our fault.
This would be a regression from the current status quo, where such
requests are rejected with a 400 badCSR error and details of the
factorization, so we are leaving our goodkey checks in place.
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.4.0 to
0.6.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ec5565b1b7"><code>ec5565b</code></a>
README.md: update documentation of module versioning</li>
<li><a
href="c8236a6712"><code>c8236a6</code></a>
unicode/bidi: remove unused global</li>
<li><a
href="ada7473102"><code>ada7473</code></a>
all: remove redundant type conversion</li>
<li>See full diff in <a
href="https://github.com/golang/text/compare/v0.4.0...v0.6.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
In #6293, we gave the ocsp-responder the ability to use a gRPC
connection to the SA to get status information for certificates, rather
than using a database connection directly. However, that change
neglected to make the database connection configuration optional: an
ocsp-responder with an SA gRPC client configured would never use its
database connection, but if it wasn't configured it would refuse to
start. Fix this oversight by making the DBConfig stanza optional.
From the go-jose/go-jose v2.6.1 merge commit:
> The square/go-jose repo had one commit "Fix EC thumbprint template"
> which existed on the v2.6.0 tag but not on the v2 branch. This means
that
> it was missed in the recent PR which merged square's v2 branch into
this
> repository's v2 branch. This also means that the current v2.6.0 tag in
this
> repo points to a commit which is not on any branch in this repo.
Now that Aaron has pushed a go-jose/go-jose v2.6.1 tag, we can upgrade
to that and remove the indirect reference to square/go-jose v2.6.0.
Related to https://github.com/letsencrypt/boulder/issues/6573
Simplify the control flow of the FinalizeOrder handler to make it easier
to read and reason about:
- Move all validation to before we set the order to Processing, and put
it all in a single helper funcion.
- Move almost all logEvent/trace handling directly into FinalizeOrder so
it cannot be missed.
- Flatten issueCertificate and issueCertificateInner into a single
helper function, now that they're no longer being called from both
ACMEv1 and v2 entry points.
- Other minor cleanups, such as making SolvedBy not return a pointer and
making matchesCSR private.
This paves the way for making both issueCertificateInner and failOrder
asynchronous, which we plan to do in the near future.
Part of #6575
Bumps
[github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2)
from 1.27.1 to 1.30.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md">github.com/aws/aws-sdk-go-v2/service/s3's
changelog</a>.</em></p>
<blockquote>
<h1>Release (2023-01-10)</h1>
<h2>Module Highlights</h2>
<ul>
<li><code>github.com/aws/aws-sdk-go-v2/service/location</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/location/CHANGELOG.md#v1210-2023-01-10">v1.21.0</a>
<ul>
<li><strong>Feature</strong>: This release adds support for two new
route travel models, Bicycle and Motorcycle which can be used with Grab
data source.</li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/rds</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/rds/CHANGELOG.md#v1400-2023-01-10">v1.40.0</a>
<ul>
<li><strong>Feature</strong>: This release adds support for configuring
allocated storage on the CreateDBInstanceReadReplica,
RestoreDBInstanceFromDBSnapshot, and RestoreDBInstanceToPointInTime
APIs.</li>
</ul>
</li>
</ul>
<h1>Release (2023-01-09)</h1>
<h2>Module Highlights</h2>
<ul>
<li><code>github.com/aws/aws-sdk-go-v2/service/ecrpublic</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/ecrpublic/CHANGELOG.md#v1150-2023-01-09">v1.15.0</a>
<ul>
<li><strong>Feature</strong>: This release for Amazon ECR Public makes
several change to bring the SDK into sync with the API.</li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/kendraranking</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/kendraranking/CHANGELOG.md#v100-2023-01-09">v1.0.0</a>
<ul>
<li><strong>Release</strong>: New AWS service client module</li>
<li><strong>Feature</strong>: Introducing Amazon Kendra Intelligent
Ranking, a new set of Kendra APIs that leverages Kendra semantic ranking
capabilities to improve the quality of search results from other search
services (i.e. OpenSearch, ElasticSearch, Solr).</li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/networkfirewall</code>:
<a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/networkfirewall/CHANGELOG.md#v1230-2023-01-09">v1.23.0</a>
<ul>
<li><strong>Feature</strong>: Network Firewall now supports the Suricata
rule action reject, in addition to the actions pass, drop, and
alert.</li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/workspacesweb</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/workspacesweb/CHANGELOG.md#v190-2023-01-09">v1.9.0</a>
<ul>
<li><strong>Feature</strong>: This release adds support for a new portal
authentication type: AWS IAM Identity Center (successor to AWS Single
Sign-On).</li>
</ul>
</li>
</ul>
<h1>Release (2023-01-06)</h1>
<h2>Module Highlights</h2>
<ul>
<li><code>github.com/aws/aws-sdk-go-v2/service/acmpca</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/acmpca/CHANGELOG.md#v1210-2023-01-06">v1.21.0</a>
<ul>
<li><strong>Feature</strong>: Added revocation parameter validation:
bucket names must match S3 bucket naming rules and CNAMEs conform to
RFC2396 restrictions on the use of special characters in URIs.</li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/auditmanager</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/auditmanager/CHANGELOG.md#v1230-2023-01-06">v1.23.0</a>
<ul>
<li><strong>Feature</strong>: This release introduces a new data
retention option in your Audit Manager settings. You can now use the
DeregistrationPolicy parameter to specify if you want to delete your
data when you deregister Audit Manager.</li>
</ul>
</li>
</ul>
<h1>Release (2023-01-05)</h1>
<h2>General Highlights</h2>
<ul>
<li><strong>Dependency Update</strong>: Updated to the latest SDK module
versions</li>
</ul>
<h2>Module Highlights</h2>
<ul>
<li><code>github.com/aws/aws-sdk-go-v2/service/accessanalyzer</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/accessanalyzer/CHANGELOG.md#v1190-2023-01-05">v1.19.0</a>
<ul>
<li><strong>Feature</strong>: Add
<code>ErrorCodeOverride</code><code>aws/smithy-go#401</code></li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/account</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/account/CHANGELOG.md#v180-2023-01-05">v1.8.0</a>
<ul>
<li><strong>Feature</strong>: Add
<code>ErrorCodeOverride</code><code>aws/smithy-go#401</code></li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/acm</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/acm/CHANGELOG.md#v1170-2023-01-05">v1.17.0</a>
<ul>
<li><strong>Feature</strong>: Add
<code>ErrorCodeOverride</code><code>aws/smithy-go#401</code></li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/acmpca</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/acmpca/CHANGELOG.md#v1200-2023-01-05">v1.20.0</a>
<ul>
<li><strong>Feature</strong>: Add
<code>ErrorCodeOverride</code><code>aws/smithy-go#401</code></li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/alexaforbusiness</code>:
<a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/alexaforbusiness/CHANGELOG.md#v1150-2023-01-05">v1.15.0</a>
<ul>
<li><strong>Feature</strong>: Add
<code>ErrorCodeOverride</code><code>aws/smithy-go#401</code></li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/amp</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/amp/CHANGELOG.md#v1160-2023-01-05">v1.16.0</a>
<ul>
<li><strong>Feature</strong>: Add
<code>ErrorCodeOverride</code><code>aws/smithy-go#401</code></li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/amplify</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/amplify/CHANGELOG.md#v1130-2023-01-05">v1.13.0</a>
<ul>
<li><strong>Feature</strong>: Add
<code>ErrorCodeOverride</code><code>aws/smithy-go#401</code></li>
</ul>
</li>
<li><code>github.com/aws/aws-sdk-go-v2/service/amplifybackend</code>: <a
href="https://github.com/aws/aws-sdk-go-v2/blob/main/service/amplifybackend/CHANGELOG.md#v1140-2023-01-05">v1.14.0</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="116a622a26"><code>116a622</code></a>
Release 2023-01-05</li>
<li><a
href="ce039452b6"><code>ce03945</code></a>
Regenerated Clients</li>
<li><a
href="095bbfff59"><code>095bbff</code></a>
Update API model</li>
<li><a
href="2998a9800a"><code>2998a98</code></a>
Regenerate clients with <code>ErrorCodeOverride</code> (<a
href="https://github-redirect.dependabot.com/aws/aws-sdk-go-v2/issues/1969">#1969</a>)</li>
<li><a
href="1b0a07d93d"><code>1b0a07d</code></a>
Release 2023-01-04</li>
<li><a
href="ff5b1c7a27"><code>ff5b1c7</code></a>
Regenerated Clients</li>
<li><a
href="cabea36bb4"><code>cabea36</code></a>
Update API model</li>
<li><a
href="cd385dc3b8"><code>cd385dc</code></a>
Update links to point to smithy.io</li>
<li><a
href="4dd79b8978"><code>4dd79b8</code></a>
Rename SyntheticClone to Synthetic</li>
<li><a
href="b302f0a86c"><code>b302f0a</code></a>
Release 2023-01-03</li>
<li>Additional commits viewable in <a
href="https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.27.1...service/s3/v1.30.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This policy existed to ensure that, when updating a dependency to a
non-release-tagged commit, we weren't accidentally bringing in a
broken version of that dependency. Since it was instituted, we have
greatly reduced the number of non-versioned dependencies. And we
have extensive tests of our own that should detect if any of the behavior
we use from one of these dependencies is broken. Therefore this
requirement has fallen by the wayside.
Update the document to reflect current practices.
These helpers are only called in one place each, and provide no real
value via abstraction. Also, they're performing gymnastics in order to
be able to use `base64.URLEncoding`, instead of simply using
`base64.RawURLEncoding`.
The core.CertificateRequest wrapper type only had two fields: an
x509.CertificateRequest, and a slice of bytes representing the unparsed
version of the same request. However, x509.CertificateRequest carries
around its own field .Raw, which serves the same purpose. Also, we
weren't even referencing the .Bytes field anyway. Therefore, get rid of
this redundant wrapper type.
This also makes it clearer just how far the original CSR makes it
through our system. I'd like a future change to remove the CSR from the
request to the CA service, replacing it with a structured object that
only exposes exactly the fields of the CSR we care about, such as names,
public key, and whether or not the must-staple extension should be set.
When attempting to add TLS probe monitoring, got the error `TLS is not a
registered Prober type`. This PR adds TLS Prober to `observer.go` to
complete its registration and adds TLS Prober to the observer README.
Co-authored-by: Samantha <hello@entropy.cat>
We use this pattern in several places: there is a query that needs to
have a variable number of placeholders (question marks) in it, depending
on how many items we are inserting or querying for. For instance, when
issuing a precertificate we add that precertificate's names to the
"issuedNames" table. To make things more efficient, we do that in a
single query, whether there is one name on the certificate or a hundred.
That means interpolating into the query string with series of question
marks that matches the number of names.
We have a helper type MultiInserter that solves this problem for simple
inserts, but it does not solve the problem for selects or more complex
inserts, and we still have a number of places that generate their
sequence of question marks manually.
This change updates addIssuedNames to use MultiInserter. To enable that,
it also narrows the interface required by MultiInserter.Insert, so it's
easier to mock in tests.
This change adds the new function db.QuestionMarks, which generates e.g.
`?,?,?` depending on the input N.
In a few places I had to rename a function parameter named `db` to avoid
shadowing the `db` package.
Although we have a metric for counting how often we reuse a valid
authorization, no code has been incrementing that metric since 2021.
Re-enable it.
Additionally, that metric doesn't help us know how old authorizations
are when they get reused, which is useful information when deciding
whether or how much to shrink our authorization reuse lifetime. Add a
new histogram metric to track the ages of authorizations when they're
attached to an order.
In live.go we use a semaphore to limit how many inflight signing
requests we can have, so a flood of OCSP traffic doesn't flood our CA
instances. If traffic exceeds our capacity to sign responses for long
enough, we want to eventually start fast-rejecting inbound requests that
are unlikely to get serviced before their deadline is reached. To do
that, add a MaxSigningWaiters config field to the OCSP responder.
Note that the files in //semaphore are forked from x/sync/semaphore,
with modifications to add the MaxWaiters field and functionality.
Fixes#6392
Add a new kind of prober to boulder-observer which makes a TLS
connection to the target hostname and expects the certificate presented
for the TLS handshake to have certain properties, such as being valid,
expired, or revoked.
Part of #5927
We use partitioning to be able to clean up old data, and partitioning is
incompatible with unique indexes. We still have a unique index on
`serial`, and these tables are downstream from there. There still may
some duplicates, like when a certificate is treated as orphaned but was
actually successfully added to the DB; when we later go to incorporate
it a duplicate will show up.
This reflects changes already made in prod.
This PR removes a unittest that coincidentally relied on these indexes
to generate an error case it needed: `TestAddPrecertificateStatusFail`.
That test was added in #5918. We can bring that test back with a
significant refactoring to change `*db.WrappedMap` to an interface, but
in the meantime we're prioritizing landing this PR so we have a more
realistic integration test environment.
Adding an insecure option to HTTP prober so that it can still check the
status of sites that we expect to be insecure (e.g. expired sites).
Co-authored-by: Aaron Gable <aaron@aarongable.com>
The SA's GetOrder method issues four separate database queries:
- get the Order object itself
- get the list of Names associated with it
- get the list of Authorization IDs associated with it
- get the contents of those Authorizations themselves
These four queries can hit different database replicas with different
amounts of replication lag, and therefore different views of the
universe. This can result in inconsistent results, such as the Order
existing, but not having any Authorization IDs associated with it.
Conduct these four queries all within a single transaction, to ensure
they all hit a single read-replica with a consistent view of the world.
Fixes#6540
Originally, WFEs had a built-in nonce service. Then we added a "remote
nonce service" via gRPC, but we kept a fallback path for when the remote
nonce service was not configured, to use a built-in nonce service. This
PR removes that fallback path.
Since the fallback path was relied on by the unittests, this also
refactors the unittests to use a gRPC-style nonce service (but in-memory
for the unittests).
Fixes#6530
Fixes#6520.
Note: The unittest for this would be fairly simple - add a duplicate
entry in the certificateStatus table. But we can't do that because our
dev environment currently has a UNIQUE KEY on that table. So adding a
unittest update for this is blocked on #6519.
Delete the NewOrder and NewAuthorizations2 methods from the SA's gRPC
interface. These methods have been replaced by the unified
NewOrderAndAuthzs method, which performs both sets of insertions in a
single transaction.
Also update the SA and RA unittests to not rely on these methods for
setting up test data that other functions-under-test rely on. In most
cases, replace calls to NewOrder with calls to NewOrderAndAuthzs. In the
SA tests specifically, replace calls to NewAuthorizations2 with a
streamlined helper function that simply does the single necessary
database insert.
Fixes#6510Fixes#5816
Update our implementation of ARI to return a renewal window entirely in
the past (i.e., suggesting immediate renewal) if the certificate in
question has been revoked for any reason. This will allow clients which
implement ARI to discover that they need to replace their certificate
without having to query OCSP directly, especially as we move into a
future where OCSP is mostly supplanted by aggregated CRLs.
Fixes#6503
In the WFE, ocsp-responder, and crl-updater, switch from using
StorageAuthorityClients to StorageAuthorityReadOnlyClients. This ensures
that these services cannot call methods which write to our database.
Fixes#6454
The `digest` value in AddCertificate's response message is never used by
any callers. Remove it, replacing the whole response message with
google.protobuf.Empty, to mirror the AddPrecertificate method.
This swap is safe, because message names are not sent on the network,
and empty message fields are omitted from the wire format entirely, so
sending the predefined Empty message is identical to sending an empty
AddCertificateResponse message. Since no client is inspecting the
response to access the digest field, sending an empty response will not
break any clients.
Fixes#6498
When a client cancels their HTTP request, that propagates into
gRPC-level cancellation. But we don't want those canceled RPCs to show
up as 500s, we want them to show up as 408s. We do that by producing a
special ProblemDetails at the gRPC level. However, for that trick to
work, we need to make sure errors from RPC methods get passed through
web.ProblemDetailsForError. There were some places where we didn't do
this and instead created a from-scratch ProblemDetails. This resulted in
spurious 500s.
My methodology was to look at every method call on each of the WFE's
fields that represents a gRPC backend: `ra`, `sa`, `accountGetter`,
`nonceService`, `remoteNonceServe`. If the error handling for that call
did not use web.ProblemDetailsForError, I changed it to use that.
Fixes#6524
In draft because still needs tests.
Followup from #6506.
As noted in that PR: I'm not aware of any way to trigger invalid UTF-8
from the layers below this, so I can't think of a good way to unittest
it.
Aaron Gable