OpenTelemetry has "semantic conventions" which are versioned
independently of the software package, as it describes the semantics of
the resources being produced. Previously, we'd combined
`resource.Default()` using the `Merge` function with our own resources.
Merge, however, doesn't handle merging resources with different semantic
conventions. This means that every dependabot PR that bumps otel will
break when the `resources.Default` has a new version.
That doesn't seem worth it for the default resources, so just provide
our own resources which have everything we care about. I've added the
PID which we didn't have before but will be interesting. We will lose
the SDK's version, but I don't think that matters.
For more discussion on this topic, see
https://github.com/open-telemetry/opentelemetry-go/issues/3769
Have the release action write the checksums to a file whose name matches
the name of the boulder deb and rpm files whose checksums it contains.
This prevents name-collision errors when we are running multiple Go
versions side-by-side in CI.
To demonstrate this, augment the try-release action to write the
checksums to the same file, print the list of files it would upload, and
print the contents of the checksums file. Also allow the try-release
workflow to be triggered manually.
Clean up how we handle identifiers throughout the Boulder codebase by
- moving the Identifier protobuf message definition from sa.proto to
core.proto;
- adding support for IP identifier to the "identifier" package;
- renaming the "identifier" package's exported names to be clearer; and
- ensuring we use the identifier package's helper functions everywhere
we can.
This will make future work to actually respect identifier types (such as
in Authorization and Order protobuf messages) simpler and easier to
review.
Part of https://github.com/letsencrypt/boulder/issues/7311
When creating an authorization, populate it with all challenges
appropriate for that identifier, regardless of whether those challenge
types are currently "enabled" in the config. This ensures that
authorizations created during a incident for which we can temporarily
disabled a single challenge type can still be validated via that
challenge type after the incident is over.
Also, when finalizing an order, check that the challenge type used to
validation each authorization is not currently disabled. This ensures
that, if we temporarily disable a single challenge due to an incident,
we don't issue any more certificates using authorizations which were
fulfilled using that disabled challenge.
Note that standard rolling deployment of this change is not safe if any
challenges are disabled at the same time, due to the possibility of an
updated RA not filtering a challenge when writing it to the database,
and then a non-updated RA not filtering it when reading from the
database. But if all challenges are enabled then this change is safe for
normal deploy.
Fixes https://github.com/letsencrypt/boulder/issues/5913
Have our gRPC server interceptor check for excessive clock skew between
its own clock and gRPC client clocks. Do this by taking advantage of the
client request timestamp that most clients already supply for the
purpose of measuring cross-service latency. If the included timestamp is
more than 10 minutes from the gRPC server's local time, immediately
error out.
To keep the integration tests -- which heavily rely on clock
manipulation -- working, use build tags to disable this behavior during
integration testing.
Fixes https://github.com/letsencrypt/boulder/issues/7684
Begin testing on go1.23. To facilitate this, also update /x/net,
golangci-lint, staticcheck, and pebble-challtestsrv to versions which
support go1.23. As a result of these updates, also fix a handful of new
lint findings, mostly regarding passing non-static (i.e. potentially
user-controlled) format strings into Sprintf-style functions.
Additionally, delete one VA unittest that was duplicating the checks
performed by a different VA unittest, but with a context timeout bug
that caused it to break when go1.23 subtly changed DialContext behavior.
Have the WFE ask the RA for authorizations, rather than asking the SA
directly. This extra layer of indirection allows us to filter out
challenges which have been disabled, so that clients don't think they
can attempt challenges that we have disabled.
Also shuffle the order of challenges within the authz objects rendered
by the API. We used to have code which does this at authz creation time,
but of course that was completely ineffectual once we stored the
challenges as just a bitmap in the database.
Update the WFE unit tests to mock RA.GetAuthorization instead of
SA.GetAuthorization2. This includes making the mock more accurate, so
that (e.g.) valid authorizations contain valid challenges, and the
challenges have their correct types (e.g. "http-01" instead of just
"http"). Also update the OTel tracing test to account for the new RPC.
Part of https://github.com/letsencrypt/boulder/issues/5913
- Add feature flag `UseKvLimitsForNewOrder`
- Add feature flag `UseKvLimitsForNewAccount`
- Flush all Redis shards before running integration or unit tests, this
avoids false positives between local testing runs
Fixes#7664
Blocked by #7676
Implements tooling in `admin` that allows an operator to
administratively pause account/identifier pairs and unpause
whole accounts. This functionality mirrors the self-service
capabilities of the SFE, so that we can administratively intervene
in the pausing and unpausing process.
The new `pause-identifier` subcommand accepts a single form
of input, specified by the `-batch-file` flag. This expects a CSV
where each row is an accountID, identifierType, identifierValue
triple.
The new `unpause-account` subcommand accepts either a single
account ID with the `-account` flag, or a text file containing a list
of account IDs with the `-batch-file` flag.
Relates to https://github.com/letsencrypt/boulder/issues/7406
Fixes https://github.com/letsencrypt/boulder/issues/7618
Updates:
github.com/weppos/publicsuffix-go from
v0.30.3-0.20240510084413-5f1d03393b3d to
v0.40.3-0.20240815124645-a8ed110559c9
golang.org/x/crypto from v0.24.0 to v0.26.0
golang.org/x/net from v0.26.0 to v0.28.0
golang.org/x/sync from v0.7.0 to v0.8.0
golang.org/x/term from v0.21.0 to v0.23.0
golang.org/x/text from v0.16.0 to v0.17.0
golang.org/x/sys from v0.21.0 to v0.23.0
These two methods were already nearly identical, their only meaningful
differences being the set of statuses they select for in their database
query, and the condition on which they prioritize entries in their
output.
Update them to have the exact same structure and logic. Most
meaningfully, update GetValidAuthorizations2 to hint the database to use
the same index which GetAuthorizations2 already hints.
- Check `CertificatesPerDomain` at newOrder and spend at Finalize time.
- Check `CertificatesPerAccountPerDomain` at newOrder and spend at
Finalize time.
- Check `CertificatesPerFQDNSet` at newOrder and spend at Finalize time.
- Fix a bug
in`FailedAuthorizationsPerDomainPerAccountSpendOnlyTransaction()` which
results in failed authorizations being spent for the exact FQDN, not the
eTLD+1.
- Remove redundant "max names" check at transaction construction time
- Enable key-value rate limits in the RA
Remove the id, identifierValue, status, and challenges fields from
sapb.NewAuthzRequest. These fields were left behind from the previous
corepb.Authorization request type, and are now being ignored by the SA.
Since the RA is no longer constructing full challenge objects to include
in the request, remove pa.ChallengesFor and replace it with the much
simpler pa.ChallengeTypesFor.
Part of https://github.com/letsencrypt/boulder/issues/5913
Fix a bug added in #7653 which sometimes attributed an "Allowed"
`Transaction` to the amalgamated "Denied" `*Decision`. Instead, always
return the most restrictive `*Decision` in the batch.
Remove a debug `fmt.Printf()` call added in #7653
Find all gRPC fields which represent DNS Names -- sometimes called
"identifier", "hostname", "domain", "identifierValue", or other things
-- and unify their naming. This naming makes it very clear that these
values are strings which may be included in the SAN extension of a
certificate with type dnsName.
As we move towards issuing IP Address certificates, all of these fields
will need to be replaced by fields which carry both an identifier type
and value, not just a single name. This unified naming makes it very
clear which messages and methods need to be updated to support
non-dnsName identifiers.
Part of https://github.com/letsencrypt/boulder/issues/7647
- Instruct callers to call *Decision.Result() to check the result of
rate limit transactions
- Preserve the Transaction within the resulting *Decision
- Generate consistently formatted verbose errors using the metadata
found in the *Decision
- Fix broken key-value rate limits integration test in
TestDuplicateFQDNRateLimit
Fixes#7577
This compile error was introduced by the combination of
https://github.com/letsencrypt/boulder/pull/7650 (which changed the type
of mockSAWithAuthzs.authzs) and
https://github.com/letsencrypt/boulder/pull/7652 (which introduced a new
usage of mockSAWithAuthzs.authzs). Because the latter PR introduced a
new usage, rather than modifying an existing usage, it didn't create a
merge conflict and wasn't caught by GitHub's mergeability checker.
Add a new "GetAuthorization" method to the RA. This method is very
similar to the SA's existing "GetAuthorization2" method, except that it
also uses the RA's built-in Policy Authority to filter out any
challenges which are currently disabled.
In a follow-up change, the WFE will be updated to use this method when
retrieving authorizations and challenges for display, so that we can
ensure disabled challenges are not presented to ACME clients.
Part of https://github.com/letsencrypt/boulder/issues/5913
Replace all of Boulder's usage of the Go stdlib "math/rand" package with
the newer "math/rand/v2" package which first became available in go1.22.
This package has an improved API and faster performance across the
board.
See https://go.dev/blog/randv2 and https://go.dev/blog/chacha8rand for
details.
This is a followup to https://github.com/letsencrypt/boulder/pull/7646,
updating two other RA methods (RevokeCertByApplicant and NewOrder) which
call different SA methods (GetValidAuthorizations2 and
GetAuthorizations2) but receive the same return type
(sapb.Authorizations) from the SA to use that type's new field.
* Make `registrationID` unsigned to match staging/production so that we
have a large pool of autoincrement IDs
* Change the primary key line to perform better filtering to appease the
query planner
Simplify SA.GetValidOrderAuthorizations2 so that it no longer conditions
the query on the status, expiry, or registration ID of the authorization
rows. This gives the query much better performance, because it no longer
tries to use an overly-large index, and fall back to large row-scans
when the query planner decides the index is too large.
While we're here, also improve the return type of
GetValidOrderAuthorizations2, so that instead of returning a map of
names to authorizations, it simply returns a list of authzs. This both
reduces the size of the gRPC message (once the old map is fully
removed), and improves its correctness because we cannot count on names
to be unique across multiple identifier types.
Finally, improve the RA code which calls SA.GetValidOrderAuthorizations2
to handle this improved return type, to make fewer assumptions about
identifier types, and to separate static authorization-checking from CAA
rechecking.
Fixes https://github.com/letsencrypt/boulder/issues/7645
Within the NewOrderAndAuthzsRequest, replace the corepb.Authorization
field with a new sapb.NewAuthzRequest message. This message has all of
the same field types and numbers, and the RA still populates all of
these fields when constructing a request, for backwards compatibility.
But it also has new fields (an Identifier carrying both type and value,
a list of challenge types, and a challenge token) which the RA
preferentially consumes if present.
This causes the content of our NewOrderAndAuthzsRequest to more closely
match the content that will be created at the database layer. Although
this may seem like a step backwards in terms of abstraction, it is also
a step forwards in terms of both efficiency (not having to transmit
multiple nearly-identical challenge objects) and correctness (being
guaranteed that the token is actually identical across all challenges).
After this change is deployed, it will be followed by a change which
removes the old fields from the NewAuthzRequest message, to realize the
efficiency gains.
Part of https://github.com/letsencrypt/boulder/issues/5913
Updates `github.com/aws/aws-sdk-go-v2` from 1.30.1 to 1.30.3
Updates `github.com/aws/aws-sdk-go-v2/config` from 1.27.24 to 1.27.27
Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.58.0 to 1.58.2
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Move the two lint-configuration keys, LintConfig and IgnoreLints, from
the top-level CA.Issuance config stanza into each individual
CA.Issuance.CertProfiles stanza. This allows us to have
differently-configured lints for different profiles, to ensure that our
linting regime is as strict as possible.
Without this change, it would be necessary for us to ignore both the
"common name included" and the "no subject key id" lints at the
top-level, when in fact each of those warnings only triggers on one of
our two profiles.
Fixes https://github.com/letsencrypt/boulder/issues/7635
Call `RA.UnpauseAccount` for valid unpause form submissions.
Determine and display the appropriate outcome to the Subscriber based on
the count returned by `RA.UnpauseAccount`:
- If the count is zero, display the "Account already unpaused" message.
- If the count equals the max number of identifiers allowed in a single
request, display a page explaining the need to visit the unpause URL
again.
- Otherwise, display the "Successfully unpaused all N identifiers"
message.
Apply per-request timeout from the SFE configuration.
Part of https://github.com/letsencrypt/boulder/issues/7406
Add three new keys to the CA's ProfileConfig:
- OmitKeyEncipherment causes the keyEncipherment Key Usage to be omitted
from certificates with RSA public keys. We currently include it for
backwards compatibility with TLS 1.1 servers that don't support modern
cipher suites, but this KU is completely useless as of TLS 1.3.
- OmitClientAuth causes the tlsClientAuthentication Extended Key Usage
to be omitted from all certificates. We currently include it to support
any subscribers who may be relying on it, but Root Programs are moving
towards single-purpose hierarchies and its inclusion is being
discouraged.
- OmitSKID causes the Subject Key Identifier extension to be omitted
from all certificates. We currently include this extension because it is
recommended by RFC 5280, but it serves little to no practical purpose
and consumes a large number of bytes, so it is now NOT RECOMMENDED by
the Baseline Requirements.
Make substantive changes to issuer.requestValid and issuer.Prepare to
implement the desired behavior for each of these options. Make a very
slight change to ra.matchesCSR to generally allow for serverAuth-only
EKUs. Improve the unit tests of both the //ca and //issuance packages to
cover the new behavior.
Part of https://github.com/letsencrypt/boulder/issues/7610
Currently, the SFE displays "An error occurred while unpausing your
account" in scenarios where it's not correct or helpful.
- Return a helpful message when a Subscriber attempts to access the
unpause form but fails to copy the entire link
- Return a helpful message when a Subscriber attempts to unpause using
an expired JWT
- Some small cleanups that make the code a little more mistake-proof.
Part of https://github.com/letsencrypt/boulder/issues/7406
Running this workflow on both `ready_for_review` and `review_requested`
was causing duplicate comments to show up when PRs were moved out of
draft mode. This is because moving out of draft mode would both trigger
a workflow run and automatically request review, which in turn would
trigger a second workflow run, and the two runs were triggered in such
quick succession that our commentMarker detection wouldn't work.
One of our goals with profiles is to allow different profiles to have
different validity periods. While the profiles already had the ability
to enforce different maximum backdates and validities, the CA still had
separate global configuration for what the backdate and validity period
should actually be.
Move the computation of the notBefore and notAfter timestamps into the
issuance package, so that it can be based on the profile's configured
backdate and validity durations. Deprecate the global "backdate" and
"expiry" config fields, as they are no longer used. Finally, add more
validation for the profile's backdate and validity.
Part of https://github.com/letsencrypt/boulder/issues/7610
Have the RA's UnpauseAccount gRPC method forward the requested account
ID to the SA's corresponding method, and in turn forward the SA's count
of unpaused identifiers back to the caller in the response.
Changing the response message from emptypb.Empty to a new
rapb.UnpauseAccountResponse is safe, because message names are not
transmitted on the wire, only message field numbers.
While we're here, drastically simplify the wfe_test and sfe_test Mock
RAs, so they don't have to implement methods that aren't actually used
by the tests.
Fixes https://github.com/letsencrypt/boulder/issues/7536
Matthew McPherrin
Aaron Gable
James Renken
Samantha Frank
Phil Porada
Adin Schmahmann
dependabot[bot]
Romuald Brunet