Commit Graph

76 Commits

Author SHA1 Message Date
James Renken 7214b285e4
identifier: Remove helper funcs from PB identifiers migration (#8236)
Remove `ToDNSSlice`, `FromProtoWithDefault`, and
`FromProtoSliceWithDefault` now that all their callers are gone. All
protobufs but one have migrated from DnsNames to Identifiers.

Remove TODOs for the exception, `ValidationRecord`, where an identifier
type isn't appropriate and it really only needs a string.

Rename `corepb.ValidationRecord.DnsName` to `Hostname` for clarity, to
match the corresponding PB's field name.

Improve various comments and docs re: IP address identifiers.

Depends on #8221 (which removes the last callers)
Fixes #8023
2025-06-13 12:55:32 -07:00
James Renken ac68828f43
Replace most uses of net.IP with netip.Addr (#8205)
Retain `net.IP` only where we directly work with `x509.Certificate` and
friends.

Fixes #5925
Depends on #8196
2025-05-27 15:05:35 -07:00
Jacob Hoffman-Andrews 388c68cb49
sa: use internal certificateStatusModel instead of core.CertificateStatus (#8159)
Part of https://github.com/letsencrypt/boulder/issues/8112
2025-05-12 14:53:08 -07:00
Jacob Hoffman-Andrews 967d722cf4
sa: use internal certificateModel (#8130)
This follows the system we've used for other types, where the SA has a
model type that is converted to a proto message for use outside the SA.

Part of #8112.
2025-04-21 13:48:29 -07:00
James Renken ff9e59d70b
core: Remove DnsNames from Order (#8108)
Remove the deprecated `DnsNames` field from the `corepb.Order` proto
message. All users of this struct use `Identifiers` instead.

This unblocks future changes that will require `Order` users to handle
different identifier types.

Part of #7311
2025-04-08 15:17:18 -07:00
James Renken 767abc73a4
core: Remove DnsName from Authorization (#8097)
Remove the deprecated `DnsName` field from the core `Authorization`
struct. All users of this struct use `Identifier` instead.

This unblocks future changes that will require `Authorization` users to
handle different identifier types.

Part of #7311
2025-04-07 15:25:59 -07:00
James Renken 3f879ed0b4
Add Identifiers to Authorization & Order structs (#7961)
Add `identifier` fields, which will soon replace the `dnsName` fields,
to:
- `corepb.Authorization`
- `corepb.Order`
- `rapb.NewOrderRequest`
- `sapb.CountFQDNSetsRequest`
- `sapb.CountInvalidAuthorizationsRequest`
- `sapb.FQDNSetExistsRequest`
- `sapb.GetAuthorizationsRequest`
- `sapb.GetOrderForNamesRequest`
- `sapb.GetValidAuthorizationsRequest`
- `sapb.NewOrderRequest`

Populate these `identifier` fields in every function that creates
instances of these structs.

Use these `identifier` fields instead of `dnsName` fields (at least
preferentially) in every function that uses these structs. When crossing
component boundaries, don't assume they'll be present, for
deployability's sake.

Deployability note: Mismatched `cert-checker` and `sa` versions will be
incompatible because of a type change in the arguments to
`sa.SelectAuthzsMatchingIssuance`.

Part of #7311
2025-03-26 10:30:24 -07:00
Aaron Gable 212a66ab49
Update go versions in CI and release (#7971)
Update from go1.23.1 to go1.23.6 for our primary CI and release builds.
This brings in a few security fixes that aren't directly relevant to us.

Add go1.24.0 to our matrix of CI and release versions, to prepare for
switching to this next major version in prod.
2025-02-19 14:37:01 -08:00
Aaron Gable 86ab2ed245
SA: Support profiles associated with authorizations (#7956)
Add "certificateProfileName" to the model used to insert new authz2 rows
and to the list of column names read when retrieving rows from the
authz2 table. Add support for this column to the functions which convert
to and from authz2 model types.

Add support for the profile field to core types so that it can be
returned by the SA.

Fixes https://github.com/letsencrypt/boulder/issues/7955
2025-01-27 14:53:30 -08:00
James Renken 3fcaebe934
core: Remove contactsPresent from Registration (#7952)
Remove the `contactsPresent` field from `corepb.Registration`, and all
places where it is set. #7933 removed all places where it was used.

Fixes #7920
2025-01-25 17:46:52 -08:00
James Renken 7da9a83deb
ra, pb: Don't expect or validate contactsPresent (#7933)
Part of #7920

There will be a followup removing the remaining places that set
`contactsPresent`.

---------

Co-authored-by: Jacob Hoffman-Andrews <jsha+github@letsencrypt.org>
2025-01-14 15:58:56 -08:00
Samantha Frank 27a77142ad
VA: Make performRemoteValidation more generic (#7847)
- Make performRemoteValidation a more generic function that returns a
new remoteResult interface
- Modify the return value of IsCAAValid and PerformValidation to satisfy
the remoteResult interface
- Include compile time checks and tests that pass an arbitrary operation
2024-11-27 15:29:33 -05:00
Samantha Frank 8bf13a90f4
VA: Make PerformValidation more like DoDCV (#7828)
- Remove Perspective and RIR from ValidationRecords
- Make ValidationResultToPB Perspective and RIR aware
- Update comment for VA.PerformValidation
- Make verificationRequestEvent more like doDCVAuditLog
- Update language used in problems created by performRemoteValidation to
be more like doRemoteDCV.
2024-11-20 14:13:55 -05:00
Samantha Frank a8cdaf8989
ratelimit: Remove legacy registrations per IP implementation (#7760)
Part of #7671
2024-11-19 18:39:21 -05:00
Aaron Gable dad9e08606
Lay the groundwork for supporting IP identifiers (#7692)
Clean up how we handle identifiers throughout the Boulder codebase by
- moving the Identifier protobuf message definition from sa.proto to
core.proto;
- adding support for IP identifier to the "identifier" package;
- renaming the "identifier" package's exported names to be clearer; and
- ensuring we use the identifier package's helper functions everywhere
we can.

This will make future work to actually respect identifier types (such as
in Authorization and Order protobuf messages) simpler and easier to
review.

Part of https://github.com/letsencrypt/boulder/issues/7311
2024-08-30 11:40:38 -07:00
Aaron Gable ced0117f6e
Remove deprecated sapb.Authorizations.Authz "map" (#7658)
This field was deprecated in
https://github.com/letsencrypt/boulder/pull/7646 and the last uses of it
were removed in https://github.com/letsencrypt/boulder/pull/7650.
2024-08-15 15:46:46 -07:00
Aaron Gable 46859a22d9
Use consistent naming for dnsName gRPC fields (#7654)
Find all gRPC fields which represent DNS Names -- sometimes called
"identifier", "hostname", "domain", "identifierValue", or other things
-- and unify their naming. This naming makes it very clear that these
values are strings which may be included in the SAN extension of a
certificate with type dnsName.

As we move towards issuing IP Address certificates, all of these fields
will need to be replaced by fields which carry both an identifier type
and value, not just a single name. This unified naming makes it very
clear which messages and methods need to be updated to support
non-dnsName identifiers.

Part of https://github.com/letsencrypt/boulder/issues/7647
2024-08-12 14:32:55 -07:00
Aaron Gable fa732df492
Remove challenge.ProvidedKeyAuthorization (#7655)
This field was deprecated in
https://github.com/letsencrypt/boulder/pull/7515, and has been fully
replaced by vapb.PerformValidationRequest.ExpectedKeyAuthorization.

Fixes https://github.com/letsencrypt/boulder/issues/7514
2024-08-12 14:08:06 -07:00
Aaron Gable 28f09341b9
Simplify GetValidOrderAuthorizations2 (#7646)
Simplify SA.GetValidOrderAuthorizations2 so that it no longer conditions
the query on the status, expiry, or registration ID of the authorization
rows. This gives the query much better performance, because it no longer
tries to use an overly-large index, and fall back to large row-scans
when the query planner decides the index is too large.

While we're here, also improve the return type of
GetValidOrderAuthorizations2, so that instead of returning a map of
names to authorizations, it simply returns a list of authzs. This both
reduces the size of the gRPC message (once the old map is fully
removed), and improves its correctness because we cannot count on names
to be unique across multiple identifier types.

Finally, improve the RA code which calls SA.GetValidOrderAuthorizations2
to handle this improved return type, to make fewer assumptions about
identifier types, and to separate static authorization-checking from CAA
rechecking.

Fixes https://github.com/letsencrypt/boulder/issues/7645
2024-08-08 10:40:40 -07:00
Phil Porada 5f616ccdb9
Upgrade go-jose from v2.6.1 to v.4.0.1 (#7345)
Upgrade from the old go-jose v2.6.1 to the newly minted go-jose v4.0.1. 
Cleans up old code now that `jose.ParseSigned` can take a list of
supported signature algorithms.

Fixes https://github.com/letsencrypt/boulder/issues/7390

---------

Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
2024-04-02 17:49:51 -04:00
Phil Porada 0e9f5d3545
va: Audit log which DNS resolver performs a lookup (#7271)
Adds the chosen DNS resolver to the VAs `ValidationRecord` object so
that for each challenge type during a validation, boulder can audit log
the resolver(s) chosen to fulfill the request..

Fixes https://github.com/letsencrypt/boulder/issues/7140
2024-02-05 14:26:39 -05:00
Phil Porada 6925fad324
Finish migration from int64 timestamps to timestamppb (#7142)
This is a cleanup PR finishing the migration from int64 timestamps to
protobuf `*timestamppb.Timestamps` by removing all usage of the old
int64 fields. In the previous PR
https://github.com/letsencrypt/boulder/pull/7121 all fields were
switched to read from the protobuf timestamppb fields.

Adds a new case to `core.IsAnyNilOrZero` to check various properties of
a `*timestamppb.Timestamp` reducing the visual complexity for receivers.

Fixes https://github.com/letsencrypt/boulder/issues/7060
2023-11-27 13:37:31 -08:00
Phil Porada fdaab3e21a
grpc: Read timestamps from timestamppb fields instead of int64 fields (#7121)
Switch to reading grpc timestamp values from the new timestamppb
protofbuf fields, completely ignoring the old int64 fields.

Part 3 of 4 for https://github.com/letsencrypt/boulder/issues/7060
2023-10-30 15:51:33 -04:00
Phil Porada a5c2772004
Add and populate new protobuf Timestamp fields (#7070)
* Adds new `google.protobuf.Timestamp` fields to each .proto file where
we had been using `int64` fields as a timestamp.
* Updates relevant gRPC messages to populate the new
`google.protobuf.Timestamp` fields in addition to the old `int64`
timestamp fields.
* Added tests for each `<x>ToPB` and `PBto<x>` functions to ensure that
new fields passed into a gRPC message arrive as intended.
* Removed an unused error return from `PBToCert` and `PBToCertStatus`
and cleaned up each call site.

Built on-top of https://github.com/letsencrypt/boulder/pull/7069
Part 2 of 4 related to
https://github.com/letsencrypt/boulder/issues/7060
2023-10-11 12:12:12 -04:00
Phil Porada 034316ef6a
Rename int64 timestamp related protobuf fields to <fieldname>NS (#7069)
Rename all of int64 timestamp fields to `<fieldname>NS` to indicate they
are Unix nanosecond timestamps.

Part 1 of 4 related to
https://github.com/letsencrypt/boulder/issues/7060
2023-09-15 13:49:07 -04:00
Aaron Gable 7e994a1216
Deprecate ROCSPStage6 feature flag (#6770)
Deprecate the ROCSPStage6 feature flag. Remove all references to the
`ocspResponse` column from the SA, both when reading from and when
writing to the `certificateStatus` table. This makes it safe to fully
remove that column from the database.

IN-8731 enabled this flag in all environments, so it is safe to
deprecate.

Part of #6285
2023-04-04 15:41:51 -07:00
Jacob Hoffman-Andrews d9872dbe41
sa: rename AddPrecertificateRequest.IssuerID (#6689)
sa: rename AddPrecertificateRequest.IssuerID
to IssuerNameID. This is in preparation for adding a similarly-named
field to AddSerialRequest.

Part of #5152.
2023-02-27 17:21:00 -05:00
Phil Porada 26e5b24585
dependencies: Replace square/go-jose.v2 with go-jose/go-jose.v2 (#6598)
Fixes #6573
2023-01-24 12:08:30 -05:00
Aaron Gable 18389c9024
Remove dead code (#5893)
Running an older version (v0.0.1-2020.1.4) of `staticcheck` in
whole-program mode (`staticcheck --unused.whole-program=true -- ./...`)
finds various instances of unused code which don't normally show up
as CI issues. I've used this to find and remove a large chunk of the
unused code, to pave the way for additional large deletions accompanying
the WFE1 removal.

Part of #5681
2022-01-19 12:23:06 -08:00
Aaron Gable 22a4e2afec
Add IssuerID field to CertificateStatus proto (#5595)
Add a new field, `issuerID`, to the `core.CertificateStatus` proto
message. Also add this field to the list of fields carried over when
converting back and forth between core and proto objects.

This should be safe to deploy without risk of miscommunication
between components that update at different times. If the SA
updates first and begins sending IssuerIDs to clients, they'll simply
ignore them. If the clients update first and try to extract IssuerIDs
from messages that don't have them, they'll simply get the zero
value, which is what they were getting before anyway.

Part of #5152 and #5532
2021-08-20 13:21:44 -07:00
Andrew Gabbitas 59bab8bac4
Make core.Registration.CreatedAt a *time.Time (#5422)
* Make core.Registration.CreatedAt a *time.time

Fixes: #5421
2021-05-21 13:44:56 -06:00
Andrew Gabbitas 5fdacbeaa6
grpc wrapper removal: Turn ra.NewRegistration into passthrough (#5397)
Turn ra.NewRegistration into passthrough

Fixes #5343
2021-04-23 13:53:22 -06:00
Andrew Gabbitas aa20bcaded
Add validated timestamp to challenges (#5253)
We do not present a validated timestamp in challenges where status = valid
as required by RFC8555.

This change is the first step to presenting challenge timestamps to the
client. It adds a timestamp to each place where we change a challenge to
valid. This only displays in the logs and will not display to the
subscriber because it is not yet stored somewhere retrievable. The next
step will be to store it in the database and then finally present it to
the client.

Part of #5198
2021-01-29 08:07:32 -08:00
Jacob Hoffman-Andrews bf7c80792d
core: move to proto3 (#5063)
Builds on #5062
Part of #5050
2020-08-31 17:58:32 -07:00
Jacob Hoffman-Andrews 0d8404bb22
core: Remove nil checks for core proto types. (#5062)
In preparation for moving to proto3 for the core/proto types
(Registration, Order, Authorization, Challenge, etc), this removes
checks that will fail when a proto2 client or server receives a message
from a proto3 client or server. Since proto3 encodes fields that have
their zero value as being absent (i.e. nil, in Go), we treat nil the
same as having a zero value.

In the process this change introduces additional checks, verifying
certain fields which should never have a zero value.

This involves factoring out registrationValid into registrationValid and
newRegistrationValid, since new registration requests lack some fields that
already-created registrations should always have.

Similarly, UpdateRegistration is changed to not verify that
`request.Update` is valid, since an update to registration object is not
a complete registration object - it may only update one field.
2020-08-31 11:47:12 -07:00
Jacob Hoffman-Andrews 8dd386b6bc
SA: Update RPC interface to proto3 (#5043)
One slightly surprising / interesting thing: Since core types like
Order and Registration are still proto2 and have pointer fields,
there are actually some places in this PR where I had to add
a `*` rather than delete an `&`, because I was taking a pointer
field from one of those core types and passing it as a field in
an SA RPC request.

Fixes #5037.
2020-08-25 10:28:41 -07:00
Aaron Gable 84799dbdf1
RA: Remove nil checks for zero-able fields (#5038)
Any field which can be zero must be allowed to be nil,
so that a proto2 server receiving requests from a proto3
client is willing to process messages with zero-value fields
encoded as missing.

Part of #4955
2020-08-18 16:54:26 -07:00
Jacob Hoffman-Andrews baf2a5be6e
sa: remove nil checks for zero-able fields. (#5034)
As part of the migration to proto3, any fields in requests that may be
zero should also be allowed to be nil. That's because proto3 will
represent those fields as absent when they have their zero value.

This is based on a manual review of the wrappers for the SA, plus
a pair of integration test runs. For the integration test runs I took these
steps:

1. Copy sa/proto to sa/proto2
2. Change sa/proto to use proto3 and regenerate.
3. In sa/*.go and cmd/boulder-sa/main.go, update the imports to use the
    proto2 version.
4. Split grpc/sa-wrappers.go into sa-server-wrappers.go and sa-wrappers.go
    (containing the client code)
5. In sa-server-wrappers.go, change the import to use sa/proto2.
6. In sa-server-wrappers.go, make a local copy of the core.StorageAuthority
    interface that uses the sa/proto2 types. This was necessary as
    a temporary kludge because of how the server wrapper internally
    uses the core.StorageAuthority interface.
7. Fix all the pointer-vs-value build errors in every other package.
8. Run integration tests.

I also performed those steps with proto2 and proto3 swapped, to confirm the
behavior when a proto2 client talks to a proto3 SA.
2020-08-18 16:20:07 -07:00
Aaron Gable e2c8f6743a
Introduce new core.AcmeChallenge type (#5012)
ACME Challenges are well-known strings ("http-01", "dns-01", and
"tlsalpn-01") identifying which kind of challenge should be used
to verify control of a domain. Because they are well-known and
only certain values are valid, it is better to represent them as
something more akin to an enum than as bare strings. This also
improves our ability to ensure that an AcmeChallenge is not
accidentally used as some other kind of string in a different
context. This change also brings them closer in line with the
existing core.AcmeResource and core.OCSPStatus string enums.

Fixes #5009
2020-08-11 15:02:16 -07:00
Aaron Gable 0f5d2064a8
Remove logic from VA PerformValidation wrapper (#5003)
Updates the type of the ValidationAuthority's PerformValidation
method to be identical to that of the corresponding auto-generated
grpc method, i.e. directly taking and returning proto message
types, rather than exploded arguments.

This allows all logic to be removed from the VA wrappers, which
will allow them to be fully removed after the migration to proto3.

Also updates all tests and VA clients to adopt the new interface.

Depends on #4983 (do not review first four commits)
Part of #4956
2020-08-06 10:45:35 -07:00
Aaron Gable 6f0016262f
Simplify database interactions (#4949)
Simplify database interactions

This change is a result of an audit of all places where
Go code directly constructs SQL queries and executes them
against a dbMap, with the goal of eliminating all instances
of constructing a well-known object type (such as a
core.CertificateStatus) from explicitly-listed database columns.
Instead, we should be relying on helper functions defined in the
sa itself to determine which columns are relevant for the
construction of any given object.

This audit did not find many places where this was occurring. It
did reveal a few simplifications, which are contained in this
change:
1) Greater use of existing SelectFoo methods provided by models.go
2) Streamlining of various SelectSingularFoo methods to always
   select by serial string, rather than user-provided WHERE clause
3) One spot (in ocsp-responder) where using a well-known type seemed
   better than using a more minimal custom type

Addresses #4899
2020-07-20 11:12:52 -07:00
Aaron Gable 7e626b63a6
Temporarily revert CA and VA proto3 migrations (#4962) 2020-07-16 14:29:42 -07:00
Aaron Gable 281575433b
Switch VA RPCs to proto3 (#4960)
This updates va.proto to use proto3 syntax, and updates
all clients of the autogenerated code to use the new types. In
particular, it removes indirection from built-in types (proto3
uses ints, rather than pointers to ints, for example).

Fixes #4956
2020-07-16 09:16:23 -07:00
Jacob Hoffman-Andrews bef02e782a
Fix nits found by staticcheck (#4726)
Part of #4700
2020-03-30 10:20:20 -07:00
Roland Bracewell Shoemaker e402156c1c Revert "Revert "Remove remaining old format authorization code from SA/… (#4502)" (#4524)
This reverts commit dc2ce4ca84.
2019-11-04 09:45:19 -05:00
Roland Bracewell Shoemaker dc2ce4ca84
Revert "Remove remaining old format authorization code from SA/… (#4502)
We need to apply some fixes for bugs introduced in #4476 before it can be deployed, as such we need to revert #4495 as there needs to be a full deploy cycle between these two changes.

This reverts commit 3ae1ae1.

😭
2019-10-23 10:45:29 -07:00
Roland Bracewell Shoemaker 3ae1ae1493 Remove remaining old format authorization code from SA/protos (#4495) 2019-10-23 09:08:38 -04:00
Roland Bracewell Shoemaker 46e0468220 Make authz2 the default storage format (#4476)
This change set makes the authz2 storage format the default format. It removes
most of the functionality related to the previous storage format, except for
the SA fallbacks and old gRPC methods which have been left for a follow-up
change in order to make these changes deployable without introducing
incompatibilities.

Fixes #4454.
2019-10-21 15:29:15 -04:00
Daniel McCarney 1cd9733c24
WFE2: allow revocation of precertificates. (#4433)
When the `features.PrecertificateRevocation` feature flag is enabled the WFE2
will allow revoking certificates for a submitted precertificate. The legacy WFE1
behaviour remains unchanged (as before (pre)certificates issued through the V1
API will be revocable with the V2 API).

Previously the WFE2 vetted the certificate from the revocation request by
looking up a final certificate by the serial number in the requested
certificate, and then doing a byte for byte comparison between the stored and
requested certificate.

Rather than adjust this logic to handle looking up and comparing stored
precertificates against requested precertificates (requiring new RPCs and an
additional round-trip) we choose to instead check the signature on the requested
certificate or precertificate and consider it valid for revocation if the
signature validates with one of the WFE2's known issuers. We trust the integrity
of our own signatures.

An integration test that performs a revocation of a precertificate (in this case
one that never had a final certificate issued due to SCT embedded errors) with
all of the available authentication mechanisms is included.

Resolves https://github.com/letsencrypt/boulder/issues/4414
2019-09-16 16:40:07 -04:00
Roland Bracewell Shoemaker e839042bae dns: Remove Authorities field from ValidationRecord (#4230) 2019-05-28 14:11:32 -04:00