Commit Graph

23 Commits

Author SHA1 Message Date
Aaron Gable 23608e19c5
Simplify docker-compose network setup (#8214)
Remove static IPs from services that can be reached by their service
name. Remove consulnet and redisnet, and have the services which
connected to those network connect directly to bouldernet instead.
Instruct docker-compose to only dynamically allocate IPs from the upper
half of the bouldernet subset, to avoid clashing with the few static IPs
we still specify.
2025-05-30 13:23:27 -07:00
Jacob Hoffman-Andrews 388c68cb49
sa: use internal certificateStatusModel instead of core.CertificateStatus (#8159)
Part of https://github.com/letsencrypt/boulder/issues/8112
2025-05-12 14:53:08 -07:00
Aaron Gable 146b78a0f7
Remove all static minica keys (#7489)
Remove the redis-tls, wfe-tls, and mail-test-srv keys which were
generated by minica and then checked in to the repo. All three are
replaced by the dynamically-generated ipki directory.

Part of https://github.com/letsencrypt/boulder/issues/7476
2024-05-17 11:45:40 -07:00
Aaron Gable e05d47a10a
Replace explicit int loops with range-over-int (#7434)
This adopts modern Go syntax to reduce the chance of off-by-one errors
and remove unnecessary loop variable declarations.

Fixes https://github.com/letsencrypt/boulder/issues/7227
2024-04-22 10:34:51 -07:00
Samantha 4ed54ff9c6
dependencies: Update go-redis from v8 to v9 (#7041)
Updated so we can access the `SetAddrs()` method added to `*redis.Ring`
in `v9` in #7042.

Part of #5545
2023-08-22 13:06:25 -04:00
Jacob Hoffman-Andrews 38fc840184
sa: refactor how metrics and logging are set up (#7031)
This eliminates the need for a pair of accessors on `db.WrappedMap` that
expose the underlying `*sql.DB` and `*borp.DbMap`.

Fixes #6991
2023-08-08 09:51:23 -07:00
Jacob Hoffman-Andrews 7d66d67054
It's borpin' time! (#6982)
This change replaces [gorp] with [borp].

The changes consist of a mass renaming of the import and comments / doc
fixups, plus modifications of many call sites to provide a
context.Context everywhere, since gorp newly requires this (this was one
of the motivating factors for the borp fork).

This also refactors `github.com/letsencrypt/boulder/db.WrappedMap` and
`github.com/letsencrypt/boulder/db.Transaction` to not embed their
underlying gorp/borp objects, but to have them as plain fields. This
ensures that we can only call methods on them that are specifically
implemented in `github.com/letsencrypt/boulder/db`, so we don't miss
wrapping any. This required introducing a `NewWrappedMap` method along
with accessors `SQLDb()` and `BorpDB()` to get at the internal fields
during metrics and logging setup.

Fixes #6944
2023-07-17 14:38:29 -07:00
Jacob Hoffman-Andrews 824417f6c0
sa: refactor db initialization (#6930)
Previously, we had three chained calls initializing a database:

 - InitWrappedDb calls NewDbMap
 - NewDbMap calls NewDbMapFromConfig

Since all three are exporetd, this left me wondering when to call one vs
the others.

It turns out that NewDbMap is only called from tests, so I renamed it to
DBMapForTest to make that clear.

NewDbMapFromConfig is only called internally to the SA, so I made it
unexported it as newDbMapFromMysqlConfig.

Also, I copied the ParseDSN call into InitWrappedDb, so it doesn't need
to call DBMapForTest. Now InitWrappedDb and DBMapForTest both
independently call newDbMapFromMysqlConfig.

I also noticed that InitDBMetrics was only called internally so I
unexported it.
2023-06-13 10:15:40 -07:00
Samantha 487680629d
cmd: TLSConfig values should be string not *string (#6872)
Fixes #6737
2023-05-08 13:21:42 -04:00
Phil Porada 17fb1b287f
cmd: Export prometheus metrics for TLS cert notBefore and notAfter fields (#6836)
Export new prometheus metrics for the `notBefore` and `notAfter` fields
to track internal certificate validity periods when calling the `Load()`
method for a `*tls.Config`. Each metric is labeled with the `serial`
field.

```
tlsconfig_notafter_seconds{serial="2152072875247971686"} 1.664821961e+09
tlsconfig_notbefore_seconds{serial="2152072875247971686"} 1.664821960e+09
```

Fixes https://github.com/letsencrypt/boulder/issues/6829
2023-04-24 16:28:05 -04:00
Aaron Gable 7e994a1216
Deprecate ROCSPStage6 feature flag (#6770)
Deprecate the ROCSPStage6 feature flag. Remove all references to the
`ocspResponse` column from the SA, both when reading from and when
writing to the `certificateStatus` table. This makes it safe to fully
remove that column from the database.

IN-8731 enabled this flag in all environments, so it is safe to
deprecate.

Part of #6285
2023-04-04 15:41:51 -07:00
Samantha 5c49231ea6
ROCSP: Remove support for Redis Cluster (#6645)
Fixes #6517
2023-02-09 17:14:37 -05:00
Samantha 6c6da76400
ROCSP: Replace Redis Cluster with a consistently sharded all-primary nodes (#6516) 2022-12-19 15:06:47 -05:00
Samantha 78ea1d2c9d
SA: Use separate schema for incidents tables (#6350)
- Move incidents tables from `boulder_sa` to `incidents_sa` (added in #6344)
- Grant read perms for all tables in `incidents_sa`
- Modify unit tests to account for new schema and grants
- Add database cleaning func for `boulder_sa`
- Adjust cleanup funcs to omit `sql-migrate` tables instead of `goose`

Resolves #6328
2022-09-09 15:17:14 -07:00
Jacob Hoffman-Andrews 18b5194f2e
rocsp: remove Metadata (#6206)
The metadata values were planned to be used for scanning Redis in
ocsp-updater. Since we won't do that, remove it. Happily, this also
allows us to get rid of shortIssuerId.

Removing the issuer check in rocsp_sa.go uncovered a "boxed nil" problem:
SA was doing a nil check against an interface field that in practice was
never nil (because it was promoted from a concrete type at construction
time). So we would always hit the ROCSP path. But one of the first steps
in that path was looking up an issuer ID. Since `test/config` never
had the issuers set, we would look up the issuer ID, not find it, and
return an error before we attempted to call storeResponse. To fix this,
I made `NewSQLStorageAuthority` take a concrete `*rocsp.WritingClient`
instead of an interface, and check for nil before assigning it to an
internal interface field.

Built on top of #6201.
2022-07-05 16:20:56 -07:00
Aaron Gable b86e9d10da
Unify how we do streaming database selects (#6176)
Create a new type `db.MappedSelector` which exposes a new
`Query` method. This method behaves similar to gorp's
`SelectFoo` methods, in that it uses the desired result type to
look up the correct table to query and uses reflection to map
the table columns to the struct fields. It behaves similarly to
the stdlib's `sql.Query` in that it returns a `Rows` object which
can be iterated over to get one row of results at a time. And it
improves both of those by using generics, rather than `interface{}`,
to provide a nicely-typed calling interface.

Use this new type to simplify the existing streaming query in
`SerialsForIncident`. Similarly use the new type to simplify
rocsp-tool's and ocsp-updater's streams of `CertStatusMetadata`.
This new type will also be used by the crl-updater's upcoming
`GetRevokedCerts` streaming query.

Fixes #6173
2022-06-24 14:31:46 -07:00
Andrew Gabbitas 87ef1b4934
Use OCSP NextUpdate to calculate Redis TTL (#6031) 2022-04-04 15:18:11 -06:00
Samantha 3e9eaf84ea
rocsp-tool: Add syslog support (#6010)
Add a logging stanza to rocsp-tool's config, and initialize a boulder
logger rather than using Go's default log facilities.

Fixes #5976
2022-03-21 14:51:56 -07:00
Jacob Hoffman-Andrews 0fc894f6bb
Fix iteration in rocsp-tool load-from-db (#5886)
rocsp-tool load-from-db scans in batches. On each iteration, it is
supposed to update its starting position based on the highest seen ID from
the last batch. However, it was always setting its starting position to
the same value, and not making progress if the DB was larger than the
batch size.
2022-01-18 14:26:50 -08:00
Andrew Gabbitas def152b722
Export Prometheus stats for Redis ClusterClient (#5828)
Redis ClusterClient has metrics available. Create a collector so
Prometheus can access the metrics on-demand.

Part of #5768
2021-12-06 18:27:18 -08:00
Jacob Hoffman-Andrews 3d7206a183
ocsp-updater: add support for writing to Redis (#5825)
If configured, ocsp-updater will write responses to Redis in parallel
with MariaDB, giving up if Redis is slower and incrementing a stat.

Factors out the ShortIDIssuer concept from rocsp-tool into
rocsp_config.
2021-12-06 14:46:46 -08:00
Jacob Hoffman-Andrews d3d5b12e59
Add --start-from-id flag to rocsp-tool (#5802)
Fixes #5783
2021-12-01 16:50:04 -08:00
Jacob Hoffman-Andrews f238409089
Split rocsp-tool into files; add some tests (#5795)
This splits rocsp-tool/main.go into main.go, client.go, issuers.go,
and inflight.go.

Adds tests for issuers and inflight, plus storeResponse in
client.go. Doesn't yet have a test for loadFromDB in client.go.

Part of #5786
2021-11-12 17:41:14 -08:00