This updates to current `master`,
bc7acd89f703743d050f5cd4a3b9746808e0fdae
Notably, it includes a bug-fix to error handling in the HTTP client,
which we found was hiding errors from CT logs, hindering our debugging.
That fix is
https://github.com/google/certificate-transparency-go/pull/1695
No release has been tagged since this PR merged, so using the `master`
commit.
A few mutual dependencies used by both Boulder and ct-go are updated,
including mysql, otel, and grpc.
Update github.com/google/certificate-transparency-go from v1.1.6 to
v1.3.1. This updates the loglist file schema to recognize logs which are
tagged as being tiled logs / implementing the static CT API.
Transitively update:
- github.com/go-sql-driver/mysql from v1.7.1 to v1.8.1
- github.com/prometheus/client_golang from v1.15.1 to v1.22.0
- github.com/prometheus/client_model from v0.4.0 to v0.6.1
- go.opentelemetry.io/otel from v1.30.0 to v1.31.0
- google.golang.org/grpc from v1.66.1 to v1.69.4
- google.golang.org/protobuf from v1.34.2 to v1.36.5
- and a variety of indirect dependencies
Remove one indirect dependency:
- github.com/matttproud/golang_protobuf_extensions
Add two new indirect dependencies:
- filippo.io/edwards25519@v1.1.0 (used by go-sql-driver to handle
mariadb's custom encryption implementation)
- github.com/munnerz/goautoneg@v0.0.0-20191010083416-a7dc8b61c822
(previously inlined into prometheus/common)
Also fix two unit tests which need minor modifications to work with
updated type signatures and behavior.
Part of https://github.com/letsencrypt/boulder/issues/7872
All 4 usages of the `maps.Keys` function from the
`golang.org/x/exp/maps` package can be refactored to a simpler
alternative. If we need it in the future, it is available in the
standard library since Go 1.23.
Updates:
- go.opentelemetry.io/contrib/instrumentation (and subpackages) from 0.52.0 to 0.55.0
- go.opentelemetry.io/otel (and subpackages) from 1.27.0 to 1.30.0
Upstream release notes:
https://github.com/open-telemetry/opentelemetry-go-contrib/releases
Also transitively updates a few golang.org/x/ dependencies, and the grpc and protobuf
dependencies.
Updates:
github.com/weppos/publicsuffix-go from
v0.30.3-0.20240510084413-5f1d03393b3d to
v0.40.3-0.20240815124645-a8ed110559c9
golang.org/x/crypto from v0.24.0 to v0.26.0
golang.org/x/net from v0.26.0 to v0.28.0
golang.org/x/sync from v0.7.0 to v0.8.0
golang.org/x/term from v0.21.0 to v0.23.0
golang.org/x/text from v0.16.0 to v0.17.0
golang.org/x/sys from v0.21.0 to v0.23.0
It's been one month since the last PSL update.
Update:
- github.com/weppos/publicsuffix-go from
v0.30.3-0.20240411085455-21202160c2ed to
v0.30.3-0.20240510084413-5f1d03393b3d
- golang.org/x/crypto from v0.22.0 to v0.23.0 (no changes)
- golang.org/x/net from v0.24.0 to v0.25.0
- golang.org/x/term from v0.19.0 to v0.20.0 (no changes)
- golang.org/x/text from v0.14.0 to v0.15.0 (no changes)
- golang.org/x/sys from v0.19.0 to v0.20.0
Fixes https://github.com/letsencrypt/boulder/issues/7483
We last updated the PSL two months ago, and the latest officially-tagged
release of publicsuffic-go is just one month old, but the domain
prompting the latest request to update the PSL landed just a few days
ago, so update to the latest commit rather than the latest release.
Also incidentally updates x/crypto, x/net, x/term, and x/sys, but brings
in no meaningful updates for any of them.
Fixes https://github.com/letsencrypt/boulder/issues/7428
Upgrade from the old go-jose v2.6.1 to the newly minted go-jose v4.0.1.
Cleans up old code now that `jose.ParseSigned` can take a list of
supported signature algorithms.
Fixes https://github.com/letsencrypt/boulder/issues/7390
---------
Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
Create a new administration tool "bin/admin" as a successor to and
replacement of "admin-revoker".
This new tool supports all the same fundamental capabilities as the old
admin-revoker, including:
- Revoking by serial, by batch of serials, by incident table, and by
private key
- Blocking a key to let bad-key-revoker take care of revocation
- Clearing email addresses from all accounts that use them
Improvements over the old admin-revoker include:
- All commands run in "dry-run" mode by default, to prevent accidental
executions
- All revocation mechanisms allow setting the revocation reason,
skipping blocking the key, indicating that the certificate is malformed,
and controlling the number of parallel workers conducting revocation
- All revocation mechanisms do not parse the cert in question, leaving
that to the RA
- Autogenerated usage information for all subcommands
- A much more modular structure to simplify adding more capabilities in
the future
- Significantly simplified tests with smaller mocks
The new tool has analogues of all of admin-revokers unit tests, and all
integration tests have been updated to use the new tool instead. A
future PR will remove admin-revoker, once we're sure SRE has had time to
update all of their playbooks.
Fixes https://github.com/letsencrypt/boulder/issues/7135
Fixes https://github.com/letsencrypt/boulder/issues/7269
Fixes https://github.com/letsencrypt/boulder/issues/7268
Fixes https://github.com/letsencrypt/boulder/issues/6927
Part of https://github.com/letsencrypt/boulder/issues/6840
Upgrade to zlint v3.6.0
Two new lints are triggered in various places:
aia_contains_internal_names is ignored in integration test
configurations, and unit tests are updated to have more realistic URLs.
The w_subject_common_name_included lint needs to be ignored where we'd
ignored n_subject_common_name_included before.
Related to https://github.com/letsencrypt/boulder/issues/7261
Add a new shared config stanza which all boulder components can use to
configure their Open Telemetry tracing. This allows components to
specify where their traces should be sent, what their sampling ratio
should be, and whether or not they should respect their parent's
sampling decisions (so that web front-ends can ignore sampling info
coming from outside our infrastructure). It's likely we'll need to
evolve this configuration over time, but this is a good starting point.
Add basic Open Telemetry setup to our existing cmd.StatsAndLogging
helper, so that it gets initialized at the same time as our other
observability helpers. This sets certain default fields on all
traces/spans generated by the service. Currently these include the
service name, the service version, and information about the telemetry
SDK itself. In the future we'll likely augment this with information
about the host and process.
Finally, add instrumentation for the HTTP servers and grpc
clients/servers. This gives us a starting point of being able to monitor
Boulder, but is fairly minimal as this PR is already somewhat unwieldy:
It's really only enough to understand that everything is wired up
properly in the configuration. In subsequent work we'll enhance those
spans with more data, and add more spans for things not automatically
traced here.
Fixes https://github.com/letsencrypt/boulder/issues/6361
---------
Co-authored-by: Aaron Gable <aaron@aarongable.com>
Update all golang.org/x/ deps to their latest available version:
- https://golang.org/x/crypto from 0.7.0 to 0.8.0
- https://golang.org/x/exp from v0.0.0-20230118134722-a68e582fa157 to
v0.0.0-20230321023759-10a507213a29
- https://golang.org/x/net from 0.8.0 to 0.9.0
- https://golang.org/x/text from 0.8.0 to 0.9.0
- https://golang.org/x/mod from 0.8.0 to 0.10.0
- https://golang.org/x/tools from 0.6.0 to 0.8.0
This only affects vendored files for /x/exp, /x/net/, and /x/tools/.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gable <aaron@letsencrypt.org>
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.4.0 to
0.6.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ec5565b1b7"><code>ec5565b</code></a>
README.md: update documentation of module versioning</li>
<li><a
href="c8236a6712"><code>c8236a6</code></a>
unicode/bidi: remove unused global</li>
<li><a
href="ada7473102"><code>ada7473</code></a>
all: remove redundant type conversion</li>
<li>See full diff in <a
href="https://github.com/golang/text/compare/v0.4.0...v0.6.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Simplify the control flow of the FinalizeOrder handler to make it easier
to read and reason about:
- Move all validation to before we set the order to Processing, and put
it all in a single helper funcion.
- Move almost all logEvent/trace handling directly into FinalizeOrder so
it cannot be missed.
- Flatten issueCertificate and issueCertificateInner into a single
helper function, now that they're no longer being called from both
ACMEv1 and v2 entry points.
- Other minor cleanups, such as making SolvedBy not return a pointer and
making matchesCSR private.
This paves the way for making both issueCertificateInner and failOrder
asynchronous, which we plan to do in the near future.
Part of #6575
In live.go we use a semaphore to limit how many inflight signing
requests we can have, so a flood of OCSP traffic doesn't flood our CA
instances. If traffic exceeds our capacity to sign responses for long
enough, we want to eventually start fast-rejecting inbound requests that
are unlikely to get serviced before their deadline is reached. To do
that, add a MaxSigningWaiters config field to the OCSP responder.
Note that the files in //semaphore are forked from x/sync/semaphore,
with modifications to add the MaxWaiters field and functionality.
Fixes#6392
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.3.7 to 0.3.8.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.3.7...v0.3.8)
This also transitively updates x/tools and x/sync, which is good because those
are unversioned packages which are otherwise ignored by dependabot.
Note that we are not affected by the vulnerability which prompted the release
of version 0.3.8; the affected files are in the language subpackage which we
do not use or vendor.
Right now, Boulder expects to be able to connect to syslog, and panics
if it's not available. We'd like to be able to log to stdout/stderr as a
replacement for syslog.
- Add a detailed timestamp (down to microseconds, same as we collect in
prod via syslog).
- Remove the escape codes for colorizing output.
- Report the severity level numerically rather than with a letter prefix.
Add locking for stdout/stderr and syslog logs. Neither the [syslog] package
nor the [os] package document concurrency-safety, and the Go rule is: if
it's not documented to be concurrent-safe, it's not. Notably the [log.Logger]
package is documented to be concurrent-safe, and a look at its implementation
shows it uses a Mutex internally.
Remove places that use the singleton `blog.Get()`, and instead pass through
a logger from main in all the places that need it.
[syslog]: https://pkg.go.dev/log/syslog
[os]: https://pkg.go.dev/os
[log.Logger]: https://pkg.go.dev/log#Logger
Fork the pieces of the Go standard library's crypto/x509
package which are relevant to parsing, handling, and
signing CRLs.
In our fork, fix an upstream parsing bug, hoist the reasonCode
out of the crlEntryExtensions for easier usability, and enforce
that CRL Numbers are never longer than 20 octets.
Part of #6199
Diff: https://github.com/prometheus/client_golang/compare/v1.7.1...v1.12.1
Changes:
* 1.12.1 / 2022-01-29
- [BUGFIX] Make the Go 1.17 collector concurrency-safe
- Use simpler locking in the Go 1.17 collector
- [BUGFIX] Reduce granularity of histogram buckets for Go 1.17 collector
- [ENHANCEMENT] API client: make HTTP reads more efficient
* 1.12.0 / 2022-01-19
- [CHANGE] example/random: Move flags and metrics into main()
- [FEATURE] API client: Support wal replay status api
- [FEATURE] Use the runtime/metrics package for the Go collector for 1.17+
- [ENHANCEMENT] API client: Update /api/v1/status/tsdb to include headStats
- [ENHANCEMENT] promhttp: Check validity of method and code label values
* 1.11.0 / 2021-06-07
- [CHANGE] Add new collectors package.
- [CHANGE] prometheus.NewExpvarCollector is deprecated, use collectors.NewExpvarCollector instead.
- [CHANGE] prometheus.NewGoCollector is deprecated, use collectors.NewGoCollector instead.
- [CHANGE] prometheus.NewBuildInfoCollector is deprecated, use collectors.NewBuildInfoCollector instead.
- [FEATURE] Add new collector for database/sql#DBStats.
- [FEATURE] API client: Add exemplars API support.
- [ENHANCEMENT] API client: Add newer fields to Rules API.
- [ENHANCEMENT] API client: Add missing fields to Targets API.
* 1.10.0 / 2021-03-18
- [CHANGE] Minimum required Go version is now 1.13.
- [CHANGE] API client: Add matchers to LabelNames and LabesValues.
- [FEATURE] API client: Add buildinfo call.
- [BUGFIX] Fix build on riscv64.
* 1.9.0 / 2020-12-17
- [FEATURE] NewPidFileFn helper to create process collectors for processes whose PID is read from a file.
- [BUGFIX] promhttp: Prevent endless loop in InstrumentHandler... middlewares with invalid metric or label names.
* 1.8.0 / 2020-10-15
- [CHANGE] API client: Use time.Time rather than string for timestamps in RuntimeinfoResult.
- [FEATURE] Export MetricVec to facilitate implementation of vectors of custom Metric types.
- [FEATURE] API client: Support /status/tsdb endpoint.
- [ENHANCEMENT] API client: Enable GET fallback on status code 501.
- [ENHANCEMENT] Remove Metric references after reslicing to free up more memory.
Additional transitive dependency updates:
* https://github.com/prometheus/common/compare/v0.10.0...v0.32.1
* https://github.com/prometheus/procfs/compare/v0.1.3...v0.7.3
* https://github.com/golang/appengine/compare/v1.6.5...v1.6.6
* cb27e3aa20...8632dd7979
* 0f9fa26af8...5a964db013
Matthew McPherrin
Aaron Gable
dependabot[bot]
Samantha Frank
Eng Zer Jun
Adin Schmahmann
Phil Porada
Matthew McPherrin
Jacob Hoffman-Andrews