This allows us to call the Google Safe Browsing calls through the VA.
If the RA config's boolean UseIsSafeDomain is true, the RA will make the RPC
call to the VA during its NewAuthorization.
If the VA config's GoogleSafeBrowsingConfig struct is not nil, the VA
will check the Google Safe Browsing API in
VA.IsSafeDomain. If the GoogleSafeBrowsingConfig struct is nil, it will
always return true.
In order to actually make requests, the VA's GoogleSafeBrowsingConfig
will need to have a directory on disk it can store the local GSB hashes
it will check first and a working Google API key for the GSB API.
Fixes#1058
Adds a (currently gated) Akamai CCU API client used to purge GET OCSP responses
from the CDN. It also contains a small tool (cmd/akamai-purger) that can be used
to purge ARLs from the command line.
If a ServiceUnavailableError is returned from GenerateOCSP backoff before
attempting to retry the call as to not to overwhelm the CA with calls that
may instantly fail.
* Moves revocation from the CA to the OCSP-Updater, the RA will mark certificates as
revoked then wait for the OCSP-Updater to create a new (final) revoked response
* Merges the ocspResponses table with the certificateStatus table and only use UPDATES
to update the OCSP response (vs INSERT-only since this happens quite often and will
lead to an extremely large table)
also moves the first OCSP responses generation from the CA to the OCSP updater. This patch lays the
ground work for moving CT submission and adding CT backfill to the OCSP updater.
* Rewrite JSONDuration as ConfigDuration that can handle both JSON and YAML unmarshaling
* Factor out RPC certificate count request struct
* Return 429 to WFE on rate limit exceeded
* Fix wonky RateLimitPolicy comment
Adds a new service, Publisher, which exists to submit issued certificates to various Certificate Transparency logs. Once submitted the Publisher will also parse and store the returned SCT (Signed Certificate Timestamp) receipts that are used to prove inclusion in a specific log in the SA database. A SA migration adds the new SCT receipt table.
The Publisher only exposes one method, SubmitToCT, which is called in a goroutine by ca.IssueCertificate as to not block any other issuance operations. This method will iterate through all of the configured logs attempting to submit the certificate, and any required intermediate certificates, to them. If a submission to a log fails it will be retried the pre-configured number of times and will either use a back-off set in a Retry-After header or a pre-configured back-off between submission attempts.
This changeset is the first of a number of changes ending with serving SCT receipts in OCSP responses and purposefully leaves out the following pieces for follow-up PRs.
* A fake CT server for integration testing
* A external tool to search the database for certificates lacking a full set of SCT receipts
* A method to construct X.509 v3 extensions containing receipts for the OCSP responder
* Returned SCT signature verification (beyond just checking that the signature is of the correct type so we aren't just serving arbitrary binary blobs to clients)
Resolves#95.
This removes TestMode from the boulder-va command, from ca.Config
(it was only used in the VA) and gets the integration config to specify
the ports it should use explicitly.
(It also removes a DBDriver field from ca.Config that was left over from
letsencrypt/boulder#624.)
Fixes#627.
This has required some substantive changes to the tests. Where
previously the foreign key constraints did not exist in the tests, now
that we use the actual production schema, they do. This has mostly led
to having to create real Registrations in the sa, ca, and ra tests. Long
term, it would be nice to fake this out better instead of needing a real
sa in the ca and ra tests.
The "goose" being referred to is <https://bitbucket.org/liamstask/goose>.
Database migrations are stored in a _db directory inside the relevant
owner service (namely, ca/_db, and sa/_db, today).
An example of migrating up with goose:
goose -path ./sa/_db -env test up
An example of creating a new migration with goose:
goose -path ./sa/_db -env test create NameOfNewMigration sql
Notice the "sql" at the end. It would be easier for us to manage sql
migrations. I would like us to stick to only them. In case we do use Go
migrations in the future, the underscore at the beginning of "_db" will
at least prevent build errors when using "..." with goose-created Go
files. Goose-created Go migrations do not compile with the go tool but
only with goose.
Fixes#111
Unblocks #623
Currently, the debug http server in every service contains just the
net/http/pprof handlers. This allows us to get CPU, blocking, and memory
profiling remotely.
Along the way, remove all the places we use http.DefaultServeMux (which
includes use of http.Handle and http.HandlerFunc) and use a NewServeMux
for each place.
Fixes#457
- Moved HandlerTimer definition from various cmd/ binaries to cmd/shell.go
- Cleaned up HandlerTimer endpoint metrics
- Moved New... counter metrics from WFE to RA and add Updated... and Finalized... ones
- Added error code and problem type counter metrics to WFE
- Added validation type / status counter metrics to VA
- Consistently return the total RTT from LookupCAA, LookupCNAME, and LookupDNSSEC method
- Added DNS RTT timing metrics to VA for the various Loookup... methods
Also, remove dependency on cfssl CLI binary, and transitive dependency cf-tls.
These are no longer necessary now that we use the local signer. And the cf-tls
dependency had drifted out of date, causing build issues when I updated cfssl to
master.
- Move dbMap construction and type converter into individual files in the sa package.
- Add DB configuration for the OCSP tool to the boulder config:
- left to the user if they want to use different boulder-config.json files
for different purposes.
- Added updater to Makefile
- Fix trailing ',' in the Boulder config, add more panic logging
- Ignore .pem files produced by the integration test
- Change RPC to use per-instance named reply-to queues.
- Finish OCSP Updater logic
- Rework RPC for OCSP to use a transfer object (due to serialization problems of x509.Certificate)
- Add SQL configuration options
- Increase the width of the authz and pending_authz tables' challenges field
- Make it configurable whether CREATE TABLE commands should run
- Added SQL debug logging (SA option: "SQLDebug")
- Added timestamps to the log prints to stdout
- Ignore *.pem in test/js
- Modified start.sh to support environment overrides for BOULDER_CONFIG, like the AMQP mode
- Changed boulder-test-config to open the server on the loopback device, so as to not cause firewall prompts on each integration test run for those of us being restrictive
- Renamed "key" column to "jwk" in DB, to avoid keyword conflict
- Set MaxLength on "jwk" column to 512
Richard Barnes
Jeff Hodges
Roland Shoemaker
Jacob Hoffman-Andrews
Tom Clegg
bifurcation
Jeremy Gillula
J.C. Jones