To keep the change small, I have not yet completely removed the
GetCertificateByShortSerial method from interfaces and the RPC. I will do taht
in a follow up change.
Adds a new service, Publisher, which exists to submit issued certificates to various Certificate Transparency logs. Once submitted the Publisher will also parse and store the returned SCT (Signed Certificate Timestamp) receipts that are used to prove inclusion in a specific log in the SA database. A SA migration adds the new SCT receipt table.
The Publisher only exposes one method, SubmitToCT, which is called in a goroutine by ca.IssueCertificate as to not block any other issuance operations. This method will iterate through all of the configured logs attempting to submit the certificate, and any required intermediate certificates, to them. If a submission to a log fails it will be retried the pre-configured number of times and will either use a back-off set in a Retry-After header or a pre-configured back-off between submission attempts.
This changeset is the first of a number of changes ending with serving SCT receipts in OCSP responses and purposefully leaves out the following pieces for follow-up PRs.
* A fake CT server for integration testing
* A external tool to search the database for certificates lacking a full set of SCT receipts
* A method to construct X.509 v3 extensions containing receipts for the OCSP responder
* Returned SCT signature verification (beyond just checking that the signature is of the correct type so we aren't just serving arbitrary binary blobs to clients)
Resolves#95.
The WFE test relies on a pre-generated cert. Since there are some sanity checks
on the dates in certs, we were getting errors during the test.
One quick fix is to have those sanity checks rely on RA's clock object, which
can be replaced with a fake for testing. In order to do that, I had to move the
sanity check (MatchesCSR) into the registration authority package, where it
makes more sense anyhow.
I also removed a handful of equality testing functions in objects.go that were
only used by MatchesCSR and whose purpose is better served by reflect.DeepEqual.
This was to avoid having to also move those equality testing functions into the
registration authority.
Challenge URIs should be determined by the WFE at fetch time, rather than stored
alongside the challenge in the DB. This simplifies a lot of the logic, and
allows to to remove a code path in NewAuthorization where we create an
authorization, then immediately save it with modifications to the challenges.
This change also gives challenges their own endpoint, which contains the
challenge id rather than the challenge's offset within its parent authorization.
This is also a first step towards replacing UpdateAuthorization with
UpdateChallenge: https://github.com/letsencrypt/boulder/issues/760.
Add an easy script to build and run the Docker instance.
Update some out-of-date information in the README.
Add goose to the Docker image.
Remove unnecessary go install step from Dockerfile.
Allow dns-test-srv to return a hardcoded address other than localhost. This was
preventing a Dockerized Boulder from answering requests from a letsencrypt
client on the host.
Change allowLoopbackAddresses to allowRestrictedAddresses and make it cover all
the private IPv4 ranges. The host IP in Docker is commonly in the 172.* range.
Fix a couple of references to lets-encrypt-preview.
This was inspired by investigation into https://github.com/letsencrypt/boulder/issues/756.
To try and reproduce, I tried running Boulder inside a container, and found some
broken things.
This has required some substantive changes to the tests. Where
previously the foreign key constraints did not exist in the tests, now
that we use the actual production schema, they do. This has mostly led
to having to create real Registrations in the sa, ca, and ra tests. Long
term, it would be nice to fake this out better instead of needing a real
sa in the ca and ra tests.
The "goose" being referred to is <https://bitbucket.org/liamstask/goose>.
Database migrations are stored in a _db directory inside the relevant
owner service (namely, ca/_db, and sa/_db, today).
An example of migrating up with goose:
goose -path ./sa/_db -env test up
An example of creating a new migration with goose:
goose -path ./sa/_db -env test create NameOfNewMigration sql
Notice the "sql" at the end. It would be easier for us to manage sql
migrations. I would like us to stick to only them. In case we do use Go
migrations in the future, the underscore at the beginning of "_db" will
at least prevent build errors when using "..." with goose-created Go
files. Goose-created Go migrations do not compile with the go tool but
only with goose.
Fixes#111
Unblocks #623
Refactor DNS problem details use
Actually store and log resolved addresses
Less convuluted get adresses function/usage
Store redirects, reconstruct transport on redirect, add redirect + lookup tests
Add another test
Review fixes
Initial bulk of review fixes (cleanups inc)
Comment cleanup
Add some more tests
Cleanups
Give addrFilter a type and add the config wiring
Expose filters
LookupHost cleanups
Remove Resolved Addresses and Redirect chain from replies to client without breaking RPC layer
Switch address/redirect logging method, add redirect loop checking + test
Review fixes + remove IPv6
Remove AddressFilter remnant + constant-ize the VA timeout
Review fixes pt. 1
Initialize validation record
Don't blank out validation reocrds
Add validation record sanity checking
Switch to shared struct
Check port is in valid range
Review fixes
Fixes#579 (which blocks #132).
This changes the SA to use a unique index on the sha256 of a
Registration's JWK's public key data instead of on the full serialized
JSON of the JWK. This corrects multiple problems:
1. MySQL/Mariadb no longer complain about key's being larger than the
largest allowed key size in an index
2. We no longer have to worry about large keys not being seen as unique
3. We no longer have to worry about the JWK's JSON being serialized with its inner keys in different orders and causing incorrectly empty queries or non-unique writes.
This change also hides the details of how Registrations are stored in
the database from the other services outside of SA. This will give us
greater flexibility if we need to move them to another database, or
change their schema, etc.
Also, adds some tests for NoSuchRegistration in the SA.
Only if the cache returns nothing for the CNAME query do we need to
look up CNAME/DNAME explicitly, in order to check CAAs on the parent
of the CNAME target rather than the parent of the original name.
This is the result of `godep save -r ./...` and
`git rm -r -f Godeps/_workspace/src/github.com/square`
Our fork is currently at the head of go-jose when Richard made the local nonce
changes, with the nonce changes added on top. In other words, the newly created
files are exactly equal to the deleted files.
In a separate commit I will bring our own go-jose fork up to the remote head,
then update our deps.
Also note: Square's go-jose repo contains a `cipher` package. Since we don't
make any changes to that package, we leave it imported as-is.
This allows us to log the remote address and registration object along with the
CSR.
Also, restore part of a comment on CertificateRequest that was deleted.
- Created some helper methods to run DNSSEC and reduce code reuse
- Support multiple DNS servers, but not in the Config file (yet)
- Fix typo; r=@rolandshoemaker
This approach performs a best-effort generation of the first OCSP response during
certificate issuance. In the event that OCSP generation fails, it logs a warning at
the Boulder-CA console, but returns successfully since the Certificate was itself
issued.
- Moved the transaction handling up to the `certificate-authority.go` file
- Simplified `certificate-authority-data.go`
- Created a mocks file in `test/` and reworked RA and CA to use it
- More audit logging to CA
- NewPendingAuthorization now uses a core.Authorization object, so
that foreign key constraints are followed
- core.Authorization now serializes RegistrationID to JSON, so it has to get
blanked out in WFE before transmission to client.
- Remove ParsedCertificate from core.Certificate, as type x509.Certificate cannot
be marshaled.
- Added AssertDeepEquals and AssertMarhsaledEquals to test-tools.go
- Caught several overloaded and misleadingly named errors in WFE
- Fix a bunch of typos in rpc-wrappers.go
- Unblank `id` in core.Registration JSON:
- It's not spec, but it's not hurting anything, and we reveal it to clients anyway.
- We need knowledge of the ID in RPC, so if we don't want to include this in the object, we need to make a transfer object to wrap it.
- Make the RPC logs much clearer as to who's talking to who
- Typo in WFE where we called a registration an authz
- Move dbMap construction and type converter into individual files in the sa package.
- Add DB configuration for the OCSP tool to the boulder config:
- left to the user if they want to use different boulder-config.json files
for different purposes.
- Added updater to Makefile
- Fix trailing ',' in the Boulder config, add more panic logging
- Ignore .pem files produced by the integration test
- Change RPC to use per-instance named reply-to queues.
- Finish OCSP Updater logic
- Rework RPC for OCSP to use a transfer object (due to serialization problems of x509.Certificate)
- Add SQL configuration options
- Increase the width of the authz and pending_authz tables' challenges field
- Make it configurable whether CREATE TABLE commands should run
- Added SQL debug logging (SA option: "SQLDebug")
- Added timestamps to the log prints to stdout
- Ignore *.pem in test/js
- Modified start.sh to support environment overrides for BOULDER_CONFIG, like the AMQP mode
- Changed boulder-test-config to open the server on the loopback device, so as to not cause firewall prompts on each integration test run for those of us being restrictive
- Renamed "key" column to "jwk" in DB, to avoid keyword conflict
- Set MaxLength on "jwk" column to 512
- Move setting the core.Registration.Key field from RA.NewRegistration to
WFE.NewRegistration to avoid a chicken-and-egg problem.
- Note: I kept the RPC wrapper object even though it now only has one field.
Seems like it's a good practice to use wrapper objects, even though we don't
everywhere.
- Don't ignore entropy underruns in challenges.go
- Correct identity crisis in Policy Authority; hopefully it will remember.
- Add a method `AuditObject` in audit-logger and convert RA/VA to use it
- Fix json typo in registration-authority that caused empty audit logs
- Fix vet issue in WFE where RegID was being printed as a 32-bit int instead of 64-bit
- Unfix the issue in WFE where RegID isn't right, per PR #215
- Auditing for general errors in executables
- Auditing for improper messages received by WFE
- Automatic audit wlogging of software errors
- Audit logging for mis-routed messages
- Audit logging for certificate requests
- Auditing for improper messages received by WFE
- Add audit events table
- Expect more details in TestRegistration in web-front-end_test.go
- Remove "extra" debug details from web-front-end.go per Issue #174
- Improve test coverage of web-front-end.go
- WFE audit updates for revocation support rebase
- Add audit messages to RPC for Improper Messages and Error Conditions
- Also note misrouted messages
- Spec says the Challenge objects contain a field "Validated" not "Completed."
- The Challenge object says "Validated" should be omitempty, but wasn't a pointer.
- Swapped to using pointers so it will not be "completed":"0001-01-01T00:00:00Z"
- Sort of related to [Issue #71 in Acme-Spec](https://github.com/letsencrypt/acme-spec/issues/71)
- Remove commented-out line from Dockerfile (whoops)
Jacob Hoffman-Andrews
Roland Shoemaker
Richard Barnes
Jeff Hodges
Jeremy Gillula
bifurcation
Tom Clegg
Romain Fliedel
J.C. Jones
Brad Warren
James 'J.C.' Jones
Peter Eckersley
Daniel Martí
J.C. Jones