* Delete Policy DB.
This is no longer needed now that we have a JSON policy file.
* Fix tests.
* Revert Dockerfile.
* Fix create_db
* Simplify user addition.
* Fix tests.
* Fix tests
* Review fixes.
https://github.com/letsencrypt/boulder/pull/1773
* Fix all errcheck errors
* Add errcheck to test.sh
* Add a new sa.Rollback method to make handling errors in rollbacks easier.
This also causes a behavior change in the VA. If a HTTP connection is
abruptly closed after serving the headers for a non-200 response, the
reported error will be the read failure instead of the non-200.
- Remove error signatures from log methods. This means fewer places where errcheck will show ignored errors.
- Pull in latest cfssl to be compatible with errorless log messages.
- Reduce the number of message priorities we support to just those we actually use.
- AuditNotice -> AuditInfo
- Remove InfoObject (only one use, switched to Info)
- Remove EmergencyExit and related functions in favor of panic
- Remove SyslogWriter / AuditLogger separate types in favor of a single interface, Logger, that has all the logging methods on it.
- Merge mock log into logger. This allows us to unexport the internals but still override them in the mock.
- Shorten names to be compatible with Go style: New, Set, Get, Logger, NewMock, etc.
- Use a shorter log format for stdout logs.
- Remove "... Starting" log messages. We have better information in the "Versions" message logged at startup.
Motivation: The AuditLogger / SyslogWriter distinction was confusing and exposed internals only necessary for tests. Some components accepted one type and some accepted the other. This made it hard to consistently use mock loggers in tests. Also, the unnecessarily fat interface for AuditLogger made it hard to meaningfully mock out.
boulder-ca was redundantly calling SetAuditLogger, leading to a warning message.
NewDbMap was calling GetAuditLogger, which is an anti-pattern, but didn't really
need to log anyhow. So deleted those lines. This cleans up some of the spam that
shows up when a test fails.
This eliminates the need the a database to store the hostname policy,
simplifying deployment. We keep the database for now, as part of our
deployability guidelines: we'll deploy, then switch config to the new style.
This also disables the obsolete whitelist checking code, but doesn't yet change
the function signature for policy.New(), to avoid bloating the pull request.
I'll fully remove the whitelist checking code in a future change when I also
remove the policy database code.
This allows secret values (PIN) to be separated from the main config.
Part of #1157.
In the process, move the CA constructor in the direction of
https://github.com/letsencrypt/boulder/wiki/Config-plan:
- Make most fields private.
- Take private key and issuer cert as constructor arguments rather than
constructing them internally.
This allows the CA test to parse the private key and issuer cert once, rather
than once per test case.
In the process, break out AMQP config into its own struct, one per service.
The AMQPConfig struct is included by composition in the config structs that need
it. If any given service lacks an AMQP config of its own, it gets a default
value from the top-level AMQP config struct, for deployability reasons.
Tightens the RPC code to take a specific AMQP config, not an over-broad
cmd.Config.
Shortens construction of specific RPC clients so they instatiate the generic
client connection themselves, simplifying per-service startup code.
Remove unused SetTimeout method on RPC clients.
Consolidate initialization of stats and logging from each main.go into cmd
package.
Define a new config parameter, `StdoutLevel`, that determines the maximum log
level that will be printed to stdout. It can be set to 6 to inhibit debug
messages, or 0 to print only emergency messages, or -1 to print no messages at
all.
Remove the existing config parameter `Tag`. Instead, choose the tag from the
basename of the currently running process. Previously all Boulder log messages
had the tag "boulder", but now they will be differentiated by process, like
"boulder-wfe".
Shorten the date format used in stdout logging, and add the current binary's
basename.
Consolidate setup function in audit-logger_test.go.
Note: Most CLI binaries now get their stats and logging from the parameters of
Action. However, a few of our binaries don't use our custom AppShell, and
instead use codegangsta/cli directly. For those binaries, we export the new
StatsAndLogging method from cmd.
Fixes https://github.com/letsencrypt/boulder/issues/852
This means after parsing the config file, setting up stats, and dialing the
syslogger. But it is still before trying to initialize the given server. This
means that we are more likely to get version numbers logged for some common
runtime failures.
Adds a new service, Publisher, which exists to submit issued certificates to various Certificate Transparency logs. Once submitted the Publisher will also parse and store the returned SCT (Signed Certificate Timestamp) receipts that are used to prove inclusion in a specific log in the SA database. A SA migration adds the new SCT receipt table.
The Publisher only exposes one method, SubmitToCT, which is called in a goroutine by ca.IssueCertificate as to not block any other issuance operations. This method will iterate through all of the configured logs attempting to submit the certificate, and any required intermediate certificates, to them. If a submission to a log fails it will be retried the pre-configured number of times and will either use a back-off set in a Retry-After header or a pre-configured back-off between submission attempts.
This changeset is the first of a number of changes ending with serving SCT receipts in OCSP responses and purposefully leaves out the following pieces for follow-up PRs.
* A fake CT server for integration testing
* A external tool to search the database for certificates lacking a full set of SCT receipts
* A method to construct X.509 v3 extensions containing receipts for the OCSP responder
* Returned SCT signature verification (beyond just checking that the signature is of the correct type so we aren't just serving arbitrary binary blobs to clients)
Resolves#95.
The ca's TestRevoke was failing occasionally.
The test was saying "has the certificate's OCSPLastUpdated been set to a
time within the last second?" as a way to see if the revocation updated
the OCSPLastUpdated. OCSPLastUpdated was not being set on revocation,
but the test still passed most of the time.
The test still passed most of the time because the creation of the
certificate (which also sets the OCSPLastUpdated) has usually happened
within the last second. So, even without revocation, the OCSPLastUpdated
was set to something in the last second because the test is fast.
Threading a clock.FakeClock through the CA induced the test to fail
consistently. Debugging and threading a FakeClock through the SA caused
changes in times reported but did not fix the test because the
OCSPLastUpdated was simply not being updated. There were not tests for
the sa.MarkCertificateRevoked API that was being called by
ca.RevokeCertificate.
Now the SA has tests for its MarkCertificateRevoked method. It uses a
fake clock to ensure not just that OCSPLastUpdated is set correctly, but
that RevokedDate is, as well. The test also checks for the
CertificateStatus's status and RevocationCode changes.
The SA and CA now use Clocks throughout instead of time.Now() allowing
for more reliable and expansive testing in the future.
The CA had to gain a public Clock field in order for the RA to use the
CertificateAuthorityImpl struct without using its constructor
function. Otherwise, the field would be nil and cause panics in the RA
tests.
The RA tests are similarly also panicking when the CAImpl attempts to
log something with its private, nil-in-those-tests log field but we're
getting "lucky" because the RA tests only cause the CAImpl to log when
they are broken.
There is a TODO there to make the CAImpl's constructor function take
just what it needs to operate instead of taking large config objects and
doing file IO and such. The Clk field should be made private and the log
field filled in for the RA tests.
Fixes#734.
This has required some substantive changes to the tests. Where
previously the foreign key constraints did not exist in the tests, now
that we use the actual production schema, they do. This has mostly led
to having to create real Registrations in the sa, ca, and ra tests. Long
term, it would be nice to fake this out better instead of needing a real
sa in the ca and ra tests.
The "goose" being referred to is <https://bitbucket.org/liamstask/goose>.
Database migrations are stored in a _db directory inside the relevant
owner service (namely, ca/_db, and sa/_db, today).
An example of migrating up with goose:
goose -path ./sa/_db -env test up
An example of creating a new migration with goose:
goose -path ./sa/_db -env test create NameOfNewMigration sql
Notice the "sql" at the end. It would be easier for us to manage sql
migrations. I would like us to stick to only them. In case we do use Go
migrations in the future, the underscore at the beginning of "_db" will
at least prevent build errors when using "..." with goose-created Go
files. Goose-created Go migrations do not compile with the go tool but
only with goose.
Fixes#111
Unblocks #623
Roland Bracewell Shoemaker