Fixes#1880.
Updates google.golang.org/grpc and github.com/jmhodges/clock, both test suites pass. A few of the gRPC interfaces changed so this also fixes those breakages.
We have seen a couple issues from folks that run into trouble using `docker-compose up` with invalid `$GOPATH`'s configured (e.g. see issues #2150, #2141, #2112).
This PR adds a sentence to the README indicate that if you see a docker "oci runtime error" or a failure to create the container it may be caused by your `$GOPATH` and to check that first.
Fixes#1576.
Adds a new package mock_metrics, with code generated by gomock, in order to test the change.
Modifies publisher.New to take a metrics.Scope and an SA, and unexport SA.
Moves core of submission loop into a separate function, singleLogSubmit, which can return an error rather than using the continue keyword. This reduces repetition of AuditErr lines, and makes it easier to put error statting in one place.
The `MailerImpl` gains a few new fields (`retryBase`, & `retryMax`). These are used with `core.RetryBackoff` in `reconnect()` to implement exponential backoff in a reconnect attempt loop. Both `expiration-mailer` and `notify-mailer` are modified to add CLI args for these 2 flags and to wire them into the `MailerImpl` via its `New()` constructor.
In `MailerImpl`'s `SendMail()` function it now detects when `sendOne` returns an `io.EOF` error indicating that the server closed the connection unexpectedly. When this case occurs `reconnect()` is invoked. If the reconnect succeeds then we invoke `sendOne` again to try and complete the message sending operation that was interrupted by the disconnect.
For integration testing purposes I modified the `mail-test-srv` to support a `-closeChance` parameter between 0 and 100. This controls what % of `MAIL` commands will result in the server immediately closing the client connection before further processing. This allows us to simulate a flaky mailserver. `test/startservers.py` is modified to start the `mail-test-srv` with a 35% close chance to thoroughly test the reconnection logic during the existing `expiration-mailer` integration tests. I took this as a chance to do some slight clean-up of the `mail-test-srv` code (mostly removing global state).
For unit testing purposes I modified the mailer `TestConnect` test to abstract out a server that can operate similar to `mail-test-serv` (e.g. can close connections artificially).
This is testing a server that **closes** a connection, and not a server that **goes away/goes down**. E.g. the `core.RetryBackoff` sleeps themselves are not being tested. The client is disconnected and attempts a reconnection which always succeeds on the first try. To test a "gone away" server would require a more substantial rewrite of the unit tests and the `mail-test-srv`/integration tests. I think this matches the experience we have with MailChimp/Mandril closing long lived connections.
This commit updates the publicsuffix-go dependency to upstream HEAD (commit
088a5b7b1f746fca57b9440f8e94f9b3302605b7 (there are no tags or releases
for this project)).
Per CONTRIBUTING.md I ran the project unit tests:
```
daniel@xxxxxxxxx:~/go/src/github.com/weppos/publicsuffix-go$ git show -s
commit 088a5b7b1f746fca57b9440f8e94f9b3302605b7
Author: Simone Carletti <weppos@weppos.net>
Date: Wed Aug 10 10:52:20 2016 +0200
autopull: 2016-08-10T06:00:16Z (#20)
daniel@xxxxxxxxx:~/go/src/github.com/weppos/publicsuffix-go$ go test ./...
? github.com/weppos/publicsuffix-go/cmd/gen [no test files]
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.008s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.029s
```
While finalizing the testing for #2101 I noticed that the `notify-mailer` would panic when `-dryRun=true` (e.g. the default value):
```
E150646 notify-mailer [AUDIT] Panic caused by err: runtime error: invalid memory address or nil pointer dereference
E150646 notify-mailer [AUDIT] Stack Trace (Current frame) goroutine 1 [running]:
github.com/letsencrypt/boulder/log.(*impl).AuditPanic(0xc820167610)
/home/daniel/go/src/github.com/letsencrypt/boulder/log/log.go:190 +0x190
panic(0x9804a0, 0xc82000e120)
/usr/local/go/src/runtime/panic.go:443 +0x4e9
github.com/letsencrypt/boulder/metrics.(*StatsdScope).Inc(0x0, 0xa9ad30, 0x11, 0x1, 0x0, 0x0)
/home/daniel/go/src/github.com/letsencrypt/boulder/metrics/scope.go:68 +0xd5
github.com/letsencrypt/boulder/mail.(*MailerImpl).SendMail(0xc8201dc360, 0xc8203d5150, 0x1, 0x1, 0x7ffeda89737c, 0x1a, 0xc8201f0000, 0x517, 0x0, 0x0)
/home/daniel/go/src/github.com/letsencrypt/boulder/mail/mailer.go:290 +0x108
main.(*mailer).run(0xc820175ea8, 0x0, 0x0)
/home/daniel/go/src/github.com/letsencrypt/boulder/cmd/notify-mailer/main.go:108 +0x362
main.main()
/home/daniel/go/src/github.com/letsencrypt/boulder/cmd/notify-mailer/main.go:365 +0x142e
```
This was caused by the `NewDryRun` constructor not initializing the `stats` member of the `MailerImpl` and is fixed in this commit.
Formerly in expiration-mailer, when we wanted to set `lastNagSent` to `Now()`, we started a transaction, read the object, updated one field, wrote it back, and closed the transaction.
This commit replaces the transaction and instead does a much simpler and more efficient `UPDATE certificateStatus SET lastNagSent = ? where serial = ?;`.
Fixes#140.
This patch allows users to specify the following revocation reasons based on my interpretation of the meaning of the codes but could use confirmation from others.
* unspecified (0)
* keyCompromise (1)
* affiliationChanged (3)
* superseded (4)
* cessationOfOperation (5)
Remove the code that echoes exit status on integration failure. It was buggy and
would always print 0, which was confusing. Instead, use `run`.
Rename run_and_comment to run_and_expect_silence, since it doesn't comment
anymore.
The create_db.sh script needs goose to create and fill the database. This is not
available yet, because it is simultaneously being installed in line 10. By moving the create_db.sh call after the `wait`, we make sure goose is available.
As a follow-up to #2098 this PR changes the sentence describing "an alias for letsencrypt" to reference an "alias to certbot" instead. This fixes what was an outdated reference to the client formerly known as Let's Encrypt, now Certbot.
The slow start guide's "working with a client" section of the README still referenced a `letsencrypt` path instead of the correct `certbot` path when describing sourcing the integration test alias script. This PR updates the path to use certbot.
Previously, the CA would log the full PEM encoded certificate signing request and the signed certificate. This PR, removes that, and logs a base64 encoded version of the request and the certificate.
Due to the fact that cfssl returns the certificate in PEM format regardless, we decode the certificate into DER and log the base64 version. If there is any failure in the decoding from PEM to DER, the CA will log the PEM encoded version of the certificate (this is the only case).
Furthermore, the Signing Success message was moved after the certificate is checked for 0 length and decoded into DER, which seems to make logical sense to me.
Fixes#1968
Part of #2080.
This change vendors `crypto/x509`, `crypto/x509/pkix`, and `encoding/asn1` from 1d5f6a765d. That commit is a direct child of the Go 1.5.4 release tag, so it contains the same code as the current Go version we are using. In that commit I rewrote imports in those packages so they depend on each other internally rather than calling out to the standard library, which would cause type disagreements.
I changed the imports in each place where we're parsing CSRs, and imported under a different name `oldx509`, both to avoid collisions and make it clear what's going on. Places that only use `x509` to parse certificates are not changed, and will use the current standard library.
This will unblock us from moving to Go 1.6, and subsequently Go 1.7.
Roland Bracewell Shoemaker
Jacob Hoffman-Andrews
Daniel McCarney
Roland Shoemaker
Blake Griffith
Ben Irving
Maarten
Okke Timm