Update github.com/google/safebrowsing and block on database health before starting VA
before starting `boulder-va`.
```
$ go test .
ok github.com/google/safebrowsing 4.510s
$ go test .
ok github.com/golang/protobuf/ptypes 0.002s
```
Fixes#2742.
This commit updates the `publicsuffix-go` dependency to f5c9a8, the tip
of master at the time of writing.
Per CONTRIBUTING.md, the unit tests were run:
```
~/go/src/github.com/weppos/publicsuffix-go$ go test
./...
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.006s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.025s
```
When running `gomock` to generate mocks in the boulder-tools image there is a requirement on `github.com/golang/mock/mockgen/model` but only during runtime (it is not required to build `gomock`). So that we don't require users to `go get` this package so that it exists in their GOPATH we need to vendor it so that it is always in the GOPATH of the boulder-tools image. In order to vendor this package (since it isn't actually used anywhere) we need to add a special file that imports this package and uses it for a variable that isn't actually used anywhere so that we can satisfy `godep`, this is done in the `test` package.
Fixes#2751.
Per review policy, running tests in updated dependencies yields:
```
$ go test ./vendor/github.com/cloudflare/cfssl/ocsp/
? github.com/letsencrypt/boulder/vendor/github.com/cloudflare/cfssl/ocsp [no test files]
```
Switches imports from `github.com/google/certificate-transparency` to `github.com/google/certificate-transparency-go` and vendors the new code. Also fixes a number of small breakages caused by API changes since the last time we vendored the code. Also updates `github.com/cloudflare/cfssl` since you can't vendor both `github.com/google/certificate-transparency` and `github.com/google/certificate-transparency-go`.
Side note: while doing this `godep` tried to pull in a number of imports under the `golang.org/x/text` repo that I couldn't find actually being used anywhere so I just dropped the changes to `Godeps/Godeps.json` and didn't add the vendored dir to the tree, let's see if this breaks any tests...
All tests pass
```
$ go test ./...
ok github.com/google/certificate-transparency-go 0.640s
ok github.com/google/certificate-transparency-go/asn1 0.005s
ok github.com/google/certificate-transparency-go/client 22.054s
? github.com/google/certificate-transparency-go/client/ctclient [no test files]
ok github.com/google/certificate-transparency-go/fixchain 0.133s
? github.com/google/certificate-transparency-go/fixchain/main [no test files]
ok github.com/google/certificate-transparency-go/fixchain/ratelimiter 27.752s
ok github.com/google/certificate-transparency-go/gossip 0.322s
? github.com/google/certificate-transparency-go/gossip/main [no test files]
ok github.com/google/certificate-transparency-go/jsonclient 25.701s
ok github.com/google/certificate-transparency-go/merkletree 0.006s
? github.com/google/certificate-transparency-go/preload [no test files]
? github.com/google/certificate-transparency-go/preload/dumpscts/main [no test files]
? github.com/google/certificate-transparency-go/preload/main [no test files]
ok github.com/google/certificate-transparency-go/scanner 0.013s
? github.com/google/certificate-transparency-go/scanner/main [no test files]
ok github.com/google/certificate-transparency-go/tls 0.033s
ok github.com/google/certificate-transparency-go/x509 1.071s
? github.com/google/certificate-transparency-go/x509/pkix [no test files]
? github.com/google/certificate-transparency-go/x509util [no test files]
```
```
$ ./test.sh
...
ok github.com/cloudflare/cfssl/api 1.089s coverage: 81.1% of statements
ok github.com/cloudflare/cfssl/api/bundle 1.548s coverage: 87.2% of statements
ok github.com/cloudflare/cfssl/api/certadd 13.681s coverage: 86.8% of statements
ok github.com/cloudflare/cfssl/api/client 1.314s coverage: 55.2% of statements
ok github.com/cloudflare/cfssl/api/crl 1.124s coverage: 75.0% of statements
ok github.com/cloudflare/cfssl/api/gencrl 1.067s coverage: 72.5% of statements
ok github.com/cloudflare/cfssl/api/generator 2.809s coverage: 33.3% of statements
ok github.com/cloudflare/cfssl/api/info 1.112s coverage: 84.1% of statements
ok github.com/cloudflare/cfssl/api/initca 1.059s coverage: 90.5% of statements
ok github.com/cloudflare/cfssl/api/ocsp 1.178s coverage: 93.8% of statements
ok github.com/cloudflare/cfssl/api/revoke 2.282s coverage: 75.0% of statements
ok github.com/cloudflare/cfssl/api/scan 2.729s coverage: 62.1% of statements
ok github.com/cloudflare/cfssl/api/sign 2.483s coverage: 83.3% of statements
ok github.com/cloudflare/cfssl/api/signhandler 1.137s coverage: 26.3% of statements
ok github.com/cloudflare/cfssl/auth 1.030s coverage: 68.2% of statements
ok github.com/cloudflare/cfssl/bundler 15.014s coverage: 85.1% of statements
ok github.com/cloudflare/cfssl/certdb/dbconf 1.042s coverage: 78.9% of statements
ok github.com/cloudflare/cfssl/certdb/ocspstapling 1.919s coverage: 69.2% of statements
ok github.com/cloudflare/cfssl/certdb/sql 1.265s coverage: 65.7% of statements
ok github.com/cloudflare/cfssl/cli 1.050s coverage: 61.9% of statements
ok github.com/cloudflare/cfssl/cli/bundle 1.023s coverage: 0.0% of statements
ok github.com/cloudflare/cfssl/cli/crl 1.669s coverage: 57.8% of statements
ok github.com/cloudflare/cfssl/cli/gencert 9.278s coverage: 83.6% of statements
ok github.com/cloudflare/cfssl/cli/gencrl 1.310s coverage: 73.3% of statements
ok github.com/cloudflare/cfssl/cli/genkey 3.028s coverage: 70.0% of statements
ok github.com/cloudflare/cfssl/cli/ocsprefresh 1.106s coverage: 64.3% of statements
ok github.com/cloudflare/cfssl/cli/revoke 1.081s coverage: 88.2% of statements
ok github.com/cloudflare/cfssl/cli/scan 1.217s coverage: 36.0% of statements
ok github.com/cloudflare/cfssl/cli/selfsign 2.201s coverage: 73.2% of statements
ok github.com/cloudflare/cfssl/cli/serve 1.133s coverage: 39.0% of statements
ok github.com/cloudflare/cfssl/cli/sign 1.210s coverage: 54.8% of statements
ok github.com/cloudflare/cfssl/cli/version 2.475s coverage: 100.0% of statements
ok github.com/cloudflare/cfssl/cmd/cfssl 1.082s coverage: 0.0% of statements
ok github.com/cloudflare/cfssl/cmd/cfssljson 1.016s coverage: 4.0% of statements
ok github.com/cloudflare/cfssl/cmd/mkbundle 1.024s coverage: 0.0% of statements
ok github.com/cloudflare/cfssl/config 2.754s coverage: 67.7% of statements
ok github.com/cloudflare/cfssl/crl 1.063s coverage: 68.3% of statements
ok github.com/cloudflare/cfssl/csr 27.016s coverage: 89.6% of statements
ok github.com/cloudflare/cfssl/errors 1.081s coverage: 81.2% of statements
ok github.com/cloudflare/cfssl/helpers 1.217s coverage: 80.4% of statements
ok github.com/cloudflare/cfssl/helpers/testsuite 7.658s coverage: 65.8% of statements
ok github.com/cloudflare/cfssl/initca 205.809s coverage: 74.2% of statements
ok github.com/cloudflare/cfssl/log 1.016s coverage: 59.3% of statements
ok github.com/cloudflare/cfssl/multiroot/config 1.107s coverage: 77.4% of statements
ok github.com/cloudflare/cfssl/ocsp 1.524s coverage: 77.7% of statements
ok github.com/cloudflare/cfssl/revoke 1.775s coverage: 79.6% of statements
ok github.com/cloudflare/cfssl/scan 1.022s coverage: 1.1% of statements
ok github.com/cloudflare/cfssl/selfsign 1.119s coverage: 70.0% of statements
ok github.com/cloudflare/cfssl/signer 1.019s coverage: 20.0% of statements
ok github.com/cloudflare/cfssl/signer/local 3.146s coverage: 81.2% of statements
ok github.com/cloudflare/cfssl/signer/remote 2.328s coverage: 71.8% of statements
ok github.com/cloudflare/cfssl/signer/universal 2.280s coverage: 67.7% of statements
ok github.com/cloudflare/cfssl/transport 1.028s
ok github.com/cloudflare/cfssl/transport/ca/localca 1.056s coverage: 94.9% of statements
ok github.com/cloudflare/cfssl/transport/core 1.538s coverage: 90.9% of statements
ok github.com/cloudflare/cfssl/transport/kp 1.054s coverage: 37.1% of statements
ok github.com/cloudflare/cfssl/ubiquity 1.042s coverage: 88.3% of statements
ok github.com/cloudflare/cfssl/whitelist 2.304s coverage: 100.0% of statements
```
Fixes#2746.
This removes the config and code to output to statsd.
- Change `cmd.StatsAndLogging` to output a `Scope`, not a `Statter`.
- Remove the prefixing of component name (e.g. "VA") in front of stats; this was stripped by `autoProm` but now no longer needs to be.
- Delete vendored statsd client.
- Delete `MockStatter` (generated by gomock) and `mocks.Statter` (hand generated) in favor of mocking `metrics.Scope`, which is the interface we now use everywhere.
- Remove a few unused methods on `metrics.Scope`, and update its generated mock.
- Refactor `autoProm` and add `autoRegisterer`, which can be included in a `metrics.Scope`, avoiding global state. `autoProm` now registers everything with the `prometheus.Registerer` it is given.
- Change va_test.go's `setup()` to not return a stats object; instead the individual tests that care about stats override `va.stats` directly.
Fixes#2639, #2733.
This commit updates the `publicsuffix-go` dependency to 908fd3b. Per
CONTRIBUTING.md the upstream unit tests were verified to pass:
```
daniel@XXXX:~/go/src/github.com/weppos/publicsuffix-go$ git log --oneline | head -n1
908fd3b autopull: 2017-04-25T06:00:35Z (#75)
daniel@XXXX:~/go/src/github.com/weppos/publicsuffix-go$ go test ./...
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.014s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.027s
```
Deletes github.com/streadway/amqp and the various RabbitMQ setup tools etc. Changes how listenbuddy is used to proxy all of the gRPC client -> server connections so we test reconnection logic.
+49 -8,221 😁Fixes#2640 and #2562.
This PR updates the `publicsuffix-go` dependency to `fb1fc94`, the
latest autopull and the HEAD of master at the time of writing.
Per CONTRIBUTING.md the tests were verified to pass:
```
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.007s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.027s
```
Updates the various gRPC/protobuf libs (google.golang.org/grpc/... and github.com/golang/protobuf/proto) and the boulder-tools image so that we can update to the newest github.com/grpc-ecosystem/go-grpc-prometheus. Also regenerates all of the protobuf definition files.
Tests run on updated packages all pass.
Unblocks #2633fixes#2636.
Switch from `gorp.v1` to `gorp.v2`. Removes `vendor/gopkg.in/gorp.v1` and vendors `vendor/gopkg/go-gorp/gorp.v2`, all tests pass.
Changes between `v1.7.1` and `v2.0.0`: c87af80f3c...4deece6103Fixes#2490.
This commit updates the `publicsuffix-go` dependency to version 0.3.2,
the latest autopull.
Per CONTRIBUTING.md the tests are verified to pass:
```
HEAD position was 5ebfcac... Fix outdated version number
HEAD is now at c12e7e9... autopull: 2017-03-04T06:00:47Z (#62)
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
=== RUN TestPublicSuffix
--- PASS: TestPublicSuffix (0.00s)
=== RUN TestEffectiveTLDPlusOne
--- PASS: TestEffectiveTLDPlusOne (0.00s)
PASS
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.007s
=== RUN TestValid
--- PASS: TestValid (0.00s)
=== RUN TestIncludePrivate
--- PASS: TestIncludePrivate (0.00s)
=== RUN TestIDNA
--- PASS: TestIDNA (0.00s)
=== RUN TestPsl
--- PASS: TestPsl (0.01s)
=== RUN TestNewListFromString
--- PASS: TestNewListFromString (0.00s)
=== RUN TestNewListFromString_IDNAInputIsUnicode
--- PASS: TestNewListFromString_IDNAInputIsUnicode (0.00s)
=== RUN TestNewListFromString_IDNAInputIsAscii
--- PASS: TestNewListFromString_IDNAInputIsAscii (0.00s)
=== RUN TestNewListFromFile
--- PASS: TestNewListFromFile (0.00s)
=== RUN TestListAddRule
--- PASS: TestListAddRule (0.00s)
=== RUN TestListFind
--- PASS: TestListFind (0.00s)
=== RUN TestNewRule_Normal
--- PASS: TestNewRule_Normal (0.00s)
=== RUN TestNewRule_Wildcard
--- PASS: TestNewRule_Wildcard (0.00s)
=== RUN TestNewRule_Exception
--- PASS: TestNewRule_Exception (0.00s)
=== RUN TestNewRule_FromASCII
--- PASS: TestNewRule_FromASCII (0.00s)
=== RUN TestNewRule_FromUnicode
--- PASS: TestNewRule_FromUnicode (0.00s)
=== RUN TestNewRuleUnicode_FromASCII
--- PASS: TestNewRuleUnicode_FromASCII (0.00s)
=== RUN TestNewRuleUnicode_FromUnicode
--- PASS: TestNewRuleUnicode_FromUnicode (0.00s)
=== RUN TestRuleMatch
--- PASS: TestRuleMatch (0.00s)
=== RUN TestRuleDecompose
--- PASS: TestRuleDecompose (0.00s)
=== RUN TestLabels
--- PASS: TestLabels (0.00s)
=== RUN TestCookieJarList
--- PASS: TestCookieJarList (0.00s)
PASS
ok github.com/weppos/publicsuffix-go/publicsuffix 0.027s
```
In the last weeks we made some large changes to the list of .RU and .SU domains in the PSL, due to some very old policy changes at the registry (2009) and more recent follow up.
Given the amount of pressure about these changes from certain users, most certainly because LE limits, I figured out you'll soon have people asking you to merge the changes. I've packaged a new release of publicsuffix-go, and updated the dependency in this PR.
$ git show master
commit c5490f26d8f43b84857ac54e23387b8ed9b100dd
Author: Simone Carletti <weppos@weppos.net>
Date: Tue Feb 7 23:26:14 2017 +0100
Release 0.3.2
➜ publicsuffix-go git:(master) go test ./...
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.023s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.039s
Please note this release also includes the .ONION as per publicsuffix/list#374
Pulls in logging improvements in OCSP Responder and the CT client, plus a handful of API changes. Also, the CT client verifies responses by default now.
This change includes some Boulder diffs to accommodate the API changes.
We recently made changes to the IANA suffixes, and you may want to pull them into the latest Boulder version.
```
➜ publicsuffix-go git:(master) git show -s
commit 3ea542729b4d7056a9d1356c9baf27bcad2bda7f
Author: Simone Carletti <weppos@weppos.net>
Date: Mon Jan 2 18:28:57 2017 +0100
Release 0.3.1
```
```
➜ publicsuffix-go git:(master) go test ./...
? github.com/weppos/publicsuffix-go/cmd/gen [no test files]
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.045s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.091s
```
v0.3.1 is tagged and signed with my PGP key.
https://github.com/weppos/publicsuffix-go/releases/tag/v0.3.1
Right now we are using a third-party client for the Google Safe Browsing API, but Google has recently released their own [Golang library](https://github.com/google/safebrowsing) which also supports the newer v4 API. Using this library will let us avoid fixing some lingering race conditions & unpleasantness in our fork of `go-safebrowsing-api`.
This PR adds support for using the Google library & the v4 API in place of our existing fork when the `GoogleSafeBrowsingV4` feature flag is enabled in the VA "features" configuration.
Resolves https://github.com/letsencrypt/boulder/issues/1863
Per `CONTRIBUTING.md` I also ran the unit tests for the new dependency:
```
daniel@XXXXXXXXXX:~/go/src/github.com/google/safebrowsing$ go test ./...
ok github.com/google/safebrowsing 3.274s
? github.com/google/safebrowsing/cmd/sblookup [no test files]
? github.com/google/safebrowsing/cmd/sbserver [no test files]
? github.com/google/safebrowsing/cmd/sbserver/statik [no test files]
? github.com/google/safebrowsing/internal/safebrowsing_proto [no test files]
ok github.com/google/safebrowsing/vendor/github.com/golang/protobuf/jsonpb 0.012s
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/jsonpb/jsonpb_test_proto [no test files]
ok github.com/google/safebrowsing/vendor/github.com/golang/protobuf/proto 0.062s
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/proto/proto3_proto [no test files]
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/protoc-gen-go [no test files]
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/protoc-gen-go/descriptor [no test files]
ok github.com/google/safebrowsing/vendor/github.com/golang/protobuf/protoc-gen-go/generator 0.017s
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/protoc-gen-go/grpc [no test files]
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/protoc-gen-go/plugin [no test files]
ok github.com/google/safebrowsing/vendor/github.com/golang/protobuf/ptypes 0.009s
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/ptypes/any [no test files]
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/ptypes/duration [no test files]
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/ptypes/empty [no test files]
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/ptypes/struct [no test files]
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/ptypes/timestamp [no test files]
? github.com/google/safebrowsing/vendor/github.com/golang/protobuf/ptypes/wrappers [no test files]
? github.com/google/safebrowsing/vendor/github.com/rakyll/statik [no test files]
? github.com/google/safebrowsing/vendor/github.com/rakyll/statik/fs [no test files]
ok github.com/google/safebrowsing/vendor/golang.org/x/net/idna 0.003s
```
There's an off-the-shelf package that provides most of the stats we care about
for gRPC using interceptors. This change vendors go-grpc-prometheus and its
dependencies, and calls out to the interceptors provided by that package from
our own interceptors.
This will allow us to get metrics like latency histograms by call, status codes
by call, and so on.
Fixes#2390.
This change vendors go-grpc-prometheus and its dependencies. Per contributing guidelines, I've run the tests on these dependencies, and they pass:
go test github.com/davecgh/go-spew/spew github.com/grpc-ecosystem/go-grpc-prometheus github.com/grpc-ecosystem/go-grpc-prometheus/examples/testproto github.com/pmezard/go-difflib/difflib github.com/stretchr/testify/assert github.com/stretchr/testify/require github.com/stretchr/testify/suite
ok github.com/davecgh/go-spew/spew 0.022s
ok github.com/grpc-ecosystem/go-grpc-prometheus 0.120s
? github.com/grpc-ecosystem/go-grpc-prometheus/examples/testproto [no test files]
ok github.com/pmezard/go-difflib/difflib 0.042s
ok github.com/stretchr/testify/assert 0.021s
ok github.com/stretchr/testify/require 0.017s
ok github.com/stretchr/testify/suite 0.012s
This PR updates the github.com/letsencrypt/pkcs11key dependency to v1.0.0.
Per CONTRIBUTING.md I checked the unit tests pass:
daniel@XXXXX:~/go/src/github.com/letsencrypt/pkcs11key$ git show -s
commit c47dab18ddd8c4d4267661870d3ba331cddc57c9
Author: Jacob Hoffman-Andrews <github@hoffman-andrews.com>
Date: Fri Sep 9 15:23:00 2016 -0400
Echo path when failing to load module. (#6)
daniel@XXXXX:~/go/src/github.com/letsencrypt/pkcs11key$ go test ./...
ok github.com/letsencrypt/pkcs11key 0.002s
ok github.com/letsencrypt/pkcs11key/uri 0.002s
This PR updates our GRPC library dep. to v1.0.3. It's likely we can update to v1.0.4 without much effort but on a first attempt it seems that the SupportPackageIsVersion3 to SupportPackageIsVersion4 change might cause some headaches so I started with 1.0.3.
The grpc/creds.go serverTransportCredentials and clientTransportCredentials needed two new funcs (Clone and OverrideServerName) to conform to the updated credentials.TransportCredentials interface.
It's tempting to remove grpc/creds.go clientTransportCredentials entirely now that the TLSCredentials from upstream has a OverrideServerName function we can use, but unfortunately it only supports one hostname and must be called a-head of ClientHandshake that we still need clientTransportCredentials for our use-case. I tried this and failed, so clientTransportCredentials remains in bgrpc/creds/creds.go.
Per CONTRIBUTING.md I've verified the unit tests pass:
daniel@XXXXXX:~/go/src/google.golang.org/grpc$ git show -s
commit b7f1379d3cbbbeb2ca3405852012e237aa05459e
Merge: 33731fd bac9e1d
Author: Qi Zhao <toqizhao@gmail.com>
Date: Mon Oct 17 16:02:05 2016 -0700
Merge pull request #903 from improbable-io/blocking-graceful-shutdown-fix
Make concurrent Server.GracefulStop calls all behave equivalently.
daniel@XXXXXX:~/go/src/google.golang.org/grpc$ go test ./...
ok google.golang.org/grpc 0.215s
ok google.golang.org/grpc/benchmark 0.017s
? google.golang.org/grpc/benchmark/client [no test files]
? google.golang.org/grpc/benchmark/grpc_testing [no test files]
? google.golang.org/grpc/benchmark/server [no test files]
? google.golang.org/grpc/benchmark/stats [no test files]
? google.golang.org/grpc/benchmark/worker [no test files]
? google.golang.org/grpc/codes [no test files]
ok google.golang.org/grpc/credentials 0.041s
? google.golang.org/grpc/credentials/oauth [no test files]
? google.golang.org/grpc/examples/helloworld/greeter_client [no test files]
? google.golang.org/grpc/examples/helloworld/greeter_server [no test files]
? google.golang.org/grpc/examples/helloworld/helloworld [no test files]
? google.golang.org/grpc/examples/route_guide/client [no test files]
? google.golang.org/grpc/examples/route_guide/routeguide [no test files]
? google.golang.org/grpc/examples/route_guide/server [no test files]
ok google.golang.org/grpc/grpclb 0.047s
? google.golang.org/grpc/grpclb/grpc_lb_v1 [no test files]
? google.golang.org/grpc/grpclog [no test files]
? google.golang.org/grpc/grpclog/glogger [no test files]
? google.golang.org/grpc/health [no test files]
? google.golang.org/grpc/health/grpc_health_v1 [no test files]
? google.golang.org/grpc/internal [no test files]
? google.golang.org/grpc/interop [no test files]
? google.golang.org/grpc/interop/client [no test files]
? google.golang.org/grpc/interop/grpc_testing [no test files]
? google.golang.org/grpc/interop/server [no test files]
ok google.golang.org/grpc/metadata 0.004s
? google.golang.org/grpc/naming [no test files]
? google.golang.org/grpc/peer [no test files]
ok google.golang.org/grpc/reflection 0.029s
? google.golang.org/grpc/reflection/grpc_reflection_v1alpha [no test files]
? google.golang.org/grpc/reflection/grpc_testing [no test files]
? google.golang.org/grpc/stress/client [no test files]
? google.golang.org/grpc/stress/grpc_testing [no test files]
? google.golang.org/grpc/stress/metrics_client [no test files]
ok google.golang.org/grpc/test 94.693s
? google.golang.org/grpc/test/codec_perf [no test files]
? google.golang.org/grpc/test/grpc_testing [no test files]
ok google.golang.org/grpc/transport 12.574s
This PR updates the `github.com/prometheus/client_golang` dependency to the v0.8.0 release tag.
For future note, `godep update github.com/prometheus/client_golang/prometheus/promhttp` doesn't work correctly for this package due to the nesting structure (See https://github.com/tools/godep/issues/164). This can be worked around by using `godep update github.com/prometheus/client_golang/...` instead ¯\\\_(ツ)\_/¯
Per `CONTRIBUTING.md` I verified the unit tests pass:
```
daniel@XXXXXX:~/go/src/github.com/prometheus/client_golang$ git show -s
commit c5b7fccd204277076155f10851dad72b76a49317
Merge: a4d14b3 1b26087
Author: Björn Rabenstein <bjoern@rabenste.in>
Date: Wed Aug 17 17:48:24 2016 +0200
Merge pull request #226 from prometheus/beorn7/alloc
Bring back zero-alloc label-value access for metric vecs
daniel@XXXXXX:~/go/src/github.com/prometheus/client_golang$ go test ./...
ok github.com/prometheus/client_golang/api/prometheus 0.003s
? github.com/prometheus/client_golang/examples/random [no test files]
? github.com/prometheus/client_golang/examples/simple [no test files]
ok github.com/prometheus/client_golang/prometheus 28.661s
ok github.com/prometheus/client_golang/prometheus/promhttp 0.013s
ok github.com/prometheus/client_golang/prometheus/push 0.007s
```
This commit updates the `go-sql-driver` dependency to the tip of master
(665b83488b90b902ce0a305ef6652e599771cdf9).
A v1.3 release and a commitment to semantic versioning[0] will soon
allow us to switch this from master to a well defined tag.
[0]
- https://github.com/go-sql-driver/mysql/issues/476#issuecomment-256004996
This commit updates the cactus `go-statsd-client` to the 2.0.2 release
tag.
Note: the most recent release is 3.1.0 but the introduction of
substatter support in 3.0.0 changes the `Statter` interface we rely on
in the boulder metrics package. We should consider follow-up work to
refactor our metrics code to be compatible with the 3.x releases.
Relates to #2315.
This pull request updates the publicsuffix-go dependency to version 0.3.0, most notably including weppos/publicsuffix-go#40 and support for IDN TLDs.
The PA's TestWillingToIssue unit test is updated to confirm that Boulder is WillingToIssue a well formed IDN domain with an IDN TLD. Prior to c5cc328 this causes the PA unit tests to fail as expected with urn:acme:error:malformed :: Name does not end in a public suffix. After
c5cc328 everything is 💯
Per CONTRIBUTING.md the unit tests are confirmed to pass:
daniel@XXXXXX:~/go/src/github.com/weppos/publicsuffix-go$ git show -s
commit 49fe4b0e8276b314e6703300ac26940d9c090a06
Author: Simone Carletti <weppos@weppos.net>
Date: Mon Nov 21 19:26:37 2016 +0100
Release 0.3.0
daniel@XXXXXX:~/go/src/github.com/weppos/publicsuffix-go$ go test ./...
? github.com/weppos/publicsuffix-go/cmd/gen [no test files]
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.007s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.042s
❤️🍺 and 🎉's to @weppos for the upstream work required for this fix. We truly appreciate your volunteer work on the PSL and the publicsuffix-go library. You're the best!
This resolves#2277.
This commit updates the `go-jose` dependency to [v1.1.0](https://github.com/square/go-jose/releases/tag/v1.1.0) (Commit: aa2e30fdd1fe9dd3394119af66451ae790d50e0d). Since the import path changed from `github.com/square/...` to `gopkg.in/square/go-jose.v1/` this means removing the old dep and adding the new one.
The upstream go-jose library added a `[]*x509.Certificate` member to the `JsonWebKey` struct that prevents us from using a direct equality test against two `JsonWebKey` instances. Instead we now must compare the inner `Key` members.
The `TestRegistrationContactUpdate` function from `ra_test.go` was updated to populate the `Key` members used in testing instead of only using KeyID's to allow the updated comparisons to work as intended.
The `Key` field of the `Registration` object was switched from `jose.JsonWebKey` to `*jose.JsonWebKey ` to make it easier to represent a registration w/o a Key versus using a value with a nil `JsonWebKey.Key`.
I verified the upstream unit tests pass per contributing.md:
```
daniel@XXXXX:~/go/src/gopkg.in/square/go-jose.v1$ git show
commit aa2e30fdd1fe9dd3394119af66451ae790d50e0d
Merge: 139276c e18a743
Author: Cedric Staub <cs@squareup.com>
Date: Thu Sep 22 17:08:11 2016 -0700
Merge branch 'master' into v1
* master:
Better docs explaining embedded JWKs
Reject invalid embedded public keys
Improve multi-recipient/multi-sig handling
daniel@XXXXX:~/go/src/gopkg.in/square/go-jose.v1$ go test ./...
ok gopkg.in/square/go-jose.v1 17.599s
ok gopkg.in/square/go-jose.v1/cipher 0.007s
? gopkg.in/square/go-jose.v1/jose-util [no test files]
ok gopkg.in/square/go-jose.v1/json 1.238s
```
This vendors the Prometheus client code, and exports metrics on the debug port, under `/metrics`.
This will currently export just the default metrics, like `go_goroutines`, `process_cpu_seconds_total`, `process_open_fds`, and `process_resident_memory_bytes`. Later work will start exporting Boulder-specific metrics, but this will allow Ops to start configuring scraping of Prometheus metrics in production.
Tests pass:
```
$ git diff master Godeps/ | sed -ne 's/^+.*ImportPath": "//p' | tr -d '",' | xargs go test
ok github.com/beorn7/perks/quantile 0.562s
ok github.com/matttproud/golang_protobuf_extensions/pbutil 0.003s
ok github.com/prometheus/client_golang/prometheus 34.418s
ok github.com/prometheus/client_golang/prometheus/promhttp 0.003s
? github.com/prometheus/client_model/go [no test files]
ok github.com/prometheus/common/expfmt 0.019s
ok github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg 0.002s
ok github.com/prometheus/common/model 0.003s
ok github.com/prometheus/procfs 0.008s
```
Part of #2284
Add feature flagged support for issuing for IDNs, fixes#597.
This patch expects that clients have performed valid IDN2008 encoding on any label that includes unicode characters. Invalid encodings (including non-compatible IDN2003 encoding) will be rejected. No script-mixing or script exclusion checks are performed as we assume that if a name is resolvable that it conforms to the registrar's policies on these matters and if it uses non-standard scripts in sub-domains etc that browsers should be the ones choosing how to display those names.
Required a full update of the golang.org/x/net tree to pull in golang.org/x/net/idna, all test suites pass.
Fixes#1880.
Updates google.golang.org/grpc and github.com/jmhodges/clock, both test suites pass. A few of the gRPC interfaces changed so this also fixes those breakages.
This commit updates the publicsuffix-go dependency to upstream HEAD (commit
088a5b7b1f746fca57b9440f8e94f9b3302605b7 (there are no tags or releases
for this project)).
Per CONTRIBUTING.md I ran the project unit tests:
```
daniel@xxxxxxxxx:~/go/src/github.com/weppos/publicsuffix-go$ git show -s
commit 088a5b7b1f746fca57b9440f8e94f9b3302605b7
Author: Simone Carletti <weppos@weppos.net>
Date: Wed Aug 10 10:52:20 2016 +0200
autopull: 2016-08-10T06:00:16Z (#20)
daniel@xxxxxxxxx:~/go/src/github.com/weppos/publicsuffix-go$ go test ./...
? github.com/weppos/publicsuffix-go/cmd/gen [no test files]
? github.com/weppos/publicsuffix-go/cmd/load [no test files]
ok github.com/weppos/publicsuffix-go/net/publicsuffix 0.008s
ok github.com/weppos/publicsuffix-go/publicsuffix 0.029s
```
Part of #2080.
This change vendors `crypto/x509`, `crypto/x509/pkix`, and `encoding/asn1` from 1d5f6a765d. That commit is a direct child of the Go 1.5.4 release tag, so it contains the same code as the current Go version we are using. In that commit I rewrote imports in those packages so they depend on each other internally rather than calling out to the standard library, which would cause type disagreements.
I changed the imports in each place where we're parsing CSRs, and imported under a different name `oldx509`, both to avoid collisions and make it clear what's going on. Places that only use `x509` to parse certificates are not changed, and will use the current standard library.
This will unblock us from moving to Go 1.6, and subsequently Go 1.7.
This PR removes the use of the global configuration variable BOULDER_CONFIG. It also removes the global configuration struct cmd.Config. Furthermore, it removes the dependency codegangsta/cli and the last bit of code that was using it cmd/single-ocsp/main.go.
This is the final (hopefully) pull request in the work to remove the reliance on a global configuration structure. Included below is a history of all other pull requests relevant in accomplishing this:
WFE (#1973)
RA (#1974)
SA (#1975)
CA (#1978)
VA (#1979)
Publisher (#2008)
OCSP Updater (#2013)
OCSP Responder (#2017)
Admin Revoker (#2053)
Expiration Mailer (#2036)
Cert Checker (#2058)
Orphan Finder (#2059)
Single OCSP (this PR)
Closes#1962
This pulls in two changes that fix the case where pkcs11 failed to load the module. One where the return value was nil, and one where an empty string could be passed to libtool-ltdl.
Tests pass for both packages.
Updates `github.com/cloudflare/cfssl` and `github.com/google/certificate-transparency/go` to current master. CFSSL has re-structured some of the `signer/local` code which should be given a once-over. Otherwise everything seems mostly benign and/or doesn't affect our usage.
Vendored tests pass.
This PR replaces the `x/net/publicsuffix` package with `weppos/publicsuffix-go`.
The conversations that leaded to this decision are #1479 and #1374. To summarize the discussion, the main issue with `x/net/publicsuffix` is that the package compiles the list into the Go source code and doesn't provide a way to easily pull updates (e.g. by re-parsing the original PSL) unless the entire package is recompiled.
The PSL update frequency is almost daily, which makes very hard to recompile the official Golang package to stay up-to-date with all the changes. Moreover, Golang maintainers expressed some concerns about rebuilding and committing changes with a frequency that would keep the package in sync with the original PSL. See https://github.com/letsencrypt/boulder/issues/1374#issuecomment-182429297
`weppos/publicsuffix-go` contains a compiled version of the list that is updated weekly (or more frequently). Moreover, the package can read and parse a PSL from a String or a File which will effectively decouple the Boulder source code with the list itself. The main benefit is that it will be possible to update the definition by simply downloading the latest list and restarting the application (assuming the list is persisted in memory).
The `letsencrypt/boulder-tools` image was recently updated, pulling in version
0.8.0 of certbot. That version stores the output of `certonly` requests in a
different path. In test.sh, we check out a specific tagged release of certbot in
order to get its integration tests. Prior to this commit, we were using
certbot 0.8.0 with the integration tests from version 0.6.0 of certbot,
which looked for `certonly` output in the wrong place, and failed.
This commit changes test.sh to checkout the 0.8.0 branch, and also removes a
temporary shim we used to make the `certbot` command call out to the
`letsencrypt` command.
Also, since the latest version of `letsencrypt/boulder-tools` includes an updated
`protoc-gen-go`, this change also updates the support packages to match.
Adds a server side unary RPC interceptor which includes basic stats. We could also use this to add a server request ID to the context.Context to identify the call through the system, but really I'd rather do that on the client side before the RPC is sent which requires the client interceptor implementation upstream. Also updates google.golang.org/grpc.
Updates #1880.
When a CAA request to Unbound times out, fall back to checking CAA via Google Public DNS' HTTPS API, through multiple proxies so as to hit geographically distributed paths. All successful multipath responses must be identical in order to succeed, and at most one can fail.
Fixes#1618
Roland Bracewell Shoemaker