Commit Graph

3235 Commits

Author SHA1 Message Date
Jacob Hoffman-Andrews 71e4af43f7 Roll forward "Run Travis tests in Docker (#1830)" (#1838)
That change broke the certbot tests because it switched to a MariaDB
10.1-specific syntax. certbot/certbot#3058 changes the certbot tests to use
Boulder's docker-compose.yml, so they will get MariaDB 10.1 automatically.
2016-05-24 15:11:22 -07:00
Jacob Hoffman-Andrews b954dcc010 Revert "Run Travis tests in Docker (#1830)" (#1834)
This reverts commit 92d94f2 and commit 0b4623f to unbreak the Certbot build.
2016-05-20 15:57:10 -07:00
Daniel McCarney 0b4623f8a3 Fixes certbot build with mariadb 10 syntax (#1833)
Certbot invokes the `test/create_db.sh` script during its integration testing. The Boulder MariaDB instance is moving to 10.1, and the `sa_db_users.sql` sql fragment for creating users has been changed to utilize a 10.1+ syntax feature to create users only if they don't exist. Since Certbot and Travis remain on 10.0 this presently breaks their build.

This pull request changes `create_db.sh` to detect if the MariaDB instance is 10.0, and if so, uses a `mariadb100_users.sql` sql fragment that maintains the 10.0 compatible way of creating users. When Certbot and Travis can support MariaDB 10.1 we can kill the `mariadb100_users.sql` file and corresponding logic.
2016-05-20 10:49:01 -07:00
Jacob Hoffman-Andrews 92d94f2558 Run Travis tests in Docker (#1830)
* MariaDB 10.1

* MariaDB 10.1 in Docker

* Run docker stuff.

* Improve test.js error.

* Lower log level

* Revert dockerfile to master

* Export debug ports, set FAKE_DNS, and remove container_name.

* Remove typo.

* Make integration-test.py wait for debug ports.

* Use 10.1 and export more Boulder ports.

* Test updates for Docker

Listen on 0.0.0.0 for utility servers.
Make integration-test.py just wait for ports rather than calling startservers.
Run docker-compose in test.sh.
Remove bypass when database exists.
Separate mailer test into its own function in integration test.
Print better errors in test.js.

* Always bring up mysql container.

* Wait for MySQL to come up.

* Put it in travis-before-install.

* Use 127

* Remove manual docker-up.

* Add ifconfig

* Switch to docker-compose run

* It works!

* Remove some spurious env vars.

* Add bash

* try running it

* Add all deps.

* Pass through env.

* Install everything in the Dockerfile.

* Fix install of ruby

* More improvements

* Revert integration test to run directly
Also remove .git from dockerignore and add some packages.

* Revert integration-test.py to master.

* Stop ignoring test/js

* Start from boulder-tools.

* Add boulder-tools.

* Tweak travis.yml

* Separate out docker-compose pull as install.

* Build in install phase; don't bother with go install in Dockerfile

* Add virtualenv

* Actually build rabbitmq-setup

* Remove FAKE_DNS

* Trivial change

* Pull boulder-tools as a separate step so it gets its own timing info.

* Install certbot and protobuf from repos.

* Use cerbot from debian backports.

* Fix clone

* Remove CERTBOT_PATH

* Updates

* Go back to letsencrypt for build.sh

* Remove certbot volume.

* go back to preinstalled letsencrypt

* Restore ENV

* Remove BASH_ENV

* Adapt reloader test so it psses when run as root.

* Fixups for review.

* Revert test.js

* Revert startservers.py

* Revert Makefile.
2016-05-19 16:29:45 -07:00
Jacob Hoffman-Andrews 6f082f397b Improve error logging in test.js (#1829)
Also fix a typo in startservers.py and quote variables in Makefile (provides more meaningful errors when they are unset).
2016-05-19 15:54:53 -07:00
Jacob Hoffman-Andrews e45cd826f7 Remove github-pr-status in test.sh. (#1827) 2016-05-16 17:51:21 -07:00
Jacob Hoffman-Andrews aa2c2529f9 Disable debug logs in integration test. (#1825) 2016-05-16 13:35:44 -07:00
Jacob Hoffman-Andrews 4e60e4bb59 Restore testing of godep restore. (#1821)
PR #1808 accidentally removed the main diff.
Also, leave out diffing of Godeps in the `go generate` test.

https://github.com/letsencrypt/boulder/pull/1821
2016-05-14 22:03:47 +02:00
Roland Bracewell Shoemaker 96a57434eb Make boulder-publisher work as a gRPC service (#1708) 2016-05-13 16:32:18 -07:00
Kane York f0cc131c44 Exact blacklist (#1816)
* policy: Implement exact blacklist matching
* fix?
* add test
* don't use real properties in tests

https://github.com/letsencrypt/boulder/pull/1816
2016-05-13 20:26:15 +02:00
Kane York fef60a8fd6 Add statsd reporting of current DB connection count (#1805)
* rename, change params, restructure
* I'm wondering how I managed that one
* use a metrics.Scope
* move method to SA, update callers
* rerun goimports
* fix compile error
* revert cmd/shell.go

https://github.com/letsencrypt/boulder/pull/1805
2016-05-12 20:33:23 -07:00
Jacob Hoffman-Andrews e62a99039d Use MarshalIndent in WFE (#1811)
* Use MarshalIndent in WFE

This makes it easier to read output without first running it through a JSON
prettifier.

* Fix test
* marshal->marshalIndent

https://github.com/letsencrypt/boulder/pull/1811
2016-05-12 20:31:36 -07:00
Jacob Hoffman-Andrews dbc81e0ede Improve error messages in validation cases. (#1812)
* Improve error messages in validation cases.

* Add challenge type to error detail.

This makes the errors easier to search for.

* %v->%s

* Revert boulder-config to master.
2016-05-12 20:30:56 -07:00
Roland Bracewell Shoemaker 28f851c375 Strip out Godep version before diffing Godeps.json (#1808)
This means our tests won't break when the Godep version is increased.
2016-05-12 17:43:46 -07:00
Kane York c4197ea7df Remove unused Validated field from Challenge (#1817)
Remove unused Validated field from core.Challenge
2016-05-12 17:33:10 -07:00
Jacob Hoffman-Andrews 546361cf68 Add a maxDBConns to ocspResponder. (#1806)
* Add a maxDBConns to ocspResponder.
2016-05-11 09:27:04 -07:00
Roland Bracewell Shoemaker b980e91e5b Bump godep to v66 (#1807) 2016-05-11 09:21:14 -07:00
Jacob Hoffman-Andrews d5dcc1f157 Remove external-cert-importer. (#1794)
Not currently used.
2016-05-10 10:45:19 -07:00
Jacob Hoffman-Andrews eeea318246 Change two-review rule to one (#1802)
Two are encouraged but no longer mandatory.
2016-05-10 10:39:59 -07:00
Kane York 09db2e5a41 Fix error path call in IssueCertificate rpc wrapper (#1799)
* Fix error path call in IssueCertificate rpc wrapper
* Bump godeps version
2016-05-09 16:01:32 -07:00
Jacob Hoffman-Andrews b3bc3d8e41 Add a MaxDBConns config parameter. (#1793) 2016-05-09 14:21:15 -07:00
Kane York 339405bcb9 Look up A and AAAA in parallel (#1760)
This allows validating IPv6-only hosts.

Fixes #593.
2016-05-09 08:38:23 -07:00
Ben Irving a41b1dd091 Update README.md for running certbot_test client (#1795) 2016-05-08 12:28:18 -07:00
Kane York 7a4aa49add Return false when ocsp blob is empty (#1771)
Return false when ocsp blob is empty
2016-05-06 17:22:19 -07:00
Jacob Hoffman-Andrews 7208052ad6 Change paths to match certbot renaming. (#1792) 2016-05-06 15:57:07 -07:00
Roland Bracewell Shoemaker efeb81d4be Use a LimitedReader when reading from HTTP response bodies (#1789)
Instead of ioutil.ReadAll
2016-05-05 13:58:14 -07:00
Jacob Hoffman-Andrews 9ed8268acd Make authz lookup more efficient (remove ORDER BY) (#1784)
- Improve speed of authz lookup.
- Remove all uses of obsolete GetLatestValidAuthorization
- Update test to call GetValidAuthorizations.
2016-05-05 13:48:32 -07:00
Jacob Hoffman-Andrews f776343d4e Remove forked repo godep restore exception. (#1786) 2016-05-05 11:22:17 -07:00
Jacob Hoffman-Andrews eb2367a536 Remove obsolete config line. (#1788)
Also move `issuerDomain` so that diffs between boulder-config.json and
boulder-config-next.json are minimized.
2016-05-05 11:22:06 -07:00
Roland Bracewell Shoemaker 35b6e83e81 Implement CAA quorum checking after failure (#1763)
When a CAA request to Unbound times out, fall back to checking CAA via Google Public DNS' HTTPS API, through multiple proxies so as to hit geographically distributed paths. All successful multipath responses must be identical in order to succeed, and at most one can fail.

Fixes #1618
2016-05-05 11:16:58 -07:00
Jacob Hoffman-Andrews 2c39c684cb Fix HostnamePolicyFile (#1785)
CA was referencing RA's config.
Also, the FailOnError function needs a non-nil error to fail.

* Bump Godep version.
2016-05-04 08:12:08 -07:00
Roland Bracewell Shoemaker 8ee578c3b7 Check JWK is valid before sending to SA (#1782)
Update vendored github.com/square/go-jose
Check key is valid before sending to SA
2016-05-03 14:26:52 -07:00
Kane York fb4955c72a Fix expiration-mailer logspam when all certs are renewed (#1770)
Fix expiration-mailer logspam when all certs are renewed
Fixes #1772
2016-05-02 13:48:28 -07:00
Jacob Hoffman-Andrews e5e4fb744a * Delete Policy DB.This is no longer needed now that we have a JSON policy file.* Fix tests.* Revert Dockerfile.* Fix create_db* Simplify user addition.* Fix tests.* Fix tests* Review fixes.https://github.com/letsencrypt/boulder/pull/1773
* Delete Policy DB.

This is no longer needed now that we have a JSON policy file.

* Fix tests.
* Revert Dockerfile.
* Fix create_db
* Simplify user addition.
* Fix tests.
* Fix tests
* Review fixes.

https://github.com/letsencrypt/boulder/pull/1773
2016-04-29 12:12:24 -07:00
Jacob Hoffman-Andrews f2f3e37b48 Add line to README for GO15VENDOREXPERIMENT. (#1779) 2016-04-29 11:26:33 -07:00
Kane York 801626fb15 Limit the length of logged HTTP response (#1778)
* Limit the length of logged HTTP response

Fixes #1777

* Fix crash, add tests
* Fix utf-8 truncation
* move more logic into helper method
* Add unit test for truncateBody

https://github.com/letsencrypt/boulder/pull/1778
2016-04-29 11:25:51 -07:00
Kane York dc15f6a55e Readd timing stats for VA (#1759)
* Fix newVARPC sanity check logic error

* Redo style of sanity check in RA to match VA

* Switch to ServerInternal(), readd log.Info

* Add tests to make sure correct metric measured

* comments on exported symbols

* review fixes

* build an array instead of storing last
2016-04-27 11:57:21 -07:00
Jacob Hoffman-Andrews 702a0dbb0c Hard-fail godep-restore. (#1767) 2016-04-25 11:49:34 -07:00
Jacob Hoffman-Andrews 6d5348f975 Run go generate in Travis (#1762)
* Fix go generate command in metrics.

The previous command only worked on OS X. This one works on Linux but not
OS X.

Also add generate phase of test.sh.

* Add mockgen to test setup.

* Fix github-pr-status output.

* Fix envvar style.

* Set xtrace.

* Fix test.sh

* Fix test.sh some more.

* Fix mockgen command.

* Add dependencies for running `go generate`.

* Add protoc-gen-go.

* Fix go get command.

* Fix generate.

* Wait for all.

* Fix generate.

* Update generated pb.

* Fix generate commands for vendored world.

* Update documentation for new vendor style.

* Update grpc package to latest.

* Update caaChecker proto with latest.

* Run go generate only over TESTPATHS

* See if Travis passes under 1.6

* Switch back to 1.5.

* Trim run command.

* Run stringer from correct directory.

* Move generate command.

* Restore and generate

* Fix path.

* list contents of GOPATH.

* Fix stringer by prebuilding.

* Try another import path.

* regenerate bcode_string.

* remove excess package

* pull jsha fork of protoc-gen-go that echoes

* Echo protoc version.

* install from source

* CD back.

* Go back to normal protoc-gen-go

* Fix path

* Move protobuf install into test/setup.sh

* Move before_install to install.

* Set PATH.

* Follow 301 with curl.

* Shuffle test order.

* Swap back test order.

* Restore all tests.

* Restore 1.5.3 to Travis.

* Remove unnecessary wait-or-exit

* Generate metrics mock with latest mockgen.

* Wrap TESTPATHS in curlies

* Remove spurious bracket
2016-04-21 15:23:06 -07:00
Jacob Hoffman-Andrews 26de87dfa2 Update grpc dep and regenerate caa-checker. (#1761)
* Update grpc dep and regenerate caa-checker.

The latest version generates a different format. This is a precursor to running
go generate in Travis.
2016-04-21 11:05:17 -07:00
Kane York b7cf618f5d context.Context as the first parameter of all RPC calls (#1741)
Change core/interfaces to put context.Context as the first parameter of all RPC calls in preparation for gRPC.
2016-04-19 11:34:36 -07:00
Kane York 0c1ddccd9c Parallelise email DNS lookup for new-reg (#1731)
* Parallelise email DNS lookup for new-reg

* Remove the 1-case switch
2016-04-19 11:11:07 -07:00
Kane York 37fd71d21e Rework how KeyAuthorization works (#1688)
* Enhance error message in NewKeyAuthFromString

* va: generate expected response and string compare

* NewKeyAuth can return error, handle that...

* wip commit

* convert ch.KeyAuthorization to string and rename

* more wip commit

* wip 3

* wip 4

* delete NewKeyAuthorizationFromString

* change to ServerInternalProblem

* fix compile error

* semantic merge conflicts are the worst

actually compiled this time.

* Replace new error with statsd increment
2016-04-19 10:54:39 -07:00
Roland Bracewell Shoemaker c6de21a53a Fix total DNS latency stat (#1751)
exchangeOne used a deferd method which contained a expression as a argument. Because of how defer works the arguments where evaluated immediately (unlike the method) causing the total latency to always be the same.
2016-04-19 10:36:44 -07:00
bmw ea52c2130b Use tools/venv.sh from client repo in integration tests (#1752) 2016-04-18 16:50:26 -07:00
Jacob Hoffman-Andrews e6c17e1717 Switch to new vendor style (#1747)
* Switch to new vendor style.

* Fix metrics generate command.

* Fix miekg/dns types_generate.

* Use generated copies of files.

* Update miekg to latest.

Fixes a problem with `go generate`.

* Set GO15VENDOREXPERIMENT.

* Build in letsencrypt/boulder.

* fix travis more.

* Exclude vendor instead of godeps.

* Replace some ...

* Fix unformatted cmd

* Fix errcheck for vendorexp

* Add GO15VENDOREXPERIMENT to Makefile.

* Temp disable errcheck.

* Restore master fetch.

* Restore errcheck.

* Build with 1.6 also.

* Match statsd.*"

* Skip errcheck unles Go1.6.

* Add other ignorepkg.

* Fix errcheck.

* move errcheck

* Remove go1.6 requirement.

* Put godep-restore with errcheck.

* Remove go1.6 dep.

* Revert master fetch revert.

* Remove -r flag from godep save.

* Set GO15VENDOREXPERIMENT in Dockerfile and remove _worskpace.

* Fix Godep version.
2016-04-18 12:51:36 -07:00
Jacob Hoffman-Andrews 9cb9f82516 Revert "Stop fetching master for migrations test. (#1730)" (#1749)
This reverts commit 8c8fc01b01.

It turns out the master fetch actually was necessary in some other cases,
particularly when building non-master branches (as opposed to PRs).
2016-04-18 08:30:58 -07:00
Jakub Warmuz d669b708e2 Custom command for integration-test.py (#1744)
Allows other ACME clients (such as simp_le), to run any command for
integration testing (without having to implement custom busy-wait
loop etc.).
2016-04-17 10:27:05 -07:00
Jakub Warmuz 1b7fd169ba Test ToS at :4000 (#1742)
Fixes #1720
2016-04-16 10:53:02 -07:00
bmw e2e5e05766 Update client package names for integration tests (#1740) 2016-04-15 12:20:56 -07:00