Commit Graph

191 Commits

Author SHA1 Message Date
Jacob Hoffman-Andrews 7208052ad6 Change paths to match certbot renaming. (#1792) 2016-05-06 15:57:07 -07:00
Jacob Hoffman-Andrews f776343d4e Remove forked repo godep restore exception. (#1786) 2016-05-05 11:22:17 -07:00
Jacob Hoffman-Andrews 702a0dbb0c Hard-fail godep-restore. (#1767) 2016-04-25 11:49:34 -07:00
Jacob Hoffman-Andrews 6d5348f975 Run go generate in Travis (#1762)
* Fix go generate command in metrics.

The previous command only worked on OS X. This one works on Linux but not
OS X.

Also add generate phase of test.sh.

* Add mockgen to test setup.

* Fix github-pr-status output.

* Fix envvar style.

* Set xtrace.

* Fix test.sh

* Fix test.sh some more.

* Fix mockgen command.

* Add dependencies for running `go generate`.

* Add protoc-gen-go.

* Fix go get command.

* Fix generate.

* Wait for all.

* Fix generate.

* Update generated pb.

* Fix generate commands for vendored world.

* Update documentation for new vendor style.

* Update grpc package to latest.

* Update caaChecker proto with latest.

* Run go generate only over TESTPATHS

* See if Travis passes under 1.6

* Switch back to 1.5.

* Trim run command.

* Run stringer from correct directory.

* Move generate command.

* Restore and generate

* Fix path.

* list contents of GOPATH.

* Fix stringer by prebuilding.

* Try another import path.

* regenerate bcode_string.

* remove excess package

* pull jsha fork of protoc-gen-go that echoes

* Echo protoc version.

* install from source

* CD back.

* Go back to normal protoc-gen-go

* Fix path

* Move protobuf install into test/setup.sh

* Move before_install to install.

* Set PATH.

* Follow 301 with curl.

* Shuffle test order.

* Swap back test order.

* Restore all tests.

* Restore 1.5.3 to Travis.

* Remove unnecessary wait-or-exit

* Generate metrics mock with latest mockgen.

* Wrap TESTPATHS in curlies

* Remove spurious bracket
2016-04-21 15:23:06 -07:00
bmw ea52c2130b Use tools/venv.sh from client repo in integration tests (#1752) 2016-04-18 16:50:26 -07:00
Jacob Hoffman-Andrews e6c17e1717 Switch to new vendor style (#1747)
* Switch to new vendor style.

* Fix metrics generate command.

* Fix miekg/dns types_generate.

* Use generated copies of files.

* Update miekg to latest.

Fixes a problem with `go generate`.

* Set GO15VENDOREXPERIMENT.

* Build in letsencrypt/boulder.

* fix travis more.

* Exclude vendor instead of godeps.

* Replace some ...

* Fix unformatted cmd

* Fix errcheck for vendorexp

* Add GO15VENDOREXPERIMENT to Makefile.

* Temp disable errcheck.

* Restore master fetch.

* Restore errcheck.

* Build with 1.6 also.

* Match statsd.*"

* Skip errcheck unles Go1.6.

* Add other ignorepkg.

* Fix errcheck.

* move errcheck

* Remove go1.6 requirement.

* Put godep-restore with errcheck.

* Remove go1.6 dep.

* Revert master fetch revert.

* Remove -r flag from godep save.

* Set GO15VENDOREXPERIMENT in Dockerfile and remove _worskpace.

* Fix Godep version.
2016-04-18 12:51:36 -07:00
bmw e2e5e05766 Update client package names for integration tests (#1740) 2016-04-15 12:20:56 -07:00
Roland Bracewell Shoemaker 8eaf247ee9 Split CAA checking out to its own service (#1647)
* Split out CAA checking service (minus logging etc)
* Add example.yml config + follow general Boulder style
* Update protobuf package to correct version
* Add grpc client to va
* Add TLS authentication in both directions for CAA client/server
* Remove go lint check
* Add bcodes package listing custom codes for Boulder
* Add very basic (pull-only) gRPC metrics to VA + caa-service
2016-04-12 23:02:41 -07:00
Kane York 25b45a45ec Errcheck errors fixed (#1677)
* Fix all errcheck errors
* Add errcheck to test.sh
* Add a new sa.Rollback method to make handling errors in rollbacks easier.
This also causes a behavior change in the VA. If a HTTP connection is
abruptly closed after serving the headers for a non-200 response, the
reported error will be the read failure instead of the non-200.
2016-04-12 16:54:01 -07:00
Jacob Hoffman-Andrews 0a627dd74b Suppress noise on running unittests.
Remove -v for go install.

Don't print success after every successful command.

Remove `run` in front of `go test` to avoid printing each command.

This reduces the number of log lines taken by successful unittests in Travis by
66%, and makes failures much more visible and easy to read.
2016-04-05 17:35:07 -07:00
Jacob Hoffman-Andrews 7fe78676b6 Improve how we test godep restore. 2016-03-20 17:23:15 -07:00
Roland Bracewell Shoemaker 736376173a Merge branch 'master' into test-linear 2016-02-16 14:41:04 -08:00
Jacob Hoffman-Andrews 8f8e89b571 Add a 'godep restore' test in Travis.
This ensures our godeps match the remote repo.
2016-02-16 12:57:19 -08:00
Jacob Hoffman-Andrews 009ed1c4f5 Run unittests in serial.
Fixes spurious local failures.
2016-02-16 12:57:03 -08:00
Jacob Hoffman-Andrews 33b8c357ef Split out setup.sh from travis-before-install.
This accomplishes two things:
 - setup.sh should now be usable by the client integration test.
 - setup.sh can be used by new project members to simplify first setup.

Update the README to indicate the new file, and to correct some out-of-date
information.
2015-11-30 23:44:45 -08:00
Jacob Hoffman-Andrews 02d5aa5831 Integration test fixes
In https://github.com/letsencrypt/boulder/pull/1110 we put
the activate command in the wrong place so it didn't run if
LETSENCRYPT_PATH was set.

Also remove SIMPLE_HTTP_PORT which is no longer necessary. It was used to keep
the build passing as the client transitioned ports. The client now defaults to
5002.
2015-11-11 13:43:46 -08:00
Jeff Hodges 3df0592b15 move sourcing of activate into test.sh
This gets us closer to allowing the client repo to use
integration-test.py. They have a different path without "venv" in it for
their virtualenv set up.

Updates #1101
2015-11-06 18:11:06 -08:00
Jeff Hodges 0af66a2080 rename amqp-integration-test.py
Make it integration-test.py

Depends on #1102, so please review that first.

Updates #1101 (it's the second step).
2015-11-06 16:31:36 -08:00
Jeff Hodges 62489a6ba9 add skipping of node or letsencrypt integration
Updates #1101
2015-11-06 14:06:07 -08:00
Jeff Hodges e8cd5c1fdd fix integration by changing pip install
The letsencrypt repo renamed its requirements.txt and I'm told we don't
actually need to use it.

Fixes #1105
2015-11-05 21:53:59 -08:00
Jacob Hoffman-Andrews f1103cb094 Fix an erorr in test.sh.
This generates spurious warnings in Travis:

test.sh: line 94: [: x: integer expression expected

And also prevents certain conditions from failing when they should.
2015-10-16 14:06:31 -07:00
Richard Barnes 3a70363366 Selectively hard fail 2015-10-12 20:36:57 -07:00
Richard Barnes 4e98b21fe2 Undo change to cause test failure 2015-10-10 13:08:50 -04:00
Richard Barnes fdae77844c Enable hard-fail and print prettier 2015-10-10 13:07:12 -04:00
Richard Barnes 0d17d030bc Revert down-scoping of test.sh 2015-10-07 10:43:17 -04:00
Richard Barnes 2de8747752 Set threshold back to 0.81 to suppress package name errors 2015-10-07 09:54:30 -04:00
Richard Barnes 5c3d851e99 Remove stray echo line 2015-10-07 09:46:09 -04:00
Richard Barnes 964794d918 Reove wrapper; move failure logic to run_and_comment 2015-10-07 09:42:45 -04:00
Richard Barnes 90484ac460 Add a wrapper that turns non-empty output into non-zero status 2015-10-06 23:12:23 -04:00
bifurcation 90050e91f5 Merge branch 'master' into golint 2015-10-05 19:23:24 -04:00
Jacob Hoffman-Andrews 6e355f8a57 test.sh: upgrade setuptools and pip to latest
Fixes https://github.com/letsencrypt/boulder/issues/921,
an interoperability issue in letsencrypt client tests.
2015-10-05 13:09:43 -07:00
Richard Barnes fd42948605 Upgrade test.sh to catch golint errors 2015-10-05 00:12:24 -04:00
Richard Barnes df87fcd739 Suppress remaining golint output 2015-10-04 21:22:49 -04:00
Jacob Hoffman-Andrews b3aca1ff2b Speed up tests.
Make `make` aware of output files so it doesn't always have to rebuild. Also
make it use `go install`, which is faster than building files individually.

Now that make is faster, use it in startservers.py to consolidate building
logic. This also has the handy side-effect that ./start.py exposes useful build
information through /build, whereas before only the .rpm packaged version did.

Additionally, this allows us to remove `make` from the Travis matrix, since we
are running `make` as part of the integration test. This means each PR only
triggers two Travis builds instead of one, which means we will get results from
Travis faster.

Also, change the Travis matrix logic to be a list of actions to run, rather than
a list of actions to skip. That fixes
https://github.com/letsencrypt/boulder/issues/817.

Enumerate specific sections of test.sh to run, rather than sections to skip.

Note: ./start.py now installs into ./bin/ instead of $GOPATH/bin.

Only set up GitHub secret file (for PR status reporting) when available, and
decrypt it into /tmp rather than $HOME, to avoid accidentally caching it once
Travis' caching features are available.

Clone letsencrypt repo into $HOME instead of $TMP, to make it possible to cache
eventually.

Remove unused `mysql` dependency in Travis.

Override default Travis install command to prevent it from adding
Godeps/_workspace to GOPATH. When that happens, it hides failures that should
arise from importing non-vendorized paths.
2015-10-01 16:28:17 -07:00
Jacob Hoffman-Andrews bc0d98a820 Fix amqp-integration-test.py when run alone.
Previously, test.sh was responsible for running venv/bin/activate, meaning that
`python test/amqp-integration-test.py` would fail to run the letsencrypt client.
Now, so long as LETSENCRYPT_PATH is already set to a valid dir (e.g. in your
.bashrc), `python test/amqp-integration-test.py` should work.
2015-09-23 17:06:14 -07:00
Jacob Hoffman-Andrews 69ddcb784c Add a test that there are no outdated migrations.
If a branch merges with a migration that is timestamped earlier than other
migrations already in master, that migration may get skipped.
2015-09-18 14:53:40 -07:00
Jeff Hodges e22d0eef9a speed up TravisCI by pre-installing test deps
This brings the TravisCI build time down to 4 minutes.

It does so by preventing the TravisCI job from having to build the
shared dependencies of the tests multiple times for each `go test` call.
2015-09-01 17:43:20 -07:00
Jacob Hoffman-Andrews 3afc1bb71a Merge pull request #624 from letsencrypt/mysql_unit
use MariaDB in the unit tests
2015-08-15 01:01:39 -07:00
Jeff Hodges 5e97aa23c6 use MariaDB in the unit tests
And delete the uses of sqlite3
2015-08-14 17:13:15 -07:00
Jeff Hodges 71c2e55f77 add all of cmd/ to test.sh and Makefile
Plus, add to test.sh all future possible packages by using `go list` and
all future possible cmd's to the Makefile with `find`.

Also, fix a bug in external-cert-importer from a bad merge somewhere.
2015-08-14 12:21:44 -07:00
Jeff Hodges 25ba6e0400 use the default, not racy covermode in TravisCI
The count covermode is racy and was causing spurious-to-us looking (but not
really) race detections in the tests. See
https://github.com/golang/go/issues/12118
2015-08-13 12:34:30 -07:00
Jacob Hoffman-Andrews 64987a3639 Merge pull request #599 from letsencrypt/mysql
use MariaDB in integration tests and start.py
2015-08-11 20:31:46 -07:00
Jacob Hoffman-Andrews cb2156817d Update test.sh comment re: Python client failure. 2015-08-10 10:31:21 -07:00
Jeff Hodges b08e6e0bad use MariaDB in integration tests and start.py
This changes moves from using SQLite in the integration tests and in the
test/boulder-config.json.

It does not port the unit tests over, unfortunately. That's a much more
invasive change.

This also updates the Dockerfile to include the MariaDB and RabbitMQ
requirements of start.py as well as adjusts the CMD to expose the
boulder server to the host machine. The Dockerfile also needed to have
its Go version bumped and the test.sh had to grow some explict
"function"s.

Updates #132
2015-08-07 23:22:52 -07:00
Jeff Hodges 9286988495 re-enable the letsencrypt client integration tests
Dependent on letsencrypt/letsencrypt#641 merging to master.
2015-07-31 15:15:49 -07:00
Jeff Hodges e0939573df clean up and harden build_letsencrypt
The run function and && combine poorly and cause exported variables to
somehow fail to cross over. run also does status code checking so we can
just use run twice and delete that extra unneeded exit command.

Along the way, move the warning message to where it belongs (creating a
temp directory instead of the user providing a stable path), and use the
new url for letsencrypt/letsencrypt.
2015-07-29 23:59:52 -07:00
Roland Shoemaker abd06564ec Merge branch 'master' into mailer 2015-07-27 12:46:19 -07:00
Roland Shoemaker bd9286dd5b Merge branch 'master' into mailer 2015-07-24 16:36:50 -07:00
Roland Shoemaker 19067da6a9 More tests 2015-07-24 16:36:03 -07:00
Jeff Hodges cfcd6dfc88 remove pkcs11 tag for build and test scripts
With #525 in place, we no longer need to worry about setting the pkcs11
tag everywhere.
2015-07-24 16:33:05 -07:00
Roland Shoemaker 3f129219dc Merge pull request #523 from letsencrypt/rm-mkcert
Remove mkcert and mkcrl
2015-07-23 17:22:05 -07:00
Jacob Hoffman-Andrews e3a62e12de Merge pull request #522 from letsencrypt/test_sh_comment
fix first failure comment in test.sh
2015-07-23 17:11:57 -07:00
Jacob Hoffman-Andrews b082b552e4 Switch to \`go test ./...\` in test.sh 2015-07-23 16:59:41 -07:00
Jeff Hodges 77418e28d3 fix first failure comment in test.sh
A better fix might be to fail earlier on when, say, go vet fails.
2015-07-23 15:26:44 -07:00
Jacob Hoffman-Andrews 6a25f554b6 Fix gofmt test.
Previously, the gofmt test was not properly causing Travis to fail, because it
would call run_and_comment on a printf command, which always succeeded. Instead,
we create a function for the gofmt check that outputs on stdout and returns a
meaningful status code, then call run_and_comment on that.

Also fix up a few places that expected TRAVIS_PULL_REQUEST=false, which doesn't
happen when running locally.
2015-07-22 14:24:36 -07:00
Jeff Hodges 7f083ba4da correct unit test skip conditional 2015-07-16 09:43:11 -07:00
Jeff Hodges 24217f1cba run tests in CI with the race detector on
The race detector has found at least one race in our current code. See
issue #465. Turn it on for the unit and integration tests running in
TravisCI.

Also, allow the local user to add new test flags with the `GOTESTFLAGS`
environment variable.

To ease speed of debugging issues, the ability to skip the unit or
integration tests is also provided.
2015-07-16 09:43:11 -07:00
Jakub Warmuz 47e636d6ee
Merge remote-tracking branch 'github/letsencrypt/master' into le-pkgs_sep_prep
Conflicts:
	test.sh
2015-07-14 22:16:04 +00:00
Jeff Hodges c110468366 harden test.sh and amqp-integration-test.py
amqp-integration-test.py gains a way to print out what processes
failed to start.

test.sh gains:

  * the ability to continue the build correctly if
LETSENCRYPT_PATH was provided but does not exist on disk.
  * an explanatory exit message if the LETSENCRYPT_PATH does
exist, but there is not finished build in it.
  * a working detection for a python 2.7 binary on OS X
2015-07-13 17:02:25 -07:00
Jakub Warmuz 247d18a390
Fix hyphens in letsencrypt install command.
Corresponds to https://github.com/letsencrypt/letsencrypt/pull/600.
2015-07-11 05:39:18 +00:00
James 'J.C.' Jones cdc59892df Merge pull request #452 from jmhodges/add_url
add TravisCI url to fancy github statuses
2015-07-10 18:18:27 -07:00
Jakub Warmuz 95503a058d
Update installation command for letsencrypt client.
Corresponds to https://github.com/letsencrypt/letsencrypt/pull/598
2015-07-10 22:25:29 +00:00
Jeff Hodges dee57814cf add TravisCI url to fancy github statuses
This will create the little "Details" links on status update that links
to the build failure.
2015-07-10 14:13:10 -07:00
Jeff Hodges efbc518cbc do not update PR if coming from non-maintainer
The github-secret.json file can't be decrypted if the PR is coming from
someone who is not a maintainer on boulder. So, just use the boring old
status updates from TravisCI and let the tests continue to run.
2015-07-09 19:01:11 -07:00
J.C. Jones d181d988f4 Updates per review comments 2015-07-08 10:51:42 -07:00
J.C. Jones 2a649b4ab1 Rework per @rolandshoemaker & regenerated secret 2015-07-02 08:56:55 -07:00
J.C. Jones dcde7936ec Switch to golang-implementation of `github-pr-status`. 2015-07-01 16:55:23 -07:00
J.C. Jones 3f80b9a7f6 Use the new npm 0.4.0 owner form 2015-07-01 16:27:21 -07:00
J.C. Jones 7f207ff5c9 Only comment if exit status is non-0 2015-06-30 16:24:38 -07:00
J.C. Jones 4d3731ba73 Issue #419 - Provide more details from Travis on PRs.
This uses a node.js module to post `status` updates to Github, and uses a Travis
secret to authenticate.

- Post comments from static analysis tools
- Change to posting from LetsEncryptBot
- For integration testing, only fail if the compile fails, or
  the NodeJS-client fails. Log if the Python client fails.
2015-06-30 15:32:50 -07:00
Jakub Warmuz f8b3750c02
Run letsencrypt/tests/boulder-integration.sh from integration tests. 2015-06-26 06:32:10 +00:00
Jacob Hoffman-Andrews d65a4611aa Exit if git clone or pip install fail. 2015-06-19 15:22:42 -07:00
Jacob Hoffman-Andrews ee4626aa0d Integration test runs without root. 2015-06-19 14:12:23 -07:00
William Budington 76d76d33cd Adding integration tests.
Travis:
  * Downloads the Let's Encrypt client
  * Installs system requirements for client
  * Sets up virtualenv

Dockerfile:
  * Buildout for development
  * Includes numerous pacakges needed for integration testing
    (including all of the above in Travis)

test.sh:
  * If no path is defined for the LE client
    * Download the Let's Encrypt client
    * Set up virtualenv

test/amqp-integration-test.py:
  * Run client test with sensible defaults
  * One test: auth for foo.com
2015-06-19 12:18:23 -07:00
Jacob Hoffman-Andrews 625eab5ad3 Move boulder to using local signer.
This allows us to use the same PKCS#11 key for both cert signing and OCSP
signing, and simplifies config and startup.

This also starts building with -tags pkcs11 in all scripts, which is required
now that the CA can choose between pkcs11 and non-pkcs11.

In order to successfully issue using a pkcs11 key, you'll need to run a version
of Go built off the master branch. The released versions are missing this
commit:
fe40cdd756,
which is necessary for PKCS#11 signing.
2015-06-05 08:13:39 -07:00
Roland Shoemaker af3abc97f6 add amqp-integration-test.py and use it in test.sh, also tell travis to give us RabbitMQ 2015-06-02 12:02:05 -07:00
J.C. Jones 1fd691564d Fix broken test, only run integration tests if unit tests pass. 2015-05-28 08:35:13 -07:00
Jacob Hoffman-Andrews fc71c2540d Replace integration test with python version.
This allows better shutdown behavior.
2015-05-26 14:11:54 -07:00
Jacob Hoffman-Andrews dc02fedc51 Don't auto-run integration test. 2015-05-22 10:30:26 -07:00
Jacob Hoffman-Andrews c1f3791b27 Fix some issues brought up by integration test 2015-05-21 17:25:56 -07:00
Jacob Hoffman-Andrews 153234204d Add an integration test using test.js
Include updates to test.js to make its output more useful as a diagnostic.

It remains a future TODO to do integration testing with the real letsencrypt
client.

Also, work around a go vet bug.
2015-05-21 16:51:40 -07:00
J.C. Jones 39a61774e7 Enforce `go fmt` is clean in the test run.
Adds a routine to test.sh to ensure committers have run `go fmt` before submission.
2015-05-18 18:42:04 -07:00
J.C. Jones ed32332204 First pass at build speedups. Feel free to use `make -j9` on your desktops! 2015-05-14 11:37:23 -07:00
Jacob Hoffman-Andrews 3eed9e3f7c Move to Square's go-jose library. 2015-05-13 17:36:38 -07:00
J.C. Jones 15808a9a6f Speed up test runs by pre-compiling sqlite3. Kudos to @jmhodges for the tip! 2015-05-01 15:02:10 -07:00
Jacob Hoffman-Andrews 9121719a49 Ignore goveralls submission failures. 2015-04-27 15:54:23 -07:00
J.C. Jones 9404753bb9 Don't print errors if realpath isn't defined. r=@jsha 2015-04-08 13:24:25 -07:00
Jacob Hoffman-Andrews d0d3994b86 Fix test.sh for running on boulder. 2015-03-26 16:35:42 -07:00
Jacob Hoffman-Andrews 9a628b7b16 Allow running test.sh from a symlinked dir. 2015-03-25 18:56:10 -07:00
J.C. Jones ccaac03eb3 Dockerfile update & don't bail out of tests early
- Also added "policy" to the test list. It got forgotten.
2015-03-25 17:11:16 -07:00
Jacob Hoffman-Andrews ddc0d7dcf5 Vendorize all dependencies with `nut`.
Nut, https://github.com/jingweno/nut, is a tool to manage Go dependencies and
versioning by vendorizing them, i.e. including them in your own repo.

This makes version management easier, as well as authenticating the contents of
this repository.

Also inthis change: Factor out the testing commands from .travis.yml to make it
easier to run them by hand. Add Vim swap files to .gitignore.
2015-03-25 15:11:40 -07:00