Commit Graph

51 Commits

Author SHA1 Message Date
Jacob Hoffman-Andrews 5dd212dd47 Add ConfigSecret.
This allows secret values to be stored externally to the config file, so that
config files can be easily shared without revealing secret data.
2015-11-19 22:41:27 -08:00
Jacob Hoffman-Andrews 5fb7be64b0 Make ServiceQueue a separate config param.
Also, make clientName strings into constants.
2015-11-18 17:40:45 -08:00
Jacob Hoffman-Andrews 7dcfcd7864 Add configurable RPC timeouts per backend.
In the process, break out AMQP config into its own struct, one per service.
The AMQPConfig struct is included by composition in the config structs that need
it. If any given service lacks an AMQP config of its own, it gets a default
value from the top-level AMQP config struct, for deployability reasons.

Tightens the RPC code to take a specific AMQP config, not an over-broad
cmd.Config.

Shortens construction of specific RPC clients so they instatiate the generic
client connection themselves, simplifying per-service startup code.

Remove unused SetTimeout method on RPC clients.
2015-11-17 19:51:51 -08:00
Jacob Hoffman-Andrews 2fc0f3143e Improve logging.
Consolidate initialization of stats and logging from each main.go into cmd
package.

Define a new config parameter, `StdoutLevel`, that determines the maximum log
level that will be printed to stdout. It can be set to 6 to inhibit debug
messages, or 0 to print only emergency messages, or -1 to print no messages at
all.

Remove the existing config parameter `Tag`. Instead, choose the tag from the
basename of the currently running process. Previously all Boulder log messages
had the tag "boulder", but now they will be differentiated by process, like
"boulder-wfe".

Shorten the date format used in stdout logging, and add the current binary's
basename.

Consolidate setup function in audit-logger_test.go.

Note: Most CLI binaries now get their stats and logging from the parameters of
Action. However, a few of our binaries don't use our custom AppShell, and
instead use codegangsta/cli directly. For those binaries, we export the new
StatsAndLogging method from cmd.

Fixes https://github.com/letsencrypt/boulder/issues/852
2015-11-11 16:52:42 -08:00
Roland Shoemaker 7675f33317 Add a Akamai CCU client and use it to purge OCSP responses on revocation and update
Adds a (currently gated) Akamai CCU API client used to purge GET OCSP responses
from the CDN. It also contains a small tool (cmd/akamai-purger) that can be used
to purge ARLs from the command line.
2015-10-27 21:45:25 -07:00
Jacob Hoffman-Andrews 194e421931 Add reconnects in AMQP. 2015-10-27 19:54:54 -07:00
Roland Shoemaker 661476f40e Backoff OCSP Updater on HSM failure
If a ServiceUnavailableError is returned from GenerateOCSP backoff before
attempting to retry the call as to not to overwhelm the CA with calls that
may instantly fail.
2015-10-26 14:06:32 -07:00
Roland Shoemaker a9865b07a0 Actually assign publisher client to updater struct 2015-10-26 12:26:47 -07:00
Roland Shoemaker ea73b4afee Fix status guard to just use status.Status, i.e. the status that was set when it was pulled from the DB 2015-10-19 15:24:44 -07:00
Roland Shoemaker fe25336924 Gate revoked certificate loop on non-zero config values 2015-10-13 16:04:11 -07:00
Roland Shoemaker 04ccd264d6 Log loop failure to start 2015-10-13 14:59:29 -07:00
Roland Shoemaker af75497397 Prevent loops from running if they are missing a param 2015-10-13 14:24:11 -07:00
Roland Shoemaker 0bb3466954 More OCSP-Updater tests 2015-10-12 16:47:10 -07:00
Roland Shoemaker 1164b037f3 Add storeResponse status guard test 2015-10-12 13:43:28 -07:00
Roland Shoemaker 54a79fa640 Review fixes pt. 1 2015-10-12 13:19:50 -07:00
Roland Shoemaker 2b604eef6e Review fixes pt. 2 2015-10-09 18:02:02 -07:00
Roland Shoemaker 8d1ea7291f Address review comments
OCSP-Responder attempts to read the OCSP response from the certificateStatus table,
if it cannot find a response there it reads the ocspResponses table to try to find a
response, if neither contains a response the not found bool is passed back to the
Responder.
2015-10-09 15:48:09 -07:00
Roland Shoemaker 10b6bb5548 Refactor certificate revocation and OCSP generation workflows
* Moves revocation from the CA to the OCSP-Updater, the RA will mark certificates as
  revoked then wait for the OCSP-Updater to create a new (final) revoked response
* Merges the ocspResponses table with the certificateStatus table and only use UPDATES
  to update the OCSP response (vs INSERT-only since this happens quite often and will
  lead to an extremely large table)
2015-10-08 18:55:11 -07:00
Roland Shoemaker 80e455f3de Revert 9ff2696 2015-10-05 13:12:03 -07:00
Roland Shoemaker 9ff26960dd COUNT and batch SELECTs so each tick all serials are retrieved 2015-10-04 21:40:06 -07:00
Roland Shoemaker 414e49921b Review fixes 2015-10-02 15:09:05 -07:00
Roland Shoemaker 356a2525f3 Add CT backfill loop to OCSP-Updater 2015-10-02 12:24:23 -07:00
Roland Shoemaker 2d0dee4ce1 Daemonize the OCSP updater tool so we are constantly updating OCSP responses.
also moves the first OCSP responses generation from the CA to the OCSP updater. This patch lays the
ground work for moving CT submission and adding CT backfill to the OCSP updater.
2015-10-01 16:36:51 -07:00
Jacob Hoffman-Andrews e97880aaa7 Audit log version info as early as possible.
This means after parsing the config file, setting up stats, and dialing the
syslogger. But it is still before trying to initialize the given server. This
means that we are more likely to get version numbers logged for some common
runtime failures.
2015-09-29 17:16:03 -07:00
Roland Shoemaker a4fbcdd65a Update naming conventions 2015-09-10 13:10:51 -07:00
Roland Shoemaker 00905ac07a Move RPCMonitor log to the RPCClient and do the collect natively 2015-09-10 12:48:35 -07:00
Roland Shoemaker d6efd496fa Merge master 2015-08-24 12:27:58 -07:00
Roland Shoemaker 7ad8847ba2 Standardize CLI UX 2015-08-23 23:17:14 -07:00
Roland Shoemaker 00307fd2ef Various standardizations and README updates
Update README.md

Finish readme fixes
2015-08-15 22:57:39 -07:00
Jeff Hodges 5e97aa23c6 use MariaDB in the unit tests
And delete the uses of sqlite3
2015-08-14 17:13:15 -07:00
Roland Shoemaker 5ac7652ef4 Major rewrite 2015-08-04 16:23:49 -07:00
J.C. Jones 4a84484912 Remove the artificial OCSP Updater limit check
Use whatever comes from the administrator. Adminstrator knows what she is doing.
2015-07-22 14:33:05 -07:00
Jeff Hodges ef54dda46a add debug http server to services
Currently, the debug http server in every service contains just the
net/http/pprof handlers. This allows us to get CPU, blocking, and memory
profiling remotely.

Along the way, remove all the places we use http.DefaultServeMux (which
includes use of http.Handle and http.HandlerFunc) and use a NewServeMux
for each place.

Fixes #457
2015-07-14 01:28:18 -07:00
Jeff Hodges 79d9b52ad0 rename dbName to dbConnect
The configuration is actually a connection string, not a database name,
and it was a bit confusing.
2015-07-10 16:17:27 -07:00
J.C. Jones 2afbcd5b44 Rework per roland 2015-07-01 09:58:40 -07:00
J.C. Jones 5845f5ff00 Don't stop the OCSP updater if there's one error, and log consistently.
- Log loop and aggregate duration timing
- Log count
2015-06-26 11:03:14 -07:00
J.C. Jones f95e9eaa83 Support TLS mutual authentication for AMQPS 2015-06-18 16:23:48 -07:00
J.C. Jones 41f5788c77 Correct most `go lint` warnings. (274 -> 5) 2015-06-16 22:18:28 -05:00
J.C. Jones ed63a524bf Bugfix: Make OCSP Response lifespans configurable. 2015-06-08 14:19:15 -07:00
J.C. Jones c21c6d0bdf Issue #249 - Refactor an interface for the RPC Client and RPC Server
- Added the skeleton of the first RPC wrapper test
- Does not close Issue #249; still need to implement a channel-RPC mechanism
2015-06-04 11:43:48 -07:00
Richard Barnes e8edbf5f21 Making capitalization consistent with Go standards 2015-06-01 02:08:47 -04:00
James 'J.C.' Jones 3e593d73c9 Merge pull request #262 from letsencrypt/ra-tests
Miscellaneous Fixes
2015-05-30 22:08:49 -07:00
Richard Barnes 7a09c78788 Issue #254 2015-05-30 13:21:36 -04:00
J.C. Jones 4518f0bf17 Migrate CADB to using GORP. 2015-05-28 23:11:03 -07:00
J.C. Jones 1008bd824d WIP on issue #202: OCSP Responder work 2015-05-28 23:11:03 -07:00
J.C. Jones e14f59c559 Rework from PR #251: 2015-05-28 23:03:48 -07:00
Jacob Hoffman-Andrews 6a60b4e4b0 Merge pull request #250 from letsencrypt/239-buildid
Issue #239 - Add a build ID method to WFE, and print Info on startup …
2015-05-28 16:08:20 -07:00
J.C. Jones 452d2bcc29 Fix copyright in OCSP Updater and Responder 2015-05-28 15:02:49 -07:00
J.C. Jones 8766edaa93 Issue #239 - Add a build ID method to WFE, and print Info on startup for all 2015-05-28 11:13:09 -07:00
J.C. Jones af0f8446eb Issue #202, Periodic OCSP Signer Tool
- Move dbMap construction and type converter into individual files in the sa package.
- Add DB configuration for the OCSP tool to the boulder config:
  - left to the user if they want to use different boulder-config.json files
    for different purposes.
- Added updater to Makefile
- Fix trailing ',' in the Boulder config, add more panic logging
- Ignore .pem files produced by the integration test
- Change RPC to use per-instance named reply-to queues.
- Finish OCSP Updater logic
- Rework RPC for OCSP to use a transfer object (due to serialization problems of x509.Certificate)
2015-05-27 22:01:29 -07:00