Commit Graph

34 Commits

Author SHA1 Message Date
Daniel McCarney ecca3492e9 csr: return berrors in VerifyCSR. (#4473)
This also adds the badCSR error type specified by RFC 8555. It is a natural fit for the errors in VerifyCSR that aren't covered by badPublicKey. The web package function for converting a berror to
a problem is updated for the new badCSR error type.

The callers (RA and CA) are updated to return the berrors from VerifyCSR as is instead of unconditionally wrapping them as a berrors.MalformedError instance. Unit/integration tests are updated accordingly.

Resolves #4418
2019-10-09 17:11:11 -07:00
Roland Bracewell Shoemaker 31ed590edd
Strip default scheme ports from Host headers (#4448)
Fixes #4447.
2019-09-27 16:14:40 -07:00
Daniel McCarney f02e9da38f
Support admin. blocking public keys. (#4419)
We occasionally have reason to block public keys from being used in CSRs
or for JWKs. This work adds support for loading a YAML blocked keys list
to the WFE, the RA and the CA (all the components already using the
`goodekey` package).

The list is loaded in-memory and is intended to be used sparingly and
not for more complicated mass blocking scenarios. This augments the
existing debian weak key checking which is specific to RSA keys and
operates on a truncated hash of the key modulus. In comparison the
admin. blocked keys are identified by the Base64 encoding of a SHA256
hash over the DER encoding of the public key expressed as a PKIX subject
public key. For ECDSA keys in particular we believe a more thorough
solution would have to consider inverted curve points but to start we're
calling this approach "Good Enough".

A utility program (`block-a-key`) is provided that can read a PEM
formatted x509 certificate or a JSON formatted JWK and emit lines to be
added to the blocked keys YAML to block the related public key.

A test blocked keys YAML file is included
(`test/example-blocked-keys.yml`), initially populated with a few of the
keys from the `test/` directory. We may want to do a more through pass
through Boulder's source code and add a block entry for every test
private key.

Resolves https://github.com/letsencrypt/boulder/issues/4404
2019-09-06 16:54:26 -04:00
Roland Bracewell Shoemaker b905691211 probs: de-capitalize identifier field in sub problems (#4424) 2019-09-06 09:35:17 -04:00
Jacob Hoffman-Andrews c777dfece6 Log the Origin header. (#4376)
XHR requests from web-based ACME clients provide the User-Agent
of the browser that initiated the request, but the hostname of the site
that originated the request is sent in the Origin header. This will let
us better analyze web-based ACME traffic.

Fixes #4370
2019-07-31 09:47:44 -07:00
Roland Bracewell Shoemaker 52dd3bd9c7 web: Log subproblems in RequestEvent (#4363) 2019-07-26 14:02:18 -04:00
Roland Bracewell Shoemaker c7debd51b9
Set namespace for sub-problems (#4361)
Fixes #4355.
2019-07-25 10:26:58 -07:00
Roland Bracewell Shoemaker a1540bd5ec web: log 200 if code not explicitly set (#4296) 2019-06-25 16:59:30 -04:00
Roland Bracewell Shoemaker 6f93942a04 Consistently used stdlib context package (#4229) 2019-05-28 14:36:16 -04:00
Daniel McCarney 6f50b2424c
web: support converting suberrors to subproblems. (#4233) 2019-05-28 14:09:02 -04:00
Daniel McCarney 204ff0446f
RA: forbid finalizing pending orders. (#4108)
This change was made on the WFE2 side but I missed making it in the RA.
2019-03-12 15:09:35 -04:00
Jacob Hoffman-Andrews 958cef1d1f Add exception for grpc timeouts in WFE. (#4014)
In https://github.com/letsencrypt/boulder/pull/3985/files we added an
exception where deadline exceeded errors in WFE wouldn't cause audit
logs, but we forgot to include gRPC responses where the code was
DeadlineExceeded.
2019-01-17 09:48:40 -05:00
Jacob Hoffman-Andrews 1563419cfb
Shrink byte size of WFE request logs. (#3997)
- Log the simple, non-whitespace-containing fields as positional
parameters to avoid the JSON overhead for them.
- Log latency in milliseconds rather than seconds (saves "0.").
- Hoist some fields from the "Extra" sub-object and give
  them shorter names. This saves the bytes for rendering the "Extra"
  field plus the bytes for the longer names.

Example output from integration tests:

Before (1687 bytes):

I205230 boulder-wfe JSON={"Endpoint":"/directory","Method":"GET","UserAgent":"Boulder integration tester","Latency":0.001,"Code":0}
I205230 boulder-wfe JSON={"Endpoint":"/acme/new-reg","Method":"HEAD","Error":"405 :: malformed :: Method not allowed","UserAgent":"Boulder integration tester","Latency":0,"Code":405}
I205230 boulder-wfe JSON={"Endpoint":"/acme/new-reg","Method":"POST","Requester":611,"Contacts":[],"UserAgent":"Boulder integration tester","Latency":0.025,"Code":201,"Payload":"{\n  \"resource\": \"new-reg\"\n}"}
I205230 boulder-wfe JSON={"Endpoint":"/acme/reg/","Slug":"611","Method":"POST","Requester":611,"Contacts":[],"UserAgent":"Boulder integration tester","Latency":0.021,"Code":202,"Payload":"{\n  \"status\": \"valid\", \n  \"resource\": \"reg\", \n  \"agreement\": \"http://boulder:4000/terms/v1\", \n  \"key\": {\n    \"e\": \"AQAB\", \n    \"kty\": \"RSA\", \n    \"n\": \"r1zCJC8Muw5K8ti-pjojivHxyNxOZye-N5aX_i7kBiHrAOp9qxgQUHUyU3COCjFPrSzScTpKoIyCwdL7x-1mPX3pby7CzGugtY9da_LZkDmsDE8LIuQkZ_wRLyh1103OQZEd71AlddMx1iwLLVl4UTICoJFUfYvXHvkqmsE5xhBPJhl-SdSrJM6F7Kn7k0WycA5ig_QPbjVbzJlQq-C65iGDJtc_LvY0FFF4exThZM7xsvucJywJMHCEWZUktm9YB-CBNA1gVbL52u22jQpX-MN52UVdqSh9ZipoJLtxKjZx31DHB_bcdgtJ8YGIE4lY_ZAax1Ut-a5WTJvVq2Hk8w\"\n  }\n}"}
I205230 boulder-wfe JSON={"Endpoint":"/acme/new-authz","Method":"POST","Requester":611,"Contacts":[],"UserAgent":"Boulder integration tester","Latency":0.031,"Code":201,"Payload":"{\n  \"identifier\": {\n    \"type\": \"dns\", \n    \"value\": \"rand.18fe4d73.xyz\"\n  }, \n  \"resource\": \"new-authz\"\n}","Extra":{"AuthzID":"PgF1JQ3TK6c1FR0wVdm_mYows_xWSsyYgyezSvSNI-0","Identifier":{"type":"dns","value":"rand.18fe4d73.xyz"}}}

After (1406 bytes):

I210117 boulder-wfe GET /directory 0 0 0 0.0.0.0 JSON={"ua":"Boulder integration tester"}
I210117 boulder-wfe HEAD /acme/new-reg 0 405 0 0.0.0.0 JSON={"Error":"405 :: malformed :: Method not allowed","ua":"Boulder integration tester"}
I210117 boulder-wfe POST /acme/new-reg 676 201 23 0.0.0.0 JSON={"Contacts":[],"ua":"Boulder integration tester","Payload":"{\n  \"resource\": \"new-reg\"\n}"}
I210117 boulder-wfe POST /acme/reg/ 676 202 23 0.0.0.0 JSON={"Slug":"676","Contacts":[],"ua":"Boulder integration tester","Payload":"{\n  \"status\": \"valid\", \n  \"resource\": \"reg\", \n  \"agreement\": \"http://boulder:4000/terms/v1\", \n  \"key\": {\n    \"e\": \"AQAB\", \n    \"kty\": \"RSA\", \n    \"n\": \"zXSFAzdzwwFGjNysmG0YE7MxAwQ8JkkvLQ7Qs7xB1h5kFM_F-W2jxYEmrRTrA0ylfuzb4RQMBrsLfv0XV8rsDIuP_t92ADBjfd25ajuuia9EGrhpHitFimEUlZjsqGQp8F49xLhDMAqm1SLBY_k1pY8TKSLHeyOyLYIKLaL3Ra9yZ63qB65oGuNhXroKqqx7nUjyZtqtUV5NUPvPgvhJgXgYKMjck3jXWgr4ZGqYyJQqNqydYSk3uJGfruChakZThwl3vbH8aUPaeoXcvPA8KaQl56JUf7jAVY3n9qKKb5mgT96vDKWUpJaI5YE1rMZIJfkaFK-ZZIhFeeKCSsSJlQ\"\n  }\n}"}
I210117 boulder-wfe POST /acme/new-authz 676 201 35 0.0.0.0 JSON={"Contacts":[],"ua":"Boulder integration tester","Payload":"{\n  \"identifier\": {\n    \"type\": \"dns\", \n    \"value\": \"rand.14ebdfd1.xyz\"\n  }, \n  \"resource\": \"new-authz\"\n}","Created":"Z-soxIEhsGlMK3GYyDqYrSlxDFEeH6q3mrd6aoi2iIs","DNSName":"rand.14ebdfd1.xyz"}
2019-01-10 09:24:06 -08:00
Jacob Hoffman-Andrews ee337cd5d1 Update wfe2. 2019-01-09 15:19:04 -08:00
Jacob Hoffman-Andrews a7da3fc58c Update tests. 2019-01-09 15:00:58 -08:00
Jacob Hoffman-Andrews 0123b35295 Make Contacts optional in logs. 2019-01-09 14:07:08 -08:00
Jacob Hoffman-Andrews d2d5ba294b Shrink byte size of WFE request logs.
- Log the simple, non-whitespace-containing fields as positional
parameters to avoid the JSON overhead for them.
- Log latency in milliseconds rather than seconds (saves "0.").
- Hoist some fields from the "Extra" sub-object and give
  them shorter names. This saves the bytes for rendering the "Extra"
  field plus the bytes for the longer names.

Example output from integration tests:

Before (1687 bytes):

I205230 boulder-wfe JSON={"Endpoint":"/directory","Method":"GET","UserAgent":"Boulder integration tester","Latency":0.001,"Code":0}
I205230 boulder-wfe JSON={"Endpoint":"/acme/new-reg","Method":"HEAD","Error":"405 :: malformed :: Method not allowed","UserAgent":"Boulder integration tester","Latency":0,"Code":405}
I205230 boulder-wfe JSON={"Endpoint":"/acme/new-reg","Method":"POST","Requester":611,"Contacts":[],"UserAgent":"Boulder integration tester","Latency":0.025,"Code":201,"Payload":"{\n  \"resource\": \"new-reg\"\n}"}
I205230 boulder-wfe JSON={"Endpoint":"/acme/reg/","Slug":"611","Method":"POST","Requester":611,"Contacts":[],"UserAgent":"Boulder integration tester","Latency":0.021,"Code":202,"Payload":"{\n  \"status\": \"valid\", \n  \"resource\": \"reg\", \n  \"agreement\": \"http://boulder:4000/terms/v1\", \n  \"key\": {\n    \"e\": \"AQAB\", \n    \"kty\": \"RSA\", \n    \"n\": \"r1zCJC8Muw5K8ti-pjojivHxyNxOZye-N5aX_i7kBiHrAOp9qxgQUHUyU3COCjFPrSzScTpKoIyCwdL7x-1mPX3pby7CzGugtY9da_LZkDmsDE8LIuQkZ_wRLyh1103OQZEd71AlddMx1iwLLVl4UTICoJFUfYvXHvkqmsE5xhBPJhl-SdSrJM6F7Kn7k0WycA5ig_QPbjVbzJlQq-C65iGDJtc_LvY0FFF4exThZM7xsvucJywJMHCEWZUktm9YB-CBNA1gVbL52u22jQpX-MN52UVdqSh9ZipoJLtxKjZx31DHB_bcdgtJ8YGIE4lY_ZAax1Ut-a5WTJvVq2Hk8w\"\n  }\n}"}
I205230 boulder-wfe JSON={"Endpoint":"/acme/new-authz","Method":"POST","Requester":611,"Contacts":[],"UserAgent":"Boulder integration tester","Latency":0.031,"Code":201,"Payload":"{\n  \"identifier\": {\n    \"type\": \"dns\", \n    \"value\": \"rand.18fe4d73.xyz\"\n  }, \n  \"resource\": \"new-authz\"\n}","Extra":{"AuthzID":"PgF1JQ3TK6c1FR0wVdm_mYows_xWSsyYgyezSvSNI-0","Identifier":{"type":"dns","value":"rand.18fe4d73.xyz"}}}

After (1406 bytes):

I210117 boulder-wfe GET /directory 0 0 0 0.0.0.0 JSON={"ua":"Boulder integration tester"}
I210117 boulder-wfe HEAD /acme/new-reg 0 405 0 0.0.0.0 JSON={"Error":"405 :: malformed :: Method not allowed","ua":"Boulder integration tester"}
I210117 boulder-wfe POST /acme/new-reg 676 201 23 0.0.0.0 JSON={"Contacts":[],"ua":"Boulder integration tester","Payload":"{\n  \"resource\": \"new-reg\"\n}"}
I210117 boulder-wfe POST /acme/reg/ 676 202 23 0.0.0.0 JSON={"Slug":"676","Contacts":[],"ua":"Boulder integration tester","Payload":"{\n  \"status\": \"valid\", \n  \"resource\": \"reg\", \n  \"agreement\": \"http://boulder:4000/terms/v1\", \n  \"key\": {\n    \"e\": \"AQAB\", \n    \"kty\": \"RSA\", \n    \"n\": \"zXSFAzdzwwFGjNysmG0YE7MxAwQ8JkkvLQ7Qs7xB1h5kFM_F-W2jxYEmrRTrA0ylfuzb4RQMBrsLfv0XV8rsDIuP_t92ADBjfd25ajuuia9EGrhpHitFimEUlZjsqGQp8F49xLhDMAqm1SLBY_k1pY8TKSLHeyOyLYIKLaL3Ra9yZ63qB65oGuNhXroKqqx7nUjyZtqtUV5NUPvPgvhJgXgYKMjck3jXWgr4ZGqYyJQqNqydYSk3uJGfruChakZThwl3vbH8aUPaeoXcvPA8KaQl56JUf7jAVY3n9qKKb5mgT96vDKWUpJaI5YE1rMZIJfkaFK-ZZIhFeeKCSsSJlQ\"\n  }\n}"}
I210117 boulder-wfe POST /acme/new-authz 676 201 35 0.0.0.0 JSON={"Contacts":[],"ua":"Boulder integration tester","Payload":"{\n  \"identifier\": {\n    \"type\": \"dns\", \n    \"value\": \"rand.14ebdfd1.xyz\"\n  }, \n  \"resource\": \"new-authz\"\n}","Created":"Z-soxIEhsGlMK3GYyDqYrSlxDFEeH6q3mrd6aoi2iIs","DNSName":"rand.14ebdfd1.xyz"}
2019-01-09 13:03:07 -08:00
Jacob Hoffman-Andrews 33b6f1c01f For deadline exceeded in WFE, skip audit log. (#3985) 2018-12-18 10:11:44 -08:00
Jacob Hoffman-Andrews 38eaeef202 Log simpler format of WFE errors. (#3974)
In #3452 we refactored error logging a little bit. In the process we
added a %#v format for internal errors. In theory this would help us
debug strange error cases better. In practice the additional type
information is almost always "&errors.BoulderError{" or
"&status.statusError{". The latter includes confusing protobuf-internal
fields like XXX_unrecognized:[]uint8(nil).

Switching to %s should simplify the logged error and make it easier to
get to the core of what's wrong.
2018-12-10 10:03:44 -08:00
Jacob Hoffman-Andrews b1be4ccaed Fix latency logging. (#3937)
In the VA, we were rendering a Duration to JSON, which gave an integer
number of nanoseconds rather than a float64 of seconds. Also, in both VA
and WFE we were rendering way more precision than we needed. Millisecond
precision is enough, and since we log latency for every WFE response,
the extra bytes are worth saving.
2018-11-14 10:52:48 -05:00
Joel Sing 9990d14654 Convert the probs functions to be formatters. (#3708)
Many of the probs.XYZ calls are of the form probs.XYZ(fmt.Sprintf(...)).
Convert these functions to take a format string and optional arguments,
following the same pattern used in the errors package. Convert the
various call sites to remove the now redundant fmt.Sprintf calls.
2018-05-11 11:51:16 -07:00
Joel Sing 8ebdfc60b6 Provide formatting logger functions. (#3699)
A very large number of the logger calls are of the form log.Function(fmt.Sprintf(...)).
Rather than sprinkling fmt.Sprintf at every logger call site, provide formatting versions
of the logger functions and call these directly with the format and arguments.

While here remove some unnecessary trailing newlines and calls to String/Error.
2018-05-10 11:06:29 -07:00
Roland Bracewell Shoemaker 9821aeb46f Split internal and public errors out in web.RequestEvent (#3682)
Splits out the old `Errors` slice into a public `Error` string and a `InternalErrors` slice. Also removes a number of occurrences of calling `logEvent.AddError` then immediately calling `wfe.sendError` with either the same internal error which caused the same error to be logged twice or no error which is slightly redundant as `wfe.sendError` calls `logEvent.AddError` internally.

Fixes #3664.
2018-05-03 09:13:33 -04:00
Roland Bracewell Shoemaker c3669f9068 Split endpoint and path in WFE+WFE2 web.RequestEvent (#3683) 2018-05-02 10:20:21 -07:00
Jacob Hoffman-Andrews 9e24cad3bb Add latency logging to WFE and WFE2. (#3617)
Fixes #3609.
2018-04-04 21:02:49 +01:00
Daniel McCarney 866627ee29 Return descriptive error when SCTs policy can't be met. (#3586)
This commit updates CTPolicy & the RA to return a distinct error when
the RA is unable to fetch the required SCTs for a certificate when
processing an issuance. This error type is plumbed up to the WFE/WFE2
where the `web/probs.go` code converts it into a server internal error
with a suitable user facing error.
2018-03-22 13:10:08 -07:00
Jacob Hoffman-Andrews d654675223 Remove BaseURL from WFE config. (#3540)
For a long time now the WFE has generated URLs based on the incoming
request rather than a hardcoded BaseURL. BaseURL is no longer set in the
prod configs.

This also allows factoring out relativeEndpoint into the web package.
2018-03-09 11:04:02 +00:00
Jacob Hoffman-Andrews eb23cb3ffc Remove "Terminated request" / "Successful request" (#3484)
The WFE logs these with every request, but with #3483,
they aren't necessary; everything other than 2xx is a failed request.
2018-02-28 15:16:36 -08:00
Jacob Hoffman-Andrews bf5dc8b929 Log status code in WFE JSON logs. (#3483)
This field was introduced to the logs in #2628 but without the code required
to fill it.
2018-02-27 10:16:45 -05:00
Jacob Hoffman-Andrews c628fc0119 Improve error reporting in WFE and WFE2 (#3452)
* Factor out sendError into web.SendError.
* In web.SendError, ensure that ierr always gets added to the list of errors logged.
2018-02-20 14:30:36 -08:00
Jacob Hoffman-Andrews c0ffa3d5d1 Remove logging of Request/ResponseNonce. (#3421)
These take up a lot of space in the logs, and we almost never reference
them.
2018-02-06 10:17:12 -05:00
Jacob Hoffman-Andrews e60251d8ca Add documentation link to rate limit errors. (#3286)
Resolves #2951
2017-12-19 15:46:18 -08:00
Jacob Hoffman-Andrews 97265c9184 Factor out context.go from wfe and wfe2. (#3086)
* Move probs.go to web.

* Move probs_test.go

* Factor out probs.go from wfe

* Move context.go

* Extract context.go into web package.

* Add a constructor for TopHandler.
2017-09-26 13:54:14 -04:00
Jacob Hoffman-Andrews 4f1f5cd689 Factor out probs.go from wfe and wfe2 (#3085)
This is shared code between both packages. Better to have it in a single shared place.

In the process, remove the unexported signatureValidationError, which was unnecessary; all returned errors from checkAlgorithm get turned into Malformed.
2017-09-18 13:08:18 -07:00