Fixes#140.
This patch allows users to specify the following revocation reasons based on my interpretation of the meaning of the codes but could use confirmation from others.
* unspecified (0)
* keyCompromise (1)
* affiliationChanged (3)
* superseded (4)
* cessationOfOperation (5)
Part of #2080.
This change vendors `crypto/x509`, `crypto/x509/pkix`, and `encoding/asn1` from 1d5f6a765d. That commit is a direct child of the Go 1.5.4 release tag, so it contains the same code as the current Go version we are using. In that commit I rewrote imports in those packages so they depend on each other internally rather than calling out to the standard library, which would cause type disagreements.
I changed the imports in each place where we're parsing CSRs, and imported under a different name `oldx509`, both to avoid collisions and make it clear what's going on. Places that only use `x509` to parse certificates are not changed, and will use the current standard library.
This will unblock us from moving to Go 1.6, and subsequently Go 1.7.
This PR removes the Shutdown{Kill,Stop}Timeout fields from `cmd/boulder-wfe/main.go`. These fields are set in `main.go` but never used. The relevant config fields only need to be used when creating an `httpdown.HTTP` in `cmd/boulder-wfe/main.go`.
Existing duration parsing was replaced with `cmd.ConfigDuration` use.
For #2001 an optimization was added to the WFE to avoid invoking the RA's `UpdateRegistration` method when a trivial (e.g. `{"resource:"reg"}`) update is received. Instead the WFE returned the trivial update back to the client immediately.
This is contrary to the ACME spec which indicates:
> Servers SHOULD NOT respond to GET requests for registration resources as these requests are not authenticated. If a client wishes to query the server for information about its account (e.g., to examine the “contact” or “certificates” fields), then it SHOULD do so by sending a POST request with an empty update. That is, it should send a JWS whose payload is trivial ({“resource”:”reg”}).
The optimization regression was captured in issue #2066 when it broke at least one client implementation.
This removes the empty reg update optimization and passes all POST's to the RA. The RA will in turn fetch the existing registration to return to the client. The second half of the #2001 optimizations remains in place, no DB UPDATE's will be performed if the new registration content doesn't differ from the existing registration content (as determine by the return of `registration.MergeUpdate`).
Since the WFE optimization is no longer in place the `FailureRegistrationAuthority` mock isn't required and is removed. Similarly `TestEmptyRegistration` in `wfe_test.go` is changed from testing for the optimization to testing for the ACME described "get registration info" behaviour.
This fixes#2066
Introduces the `authorizationLifetimeDays` and `pendingAuthorizationLifetimeDays` configuration options for `RA`.
If the values are missing from configuration, the code defaults back to the current values (300/7 days).
fixes#2024
This PR adds two optimizations to fix the optimistic lock errors observed in #1986.
First, the WFE now returns early for registration POST's (before invoking the RA and SA) when the POST body is the trivial update (`{"resource":"reg"}`). This prevents any DB operations from being performed when there is no work to be done.
Second, the RA now tracks whether a update actually changes the base registration's `Contact` slice, or `Agreement` string. If the proposed update doesn't change either of these fields then the RA will return early before handing the update to the SA.
Both changes save database operations from being performed needlessly and will help avoid the optimistic lock errors we observed when a problematic client was POSTing the trivial update repeatedly in a short period.
The fix was verified as follows: I checked out master and artificially introduced lock contention into the SA by adding a 2s sleep into `UpdateRegistration` between fetching the `existingRegModel` to get the `LockCol` value and calling `ssa.dbMap.Update`. With the sleep in place & two certbot clients posting matching registration updates the lock contention error is produced as expected. After checking out the `empty-reg-updates` branch, re-adding the sleep to the SA, and performing the same two client reg updates no error is produced.
Adds a test for CSRs generated using a pre-1.0.2 version of OpenSSL and a buggy client which will fail to parse with Golang 1.6+.
This test checks the values of the bytes in the 8th and 9th offsets, which in a properly formatted CSR should be the version integer declaration bytes, and if the malformed values are present will return a error to the user informing them that they are using an old version of OpenSSL and/or a client which doesn't explicitly set the CSR version.
Fixes#1902.
The `regID` parameter in the PA's `WillingToIssue` function was originally used for whitelisting purposes, but is not used any longer. This PR removes it.
The RA UpdateRegistration function merges a base registration object with an update by calling Registration.MergeUpdate. Prior to this commit MergeUpdate only allowed the updated registration object to overwrite the Contact field of the existing registration if the updated reg. defined at least one AcmeURL. This prevented clients from being able to outright remove the contact associated with an existing registration.
This commit removes the len() check on the input.Contact in MergeUpdate to allow the r.Contact field to be overwritten by a []*core.AcmeURL(nil) Contact field. Subsequently clients can now send an empty contacts list in the update registration POST in order to remove their reg contact.
Fixes#1846
* Allow removing registration contact.
* Adds a test for `MergeUpdate` contact removal.
* Change `Registration.Contact` type to `*[]*core.AcmeURL`.
* End validateContacts early for empty contacts
* Test removing reg. contact more thoroughly.
Presently clients may request a new AuthZ be created for a domain that they have already proved authorization over. This results in unnecessary bloat in the authorizations table and duplicated effort.
This commit alters the `NewAuthorization` function of the RA such that before going through the work of creating a new AuthZ it checks whether there already exists a valid AuthZ for the domain/regID that expires in more than 24 hours from the current date. If there is, then we short circuit creation and return the existing AuthZ. When this case occurs the `RA.ReusedValidAuthz` counter is incremented to provide visibility.
Since clients requesting a new AuthZ and getting an AuthZ back expect to turn around and post updates to the corresponding challenges we also return early in `UpdateAuthorization` when asked to update an AuthZ that is already valid. When this case occurs the `RA.ReusedValidAuthzChallenge` counter is incremented.
All of the above behaviour is gated by a new RA config flag `reuseValidAuthz`. In the default case (false) the RA does **not** reuse any AuthZ's and instead maintains the historic behaviour; always creating a new AuthZ when requested, irregardless of whether there are already valid AuthZ's that could be reused. In the true case (enabled only in `boulder-config-next.json`) the AuthZ reuse described above is enabled.
Resolves#1854
Resolves#1810 by automatically updating the RA ratelimit.RateLimitConfig whenever the backing config file is changed. Much like the Policy Authority uses a reloader instance to support updating the Hostname policy on the fly, this PR changes the Registration Authority to use a reloader for the rate limit policy file.
Access to the ra.rlPolicies member is protected with a RWMutex now that there is a potential for the values to be reloaded while a reader is active.
A test is introduced to ensure that writing a new policy YAML to the policy config file results in new values being set in the RA's rlPolicies instance.
https://github.com/letsencrypt/boulder/pull/1894
* WFE returns headers Boulder-Request-ID and Boulder-Requester
* improve test
* add the requestHeader after calls to verifyPOST
* move call of addRequesterHeader in NewRegistration
* move setting of request header to context, improve test
* remove pointless comment
This PR implements `http.StripPrefix` as a wrapper to the existing handler in `wfe.go`. `StripPrefix` is used to remove the path that it expects for each handler. Such as /acme/reg/. The remaining path is used as a slug, since multiple slugs are outside the scope of the specification.
Several tests bypassed the `mux.Handle()` and called the wrapped handler directly. Ex `Registration`. As a result of this, many tests had to be modified to no longer pass in the full path. `request.URL.Path` should now only ever contain the slug (if there is one).
Fixes#437
In this PR, logger is passed to the following callers:
NewWebFrontEndImpl
NewCertificateAuthorityImpl
NewValidationAuthorityImpl
NewAmqpRPCServer
newUpdater
NewRegistrationAuthorityServer
This reduces the usage of a global singleton logger and allows tests to consistently use a mock logger.
Fixes#1642
* remove blog.Get() in wfe
* remove blog.Get() from va
* remove Blog.Get() from ca
* remove blog.Get() from oscp updater, ampq rpc server, registration authority server
* removed some pointless logging code
* remove one added newline
* fix format issue
* fix setup function to return *blog.Mock instead of being passed in
* remove useless blog.NewMock() call
- Run both gRPC and AMQP servers simultaneously
- Take explicit constructor parameters and unexport fields that were previously set by users
- Remove transitional DomainCheck code in RA now that GSB is enabled.
- Remove some leftover UpdateValidation dummy methods.
The rate limiting code previously lived in the `cmd` package without a clear justification for why. This commit moves the rate limiting code to its own `ratelimit` package and updates import paths as required. Notably all references from the `cmd` package's exported `LoadRateLimitPolicies`, `RateLimitPolicy`, and `RateLimitConfig` were moved to use `ratelimit`. This removed the `cmd` import from a couple of callers (nice!).
Prior to this PR the /directory JSON result was built once in Handler() and returned as-is for all requests. Each endpoint URL was fully qualified as an absolute URL using the BaseURL configuration
parameter. This required a configuration change in order to tweak the origin being used for subsequent requests. Returning purely relative URLs (e.g. /acme/new-reg vs http://localhost:4000/acme/new-reg) would break clients that assume absolute paths and we don't want that.
This PR introduces a new behaviour where the /directory JSON is built per-request using the HTTP Host header in place of the BaseURL. Clients will still receive a fully qualified URL in each directory entry but we gain the ability to more easily control the host without requiring config changes. To allow gradual migration via the config file we use the old /directory behaviour when a BaseURL is specified in the configuration file. This will address #1823.
Since the request.URL is not populated (Spare the Path attribute) we can not use request.URL.Scheme for the initial http:// vs https:// prefix when constructing the URLs and instead differentiate between the two cases using the req.TLS attribute. For cases (such as in production) where another service is terminating the initial request and making a subsequent HTTP request to the WFE we support the X-Forwarded-Proto header to ensure we use the original request's protocol when building URLs.
Many unit tests for the WFE assumed that when there is no BaseURL specified and no Host header is sent in the request, that the output will return relative paths. This PR changes that behaviour to always return absolute URLs by defaulting to localhost for the Host when it is not specified via the initial request or the BaseURL config option. This PR changes the expected test output to match this behaviour.
* Split CSR testing and name hoisting into own functions, verify CSR in RA & CA
* Move tests around and various other fixes
* 1.5.3 doesn't have the needed stringer
* Move functions to their own lib
* Remove unused imports
* Move MaxCNLength and BadSignatureAlgorithms to csr package
* Always normalizeCSR in VerifyCSR and de-export it
* Update comments
* Use MarshalIndent in WFE
This makes it easier to read output without first running it through a JSON
prettifier.
* Fix test
* marshal->marshalIndent
https://github.com/letsencrypt/boulder/pull/1811
* Fix all errcheck errors
* Add errcheck to test.sh
* Add a new sa.Rollback method to make handling errors in rollbacks easier.
This also causes a behavior change in the VA. If a HTTP connection is
abruptly closed after serving the headers for a non-200 response, the
reported error will be the read failure instead of the non-200.
- Remove error signatures from log methods. This means fewer places where errcheck will show ignored errors.
- Pull in latest cfssl to be compatible with errorless log messages.
- Reduce the number of message priorities we support to just those we actually use.
- AuditNotice -> AuditInfo
- Remove InfoObject (only one use, switched to Info)
- Remove EmergencyExit and related functions in favor of panic
- Remove SyslogWriter / AuditLogger separate types in favor of a single interface, Logger, that has all the logging methods on it.
- Merge mock log into logger. This allows us to unexport the internals but still override them in the mock.
- Shorten names to be compatible with Go style: New, Set, Get, Logger, NewMock, etc.
- Use a shorter log format for stdout logs.
- Remove "... Starting" log messages. We have better information in the "Versions" message logged at startup.
Motivation: The AuditLogger / SyslogWriter distinction was confusing and exposed internals only necessary for tests. Some components accepted one type and some accepted the other. This made it hard to consistently use mock loggers in tests. Also, the unnecessarily fat interface for AuditLogger made it hard to meaningfully mock out.
Also, when a certificate already exists, treat that as info, not error.
Update mock logger to allow matching by log level, and fix WFE and VA tests
correspondingly.
Also:
- Use MockCA in the RA test instead of a real CA.
- Since the mock CA doesn't write to an SA, remove a part of the RA test that
checked that the certificate was written. That code is already tested in the CA,
where the test belongs.
- Format the constants in RA test to be more copy-and-pasteable.
- Remove Printf in mocks/log.go and test/db.go to make failed test output more readable.
This uses ProblemDetails throughout the wfe. This is the last step in
allowing the backend services to pass ProblemDetails from RPCs through
to the user.
Updates #1153.
Fixes#1161.
This allows us to call the Google Safe Browsing calls through the VA.
If the RA config's boolean UseIsSafeDomain is true, the RA will make the RPC
call to the VA during its NewAuthorization.
If the VA config's GoogleSafeBrowsingConfig struct is not nil, the VA
will check the Google Safe Browsing API in
VA.IsSafeDomain. If the GoogleSafeBrowsingConfig struct is nil, it will
always return true.
In order to actually make requests, the VA's GoogleSafeBrowsingConfig
will need to have a directory on disk it can store the local GSB hashes
it will check first and a working Google API key for the GSB API.
Fixes#1058
This change makes the use of requestEvent more prominent and provides
some nice handler types to make that better.
In order to avoid stomping on Error, requestEvent gains an Errors string
slice.
Fixes#552
Currently, the whitelisted registration ID is one that is impossible for the
database to return. Once the partner's registration is in place, we can
deploy a change to it.
Fixes#810
instead of submitted key. This minimizes the chances of unexpected JWK fields in
the submitted key altering its interpretation without altering the lookup in the
registrations table.
In the process, fix handling of NoSuchRegistration responses.
Fixes https://github.com/letsencrypt/boulder/issues/865.
* Rewrite JSONDuration as ConfigDuration that can handle both JSON and YAML unmarshaling
* Factor out RPC certificate count request struct
* Return 429 to WFE on rate limit exceeded
* Fix wonky RateLimitPolicy comment
To keep the change small, I have not yet completely removed the
GetCertificateByShortSerial method from interfaces and the RPC. I will do taht
in a follow up change.
The WFE test relies on a pre-generated cert. Since there are some sanity checks
on the dates in certs, we were getting errors during the test.
One quick fix is to have those sanity checks rely on RA's clock object, which
can be replaced with a fake for testing. In order to do that, I had to move the
sanity check (MatchesCSR) into the registration authority package, where it
makes more sense anyhow.
I also removed a handful of equality testing functions in objects.go that were
only used by MatchesCSR and whose purpose is better served by reflect.DeepEqual.
This was to avoid having to also move those equality testing functions into the
registration authority.
The previous cert was causing a test failure because it had a NotBefore date too
far in the past.
This is a stopgap fix to fix the build. The real fix, coming soon, will be to
use a fake clock in the WFE test so we can set it to match the date in the test cert.
The HTML reply pointed to the new-reg URL, when it should point to the
directory.
Also fix https://github.com/letsencrypt/boulder/issues/717 by checking first
whether the request path is exactly "/" and giving 404 otherwise.
Challenge URIs should be determined by the WFE at fetch time, rather than stored
alongside the challenge in the DB. This simplifies a lot of the logic, and
allows to to remove a code path in NewAuthorization where we create an
authorization, then immediately save it with modifications to the challenges.
This change also gives challenges their own endpoint, which contains the
challenge id rather than the challenge's offset within its parent authorization.
This is also a first step towards replacing UpdateAuthorization with
UpdateChallenge: https://github.com/letsencrypt/boulder/issues/760.
The RA did not have any code to test what occurred when a challenge
failed. This let in the authz schema change in #705.
This change sets the expires column in authz back to NULLable and fixes
the RA tests (including, using clock.Clocks in the RA).
Fixes#744.
This function splits out Authorization into its three separate cases:
GET a challenge
POST a challenge
GET an authorization
This is a refactoring I needed to do on the way to
https://github.com/letsencrypt/boulder/issues/638. I split it out to avoid
having that change grow too big and full of ancillary refactorings.
Roland Bracewell Shoemaker