Support BYO server certs AND user bundles

The original implementation assumed that if you brought your own server
cert, then users certs would be signed by the same CA, but this will
make it quite challenging for large enterprises who would be forced to
manage certs for users, or worse, buy them from the same external CA.
Since the UCP controller already trusts multiple root, there's no reason
we can't add another.

Prior to this change the CA stored next to the server cert was the
"full" trust chain including the root CAs.  With this change, we flip
that around and use the swarm cert CA for the controller.  This is a
tiny bit messy, because we have to be careful not to accidentally wind
up with that CA on the cluster components other than the controller,
so I've enhanced our integration tests to cover this case specifically
and make sure we don't mistakenly open the system up.  In doing so,
I had to refine the integration test so all the servers were signed by
the same CA (the prior code was sloppy and used a fresh CA for each HA
node, which meant the bundles broke on the replica controllers.)

In the future, we'll likely have intermediaries with differnet
privileges/scopes, and may revisit the multiple root CA model, so this
seems like a reasonable compromise to keep the code churn down for now.
This commit is contained in:
Daniel Hiltgen 2016-02-02 20:09:11 -08:00 committed by Joao Fernandes
parent e040139d6b
commit 596200b888
5 changed files with 4 additions and 231 deletions

207
certs.md
View File

@ -1,207 +0,0 @@
<!--[metadata]>
+++
title ="Manually setting up a CA"
description="Docker Universal Control Plane"
[menu.main]
parent="mn_ucp"
+++
<![end-metadata]-->
# Manually set up a Certificate Authority
A few features of UCP require an external CA (cfssl or equivalent) to sign
certs. By default in developer mode, these services aren't created (maybe we'll
set them up someday, but there's some bootstrapping challenges...) The
following instructions explain how to set this up manually so you can do
developer mode with CA support.
```bash
sudo mkdir -p /etc/docker/ssl/orca
sudo sh -c 'cat << EOF > /etc/docker/ssl/orca/config.json
{
"roots": {
"orca": {
"key": "/etc/docker/ssl/orca/orca_ca_key.pem",
"certificate": "/etc/docker/ssl/orca/orca_ca.pem"
},
"swarm": {
"key": "/etc/docker/ssl/orca/swarm_ca_key.pem",
"certificate": "/etc/docker/ssl/orca/swarm_ca.pem"
}
},
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"client": {
"usages": [
"signing",
"key encipherment",
"client auth"
],
"expiry": "87600h"
},
"node": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
},
"intermediate": {
"usages": [
"signing",
"key encipherment",
"cert sign",
"crl sign"
],
"is_ca": true,
"expiry": "87600h"
}
}
}
}
EOF'
sudo sh -c 'cat << EOF > /etc/docker/ssl/orca/orca_ca.json
{
"key": {
"algo": "rsa",
"size": 4096
},
"CN": "Orca Root CA"
}
EOF'
sudo sh -c 'cat << EOF > /etc/docker/ssl/orca/swarm_ca.json
{
"key": {
"algo": "rsa",
"size": 4096
},
"CN": "Swarm Root CA"
}
EOF'
docker run --rm -v /etc/docker/ssl/orca:/etc/docker/ssl/orca -w /etc/docker/ssl/orca dockerorca/orca-cfssl genkey -initca orca_ca.json | \
docker run --rm -i -v /etc/docker/ssl/orca:/etc/docker/ssl/orca --entrypoint cfssljson -w /etc/docker/ssl/orca dockerorca/orca-cfssl -bare orca_ca
docker run --rm -v /etc/docker/ssl/orca:/etc/docker/ssl/orca -w /etc/docker/ssl/orca dockerorca/orca-cfssl genkey -initca swarm_ca.json | \
docker run --rm -i -v /etc/docker/ssl/orca:/etc/docker/ssl/orca --entrypoint cfssljson -w /etc/docker/ssl/orca dockerorca/orca-cfssl -bare swarm_ca
# Just to keep the naming consistent...
sudo mv /etc/docker/ssl/orca/orca_ca-key.pem /etc/docker/ssl/orca/orca_ca_key.pem
sudo mv /etc/docker/ssl/orca/swarm_ca-key.pem /etc/docker/ssl/orca/swarm_ca_key.pem
```
Once you've generated the root cert (above) you can start the servers
```bash
docker run -d \
-v /etc/docker/ssl/orca/orca_ca.pem:/etc/cfssl/ca.pem:ro \
-v /etc/docker/ssl/orca/orca_ca_key.pem:/etc/cfssl/ca-key.pem:ro \
-v /etc/docker/ssl/orca/config.json:/etc/cfssl/config.json:ro \
--name orca-ca \
dockerorca/orca-cfssl serve --address 0.0.0.0 -config config.json
docker run -d \
-v /etc/docker/ssl/orca/swarm_ca.pem:/etc/cfssl/ca.pem:ro \
-v /etc/docker/ssl/orca/swarm_ca_key.pem:/etc/cfssl/ca-key.pem:ro \
-v /etc/docker/ssl/orca/config.json:/etc/cfssl/config.json:ro \
--name orca-swarm-ca \
dockerorca/orca-cfssl serve --address 0.0.0.0 -config config.json
```
Now you can generate a server cert (you might want to edit this to make the hostname/ip match your config
```bash
sudo sh -c 'cat << EOF > /etc/docker/ssl/orca/server.json
{
"hosts": [
"127.0.0.1"
],
"key": {
"algo": "rsa",
"size": 4096
},
"CN": "My Server"
}
EOF'
docker run --rm --link=orca-swarm-ca:swarm_ca \
-v /etc/docker/ssl/orca:/etc/docker/ssl/orca \
-w /etc/docker/ssl/orca \
--entrypoint=/bin/sh \
dockerorca/orca-cfssl -c \
'cfssl gencert -remote $SWARM_CA_PORT_8888_TCP_ADDR -profile=node server.json' | \
docker run --rm -i -v /etc/docker/ssl/orca:/etc/docker/ssl/orca \
-w /etc/docker/ssl/orca \
--entrypoint=cfssljson dockerorca/orca-cfssl \
-bare swarm_orca_server
docker run --rm --link=orca-ca:orca_ca \
-v /etc/docker/ssl/orca:/etc/docker/ssl/orca \
-w /etc/docker/ssl/orca \
--entrypoint=/bin/sh \
dockerorca/orca-cfssl -c \
'cfssl gencert -remote $ORCA_CA_PORT_8888_TCP_ADDR -profile=node server.json' | \
docker run --rm -i -v /etc/docker/ssl/orca:/etc/docker/ssl/orca \
-w /etc/docker/ssl/orca \
--entrypoint=cfssljson dockerorca/orca-cfssl \
-bare orca_server
# Just to keep the naming consistent...
sudo mv /etc/docker/ssl/orca/swarm_orca_server-key.pem /etc/docker/ssl/orca/swarm_orca_server_key.pem
sudo mv /etc/docker/ssl/orca/orca_server-key.pem /etc/docker/ssl/orca/orca_server_key.pem
```
**Now you can run UCP**
(proxy)
```bash
docker run -d \
-v /etc/docker/ssl/orca:/etc/docker/ssl/orca:ro \
-v /var/run/docker.sock:/var/run/docker.sock \
-e SSL_CA=/etc/docker/ssl/orca/swarm_ca.pem \
-e SSL_CERT=/etc/docker/ssl/orca/swarm_orca_server.pem \
-e SSL_KEY=/etc/docker/ssl/orca/swarm_orca_server_key.pem \
--name orca-proxy \
dockerorca/orca-proxy
```
(rethinkdb)
```bash
docker run -d \
--name orca-db \
dockerorca/rethinkdb
```
Then orca itself
```bash
docker run --rm -it \
-v /etc/docker/ssl/orca:/etc/docker/ssl/orca:ro \
--link orca-ca:orca_ca \
--link orca-swarm-ca:swarm_ca \
--link orca-proxy:proxy \
--link orca-db:rethinkdb \
--name orca-controller \
dockerorca/orca \
--debug server \
--docker tcp://proxy:2375 \
--tls-ca-cert /etc/docker/ssl/orca/swarm_ca.pem \
--tls-cert /etc/docker/ssl/orca/swarm_orca_server.pem \
--tls-key /etc/docker/ssl/orca/swarm_orca_server_key.pem \
--orca-tls-ca-cert /etc/docker/ssl/orca/orca_ca.pem \
--orca-tls-cert /etc/docker/ssl/orca/orca_server.pem \
--orca-tls-key /etc/docker/ssl/orca/orca_server_key.pem
```

View File

@ -642,12 +642,7 @@ for the UCP controller. This cert is used by UCP's main management web UI
and the Docker remote API. The remote API is visible to the Docker CLI. In this release, the Swarm Root CA is always managed by UCP.
The external UCP Root CA model supports customers managing their own CA, or
purchasing certs from a commercial CA. When operating in this mode, UCP can
not generate regular user certificates, as those must be managed and signed
externally, however admin account certs can be generated as they are signed by
the internal Swarm Root CA. Normal user accounts should be signed by the same
external Root CA (or a trusted intermediary), and the public keys manually added
through the UI.
purchasing certs from a commercial CA.
The first time you install, we recommend you skip user-supplied certs and use
the default certificates instead. The default TLS certificate files are placed

View File

@ -716,12 +716,7 @@ for the UCP controller. This cert is used by UCP's main management web UI
and the Docker remote API. The remote API is visible to the Docker CLI. In this release, the Swarm Root CA is always managed by UCP.
The external UCP Root CA model supports customers managing their own CA, or
purchasing certs from a commercial CA. When operating in this mode, UCP can
not generate regular user certificates, as those must be managed and signed
externally, however admin account certs can be generated as they are signed by
the internal Swarm Root CA. Normal user accounts should be signed by the same
external Root CA (or a trusted intermediary), and the public keys manually added
through the UI.
purchasing certs from a commercial CA.
The first time you install, we recommend you skip user-supplied certs and use
the default certificates instead. The default TLS certificate files are placed

View File

@ -635,12 +635,7 @@ for the UCP controller. This cert is used by UCP's main management web UI
and the Docker remote API. The remote API is visible to the Docker CLI. In this release, the Swarm Root CA is always managed by UCP.
The external UCP Root CA model supports customers managing their own CA, or
purchasing certs from a commercial CA. When operating in this mode, UCP can
not generate regular user certificates, as those must be managed and signed
externally, however admin account certs can be generated as they are signed by
the internal Swarm Root CA. Normal user accounts should be signed by the same
external Root CA (or a trusted intermediary), and the public keys manually added
through the UI.
purchasing certs from a commercial CA.
The first time you install, we recommend you skip user-supplied certs and use
the default certificates instead. The default TLS certificate files are placed

View File

@ -598,12 +598,7 @@ for the UCP controller. This cert is used by UCP's main management web UI
and the Docker remote API. The remote API is visible to the Docker CLI. In this release, the Swarm Root CA is always managed by UCP.
The external UCP Root CA model supports customers managing their own CA, or
purchasing certs from a commercial CA. When operating in this mode, UCP can
not generate regular user certificates, as those must be managed and signed
externally, however admin account certs can be generated as they are signed by
the internal Swarm Root CA. Normal user accounts should be signed by the same
external Root CA (or a trusted intermediary), and the public keys manually added
through the UI.
purchasing certs from a commercial CA.
The first time you install, we recommend you skip user-supplied certs and use
the default certificates instead. The default TLS certificate files are placed