Merge pull request #702 from docker/kube-net-encryption-669

Created Kube net encryption 669. Ready for review.
2018-09-19 11:14:14 -04:00 · 2018-09-19 11:14:14 -04:00 · 8ab6d7c541
parent 5648489031 6e1214e9e4
commit 8ab6d7c541
4 changed files with 241 additions and 0 deletions
--- a/_data/toc.yaml
+++ b/_data/toc.yaml
@ -1733,6 +1733,8 @@ manuals:
        path: /ee/ucp/kubernetes/create-service-account/
      - title: Install a CNI plugin
        path: /ee/ucp/kubernetes/install-cni-plugin/
+      - title: Kubernetes network encryption
+        path: /ee/ucp/kubernetes/kubernetes-network-encryption/
    - title: API reference
      path: /reference/ucp/3.0/api/
      nosync: true
--- a/ee/images/kubernetes-network-encryption.png
+++ b/ee/images/kubernetes-network-encryption.png
--- a/ee/ucp/kubernetes/kubernetes-network-encryption.md
+++ b/ee/ucp/kubernetes/kubernetes-network-encryption.md
@ -0,0 +1,74 @@
+---
+title: Kubernetes Network Encryption
+description: Learn how to configure network encryption in Kubernetes
+keywords: ucp, cli, administration, kubectl, Kubernetes, security, network, ipsec, ipip, esp, calico
+---
+
+Docker Enterprise Edition provides data-plane level IPSec network encryption to securely encrypt application 
+traffic in a Kubernetes cluster. This secures application traffic within a cluster when running in untrusted 
+infrastructure or environments. It is an optional feature of UCP that is enabled by deploying the Secure Overlay 
+components on Kuberenetes when using the default Calico driver for networking configured for IPIP tunnelling 
+(the default configuration).
+
+Kubernetes network encryption is enabled by two components in UCP: the SecureOverlay Agent and SecureOverlay 
+Master. The agent is deployed as a per-node service that manages the encryption state of the data plane. The 
+agent controls the IPSec encryption on Calico’s IPIP tunnel traffic between different nodes in the Kubernetes 
+cluster. The master is the second component, deployed on a UCP manager node, which acts as the key management 
+process that configures and periodically rotates the encryption keys.
+
+Kubernetes network encryption uses AES Galois Counter Mode (AES-GCM) with 128-bit keys by default. Encryption 
+is not enabled by default and requires the SecureOverlay Agent and Master to be deployed on UCP to begin 
+encrypting traffic within the cluster. It can be enabled or disabled at any time during the cluster lifecycle. 
+However, it should be noted that it can cause temporary traffic outages between pods during the first few minutes 
+of traffic enabling/disabling. When enabled, Kubernetes pod traffic between hosts is encrypted at the IPIP tunnel 
+interface in the UCP host.
+
+![Kubernetes Network Encryption](/ee/images/kubernetes-network-encryption.png)
+
+## Requirements
+
+Kubernetes Network Encryption is supported for the following platforms:
+* Docker Enterprise 2.1+ (UCP 3.1+)
+* Kubernetes 1.11+
+* On-premise, AWS, GCE
+* Azure is not supported for network encryption as encryption utilizes Calico’s IPIP tunnel
+* Only supported when using UCP’s default Calico CNI plugin
+* Supported on all Docker Enterprise supported Linux OSes
+
+## Configuring MTUs
+
+Before deploying the SecureOverlay components one must ensure that Calico is configured so that the IPIP tunnel 
+MTU leaves sufficient headroom for the encryption overhead.   Encryption adds 26 bytes of overhead but every IPSec 
+packet size must be a multiple of 4 bytes.  IPIP tunnels require 20 bytes of encapsulation overhead.  So the IPIP 
+tunnel interface MTU must be no more than "EXTMTU - 46 - ((EXTMTU - 46) modulo 4)" where EXTMTU is the minimum MTU 
+of the external interfaces.   An IPIP MTU of 1452 should generally be safe for most deployments. 
+
+Changing UCP's MTU requires updating the UCP configuration.  This process is described [here](/ee/ucp/admin/configure/ucp-configuration-file).  
+
+The user must update the following values to the new MTU:
+
+     [cluster_config]
+      ...
+      calico_mtu = "1452"
+      ipip_mtu = "1452"
+      ...
+
+## Configuring SecureOverlay
+
+Once the cluster nodes’ MTUs are properly configured, deploy the SecureOverlay components using the Secure Overlay YAML file to UCP.
+
+[Download the Secure Overlay YAML file here.](ucp-secureoverlay.yml)
+
+After downloading the YAML file, run the following command from any machine with the properly configured kubectl environment and the proper UCP bundle's credentials:
+
+```
+$ kubectl apply -f ucp-secureoverlay.yml
+```
+
+Run this command at cluster installation time before starting any workloads.
+
+To remove the encryption from the system, issue the command:
+
+```
+$ kubectl delete -f ucp-secureoverlay.yml
+```
--- a/ee/ucp/kubernetes/ucp-secureoverlay.yml
+++ b/ee/ucp/kubernetes/ucp-secureoverlay.yml
@ -0,0 +1,165 @@
+######################
+# Cluster role for key management jobs
+######################
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: ucp-secureoverlay-mgr
+rules:
+  - apiGroups: [""]
+    resources:
+      - secrets
+    verbs:
+      - get
+      - update
+---
+######################
+# Cluster role binding for key management jobs
+######################
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: ucp-secureoverlay-mgr
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: ucp-secureoverlay-mgr
+subjects:
+- kind: ServiceAccount
+  name: ucp-secureoverlay-mgr
+  namespace: kube-system
+---
+######################
+# Service account for key management jobs
+######################
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: ucp-secureoverlay-mgr
+  namespace: kube-system
+---
+######################
+# Cluster role for secure overlay per-node agent
+######################
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: ucp-secureoverlay-agent
+rules:
+  - apiGroups: [""]
+    resources:
+      - nodes
+    verbs:
+      - get
+      - list
+      - watch
+---
+######################
+# Cluster role binding for secure overlay per-node agent
+######################
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: ucp-secureoverlay-agent
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: ucp-secureoverlay-agent
+subjects:
+- kind: ServiceAccount
+  name: ucp-secureoverlay-agent
+  namespace: kube-system
+---
+######################
+# Service account secure overlay per-node agent
+######################
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: ucp-secureoverlay-agent
+  namespace: kube-system
+---
+######################
+# K8s secret of current key configuration
+######################
+apiVersion: v1
+kind: Secret
+metadata:
+  name: ucp-secureoverlay
+  namespace: kube-system
+type: Opaque
+data:
+  keys: ""
+---
+######################
+# DaemonSet for secure overlay per-node agent
+######################
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+  name: ucp-secureoverlay-agent
+  namespace: kube-system
+  labels:
+    k8s-app: ucp-secureoverlay-agent
+spec:
+  selector:
+    matchLabels:
+      k8s-app: ucp-secureoverlay-agent
+  updateStrategy:
+    type: RollingUpdate
+  template:
+    metadata:
+      labels:
+        k8s-app: ucp-secureoverlay-agent
+      annotations:
+        scheduler.alpha.kubernetes.io/critical-pod: ''
+    spec:
+      hostNetwork: true
+      priorityClassName: system-node-critical
+      terminationGracePeriodSeconds: 10
+      serviceAccountName: ucp-secureoverlay-agent
+      containers:
+      - name: ucp-secureoverlay-agent
+        image: ucp-secureoverlay-agent:3.1.0
+        securityContext:
+          capabilities:
+            add: ["NET_ADMIN"]
+        env:
+        - name: MY_NODE_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: spec.nodeName
+        volumeMounts:
+        - name: ucp-secureoverlay
+          mountPath: /etc/secureoverlay/
+          readOnly: true
+      volumes:
+      - name: ucp-secureoverlay
+        secret:
+          secretName: ucp-secureoverlay
+---
+######################
+# Deployment for manager of the whole cluster (to rotate keys)
+######################
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ucp-secureoverlay-mgr
+  namespace: kube-system
+spec:
+  selector:
+    matchLabels:
+      app: ucp-secureoverlay-mgr
+  replicas: 1
+  template:
+    metadata:
+      name: ucp-secureoverlay-mgr
+      namespace: kube-system
+      labels:
+        app: ucp-secureoverlay-mgr
+    spec:
+      serviceAccountName: ucp-secureoverlay-mgr
+      restartPolicy: Always
+      containers:
+      - name: ucp-secureoverlay-mgr
+        image: ucp-secureoverlay-mgr:3.1.0