mirror of https://github.com/docker/docs.git
Merge pull request #8346 from docker/ha-nginx-controller
Ha nginx controller updates
This commit is contained in:
paigehargrave
GitHub
commit
a89ba775a6
Binary file not shown.
|
After Width: | Height: | Size: 110 KiB |
|
|
@ -7,6 +7,8 @@ redirect_from:
|
|||
- /ee/ucp/kubernetes/deploy-ingress-controller/
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
When you deploy a Kubernetes application, you may want to make it accessible
|
||||
to users using hostnames instead of IP addresses.
|
||||
|
||||
|
|
@ -19,87 +21,180 @@ Use an ingress controller when you want to:
|
|||
* Give your Kubernetes app an externally-reachable URL.
|
||||
* Load-balance traffic to your app.
|
||||
|
||||
Kubernetes provides an NGINX ingress controller that you can use in Docker EE
|
||||
without modifications.
|
||||
Learn about [ingress in Kubernetes](https://v1-8.docs.kubernetes.io/docs/concepts/services-networking/ingress/).
|
||||
Ingress is an API object that manages external access to the services in a cluster, typically HTTP. In UCP 3.1, Ingress must be deployed separately. There are a variety of ingress controllers, but for UCP, the supported one is the [NGINX Ingress Controller](https://github.com/kubernetes/ingress-nginx). The following instructions provide highly-available production deployment of this controller.
|
||||
|
||||
## Create a dedicated namespace
|
||||
## Prerequisites
|
||||
|
||||
1. Navigate to the **Namespaces** page, and click **Create**.
|
||||
2. In the **Object YAML** editor, append the following text.
|
||||
```yaml
|
||||
metadata:
|
||||
name: ingress-nginx
|
||||
```
|
||||
- UCP deployed and properly configured
|
||||
- Two or three dedicated infrae nodes deployed as UCP worker nodes
|
||||
- An external load-balancer fronting these nodes with an associated VIP that resolves the application DNS (for example, `*.app.docker.mycompany.com`)
|
||||
|
||||
The finished YAML should look like this.
|
||||
### Step 1: Labeling infrastructure nodes
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ingress-nginx
|
||||
```
|
||||
3. Click **Create**.
|
||||
4. In the **ingress-nginx** namespace, click the **More options** icon,
|
||||
and in the context menu, select **Set Context**.
|
||||
To deploy the NGINX controller on the infra nodes, the nodes must be labeled. Identify 2 or 3 nodes as the infra nodes to host the ingress controller. The following example uses three nodes: `dockeree-worker-linux-1`, `dockeree-worker-linux-2`, and `dockeree-worker-linux-3`:
|
||||
|
||||
{: .with-border}
|
||||
```
|
||||
🐳 → kubectl get nodes
|
||||
NAME STATUS ROLES AGE VERSION
|
||||
dockeree-worker-linux-1 Ready <none> 5d v1.8.11-docker-8d637ae
|
||||
dockeree-worker-linux-2 Ready <none> 5d v1.8.11-docker-8d637ae
|
||||
dockeree-worker-linux-3 Ready <none> 5d v1.8.11-docker-8d637ae
|
||||
|
||||
## Create a grant
|
||||
```
|
||||
|
||||
The default service account that's associated with the `ingress-nginx`
|
||||
namespace needs access to Kubernetes resources, so create a grant with
|
||||
`Restricted Control` permissions.
|
||||
For this example, the label `infra.role=ingress` is used to label the nodes:
|
||||
|
||||
1. From UCP, navigate to the **Grants** page, and click **Create Grant**.
|
||||
2. Within the **Subject** pane, select **Service Account**. For the
|
||||
**Namespace** select **ingress-nginx**, and select **default** for
|
||||
the **Service Account**. Click **Next**.
|
||||
3. Within the **Role** pane, select **Restricted Control**, and then click
|
||||
**Next**.
|
||||
4. Within the **Resource Set** pane, select the **Type** **Namespace**, and
|
||||
select the **Apply grant to all existing and new namespaces** toggle.
|
||||
5. Click **Create**.
|
||||
```
|
||||
🐳 → kubectl label node dockeree-worker-linux-1 infra.role=ingress
|
||||
node "dockeree-worker-linux-1" labeled
|
||||
🐳 → kubectl label node dockeree-worker-linux-2 infra.role=ingress
|
||||
node "dockeree-worker-linux-2" labeled
|
||||
🐳 → kubectl label node dockeree-worker-linux-3 infra.role=ingress
|
||||
node "dockeree-worker-linux-3" labeled
|
||||
|
||||
## Deploy NGINX ingress controller
|
||||
```
|
||||
|
||||
The cluster is ready for the ingress controller deployment, which has three
|
||||
main components:
|
||||
### Step 2: Create a Dedicated Namespace
|
||||
|
||||
- a simple HTTP server, named `default-http-backend`,
|
||||
- an ingress controller, named `nginx-ingress-controller`, and
|
||||
- a service that exposes the app, named `ingress-nginx`.
|
||||
A dedicated namespace, for example, `infra`, is needed for all infrastructure deployment activities. You also need a service account to enable the ingress controller to work with the Kubernetes API. After creating a namespace and a service account, you must create an RBAC policy to only allow infrastructure deployments on the dedicated nodes that were labelled in the previous example.
|
||||
|
||||
Navigate to the **Create Kubernetes Object** page, and in the **Object YAML**
|
||||
editor, paste the following YAML.
|
||||
To create a namespace and a service account, simply use the following [YAML file](config/ns-and-sa.yaml) and apply it via the CLI or the UI:
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1beta2
|
||||
```
|
||||
🐳 → cat ns-and-sa.yaml
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: infra
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: nginx-ingress-service-account
|
||||
namespace: infra
|
||||
```
|
||||
|
||||
Then apply the namespace and service account via `kubectl`:
|
||||
|
||||
```
|
||||
🐳 → kubectl apply -f ns-and-sa.yaml
|
||||
namespace "infra" created
|
||||
serviceaccount "nginx-ingress-service-account" created
|
||||
|
||||
```
|
||||
|
||||
### Step 3: Create an RBAC Policy
|
||||
|
||||
Apply an RBAC role-binding for NGINX controller access to the API Server, as shown in the following example:
|
||||
|
||||
|
||||
```
|
||||
🐳 → cat ingress-rbac.yaml
|
||||
## Source: https://github.com/nginxinc/kubernetes-ingress/blob/master/deployments/rbac/rbac.yaml
|
||||
kind: ClusterRole
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: nginx-ingress-cluster-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- services
|
||||
- endpoints
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- secrets
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- create
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- pods
|
||||
verbs:
|
||||
- list
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- events
|
||||
verbs:
|
||||
- create
|
||||
- patch
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resources:
|
||||
- ingresses
|
||||
verbs:
|
||||
- list
|
||||
- watch
|
||||
- get
|
||||
- apiGroups:
|
||||
- "extensions"
|
||||
resources:
|
||||
- ingresses/status
|
||||
verbs:
|
||||
- update
|
||||
---
|
||||
kind: ClusterRoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: nginx-ingress-cluster-rb
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-ingress-service-account
|
||||
namespace: infra
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: nginx-ingress-cluster-role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
```
|
||||
|
||||
Then apply it with `kubectl`:
|
||||
|
||||
```
|
||||
🐳 → kubectl apply -f ingress-rbac.yaml
|
||||
```
|
||||
|
||||
### Step 4: Deploy NGINX Controller
|
||||
|
||||
Note that following example uses hostPorts for controller ports. This exposes the host port ( selected in a high range using `hostPort: 38080`) directly into the nodes. Port 38080 is used for HTTP and port 38443 is used for HTTPS. Make sure that your loadbalancer forwards to the applicable ports on the nodes. You can change them as needed.
|
||||
|
||||
|
||||
```
|
||||
🐳 → cat nginx-ingress-deployment.yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: default-http-backend
|
||||
labels:
|
||||
app: default-http-backend
|
||||
namespace: ingress-nginx
|
||||
namespace: infra
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: default-http-backend
|
||||
replicas: 3
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: default-http-backend
|
||||
annotations:
|
||||
seccomp.security.alpha.kubernetes.io/pod: docker/default
|
||||
spec:
|
||||
terminationGracePeriodSeconds: 60
|
||||
containers:
|
||||
- name: default-http-backend
|
||||
# Any image is permissible as long as:
|
||||
# 1. It serves a 404 page at /
|
||||
# 2. It serves 200 on a /healthz endpoint
|
||||
image: gcr.io/google_containers/defaultbackend:1.4
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
|
|
@ -117,12 +212,14 @@ spec:
|
|||
requests:
|
||||
cpu: 10m
|
||||
memory: 20Mi
|
||||
nodeSelector:
|
||||
infra.role: ingress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: default-http-backend
|
||||
namespace: ingress-nginx
|
||||
namespace: infra
|
||||
labels:
|
||||
app: default-http-backend
|
||||
spec:
|
||||
|
|
@ -136,7 +233,7 @@ kind: ConfigMap
|
|||
apiVersion: v1
|
||||
metadata:
|
||||
name: nginx-configuration
|
||||
namespace: ingress-nginx
|
||||
namespace: infra
|
||||
labels:
|
||||
app: ingress-nginx
|
||||
---
|
||||
|
|
@ -144,21 +241,21 @@ kind: ConfigMap
|
|||
apiVersion: v1
|
||||
metadata:
|
||||
name: tcp-services
|
||||
namespace: ingress-nginx
|
||||
namespace: infra
|
||||
---
|
||||
kind: ConfigMap
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: udp-services
|
||||
namespace: ingress-nginx
|
||||
namespace: infra
|
||||
---
|
||||
apiVersion: apps/v1beta2
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: nginx-ingress-controller
|
||||
namespace: ingress-nginx
|
||||
namespace: infra
|
||||
spec:
|
||||
replicas: 1
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: ingress-nginx
|
||||
|
|
@ -169,7 +266,6 @@ spec:
|
|||
annotations:
|
||||
prometheus.io/port: '10254'
|
||||
prometheus.io/scrape: 'true'
|
||||
seccomp.security.alpha.kubernetes.io/pod: docker/default
|
||||
spec:
|
||||
initContainers:
|
||||
- command:
|
||||
|
|
@ -181,9 +277,10 @@ spec:
|
|||
name: sysctl
|
||||
securityContext:
|
||||
privileged: true
|
||||
serviceAccountName: nginx-ingress-service-account
|
||||
containers:
|
||||
- name: nginx-ingress-controller
|
||||
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.20.0
|
||||
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.21.0
|
||||
args:
|
||||
- /nginx-ingress-controller
|
||||
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
|
||||
|
|
@ -191,7 +288,6 @@ spec:
|
|||
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
|
||||
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
|
||||
- --annotations-prefix=nginx.ingress.kubernetes.io
|
||||
- --enable-ssl-passthrough
|
||||
env:
|
||||
- name: POD_NAME
|
||||
valueFrom:
|
||||
|
|
@ -204,8 +300,12 @@ spec:
|
|||
ports:
|
||||
- name: http
|
||||
containerPort: 80
|
||||
hostPort: 38443
|
||||
protocol: TCP
|
||||
- name: https
|
||||
containerPort: 443
|
||||
hostPort: 38443
|
||||
protocol: TCP
|
||||
livenessProbe:
|
||||
failureThreshold: 3
|
||||
httpGet:
|
||||
|
|
@ -225,27 +325,121 @@ spec:
|
|||
periodSeconds: 10
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 1
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: ingress-nginx
|
||||
namespace: ingress-nginx
|
||||
spec:
|
||||
type: NodePort
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: 80
|
||||
protocol: TCP
|
||||
- name: https
|
||||
port: 443
|
||||
targetPort: 443
|
||||
protocol: TCP
|
||||
selector:
|
||||
app: ingress-nginx
|
||||
nodeSelector:
|
||||
infra.role: ingress
|
||||
```
|
||||
|
||||
Deploy the controller using `kubectl` and verify pods are deployed successfully:
|
||||
|
||||
```
|
||||
🐳 → kubectl apply -f nginx-ingress-deployment.yaml
|
||||
deployment.extensions "default-http-backend" created
|
||||
service "default-http-backend" created
|
||||
configmap "nginx-configuration" created
|
||||
configmap "tcp-services" created
|
||||
configmap "udp-services" created
|
||||
deployment.extensions "nginx-ingress-controller" created
|
||||
|
||||
🐳 → kubectl get pod -n infra -o wide
|
||||
NAME READY STATUS RESTARTS AGE IP NODE
|
||||
default-http-backend-7ff9774865-hsj46 1/1 Running 0 1m 192.168.145.6 dockeree-worker-linux-1
|
||||
default-http-backend-7ff9774865-kcqhj 1/1 Running 0 1m 192.168.116.145 dockeree-worker-linux-3
|
||||
default-http-backend-7ff9774865-xq566 1/1 Running 0 1m 192.168.247.210 dockeree-worker-linux-2
|
||||
nginx-ingress-controller-6b987cbbc6-4qqz8 1/1 Running 0 1m 192.168.145.7 dockeree-worker-linux-1
|
||||
nginx-ingress-controller-6b987cbbc6-h6rmg 1/1 Running 0 1m 192.168.116.146 dockeree-worker-linux-3
|
||||
nginx-ingress-controller-6b987cbbc6-hkw86 1/1 Running 0 1m 192.168.247.211 dockeree-worker-linux-2
|
||||
```
|
||||
|
||||
### Step 5: Deploy an application and its ingress rule
|
||||
|
||||
After the controller is successfully deployed, you can create ingress objects to expose applications externally. The following example application deployment uses deployment yaml consisting of `Service` and `Deployment` objects that deploy the application with the Docker image `ehazlett/docker-demo` and create a service associated with it.
|
||||
|
||||
|
||||
```
|
||||
🐳 → cat dockerdemo.yaml
|
||||
|
||||
kind: Service
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
namespace: default
|
||||
name: docker-demo-svc
|
||||
spec:
|
||||
selector:
|
||||
app: dockerdemo
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 8080
|
||||
targetPort: 8080
|
||||
---
|
||||
apiVersion: apps/v1beta2
|
||||
kind: Deployment
|
||||
metadata:
|
||||
namespace: default
|
||||
name: dockerdemo-deploy
|
||||
labels:
|
||||
app: dockerdemo
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: dockerdemo
|
||||
strategy:
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: dockerdemo
|
||||
spec:
|
||||
containers:
|
||||
- image: ehazlett/docker-demo
|
||||
name: docker-demo-container
|
||||
env:
|
||||
- name: app
|
||||
value: dockerdemo
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
```
|
||||
|
||||
|
||||
To expose this application with a valid URL, for example, `dockerdemo.app.docker.example.com`, you must create an `Ingress` rule. The ingress rule is used by the NGINX controller to generate the proper NGINX configuration for proper load-balancing. For example:
|
||||
|
||||
```
|
||||
🐳 → cat dockerdemo.ingress.yaml
|
||||
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: dockerdemo-ingress
|
||||
namespace: default
|
||||
annotations:
|
||||
kubernetes.io/ingress.class: "nginx"
|
||||
spec:
|
||||
rules:
|
||||
- host: dockerdemo.app.docker.example.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
backend:
|
||||
serviceName: docker-demo-svc
|
||||
servicePort: 8080
|
||||
```
|
||||
|
||||
```
|
||||
🐳 → kubectl apply -f dockerdemo.ingress.yaml
|
||||
```
|
||||
|
||||
You can apply ingress rules separately or combine them with the application deployment yaml file.
|
||||
|
||||
```
|
||||
|
||||
🐳 → kubectl get ingress
|
||||
NAME HOSTS ADDRESS PORTS AGE
|
||||
dockerdemo-ingress dockerdemo.app.docker.example.com 80 7d
|
||||
```
|
||||
|
||||
Assuming you have already registered a DNS record for your application pointing to the external load-balancer fronting the `infra` nodes, you should be able to access your application using the URL.You can also scale the docker demo deployment to test how ingress correctly routes traffic to all the backend pods!
|
||||
|
||||
|
||||
|
||||
## Check your deployment
|
||||
|
||||
The `default-http-backend` provides a simple service that serves a 404 page
|
||||
|
|
|
|||
Loading…
Reference in New Issue