kubernetes
/
website
mirror of https://github.com/kubernetes/website.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Address review comments.

This commit is contained in:
Anthony Yeh 2016-12-02 13:27:43 -08:00
parent c20e898491
commit 8d12e8c888
1 changed files with 70 additions and 66 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

136
docs/tutorials/stateful-application/run-replicated-stateful-application.md
View File

@ -13,7 +13,7 @@ assignees:
{% capture overview %} {% capture overview %}
This page shows how to run a replicated stateful application using a This page shows how to run a replicated stateful application using a
[StatefulSet](/docs/concepts/controllers/statefulsets/) controller. [StatefulSet](/docs/concepts/abstractions/controllers/statefulsets/) controller.
The example is a MySQL single-master topology with multiple slaves running The example is a MySQL single-master topology with multiple slaves running
asynchronous replication. asynchronous replication.
@ -29,11 +29,11 @@ on general patterns for running stateful applications in Kubernetes.
* {% include default-storage-class-prereqs.md %} * {% include default-storage-class-prereqs.md %}
* This tutorial assumes you are familiar with * This tutorial assumes you are familiar with
[PersistentVolumes](/docs/user-guide/persistent-volumes/) [PersistentVolumes](/docs/user-guide/persistent-volumes/)
and [StatefulSets](/docs/concepts/controllers/statefulsets/), and [StatefulSets](/docs/concepts/abstractions/controllers/statefulsets/),
as well as other core concepts like [Pods](/docs/user-guide/pods/), as well as other core concepts like [Pods](/docs/user-guide/pods/),
[Services](/docs/user-guide/services/), and [Services](/docs/user-guide/services/), and
[ConfigMaps](/docs/user-guide/configmap/). [ConfigMaps](/docs/user-guide/configmap/).
* Some familiarity with MySQL will help, but this tutorial aims to present * Some familiarity with MySQL helps, but this tutorial aims to present
general patterns that should be useful for other systems. general patterns that should be useful for other systems.
{% endcapture %} {% endcapture %}
@ -56,57 +56,59 @@ and a StatefulSet.
#### ConfigMap #### ConfigMap
Create the ConfigMap by saving the following manifest to `mysql-configmap.yaml` Create the ConfigMap from the following YAML configuration file:
and running:
```shell ```shell
kubectl create -f mysql-configmap.yaml export REPO=https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/{{page.docsbranch}}
kubectl create -f $REPO/docs/tutorials/stateful-application/mysql-configmap.yaml
``` ```
{% include code.html language="yaml" file="mysql-configmap.yaml" ghlink="/docs/tutorials/stateful-application/mysql-configmap.yaml" %} {% include code.html language="yaml" file="mysql-configmap.yaml" ghlink="/docs/tutorials/stateful-application/mysql-configmap.yaml" %}
This ConfigMap provides `my.cnf` overrides that let you independently control This ConfigMap provides `my.cnf` overrides that let you independently control
configuration on the master and the slaves. configuration on the MySQL master and slaves.
In this case, you want the master to be able to serve replication logs to slaves In this case, you want the master to be able to serve replication logs to slaves
and you want slaves to reject any writes that don't come via replication. and you want slaves to reject any writes that don't come via replication.
There's nothing special about the ConfigMap itself that causes different There's nothing special about the ConfigMap itself that causes different
portions to apply to different Pods. portions to apply to different Pods.
Each Pod will decide which portion to look at as it's initializing, Each Pod decides which portion to look at as it's initializing,
based on information provided by the StatefulSet controller. based on information provided by the StatefulSet controller.
#### Services #### Services
Create the Services by saving the following manifest to `mysql-services.yaml` Create the Services from the following YAML configuration file:
and running:
```shell ```shell
kubectl create -f mysql-services.yaml export REPO=https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/{{page.docsbranch}}
kubectl create -f $REPO/docs/tutorials/stateful-application/mysql-services.yaml
``` ```
{% include code.html language="yaml" file="mysql-services.yaml" ghlink="/docs/tutorials/stateful-application/mysql-services.yaml" %} {% include code.html language="yaml" file="mysql-services.yaml" ghlink="/docs/tutorials/stateful-application/mysql-services.yaml" %}
The Headless Service provides a home for the DNS entries that the StatefulSet The Headless Service provides a home for the DNS entries that the StatefulSet
controller will create for each Pod that's part of the set. controller creates for each Pod that's part of the set.
Because the Headless Service is named `mysql`, the Pods will be accessible by Because the Headless Service is named `mysql`, the Pods are accessible by
resolving `<pod-name>.mysql` from within any other Pod in the same Kubernetes resolving `<pod-name>.mysql` from within any other Pod in the same Kubernetes
cluster and namespace. cluster and namespace.
The Client Service, called `mysql-read`, is a normal Service with its own The Client Service, called `mysql-read`, is a normal Service with its own
cluster IP that will distribute connections across all MySQL Pods that report cluster IP that distributes connections across all MySQL Pods that report
being Ready. The set of endpoints will include the master and all slaves. being Ready. The set of potential endpoints includes the MySQL master and all
slaves.
Note that only read queries can use the load-balanced Client Service. Note that only read queries can use the load-balanced Client Service.
Because there is only one master, clients should connect directly to the master Because there is only one MySQL master, clients should connect directly to the
Pod (through its DNS entry within the Headless Service) to execute writes. MySQL master Pod (through its DNS entry within the Headless Service) to execute
writes.
#### StatefulSet #### StatefulSet
Finally, create the StatefulSet by saving the following manifest to Finally, create the StatefulSet from the following YAML configuration file:
`mysql-statefulset.yaml` and running:
```shell ```shell
kubectl create -f mysql-statefulset.yaml export REPO=https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/{{page.docsbranch}}
kubectl create -f $REPO/docs/tutorials/stateful-application/mysql-statefulset.yaml
``` ```
{% include code.html language="yaml" file="mysql-statefulset.yaml" ghlink="/docs/tutorials/stateful-application/mysql-statefulset.yaml" %} {% include code.html language="yaml" file="mysql-statefulset.yaml" ghlink="/docs/tutorials/stateful-application/mysql-statefulset.yaml" %}
@ -152,7 +154,7 @@ properties to perform orderly startup of MySQL replication.
Before starting any of the containers in the Pod spec, the Pod first runs any Before starting any of the containers in the Pod spec, the Pod first runs any
[Init Containers](/docs/user-guide/production-pods/#handling-initialization) [Init Containers](/docs/user-guide/production-pods/#handling-initialization)
in the order defined. in the order defined.
In the StatefulSet manifest, you will find these defined within the In the StatefulSet manifest, you can find these defined within the
`pod.beta.kubernetes.io/init-containers` annotation. `pod.beta.kubernetes.io/init-containers` annotation.
The first Init Container, named `init-mysql`, generates special MySQL config The first Init Container, named `init-mysql`, generates special MySQL config
@ -168,19 +170,19 @@ properties.
The script in the `init-mysql` container also applies either `master.cnf` or The script in the `init-mysql` container also applies either `master.cnf` or
`slave.cnf` from the ConfigMap by copying the contents into `conf.d`. `slave.cnf` from the ConfigMap by copying the contents into `conf.d`.
Because the example topology consists of a single master and any number of Because the example topology consists of a single MySQL master and any number of
slaves, the script simply assigns ordinal `0` to be the master, and everyone slaves, the script simply assigns ordinal `0` to be the master, and everyone
else to be slaves. else to be slaves.
Combined with the StatefulSet controller's Combined with the StatefulSet controller's
[deployment order guarantee](/docs/concepts/controllers/statefulsets/#deployment-and-scaling-guarantee), [deployment order guarantee](/docs/concepts/abstractions/controllers/statefulsets/#deployment-and-scaling-guarantee),
this ensures the master is Ready before creating slaves, so they can begin this ensures the MySQL master is Ready before creating slaves, so they can begin
replicating. replicating.
#### Cloning existing data #### Cloning existing data
In general, when a new Pod joins the set as a slave, it must assume the master In general, when a new Pod joins the set as a slave, it must assume the MySQL
might already have data on it. It also must assume that the replication logs master might already have data on it. It also must assume that the replication
might not go all the way back to the beginning of time. logs might not go all the way back to the beginning of time.
These conservative assumptions are the key to allowing a running StatefulSet These conservative assumptions are the key to allowing a running StatefulSet
to scale up and down over time, rather than being fixed at its initial size. to scale up and down over time, rather than being fixed at its initial size.
@ -192,9 +194,9 @@ so its local state is consistent enough to begin replicating from the master.
MySQL itself does not provide a mechanism to do this, so the example uses a MySQL itself does not provide a mechanism to do this, so the example uses a
popular open-source tool called Percona XtraBackup. popular open-source tool called Percona XtraBackup.
During the clone, the source MySQL server might suffer reduced performance. During the clone, the source MySQL server might suffer reduced performance.
To minimize impact on the master, the script instructs each Pod to clone from To minimize impact on the MySQL master, the script instructs each Pod to clone
the Pod whose ordinal index is one lower. from the Pod whose ordinal index is one lower.
This works because the StatefulSet controller will always ensure Pod `N` is This works because the StatefulSet controller always ensures Pod `N` is
Ready before starting Pod `N+1`. Ready before starting Pod `N+1`.
#### Starting replication #### Starting replication
@ -210,10 +212,10 @@ If so, it waits for `mysqld` to be ready and then executes the
`CHANGE MASTER TO` and `START SLAVE` commands with replication parameters `CHANGE MASTER TO` and `START SLAVE` commands with replication parameters
extracted from the XtraBackup clone files. extracted from the XtraBackup clone files.
Once a slave begins replication, by default it will remember its master and Once a slave begins replication, it remembers its MySQL master and
reconnect automatically if the server is restarted or the connection dies. reconnects automatically if the server restarts or the connection dies.
Also, because slaves look for the master at its stable DNS name Also, because slaves look for the master at its stable DNS name
(`mysql-0.mysql`), they will automatically find the master even if it gets a new (`mysql-0.mysql`), they automatically find the master even if it gets a new
Pod IP due to being rescheduled. Pod IP due to being rescheduled.
Lastly, after starting replication, the `xtrabackup` container listens for Lastly, after starting replication, the `xtrabackup` container listens for
@ -223,7 +225,7 @@ case the next Pod loses its PersistentVolumeClaim and needs to redo the clone.
### Sending client traffic ### Sending client traffic
You can send test queries to the master (hostname `mysql-0.mysql`) You can send test queries to the MySQL master (hostname `mysql-0.mysql`)
by running a temporary container with the `mysql:5.7` image and running the by running a temporary container with the `mysql:5.7` image and running the
`mysql` client binary. `mysql` client binary.
@ -336,15 +338,15 @@ kubectl exec mysql-2 -c mysql -- mv /usr/bin/mysql.off /usr/bin/mysql
#### Delete Pods #### Delete Pods
The StatefulSet will also recreate Pods if they're deleted, similar to what a The StatefulSet also recreates Pods if they're deleted, similar to what a
ReplicaSet does for stateless Pods. ReplicaSet does for stateless Pods.
```shell ```shell
kubectl delete pod mysql-2 kubectl delete pod mysql-2
``` ```
The StatefulSet controller will notice that no `mysql-2` Pod exists anymore, The StatefulSet controller notices that no `mysql-2` Pod exists anymore,
and will create a new one with the same name and linked to the same and creates a new one with the same name and linked to the same
PersistentVolumeClaim. PersistentVolumeClaim.
You should see server ID `102` disappear from the loop output for a while You should see server ID `102` disappear from the loop output for a while
and then return on its own. and then return on its own.
@ -368,8 +370,8 @@ NAME READY STATUS RESTARTS AGE IP NODE
mysql-2 2/2 Running 0 15m 10.244.5.27 kubernetes-minion-group-9l2t mysql-2 2/2 Running 0 15m 10.244.5.27 kubernetes-minion-group-9l2t
``` ```
Then drain the Node by running the following command, which will cordon it so Then drain the Node by running the following command, which cordons it so
no new Pods may schedule there, and then evict any existing Pods. no new Pods may schedule there, and then evicts any existing Pods.
Replace `<node-name>` with the name of the Node you found in the last step. Replace `<node-name>` with the name of the Node you found in the last step.
This might impact other applications on the Node, so it's best to This might impact other applications on the Node, so it's best to
@ -461,7 +463,7 @@ You can see this by running:
kubectl get pvc -l app=mysql kubectl get pvc -l app=mysql
``` ```
Which will show that all 5 PVCs still exist, despite having scaled the Which shows that all 5 PVCs still exist, despite having scaled the
StatefulSet down to 3: StatefulSet down to 3:
``` ```
@ -484,42 +486,44 @@ kubectl delete pvc data-mysql-4
{% capture cleanup %} {% capture cleanup %}
* Cancel the `SELECT @@server_id` loop by pressing **Ctrl+C** in its terminal, 1. Cancel the `SELECT @@server_id` loop by pressing **Ctrl+C** in its terminal,
or running the following from another terminal: or running the following from another terminal:
```shell ```shell
kubectl delete pod mysql-client-loop --now kubectl delete pod mysql-client-loop --now
``` ```
* Delete the StatefulSet. This will also begin terminating the Pods. 1. Delete the StatefulSet. This also begins terminating the Pods.
```shell ```shell
kubectl delete statefulset mysql kubectl delete statefulset mysql
``` ```
* Verify that the Pods disappear. 1. Verify that the Pods disappear.
They might take some time to finish terminating. They might take some time to finish terminating.
```shell ```shell
kubectl get pods -l app=mysql kubectl get pods -l app=mysql
``` ```
You'll know the Pods have terminated when the above returns: You'll know the Pods have terminated when the above returns:
``` ```
No resources found. No resources found.
``` ```
* Delete the ConfigMap, Services, and PersistentVolumeClaims. 1. Delete the ConfigMap, Services, and PersistentVolumeClaims.
```shell ```shell
kubectl delete configmap,service,pvc -l app=mysql kubectl delete configmap,service,pvc -l app=mysql
``` ```
* If you manually provisioned PersistentVolumes, you will also need to manually 1. If you manually provisioned PersistentVolumes, you also need to manually
delete them. If you used a dynamic provisioner, it will automatically delete delete them, as well as release the underlying resources.
the PersistentVolumes when it sees you have deleted the If you used a dynamic provisioner, it automatically deletes the
PersistentVolumeClaims above. PersistentVolumes when it sees that you deleted the PersistentVolumeClaims.
Some dynamic provisioners (such as those for EBS and PD) also release the
underlying resources upon deleting the PersistentVolumes.
{% endcapture %} {% endcapture %}