kubernetes
/
kops
mirror of https://github.com/kubernetes/kops.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #4359 from jeyglk/feature/single-to-multi-master-doc-improvements 

Update single-to-multi-master.md
This commit is contained in:
k8s-ci-robot 2018-01-30 22:42:03 -08:00 committed by GitHub
parent cc67497776 d00de67de8
commit f58d7b2de4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 59 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

68
docs/single-to-multi-master.md
View File

@ -10,6 +10,8 @@ Please follow all the backup steps before attempting it. Please read the
[etcd admin guide](https://github.com/coreos/etcd/blob/v2.2.1/Documentation/admin_guide.md) [etcd admin guide](https://github.com/coreos/etcd/blob/v2.2.1/Documentation/admin_guide.md)
before attempting it. before attempting it.
We can migrate from a single-master cluster to a multi-master cluster, but this is a complicated operation. It is easier to create a multi-master cluster using Kops (described [here](https://github.com/kubernetes/kops/blob/master/docs/high_availability.md)). If possible, try to plan this at time of cluster creation.
During this procedure, you will experience **downtime** on the API server, but During this procedure, you will experience **downtime** on the API server, but
not on the end user services. During this downtime, existing pods will continue not on the end user services. During this downtime, existing pods will continue
to work, but you will not be able to create new pods and any existing pod that to work, but you will not be able to create new pods and any existing pod that
@ -65,6 +67,11 @@ a different AZ from the existing one.
```bash ```bash
$ kops create instancegroup master-<availability-zone2> --subnet <availability-zone2> --role Master $ kops create instancegroup master-<availability-zone2> --subnet <availability-zone2> --role Master
```
Example:
```bash
$ kops create ig master-eu-west-1b --subnet eu-west-1b --role Master
``` ```
* ``maxSize`` and ``minSize`` should be 1, * ``maxSize`` and ``minSize`` should be 1,
@ -77,6 +84,12 @@ also required. However, real EC2 instance is not required until the second maste
```bash ```bash
$ kops create instancegroup master-<availability-zone3> --subnet <availability-zone3> --role Master $ kops create instancegroup master-<availability-zone3> --subnet <availability-zone3> --role Master
```
Example:
```bash
$ kops create ig master-eu-west-1c --subnet eu-west-1c --role Master
``` ```
* ``maxSize`` and ``minSize`` should be **0**, * ``maxSize`` and ``minSize`` should be **0**,
@ -91,16 +104,39 @@ reference a third one, even if we have not created it yet.*
$ kops edit cluster example.com $ kops edit cluster example.com
``` ```
* In ``.spec.etcdClusters`` 2 new members in each cluster, one for each new * In ``.spec.etcdClusters`` add 2 new members in each cluster, one for each new
availability zone. availability zone.
```yaml ```yaml
- instanceGroup: master-<availability-zone2> - instanceGroup: master-<availability-zone2>
name: <availability-zone2> name: <availability-zone2-name>
- instanceGroup: master-<availability-zone3> - instanceGroup: master-<availability-zone3>
name: <availability-zone3> name: <availability-zone3-name>
``` ```
Example:
```yaml
etcdClusters:
- etcdMembers:
- instanceGroup: master-eu-west-1a
name: a
- instanceGroup: master-eu-west-1b
name: b
- instanceGroup: master-eu-west-1c
name: c
name: main
- etcdMembers:
- instanceGroup: master-eu-west-1a
name: a
- instanceGroup: master-eu-west-1b
name: b
- instanceGroup: master-eu-west-1c
name: c
name: events
```
## 3 - Add a new master ## 3 - Add a new master
### a - Add a new member to the etcd clusters ### a - Add a new member to the etcd clusters
@ -108,8 +144,15 @@ $ kops edit cluster example.com
**The clusters will stop to work until the new member is started**. **The clusters will stop to work until the new member is started**.
```bash ```bash
$ kubectl --namespace=kube-system exec etcd-server-ip-172-20-36-161.ec2.internal -- etcdctl member add etcd-<availability-zone2> http://etcd-<availability-zone2>.internal.example.com:2380 $ kubectl --namespace=kube-system exec etcd-server-ip-172-20-36-161.ec2.internal -- etcdctl member add etcd-<availability-zone2-name> http://etcd-<availability-zone2-name>.internal.example.com:2380 \
$ kubectl --namespace=kube-system exec etcd-server-events-ip-172-20-36-161.ec2.internal -- etcdctl --endpoint http://127.0.0.1:4002 member add etcd-events-<availability-zone2> http://etcd-events-<availability-zone2>.internal.example.com:2381 && kubectl --namespace=kube-system exec etcd-server-events-ip-172-20-36-161.ec2.internal -- etcdctl --endpoint http://127.0.0.1:4002 member add etcd-events-<availability-zone2-name> http://etcd-events-<availability-zone2-name>.internal.example.com:2381
```
Example:
```bash
$ kubectl --namespace=kube-system exec etcd-server-ip-172-20-36-161.ec2.internal -- etcdctl member add etcd-b http://etcd-b.internal.example.com:2380 \
&& kubectl --namespace=kube-system exec etcd-server-events-ip-172-20-36-161.ec2.internal -- etcdctl --endpoint http://127.0.0.1:4002 member add etcd-events-b http://etcd-events-b.internal.example.com:2381
``` ```
### b - Launch the new master ### b - Launch the new master
@ -132,7 +175,7 @@ Reinitialize the etcd instances:
* Delete the containers and the data directories: * Delete the containers and the data directories:
```bash ```bash
root@ip-172-20-116-230:~# docker stop $(docker ps | grep "etcd:2.2.1" | awk '{print $1}') root@ip-172-20-116-230:~# docker stop $(docker ps | grep "gcr.io/google_containers/etcd" | awk '{print $1}')
root@ip-172-20-116-230:~# rm -r /mnt/master-vol-03b97b1249caf379a/var/etcd/data-events/member/ root@ip-172-20-116-230:~# rm -r /mnt/master-vol-03b97b1249caf379a/var/etcd/data-events/member/
root@ip-172-20-116-230:~# rm -r /mnt/master-vol-0dbfd1f3c60b8c509/var/etcd/data/member/ root@ip-172-20-116-230:~# rm -r /mnt/master-vol-0dbfd1f3c60b8c509/var/etcd/data/member/
``` ```
@ -175,8 +218,15 @@ $ kops edit instancegroup master-<availability-zone3>
### b - Add a new member to the etcd clusters ### b - Add a new member to the etcd clusters
```bash ```bash
$ kubectl --namespace=kube-system exec etcd-server-ip-172-20-36-161.ec2.internal -- etcdctl member add etcd-<availability-zone3> http://etcd-<availability-zone3>.internal.example.com:2380 $ kubectl --namespace=kube-system exec etcd-server-ip-172-20-36-161.ec2.internal -- etcdctl member add etcd-<availability-zone3-name> http://etcd-<availability-zone3-name>.internal.example.com:2380 \
$ kubectl --namespace=kube-system exec etcd-server-events-ip-172-20-36-161.ec2.internal -- etcdctl --endpoint http://127.0.0.1:4002 member add etcd-events-<availability-zone3> http://etcd-events-<availability-zone3>.internal.example.com:2381 && kubectl --namespace=kube-system exec etcd-server-events-ip-172-20-36-161.ec2.internal -- etcdctl --endpoint http://127.0.0.1:4002 member add etcd-events-<availability-zone3-name> http://etcd-events-<availability-zone3-name>.internal.example.com:2381
```
Example:
```bash
$ kubectl --namespace=kube-system exec etcd-server-ip-172-20-36-161.ec2.internal -- etcdctl member add etcd-c http://etcd-c.internal.example.com:2380 \
&& kubectl --namespace=kube-system exec etcd-server-events-ip-172-20-36-161.ec2.internal -- etcdctl --endpoint http://127.0.0.1:4002 member add etcd-events-c http://etcd-events-c.internal.example.com:2381
``` ```
### c - Launch the third master ### c - Launch the third master
@ -197,7 +247,7 @@ $ kops edit instancegroup master-<availability-zone3>
* Delete the containers and the data directories: * Delete the containers and the data directories:
```bash ```bash
root@ip-172-20-139-130:~# docker stop $(docker ps | grep "etcd:2.2.1" | awk '{print $1}') root@ip-172-20-139-130:~# docker stop $(docker ps | grep "gcr.io/google_containers/etcd" | awk '{print $1}')
root@ip-172-20-139-130:~# rm -r /mnt/master-vol-019796c3511a91b4f//var/etcd/data-events/member/ root@ip-172-20-139-130:~# rm -r /mnt/master-vol-019796c3511a91b4f//var/etcd/data-events/member/
root@ip-172-20-139-130:~# rm -r /mnt/master-vol-0c89fd6f6a256b686/var/etcd/data/member/ root@ip-172-20-139-130:~# rm -r /mnt/master-vol-0c89fd6f6a256b686/var/etcd/data/member/
``` ```