add a guide to set up kubemark cluster
This commit is contained in:
jingxueli
parent
29871d3459
commit
2f0ce0fe6b
|
|
@ -0,0 +1,93 @@
|
|||
apiVersion: v1
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
name: hollow-node
|
||||
namespace: kubemark
|
||||
spec:
|
||||
replicas: {{numreplicas}}
|
||||
selector:
|
||||
name: hollow-node
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
name: hollow-node
|
||||
spec:
|
||||
initContainers:
|
||||
- name: init-inotify-limit
|
||||
image: docker.io/busybox:latest
|
||||
command: ['sysctl', '-w', 'fs.inotify.max_user_instances=200']
|
||||
securityContext:
|
||||
privileged: true
|
||||
volumes:
|
||||
- name: kubeconfig-volume
|
||||
secret:
|
||||
secretName: kubeconfig
|
||||
- name: logs-volume
|
||||
hostPath:
|
||||
path: /var/log
|
||||
containers:
|
||||
- name: hollow-kubelet
|
||||
image: {{kubemark_image_registry}}/kubemark:{{kubemark_image_tag}}
|
||||
ports:
|
||||
- containerPort: 4194
|
||||
- containerPort: 10250
|
||||
- containerPort: 10255
|
||||
env:
|
||||
- name: CONTENT_TYPE
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: node-configmap
|
||||
key: content.type
|
||||
- name: NODE_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- /kubemark --morph=kubelet --name=$(NODE_NAME) --kubeconfig=/kubeconfig/kubelet.kubeconfig $(CONTENT_TYPE) --alsologtostderr --v=2
|
||||
volumeMounts:
|
||||
- name: kubeconfig-volume
|
||||
mountPath: /kubeconfig
|
||||
readOnly: true
|
||||
- name: logs-volume
|
||||
mountPath: /var/log
|
||||
resources:
|
||||
requests:
|
||||
cpu: 20m
|
||||
memory: 50M
|
||||
securityContext:
|
||||
privileged: true
|
||||
- name: hollow-proxy
|
||||
image: {{kubemark_image_registry}}/kubemark:{{kubemark_image_tag}}
|
||||
env:
|
||||
- name: CONTENT_TYPE
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: node-configmap
|
||||
key: content.type
|
||||
- name: NODE_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- /kubemark --morph=proxy --name=$(NODE_NAME) --use-real-proxier=false --kubeconfig=/kubeconfig/kubeproxy.kubeconfig $(CONTENT_TYPE) --alsologtostderr --v=2
|
||||
volumeMounts:
|
||||
- name: kubeconfig-volume
|
||||
mountPath: /kubeconfig
|
||||
readOnly: true
|
||||
- name: logs-volume
|
||||
mountPath: /var/log
|
||||
resources:
|
||||
requests:
|
||||
cpu: 20m
|
||||
memory: 50M
|
||||
tolerations:
|
||||
- effect: NoExecute
|
||||
key: node.kubernetes.io/unreachable
|
||||
operator: Exists
|
||||
- effect: NoExecute
|
||||
key: node.kubernetes.io/not-ready
|
||||
operator: Exists
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
## Introduction
|
||||
This document serves to understand how to set up kubemark cluster given that a base cluster (to run hollow-node pods) and separate master (to act as master for the hollow nodes) are already present.
|
||||
|
||||
## Precondition
|
||||
You need kubemark master and external cluster to set up a kubemark cluster.
|
||||
|
||||
The functions are as follows:
|
||||
|
||||
- kubemark master: can be StandAlone or HA, used to be the kubemark cluster's master
|
||||
- external cluster: used to create hollow nodes for the kubemark cluster
|
||||
|
||||
## Steps:
|
||||
1. Build kubemark image
|
||||
|
||||
If you want to build/use your own kubemark image, do as follows. Otherwise skip this section and just use the latest image `staging-k8s.gcr.io/kubemark:latest` from public repo.
|
||||
|
||||
- i. pull kubernetes code
|
||||
|
||||
```
|
||||
cd $GOPATH/src/k8s.io/
|
||||
git clone git@github.com:kubernetes/kubernetes.git
|
||||
```
|
||||
|
||||
- ii. build kubemark binary
|
||||
|
||||
```
|
||||
./hack/build-go.sh cmd/kubemark/
|
||||
cp $GOPATH/src/k8s.io/kubernetes/_output/bin/kubemark $GOPATH/src/k8s.io/kubernetes/cluster/images/kubemark/
|
||||
```
|
||||
|
||||
- iii. build kubemark image
|
||||
|
||||
```
|
||||
cd $GOPATH/src/k8s.io/kubernetes/cluster/images/kubemark/
|
||||
make build
|
||||
```
|
||||
|
||||
Then you can get the image named `staging-k8s.gcr.io/kubemark:latest` locally.
|
||||
|
||||
- iv. push kubemark image
|
||||
|
||||
```
|
||||
docker tag staging-k8s.gcr.io/kubemark:latest {{kubemark_image_registry}}/kubemark:{{kubemark_image_tag}}
|
||||
docker push {{kubemark_image_registry}}/kubemark:{{kubemark_image_tag}}
|
||||
```
|
||||
|
||||
2. Create hollow nodes
|
||||
|
||||
- i. create namespace, configmap and secret
|
||||
|
||||
Copy kubemark master's kubeconfig which is used to configure access, put it on a master of external cluster, rename it as config.
|
||||
|
||||
```
|
||||
kubectl create ns kubemark
|
||||
kubectl create configmap node-configmap -n kubemark --from-literal=content.type="test-cluster"
|
||||
kubectl create secret generic kubeconfig --type=Opaque --namespace=kubemark --from-file=kubelet.kubeconfig=config --from-file=kubeproxy.kubeconfig=config
|
||||
```
|
||||
|
||||
- ii. apply yaml to create hollow nodes
|
||||
|
||||
You can use `hollow-node_simplified_template.yaml` in the current directory, or use `hollow-node_template.yaml` in `kubernetes/test/kubemark/resources/`.
|
||||
|
||||
Note:
|
||||
|
||||
- the parameters `{{numreplicas}}` means the number of hollow nodes in the kubemark cluster
|
||||
- the parameters `{{numreplicas}}`, `{{kubemark_image_registry}}` and `{{kubemark_image_tag}}` need to be filled in the simplified template
|
||||
- your external cluster should have enough resources to be able to run `{{numreplicas}}` no. of hollow-node pods
|
||||
|
||||
```
|
||||
kubectl create -f hollow-node_simplified_template.yaml
|
||||
```
|
||||
|
||||
Waiting for these hollow-node pods to be running. Then you can see these pods register as kubemark master's nodes.
|
||||
|
||||
Finally, kubemark master and external cluster set up the kubemark cluster.
|
||||
Loading…
Reference in New Issue