Example running OpenShift Origin as pod on Kubernetes
This commit is contained in:
derekwaynecarr
parent
ced265bd66
commit
a12f044851
|
|
@ -0,0 +1,110 @@
|
||||||
|
## OpenShift Origin example
|
||||||
|
|
||||||
|
This example shows how to run OpenShift Origin as a pod on an existing Kubernetes cluster.
|
||||||
|
|
||||||
|
This example demonstrates usage of a pod with a secret volume mount.
|
||||||
|
|
||||||
|
### Step 0: Prerequisites
|
||||||
|
|
||||||
|
This example assumes that you have a basic understanding of Kubernetes and that you have forked the repository and [turned up a Kubernetes cluster](https://github.com/GoogleCloudPlatform/kubernetes#contents):
|
||||||
|
|
||||||
|
This example has been tested against the **gce** and **vagrant** based KUBERNETES_PROVIDER.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ cd kubernetes
|
||||||
|
$ export KUBERNETES_PROVIDER=gce
|
||||||
|
$ hack/dev-build-and-up.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 1: Generate resources
|
||||||
|
|
||||||
|
The demonstration will require the following resources:
|
||||||
|
|
||||||
|
1. A Kubernetes Secret that contains information needed to securely communicate to your Kubernetes master as an administrator
|
||||||
|
2. A Kubernetes Pod that contains information for how to run OpenShift Origin that consumes this Secret securely
|
||||||
|
3. A Kubernetes Service that exposes OpenShift Origin API via an external load balancer
|
||||||
|
4. A Kubernetes Service that exposes OpenShift Origin UI via an external load balancer
|
||||||
|
|
||||||
|
To generate these resources, we will run a script that introspects your configured KUBERNETES_PROVIDER:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ examples/openshift-origin/resource-generator.sh
|
||||||
|
```
|
||||||
|
A Kubernetes Secret was generated that contains the following data:
|
||||||
|
|
||||||
|
1. kubeconfig: a valid kubeconfig file that is used by OpenShift Origin to communicate to the master
|
||||||
|
2. kube-ca: a certificate authority for the Kubernetes master
|
||||||
|
3. kube-auth-path: a Kubernetes authorization file
|
||||||
|
4. kube-cert: a Kubernetes certificate
|
||||||
|
5. kube-key: a Kubernetes key file
|
||||||
|
|
||||||
|
As required by a Kubernetes secret, each piece of data is base64 encoded - with no line wraps.
|
||||||
|
|
||||||
|
You can view the file by doing:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ cat examples/openshift-origin/secret.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Caution: This file contains all of the required information to operate as a Kubernetes admin on your cluster, so only share this file with trusted parties.
|
||||||
|
|
||||||
|
A Kubernetes Pod file was generated that can run OpenShift Origin on your cluster.
|
||||||
|
|
||||||
|
The OpenShift Origin pod file has a volume mount that references the Kubernetes secret we created to know how to work with the underlying Kubernetes provider.
|
||||||
|
|
||||||
|
You can view the file by doing:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ cat examples/openshift-origin/pod.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Finally, a Kubernetes service was generated for the UI and the API and available via an external load balancer:
|
||||||
|
|
||||||
|
``shell
|
||||||
|
$ cat examples/openshift-origin
|
||||||
|
|
||||||
|
### Step 2: Create the secret in Kubernetes
|
||||||
|
|
||||||
|
To provision the secret on Kubernetes:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ cluster/kubectl.sh create -f examples/openshift-origin/secret.json
|
||||||
|
```
|
||||||
|
|
||||||
|
You should see your secret resource was created by listing:
|
||||||
|
```shell
|
||||||
|
$ cluster/kubectl.sh get secrets
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Provisioning OpenShift Origin
|
||||||
|
|
||||||
|
To create the OpenShift Origin pod:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ cluster/kubectl.sh create -f examples/openshift-origin/pod.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 4: Provisioning OpenShift Origin Services
|
||||||
|
|
||||||
|
To create the OpenShift Origin Services that expose the API and UI:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ cluster/kubectl.sh create -f examples/openshift-origin/ui-service.json
|
||||||
|
$ cluster/kubectl.sh create -f examples/openshift-origin/api-service.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 5: Open Firewall Ports
|
||||||
|
|
||||||
|
If you are running on GCE, you need to open the following ports:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ gcloud compute instances list
|
||||||
|
|
||||||
|
FIND THE MINION NAME PREFIX
|
||||||
|
|
||||||
|
$ gcloud compute firewall-rules create openshift-origin-node-8444 --allow tcp:8444 --target-tags kubernetes-minion-prq8
|
||||||
|
$ gcloud compute firewall-rules create openshift-origin-node-8443 --allow tcp:8443 --target-tags kubernetes-minion-prq8
|
||||||
|
```
|
||||||
|
### Step 4: Try out OpenShift Origin
|
||||||
|
|
||||||
|
TODO add more detail here:
|
||||||
|
|
@ -0,0 +1,37 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Copyright 2014 Google Inc. All rights reserved.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
# Generates secret, creates secret on kube, creates pod on kube
|
||||||
|
|
||||||
|
set -o errexit
|
||||||
|
set -o nounset
|
||||||
|
set -o pipefail
|
||||||
|
|
||||||
|
ORIGIN=$(dirname "${BASH_SOURCE}")
|
||||||
|
KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
|
||||||
|
|
||||||
|
## Generate resources
|
||||||
|
${ORIGIN}/resource-generator.sh
|
||||||
|
|
||||||
|
## Create the secret
|
||||||
|
${KUBE_ROOT}/cluster/kubectl.sh create -f ${ORIGIN}/secret.json
|
||||||
|
|
||||||
|
## Create the pod
|
||||||
|
${KUBE_ROOT}/cluster/kubectl.sh create -f ${ORIGIN}/pod.json
|
||||||
|
|
||||||
|
## Create the services
|
||||||
|
${KUBE_ROOT}/cluster/kubectl.sh create -f ${ORIGIN}/api-service.json
|
||||||
|
${KUBE_ROOT}/cluster/kubectl.sh create -f ${ORIGIN}/ui-service.json
|
||||||
|
|
@ -0,0 +1,34 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Copyright 2014 Google Inc. All rights reserved.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
# Deletes pod, deletes secret
|
||||||
|
|
||||||
|
set -o errexit
|
||||||
|
set -o nounset
|
||||||
|
set -o pipefail
|
||||||
|
|
||||||
|
ORIGIN=$(dirname "${BASH_SOURCE}")
|
||||||
|
KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
|
||||||
|
|
||||||
|
## Delete the services
|
||||||
|
${KUBE_ROOT}/cluster/kubectl.sh delete services origin-api
|
||||||
|
${KUBE_ROOT}/cluster/kubectl.sh delete services origin-ui
|
||||||
|
|
||||||
|
## Delete the pod
|
||||||
|
${KUBE_ROOT}/cluster/kubectl.sh delete pods openshift
|
||||||
|
|
||||||
|
## Delete the secret
|
||||||
|
${KUBE_ROOT}/cluster/kubectl.sh delete secrets kubernetes-secret
|
||||||
|
|
@ -0,0 +1,18 @@
|
||||||
|
apiVersion: v1
|
||||||
|
clusters:
|
||||||
|
- cluster:
|
||||||
|
certificate-authority: /etc/secret-volume/kube-ca
|
||||||
|
server: https://146.148.35.28
|
||||||
|
name: kubernetes
|
||||||
|
contexts:
|
||||||
|
- context:
|
||||||
|
cluster: kubernetes
|
||||||
|
user: kubernetes-admin
|
||||||
|
name: kubernetes
|
||||||
|
current-context: kubernetes
|
||||||
|
kind: Config
|
||||||
|
preferences: {}
|
||||||
|
users:
|
||||||
|
- name: kubernetes-admin
|
||||||
|
user:
|
||||||
|
auth-path: /etc/secret-volume/kube-auth-path
|
||||||
|
|
@ -0,0 +1,198 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Copyright 2014 Google Inc. All rights reserved.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
# Generates pod and secret to deploy origin against configured Kubernetes provider
|
||||||
|
|
||||||
|
set -o errexit
|
||||||
|
set -o nounset
|
||||||
|
set -o pipefail
|
||||||
|
|
||||||
|
ORIGIN=$(dirname "${BASH_SOURCE}")
|
||||||
|
KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
|
||||||
|
source "${KUBE_ROOT}/cluster/kubectl.sh" > /dev/null 2>&1
|
||||||
|
|
||||||
|
# Check all prerequisites are on the path
|
||||||
|
HAVE_JQ=$(which jq)
|
||||||
|
if [[ -z ${HAVE_JQ} ]]; then
|
||||||
|
echo "Please install jq"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
HAVE_BASE64=$(which base64)
|
||||||
|
if [[ -z ${HAVE_BASE64} ]]; then
|
||||||
|
echo "Please install base64"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Capture information about your kubernetes cluster
|
||||||
|
TEMPLATE="--template=\"{{ index . \"current-context\" }}\""
|
||||||
|
CURRENT_CONTEXT=$( "${kubectl}" "${config[@]:+${config[@]}}" config view -o template "${TEMPLATE}" )
|
||||||
|
|
||||||
|
TEMPLATE="--template=\"{{ index . \"contexts\" ${CURRENT_CONTEXT} \"cluster\" }}\""
|
||||||
|
CURRENT_CLUSTER=$( "${kubectl}" "${config[@]:+${config[@]}}" config view -o template "${TEMPLATE}" )
|
||||||
|
|
||||||
|
TEMPLATE="--template=\"{{ index . \"contexts\" ${CURRENT_CONTEXT} \"user\" }}\""
|
||||||
|
CURRENT_USER=$( "${kubectl}" "${config[@]:+${config[@]}}" config view -o template "${TEMPLATE}" )
|
||||||
|
|
||||||
|
TEMPLATE="--template={{ index . \"clusters\" ${CURRENT_CLUSTER} \"certificate-authority\" }}"
|
||||||
|
CERTIFICATE_AUTHORITY=$( "${kubectl}" "${config[@]:+${config[@]}}" config view -o template "${TEMPLATE}" )
|
||||||
|
|
||||||
|
TEMPLATE="--template={{ index . \"clusters\" ${CURRENT_CLUSTER} \"server\" }}"
|
||||||
|
KUBE_MASTER=$( "${kubectl}" "${config[@]:+${config[@]}}" config view -o template "${TEMPLATE}" )
|
||||||
|
|
||||||
|
TEMPLATE="--template={{ index . \"users\" ${CURRENT_USER} \"auth-path\" }}"
|
||||||
|
AUTH_PATH=$( "${kubectl}" "${config[@]:+${config[@]}}" config view -o template "${TEMPLATE}" )
|
||||||
|
|
||||||
|
# Build an auth_path file to embed as a secret
|
||||||
|
AUTH_PATH_DATA=$(cat ${AUTH_PATH} )
|
||||||
|
KUBE_USER=$( echo ${AUTH_PATH_DATA} | jq '.User' )
|
||||||
|
KUBE_PASSWORD=$( echo ${AUTH_PATH_DATA} | jq '.Password' )
|
||||||
|
KUBE_CERT_FILE=$( echo ${AUTH_PATH_DATA} | jq '.CertFile' )
|
||||||
|
KUBE_KEY_FILE=$( echo ${AUTH_PATH_DATA} | jq '.KeyFile' )
|
||||||
|
|
||||||
|
cat <<EOF >"${ORIGIN}/origin-auth-path"
|
||||||
|
{
|
||||||
|
"User": ${KUBE_USER},
|
||||||
|
"Password": ${KUBE_PASSWORD},
|
||||||
|
"CAFile": "/etc/secret-volume/kube-ca",
|
||||||
|
"CertFile": "/etc/secret-volume/kube-cert",
|
||||||
|
"KeyFile": "/etc/secret-volume/kube-key"
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Collect all the secrets and encode as base64
|
||||||
|
ORIGIN_KUBECONFIG_DATA=$( cat ${ORIGIN}/origin-kubeconfig.yaml | base64 --wrap=0)
|
||||||
|
ORIGIN_CERTIFICATE_AUTHORITY_DATA=$(cat ${CERTIFICATE_AUTHORITY} | base64 --wrap=0)
|
||||||
|
ORIGIN_AUTH_PATH_DATA=$(cat ${ORIGIN}/origin-auth-path | base64 --wrap=0)
|
||||||
|
ORIGIN_CERT_FILE=$( cat ${KUBE_CERT_FILE//\"/} | base64 --wrap=0)
|
||||||
|
ORIGIN_KEY_FILE=$( cat ${KUBE_KEY_FILE//\"/} | base64 --wrap=0)
|
||||||
|
|
||||||
|
cat <<EOF >"${ORIGIN}/secret.json"
|
||||||
|
{
|
||||||
|
"apiVersion": "v1beta2",
|
||||||
|
"kind": "Secret",
|
||||||
|
"id": "kubernetes-secret",
|
||||||
|
"data": {
|
||||||
|
"kubeconfig": "${ORIGIN_KUBECONFIG_DATA}",
|
||||||
|
"kube-ca": "${ORIGIN_CERTIFICATE_AUTHORITY_DATA}",
|
||||||
|
"kube-auth-path": "${ORIGIN_AUTH_PATH_DATA}",
|
||||||
|
"kube-cert": "${ORIGIN_CERT_FILE}",
|
||||||
|
"kube-key": "${ORIGIN_KEY_FILE}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
echo "Generated Kubernetes Secret file: ${ORIGIN}/secret.json"
|
||||||
|
|
||||||
|
# Generate an OpenShift Origin pod
|
||||||
|
# TODO: In future, move this to a replication controller when we are not running etcd in container
|
||||||
|
|
||||||
|
cat <<EOF >"${ORIGIN}/pod.json"
|
||||||
|
{
|
||||||
|
"apiVersion": "v1beta1",
|
||||||
|
"id": "openshift",
|
||||||
|
"kind": "Pod",
|
||||||
|
"labels": {"name": "origin"},
|
||||||
|
"desiredState": {
|
||||||
|
"manifest": {
|
||||||
|
"containers": [
|
||||||
|
{
|
||||||
|
"command": [
|
||||||
|
"start",
|
||||||
|
"master",
|
||||||
|
"--kubernetes=${KUBE_MASTER}",
|
||||||
|
"--kubeconfig=/etc/secret-volume/kubeconfig",
|
||||||
|
"--public-kubernetes=https://10.245.1.3:8443",
|
||||||
|
"--public-master=https://10.245.1.3:8443",
|
||||||
|
],
|
||||||
|
"image": "openshift/origin:latest",
|
||||||
|
"imagePullPolicy": "PullIfNotPresent",
|
||||||
|
"name": "origin",
|
||||||
|
"ports": [
|
||||||
|
{
|
||||||
|
"name": "https-api",
|
||||||
|
"containerPort": 8443,
|
||||||
|
"hostPort": 8443,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "https-ui",
|
||||||
|
"containerPort": 8444,
|
||||||
|
"hostPort": 8444,
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"volumeMounts": [
|
||||||
|
{
|
||||||
|
"mountPath": "/etc/secret-volume",
|
||||||
|
"name": "secret-volume",
|
||||||
|
"readOnly": true
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"restartPolicy": {
|
||||||
|
"never": {}
|
||||||
|
},
|
||||||
|
"version": "v1beta2",
|
||||||
|
"volumes": [
|
||||||
|
{
|
||||||
|
"name": "secret-volume",
|
||||||
|
"source": {
|
||||||
|
"secret": {
|
||||||
|
"target": {
|
||||||
|
"kind": "Secret",
|
||||||
|
"name": "kubernetes-secret",
|
||||||
|
"namespace": "default"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
echo "Generated Kubernetes Pod file: ${ORIGIN}/pod.json"
|
||||||
|
|
||||||
|
cat <<EOF >"${ORIGIN}/api-service.json"
|
||||||
|
{
|
||||||
|
"apiVersion": "v1beta1",
|
||||||
|
"kind": "Service",
|
||||||
|
"id": "origin-api",
|
||||||
|
"port": 8443,
|
||||||
|
"containerPort": "https-api",
|
||||||
|
"selector": { "name": "origin" },
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
echo "Generated Kubernetes Service file: ${ORIGIN}/api-service.json"
|
||||||
|
|
||||||
|
cat <<EOF >"${ORIGIN}/ui-service.json"
|
||||||
|
{
|
||||||
|
"apiVersion": "v1beta1",
|
||||||
|
"kind": "Service",
|
||||||
|
"id": "origin-ui",
|
||||||
|
"port": 8444,
|
||||||
|
"containerPort": "https-ui",
|
||||||
|
"selector": { "name": "origin" },
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
echo "Generated Kubernetes Service file: ${ORIGIN}/ui-service.json"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Loading…
Reference in New Issue