From 6c54d16ebc07cbe71b1fbddf3a9fd0d97f8112b5 Mon Sep 17 00:00:00 2001 From: "Madhusudan.C.S" Date: Fri, 17 Mar 2017 01:39:09 -0700 Subject: [PATCH 1/3] Update kubefed documentation. --- .../set-up-cluster-federation-kubefed.md | 199 +++++++++++++++++- 1 file changed, 192 insertions(+), 7 deletions(-) diff --git a/docs/tutorials/federation/set-up-cluster-federation-kubefed.md b/docs/tutorials/federation/set-up-cluster-federation-kubefed.md index 4af8db60b5..ef7e9a01be 100644 --- a/docs/tutorials/federation/set-up-cluster-federation-kubefed.md +++ b/docs/tutorials/federation/set-up-cluster-federation-kubefed.md @@ -16,7 +16,7 @@ existing federation control plane. This guide explains how to administer a Kubernetes Cluster Federation using `kubefed`. -> Note: `kubefed` is an alpha feature in Kubernetes 1.5. +> Note: `kubefed` is in beta as of Kubernetes 1.6. ## Prerequisites @@ -40,7 +40,7 @@ of available binaries on the [release page](https://github.com/kubernetes/kubern ```shell -curl -O https://storage.googleapis.com/kubernetes-release/release/v1.5.2/kubernetes-client-linux-amd64.tar.gz +curl -O https://storage.googleapis.com/kubernetes-release/release/{{page.version}}/kubernetes-client-linux-amd64.tar.gz tar -xzvf kubernetes-client-linux-amd64.tar.gz sudo cp kubernetes/client/bin/kubefed /usr/local/bin sudo chmod +x /usr/local/bin/kubefed @@ -84,18 +84,28 @@ the following: * Federation name * `--host-cluster-context`, the `kubeconfig` context for the host cluster +* `--dns-provider`, one of `'google-clouddns'`, `aws-route53` or `coredns` * `--dns-zone-name`, a domain name suffix for your federated services +If your host cluster is running in a non-cloud environment or an +environment that doesn't support common cloud primitives such as +load balancers, you might need additional flags. Please see the +[on-premises host clusters](#on-premises-host-clusters) section below. + The following example command deploys a federation control plane with the name `fellowship`, a host cluster context `rivendell`, and the -domain suffix `example.com`: +domain suffix `example.com.`: ```shell -kubefed init fellowship --host-cluster-context=rivendell --dns-zone-name="example.com" +kubefed init fellowship \ + --host-cluster-context=rivendell \ + --dns-provider="google-clouddns" \ + --dns-zone-name="example.com." ``` The domain suffix specified in `--dns-zone-name` must be an existing domain that you control, and that is programmable by your DNS provider. +It must also end with a trailing dot. `kubefed init` sets up the federation control plane in the host cluster and also adds an entry for the federation API server in your @@ -110,6 +120,158 @@ kubectl config use-context fellowship where `fellowship` is the name of your federation. +### Basic and token authentication support + +`kubefed init` by default only generates TLS certificates and keys +to authenticate with the federation API server and writes them to +your local kubeconfig file. If you wish to enable basic authentication +or token authentication for debugging purposes, you can enable them by +passing the `--apiserver-enable-basic-auth` flag or the +`--apiserver-enable-token-auth` flag. + +```shell +kubefed init fellowship \ + --host-cluster-context=rivendell \ + --dns-provider="google-clouddns" \ + --dns-zone-name="example.com." \ + --apiserver-enable-basic-auth=true \ + --apiserver-enable-token-auth=true +``` + +### Passing command line arguments to federation components + +`kubefed init` bootstraps a federation control plane with default +arguments to federation API server and federation controller manager. +Some of these arguments are derived from `kubefed init`'s flags. +However, you can override these command line arguments by passing +them via the appropriate override flags. + +You can override the federation API server arguments by passing them +to `--apiserver-arg-overrides` and override the federation controller +manager arguments by passing them to +`--controllermanager-arg-overrides`. + +```shell +kubefed init fellowship \ + --host-cluster-context=rivendell \ + --dns-provider="google-clouddns" \ + --dns-zone-name="example.com." \ + --apiserver-arg-overrides="--anonymous-auth=false, --v=4" \ + --controllermanager-arg-overrides="--controllers=services=false" +``` + +### Configuring DNS provider + +Federated service controller programs a DNS provider to expose +federated services via DNS names. Certain cloud providers +automatically provide the configuration required to program the +DNS provider if the host cluster's cloud provider is same as the DNS +provider. In all other cases, you have to provide the DNS provider +configuration to your federation controller manager which will in-turn +be passed to the federated service controller. You can provide this +configuration to federation controller manager by storing it in a file +and passing the file's local filesystem path to `kubefed init`'s +`--dns-provider-config` flag. For example, save the config below in +`$HOME/coredns-provider.conf`. + +```ini +[Global] +etcd-endpoints = http://etcd-cluster.ns:2379 +zones = example.com +``` + +And then pass this file to `kubefed init`: + +```shell +kubefed init fellowship \ + --host-cluster-context=rivendell \ + --dns-provider="coredns" \ + --dns-zone-name="example.com." \ + --dns-provider-config="$HOME/coredns-provider.conf" +``` + +### On-premises host clusters + +#### API server service type + +`kubefed init` exposes the federation API server as a Kubernetes +[service](/docs/user-guide/services/) on the host cluster. By default, +this service is exposed as a +[load balanced service](/docs/user-guide/services/#type-loadbalancer). +Most on-premises and bare-metal enviroments, and some cloud +environments lack support for load balanced services. `kubefed init` +allows exposing the federation API server as a +[`NodePort` service](/docs/user-guide/services/#type-nodeport) on +such environments. This can be accomplished by passing +the `--api-server-service-type=NodePort` flag. You can also specify +the preferred address to advertise the federation API server by +passing the `--api-server-advertise-address=` +flag. Otherwise, one of the host cluster's node address is chosen as +the default. + +```shell +kubefed init fellowship \ + --host-cluster-context=rivendell \ + --dns-provider="google-clouddns" \ + --dns-zone-name="example.com." \ + --api-server-service-type="NodePort" \ + --api-server-advertise-address="10.0.10.20" +``` + +#### Dynamic provisioning of [`etcd`](https://coreos.com/etcd/docs/latest/) storage + +Federation control plane stores its state in +[`etcd`](https://coreos.com/etcd/docs/latest/). +[`etcd`](https://coreos.com/etcd/docs/latest/) data must be stored in +a persistent storage volume to ensure correct operation across +federation control plane restarts. On host clusters that support +[dynamic provisioning of storage volumes](/docs/user-guide/persistent-volumes/#dynamic), +`kubefed init` dynamically provisions a +[`PersistentVolume`](/docs/user-guide/persistent-volumes/#persistent-volumes) +and binds it to a +[`PersistentVolumeClaim`](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) +to store [`etcd`](https://coreos.com/etcd/docs/latest/) data. If your +host cluster doesn't support dynamic provisioning, then you pass +`--etcd-persistent-storage=false` to `kubefed init` to disable +storage provisioning and provide your own +[`PersistentVolume`](/docs/user-guide/persistent-volumes/#persistent-volumes) and +[`PersistentVolumeClaim`](/docs/user-guide/persistent-volumes/#persistentvolumeclaims). + +```shell +kubefed init fellowship \ + --host-cluster-context=rivendell \ + --dns-provider="google-clouddns" \ + --dns-zone-name="example.com." \ + --etcd-persistent-storage=false +``` + +`kubefed init` still doesn't support attaching an existing +[`PersistentVolumeClaim`](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) +to the federation control plane that it bootstraps. We are planning to +support this in a future version of `kubefed`. + +#### CoreDNS support + +Federated services now support [CoreDNS](https://coredns.io/) as one +of the DNS providers. If you are running your clusters and federation +in an environment that does not have access to cloud-based DNS +providers, then you can run your own [CoreDNS](https://coredns.io/) +instance and publish the federated service DNS names to that server. + +You can configure your federation to use +[CoreDNS](https://coredns.io/), by passing appropriate values to +`kubefed init`'s `--dns-provider` and `--dns-provider-config` flags. + +```shell +kubefed init fellowship \ + --host-cluster-context=rivendell \ + --dns-provider="coredns" \ + --dns-zone-name="example.com." \ + --dns-provider-config="$HOME/coredns-provider.conf" +``` + +For more information see +[Setting up CoreDNS as DNS provider for Cluster Federation](/docs/tutorials/federation/set-up-coredns-provider-federation/) ## Adding a cluster to a federation @@ -121,8 +283,19 @@ To use `kubefed join`, you'll need to provide the name of the cluster you want to add to the federation, and the `--host-cluster-context` for the federation control plane's host cluster. +> Note: The name that you provide to the `join` command is used as the +joining cluster's identity in federation. This name should adhere to +the rules described in the +[identifiers doc](/docs/user-guide/identifiers/#names). If the context +corresponding to your joining cluster conforms to these rules then you +can use the same name in the join command. Otherwise, you will have to +choose a different name for your cluster's identity. For more +information, please see the +[naming rules and customization](#naming-rules-and-customization) +section below. + The following example command adds the cluster `gondor` to the -federation with host cluster `rivendell`: +federation running on host cluster `rivendell`: ``` kubefed join gondor --host-cluster-context=rivendell @@ -136,8 +309,9 @@ cluster. ### Naming rules and customization -The cluster name you supply to `kubefed join` must be a valid RFC 1035 -label. +The cluster name you supply to `kubefed join` must be a valid +[RFC 1035](https://www.ietf.org/rfc/rfc1035.txt) label and are +enumerated in the [Identifiers doc](/docs/user-guide/identifiers/#names). Furthermore, federation control plane requires credentials of the joined clusters to operate on them. These credentials are obtained @@ -182,6 +356,17 @@ specification, all you need to do is supply the secret name via the `--secret-name` flag. `kubefed join` automatically creates the secret for you. +### `kube-dns` configuration + +`kube-dns` configuration must be updated in each joining cluster to +enable federated service discovery. If the joining Kubernetes cluster +is version 1.5 or newer and your `kubefed` version 1.6 or later, then +this configuration is automatically managed for you when the clusters +are joined or unjoined using `kubefed` `join` or `unjoin` commands. + +In all other cases, you must update `kube-dns` configuration manually +as described in the +[Updating KubeDNS section of the admin guide](/docs/admin/federation/) ## Removing a cluster from a federation From 5b50e8d04e27ca535307bebcdff531a99727a059 Mon Sep 17 00:00:00 2001 From: "Madhusudan.C.S" Date: Fri, 17 Mar 2017 22:36:26 -0700 Subject: [PATCH 2/3] Address review comments. --- .../tutorials/federation/set-up-cluster-federation-kubefed.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/federation/set-up-cluster-federation-kubefed.md b/docs/tutorials/federation/set-up-cluster-federation-kubefed.md index ef7e9a01be..ae246bbac0 100644 --- a/docs/tutorials/federation/set-up-cluster-federation-kubefed.md +++ b/docs/tutorials/federation/set-up-cluster-federation-kubefed.md @@ -177,7 +177,7 @@ and passing the file's local filesystem path to `kubefed init`'s ```ini [Global] etcd-endpoints = http://etcd-cluster.ns:2379 -zones = example.com +zones = example.com. ``` And then pass this file to `kubefed init`: @@ -218,7 +218,7 @@ kubefed init fellowship \ --api-server-advertise-address="10.0.10.20" ``` -#### Dynamic provisioning of [`etcd`](https://coreos.com/etcd/docs/latest/) storage +#### Dynamic provisioning of etcd storage Federation control plane stores its state in [`etcd`](https://coreos.com/etcd/docs/latest/). From b4ffa837a59642d80c50a2cbbcc010a9bd4239d8 Mon Sep 17 00:00:00 2001 From: "Madhusudan.C.S" Date: Mon, 20 Mar 2017 13:29:59 -0700 Subject: [PATCH 3/3] Address review comments. --- .../set-up-cluster-federation-kubefed.md | 99 ++++++++++++++----- 1 file changed, 74 insertions(+), 25 deletions(-) diff --git a/docs/tutorials/federation/set-up-cluster-federation-kubefed.md b/docs/tutorials/federation/set-up-cluster-federation-kubefed.md index ae246bbac0..f0f97dcfcd 100644 --- a/docs/tutorials/federation/set-up-cluster-federation-kubefed.md +++ b/docs/tutorials/federation/set-up-cluster-federation-kubefed.md @@ -16,7 +16,7 @@ existing federation control plane. This guide explains how to administer a Kubernetes Cluster Federation using `kubefed`. -> Note: `kubefed` is in beta as of Kubernetes 1.6. +> Note: `kubefed` is a beta feature in Kubernetes 1.6. ## Prerequisites @@ -27,21 +27,37 @@ for installation instructions for your platform. ## Getting `kubefed` -Download the client tarball corresponding to Kubernetes version 1.5 -or later -[from the release page](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md), -extract the binaries in the tarball to one of the directories -in your `$PATH` and set the executable permission on those binaries. +Download the client tarball corresponding to the latest release and +extract the binaries in the tarball with the commands: -Note: The URL in the curl command below downloads the binaries for -Linux amd64. If you are on a different platform, please use the URL -for the binaries appropriate for your platform. You can find the list -of available binaries on the [release page](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#client-binaries-1). +```shell +# Linux +curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s +https://storage.googleapis.com/kubernetes-release/release/stable.txt)/kubernetes-client-linux-amd64.tar.gz +tar -xzvf kubernetes-client-linux-amd64.tar.gz + +# OS X +curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s +https://storage.googleapis.com/kubernetes-release/release/stable.txt)/kubernetes-client-darwin-amd64.tar.gz +tar -xzvf kubernetes-client-darwin-amd64.tar.gz + +# Windows +curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s +https://storage.googleapis.com/kubernetes-release/release/stable.txt)/kubernetes-client-windows-amd64.tar.gz +tar -xzvf kubernetes-client-windows-amd64.tar.gz +``` + +> Note: The URLs in the curl commands above download the binaries for +`amd64`. If you are on a different architecture, please use a URL +appropriate for your architecture. You can find the list of available +binaries on the +[release page](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#client-binaries-1). + +Copy the extracted binaries to one of the directories in your `$PATH` +and set the executable permission on those binaries. ```shell -curl -O https://storage.googleapis.com/kubernetes-release/release/{{page.version}}/kubernetes-client-linux-amd64.tar.gz -tar -xzvf kubernetes-client-linux-amd64.tar.gz sudo cp kubernetes/client/bin/kubefed /usr/local/bin sudo chmod +x /usr/local/bin/kubefed sudo cp kubernetes/client/bin/kubectl /usr/local/bin @@ -109,7 +125,7 @@ It must also end with a trailing dot. `kubefed init` sets up the federation control plane in the host cluster and also adds an entry for the federation API server in your -local kubeconfig. Note that in the alpha release in Kubernetes 1.5, +local kubeconfig. Note that in the beta release in Kubernetes 1.6, `kubefed init` does not automatically set the current context to the newly deployed federation. You can set the current context manually by running: @@ -156,13 +172,13 @@ kubefed init fellowship \ --host-cluster-context=rivendell \ --dns-provider="google-clouddns" \ --dns-zone-name="example.com." \ - --apiserver-arg-overrides="--anonymous-auth=false, --v=4" \ + --apiserver-arg-overrides="--anonymous-auth=false,--v=4" \ --controllermanager-arg-overrides="--controllers=services=false" ``` -### Configuring DNS provider +### Configuring a DNS provider -Federated service controller programs a DNS provider to expose +The Federated service controller programs a DNS provider to expose federated services via DNS names. Certain cloud providers automatically provide the configuration required to program the DNS provider if the host cluster's cloud provider is same as the DNS @@ -218,7 +234,7 @@ kubefed init fellowship \ --api-server-advertise-address="10.0.10.20" ``` -#### Dynamic provisioning of etcd storage +#### Provisioning storage for etcd Federation control plane stores its state in [`etcd`](https://coreos.com/etcd/docs/latest/). @@ -231,12 +247,44 @@ federation control plane restarts. On host clusters that support and binds it to a [`PersistentVolumeClaim`](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) to store [`etcd`](https://coreos.com/etcd/docs/latest/) data. If your -host cluster doesn't support dynamic provisioning, then you pass -`--etcd-persistent-storage=false` to `kubefed init` to disable -storage provisioning and provide your own -[`PersistentVolume`](/docs/user-guide/persistent-volumes/#persistent-volumes) and +host cluster doesn't support dynamic provisioning, you can also +statically provision a +[`PersistentVolume`](/docs/user-guide/persistent-volumes/#persistent-volumes). +`kubefed init` creates a +[`PersistentVolumeClaim`](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) +that has the following configuration: + +```yaml +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + annotations: + volume.alpha.kubernetes.io/storage-class: "yes" + labels: + app: federated-cluster + name: fellowship-federation-apiserver-etcd-claim + namespace: federation-system +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi +``` + +To statically provision a +[`PersistentVolume`](/docs/user-guide/persistent-volumes/#persistent-volumes), +you must ensure that the +[`PersistentVolume`](/docs/user-guide/persistent-volumes/#persistent-volumes) +that you create has the matching storage class, access mode and +at least as much capacity as the requested [`PersistentVolumeClaim`](/docs/user-guide/persistent-volumes/#persistentvolumeclaims). +Alternatively, you can disable persistent storage completely +by passing `--etcd-persistent-storage=false` to `kubefed init`. +However, we do not recommended this because your federation control +plane cannot survive restarts in this mode. + ```shell kubefed init fellowship \ --host-cluster-context=rivendell \ @@ -360,9 +408,10 @@ for you. `kube-dns` configuration must be updated in each joining cluster to enable federated service discovery. If the joining Kubernetes cluster -is version 1.5 or newer and your `kubefed` version 1.6 or later, then -this configuration is automatically managed for you when the clusters -are joined or unjoined using `kubefed` `join` or `unjoin` commands. +is version 1.5 or newer and your `kubefed` is version 1.6 or newer, +then this configuration is automatically managed for you when the +clusters are joined or unjoined using `kubefed` `join` or `unjoin` +commands. In all other cases, you must update `kube-dns` configuration manually as described in the @@ -382,7 +431,7 @@ kubefed unjoin gondor --host-cluster-context=rivendell ## Turning down the federation control plane: Proper cleanup of federation control plane is not fully implemented in -this alpha release of `kubefed`. However, for the time being, deleting +this beta release of `kubefed`. However, for the time being, deleting the federation system namespace should remove all the resources except the persistent storage volume dynamically provisioned for the federation control plane's etcd. You can delete the federation