kubernetes
/
autoscaler
mirror of https://github.com/kubernetes/autoscaler.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
autoscaler/cluster-autoscaler/cloudprovider/brightbox/README.md

5.8 KiB
Raw Permalink Blame History

Cluster Autoscaler for Brightbox Cloud

This cloud provider implements the autoscaling function for Brightbox Cloud. The autoscaler should work on any Kubernetes clusters running on Brightbox Cloud, however the approach is tailored to clusters built with the Kubernetes Cluster Builder

How Autoscaler works on Brightbox Cloud

The autoscaler looks for Server Groups named after the cluster-name option passed to the autoscaler (--cluster-name).

A group named with a suffix of the cluster-name (e.g. k8s-worker.k8s-test.cluster.local) is a candidate to be a scaling group. The autoscaler will then check the description to see if it is a pair of integers separated by a colon (e.g. 1:4). If it finds those numbers then they will become the minimum and maximum server size for that group, and autoscaler will attempt to scale the group between those sizes.

The type of server, the image used and the target zone will be dynamically determined from the existing members. If these differ, or there are no existing servers, autoscaler will log an error and will not scale that group.

A group named precisely the same as the cluster-name (e.g. k8s-test.cluster.local) is considered to be the default cluster group and all autoscaled servers created are placed within it as well as the scaling group.

The Brightbox Cloud provider only supports auto-discovery mode using this pattern. node-group-auto-discovery and nodes options are effectively ignored.

Cluster configuration

If you are using the Kubernetes Cluster Builder set the worker_min and worker_max values to scale the worker group, and the storage_min and storage_max values to scale the storage group.

The Cluster Builder will ensure the group name and description are updated with the correct values in the format that autoscaler can recognise.

Generally it is best to keep the min and the count values to be the same within the Cluster Buider and let autoscaler create and destroy servers dynamically up the the max value.

While using autoscaler you may find that the Cluster Builder recreates servers that have been scaled down, if you use the manifests to maintain the cluster for other reasons (changing the management address for example). This is a limitation of the Terraform state database, and autoscaler will scale the cluster back down during the next few minutes.

Autoscaler Brightbox cloudprovider configuration

The Brightbox Cloud cloudprovider is configured via Environment Variables suppied to the autoscaler pod. The easiest way to do this is to create a secret containing the variables within the kube-system namespace.

apiVersion: v1
kind: Secret
metadata:
  name: brightbox-credentials
  namespace: kube-system
type: Opaque
data:
  BRIGHTBOX_API_URL: <base 64 of api URL>
  BRIGHTBOX_CLIENT: <bas64 of Brighbox Cloud client id>
  BRIGHTBOX_CLIENT_SECRET: <base64 of Brightbox Cloud client id secret>
  BRIGHTBOX_KUBE_JOIN_COMMAND: <base64 of cluster join command>
  BRIGHTBOX_KUBE_VERSION: <base 64 of installed k8s version>

The join command can be obtained from the kubeadm token command

$ kubeadm token create --ttl 0 --description 'Cluster autoscaling token' --print-join-command

Brightbox API Clients can be created in the Brightbox Manager

Cluster Configuration

The Kubernetes Cluster Builder creates a brightbox-credentials secret in the kube-system namespace ready to use.

Checking the environment

You can check the brightbox-credentials secret by running the check-env job from the examples directory.

$ kubectl apply -f examples/check-env.yaml
job.batch/check-env created
$ kubectl -n kube-system logs job/check-env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=check-env-hbh6m
_BASH_GPG_KEY=7C0135FB088AAF6C66C650B9BB5869F064EA74AB
_BASH_VERSION=5.0
_BASH_PATCH_LEVEL=0
_BASH_LATEST_PATCH=11
BRIGHTBOX_KUBE_VERSION=1.17.0
...
$ kubectl delete -f examples/check-env.yaml
job.batch "check-env" deleted

Running the Autoscaler

  1. Clone this repository and change into this directory.
  2. Edit the examples/config.rb file and adjust the config hash.
  3. Alter the cluster name if required. (If you are using the Kubernetes Cluster Builder, this will be cluster_name and cluster_domainname joined with a '.')

Then generate and apply the manifests

$ make deploy TAG=<version>

where TAG is the version you wish to use (1.17, 1.18, etc.)

As the Brightbox cloud-provider auto-detects and potentially scales all the worker groups, the example deployment file runs the autoscaler on the master nodes. This avoids it accidentally killing itself.

Viewing the cluster-autoscaler options

Cluster autoscaler has many options that can be adjusted to better fit the needs of your application. To view them run

$ kubectl create job ca-options --image=brightbox/cluster-autoscaler-brightbox:dev -- ./cluster-autoscaler -h
$ kubectl log job/ca-options

Remove the job in the normal way with kubectl delete job/ca-options

You can read more details about some of the options in the main FAQ

Building the Brightbox Cloud autoscaler

Extract the repository to a machine running docker and then run the make command

$ make build

This builds an autoscaler containing only the Brightbox Cloud provider, tagged as brightbox/cluster-autoscaler-brightbox:dev. To build any other version add a TAG variable

make build TAG=1.1x