kops-controller manages a few node-role node-labels. We
now remove any extra managed labels that land on the node.
This means we will now actively remove the extra node label if we
previously erroneously applied to a control-plane node; previous code
changes stopped applying it.
If we deserialize the yaml, we don't go through the version-conversion
logic. That logic maps from Master -> ControlPlane, so without that
logic we see unexpected values in the "string enums".
We were silently ignoring unknown roles, which makes it hard to know
when our expectations aren't met. It looks like the rename of the
role from "Master" to "ControlPlane" may have caused some drift
against our expectations also.
We add the hosts plugin to CoreDNS, and we populate a ConfigMap from
kops-controller (when in gossip mode).
This enables resolution of the internal apiserver DNS name from Pods,
even when gossip mode (k8s.local) is in use. This should fix the
failing e2e tests which are assuming that the name in the JWT token is
resolvable from inside the cluster.
This is also a possible step towards a simpler gossip mode, now that
we have a central controller.
When using a "mixed instance policy"[1] instance group spot and onDemand nodes are part of the same
ASG. The ASG handles the percentage of spot vs onDemand instances. There are no annotations, EC2 tags or labels to identify which
instances are onDemand vs spot. There is a field called `InstanceLifecycle` accessible through `EC2.DescribeInstances`.
The field `InstanceLifecycle` is available only in `spot` and
`scheduled` AWS EC2 instance types.
This PR introduces a new label to be attached on AWS EC2 spot nodes.
The label is:
```
node-role.kubernetes.io/spot-worker: "true"
```
or
```
node-role.kubernetes.io/scheduled-worker: "true"
```
[^1]: https://github.com/kubernetes/kops/blob/master/docs/instance_groups.md#mixedinstancepolicy-aws-only
The client-go signature for most methods adds a context.Context
object, and also makes Options mandatory. Feed through a
context.Context through many of our methods (but use context.TODO to
stop it getting totally out of hand!)