## Release notes for kops 1.17 series

(The kops 1.17 release has not been released yet; this is a document to gather
the notes prior to the release).

# Significant changes

* The default instance type for AWS is now t3.medium. This should provide better performance and reduced costs in clusters where the average CPU usage is low.

# Breaking changes

* Terraform users on AWS may need to rename some resources in their state file in order to prepare for future Terraform 0.12 support. See Required Actions below.

* Please see the notes in the 1.15 release about the apiGroup changing from kops
  to kops.k8s.io

* Since 1.16, a controller is now used to apply labels to nodes.  If
  you are not using AWS, GCE or OpenStack your (non-master) nodes may
  not have labels applied correctly.

# Required Actions

* Terraform users on AWS may need to rename resources in their terraform state file in order to prepare for future Terraform 0.12 support.
  Terraform 0.12 [no longer supports resource names starting with digits](https://www.terraform.io/upgrade-guides/0-12.html#pre-upgrade-checklist). In Kops, both the default route and additional VPC CIDR associations are affected. See [#7957](https://github.com/kubernetes/kops/pull/7957) for more information.
  * The default route was named `aws_route.0-0-0-0--0` and will now be named `aws_route.route-0-0-0-0--0`.
  * Additional CIDR blocks associated with a VPC were similarly named the hyphenated CIDR block with two hyphens for the `/`, for example `aws_vpc_ipv4_cidr_block_association.10-1-0-0--16`. These will now be prefixed with `cidr-`, for example `aws_vpc_ipv4_cidr_block_association.cidr-10-1-0-0--16`.
  
  To prevent downtime, follow these steps with the new version of Kops:
  ```
  kops update cluster --target terraform ...
  terraform plan
  # Observe any aws_route or aws_vpc_ipv4_cidr_block_association resources being destroyed and recreated
  # Run these commands as necessary. The exact names may differ; use what is outputted by terraform plan
  terraform state mv aws_route.0-0-0-0--0 aws_route.route-0-0-0-0--0
  terraform state mv aws_vpc_ipv4_cidr_block_association.10-1-0-0--16 aws_vpc_ipv4_cidr_block_association.cidr-10-1-0-0--16
  terraform plan
  # Ensure these resources are no longer being destroyed and recreated
  terraform apply
  ```

* Kubernetes 1.9 users will need to enable the PodPriority feature gate. This is required for newer versions of Kops.

  To enable the Pod priority feature, follow these steps:
  ```
  kops edit cluster
  # Add the following section
  spec:
    kubelet:
      featureGates:
        PodPriority: "true"
  ```
 
* If either a Kops 1.17 alpha release or a custom Kops build was used on a cluster,
  a kops-controller Deployment may have been created that should get deleted because it has been replaced with a DaemonSet.
  Run `kubectl -n kube-system delete deployment kops-controller` after upgrading to Kops 1.17.0-alpha.2 or later.

# Deprecations

* Support for Kubernetes releases prior to 1.9 is deprecated and will be removed in kops 1.18.

* The `kops/v1alpha1` API is deprecated and will be removed in kops 1.18. Users of `kops replace` will need to supply v1alpha2 resources.

# Known Issues

* Kops 1.17.0-beta.1 included an update for AWS IAM Authenticator to 0.5.0.
  This version fails to use the volume mounted ConfigMap causing API authentication issues for clients with aws-iam-authenticator credentials.
  Any cluster with `spec.authentication.aws` defined according to the [docs](../authentication.md#aws-iam-authenticator) without overriding the `spec.authentication.aws.image` is affected.
  The workaround is to specify the old 0.4.0 image with `spec.authentication.aws.image=602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.4.0`.
 
# Full change list since 1.16.0 release

## 1.16.0-alpha.1 to 1.17.0-alpha.1

* Add release notes for 1.16.0-alpha.1 [@justinsb](https://github.com/justinsb) [#7896](https://github.com/kubernetes/kops/pull/7896)
* stable channel: promote kubernetes 1.13.12, 1.14.8 etc [@justinsb](https://github.com/justinsb) [#7891](https://github.com/kubernetes/kops/pull/7891)
* Don't update first node in instancegroup if cluster fails validation [@johngmyers](https://github.com/johngmyers),[@justinsb](https://github.com/justinsb) [#7872](https://github.com/kubernetes/kops/pull/7872)
* add missing priorityClassName to flannel DaemonSet [@EladDolev](https://github.com/EladDolev) [#7842](https://github.com/kubernetes/kops/pull/7842)
* fix broken links [@dj80hd](https://github.com/dj80hd) [#7901](https://github.com/kubernetes/kops/pull/7901)
* Fix rendering of the Node Authorizer template [@KashifSaadat](https://github.com/KashifSaadat) [#7916](https://github.com/kubernetes/kops/pull/7916)
* Fix fork bomb in Makefile [@johngmyers](https://github.com/johngmyers) [#7935](https://github.com/kubernetes/kops/pull/7935)
* Unhide docs make logging [@mikesplain](https://github.com/mikesplain) [#7936](https://github.com/kubernetes/kops/pull/7936)
* Upgrade AWS VPC CNI to 1.5.5 [@rifelpet](https://github.com/rifelpet) [#7938](https://github.com/kubernetes/kops/pull/7938)
* Correct spelling mistakes [@yuxiaobo96](https://github.com/yuxiaobo96) [#7922](https://github.com/kubernetes/kops/pull/7922)
* Fix flannel CNI version to use 0.2.0 [@srikiz](https://github.com/srikiz) [#7924](https://github.com/kubernetes/kops/pull/7924)
* Update vendoring documentation for go modules [@rifelpet](https://github.com/rifelpet) [#7937](https://github.com/kubernetes/kops/pull/7937)
* Remove duplication and update release details [@mikesplain](https://github.com/mikesplain) [#7939](https://github.com/kubernetes/kops/pull/7939)
* Updated documentation on how to move from single to multi master [@mccare](https://github.com/mccare) [#7439](https://github.com/kubernetes/kops/pull/7439)
* Create PodDisruptionBudget for kube-dns in kube-system namespace [@hakman](https://github.com/hakman) [#7856](https://github.com/kubernetes/kops/pull/7856)
* Add support for newer Docker versions [@hakman](https://github.com/hakman) [#7860](https://github.com/kubernetes/kops/pull/7860)
* Machine types updates [@mikesplain](https://github.com/mikesplain) [#7947](https://github.com/kubernetes/kops/pull/7947)
* fix 404 urls in docs [@tanjunchen](https://github.com/tanjunchen) [#7943](https://github.com/kubernetes/kops/pull/7943)
* Fix generation of documentation /sitemap.xml file [@aledbf](https://github.com/aledbf) [#7949](https://github.com/kubernetes/kops/pull/7949)
* Kops site link [@mikesplain](https://github.com/mikesplain) [#7950](https://github.com/kubernetes/kops/pull/7950)
* Fix netlify mixed content [@mikesplain](https://github.com/mikesplain) [#7953](https://github.com/kubernetes/kops/pull/7953)
* Fix goimports errors [@rifelpet](https://github.com/rifelpet) [#7955](https://github.com/kubernetes/kops/pull/7955)
* Upate Lyft CNI to v0.5.1 [@maruina](https://github.com/maruina) [#7402](https://github.com/kubernetes/kops/pull/7402)
* Add relnotes for 1.16.0-alpha.2 [@justinsb](https://github.com/justinsb) [#7962](https://github.com/kubernetes/kops/pull/7962)
* Bump version of alpha [@mikesplain](https://github.com/mikesplain) [#7963](https://github.com/kubernetes/kops/pull/7963)
* Add relnotes for 1.15.0 [@justinsb](https://github.com/justinsb) [#7964](https://github.com/kubernetes/kops/pull/7964)
* Update feature flag documentation [@rifelpet](https://github.com/rifelpet) [#7969](https://github.com/kubernetes/kops/pull/7969)
* Bazel upgrade [@mikesplain](https://github.com/mikesplain) [#7933](https://github.com/kubernetes/kops/pull/7933)
* Upgrade AWS SDK [@rifelpet](https://github.com/rifelpet) [#7972](https://github.com/kubernetes/kops/pull/7972)
* Fix panic when ssh key not exists on digitalocean [@prksu](https://github.com/prksu) [#7941](https://github.com/kubernetes/kops/pull/7941)
* Upgrade go to 1.13 [@rifelpet](https://github.com/rifelpet) [#7973](https://github.com/kubernetes/kops/pull/7973)
* Use correct values for CALICO_IPV4POOL_IPIP in Calico v3 [@zacblazic](https://github.com/zacblazic) [#7899](https://github.com/kubernetes/kops/pull/7899)
* Put kubernetes 1.17.0-beta.2 into channels [@justinsb](https://github.com/justinsb) [#7982](https://github.com/kubernetes/kops/pull/7982)
* Update compatability matrix [@mikesplain](https://github.com/mikesplain) [#7984](https://github.com/kubernetes/kops/pull/7984)
* Promote peter & ryan & zetaab to approvers [@justinsb](https://github.com/justinsb) [#7983](https://github.com/kubernetes/kops/pull/7983)
* upgrade the time api [@tanjunchen](https://github.com/tanjunchen) [#7910](https://github.com/kubernetes/kops/pull/7910)
* sysctls.go: Fix some comments [@eest](https://github.com/eest) [#7923](https://github.com/kubernetes/kops/pull/7923)
* Ignore devcontainer for vscode remote-containers [@granular-ryanbonham](https://github.com/granular-ryanbonham) [#7987](https://github.com/kubernetes/kops/pull/7987)
* Dont run travis with Go 1.11 [@rifelpet](https://github.com/rifelpet) [#7988](https://github.com/kubernetes/kops/pull/7988)
* Change doc cross-references from absolute to relative links [@johngmyers](https://github.com/johngmyers) [#7907](https://github.com/kubernetes/kops/pull/7907)
* Correct link error:404 [@yuxiaobo96](https://github.com/yuxiaobo96) [#7954](https://github.com/kubernetes/kops/pull/7954)
* Update apiVersion in docs and tests [@johngmyers](https://github.com/johngmyers) [#7906](https://github.com/kubernetes/kops/pull/7906)
* [aws-iam-authenticator] Docs - Steps to disable DaemonSet Temporarily [@bhegazy](https://github.com/bhegazy) [#7926](https://github.com/kubernetes/kops/pull/7926)
* Add indent template function and use it to fix KubeDNS.ExternalCoreFile rendering [@rochacon](https://github.com/rochacon) [#7979](https://github.com/kubernetes/kops/pull/7979)
* fix golint failures [@FayerZhang](https://github.com/FayerZhang) [#7894](https://github.com/kubernetes/kops/pull/7894)
* [Issue-7870] kops controller support for digital ocean [@srikiz](https://github.com/srikiz) [#7961](https://github.com/kubernetes/kops/pull/7961)
* cleanup whitespace in root.go [@joshbranham](https://github.com/joshbranham) [#7997](https://github.com/kubernetes/kops/pull/7997)
* Run goimports from locked version in go.mod [@justinsb](https://github.com/justinsb) [#7998](https://github.com/kubernetes/kops/pull/7998)
* Remove note about 1.15 not being released [@kaspernissen](https://github.com/kaspernissen) [#8000](https://github.com/kubernetes/kops/pull/8000)
* fix(openstack): fix additional security groups on instance groups [@mitch000001](https://github.com/mitch000001) [#8004](https://github.com/kubernetes/kops/pull/8004)
* DOCS: fix simple typo in readme [@lpmi-13](https://github.com/lpmi-13) [#8005](https://github.com/kubernetes/kops/pull/8005)
* Spotinst: Upgrade the Spotinst SDK to version 1.36 [@liranp](https://github.com/liranp) [#8003](https://github.com/kubernetes/kops/pull/8003)
* Release 1.17.0-alpha.1 [@justinsb](https://github.com/justinsb) [#7985](https://github.com/kubernetes/kops/pull/7985)

## 1.17.0-alpha.1 to 1.17.0-alpha.2

* Fix mounting Calico "flexvol-driver-host" in CoreOS [@hakman](https://github.com/hakman) [#8062](https://github.com/kubernetes/kops/pull/8062)
* Cherry-pick #8074 to release-1.17 [@johngmyers](https://github.com/johngmyers) [#8084](https://github.com/kubernetes/kops/pull/8084)
* Bump cilium version to 1.6.4 [@olemarkus](https://github.com/olemarkus) [#8022](https://github.com/kubernetes/kops/pull/8022)
* Complete support for Flatcar [@mazzy89](https://github.com/mazzy89) [#7545](https://github.com/kubernetes/kops/pull/7545)
* Canal v3.10 manifest for k8s v1.15+ [@KashifSaadat](https://github.com/KashifSaadat),[@hakman](https://github.com/hakman) [#7917](https://github.com/kubernetes/kops/pull/7917)
* Cherry pick #8095 [@zetaab](https://github.com/zetaab) [#8096](https://github.com/kubernetes/kops/pull/8096)
* test validateCluster twice to make sure it does not flap [@zetaab](https://github.com/zetaab),[@johngmyers](https://github.com/johngmyers) [#8088](https://github.com/kubernetes/kops/pull/8088)
* Add inf1 isntances [@mikesplain](https://github.com/mikesplain) [#8128](https://github.com/kubernetes/kops/pull/8128)
* Add CapacityOptimized to list of supported spot allocation strategies [@gjtempleton](https://github.com/gjtempleton) [#7406](https://github.com/kubernetes/kops/pull/7406)
* Update Calico to v3.10.2 [@hakman](https://github.com/hakman) [#8104](https://github.com/kubernetes/kops/pull/8104)
* Openstack: Fix cluster floating ips [@mitch000001](https://github.com/mitch000001) [#8115](https://github.com/kubernetes/kops/pull/8115)
* cilium: don't try to mount sys/fs/bpf if already mounted [@justinsb](https://github.com/justinsb) [#7832](https://github.com/kubernetes/kops/pull/7832)
* Update copyrights for 2020 [@johngmyers](https://github.com/johngmyers) [#8241](https://github.com/kubernetes/kops/pull/8241)
* Fix protokube osx build [@mikesplain](https://github.com/mikesplain) [#8263](https://github.com/kubernetes/kops/pull/8263)
* Set CLUSTER_NAME env var on amazon-vpc-cni pods [@rifelpet](https://github.com/rifelpet) [#8274](https://github.com/kubernetes/kops/pull/8274)
* Add deprecation warning for older k8s versions [@rifelpet](https://github.com/rifelpet) [#8176](https://github.com/kubernetes/kops/pull/8176)
* Remove kops-controller deployment [@rifelpet](https://github.com/rifelpet) [#8273](https://github.com/kubernetes/kops/pull/8273)
* Don't output empty sections in the manifests [@justinsb](https://github.com/justinsb) [#8317](https://github.com/kubernetes/kops/pull/8317)
* Cloud controller template function [@DavidSie](https://github.com/DavidSie) [#7992](https://github.com/kubernetes/kops/pull/7992)
* Configuration to specify no SSH key [@austinmoore-](https://github.com/austinmoore-) [#7096](https://github.com/kubernetes/kops/pull/7096)
* tests: increase timeout in rolling update tests [@justinsb](https://github.com/justinsb) [#8139](https://github.com/kubernetes/kops/pull/8139)
* Fix crossbuild-nodeup-in-docker [@johngmyers](https://github.com/johngmyers) [#8343](https://github.com/kubernetes/kops/pull/8343)
* update gophercloud dependency [@zetaab](https://github.com/zetaab) [#8347](https://github.com/kubernetes/kops/pull/8347)
* Update Terraform resource names to be 0.12 compatible. [@rifelpet](https://github.com/rifelpet) [#7957](https://github.com/kubernetes/kops/pull/7957)
* Allow local filesystem state stores (to aid CI pull-request workflows) [@ari-becker](https://github.com/ari-becker),[@rifelpet](https://github.com/rifelpet) [#6465](https://github.com/kubernetes/kops/pull/6465)
* Fix issues with older versions of k8s for basic clusters [@hakman](https://github.com/hakman) [#8248](https://github.com/kubernetes/kops/pull/8248)
* Use IAMPrefix() for hostedzone [@lazzarello](https://github.com/lazzarello) [#8366](https://github.com/kubernetes/kops/pull/8366)
* Fix scheduler policy configmap args [@vvbogdanov87](https://github.com/vvbogdanov87) [#8386](https://github.com/kubernetes/kops/pull/8386)
* Add Cilium.EnablePolicy back into templates [@olemarkus](https://github.com/olemarkus) [#8379](https://github.com/kubernetes/kops/pull/8379)
* Bump etcd-manager to 3.0.20200116 (#8310) [@mmerrill3](https://github.com/mmerrill3) [#8399](https://github.com/kubernetes/kops/pull/8399)
* CoreDNS default image bump to 1.6.6 to resolve CVE [@gjtempleton](https://github.com/gjtempleton) [#8333](https://github.com/kubernetes/kops/pull/8333)
* Don't load nonexistent calico-client cert when CNI is Cilium [@johngmyers](https://github.com/johngmyers) [#8338](https://github.com/kubernetes/kops/pull/8338)
* Kops releases - prefix git tags with v [@rifelpet](https://github.com/rifelpet) [#8373](https://github.com/kubernetes/kops/pull/8373)
* EBS Root Volume Termination [@tioxy](https://github.com/tioxy) [#7865](https://github.com/kubernetes/kops/pull/7865)
* Alicloud: etcd-manager support [@bittopaz](https://github.com/bittopaz) [#8016](https://github.com/kubernetes/kops/pull/8016)

## 1.17.0-alpha.2 to 1.17.0-alpha.3

* Add missing priorityClassName for critical pods [@johngmyers](https://github.com/johngmyers) [#8200](https://github.com/kubernetes/kops/pull/8200)
* Alicloud: allow use RAM role for OSS client [@bittopaz](https://github.com/bittopaz) [#8025](https://github.com/kubernetes/kops/pull/8025)
* Update coredns to 1.6.7 [@maruina](https://github.com/maruina) [#8452](https://github.com/kubernetes/kops/pull/8452)
* Fix Github download url for nodeup [@adri](https://github.com/adri),[@justinsb](https://github.com/justinsb) [#8468](https://github.com/kubernetes/kops/pull/8468)

## 1.17.0-alpha.3 to 1.17.0-alpha.4

* Cilium - Add missing Identity Allocation Mode to Operator Template [@daviddyball](https://github.com/daviddyball) [#8445](https://github.com/kubernetes/kops/pull/8445)
* Revert "Update coredns to 1.6.7" [@gjtempleton](https://github.com/gjtempleton) [#8502](https://github.com/kubernetes/kops/pull/8502)
* GCS: Don't try to set ACLs if bucket-policy only is set [@justinsb](https://github.com/justinsb) [#8493](https://github.com/kubernetes/kops/pull/8493)
* Make it possible to enable Prometheus metrics for Cilium [@olemarkus](https://github.com/olemarkus) [#8433](https://github.com/kubernetes/kops/pull/8433)
* Update cilium to 1.6.6 [@olemarkus](https://github.com/olemarkus) [#8484](https://github.com/kubernetes/kops/pull/8484)

## 1.17.0-alpha.4 to 1.17.0-beta.1

* Stabilize sequence of "export xx=xxx" statements [@bittopaz](https://github.com/bittopaz) [#8247](https://github.com/kubernetes/kops/pull/8247)
* Add events RBAC permissions to kops-controller [@rifelpet](https://github.com/rifelpet) [#8535](https://github.com/kubernetes/kops/pull/8535)
* Fix DNS loop on Ubuntu 18.04 (Bionic) [@hakman](https://github.com/hakman) [#8353](https://github.com/kubernetes/kops/pull/8353)
* Update AWS IAM Authenticator to 0.5.0 [@rifelpet](https://github.com/rifelpet) [#8423](https://github.com/kubernetes/kops/pull/8423)
* Update amazon-vpc-cni-k8s to v1.6.0 [@hakman](https://github.com/hakman) [#8538](https://github.com/kubernetes/kops/pull/8538)
* Update IAM permissions for amazon-vpc-cni-k8s 1.6.0 [@rifelpet](https://github.com/rifelpet) [#8548](https://github.com/kubernetes/kops/pull/8548)
* Switch AWS IAM Authenticator to use non-scratch image [@rifelpet](https://github.com/rifelpet) [#8555](https://github.com/kubernetes/kops/pull/8555)
* add s3 region [@zetaab](https://github.com/zetaab) [#8592](https://github.com/kubernetes/kops/pull/8592)
* Update coredns to 1.6.7 [@maruina](https://github.com/maruina) [#8602](https://github.com/kubernetes/kops/pull/8602)
* Cilium fix bpffs check [@olemarkus](https://github.com/olemarkus) [#8599](https://github.com/kubernetes/kops/pull/8599)
* Fix periodic e2e test for Ubuntu 16.04 [@hakman](https://github.com/hakman) [#8160](https://github.com/kubernetes/kops/pull/8160)
* Bump Cilium to 1.7 for k8s 1.12+ [@olemarkus](https://github.com/olemarkus) [#8589](https://github.com/kubernetes/kops/pull/8589)
* Pass the cloud object to validator from caller [@johngmyers](https://github.com/johngmyers) [#7925](https://github.com/kubernetes/kops/pull/7925)
* Custom sysctl Parameters [@ripta](https://github.com/ripta) [#7730](https://github.com/kubernetes/kops/pull/7730)
* Automatically install dependencies with local packages [@hakman](https://github.com/hakman) [#8020](https://github.com/kubernetes/kops/pull/8020)
* Add support for custom env vars in amazon-vpc-cni [@rifelpet](https://github.com/rifelpet) [#8276](https://github.com/kubernetes/kops/pull/8276)