Merge pull request #3142 from prachetasp/psp/canal-tolerations

Automatic merge from submit-queue

Fix for Canal Taints and Tolerations

This PR fixes issues with taints and tolerations for the Canal network type.

There are 2 problems:

1. Canal pods will not schedule onto dedicated instancegroups due to their too-strict tolerations. A similar fix for calico was merged recently #3097 - this is the same fix for Canal instead.
2. When booting a cluster with 1.7.0 we ran into the same issue mentioned in projectcalico/canal#77 and likely experienced in #2856. Specifically one of our dedicated instancegroups and the master nodes had no Taints attached. To mitigate we simply bump the versions of calico/node and calico/cni to match the versions that resolved projectcalico/canal#77.

When using these fixes to boot our k8s 1.7.3 cluster with this branch the Taints showed up and Calico scheduled on our masters and our dedicated IG.

I am not sure if there are associated tests that need to be updated here - still not familiar enough with the kops repo to tell.

upup/models/cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.6.yaml.template

@@ -71,19 +71,21 @@ spec:
       hostNetwork: true
       serviceAccountName: canal
       tolerations:
-        # Allow the pod to run on the master.  This is required for
-        # the master to communicate with pods.
-        - key: node-role.kubernetes.io/master
-          effect: NoSchedule
         # Mark the pod as a critical add-on for rescheduling.
         - key: "CriticalAddonsOnly"
           operator: "Exists"
+        - effect: NoExecute
+          operator: Exists
+        # Allow the pod to run on all nodes.  This is required
+        # for cluster communication
+        - effect: NoSchedule
+          operator: Exists
       containers:
         # Runs calico/node container on each Kubernetes node.  This
         # container programs network policy and routes on each
         # host.
         - name: calico-node
-          image: quay.io/calico/node:v1.2.1
+          image: quay.io/calico/node:v1.3.0
           env:
             # Use Kubernetes API as the backing datastore.
             - name: DATASTORE_TYPE
@@ -130,7 +132,7 @@ spec:
         # This container installs the Calico CNI binaries
         # and CNI network config file on each node.
         - name: install-cni
-          image: quay.io/calico/cni:v1.8.3
+          image: quay.io/calico/cni:v1.9.1
           command: ["/install-cni.sh"]
           env:
             # The CNI network config to install on each node.
@@ -337,4 +339,4 @@ roleRef:
 subjects:
 - kind: ServiceAccount
   name: canal
-  namespace: kube-system
+  namespace: kube-system

upup/pkg/fi/cloudup/bootstrapchannelbuilder.go

@@ -413,7 +413,8 @@ func (b *BootstrapChannelBuilder) buildManifest() (*channelsapi.Addons, map[stri
 
 	if b.cluster.Spec.Networking.Canal != nil {
 		key := "networking.projectcalico.org.canal"
-		version := "1.1"
+		// Locking canal addon version to 1.3 (same as Calico node). Best to maintain lockstep for sanity
+		version := "1.3"
 
 		{
 			location := key + "/pre-k8s-1.6.yaml"