Commit Graph

1322 Commits

Author SHA1 Message Date
Justin SB 2d143e6340 Update tests for ResourceHolder -> Resource 2020-12-19 09:51:58 -05:00
Justin SB 45d11ba12c Replace (some) deprecated ResourceHolder with Resource
This removes more of the deprecated type, but it also simplifies
refactoring the GCE InstanceTemplate.
2020-12-19 09:51:43 -05:00
Kubernetes Prow Robot 9561ed38c5
Merge pull request #10471 from akshedu/chore/add_new_pod_scale_up_delay_cluster_autoscaler
Add new-pod-scale-up-delay in Cluster Autoscaler spec
2020-12-19 02:42:24 -08:00
Akshansh Gupta 830fef7959 addons(cluster-autoscaler): Add newPodScaleUpDelay in clusterAutoscaler spec 2020-12-19 14:30:18 +05:30
Kubernetes Prow Robot e747047db7
Merge pull request #10452 from spotinst/feat-elastigroup-subnets
Spotinst: Support for multiple subnets per zone
2020-12-19 00:00:24 -08:00
Kubernetes Prow Robot a013aaac28
Merge pull request #10449 from spotinst/feat-ocean-autoscaler
Spotinst: Expose Ocean Headroom percentage and autoconfig labels
2020-12-18 23:18:24 -08:00
Kubernetes Prow Robot ef8c36999a
Merge pull request #10404 from seh/allow-use-of-calico-vxlan-backend
Calico: Allow operators to choose which encapsulation mode to use
2020-12-18 10:54:25 -08:00
Steven E. Harris f0f45b71fd Allow use of Calico's VXLAN networking backend
Introduce a new "encapsulationMode" field in Calico's portion of the
Cluster specification to allow switching between the the IP-in-IP and
VXLAN encapsulation protocols. For now, we accept the values "ipip"
and "vxlan," and forgo a possible "none" value that would disable
encapsulation altogether (at least for the default Calico IP pool).

Augment the default-populating procedure for Calico to take this field
into account when deciding both which networking backend to use and
whether to use IP-in-IP or VXLAN encapsulation for the default IP
pool. Note that these values supplied for the "CALICO_IPV4POOL_IPIP"
and "CALICO_IPV4POOL_VXLAN" environment variables in the "calico-node"
DaemonSet pod spec only matter for creating the "default" IPPool pool
object when no such objects already exist.

Generalize the documentation for the "crossSubnet" field to cover
environments more broad than just AWS, as Calico can employ this
selective encapsulation in any environment in which it can detect
boundaries between subnets.
2020-12-18 10:55:11 -05:00
liranp 22a07ff7a2
feat(spot/elastigroup): configure subnets without zones 2020-12-17 20:38:43 +02:00
liranp 55b27582c6
feat(spot/ocean): expose headroom percentage and autoconfig labels 2020-12-17 17:33:38 +02:00
Justin SB 72329db188 IAM ServiceAccount Roles: truncate name at 64 characters
The maximum IAM role name length is 64 characters, which we hit much
more often now that we are constructing complex names.  Use our normal
strategy of adding a hash when we truncate.

This is not a breaking change, because these names were not valid
previously.
2020-12-16 13:38:38 -05:00
Ole Markus With 4fa6f56ecd Use the kubernetes-sigs version of yaml 2020-12-15 10:38:01 +01:00
Ciprian Hacman 1c6618bdd9 Update tests 2020-12-13 13:28:41 +02:00
Ciprian Hacman d2b34eac90 Add support for containerd v1.3.9 for older k8s versions 2020-12-13 13:28:36 +02:00
Ciprian Hacman ab9d30a015 Order by name fields in CalicoNetworkingSpec 2020-12-11 18:23:49 +02:00
Justin SB dc48ca6905 Update etcd-manager to 20201209
Highlights:

* Fix arm64 images, which were built with an incorrect base image.
* Initial (experimental) Azure support

Full change list:

* Update Kops dependency for Azure Blob Storage support [#372](https://github.com/kopeio/etcd-manager/pull/372)
* Exclude gazelle from tools/deb-tools [#373](https://github.com/kopeio/etcd-manager/pull/373)
* Regenerate bazel in tools/deb-tools [#374](https://github.com/kopeio/etcd-manager/pull/374)
* Release notes for 3.0.20201202 [#375](https://github.com/kopeio/etcd-manager/pull/375)
* Remove travis CI [#377](https://github.com/kopeio/etcd-manager/pull/377)
* Fix vendor generation for tools/deb-tools subproject [#376](https://github.com/kopeio/etcd-manager/pull/376)
* Add script to verify image hashes [#380](https://github.com/kopeio/etcd-manager/pull/380)
* Fix some incorrect base image hashes for arm64 [#379](https://github.com/kopeio/etcd-manager/pull/379)
* Support Azure [#378](https://github.com/kopeio/etcd-manager/pull/378)
* Add more descriptions to wait loops [#383](https://github.com/kopeio/etcd-manager/pull/383)
* Rename fields in the azure client struct [#382](https://github.com/kopeio/etcd-manager/pull/382)
* Fix small typo in code comment [#381](https://github.com/kopeio/etcd-manager/pull/381)
2020-12-09 09:30:44 -05:00
Kubernetes Prow Robot bee16c052d
Merge pull request #10324 from bharath-123/feature/aws-imdv2
Add support for AWS IMDS v2
2020-12-07 22:55:11 -08:00
Ciprian Hacman 265bf4d106 Add option for setting the volume encryption key in AWS 2020-12-08 07:08:09 +02:00
Kubernetes Prow Robot 2f6c67e92c
Merge pull request #10364 from johngmyers/custom-account-issuer
Use custom-configured ServiceAccountIssuer when present
2020-12-07 19:39:11 -08:00
Bharath Vedartham 7f6e125733 Add support for aws ec2 instance metadata v2
A new field is add to the InstanceGroup spec with 2 sub fields,
HTTPPutResponseHopLimit and HTTPTokens. These fields enable the user
to disable IMDv1 for instances within an instance group.

By default, both IMDv1 and IMDv2 are enabled in instances in an instance group.
2020-12-07 02:57:02 +05:30
Ciprian Hacman e11d934268 Add option to reuse existing Elastic IPs for NAT gateways 2020-12-06 09:37:17 +02:00
Kubernetes Prow Robot ec691116a9
Merge pull request #10357 from rdrgmnzs/gzip-nodeup-heredocs
Give users the option to gzip and base64 encode the heredocs in the nodeup.sh user-data
2020-12-04 13:37:38 -08:00
Rodrigo Menezes 3fb12c66ae gzip and base64 encode the heredocs in the nodeup.sh portion of user-data 2020-12-04 10:46:18 -08:00
John Gardiner Myers e7508cc973 Use custom-configured ServiceAccountIssuer when present 2020-12-04 09:03:03 -08:00
Kubernetes Prow Robot 0fecffbfe0
Merge pull request #10284 from johngmyers/service-account-issuer
Set --service-account-issuer for k8s 1.20+
2020-12-04 08:07:59 -08:00
John Gardiner Myers 4f5def8610 Address review comment 2020-12-03 23:24:43 -08:00
Kubernetes Prow Robot 1b45f876a4
Merge pull request #10335 from hakman/same-tg-multiple-igs
Allow attaching same external target group to multiple instance groups
2020-12-02 21:38:59 -08:00
Ciprian Hacman e57cd534b5 Allow attaching same external target group to multiple instance groups 2020-12-03 06:59:59 +02:00
Kubernetes Prow Robot 443567426e
Merge pull request #9704 from nckturner/aws-cloud-controller
Add aws-cloud-controller-manager config to addons
2020-12-02 12:17:00 -08:00
Justin SB 0ea98a1e87 Update etcd-manager to 3.0.20201202
The important PR we want to pick up is 369, fixing a bug when
ListenMetricsURLS is set as an env var.

Full changelist:

* Release notes for 3.0.20201117 [#364](https://github.com/kopeio/etcd-manager/pull/364)
* Fix gofmt [#365](https://github.com/kopeio/etcd-manager/pull/365)
* Add gofmt check to github actions [#366](https://github.com/kopeio/etcd-manager/pull/366)
* Add boilerplate to tools/deb-tools/main.go [#367](https://github.com/kopeio/etcd-manager/pull/367)
* Do not set ListenMetricsURLS [#369](https://github.com/kopeio/etcd-manager/pull/369)
* Fix bazel formatting [#370](https://github.com/kopeio/etcd-manager/pull/370)
2020-12-02 12:08:37 -05:00
Ciprian Hacman e11156135b Update Docker to v19.03.14 2020-12-02 10:11:27 +02:00
Ciprian Hacman 2b6d730354 Update containerd to v1.4.3 2020-12-02 09:53:57 +02:00
Rodrigo Menezes c9af4de9cf Remove copywrite from nodeup scripts to reduce the user-data size 2020-11-30 12:49:25 -08:00
Nick Turner c9feb36f3f Add aws-cloud-controller-manager config to addons
- Config at aws-cloud-controller.addons.k8s.io/k8s-1.18.yaml.template
- AWSCCMTag function for CCM image tag
2020-11-30 01:35:07 -08:00
Kubernetes Prow Robot 16e922141a
Merge pull request #10296 from hakman/remove-legacy-elb-name
Remove support for using legacy ELB name
2020-11-22 11:31:42 -08:00
Ciprian Hacman ffe0af8629 Remove support for using legacy ELB name 2020-11-22 08:24:12 +02:00
Ciprian Hacman 338fb43f8b Update kOps version after 1.19.0-beta.2 release 2020-11-22 08:10:50 +02:00
Kubernetes Prow Robot 44465075b3
Merge pull request #10276 from hakman/fix-asg
Parse TargetGroup names from ARNs
2020-11-21 12:21:33 -08:00
John Gardiner Myers 9607b9955c Set --service-account-issuer for k8s 1.20+ 2020-11-20 22:20:39 -08:00
Ciprian Hacman 4853bf982a Use etcd v3.4.13 for k8s v1.19+ 2020-11-20 14:25:22 +02:00
Ciprian Hacman 19345c3f7f Order attached TargetGrups list by name 2020-11-20 10:40:27 +02:00
Ciprian Hacman fdcc2607bf Parse TargetGrup names from ARNs 2020-11-20 10:40:26 +02:00
Frank Yang 93dcaddc48 feat(aws): add PolicyNames for ELB to change listener's security policy 2020-11-19 16:07:21 +08:00
Justin SB d516fb7d9c Update etcd-manager to 3.0.20201117
Release notes for 3.0.20201117:

* Release notes for 3.0.20200531
* Adds support for using OS application credentials
* Fixes usage of OpenStack Swift reauthentication
* Move from debian-hyperkube-base to debian-base
* Add license headers to each file
* Fix some typos picked up by verify-spelling
* Fix some problems with trailing spaces
* Add support for etcd 3.4.13
* Switch to gcr.io/cloud-marketplace-containers/google/debian10 - Fix
 for #340 option 1
* Support for ARM64
* BUG: OpenStack ignore AvailabilityZone in discovery
* Added full cinder ID to candidateDeviceNodes
* feat(etcd-manager-ctl): use backupname to delete backup instead of timestamp
* Update kops to pick up AllowAuth Openstack
* Build base image by raw expansion of deb packages
* Switch the cloudbuild docker image, locking to 2.2.0
* Fix build on case-insensitive file systems (MacOS)
* Set AltNames on server certificates
* govet: Fix a log message
2020-11-17 22:03:30 -05:00
John Gardiner Myers 1165fd381e Remove more code specific to unsupported etcd v2 2020-11-15 22:21:24 -08:00
Kubernetes Prow Robot db473a11cd
Merge pull request #10194 from elblivion/etcdmanager-logverbosity
Make etcd-manager log verbosity configurable
2020-11-12 10:39:07 -08:00
Anthony Stanton c117d8d924 feat: Make etcd-manager log verbosity configurable 2020-11-12 09:58:09 +01:00
Ciprian Hacman 3e8770f763 Update kOps version after 1.19.0-beta.1 release 2020-11-11 10:15:39 +02:00
Kubernetes Prow Robot ddb3a38e28
Merge pull request #10190 from spotinst/feat-ocean-resource-limits
Spotinst: Configure Resource Limits in Ocean Auto Scaler
2020-11-10 18:15:48 -08:00
Kubernetes Prow Robot e43efbe102
Merge pull request #10157 from rifelpet/acm-nlb
Setup a second NLB listener when an AWS ACM certificate is used
2020-11-10 10:36:41 -08:00
Ciprian Hacman 0934374fe2 Fix various NLB nits 2020-11-10 17:30:23 +02:00
Peter Rifel 4758ea9f2f
Address feedback 2020-11-09 17:24:32 -06:00
Ciprian Hacman 32658075d3 Fix disabling spot instances when using launch templates 2020-11-08 19:11:45 +02:00
liranp fce6a22755
feat(spot/ocean): configure resource limits 2020-11-07 20:32:51 +02:00
Kubernetes Prow Robot 6a57543f6e
Merge pull request #10179 from olemarkus/sgr-consistent-naming
Consistent naming of security group rules
2020-11-07 02:07:37 -08:00
Ole Markus With fab694d290 Add ability to consistently name sgrs
In order to let kops fully control the rules for each security group we need to be able to generate names from the info in AWS. This is similar to the approach we used for openstack

Update pkg/model/firewall.go

Co-authored-by: Ciprian Hacman <ciprianhacman@gmail.com>
2020-11-07 10:27:19 +01:00
Jack Andersen 281e6140d9 Compare KubernetesAPIAccess to OpenStack allowedCIDRs deterministically 2020-11-07 00:29:24 -05:00
Peter Rifel 370092cb5a
Update TG ports rather than protocols when adding/removing ACM certs from listeners
This also renames the TGs to be more descriptive, with tcp and tls prefixes.
2020-11-06 11:09:38 -06:00
Peter Rifel 9242c34a38
Setup a second NLB listener on 8443 when sslCertificate is set 2020-11-06 11:09:37 -06:00
Peter Rifel 6c5b2fc58f
Add support for multiple NLB listeners and target groups 2020-11-06 11:09:36 -06:00
Ole Markus With 3c76610688 Remove the commented code. We can always retrieve it later 2020-11-06 09:53:10 +01:00
Ciprian Hacman a3a0b91b5f Order policy document sections alphabetically 2020-11-04 16:15:00 +02:00
Kubernetes Prow Robot 578920e921
Merge pull request #10162 from rifelpet/nlb-sg
Fix additionalSecurityGroups support for NLB
2020-11-03 08:02:16 -08:00
Peter Rifel 860249f6b7
Fix additionalSecurityGroups support for NLB
We were correctly adding the security groups to the master ASGs but identified them incorrectly.
2020-11-03 08:22:24 -06:00
Peter Rifel f08284834e
Move NLB's VPC CIDR security group rule logic into model
This way the security group rule task doesn't need to be aware of VPCs, since we know the VPC CIDR ahead of time via cluster spec.

This also fixes the terraform and cloudformation rendering of this rule (see the added cidr block in the integration test outputs)

These rules are for NLB's health checks. The AWS docs recommend allowing access from the entire VPC CIDRs
Also add rules for additionalNetworkCIDRs, supporting VPCs with multiple CIDR blocks.
2020-11-03 08:13:32 -06:00
Christian Joun e91ed11449
Implement API load balancer class with NLB and ELB support on AWS (#9011)
* refactor TargetLoadBalancer to use DNSTarget interface instead of LoadBalancer

* add LoadBalancerClass fields into api

* make api machinery

* WIP: Implemented API loadbalancer class, allowing NLB and ELB support on AWS for new clusters.

* perform vendoring related tasks and apply fixes identified from hack/

dissallow spotinst + nlb
remove reflection in status_discovery.go
Add precreated additional security groups to the Master nodes in case of NLB
Remove support for attaching individual instances to NLB; only rely on ASG attachments
Don't specify Classic loadbalancer in GCE integration test

* add utility function to the kops model context to make LoadBalancer comparisons simpler

* use DNSTarget interface when locating DNSName of API ELB

* wip: create target group task

* Consolidate TargetGroup tasks

* Use context helper for determining api load balancer type to avoid nil pointers

* Update NLB creation to use target group ARN from separate task rather than creating a TG in-line

* Address staticcheck and bazel failures

* Removing NLB Attachment tasks because they're not used since we switched to defining them as a part of the ASGs

* Address PR review feedback

* Only set LB Class field for AWS clusters, fix nil pointer

* Move target group attributes from NLB task to TG task, removing unused attributes

* Add terraform and cloudformation support for NLBs, listeners, and target groups

* Update integration test for NLB support

* Fix NLB name format to pass terraform validation

* Preserve security group rule names when switching ELB to NLB to reduce destructive terraform changes

* Use elbv2 enums and address some TODOs

* Set healthcheck values in target group

* Find TG tags, fix NLB name detection

* Fix more spurious changes reported by lifecycle integration test

* Fix spotinst validation, more code cleanup

* Address more PR feedback

* ReconcileTargetGroups unit test + more code simplification

* Addressing PR feedback Renaming task 1. awstasks.LoadBalancer -> awstasks.ClassicLoadBalancer

* Addressing PR feedback Renaming task: ELBName() -> CLBName() / LinkToELB() -> LinkToCLB()

* Addressing PR feedback: Various text changes

* fix export of kubecfg

* address TargetGroup should have the same name as the NLB

* should address error when fetching tags due to missing ARN

* Update expected and crds

* Add feature table to NLB docs

* Address more feedback and remove some TODOs that arent applicable anymore

* Update spotinst validation error message

Co-authored-by: Peter Rifel <pgrifel@gmail.com>
2020-11-02 05:28:52 -08:00
Ciprian Hacman 91d9c061dd Simplify etcd options builder 2020-10-30 09:11:00 +02:00
John Gardiner Myers 2ac17bee69 Remove code for no-longer-supported k8s releases 2020-10-29 16:45:53 -07:00
Peter Rifel 6318e90128
Ignore changes reported by subsequent updates
Usually this is an "actual.Foo = e.Foo" one-liner but we don't know which LB attached to an ASG is the API ELB so it's a bit more complicated
2020-10-29 12:34:20 -05:00
Peter Rifel 7497edaf7c
Lookup LoadBalancerName when only the LB task name is known 2020-10-29 12:13:23 -05:00
Kubernetes Prow Robot f466403912
Merge pull request #9794 from rdrgmnzs/lb-attachment
Prevent unintended resource updates to LB attatchments
2020-10-28 15:18:59 -07:00
Rodrigo Menezes 41adf07e15 cleanup code 2020-10-28 11:11:58 -07:00
Javi Polo c2684bcf7b Add nodeLocalDNSCache.kubeDnsOnly option 2020-10-27 10:46:25 +01:00
Rodrigo Menezes 9bd0a7aedb Add instruction for no downtime 2020-10-26 18:11:46 -07:00
Rodrigo Menezes dbbd0dd802 Move external LB and target group to inline as well. 2020-10-26 17:30:06 -07:00
Rodrigo Menezes 82d0ebdb56 Prevent unintended resource updates to LB attatchments 2020-10-26 17:29:07 -07:00
Kubernetes Prow Robot c9aa53895a
Merge pull request #10048 from hakman/container-runtime-assets
Install container runtime packages as assets
2020-10-25 21:03:01 -07:00
Kubernetes Prow Robot fbb172c08c
Merge pull request #9575 from johngmyers/node-labels
Take node labels from cloud tags on AWS
2020-10-23 04:01:45 -07:00
binkkatal e32717f31d FIX: Change int fields to string
The ./hack/update-expected.sh script generates some fields which are
required to be string fields and hence results in linting errors.

This PR changes those fields to string/*string and removes lint
warnings.
2020-10-20 19:28:20 +05:30
Kubernetes Prow Robot 18ffb493bf
Merge pull request #10061 from zetaab/fixegress
do not create egress rules when using vipacl octavia
2020-10-16 10:01:26 -07:00
Jesse Haka 33e2de60e5 do not create egress rules when using vipacl octavia 2020-10-16 14:11:22 +03:00
Ole Markus With 29a1cb2a9f If we use node local dns, always use the nld local ip as cluster dns 2020-10-16 12:46:17 +02:00
Ciprian Hacman 23e73a5b8e Release 1.19.0-alpha.5 2020-10-15 07:09:46 +03:00
Ciprian Hacman 852bebe165 Install container runtime packages as assets - Misc 2020-10-14 15:41:51 +03:00
Kubernetes Prow Robot 8c6bb14e15
Merge pull request #10033 from hakman/container-runtime-defaults
Update Docker version defaults for older k8s versions
2020-10-10 23:14:47 -07:00
Ciprian Hacman 2c15acfa44 Enable Calico AWS src/dest check permissions when CrossSubnet is set 2020-10-10 04:17:19 +03:00
Ciprian Hacman 95f9228e54 Update Docker version defaults for older k8s versions 2020-10-09 17:12:37 +03:00
Ciprian Hacman d0349fd6bb Open etcd port only when Calico uses "etcd" datastore 2020-10-09 09:33:38 +03:00
monicagangwar a63ccd5163 [calico] awsSrcDstCheck to disable src/dest checks in AWS
* replacing k8s-ec2-srcdst with calico's config awsSrcDstCheck and
  flag FELIX_AWSSRCDSTCHECK
* documentation and iam changes for calico awsSrcDstCheck
2020-10-08 17:17:23 +05:30
Ole Markus With 7c8ff94631 Make setupmockopenstack standalone 2020-10-01 19:15:39 +02:00
Kubernetes Prow Robot 4840582429
Merge pull request #9996 from rifelpet/additional-network-cidr
Fix support for multiple additionalNetworkCIDR blocks
2020-10-01 03:52:56 -07:00
Ole Markus With 7eb1489945 Bump cilium to 1.8.4 2020-10-01 10:21:10 +02:00
Kubernetes Prow Robot 13cbd84886
Merge pull request #9967 from olemarkus/cilium-hubble-pointer
Cilium hubble pointer
2020-09-30 12:36:54 -07:00
Peter Rifel db1b4e301c
Reconcile deletion of VPC CIDR block associations 2020-09-30 09:34:22 -05:00
Peter Rifel 4bcfebebcc
Fix the detection and rendering of multiple additionalNetworkCIDR blocks 2020-09-27 20:12:09 -05:00
Ole Markus With 6797998ac1 Consolidate all buildMinimalClusters into a generic test cluster builder 2020-09-19 19:55:19 +02:00
Kubernetes Prow Robot bca601d1da
Merge pull request #9969 from hakman/docker-19.03.13
Update Docker to v19.03.13
2020-09-18 10:46:46 -07:00
Kubernetes Prow Robot 255cd59b67
Merge pull request #9964 from rifelpet/sa-partition
Add AWS partition support to iam service account roles
2020-09-18 06:48:46 -07:00
Kubernetes Prow Robot e7bfedd1ac
Merge pull request #9921 from olemarkus/nth
Add addon for aws node termination handler
2020-09-18 03:10:45 -07:00
Ciprian Hacman 96e3fefd85 Update Docker to v19.03.13 2020-09-18 12:14:43 +03:00