Commit Graph

369 Commits

Author SHA1 Message Date
Peter Rifel 0bd7348ad9
Fix ARN partition in SQS queue policy 2021-10-29 23:08:30 -05:00
Peter Rifel c734f5c08d
Update IAMBuilder to include the current partition in ARNs 2021-10-29 23:07:31 -05:00
Kubernetes Prow Robot 228c82cb6e
Merge pull request #12571 from rifelpet/sqs-arn
Use the SQS Queue's ARN reference
2021-10-26 22:19:26 -07:00
Peter Rifel cedb8f813c
Use the SQS Queue's ARN reference 2021-10-20 20:47:26 -07:00
liranp b3a3526ad0
feat(spot/ocean): get instance types from `mixedInstancesPolicy` 2021-10-18 16:08:45 +03:00
Peter Rifel b1fa018c36
Don't hard-code the SQS Queue ARN partition 2021-10-15 09:49:57 -07:00
liranp 30f09f9f07
feat(spot): new metadata label: utilize-commitments 2021-10-12 01:32:09 +03:00
Kubernetes Prow Robot 3dc1d25454
Merge pull request #12439 from rifelpet/nth-truncate
Truncate cluster name in NTH EventBridgeRules
2021-09-30 00:58:07 -07:00
Peter Rifel 3311e45767
Truncate cluster name prefix used in event bridge rules 2021-09-29 19:12:49 -05:00
Charles-Edouard Brétéché 7c8c9b9a23 feat: add support for custom audience in aws oidc provider
fix: missing json tags


fix: code gen


fix: switch to additional audiences


fix: oidc task


fix: add integration test
2021-09-28 22:39:56 +02:00
Kubernetes Prow Robot 1774e6cae3
Merge pull request #12321 from dezmodue/private_bastion
Add option to create an internal load balancer for the bastion
2021-09-24 07:23:24 -07:00
Kubernetes Prow Robot 74f9a8e2fb
Merge pull request #12342 from eddycharly/irsa-wildcard
feat: add support for wildcard in roles generated for IRSA
2021-09-22 16:09:10 -07:00
Charles-Edouard Brétéché 5f523366d6 feat: add support for wildcard in roles generated for IRSA 2021-09-23 00:24:45 +02:00
justinsb 99764fb168 AWS: Move some subnet functions into AWS model
We want to move all these eventually, and this is preparing for better
GCE subnet support.
2021-09-19 12:08:09 -04:00
Simone Sciarrati 61763d488a Add option to create an internal load balancer for the bastion 2021-09-18 20:47:55 +02:00
Ole Markus With d98994686a Use sg rule ids and tags where possible 2021-09-12 14:32:58 +02:00
Peter Rifel 60c86e1a44
Enable IMDS IPv6 endpoint when IPv6AddressCount > 0 2021-09-09 07:24:14 -05:00
Ole Markus With b52008d9b6 Add instance state change notification to nth 2021-08-31 22:54:21 +02:00
Kubernetes Prow Robot bb38a3e52e
Merge pull request #12067 from h3poteto/iss-11608
Support AWS LB access log configuration in cluster spec
2021-08-25 16:51:23 -07:00
Ole Markus With 0439bb0d76 Remove UseServiceAccountIAM feature flag and rename feature to UseServiceAccountExternalPermissions 2021-08-07 21:20:03 +02:00
AkiraFukushima 2fd69ba3a3
Remove access log attributes when the spec is removed from cluster spec 2021-08-03 17:45:20 +09:00
AkiraFukushima 226cbe5561
Support AWS LB access log configuration for NetworkLoadBalancer 2021-08-03 12:12:16 +09:00
Peter Rifel a0a6e3c974
Cleanup various references to LaunchConfigurations 2021-07-29 22:25:01 -04:00
AkiraFukushima 50ab82ed04
Support AWS LB access log configuration in cluster spec 2021-07-29 22:39:23 +09:00
John Gardiner Myers e9fc12b4f3 Fix certificate bootstrap for non-kops-controller-bootstrap cloud providers 2021-07-18 13:37:19 -07:00
John Gardiner Myers c35d101a89 Refactor keysets for etcd-manager 2021-07-08 18:46:03 -07:00
Ole Markus With aefa906491 Do not set both CIDR and IPv6CIDR on sg rules 2021-07-03 07:57:35 +02:00
John Gardiner Myers 1e0c6cb1aa Refactor apiserver-aggregator-ca 2021-07-01 22:25:47 -07:00
Kubernetes Prow Robot 19ffc06d3d
Merge pull request #11853 from johngmyers/override-issuer
Allow overriding the ServiceAccountIssuer for IRSA
2021-07-01 04:43:54 -07:00
John Gardiner Myers 3de05a500e Refactor etcd-clients-ca keyset for api-server 2021-06-30 18:55:30 -07:00
Kubernetes Prow Robot ee048e89e7
Merge pull request #11872 from johngmyers/refactor-serviceaccount
Refactor nodeup APIServer builder, part one
2021-06-28 10:42:01 -07:00
Kubernetes Prow Robot 917c965c8f
Merge pull request #11873 from hakman/avoid_spurious_changes
Avoid spurious changes for ASG InstanceProtection and LT InstanceMonitoring
2021-06-27 19:59:24 -07:00
John Gardiner Myers e1df9f09dd Refactor service-account public keys 2021-06-27 08:45:06 -07:00
Kubernetes Prow Robot 22c11c10f1
Merge pull request #11848 from johngmyers/cilium-etcd-client
Refactor etcd-client-cilium secrets
2021-06-27 04:01:24 -07:00
Ciprian Hacman 348eed772a Avoid spurious changes for ASG InstanceProtection and LT InstanceMonitoring 2021-06-27 10:08:13 +03:00
Kubernetes Prow Robot 51daab932e
Merge pull request #11870 from hakman/ipv6_use_dualstack_nlb
Use DualStack API NLB for IPv6
2021-06-26 12:45:24 -07:00
Ciprian Hacman 7969f57d07 Address review comments 2021-06-26 21:27:00 +03:00
Ole Markus With dc79acb1bb Don't reconcile roles and policies if a profile is provided 2021-06-26 19:45:19 +02:00
Ciprian Hacman 7bc629b683 Use DualStack API NLB for IPv6 2021-06-26 19:16:46 +03:00
John Gardiner Myers 2faf28379a Refactor etcd-client-cilium secrets 2021-06-25 23:57:23 -07:00
John Gardiner Myers 24d1706848 Allow overriding the ServiceAccountIssuer for IRSA 2021-06-25 18:33:07 -07:00
John Gardiner Myers 5687b0d5dc Weaken some interfaces 2021-06-21 23:11:47 -07:00
Ole Markus With b2588b637b fix missing lifecycle when deleting iam roles 2021-06-16 13:59:19 +02:00
Ciprian Hacman eb574a414c Don't set Subnet dependency on AmazonIPv6CIDR for shared VPCs 2021-06-13 12:25:42 +02:00
Kubernetes Prow Robot cfc93e5178
Merge pull request #9294 from johngmyers/refactor-nodeup-context
Remove InstanceGroup from NodeupModelContext
2021-06-12 13:43:01 -07:00
Kubernetes Prow Robot 92af7b88f4
Merge pull request #11523 from hakman/ipv6_cidr_subnet
Calculate IPv6 subnet CIDR based on cluster CIDR
2021-06-10 21:40:13 -07:00
Kubernetes Prow Robot 4005c209ff
Merge pull request #11604 from spotinst/feat-aws-nlb
Spotinst: Support for API Load Balancer with AWS/NLB
2021-06-10 04:29:28 -07:00
Ciprian Hacman 99268697c0 Add Subnet dependency on VPCAmazonIPv6CIDRBlock 2021-06-09 09:57:53 +03:00
John Gardiner Myers eb09d31a3c Pass AuxConfig to nodeup 2021-06-03 21:04:21 -07:00
John Gardiner Myers 7c9e7e9286 Make Lifecycle field non-pointer 2021-06-02 23:02:16 -07:00
Peter Rifel efef53cb2a
Add more lifecycles to HasLifecycle tasks 2021-06-01 23:08:49 -05:00
John Gardiner Myers 2b146d31d6 Set Lifecycle in APILoadBalancerBuilder 2021-05-31 10:39:33 -07:00
John Gardiner Myers 64dac12216 Set Lifecycle in AutoscalingGroupModelBuilder 2021-05-31 10:39:33 -07:00
John Gardiner Myers 024b3653c0 Set lifecycle on WarmPool task 2021-05-28 20:05:44 -07:00
liranp 1d97fbd78c
feat(spot): support for api load balancer with aws/nlb 2021-05-26 03:35:37 +03:00
Kubernetes Prow Robot 4a5d04d94f
Merge pull request #11497 from johngmyers/cleanup-iam
Cleanup orphaned IAM service account roles in direct render
2021-05-19 18:35:05 -07:00
Ciprian Hacman cedbe1f360 Add initial support for configuring IPv6 with AWS 2021-05-19 06:21:07 +03:00
Ole Markus With d3581ebb84 bump aws lb controller to 2.2.0 2021-05-16 18:26:23 +02:00
John Gardiner Myers 4baf2cbdcf Delete IAM roles no longer in the model 2021-05-15 12:03:23 -07:00
John Gardiner Myers 0c1f9f4772 Refactor LaunchTemplate.SecurityGroups 2021-05-11 14:48:00 -07:00
John Gardiner Myers 5d3af39311 Refactor LaunchTemplate.UserData 2021-05-11 14:48:00 -07:00
John Gardiner Myers 4a5e46922f Refactor LaunchTemplate.Tenancy 2021-05-11 14:48:00 -07:00
John Gardiner Myers 4d9018282c Refactor LaunchTemplate.SSHKey 2021-05-11 14:48:00 -07:00
John Gardiner Myers b0bcf40921 Refactor LaunchTemplate.RootVolumeEncryptionKey 2021-05-11 14:48:00 -07:00
John Gardiner Myers 945e56294f Refactor LaunchTemplate.RootVolumeEncryption 2021-05-11 14:48:00 -07:00
John Gardiner Myers 1a39c9060e Refactor LaunchTemplate.RootVolumeSize 2021-05-11 14:48:00 -07:00
John Gardiner Myers 3097a3a746 Refactor LaunchTemplate.RootVolumeOptimization 2021-05-11 14:48:00 -07:00
John Gardiner Myers 436dbe8435 Refactor LaunchTemplate.RootVolumeIops 2021-05-11 14:47:57 -07:00
John Gardiner Myers 01a55812ac Refactor LaunchTemplate.RootVolumeType 2021-05-11 13:38:20 -07:00
John Gardiner Myers a4898c9d7d Refactor LaunchTemplate.InstanceType 2021-05-10 23:22:41 -07:00
John Gardiner Myers d2adf498f6 Refactor LaunchTemplate.InstanceMonitoring 2021-05-10 23:12:21 -07:00
John Gardiner Myers a1db8f1e82 Refactor LaunchTemplate.InstanceInterruptionBehavior 2021-05-10 23:11:17 -07:00
John Gardiner Myers d0793bd6ed Refactor LaunchTemplate.ImageID 2021-05-10 23:08:21 -07:00
John Gardiner Myers bfd8034cce Refactor LaunchTemplate.IAMInstanceProfile 2021-05-10 23:08:21 -07:00
John Gardiner Myers 07aa346e68 Refactor LaunchTemplate.HTTPTokens 2021-05-10 23:08:20 -07:00
John Gardiner Myers 98502cd0b2 Refactor LaunchTemplate.HTTPPutResponseHopLimit 2021-05-10 23:08:16 -07:00
John Gardiner Myers 33590eb617 Refactor LaunchTemplate.CPUCredits 2021-05-10 23:07:24 -07:00
John Gardiner Myers 0557414111 Refactor LaunchTemplate.BlockDeviceMappings 2021-05-10 22:51:00 -07:00
John Gardiner Myers 4657cb94d6 Refactor LaunchTemplate.AssociatePublicIP 2021-05-10 22:47:48 -07:00
Ole Markus With 6f8b3647cf Add support for IRSA in he api
Apply suggestions from code review

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2021-05-01 16:03:42 +02:00
Ole Markus With 460586833b Add toggle for AWS OIDC provider. Free it from any feature flag 2021-04-30 19:19:06 +02:00
Ole Markus With 0f545f8659 Split oidc_provider
* one builder concerned with publishing issuer discovery metadata
* one builder concerned with creating aws oidc provider
2021-04-30 18:05:20 +02:00
Ciprian Hacman 4a0fa78b20 Run hack/update-bazel.sh 2021-04-30 14:50:46 +03:00
Ciprian Hacman 0e651dd8fc Use AWSModelContext in remaining awsmodel files 2021-04-30 14:50:46 +03:00
Ciprian Hacman 137fe6c2bb Move firewall to awsmodel 2021-04-30 14:50:46 +03:00
Ciprian Hacman fcba0043d0 Move iam to awsmodel 2021-04-30 12:37:28 +03:00
Ciprian Hacman 4dfe58de7a Move network to awsmodel 2021-04-30 12:04:06 +03:00
Ciprian Hacman ca02c04793 Move sshkey to awsmodel 2021-04-30 12:04:06 +03:00
Kubernetes Prow Robot 942f183157
Merge pull request #11336 from olemarkus/sqs-fix-flap
Fix SQS resource flapping
2021-04-27 22:08:49 -07:00
Ole Markus With f16cafb8ef Make hook task name unique while the hook name is consistent
Since tasks need to be unique, but we need to reuse the hookname across all ASGs, we distinguish between task and actual name of the hook
2021-04-27 20:57:19 +02:00
Ole Markus With 849ff56c96 Fix SQS resource flapping
* one case of AWS returning different JSON than we passed
* AWS returning a field we do not (and can not) build an expected value of
2021-04-27 20:47:24 +02:00
John Gardiner Myers 428041bc0f Add cluster-level warmPool settings 2021-04-25 20:22:04 -07:00
John Gardiner Myers 5ad32230bb Fix typo 2021-04-25 13:42:12 -07:00
John Gardiner Myers 044b5f6d0d Allow disabling warm pool by setting WarmPool.MaxSize to 0 2021-04-24 16:35:46 -07:00
Ole Markus With 1ec0bd18e8 Enable support for the ASG WarmPool lifecycle hook
Update pkg/model/iam/iam_builder.go

Co-authored-by: Ciprian Hacman <ciprianhacman@gmail.com>
2021-04-24 09:40:52 +02:00
Kubernetes Prow Robot 2649cbc598
Merge pull request #10995 from haugenj/release-1.19
Add NTH Queue Processor Mode
2021-04-22 12:15:58 -07:00
Ole Markus With 020652e096 Add ability to enable/configure warm pool for ASG
Apply suggestions from code review

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>

Apply suggestions from code review

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2021-04-20 09:02:09 +02:00
Jason Haugen 7e48dad4d2 add ManagedAsgTag, merge templates, improve docs 2021-04-19 16:51:08 -05:00
Jason Haugen cceb9dd296 lifecycle integ test, docs, & small cleanup 2021-04-19 15:43:06 -05:00
Jason Haugen 318a116ba6 fix staticcheck 2021-04-19 15:43:05 -05:00
Jason Haugen c8bb48ba81 fix existing tests 2021-04-19 15:43:05 -05:00
Jason Haugen d07b067249 Add NTH queue-processor mode 2021-04-19 15:43:05 -05:00
John Gardiner Myers fdc61b4bdb Rename the service account key 2021-04-11 08:11:27 -07:00
liranp 97370b0adc
fix(spot/ocean): configure headroom resources only at the vng level 2021-04-06 23:41:40 +03:00
Ole Markus With 20bd724f5e Add support for scaling out the control plane with dedicated apiserver nodes
Ensure apiserver role can only be used on AWS (because of firewalling)

Apply api-server label to CP as well

Consolidate node not ready validation message

Guard apiserver nodes with a feature flag

Rename Apiserver role to APIServer

Add an integration test for apiserver nodes

Rename Apiserver role to APIServer

Enumerate all roles in rolling update docs

Apply suggestions from code review

Co-authored-by: Steven E. Harris <seh@panix.com>
2021-03-20 20:57:00 +01:00
Ole Markus With 397f58deb4 Fix comments from review 2021-03-19 20:51:18 +01:00
Ole Markus With 5178571db5 Comment where the CA sha1s come from 2021-03-19 20:07:57 +01:00
Ole Markus With 1900548213 Upload JWKS files as world readable 2021-03-19 20:07:38 +01:00
Ole Markus With 2c1f88f40e Do not need thumbprints to be resources 2021-03-19 20:05:37 +01:00
Ole Markus With ed166313d2 Use well-known s3 fingerprints 2021-03-19 20:03:28 +01:00
Peter Rifel 7c900b7fae Generate and upload keys.json + discovery.json to public store
Generate and upload keys.json + discovery.json to public store

Don't enable anonymous auth on publicjwks

Remove tests that won't work using FS VFS anymore
2021-03-19 20:03:26 +01:00
liranp dc1ee9402a
feat(spot/ocean): support for block device mappings in launchspec 2021-03-10 15:30:39 +02:00
Bharath Vedartham 0c0767c0c9 Remove support for launch configurations 2021-03-09 09:04:15 +02:00
Ole Markus With c6a741a148 Move dns and external_access to awsmodel 2021-03-07 22:07:17 +01:00
Ole Markus With d415fdf1a1 Move bastion model to awsmodel 2021-03-07 22:06:20 +01:00
Ole Markus With 896f1740c6 Rename spotinst symbols and merge spotinstmodel with awsmodel 2021-03-07 22:06:12 +01:00
Peter Rifel ce51ec44bc
Use new CPUCredits IG spec field in launch templates 2021-03-02 22:54:29 -06:00
liranp 2abdb90c54
fix: don't skip lb attachments when hybrid is enabled 2021-03-01 14:07:22 +02:00
Kubernetes Prow Robot 1b42286cfe
Merge pull request #10832 from rifelpet/aws-sdk
Add Tagging to Instance Profiles and OIDC Providers
2021-02-24 05:40:50 -08:00
Timothy Clarke 1577b0a54b
Adding Elastic IP Allocations to NLB API 2021-02-18 12:27:28 +00:00
Peter Rifel d52fd9f76c
Add tagging support to AWS Instance Profiles and OIDC Providers 2021-02-15 16:48:43 -06:00
Kubernetes Prow Robot cd10383fa0
Merge pull request #10741 from codablock/nlb-subnets
Allow to control which subnets and IPs get used for the API loadbalancer
2021-02-14 14:23:06 -08:00
Alexander Block 295fb11ac2 Better readable modification assigning of PrivateIPv4Address 2021-02-10 09:39:32 +01:00
Alexander Block 2c0f9809eb Move validation of ClusterSubnetSpec into pkg/apis/kops/validation 2021-02-10 09:36:39 +01:00
Alexander Block c6eca9db81 Fix check for empty privateIPv4Address 2021-02-10 08:21:22 +01:00
Kubernetes Prow Robot 4507be8e13
Merge pull request #10469 from justinsb/boot_nodes_from_kops_controller
Boot nodes without state store access
2021-02-08 11:28:19 -08:00
Peter Rifel e7ede2b13e
Use EnsureTask instead of prepending IG names to external ELB tasks
This way we end up with one CLB task per CLB regardless of how many ASGs to which it is attached.
2021-02-07 10:45:38 -06:00
Alexander Block 6facd1b8ab Allow to explicitely choose subnets and private IPs for the API loadbalancer 2021-02-05 17:53:20 +01:00
Alexander Block 49e7ec8890 Use SubnetMappings for NLBs instead of Subnets
SubnetMappings allow to explicitely set the private IPv4 address that
must be used for the NLB.

SubnetMappings and Subnets in the AWS API are compatible as long as the
address settings are not changes, making this commit backwards compatible.
2021-02-05 17:53:20 +01:00
Ciprian Hacman f8d3b76556 Default IMDSv2 to "optional" for AWS 2021-01-29 14:02:14 +02:00
Ciprian Hacman 5fcd4e4b28 Allow attaching same external load balancer to multiple instance groups 2021-01-27 16:25:39 +02:00
Ciprian Hacman d889d61ddb Set default IMDS v2 to "required" for instances in AWS 2021-01-21 11:35:41 +02:00
Ciprian Hacman c8a9b2fb3e Set default volume encryption to "true" for instances in AWS 2021-01-21 11:27:02 +02:00
Ciprian Hacman 18bb14ffed Set default volume type to "gp3" for instances in AWS 2021-01-21 11:27:02 +02:00
Ciprian Hacman 85fbf1c6a2 Add iops field for gp3 volumes only with launch templates 2021-01-21 11:27:02 +02:00
Ole Markus With afbd057286 Use consistent naming for the remaining SGRs 2021-01-14 12:57:33 +01:00
Justin SB d5294b0b7c Update test data for richer bootstrap script 2021-01-09 13:29:18 -05:00
Ciprian Hacman a7bb949936 Add possibility to set volume throughput for gp3 volumes 2021-01-05 13:18:32 +02:00
Steven E. Harris 2a89d25ed0 Test that launch templates include additional SGs 2021-01-04 08:38:25 -05:00
Steven E. Harris 252d4177f0 Only include API server SGs in IGs for masters
When using an AWS NLB in front of the Kubernetes API servers, we can't
attach the EC2 security groups nominated in the Cluster
"spec.api.loadBalancer.additionalSecurityGroups" field directly to the
load balancer, as NLBs don't have associated security groups. Instead,
we intend to attach those nominated security groups to the machines
that will receive network traffic forwarded from the NLB's
listeners. For the API servers, since that program runs only on the
master or control plane machines, we need only attach those security
groups to the machines that will host the "kube-apiserver" program, by
way of the ASG launch templates that come from kOps InstanceGroups of
role "master."

We were mistakenly including these security groups in launch templates
derived from InstanceGroups of all of our three current roles:
"bastion," "master," and "node." Instead, skip InstanceGroups of the
"bastion" and "node" roles and only target those of role "master."
2021-01-04 08:38:25 -05:00
Steven E. Harris ad4ac4f474 Test that AWS launch templates include wrong SG 2021-01-04 08:38:25 -05:00
Kubernetes Prow Robot bee16c052d
Merge pull request #10324 from bharath-123/feature/aws-imdv2
Add support for AWS IMDS v2
2020-12-07 22:55:11 -08:00
Ciprian Hacman 265bf4d106 Add option for setting the volume encryption key in AWS 2020-12-08 07:08:09 +02:00
Bharath Vedartham 7f6e125733 Add support for aws ec2 instance metadata v2
A new field is add to the InstanceGroup spec with 2 sub fields,
HTTPPutResponseHopLimit and HTTPTokens. These fields enable the user
to disable IMDv1 for instances within an instance group.

By default, both IMDv1 and IMDv2 are enabled in instances in an instance group.
2020-12-07 02:57:02 +05:30
Kubernetes Prow Robot 0fecffbfe0
Merge pull request #10284 from johngmyers/service-account-issuer
Set --service-account-issuer for k8s 1.20+
2020-12-04 08:07:59 -08:00
John Gardiner Myers 4f5def8610 Address review comment 2020-12-03 23:24:43 -08:00
Ciprian Hacman e57cd534b5 Allow attaching same external target group to multiple instance groups 2020-12-03 06:59:59 +02:00
Ciprian Hacman 19345c3f7f Order attached TargetGrups list by name 2020-11-20 10:40:27 +02:00
Ciprian Hacman fdcc2607bf Parse TargetGrup names from ARNs 2020-11-20 10:40:26 +02:00
Frank Yang 93dcaddc48 feat(aws): add PolicyNames for ELB to change listener's security policy 2020-11-19 16:07:21 +08:00