Node termination handler as all daemonSets may play a critical role in
capacity planning, define resource policy for chosing instanceType etc.
In this commit, we enable users to define resources themselves to meet
their needs and also removed limits to convey with the chosen strategy
to avoid limits on such components.
Signed-off-by: dntosas <ntosas@gmail.com>
- Update manifests
- Bump components version
- Add API capability of setting Version + VolumeLimit
- Remove snapshot-controller resources as it should be independent from
any CSI driver
Signed-off-by: dntosas <ntosas@gmail.com>
Apply suggestions from code review
Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
Apply suggestions from code review
Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
Cilium as a CNI is a critical component for the cluster so it would be safe
to have some guaranteed resources as well as allowing the users to
define them based on their needs.
In this commit, we init default requested resources and add the
capability of user-defined values.
Signed-off-by: dntosas <ntosas@gmail.com>
* 'sv' may have 'nil' or other unexpected value as its corresponding error variable may be not 'nil'
* Apply suggestions from code review
Co-authored-by: Ciprian Hacman <ciprian@hakman.dev>
This commit also introduces support for adding token projection volumes for well-known SAs.
Slightly less complicated than explicitly parsing the objects for a manifest
Ensure apiserver role can only be used on AWS (because of firewalling)
Apply api-server label to CP as well
Consolidate node not ready validation message
Guard apiserver nodes with a feature flag
Rename Apiserver role to APIServer
Add an integration test for apiserver nodes
Rename Apiserver role to APIServer
Enumerate all roles in rolling update docs
Apply suggestions from code review
Co-authored-by: Steven E. Harris <seh@panix.com>
Should not be needed; dns-controller should run on the control-plane
node so there should not be a bootstrapping problem with the nodes.
Reverts #10529
Generate and upload keys.json + discovery.json to public store
Don't enable anonymous auth on publicjwks
Remove tests that won't work using FS VFS anymore
When choosing a default value for the Cluster spec's
"cloudConfig.manageStorageClasses" field, first check whether a user
specified a concrete value for the related OpenStack
"blockStorage.createStorageClass" field. If so, use that value as the
effective default value for the former field as well, so as to avoid
an unnecessary conflict between the field values on the second
validation pass.
The public api url cannot be used by pods and nodes if access is restricted. So by default we need to use the internal one.
This should finally pass the OIDC e2e test
For public access, api server must be publically available and anonymous
auth must be enabled
As Azure doesn't allow "/" in label keys, we used to convert "/" to
"_", but we didn't convert back "_" to "/" in kops-controller.
Change the code to store a label key in a tag value so that we don't
need to worry about escaping.
I opted to just use a *string for the TGW ID in the route task rather than a full TGW task because kops will never be creating TGWs itself, they will always be defined by ID.
SubnetMappings allow to explicitely set the private IPv4 address that
must be used for the NLB.
SubnetMappings and Subnets in the AWS API are compatible as long as the
address settings are not changes, making this commit backwards compatible.
This ensures that we're using our settings for downloading nodeup
itself and any assets that nodeup downloads. This is a workaround for
reported problems with the initial download on some kernels otherwise.
Issue #10206
Upstream changes:
* Release notes for 3.0.20201209
* Handling hanging list command from etcd client during reconciliation loop
* Update dependencies to match kOps 1.19
* Include IPs in peer certificates
* Use etcd v3.4.13 in tests
* Eager-broadcast leadership on every leader iteration
* Update Go to 1.15.7
Currently, kOps uses cgroupfs cgroup driver for the kubelet and CRIs. This PR defaults
the cgroup driver to systemd for clusters created with k8s versions >= 1.20.
Using systemd as the cgroup-driver is the recommended way as per
https://kubernetes.io/docs/setup/production-environment/container-runtimes/
- Resources
We enable users to set their desired capacity for cluster-autoscaler addon.
There are edge cases, especially in big clusters, where autoscaler needs
to reconcile a large number of objects thus may need increased memory or
increased cpu to avoid saturation.
- Metrics
Cluster autoscaler provides valuable insights for monitoring capacity
allocation and scheduling aspects of a cluster. In this commit, we
add proper annotation on deployment to enable Prometheus scrape metrics.
We also bump patch version of container images.
Signed-off-by: dntosas <ntosas@gmail.com>
kops-controller can now serve the instance group & cluster config to
nodes, as part of the bootstrap process.
This enables nodes to boot without access to the state
store (i.e. without S3 / GCS / etc permissions)
Feature-flagged behind the KopsControllerStateStore feature-flag.
These are needed by protokube to create the kops-controller DNS record to allow nodes to bootstrap.
See these logs: https://storage.googleapis.com/kubernetes-jenkins/logs/e2e-kops-grid-scenario-public-jwks/1345956556562239488/artifacts/ip-172-20-48-1.sa-east-1.compute.internal/protokube.log
```
I0104 05:03:51.264472 6482 dnscache.go:74] querying all DNS zones (no cached results)
I0104 05:03:51.264570 6482 route53.go:53] AWS request: route53 ListHostedZones
W0104 05:03:51.389485 6482 dnscontroller.go:124] Unexpected error in DNS controller, will retry: error querying for zones: error querying for DNS zones: AccessDenied: User: arn:aws:sts::768319786644:assumed-role/masters.e2e-kops-scenario-public-jwks.test-cncf-aws.k8s.io/i-05b1db10d1a5b8637 is not authorized to perform: route53:ListHostedZones
```
and the nodeup logs on nodes that couldn't join the cluster:
```
Jan 04 04:55:53.500187 ip-172-20-38-84 nodeup[2070]: W0104 04:55:53.500117 2070 executor.go:131] error running task "BootstrapClient/BootstrapClient" (9m52s remaining to succeed): Post "https://kops-controller.internal.e2e-kops-scenario-public-jwks.test-cncf-aws.k8s.io:3988/bootstrap": dial tcp: lookup kops-controller.internal.e2e-kops-scenario-public-jwks.test-cncf-aws.k8s.io on 127.0.0.53:53: no such host
```
add io2 case and correct IOPS minimum value check
add gp3 case
add io2 and gp3 parameter ratio validation logic
add volumeThroughput parameter for disks that support it
add volumeThroughput components throughout ebs structs
add volumeThroughput to versioned api
updated api machinery and crds
apimachinery update
When using an AWS NLB in front of the Kubernetes API servers, we can't
attach the EC2 security groups nominated in the Cluster
"spec.api.loadBalancer.additionalSecurityGroups" field directly to the
load balancer, as NLBs don't have associated security groups. Instead,
we intend to attach those nominated security groups to the machines
that will receive network traffic forwarded from the NLB's
listeners. For the API servers, since that program runs only on the
master or control plane machines, we need only attach those security
groups to the machines that will host the "kube-apiserver" program, by
way of the ASG launch templates that come from kOps InstanceGroups of
role "master."
We were mistakenly including these security groups in launch templates
derived from InstanceGroups of all of our three current roles:
"bastion," "master," and "node." Instead, skip InstanceGroups of the
"bastion" and "node" roles and only target those of role "master."
Introduce a new "encapsulationMode" field in Calico's portion of the
Cluster specification to allow switching between the the IP-in-IP and
VXLAN encapsulation protocols. For now, we accept the values "ipip"
and "vxlan," and forgo a possible "none" value that would disable
encapsulation altogether (at least for the default Calico IP pool).
Augment the default-populating procedure for Calico to take this field
into account when deciding both which networking backend to use and
whether to use IP-in-IP or VXLAN encapsulation for the default IP
pool. Note that these values supplied for the "CALICO_IPV4POOL_IPIP"
and "CALICO_IPV4POOL_VXLAN" environment variables in the "calico-node"
DaemonSet pod spec only matter for creating the "default" IPPool pool
object when no such objects already exist.
Generalize the documentation for the "crossSubnet" field to cover
environments more broad than just AWS, as Calico can employ this
selective encapsulation in any environment in which it can detect
boundaries between subnets.
Ole Markus With