Automatic merge from submit-queue.
work on using files assets
Basic MVP for file assests.
- using file assest builder
- able to upload files
- using URL structs instead of strings everywhere
Automatic merge from submit-queue.
Adds permissions for ELB and NLB req'd by 1.9
Adds appropriate IAM permissions to Masters (in restrictive mode) for ELB and NLB.
Closes https://github.com/kubernetes/kops/issues/3883
File assets and the SHA files are uploaded to the new location. Files
when are users uses s3 are upload public read only. The copyfile task
uses only the existing SHA value.
This PR include major refactoring of the use of URLs. Strings are no
longer categnated, but converted into a URL struct and path.Join is
utlilized.
A new values.go file is included so that we can start refactoring more
code out of the "fi" package.
A
Automatic merge from submit-queue.
add imagePullProgressDeadline to kubelet config
Support the kubelet runtime flag `--image-pull-progress-deadline` by mapping the config key `imagePullProgressDeadline`
This supports extending the deadline to pull new images, as detailed in [this issue](https://github.com/openshift/origin/issues/13122)
Automatic merge from submit-queue.
Fix node counts
When running `kops validate`, and the cluster size is greater than the minimum configuration, the display message is
<img width="153" alt="screen shot 2017-12-07 at 1 10 14 pm" src="https://user-images.githubusercontent.com/11003242/33738958-1571943e-db50-11e7-9156-f034c9af7d9c.png">
This PR sets the NodeCount value to the length of the nodes array instead of the configuration.
Automatic merge from submit-queue.
Allow additional SGs to be added to API loadbalancer
Allow adding precreated additional security groups to the API loadbalancer using cluster spec:
```yaml
spec:
api:
loadBalancer:
type: Public
additionalSecurityGroups:
- sg-exampleid3
- sg-exampleid4
```
- [x] Adding additionalSecurityGroups cluster spec
- [x] Adding validation for repeated security groups
- [x] Adding validation for API loadbalancer security groups
- [x] Integration test for API loadbalancer and its security groups
- [x] Update API docs and cluster.spec docs
Automatic merge from submit-queue.
Add additionalNetworkCIDRs to support VPCs with multiple CIDRs in AWS
Add additionalNetworkCIDRs to support VPCs with multiple CIDRs in AWS.
@justinsb I cannot find anywhere that does a check on an existing VPC to see if the networkCIDR matches what is on the VPC defined, I was looking for that so I can add a similar check for this. Am I missing something or is there really no check like that?
Automatic merge from submit-queue.
Let a user set a hostnameOverride when the cloud provider is aws.
Let a user use the hostname or set a hostnameOverride when the cloud provider is aws. This would allow for a more descriptive name to be used. The name of the hosts when using @hostname can be set by using a hook or some other method.
We've done this in the API already, but we had a single CAStore
interface that did Keysets and SSHCredentials. Separate out
SSHCredentials into SSHCredentialStore, and start using API objects as
our primary representation.
Automatic merge from submit-queue.
Fix spurious shared InternetGateway renaming
This fixes the original issue described in #476 where `kops update cluster` would continuously report renaming a shared IGW even though it never actually renames it. I also added tests to confirm the behavior change.
I removed the bug mention from the docs although if you would prefer to keep it and instead say `In kops versions before X, ...` I can do that too
Automatic merge from submit-queue.
When using private DNS add ELB name to the api certificate
This fixes issue #2032 by using the gossip paths with private dns as well:
* When creating the api server certificate, include the ELB hostname
* When generating kubeconfig, use the ELB hostname as the api server name
Automatic merge from submit-queue.
Add missing permissions for NLB creation
Without this permissions is not possible to create a network load balancer (alpha in k8s >= 1.9)
Automatic merge from submit-queue.
DNS Controller Optional
The current implementation enforces a dns-controller is running; given the user can switch the make the kube-apiserver server Internal and then reuse the dns for the masterInternalName; this effectlively removes the need to run the service (assuming your not using it for pods, node and service dns)
- adding a disableDnsController to the ExternalDNS spec provides a toggle on the addon (name is definitely up for debate)
- the default behaviour remains, the dns-controller is always pushed as an addon
Automatic merge from submit-queue.
Respect the shared tag when deleting route tables
Fixes#3828.
Modifies the `buildTrackerForRouteTable` function (used by `ListRouteTables`) to set the `Shared` field of each returned route table resource, based on the presence of the `kubernetes.io/cluster/<clustername>: shared` tag. This prevents route tables with this tag from being deleted.
WIP while I add some more tests.
The current implementation requires enforces a dns-controller is running; given the user can switch the make the kube-apiserver server Internal and then reuse the dns for the masterInternalName; this effectlively removes the need to run the service (assuming your not using it for pods, node and service dns)
- adding a disableDnsController to the ExternalDNS spec provides a toggle on the addon (name is definitely up for debate)
- the default behaviour remains, the dns-controller is always pushed as an addon
Automatic merge from submit-queue.
Changing the prefix of the ResourceTag condition
The prefix was `ec2` and it was not working, changing it to `autoscaling` should do the trick. This should fix#3871
Automatic merge from submit-queue.
Improving UX for placeholder IP Address
Before the `kops validate cluster` attempts to connect to the K8s API
endpoint, the code now checks to see if the API DNS Entry is the kops
placeholder IP Address 203.0.113.123. It prints a message to the user
and err's. There is a new init func in validate cluster that disables
CGO based DNS for Darwin OS. Darwin does two things with kops
validates; it caches the IP address, and it does not return the
placeholder IP address. We cannot use CGO base DNS with kops validate with OSX.
Before the `kops validate cluster` attempts to connect to the K8s API
endpoint, the code now checks to see if the API DNS Entry is the kops
placeholder IP Address 203.0.113.123. It prints a message to the user
and err's. There is a new init func in validate cluster that disables
CGO based DNS for Darwin OS. Darwin does two things with kops
validates; it caches the IP address, and it does not return the
placeholder IP address. We cannot use CGO base DNS with kops validate.
Automatic merge from submit-queue.
Include encryptionConfig setting within userdata for masters.
When updating the kops `ClusterSpec.EncryptionConfig` setting, a `kops update cluster / kops rolling-update cluster` mentions no changes required, and so a forced rolling-update is required for the setting to be picked up.
Including this in bootstrapscript will add the setting to the user-data for the master nodes only (because it applies to the KubeAPIServer), marking them as requiring a rolling-update.
Automatic merge from submit-queue.
add openstack cloud provider
Add an Openstack cloud provider. It does not implement all the interfaces of fi.Cloud, hence, can not create a cluster, but it can pass the work flow of creating cluster for the command like "kops create cluster --cloud openstack --zones nova -v 15 --target direct --yes myoscluster4.k8s.local"
Which issue this PR fixes: #3819
Automatic merge from submit-queue.
Add a default NodeLabel with the InstanceGroup name
As requested in https://github.com/kubernetes/kops/issues/2999, this change just auto-populates new InstanceGroup specs with a default node label containing the name of the instance group. It would be really useful for those of us managing environments with multiple instance groups.
It allows an admin to easily view the instance groups using kubectl:
```
kubectl get nodes --label-columns kops.k8s.io/instancegroup
NAME STATUS AGE VERSION INSTANCEGROUP
ip-172-20-108-120.eu-west-1.compute.internal Ready,node 3m v1.7.4 xtra-large
ip-172-20-117-133.eu-west-1.compute.internal Ready,master 14m v1.7.4 master-eu-west-1c
ip-172-20-32-139.eu-west-1.compute.internal Ready,master 14m v1.7.4 master-eu-west-1a
ip-172-20-32-92.eu-west-1.compute.internal Ready,node 12m v1.7.4 nodes
ip-172-20-67-184.eu-west-1.compute.internal Ready,master 13m v1.7.4 master-eu-west-1b
```
Automatic merge from submit-queue.
Add Node IAM permissions to access kube-router key in S3.
Fixes#3792
An additional S3 IAM permission is added to the nodes policy when `Networking.Kuberouter` is specified.
Automatic merge from submit-queue.
Add node-to-master IPIP to kuberouter
Like Calico and Romana, Kube Router needs IPIP traffic from nodes to masters to be allowed. This adds that firewall rule for all clusters set up with Kube Router.
See:
https://github.com/cloudnativelabs/kube-router/issues/208
Automatic merge from submit-queue.
Template Fixes
- fixing an issue in the way it handled inline yaml documents
- cleaning it up somewhat, the prior version was a little cryptic to read; it's easier to just split into docs, format if required and rejoin at the end.
Apologizes @justinsb ... the prior PR didn't handle YAML separators correctly; I've find it somewhat simpler by a split and join rather than using indexes and such
- fixing an issue in the way it handled inline yaml documents
- cleaning it up somewhat, the prior version was a little cryptic to read; it's easier to just split into docs, format and rejoin at the end
Automatic merge from submit-queue.
Update IAM roles documentation based on recent changes.
The [IAM Roles documentation](https://github.com/kubernetes/kops/blob/master/docs/iam_roles.md) has been updated to reflect recent hardening on the policies generated for Master & Compute nodes.
Fixes#3557.
The challenge here is that we normally only get the instance url. So we
have to do another call to GCE, but we also don't want to do one call
per instance.
Instead, we create a dump operation context object which we pass into the dump.
Automatic merge from submit-queue.
GCS: Use ACLs for GCE permissions
This needs less permissions, but also allows for more granular control over
access to files than whole-bucket permissions that IAM gives us.
This lets us configure cross-project permissions while ourselves needing
minimal permissions, but also gives us a nice hook for future lockdown
of object-level permissions.
Automatic merge from submit-queue.
API docs updates
- tweak to .gitignore
- updating api docs
- updated docs/apireference/README.md with how to generate docs
@justinsb any idea what happened to the hack script? `hack/make-gendocs.sh` did not work last time I tried to do this, but I updated the instructions in the `docs/apireference/README.md` for how to get it to work now.
This ensures that the cluster can read the kops state store files, even
if the GCS bucket is in a different project.
We automatically set up an IAM access policy that grants access.
Automatic merge from submit-queue.
Implement DigitalOcean Droplet FI Task
Implements cloudup fi tasks for DigitalOcean droplets. It makes a few assumptions to reduce the size of this PR, those will be addressed in future PRs.
Also does some cleanup in the DigitalOcean `dns` package.
Justin Santa Barbara
