We'll need to move the current release notes in master for 1.7.1 to 1.8.0 since
they were not included in this release at this time and cherry-pick
these notes into master.
Automatic merge from submit-queue.
Initial bazel support
Builds on the 1.8 version bump
The "trick" is to strip the BUILD & BUILD.bazel files from the vendor-ed deps.
Will rebase after 1.8 version bump merges.
Automatic merge from submit-queue. .
promoting drain and validate by setting feature flag to true
I am unable to recreate https://github.com/kubernetes/kops/issues/2407, and frankly, it may be an edge case. We could warn a user if their wait times are low, but that would be another PR.
This PR moves Drain and Validate functionality for rolling-updates into the default user experience, setting the Feature Flag to true.
Per feedback, I am using the node and master interval times for the validation.
Automatic merge from submit-queue. .
Support encryption-at-rest for the kube-apiserver
This PR adds support for enabling encryption-at-rest for data in etcd, via the kube-apiserver (as per https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data).
I've put the functionality behind a feature flag, `+EnableDataEncryption`. It can then be enabled per-cluster by using `--enable-encryption-config` on the command line, or by adding a `kubeEncryptionConfig` section to the cluster spec. This is passed through to the kube-apiserver by the nodeup process. I'm not sure if this is the best way of doing it right now, but it is working.
Fixes#3356.
Automatic merge from submit-queue. .
improve documentation for secrets
I was struggling on how to change a the api basic auth password for my cluster and found this to be the only way to do it. If there is an easier way, let me know and i can adapt the docs again :)
- add a new kops secret command to store encryption config
- add the experimential-encryption-provider-config flag to the kube-apiserver
- add functionality to nodeup to create the stored encryption config in the correct path
Automatic merge from submit-queue. .
Allow specifying a SSH key name for AWS
Related to #2309, this allows naming an existing key pair using the
cluster spec field `sshKeyName`.
In our use case, kops can now be used without providing the ability to
create EC2 key pairs.
Automatic merge from submit-queue
Add romana to built-in CNI options
This PR adds `romana` as a networking option for kops.
It installs the latest "preview" release of Romana v2.0, which provides the expected features in terms of IP allocations and route configuration. Network policy features are being ported to 2.0 and will be in the final release. (We intend to submit a followup PR for kops as part of that rolling out that release.)
Note: in this setup, we're using the etcd cluster that kops deploys for k8s. This isn't ideal, but some possibilities (eg: StatefulSets) aren't practical for the CNI itself, and creating a parallel etcd cluster via manifests seemed to be a more-intrusive approach than using the existing one.
If this is a concern or problem, then I'm very open to discussing and implementing it based on your suggestions.
Also, some functionality is exclusive to AWS environments. Other cloud platforms are on Romana's roadmap but not developed yet. Let me know that restriction needs to be enforced in code or directly documented.
Automatic merge from submit-queue
Add a doc to describe how to use kops in AWS China
Closes#2933, closes#3282.
@wangycc, @mgyong, @hb404 This doc might be helpful for you guys to deploy a cluster in AWS China Region.
@chrislovecnm, @justinsb Please help to review. Thanks.
Automatic merge from submit-queue
small docs improvement
Make cmd1 populate the var for cmd2
Using `tee /dev/stderr` allows the user to still get output if they copy-pasta the first command. The other changes prepare the HZC variable so it can be used if they copy-pasta the second command.
Automatic merge from submit-queue
Added ability to set --service-node-port-range
Addresses: #3052 and #3326
From what I can tell, `--service-node-port-range` has been a valid option forever, so I haven't worried about k8s versions.
@justinsb @chrislovecnm let me know if any changes are required.
Using `tee /dev/stderr` allows the user to still get output if they copy-pasta the first command. The other changes prepare the HZC variable so it can be used if they copy-pasta the second command.
Automatic merge from submit-queue
Flannel: change default backend type
We support udp, which has to the default for backwards-compatibility,
but also new clusters will now use vxlan.
Extending the current implementation of toolbox template to include multiple files and snippets. Note, i've removed the requirements for defaults as I think people should be forced to specifically pass them.
- fixing the vetting iseues to the method YamlToJson -> YAMLToJSON
- adding a safety check to ensure templates don't reference an unknown value
- extending the unit test to ensure the above works on main and snippets
- include the ability to specify multiple configuration files, useful for common.yaml and prod.yaml etc
Requested Changes - Toolbox Templating
Added the requested changes
- moved the templater into it's own package rather than using base util
- moved to using the sprig library for additional template function
- @note: i couldn't find a native way in sprig to do snippets, also the i've overloaded the indent as it appears to do the indent on all lines rather than on the newline, meaning i'd have to shift my first line back by the indent to get it to work, which seems ugly
Related to #2309, this allows naming an existing key pair using the
cluster spec field `sshKeyName`.
In our use case, kops can now be used without providing the ability to
create EC2 key pairs.
aws.md linked to advanced_create.md which was deleted in
https://github.com/kubernetes/kops/pull/2725. Its contents were added to
high_availability.md. Update the link to point to the relevant section in
high_availability.md.
Signed-off-by: Steven Santos Erenst <stevenerenst@gmail.com>
Automatic merge from submit-queue
Adds DNSControllerSpec and WatchIngress flag
This PR is in reference to #2496, #2468 and the issues referenced in there relating to use of the watch-ingress flag.
This PR attempts to rectify this situation and gives users who want it, the option to turn on watch-ingress without forcing it on them. Also spits out a warning to the logs about potential side effects.
Includes notes in `docs/cluster_spec.md` to explain.
Automatic merge from submit-queue
Cluster / InstanceGroup File Assets
@chrislovecnm @justinsb ...
The current implementation does not make it ease to fully customize nodes before kube install. This PR adds the ability to include file assets in the cluster and instaneGroup spec which can be consumed by nodeup. Allowing those whom need (i.e. me :-)) greater flexibilty around their nodes. @Note, nothing is enforced, so unless you've specified anything everything is as the same
- updated the cluster_spec.md to reflect the changes
- permit users to place inline files into the cluster and instance group specs
- added the ability to template the files, the Cluster and InstanceGroup specs are passed into context
- cleaned up and missed comment, unordered imports etc along the journey
notes: In addition to this; need to look at the detecting the changes in the cluster and instance group spec. Think out loud perhaps using a last_known_configuration annotation, similar to kubernetes
- removed the Mode field from the FileAsset spec
- removed the ability to template the content
- removed the need to specify the Path and instead default to /srv/kubernetes/assets/<name>
- change the FileAssets from []*FileAssets to []FileAssets
The current implementation does not make it ease to fully customize nodes before kube install. This PR adds the ability to include file assets in the cluster and instaneGroup spec which can be consumed by nodeup. Allowing those whom need (i.e. me :-)) greater flexibilty around their nodes. @Note, nothing is enforced, so unless you've specified anything everything is as the same
- updated the cluster_spec.md to reflect the changes
- permit users to place inline files into the cluster and instance group specs
- added the ability to template the files, the Cluster and InstanceGroup specs are passed into context
- cleaned up and missed comment, unordered imports etc along the journey
Automatic merge from submit-queue
Changes on CoreOS related documentation.
- Changed "experimental" to "production-ready" status on CoreOS
images.
- Added a complete tutorial/exercise using a multi-master setup
with CoreOS
Automatic merge from submit-queue
Add documentation on handling node resources
At a minimum, this is meant to give more context on why the feature in https://github.com/kubernetes/kops/pull/2982 was added and attempts to give some recommendations of what to consider when evaluating node system resources.
I hope this spurs some discussion and that the recommendations I make maybe be assessed further. For example ... in one of the links I referenced, we're advised to set `system-reserved` **only if we know what we are doing** (which I can't say I do 💯% ... 🤷♂️) and we're even warned to only set it if you really need to.
- switched to using an array of roles rather than boolean flags for node selection
- fixed up the README to reflect the changes
- added the docker.service as a Requires to all docker exec hooks
- extending the hooks to permit adding hooks per instancegroup as well
- @note, instanceGroup are permitted to override the cluster wide one for ease of testing
- updated the documentation to reflect the changes
- on the journey tried to fix an go idioms such as import ordering, comments for global export etc
- @question: v1alpha1 doesn't appear to have Subnet fields, are these different version being used anywhere?
The present implementation of hooks only perform for docker exec, which isn't that flexible. This PR permits the user to greater customize systemd units on the instances
- cleaned up the manifest code, added tests and permit setting a section raw
- added the ability to filter hooks via master and node roles
- updated the documentation to reflect the changes
- cleaned up some of the vetting issues
The current implementation does not permit the user to order the hooks. This PR adds optional Requires, Before and Documentation to the HookSpec which is added the systemd unit if specified.
Automatic merge from submit-queue
Kubelet API Certificate
A while back options to permit secure kube-apiserver to kubelet api was [PR2381](https://github.com/kubernetes/kops/pull/2831) using the server.cert and server.key as testing grounds. This PR formalizes the options and generates a client certificate on their behalf (note, the server{.cert,key} can no longer be used post 1.7 as the certificate usage is checked i.e. it's not using a client cert). The users now only need to add anonymousAuth: false to enable secure api to kubelet. I'd like to make this default to all new builds i'm not sure where to place it.
- updated the security.md to reflect the changes
- issue a new client kubelet-api certificate used to secure authorize comms between api and kubelet
- fixed any formatting issues i came across on the journey
Automatic merge from submit-queue
Clarify docs: rename spec/specification into desired configuration
The cluster state in S3 has (among others) two files: `cluster.spec` and `config`.
When the documentation mentioned "create or update cluster spec" for example, it was confusing what was actually updated. It's not the cluster.spec file.
As I understand, `cluster.spec` should only be created/updated after `kops update --yes` is run.
I changed the docs for `kops get`, `kops create`, `kops replace`, `kops edit`.
I did NOT change those files: `kops_rolling-update.md`, `kops_rolling-update_cluster.md` as I think those actually use `cluster.spec`.
In the S3 bucket, the file cluster.spec is not actually the spec, but the
actual configuration. The file config is the spec. To avoid confusion,
this commit changes spec/specification into 'desired configuration' in
the documentation, to avoid associating cluster.spec with a cluster
'specification' that the users should use.
- fixed the various issues highlighted in https://github.com/kubernetes/kops/pull/3125
- changed the docuementation to make more sense
- changed the logic of the UseSecureKubelet to return early
A while back options to permit secure kube-apiserver to kubelet api was https://github.com/kubernetes/kops/pull/2831 using the server.cert and server.key as testing grouns. This PR formalizes the options and generates a client certificate on their behalf (note, the server{.cert,key} can no longer be used post 1.7 as the certificate usage is checked i.e. it's not using a client cert). The users now only need to add anonymousAuth: false to enable secure api to kubelet. I'd like to make this default to all new builds i'm not sure where to place it.
- updated the security.md to reflect the changes
- issue a new client kubelet-api certificate used to secure authorize comms between api and kubelet
- fixed any formatting issues i came across on the journey
Automatic merge from submit-queue
Add `kops create secret dockerconfig` feature
This adds a well-known secret name `dockerconfig` which will automatically
be used if present to create `/root/.docker/config.json` on all nodes. This will
allow private registries to be used for kops hooks as well as any k8s images
without the need to define `imagePullSecrets` in every namespace.
closes https://github.com/kubernetes/kops/issues/2505
Automatic merge from submit-queue
docs/topology - Fix the rolling-update command
This patch updates the `kops rolling-update` command which should be
run after making changes to the API server topology.
Fixes#3093
Signed-off-by: M S Vishwanath Bhat <msvbhat@gmail.com>
Automatic merge from submit-queue
remove source/destination check requirement for kube-router from docs
Kube-router automatically disables source/destination checks for the AWS
EC2 instances now user need not perform explicitly.
Fixed as part of https://github.com/cloudnativelabs/kube-router/issues/35
Automatic merge from submit-queue
AWS: root volume provisioned IOPS support
It is possible to set `rootVolumeType: gp2` so an instance group will use `gp2` as disk type (https://github.com/kubernetes/kops/blob/master/docs/instance_groups.md)
If the option is set to `rootVolumeType: io1` then the below error is thrown when building the cluster:
`W0731 13:44:44.298875 8784 executor.go:109] error running task "LaunchConfiguration/master-eu-west-1a.masters.cluster.name" (9m48s remaining to succeed): error creating AutoscalingLaunchConfiguration: ValidationError: Iops is required for a provisioned IOPS volume.
status code: 400, request id: 072e23b2-75ee-11e7-968a-191d6f6e9343`
This PR tries to address this so if anyone needs, it can use `io1` as disk type for the root volume.
In order to define the number of Iops set `rootVolumeIops: <number>` (minimum is 100)
If `rootVolumeType: io1` is set and `rootVolumeIops` is not defined, then `rootVolumeIops` defaults to 100.
I am not quite familiar with the Kops codebase and this is my first encounter with Go, so some changes might not be suitable (I also do not have a software developer background so bear with me :) ). Please review and add feed-back.
I've included a small change to the Makefile as I compiled Kops on a Mac and didn't had $GOPATH set after installing Go. I thought that pulling this via `go env` would be more flexible, but I am not 100% sure as I don't do much development in this area and not sure about possible use cases.
This patch updates the `kops rolling-update` command which should be
run after making changes to the API server topology.
Signed-off-by: M S Vishwanath Bhat <msvbhat@gmail.com>
Image rewriting involves a yaml format of the manifests, which makes for
a large and hard to read diff. Add a feature flag to disable it, along
with a workaround to the release notes.
* Use the pretty.LongDesc function (which doesn't pipe it through
markdown -> ascii)
* Use pretty.Bash helper to avoid problems with backticks in
go-backtick-quoted-strings
* Make sure indentation is consistent
Justin Santa Barbara
Kubernetes Submit Queue
Alberto
Rahul Mahale
chrislovecnm
Manatsawin Hanmongkolchai
Mike Splain
jatan28
Rohith
John Laswell
Julian Simioni
georgebuckerfield
Timothy Josefik
Andrey Chernih
Andrey Falko
Marius Davidsen
Kashif Saadat
Hanfei Shen
Reen Lokum
Caleb Gilmour
Bruno Bronosky
Francesco Vasco
Victor Palade
Robin Percy
John Zeringue
Steven Santos Erenst
Krishna m-Kumar
Eric Hole
Sebastian Melchior
Rajadeepan D Ramesh
Erik Stidham
Murali Reddy
Dave Powell
Alexander Brandstedt
lichuqiang
Reinaldo Martinez
lsowen
Orr Chen
King'ori Maina
Quentin Nerden
Nathan Smith
asifdxtreme
Derek VerLee
Sathiya Shunmugasundaram
Grischa Ekart
Blake
Devipriya Sarkar
Markus Padourek
Ionut Craciunescu
Paul Capestany
M S Vishwanath Bhat
AlexB138
Jean-Pierre Huynh
craigmonson
Alexander Brandstedt
Otto Yiu
Bradley
Cole Wagner
Phillip Wittrock
Frederik Nordahl Jul Sabroe
j14s
Steve Coffman
Benedict Hartley
Giancarlo Rubio
Tyler Roscoe
Erick Tryzelaar
Daniel Bryant
Justin Burnham