Commit Graph

113 Commits

Author SHA1 Message Date
Justin Santa Barbara 1b91f417e5 Build IAM policy in code
Easier to get right than relying on string manipulation, but we're still
doing the same policies, with the improvements as done by @weargoogles.
2016-08-27 21:18:23 -04:00
Justin Santa Barbara a3eda654db Revert "Revert "include change to node policy to cover #363""
This reverts commit ca1a52ff3e.
2016-08-27 17:38:01 -04:00
Justin Santa Barbara 4df50773c1 Revert "Revert "Restrict master access to state store bucket""
This reverts commit c11a370c9a.
2016-08-27 17:37:55 -04:00
Justin Santa Barbara c11a370c9a Revert "Restrict master access to state store bucket"
This reverts commit 369a6ea1db.
2016-08-27 16:31:53 -04:00
Justin Santa Barbara ca1a52ff3e Revert "include change to node policy to cover #363"
This reverts commit 969af97b60.
2016-08-27 16:31:38 -04:00
Pete Wildsmith 969af97b60 include change to node policy to cover #363 2016-08-24 17:19:54 +01:00
Pete Wildsmith 369a6ea1db Restrict master access to state store bucket
This change increases the specificity of the master's state store bucket contents permission to only the top-level folder named after the cluster.

Fixes #365
2016-08-24 17:03:10 +01:00
Justin Santa Barbara dc63e307df Allow pluggable networking: classic, kubenet, external 2016-08-22 12:32:15 -04:00
Justin Santa Barbara 9829eb8579 Make it easy to run a custom build
We introduce two env vars:

* NODEUP_URL url to nodeup that we want to run
* PROTOKUBE_IMAGE specifies the protokube Docker image to run
2016-08-19 00:13:56 -04:00
Justin Santa Barbara 10fe716125 DNS is a SPOF; make sure there are replicas
Change the default to a more sensible starting point.
2016-08-14 22:47:40 -04:00
Justin Santa Barbara 7699dc8fd2 Merge pull request #294 from justinsb/use_ssh_key
SSH key improvements
2016-08-11 22:28:41 -04:00
Justin Santa Barbara a3cfec6c24 Support changing the SSH public key
This requires that we include the OpenSSH fingerprint in the AWS key
name.
2016-08-11 12:00:52 -04:00
Justin Santa Barbara 8fb4215e17 Run CI versions of k8s
CI versions are not pushed to gcr.io, so we need to preload the images
by downloading them and doing a docker load.
2016-08-11 01:32:42 -04:00
Mykhailo Oleksiuk fad3d3a4f4 move --no-associate-public-ip to instance group 2016-08-06 14:46:46 +03:00
Mykhailo Oleksiuk a860fdbdfd add parameter --no-associate-public-ip 2016-08-04 17:19:20 +03:00
Justin Santa Barbara 2b3f55563e Run the master on the pod network, unless IsolateMaster=true
The master is now registered as a Node.  It is marked as Unschedulable,
so normal pods will not run on it.  But Daemonsets will, and it is
surprising that they don't work unless hostNetwork=true.

The default is now what seems to be expected:
* we allocate the master a real CIDR on the pod network
* kube-proxy runs on the master, so it can talk to pods
* we run kubelet on the master with enable-debugging-handlers, so
  kubectl logs etc works

To get the old behaviour, edit the cluster spec and set
`isolateMasters: true`
2016-07-28 12:12:16 -04:00
Fotios Lindiakos be2fcca933 Remove trimming in AWS templates 2016-07-26 11:14:55 -04:00
Justin Santa Barbara 5c87261622 Make sure master uses 127.0.0.1:8080 to reach apiserver
This should be a little faster
2016-07-25 10:11:42 -04:00
Justin Santa Barbara 9e9855d1a4 Simpler upgrade procedure: reuse subnet
By reusing the subnet & security groups, we are able to skip the ELB
steps of the upgrade procedure.  The new cluster also has the same
identity as the old cluster for security groups, so we don't need to
reconfigure ELB etc.

Fixes #175
Fixes #174
2016-07-22 11:47:12 -04:00
Justin Santa Barbara 11d51b04a9 Adapt IAM policies when running in cn-north-1
Fix #27
2016-07-21 22:19:43 -04:00
Justin Santa Barbara 2fa3bcc952 UX: Split create command into `create` and `update`
We separate out the `create cluster` operation from the `update cluster`
operation.  Now create cluster only creates the spec (unless you pass
--yes), and is only for new clusters.

`update cluster` works on new or existing clusters, and should be called
to apply changes.

`update` is not the best name, because it means something different in
kubectl, but I think it's a good start.
2016-07-21 11:54:09 -04:00
Justin Santa Barbara 302f23463e Configuration of admin access to ports 22 and master-443
Fix #143
2016-07-14 10:33:26 -04:00
Justin Santa Barbara f771c2af4c Add support for spot instances
Fixes #58
2016-07-10 23:56:16 -04:00
Justin Santa Barbara 5b8b4d4da3 Detect & delete new ASG launch configs
We now output a ClusterName property into the launchconfig, even though
we don't technically need it.  But it allows us to more easily detect
the cluster, and it generally seems like a good idea.

Also rename to 'autoscaling-config' and clean up the cluster name
detection logic.

Fix #96
2016-07-09 22:07:24 -04:00
Justin Santa Barbara 126c508426 Fix model: numbers must be quoted 2016-07-09 01:41:04 -04:00
Justin Santa Barbara 13e514aeac Merge pull request #93 from justinsb/fix_24
Allow configurable RootDeviceSize & RootDeviceType
2016-07-09 01:25:20 -04:00
Justin Santa Barbara b42765816e Change node role tag to match master pattern
It's not currently used, and we hadn't updated it to match the better
pattern.

k8s.io/role=master can only be in one role
k8s.io/role/master=1 allows for multiple roles
2016-07-08 22:02:32 -04:00
Justin Santa Barbara 13b8e81bd6 Allow configurable RootDeviceSize & RootDeviceType
This allows for a larger EBS root volume (and we now default to 20GB,
just like kube-up did).

We remove the BlockDeviceMappings support because it wasn't used and
made things a lot more complicated.  We always map the ephemeral
devices.

Issue #24
2016-07-08 01:11:14 -04:00
Justin Santa Barbara 9c2f92f289 AWS: set hostname-override from metadata service
This is a weird edge case, because it can't be determined in advance.

We carve out a special well-known name, `@aws`, which nodeup/protokube
will expand to the local-hostname from the aws metadata service when it
is found in the HostnameOverride value.

Ideally we wouldn't do this at all now that we have DNS integration, but
we first want to get into the tested & working configuration!

Fixes #19
2016-07-05 11:36:47 -04:00
Justin Santa Barbara 9ff5dcd902 Make IAM permission abstraction more generic
At least in the JSON representation!
2016-06-27 15:48:16 -04:00
Justin Santa Barbara d1f6f4bfab Remove AdmissionControl from top level 2016-06-27 15:37:21 -04:00
Justin Santa Barbara 947a045667 Rename DNSDomain -> ClusterDNSDomain for clarity 2016-06-27 15:36:11 -04:00
Justin Santa Barbara 26d05341b4 Move options to common stage, so that it works with terraform generation 2016-06-27 15:21:31 -04:00
Justin Santa Barbara c36607644b Better shared VPC support: more validation 2016-06-27 15:00:51 -04:00
Justin Santa Barbara a0d8302255 Merge pull request #156 from slack/protokube-dns
upup/protokube: tell protokube to use --dns-zone-name
2016-06-27 00:41:11 -04:00
Justin Santa Barbara b6cf38c96e AllocateNodeCIDRs need no longer be "bubbled down"
We have it on the KCM config; just set it there
2016-06-27 00:32:19 -04:00
Justin Santa Barbara eeed4a3031 Rationalize API to something we want to support forever 2016-06-26 23:09:02 -04:00
Justin Santa Barbara ee325435e6 Rationalize properties to the minimal set 2016-06-26 09:45:05 -04:00
Jason Hansen 0d276591d5 upup/cloudup: use configured URL for nodeup location 2016-06-26 04:26:37 +00:00
Jason Hansen 9ebe071855 upup/dns: lets protokube use sub-sub domains 2016-06-26 03:28:23 +00:00
Justin Santa Barbara ac8ca9ad06 Merge pull request #126 from justinsb/upup_use_vfs
upup: use vfs for secretstore/keystore
2016-06-23 10:26:42 -04:00
Justin Santa Barbara 93f634b428 upup: use vfs for secretstore/keystore
This is needed so that we can have encrypted storage and complex keys
(e.g. multiple CA certs).  Multiple CA certs are needed for an in-place
upgrade from kube-up v1.
2016-06-23 08:58:54 -04:00
Justin Santa Barbara fcc1f57c2d Updates for 1.3: Docker 1.11.2, 1.3 image 2016-06-23 08:58:23 -04:00
Justin Santa Barbara 0559ec1210 upup: Support for shared VPCs
A lot of work that had to happen here:

* Better reuse of config
* Ability to mark VPC & InternetGateway as shared
* Find models relative to the executable, to run from a dir-per-cluster

Fixes #95
2016-06-13 11:37:06 -04:00
Justin Santa Barbara 5edefd92a4 upup: switch from skydns to kube-dns 2016-06-13 11:32:24 -04:00
Justin Santa Barbara b52877e2ce upup: separate node & master zone configuration; validate
We allow --zones & --master-zones to be specified separately now, but we
validate for common errors (using a region where you meant a zone,
duplicating a zone, spanning regions, entering an invalid AZ etc)
2016-06-11 21:06:31 -04:00
Justin Santa Barbara 52496ac73a upup: split launchconfiguration from ASG
It is much more logical this way, and mirrors the way GCE & terraform
work.
2016-06-10 11:36:17 -04:00
Justin Santa Barbara 7c478c794f upup: harmonize model with official config types 2016-06-10 01:28:44 -04:00
Justin Santa Barbara 6e203da852 upup: split model into two parts
This is probably a good idea anyway, but it also lets us side-step the
terraform no-dots-in-tags bug.
2016-06-09 23:14:36 -04:00
Justin Santa Barbara c826f46a60 upup: support for terraform on AWS
All seems good except for a bug with volume tagging
2016-06-08 12:19:15 -04:00
Justin Santa Barbara 42e32f7379 upup: include kope-routing, but only if _kope_routing is set 2016-06-08 12:18:04 -04:00
Justin Santa Barbara 1eaf0d36a8 upup: HA support
Specifying multiple zones will bring up an HA cluster.
2016-06-07 15:44:00 -04:00
Justin Santa Barbara 8a53149eef upup: use protokube
This wires up protokube, replacing a lot of the nodeup functionality.

The option of not using protokube is retained, via the _not_protokube
tag.
2016-06-07 15:32:37 -04:00
Justin Santa Barbara 6cf5cd423e upup: apply IAM changes
We now apply changes to IAM policies, and print the diffs.
2016-06-07 15:17:59 -04:00
Justin Santa Barbara 062d2ece96 Merge pull request #54 from justinsb/upup_protokube
Use protokube with upup
2016-06-07 08:56:23 -04:00
Justin Santa Barbara 71c2835007 upup: don't hard-code v1.2.2 in image names 2016-06-04 16:12:51 -04:00
Justin Santa Barbara 540c46fe4e nodeup: pass --master argument to protokube 2016-06-04 16:11:11 -04:00
Justin Santa Barbara a4408f76be upup: better secrets support
Start creating commands to manage secrets, and also stop implicitly
creating them.
2016-05-30 18:47:20 -04:00
Mike Danese a4344e1564 Merge pull request #19 from justinsb/upup_keypair
upup: better keypair support
2016-05-17 09:33:24 -07:00
Justin Santa Barbara 1c97a94d87 Rework keypair to fit our change model
We also remove another special-case context (pki), so that it is just
another object type.
2016-05-15 21:46:53 -04:00
Justin Santa Barbara 370727aa44 Support for persistent disks, users & symlinks
These are required to mount the master PD.  We use the kubernetes
safe-format-and-mount implementation.
2016-05-13 13:29:56 -04:00
Justin Santa Barbara caccb8953f UpUp: AWS support
Adds AWS support for both cloudup & nodeup.
Also cleaning up things found along the way!
2016-05-09 13:08:27 -04:00
Justin Santa Barbara d4c2cfaae7 Initial version of upup: cloudup & nodeup
* GCE support only
* Key and secret generation
* "Direct mode" makes API calls
* "Dry run mode" previews the changes
* Terraform output (though key generation not working for master ip)
* cloud-init output (though debian image does not ship with cloud-init)
2016-05-06 16:01:33 -04:00