Commit Graph

1163 Commits

Author SHA1 Message Date
Justin Santa Barbara 613b7fea61 Map ELB attributes to terraform
Requires moving them under the LoadBalancer awstask, sadly
2016-12-18 21:55:40 -05:00
Justin Santa Barbara 0be724b696 Simplify terraform ELB tasks 2016-12-18 21:55:40 -05:00
Justin Santa Barbara c01c2af656 Mark ObjectMeta as a named field
This will work around some apimachinery bugs
(https://github.com/kubernetes/client-go/issues/8)
2016-12-14 22:26:57 -05:00
Justin Santa Barbara a97ab00788 Disable scheme:internal on ELB 2016-12-12 12:26:52 -05:00
icereval 0331f70f11 internal aws elb 2016-12-11 14:51:33 -05:00
icereval bf62eb7019 fill in RenderTerraform methods for private topology 2016-12-10 17:29:46 -05:00
alok87 99aa9d6490 Merge remote-tracking branch 'kopsrepo/master' into bastion_improvements
* kopsrepo/master: (29 commits)
  Add verify-boilerplate target
  Add logging of AWS retries
  adding hack/verify-boilerplate.sh to make ci target
  Print time remaining to succeed as a positive value
  adding hack/verify-boilerplate.sh to make ci target
  updating headers, OMG we need this in the ci
  Format resource diffs
  Include error in message when we fail to query AZs
  Import tidying
  Apply gofmt
  Update cmd/kops/validate_cluster for refactor
  Move to pkg/validation and tidy up
  Update command building pattern, a few tweaks
  adds more machine types
  fix path to adding feature doc
  Update dns-controller README
  bug in my fix header script
  updating header
  bumping weave version
  Remove old file
  ...
2016-12-04 17:24:29 +05:30
alok87 66d2e4791d IdleTimeout configurable from editcluster 2016-12-04 16:35:39 +05:30
alok87 ef73285659 Connection settings loadbalancer 2016-12-03 02:38:22 +05:30
chrislovecnm 1bbbe0b71d bumping weave version 2016-11-30 22:06:12 -07:00
alok87 fa18857b43 Auto generated fi tasks for loadbalancer attributes 2016-11-30 09:16:22 +05:30
alok87 f0b80503c3 Configure LoadBalancer Attributes 2016-11-30 07:35:45 +05:30
alok87 edf22f3797 Bastion DNS as an option and not by default 2016-11-23 12:40:45 +05:30
alok87 6b17c27572 Bastion Improvements 2016-11-23 12:37:42 +05:30
chrislovecnm a47e0ccc10 missed addons 2016-11-22 16:29:05 -05:00
chrislovecnm 6e9a88151a starting work on limits 2016-11-22 13:20:57 -07:00
chrislovecnm 25ee1e4cdb adding weave support 2016-11-16 15:48:32 -07:00
Justin Santa Barbara ee44353cde Add support for kopeio networking 2016-11-16 14:20:23 -05:00
alok87 839707debe Comment should be inside the if block Fix for - https://github.com/kubernetes/kops/issues/862 2016-11-10 21:50:10 +05:30
Kris Childress 6f78e0ca18 Flipping associatePublicIP bool for nodes/bastion/master in private topology 2016-11-08 15:16:41 -08:00
Kris Childress c1644cc4e7 Remove refs to `privatemasters` 2016-11-08 15:16:41 -08:00
Kris Childress cc2e920008 Fix for https://github.com/kubernetes/kops/pull/694#issuecomment-258308027 2016-11-08 15:16:41 -08:00
Kris Childress 712882f080 K8s API
- Fixing Kubernetes API forwarding in the ELB
 - Fixing DNS for kubectul
 - Fixing Suggestions: output for bastion
2016-11-08 15:16:41 -08:00
Kris Childress 78ecdb2165 Moar YAML cleanup and putting finishing touches on k8s debugging for tomorrow - Oh etcd... <3 2016-11-08 15:16:41 -08:00
Kris Childress 3f4bc39d52 Yaml Docs cleanup 2016-11-08 15:16:41 -08:00
Kris Childress 37f5bb7d57 Working networking commit!
- Stick bastion in ASG
 - ELBs for API and Bastion
2016-11-08 15:16:41 -08:00
Kris Childress 0857ed1732 Working Bastion with ELB - now time to start on the k8s API :) :) :) 2016-11-08 15:16:41 -08:00
Kris Childress 312621b0d0 Pushing up some last minute tweaks before asking for help and feedback from testing 2016-11-08 15:16:41 -08:00
Kris Childress e962f9c5fd Adding bastion support 2016-11-08 15:16:41 -08:00
Kris Childress cebdde3fb4 Woo! Time to start playing with private networks in AWS!! 2016-11-08 15:16:41 -08:00
Kris Childress 835e24f788 Working EIP and NGW CRUD for private networking..
Next step.. lets piece them all together
2016-11-08 15:16:41 -08:00
Kris Childress a3dd1257ce Working ElasticIP associations on subnet. Delete and Create! 2016-11-08 15:16:41 -08:00
Kris Childress c1e8dbe9d6 More work on the network and EIP things 2016-11-08 15:16:41 -08:00
Kris Childress 9bd9e30bdd Adding another large commit after a make codegen 2016-11-08 15:16:41 -08:00
Kris Childress a1c5c77b23 docs 2016-11-08 15:16:41 -08:00
Kris Childress a1ca6b7a5b More progress - getting out to Github so I can switch laptops... will be needing 8 cores today :D 2016-11-08 15:16:41 -08:00
Kris Childress 8f30225b32 Switching over branches 2016-11-08 15:16:41 -08:00
Kris Childress 8fba14b85b Small refactor - getting ready to start YAML 2016-11-08 15:16:41 -08:00
Kris Childress 000e847af2 Topology Initial Commit
- Refactor private networking -> topology
- Define new topology models (no changes yet)
- Docs
- Create cluster --topology and -t
- New functions for topology templating
2016-11-08 15:16:41 -08:00
Justin Santa Barbara e8816f0643 Remove security group rules that match our filter
We configure a filter so that we only remove rules on port 22 & 443

Fix #478
2016-10-20 00:10:18 -04:00
Justin Santa Barbara d780c8ee9b Merge pull request #424 from tazjin/ig-subnets
Support for multiple admin access CIDRs
2016-10-20 00:08:20 -04:00
Vincent Ambo c0dad70d1f Support multiple admin access CIDRs
This modifies the templates to appropriately create resources for
different access CIDRs specified in the cluster configuration.

On AWS this leads to the creation of multiple security group rules which
will not currently be cleaned up if a CIDR is removed.

This issue is tracked in kubernetes/kops#145

Changes:
* change AdminCIDR() to return slice of configured CIDRs
* aws: change templates to create security group rule per CIDR
* gce: set 'sourceRanges' for firewall rule to configured CIDRs
2016-10-16 12:27:24 +02:00
Justin Santa Barbara 2af1fde49d Reuse the route table when importing
Not only is this lower-impact, but it also avoid a bug because the
subnets were considered "shared", and thus we would not manage the
route-table any more.
2016-10-15 14:18:30 -04:00
Justin Santa Barbara 7c37b096e3 remove _master_dns tag; we rely on DNS 2016-10-11 00:29:18 -04:00
Justin Santa Barbara a529ffbb65 Configure dns-controller with ID of hosted zones
Fix #584
2016-10-06 13:12:27 -04:00
Justin Santa Barbara 204d1364ac Switch to image published under kope account 2016-10-01 17:30:52 -04:00
Justin Santa Barbara 146babbd27 Disable ingress DNS integration for 1.4.0
There are still some problems with the default nginx controller
2016-10-01 17:25:11 -04:00
Justin Santa Barbara 1a4558a736 Fix DNS deployment manifest 2016-10-01 15:26:10 -04:00
Justin Santa Barbara 655a61588e Switch all the final switches for release 1.4
Also apply the 1.4 schema changes.
2016-10-01 13:50:19 -04:00
Justin Santa Barbara 3ead9fe0ce Create addons for 1.4
(It isn't activated yet though)
2016-10-01 09:35:20 -04:00
Justin Santa Barbara 8839e67f0b Merge fixups 2016-09-24 11:46:34 -04:00
Justin Santa Barbara 41e2bee204 Merge pull request #495 from justinsb/setup_machine_id
Call /bin/systemd-machine-id-setup as part of init
2016-09-24 11:42:44 -04:00
Justin Santa Barbara d494d83436 Merge pull request #452 from yissachar/support-shared-subnets
Add support for shared subnets
2016-09-24 11:41:28 -04:00
Justin Santa Barbara d7639691e9 Call /bin/systemd-machine-id-setup as part of init
Just in case nobody else sets it!
2016-09-24 10:18:30 -04:00
Justin Santa Barbara 9356b5b215 Merge pull request #460 from justinsb/security_group_rule_removal
Support deletion of items
2016-09-20 11:42:42 -04:00
Justin Santa Barbara 352bc52a9f Honor minSize/maxSize for ASGs for master
Normally we expect the size to be 1, but it turns out there is an
exception - in the case when we want to suspend a cluster.  So honor the
values if the user sets them.

Thanks for spotting @sekka1

Fix #403
2016-09-17 23:17:18 -04:00
Justin Santa Barbara f8bbdb1467 Support deletion of items
We don't normally need to delete items, but we do need to purge old
security group rules.
2016-09-17 23:06:15 -04:00
Yissachar Radcliffe 5217bd432d Add support for shared subnets 2016-09-16 12:17:44 -04:00
Justin Santa Barbara 6d139d06d1 Support labels on k8s nodes and AWS instances
A lot of supporting work was needed, including improvements to the model
and model generation logic.
2016-09-13 12:47:16 -04:00
Justin Santa Barbara b9c20a7c0d Fix logic around `or nillable true` in text template
A false value is also treated as false, so the expression will always be
true
2016-09-09 11:35:49 -04:00
Justin Santa Barbara 8c1cbec9b6 Default AssociatePublicIP to true
If AssociatePublicIP is nil, treat that as true.

The full fix is likely to version InstanceGroups, but this is also
"defense in depth".
2016-09-09 10:12:26 -04:00
Justin Santa Barbara 9ee663764f Merge pull request #378 from justinsb/reapply_365
Reapply #365
2016-09-09 10:04:55 -04:00
Justin Santa Barbara ebf84d33d6 Merge pull request #273 from moleksyuk/master
Add no-public-ip option to instance groups
2016-09-08 11:45:43 -04:00
Justin Santa Barbara 62d5451b25 Initial (experimental) Ubuntu 16.04 support 2016-09-08 10:20:42 -04:00
Justin Santa Barbara d3ab070b0d Use go-bindata to embed our models
This allows us to have single-file deployment
2016-09-07 11:56:03 -04:00
Mykhailo Oleksiuk aa6693a6ed megre from upstream 2016-09-01 13:23:50 +03:00
Justin Santa Barbara 1b91f417e5 Build IAM policy in code
Easier to get right than relying on string manipulation, but we're still
doing the same policies, with the improvements as done by @weargoogles.
2016-08-27 21:18:23 -04:00
Justin Santa Barbara a3eda654db Revert "Revert "include change to node policy to cover #363""
This reverts commit ca1a52ff3e.
2016-08-27 17:38:01 -04:00
Justin Santa Barbara 4df50773c1 Revert "Revert "Restrict master access to state store bucket""
This reverts commit c11a370c9a.
2016-08-27 17:37:55 -04:00
Justin Santa Barbara c11a370c9a Revert "Restrict master access to state store bucket"
This reverts commit 369a6ea1db.
2016-08-27 16:31:53 -04:00
Justin Santa Barbara ca1a52ff3e Revert "include change to node policy to cover #363"
This reverts commit 969af97b60.
2016-08-27 16:31:38 -04:00
Pete Wildsmith 969af97b60 include change to node policy to cover #363 2016-08-24 17:19:54 +01:00
Pete Wildsmith 369a6ea1db Restrict master access to state store bucket
This change increases the specificity of the master's state store bucket contents permission to only the top-level folder named after the cluster.

Fixes #365
2016-08-24 17:03:10 +01:00
Justin Santa Barbara 7699dc8fd2 Merge pull request #294 from justinsb/use_ssh_key
SSH key improvements
2016-08-11 22:28:41 -04:00
Justin Santa Barbara a3cfec6c24 Support changing the SSH public key
This requires that we include the OpenSSH fingerprint in the AWS key
name.
2016-08-11 12:00:52 -04:00
Justin Santa Barbara 8fb4215e17 Run CI versions of k8s
CI versions are not pushed to gcr.io, so we need to preload the images
by downloading them and doing a docker load.
2016-08-11 01:32:42 -04:00
Mykhailo Oleksiuk fad3d3a4f4 move --no-associate-public-ip to instance group 2016-08-06 14:46:46 +03:00
Mykhailo Oleksiuk a860fdbdfd add parameter --no-associate-public-ip 2016-08-04 17:19:20 +03:00
Justin Santa Barbara 2b3f55563e Run the master on the pod network, unless IsolateMaster=true
The master is now registered as a Node.  It is marked as Unschedulable,
so normal pods will not run on it.  But Daemonsets will, and it is
surprising that they don't work unless hostNetwork=true.

The default is now what seems to be expected:
* we allocate the master a real CIDR on the pod network
* kube-proxy runs on the master, so it can talk to pods
* we run kubelet on the master with enable-debugging-handlers, so
  kubectl logs etc works

To get the old behaviour, edit the cluster spec and set
`isolateMasters: true`
2016-07-28 12:12:16 -04:00
Fotios Lindiakos be2fcca933 Remove trimming in AWS templates 2016-07-26 11:14:55 -04:00
Justin Santa Barbara 9e9855d1a4 Simpler upgrade procedure: reuse subnet
By reusing the subnet & security groups, we are able to skip the ELB
steps of the upgrade procedure.  The new cluster also has the same
identity as the old cluster for security groups, so we don't need to
reconfigure ELB etc.

Fixes #175
Fixes #174
2016-07-22 11:47:12 -04:00
Justin Santa Barbara 11d51b04a9 Adapt IAM policies when running in cn-north-1
Fix #27
2016-07-21 22:19:43 -04:00
Justin Santa Barbara 302f23463e Configuration of admin access to ports 22 and master-443
Fix #143
2016-07-14 10:33:26 -04:00
Justin Santa Barbara f771c2af4c Add support for spot instances
Fixes #58
2016-07-10 23:56:16 -04:00
Justin Santa Barbara 5b8b4d4da3 Detect & delete new ASG launch configs
We now output a ClusterName property into the launchconfig, even though
we don't technically need it.  But it allows us to more easily detect
the cluster, and it generally seems like a good idea.

Also rename to 'autoscaling-config' and clean up the cluster name
detection logic.

Fix #96
2016-07-09 22:07:24 -04:00
Justin Santa Barbara 126c508426 Fix model: numbers must be quoted 2016-07-09 01:41:04 -04:00
Justin Santa Barbara 13e514aeac Merge pull request #93 from justinsb/fix_24
Allow configurable RootDeviceSize & RootDeviceType
2016-07-09 01:25:20 -04:00
Justin Santa Barbara b42765816e Change node role tag to match master pattern
It's not currently used, and we hadn't updated it to match the better
pattern.

k8s.io/role=master can only be in one role
k8s.io/role/master=1 allows for multiple roles
2016-07-08 22:02:32 -04:00
Justin Santa Barbara 13b8e81bd6 Allow configurable RootDeviceSize & RootDeviceType
This allows for a larger EBS root volume (and we now default to 20GB,
just like kube-up did).

We remove the BlockDeviceMappings support because it wasn't used and
made things a lot more complicated.  We always map the ephemeral
devices.

Issue #24
2016-07-08 01:11:14 -04:00
Justin Santa Barbara 947a045667 Rename DNSDomain -> ClusterDNSDomain for clarity 2016-06-27 15:36:11 -04:00
Justin Santa Barbara 26d05341b4 Move options to common stage, so that it works with terraform generation 2016-06-27 15:21:31 -04:00
Justin Santa Barbara c36607644b Better shared VPC support: more validation 2016-06-27 15:00:51 -04:00
Justin Santa Barbara a0d8302255 Merge pull request #156 from slack/protokube-dns
upup/protokube: tell protokube to use --dns-zone-name
2016-06-27 00:41:11 -04:00
Justin Santa Barbara b6cf38c96e AllocateNodeCIDRs need no longer be "bubbled down"
We have it on the KCM config; just set it there
2016-06-27 00:32:19 -04:00
Justin Santa Barbara eeed4a3031 Rationalize API to something we want to support forever 2016-06-26 23:09:02 -04:00
Justin Santa Barbara ee325435e6 Rationalize properties to the minimal set 2016-06-26 09:45:05 -04:00
Jason Hansen 0d276591d5 upup/cloudup: use configured URL for nodeup location 2016-06-26 04:26:37 +00:00
Justin Santa Barbara ac8ca9ad06 Merge pull request #126 from justinsb/upup_use_vfs
upup: use vfs for secretstore/keystore
2016-06-23 10:26:42 -04:00
Justin Santa Barbara 93f634b428 upup: use vfs for secretstore/keystore
This is needed so that we can have encrypted storage and complex keys
(e.g. multiple CA certs).  Multiple CA certs are needed for an in-place
upgrade from kube-up v1.
2016-06-23 08:58:54 -04:00
Justin Santa Barbara fcc1f57c2d Updates for 1.3: Docker 1.11.2, 1.3 image 2016-06-23 08:58:23 -04:00
Justin Santa Barbara 0559ec1210 upup: Support for shared VPCs
A lot of work that had to happen here:

* Better reuse of config
* Ability to mark VPC & InternetGateway as shared
* Find models relative to the executable, to run from a dir-per-cluster

Fixes #95
2016-06-13 11:37:06 -04:00
Justin Santa Barbara b52877e2ce upup: separate node & master zone configuration; validate
We allow --zones & --master-zones to be specified separately now, but we
validate for common errors (using a region where you meant a zone,
duplicating a zone, spanning regions, entering an invalid AZ etc)
2016-06-11 21:06:31 -04:00
Justin Santa Barbara 52496ac73a upup: split launchconfiguration from ASG
It is much more logical this way, and mirrors the way GCE & terraform
work.
2016-06-10 11:36:17 -04:00
Justin Santa Barbara 6e203da852 upup: split model into two parts
This is probably a good idea anyway, but it also lets us side-step the
terraform no-dots-in-tags bug.
2016-06-09 23:14:36 -04:00
Justin Santa Barbara c826f46a60 upup: support for terraform on AWS
All seems good except for a bug with volume tagging
2016-06-08 12:19:15 -04:00
Justin Santa Barbara 42e32f7379 upup: include kope-routing, but only if _kope_routing is set 2016-06-08 12:18:04 -04:00
Justin Santa Barbara 1eaf0d36a8 upup: HA support
Specifying multiple zones will bring up an HA cluster.
2016-06-07 15:44:00 -04:00
Justin Santa Barbara 6cf5cd423e upup: apply IAM changes
We now apply changes to IAM policies, and print the diffs.
2016-06-07 15:17:59 -04:00
Justin Santa Barbara 71c2835007 upup: don't hard-code v1.2.2 in image names 2016-06-04 16:12:51 -04:00
Justin Santa Barbara a4408f76be upup: better secrets support
Start creating commands to manage secrets, and also stop implicitly
creating them.
2016-05-30 18:47:20 -04:00
Justin Santa Barbara 1c97a94d87 Rework keypair to fit our change model
We also remove another special-case context (pki), so that it is just
another object type.
2016-05-15 21:46:53 -04:00
Justin Santa Barbara caccb8953f UpUp: AWS support
Adds AWS support for both cloudup & nodeup.
Also cleaning up things found along the way!
2016-05-09 13:08:27 -04:00
Justin Santa Barbara d4c2cfaae7 Initial version of upup: cloudup & nodeup
* GCE support only
* Key and secret generation
* "Direct mode" makes API calls
* "Dry run mode" previews the changes
* Terraform output (though key generation not working for master ip)
* cloud-init output (though debian image does not ship with cloud-init)
2016-05-06 16:01:33 -04:00