Commit Graph

227 Commits

Author SHA1 Message Date
Justin Santa Barbara 2a34e6d00e Output variables from terraform, for reuse in a module
Issue #1026
2017-02-17 11:05:06 -05:00
Justin Santa Barbara fec89f0f33 Skip DNS precreate on CF; a few test fixes 2017-02-17 11:04:39 -05:00
Justin Santa Barbara 7d68f744f6 Initial support for cloudformation output 2017-02-17 10:59:16 -05:00
Justin Santa Barbara 55b6d86454 Move more options to code
User reports of kubelet flags not being passed; moved more to code.

Also found & fixed the likely root-cause issue: we have two copies of
the cluster spec and were not being precise about which one we wanted to
use at all times.
2017-02-15 13:11:12 -05:00
Justin Santa Barbara 2d37ab1ca5 Recommend a k8s version based on each kops version
So the flow is that we recommend (or strongly recommend) a new kops
version when one is required for a new version, and then the new kops
version will recommend (or strongly recommend) a new k8s version.

We don't have a notion of multiple recommended k8s versions per kops
version - that is what channels are for.

Users are always free to disregard updates, even "required" ones by
setting a flag.
2017-02-06 01:06:03 -05:00
Justin Santa Barbara 379e0ca098 Channel manifest should be version aware
* We can target AMIs to kubernetes versions
* We can recommend / force a kops upgrade
* We can recommend / force a kubernetes upgrade
2017-01-25 23:13:48 -05:00
Justin Santa Barbara 9e015285f8 validation: Validate we specify ids for all subnets
Move our validation to the apimachinery style.  And then add a
validation that we specify IDs either for all subnets or no subnets.
2017-01-24 12:38:52 -05:00
Zach Loafman 7079729f26 Revert "Revert "Load protokube from http/https"" 2017-01-04 09:32:24 -08:00
Zach Loafman d69fd6989c Revert "Load protokube from http/https" 2017-01-04 07:54:20 -08:00
Justin Santa Barbara 66d5d55195 Fixes per code review 2016-12-31 16:03:23 -05:00
Justin Santa Barbara 5d6d0f5f5d Load protokube from http/https
This helps us treat protokube as being paired with nodeup, and is a step
towards registry-less deployments (and isolated deployments) along with
moving away from our deprecated gcr.io usage.
2016-12-31 15:21:20 -05:00
Justin Santa Barbara aa6c214a12 Fixes per code-review 2016-12-28 14:05:00 -05:00
Justin Santa Barbara d449f40a37 Pre-create DNS records with placeholder values
Fixes #928
2016-12-28 13:33:23 -05:00
Justin Santa Barbara ef14a1d172 Private DNS initial implementation - via feature flag 2016-12-26 14:03:31 -05:00
Justin Santa Barbara fed68310fa Schema v1alpha2
* Zones are now subnets
* Utility subnet is no longer part of Zone
* Bastion InstanceGroup type added instead
* Etcd clusters defined in terms of InstanceGroups, not zones
* AdminAccess split into SSHAccess & APIAccess
* Dropped unused Multizone flag
2016-12-18 21:56:57 -05:00
Justin Santa Barbara 613b7fea61 Map ELB attributes to terraform
Requires moving them under the LoadBalancer awstask, sadly
2016-12-18 21:55:40 -05:00
Justin Santa Barbara 0be724b696 Simplify terraform ELB tasks 2016-12-18 21:55:40 -05:00
Chris Love 1516b3b822 Merge pull request #1165 from robertojrojas/issue_1120
fixes issue #1120 - User can specify CNI_VERSION_URL
2016-12-15 15:31:11 -07:00
Roberto J Rojas 99c42497c3 fixes #1120 - PR #1165 - changes as per review 2016-12-15 10:36:41 -05:00
Justin Santa Barbara c01c2af656 Mark ObjectMeta as a named field
This will work around some apimachinery bugs
(https://github.com/kubernetes/client-go/issues/8)
2016-12-14 22:26:57 -05:00
Roberto J Rojas 1c5e668ffe fixes issue #1120 - User can specify CNI_VERSION_URL 2016-12-14 20:48:27 -05:00
Justin Santa Barbara cc7419adeb Bump version of nodeup that we depend on 2016-12-13 01:44:37 -05:00
Kris Childress 3f0bd708d1 Merge pull request #1114 from icereval/feature/terraform-private-topology
WIP: Fill in RenderTerraform methods for private topology
2016-12-12 10:23:53 -07:00
Justin Santa Barbara afd0c25abe First model -> tf test 2016-12-11 17:11:10 -05:00
icereval bf62eb7019 fill in RenderTerraform methods for private topology 2016-12-10 17:29:46 -05:00
Justin Santa Barbara dbab97e508 Use versioned marshaling where we can
In particular during a kops edit
2016-12-08 22:17:26 -05:00
alok87 a413ea5ac3 Merge remote-tracking branch 'kopsrepo/master' into bastion_improvements
* kopsrepo/master:
  gcs-upload: Use a no-clobber copy instead
  gcs-upload: Fix cache-control on other files as well
  changes from code review
  doc updates
  unit tests with fakes
  it is working in alpha
  working on the start of validate
  Starting work on node lookup and validation
  starting porting node code
  Fix retries for AutoScalingGroup pending delete
  Apply gofmt to pkg directory
  Avoid tests hitting kubernetes stable.txt HTTP file
  Fix printing of max size on instance group
  Disable kubelet from starting until after volume mounts
  Fix Cluster parsing error message
  bumping stable channel to k8s 1.4.6
  support more zones(cn-north-1a/b) for cloud provider guess
2016-11-30 07:58:41 +05:30
alok87 f0b80503c3 Configure LoadBalancer Attributes 2016-11-30 07:35:45 +05:30
Zach Loafman e60c36068f Fix retries for AutoScalingGroup pending delete
This:
- reworks how retries are handled in fi/executor.go to a time-based scheme
- changes the single-task limit to 10m (from about 30s of no-progress)
- eliminates the inner IAM propagation retry for LaunchConfigurations,
because the task itself will just be redriven for a while. This also
eliminates any long-pole delay caused by this error (since task Run()
should be 'fast').
2016-11-28 17:04:02 -08:00
alok87 edf22f3797 Bastion DNS as an option and not by default 2016-11-23 12:40:45 +05:30
Justin Santa Barbara 6f20979790 Adapt unit tests to sets.String; convert node tags also 2016-11-19 16:23:51 -05:00
Justin Santa Barbara ee44353cde Add support for kopeio networking 2016-11-16 14:20:23 -05:00
Justin Santa Barbara f709f909e4 Options through code 2016-11-14 12:26:42 -05:00
Justin Santa Barbara ae84beddb7 Use more api machinery when writing 2016-11-13 22:11:36 -05:00
Justin Santa Barbara 933506bdbb Apply gofmt 2016-11-13 21:06:55 -05:00
Kris Childress 835e24f788 Working EIP and NGW CRUD for private networking..
Next step.. lets piece them all together
2016-11-08 15:16:41 -08:00
Kris Childress 9bd9e30bdd Adding another large commit after a make codegen 2016-11-08 15:16:41 -08:00
Kris Childress a1c5c77b23 docs 2016-11-08 15:16:41 -08:00
Kris Childress 000e847af2 Topology Initial Commit
- Refactor private networking -> topology
- Define new topology models (no changes yet)
- Docs
- Create cluster --topology and -t
- New functions for topology templating
2016-11-08 15:16:41 -08:00
Justin Santa Barbara 205a027b8a CNI: update version to latest
This isn't ideal, because it isn't versioned, but there is an important
bugfix - otherwise pods are allocated a .255 IP, which is reserved for
broadcast.

Issue #724
2016-10-26 11:27:20 -04:00
Justin Santa Barbara 621dec765d Check NS records as part of kops apply
Issue #585
2016-10-19 00:16:38 -04:00
Chris Love 6ceeb25d10 Merge pull request #654 from justinsb/govet_fixes
govet fixes; add govet makefile target
2016-10-15 21:00:36 -06:00
chrislovecnm 8fa2aac99f fixing more headers 2016-10-15 19:20:56 -06:00
Justin Santa Barbara 3f54852497 govet fixes; add govet makefile target 2016-10-15 13:25:31 -04:00
Justin Santa Barbara 5ac44018d4 Move API to pkg/apis/kops
This is a breaking change for people using the API (sorry), but is
hopefully a simple search and replace:

"k8s.io/kops/upup/pkg/api"
 -> api "k8s.io/kops/pkg/apis/kops"

"k8s.io/kops/upup/pkg/api/registry"
 -> "k8s.io/kops/pkg/apis/kops/registry"

This is the "correct" place for it in the k8s API infrastructure - we
are working towards a versioned API here.
2016-10-11 08:52:54 -04:00
Justin Santa Barbara 6eeb7bf596 Prepare for 1.4.1: bump nodeup; add docs 2016-10-10 10:55:48 -04:00
Justin Santa Barbara d965f14e4b Add apply to example 2016-10-07 02:27:41 -04:00
Justin Santa Barbara afa8682dac Default ApplyClusterCmd to all the instance groups
Simpler, and ensures "one code path"
2016-10-07 01:13:26 -04:00
Justin Santa Barbara f7fa324858 Initial k8s-style kops API
We try to emulate the k8s Clientset approach
2016-10-07 01:13:02 -04:00
Justin Santa Barbara 655a61588e Switch all the final switches for release 1.4
Also apply the 1.4 schema changes.
2016-10-01 13:50:19 -04:00
Justin Santa Barbara 88ec3db1bf ManagedFile support
A managed file is templated kops-side, but then stored in the S3 bucket
(aka state store)

This will be used to pass the channel containing the core addons.
2016-10-01 10:21:45 -04:00
Justin Santa Barbara 647618b755 Store channel in cluster, use it to determine version 2016-10-01 00:04:09 -04:00
Justin Santa Barbara ef49855a80 Less verbose logging 2016-09-30 23:42:24 -04:00
Justin Santa Barbara 3b1e467ee5 Create version tags 2016-09-30 00:07:01 -04:00
Justin Santa Barbara 5f8d68ec85 Refactor shared packages into util directory 2016-09-25 18:27:09 -04:00
Justin Santa Barbara bb01cf6ddd Make AWSCloud an interface, mock it out in tests
Beginnings of a mock for the AWSCloud, so that hopefully we aren't
calling out to AWS at all in the tests.  We will likely start mocking
the actual EC2 APIs in future, but this seems a good starting point.

Fix #425
2016-09-24 15:52:52 -04:00
Justin Santa Barbara 8839e67f0b Merge fixups 2016-09-24 11:46:34 -04:00
Justin Santa Barbara addaf5e4b4 Don't add terraform to --out
Fix #486
2016-09-24 11:25:39 -04:00
Justin Santa Barbara 9356b5b215 Merge pull request #460 from justinsb/security_group_rule_removal
Support deletion of items
2016-09-20 11:42:42 -04:00
Justin Santa Barbara d440fb0763 When validating DNS names, ignoring trailing dot 2016-09-19 10:20:57 -04:00
Justin Santa Barbara f8bbdb1467 Support deletion of items
We don't normally need to delete items, but we do need to purge old
security group rules.
2016-09-17 23:06:15 -04:00
Justin Santa Barbara 6d139d06d1 Support labels on k8s nodes and AWS instances
A lot of supporting work was needed, including improvements to the model
and model generation logic.
2016-09-13 12:47:16 -04:00
Justin Santa Barbara 3942ffa8fb Deeper DNS validation: don't create hosted zone
Avoids a problem with TF output, but also lays the groundwork for deeper
DNS validation, which is probably the #1 source of problems right now.
2016-09-13 12:45:12 -04:00
Justin Santa Barbara d3ab070b0d Use go-bindata to embed our models
This allows us to have single-file deployment
2016-09-07 11:56:03 -04:00
Justin Santa Barbara dc63e307df Allow pluggable networking: classic, kubenet, external 2016-08-22 12:32:15 -04:00
Justin Santa Barbara 9829eb8579 Make it easy to run a custom build
We introduce two env vars:

* NODEUP_URL url to nodeup that we want to run
* PROTOKUBE_IMAGE specifies the protokube Docker image to run
2016-08-19 00:13:56 -04:00
Justin Santa Barbara 5fb39913cc If no changes are needed in an update, don't print a confusing message
Fix #208
2016-08-14 23:40:18 -04:00
Justin Santa Barbara 7699dc8fd2 Merge pull request #294 from justinsb/use_ssh_key
SSH key improvements
2016-08-11 22:28:41 -04:00
Justin Santa Barbara a3cfec6c24 Support changing the SSH public key
This requires that we include the OpenSSH fingerprint in the AWS key
name.
2016-08-11 12:00:52 -04:00
Justin Santa Barbara 219c6e4308 Use the SSH key from the secret store
It is an error if there is not exactly one, but we have commands to
manage it now.
2016-08-11 12:00:46 -04:00
Justin Santa Barbara 8fb4215e17 Run CI versions of k8s
CI versions are not pushed to gcr.io, so we need to preload the images
by downloading them and doing a docker load.
2016-08-11 01:32:42 -04:00
Justin Santa Barbara 39a3d71def Revert "Import SSH public key into the keystore"
This reverts commit bd3ab166b7.
2016-07-29 13:06:37 -04:00
Justin Santa Barbara 3e3ca5a2b8 Merge pull request #222 from justinsb/fix_204
Run the master on the pod network, unless IsolateMaster=true
2016-07-29 00:51:50 -04:00
Justin Santa Barbara bd3ab166b7 Import SSH public key into the keystore
This means it only needs to be specified during `kops create`.  We
remove the option from `kops update` for consistency.

This will shortly be manageable using the secrets functionality.

Fix #221
2016-07-29 00:45:37 -04:00
Justin Santa Barbara 2b3f55563e Run the master on the pod network, unless IsolateMaster=true
The master is now registered as a Node.  It is marked as Unschedulable,
so normal pods will not run on it.  But Daemonsets will, and it is
surprising that they don't work unless hostNetwork=true.

The default is now what seems to be expected:
* we allocate the master a real CIDR on the pod network
* kube-proxy runs on the master, so it can talk to pods
* we run kubelet on the master with enable-debugging-handlers, so
  kubectl logs etc works

To get the old behaviour, edit the cluster spec and set
`isolateMasters: true`
2016-07-28 12:12:16 -04:00
Justin Santa Barbara 11d51b04a9 Adapt IAM policies when running in cn-north-1
Fix #27
2016-07-21 22:19:43 -04:00
Justin Santa Barbara 2fa3bcc952 UX: Split create command into `create` and `update`
We separate out the `create cluster` operation from the `update cluster`
operation.  Now create cluster only creates the spec (unless you pass
--yes), and is only for new clusters.

`update cluster` works on new or existing clusters, and should be called
to apply changes.

`update` is not the best name, because it means something different in
kubectl, but I think it's a good start.
2016-07-21 11:54:09 -04:00