kubernetes
/
kops
mirror of https://github.com/kubernetes/kops.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,172 Commits 31 Branches 256 Tags 544 MiB
Commit Graph

1453 Commits

Author SHA1 Message Date
Justin Santa Barbara 0a6260f7c4 Merge pull request #130 from justinsb/upup_vfs_context 
upup: Add VFS context object, centralize usage
 2016-06-23 10:30:09 -04:00
Justin Santa Barbara 3185a3fe5c Fix upgrade 2016-06-23 10:28:17 -04:00
Justin Santa Barbara 75af0d089d upup: Add VFS context object, centralize usage 
Replaces ad-hoc construction of VFS paths
 2016-06-23 10:27:32 -04:00
Justin Santa Barbara 506d0d555a Merge pull request #127 from justinsb/upup_upgrade_tooling 
upup: improved delete; create upup export and upup upgrade
 2016-06-23 10:27:32 -04:00
Justin Santa Barbara 90d7fb87ad upup: improved delete; create upup export and upup upgrade 2016-06-23 08:59:20 -04:00
Justin Santa Barbara 93f634b428 upup: use vfs for secretstore/keystore 
This is needed so that we can have encrypted storage and complex keys
(e.g. multiple CA certs).  Multiple CA certs are needed for an in-place
upgrade from kube-up v1.
 2016-06-23 08:58:54 -04:00
Justin Santa Barbara 75f386235a upup: rolling-update add mutex lock 2016-06-17 09:36:43 -04:00
Justin Santa Barbara 1e7159a923 upup: detach internet gateway if vpc is to be deleted 2016-06-17 09:36:27 -04:00
Justin Santa Barbara 3ede6c1f4a upup: uncompress gzipped data in delete cluster 2016-06-17 09:36:12 -04:00
Justin Santa Barbara 3040676a3b upup: enable subnet changes on ASG 
For kube-up upgrade
 2016-06-15 22:56:19 -04:00
Justin Santa Barbara 22bf484777 upup: Assign subnet CIDRs based on AZs 2016-06-15 14:03:19 -04:00
Justin Santa Barbara b147091e5c upup: fix dryrun when adding a subnet 
When adding a subnet in --dryrun, the ID might not be set during a Find
 2016-06-15 14:03:19 -04:00
Justin Santa Barbara a10c76a954 upup: prevent spurious comparison failure on VPC/IGW 2016-06-15 14:03:19 -04:00
Justin Santa Barbara eb23e22b3c upup: rolling-update should take --name, not --cluster-id 2016-06-13 17:37:30 -04:00
Justin Santa Barbara 14bae04e01 upup: turn down a few more logging messages 2016-06-13 17:37:30 -04:00
Justin Santa Barbara 0559ec1210 upup: Support for shared VPCs 
A lot of work that had to happen here:

* Better reuse of config
* Ability to mark VPC & InternetGateway as shared
* Find models relative to the executable, to run from a dir-per-cluster

Fixes #95
 2016-06-13 11:37:06 -04:00
Justin Santa Barbara 530b4c79db upup: experimental addon management 2016-06-13 11:35:00 -04:00
Justin Santa Barbara 55574ec3f5 upup: apply gofmt 2016-06-11 23:09:44 -04:00
Justin Santa Barbara 226f0284b1 upup: Fix log message 2016-06-11 23:09:44 -04:00
Justin Santa Barbara 72afa859a3 upup: delete cluster should take -name to be consistent 
Still not entirely sure on cluster-id vs name, but name is consistent
 2016-06-11 23:08:40 -04:00
Justin Santa Barbara 96247daf6d Merge pull request #101 from justinsb/upup_fix_delete_asg 
upup: fixes for ASG deletion
 2016-06-11 23:06:00 -04:00
Justin Santa Barbara f9ac7af7df Merge pull request #100 from justinsb/upup_zones 
upup: separate node & master zone configuration; validate
 2016-06-11 23:05:55 -04:00
Justin Santa Barbara ee056fa975 upup: Retry if race detected in FindOrCreateSecret 
We do a read / create-if-empty.  If the create fails because of a
concurrent creation, we retry the operation.

Fix #73
 2016-06-11 22:40:47 -04:00
Justin Santa Barbara 0263826163 upup: delete cluster should check status of ASG 
Before deleting the ASG, we need to check it exists.

Issue #99
 2016-06-11 21:33:56 -04:00
Justin Santa Barbara e63426ce65 upup: delete cluster should eventually give up 
If it isn't making progress, eventually we should bail out so that we
can use this in scripts.  However, we don't need to be too aggressive,
because it is not good to leak resources, and a user will likely
Control-C us pretty fast when they see they're in a loop.

Issue #99
 2016-06-11 21:33:56 -04:00
Justin Santa Barbara b52877e2ce upup: separate node & master zone configuration; validate 
We allow --zones & --master-zones to be specified separately now, but we
validate for common errors (using a region where you meant a zone,
duplicating a zone, spanning regions, entering an invalid AZ etc)
 2016-06-11 21:06:31 -04:00
Justin Santa Barbara d489fccd28 upup: make NodeConfig be JSON-clean 
We use JSON-merging in our option parsing, so we need it to only
populate values that are actually set.
 2016-06-10 13:37:48 -04:00
Justin Santa Barbara f3ebc088f1 upup: make LaunchConfiguration compatible with terraform 
This way we can output a LaunchConfiguration prefix into terrform that
we can then read later, so that we can create with terraform and then
transfer to another mode of operation if desired.
 2016-06-10 13:36:36 -04:00
Justin Santa Barbara c54ef2773d upup: support Base64 and raw-string JSON encoding of certs/keys 
For compatability reasons, we write the certificate & keys as base64
encoded strings.  I don't think we have to any more, but we have to be
able to parse it.
 2016-06-10 13:35:40 -04:00
Justin Santa Barbara 7543d1c19d upup: apply gofmt 
Missed a few gofmt places
 2016-06-10 11:41:10 -04:00
Justin Santa Barbara 1cbcb82fe9 upup: use JSON as fallback in --dryrun printing 
Rather than giving up with an error, we can simply use JSON format.
 2016-06-10 11:38:21 -04:00
Justin Santa Barbara 9b32edfb81 upup: add experimental cluster rolling-update command 
Little more than a hack right now, but a good place to start.
 2016-06-10 11:36:57 -04:00
Justin Santa Barbara 52496ac73a upup: split launchconfiguration from ASG 
It is much more logical this way, and mirrors the way GCE & terraform
work.
 2016-06-10 11:36:17 -04:00
Justin Santa Barbara eeabcc0cb1 upup: fix options processing 
We want to make sure that any values we set are treated both as defaults
and as overrides (i.e. the options shouldn't be able to override a value
the user has explicitly set)
 2016-06-10 11:35:31 -04:00
Justin Santa Barbara 6e7c4209de upup: Simple tag fixes to avoid comparison failures 
We populate the actual expected tags in-place now
 2016-06-10 11:33:33 -04:00
Justin Santa Barbara 2c5e0c0164 upup: populate Name, to support --dryrun 
Some of these types were coming up as spurious changes.
 2016-06-10 11:33:29 -04:00
Justin Santa Barbara e10f09a81f Merge pull request #83 from justinsb/upup_node_sizes 
upup: node size on CLI and more instance type families
 2016-06-10 11:32:49 -04:00
Justin Santa Barbara 2bacc2035c upup: define m3, m4, c4 families 
This allows us to set up their ephemeral volumes
 2016-06-10 11:31:20 -04:00
Justin Santa Barbara 7c478c794f upup: harmonize model with official config types 2016-06-10 01:28:44 -04:00
Justin Santa Barbara 6e203da852 upup: split model into two parts 
This is probably a good idea anyway, but it also lets us side-step the
terraform no-dots-in-tags bug.
 2016-06-09 23:14:36 -04:00
Justin Santa Barbara fa860ca730 Merge pull request #75 from justinsb/upup_vfs 
upup: Add VFS for storing state in S3 or locally
 2016-06-09 23:12:54 -04:00
Justin Santa Barbara 338f158d22 upup: Add VFS for storing state in S3 or locally 
This also fixes a few concurrency issues, because we're writing in one
place now.
 2016-06-09 23:08:10 -04:00
Justin Santa Barbara 93284d5d13 upup: reformat terraform output 
Using string manipulation, sadly, but I believe it's safe as we don't
use heredocs
 2016-06-09 23:07:12 -04:00
Justin Santa Barbara c826f46a60 upup: support for terraform on AWS 
All seems good except for a bug with volume tagging
 2016-06-08 12:19:15 -04:00
Justin Santa Barbara 70d7c1b92e upup: avoid warning when building filters with nil name 2016-06-08 12:18:07 -04:00
Justin Santa Barbara 42e32f7379 upup: include kope-routing, but only if _kope_routing is set 2016-06-08 12:18:04 -04:00
Justin Santa Barbara 59124fd6fb upup: include count of tasks as we are executing them 2016-06-08 12:16:19 -04:00
Justin Santa Barbara 4d331e2881 upup: Reattempt up to 3 times when no progress made 
Now that we're running in parallel, sometimes AWS eventual consistency
causes us problems.  We now retry up to 3 times, sleeping 10 seconds in
between each run even when we aren't making progress.
 2016-06-08 12:16:19 -04:00
Justin Santa Barbara 44507e64cf upup: remove some bad logging 
Was left over from move to simpler unmarshal model
 2016-06-08 12:16:19 -04:00
Justin Santa Barbara 1eaf0d36a8 upup: HA support 
Specifying multiple zones will bring up an HA cluster.
 2016-06-07 15:44:00 -04:00
Justin Santa Barbara 6ea399fc79 upup: Add back in CompareWithID functions 
Removed when we pulled this functionality out of the go generator, but
then the manual code not re-added.

Also a few other small tweaks to Tasks.
 2016-06-07 15:37:07 -04:00
Justin Santa Barbara 4c0f54f95d upup: run create & delete in parallel 
It makes logging harder, but we want to start testing it!
 2016-06-07 15:23:07 -04:00
Justin Santa Barbara 6cf5cd423e upup: apply IAM changes 
We now apply changes to IAM policies, and print the diffs.
 2016-06-07 15:17:59 -04:00
Justin Santa Barbara c95656177f cloudup: Add support for ELB health checks 2016-06-07 09:01:35 -04:00
Justin Santa Barbara 062d2ece96 Merge pull request #54 from justinsb/upup_protokube 
Use protokube with upup
 2016-06-07 08:56:23 -04:00
Justin Santa Barbara 274696c0c5 Merge pull request #58 from justinsb/upup_smarter_delete 
upup: better cluster delete
 2016-06-07 08:47:21 -04:00
Justin Santa Barbara cf840ef21a upup: better cluster delete 2016-06-07 08:46:28 -04:00
Justin Santa Barbara e18833fcb7 Merge pull request #57 from justinsb/upup_resource_tagging 
cloudup: better retrying around resource tagging
 2016-06-07 08:43:21 -04:00
Justin Santa Barbara 9d6847ff49 cloudup: better retrying around resource tagging 2016-06-04 16:14:00 -04:00
Justin Santa Barbara 82f373cba0 nodeup: make sure tasks implement HasDependencies 2016-06-04 16:13:25 -04:00
Justin Santa Barbara 540c46fe4e nodeup: pass --master argument to protokube 2016-06-04 16:11:11 -04:00
Justin Santa Barbara c774777160 Merge pull request #50 from justinsb/upup_keys 
upup: better secrets support
 2016-06-04 15:57:08 -04:00
Justin Santa Barbara 4a85ef64fb Merge pull request #48 from justinsb/upup_elb 
upup: simple ELB support
 2016-06-04 15:56:57 -04:00
Justin Santa Barbara b9597c53ec Merge pull request #47 from justinsb/upup_delete_cluster 
upup: simple command to delete (AWS) clusters
 2016-06-04 15:56:51 -04:00
Justin Santa Barbara 5f4c03cd11 Merge pull request #46 from justinsb/upup_route53 
upup: Simple route53 (DNS) support
 2016-06-04 15:56:48 -04:00
Justin Santa Barbara a4408f76be upup: better secrets support 
Start creating commands to manage secrets, and also stop implicitly
creating them.
 2016-05-30 18:47:20 -04:00
Justin Santa Barbara 095bf95fc9 upup: simple command to delete (AWS) clusters 
A relatively dumb retry strategy to work around dependencies, but it
works and it is difficult to do _much_ better.
 2016-05-30 17:57:40 -04:00
Justin Santa Barbara 3a660fe8fa upup: simple ELB support 
Add support for ELB, though this doesn't wire it up yet
 2016-05-30 17:53:05 -04:00
Justin Santa Barbara 3389323392 upup: Simple route53 (DNS) support 2016-05-30 17:50:55 -04:00
Justin Santa Barbara 3aeaf73788 upup: move string-slice comparison functions to utils 2016-05-30 17:46:36 -04:00
Justin Santa Barbara 1aeea67510 upup: replace parse code with standard json unmarshaller 
Instead of reimplementing the unmarshal code, we implement a trick: we
implement an alternative JSON representation of an object: a string.

We unmarshal as normal, and then we reconcile these pointer values to
the primary values, by walking the unmarshalled tree.
 2016-05-30 17:46:36 -04:00
Justin Santa Barbara c72593fcf7 Use generators in tasks 2016-05-30 17:46:36 -04:00
Justin Santa Barbara 16559c9010 upup: Executor that performs some retry logic 
If there is an error performing a task, we will reattempt it as long as
forward progress is still being made (i.e. at least one other task
completed successfully)

This makes everything more reliable (though we should still fix these
problems), but it also lays the groundwork for parallel execution.
 2016-05-22 21:02:53 +01:00
Justin Santa Barbara 505ef056d9 Add upup command, with a subcommand to generate kubecfg 2016-05-17 12:54:13 -04:00
Justin Santa Barbara bbc460f5a7 Refactor Certificate and PrivateKey to implement io.WriterTo 2016-05-17 12:54:13 -04:00
Mike Danese 19bb767767 Merge pull request #28 from justinsb/upup_aws_retry_tags_on_eventual_consistency_error 
upup: tolerate aws eventual-consistency errors on tags
 2016-05-17 09:34:53 -07:00
Mike Danese 01590a9704 Merge pull request #27 from justinsb/upup_aws_image_aliases 
upup: Map matching image ids to source name
 2016-05-17 09:34:42 -07:00
Mike Danese 026bb16e49 Merge pull request #26 from justinsb/upup_aws_instance_userdata 
upup: Fetch instance userdata on AWS
 2016-05-17 09:34:30 -07:00
Mike Danese 7466c48a75 Merge pull request #25 from justinsb/upup_dryrun_print_interface 
upup: fix dryrun spurious printing of interface values
 2016-05-17 09:34:17 -07:00
Mike Danese d5928df6a4 Merge pull request #24 from justinsb/upup_aws_iamrole_policies 
upup: Perform JSON comparison on IAMRole PolicyDocuments
 2016-05-17 09:34:03 -07:00
Mike Danese cf98b75d39 Merge pull request #23 from justinsb/upup_aws_instance_tags 
upup: Discover tags on AWS instances
 2016-05-17 09:33:50 -07:00
Mike Danese 03ae922b01 Merge pull request #22 from justinsb/upup_aws_keypairs 
upup: Fix fingerprint calculation in AWS keypair
 2016-05-17 09:33:38 -07:00
Mike Danese a4344e1564 Merge pull request #19 from justinsb/upup_keypair 
upup: better keypair support
 2016-05-17 09:33:24 -07:00
Mike Danese d6026a431e Merge pull request #18 from justinsb/upup_reflection_redux 
upup: Make reflective walk more logical, use for dryrun change printing
 2016-05-17 09:33:16 -07:00
Mike Danese b48b503a7c Merge pull request #17 from justinsb/upup_gce_metadata 
upup: Discover metadata on GCE instance
 2016-05-17 09:33:11 -07:00
Mike Danese 5aa64c72a6 Merge pull request #16 from justinsb/upup_image_url 
upup: Map GCE image-url to string consistently
 2016-05-17 09:33:02 -07:00
Justin Santa Barbara c68aeec94d upup: tolerate aws eventual-consistency errors on tags 
AWS will sometimes return an error like "resource not found" when a
DescribeTags or CreateTags call immediately follows creation of the
resource.  Introduce a retry-loop when we get an error that is of the
appropriate type.
 2016-05-16 12:08:28 -04:00
Justin Santa Barbara 14ac983241 upup: Map matching image ids to source name 
This avoids spurious changes, and also is more intuitive for the user -
whatever name the user gave it, if it resolves to the same image, that
is the name we will use.
 2016-05-16 11:20:24 -04:00
Justin Santa Barbara a00e974a95 upup: Fetch instance userdata on AWS 
We simply weren't doing this until now
 2016-05-16 11:12:13 -04:00
Justin Santa Barbara e90f5c78e1 upup: fix dryrun spurious printing of interface values 
We missed the case where the interface changed (this only shows up if
other fields change also)
 2016-05-16 10:58:44 -04:00
Justin Santa Barbara ba77aea0ea upup: Perform JSON comparison on IAMRole PolicyDocuments 
AWS reformats them (inserting lots of whitespace), making a string
comparison incorrect.  Instead we parse to JSON and do a
reflect.DeepEqual check; if they are the same then we pretend the actual
value was the expected value.
 2016-05-15 23:49:01 -04:00
Justin Santa Barbara e6ce439fcd upup: Discover tags on AWS instances 
We simply weren't mapping them previously.
 2016-05-15 23:32:49 -04:00
Justin Santa Barbara 8752db39aa upup: Fix fingerprint calculation in AWS keypair 
Both fix the calculation itself to match AWS's weird fingerprint
algorithm, and also fix the comparison logic by which we infer that if
the fingerprint matches, that the public key matches also.
 2016-05-15 23:28:19 -04:00
Justin Santa Barbara 1c97a94d87 Rework keypair to fit our change model 
We also remove another special-case context (pki), so that it is just
another object type.
 2016-05-15 21:46:53 -04:00
Justin Santa Barbara c4742045ab Support changing GCE instance metadata 
This works, but is only a part-way there because:

1. We need to trigger cloudup
2. Our manifests don't pick up the change (e.g. kube-apiserver)
 2016-05-15 17:46:34 -04:00
Justin Santa Barbara ac220d2ba1 Allow Render methods to take fi.Context 
We call the Render methods on Tasks by reflection, and some of them
don't care about the Target, but do care about the Context (e.g. the PKI
tasks, which only care about the CAStore)
 2016-05-15 17:43:04 -04:00
Justin Santa Barbara cb7e20be3b Create HasAddress interface 
AWS ElasticIPs and GCE Addresses implement HasAddress.  This will be
used by dependent tasks to extract the IP address at runtime.
 2016-05-15 17:42:59 -04:00
Justin Santa Barbara 02d980c681 Make reflective walk more logical, use for dryrun change printing 
Remove a bunch of inconsistencies so that the reflective walk is not
suprising, and also rename it to ReflectRecursive.

Then use this for dry-run change printing.
 2016-05-15 17:41:19 -04:00
Justin Santa Barbara 0976ee1a73 Discover metadata on GCE instance 
We simply weren't mapping it before, which was causing spurious changes.
 2016-05-15 17:40:32 -04:00
Justin Santa Barbara 5a61c35052 Map GCE image-url to string consistently 
We need to reverse our image shortening consistently with how we resolve
the image, so that --dryrun does not report spurious changes.
 2016-05-15 17:39:26 -04:00
Justin Santa Barbara 370727aa44 Support for persistent disks, users & symlinks 
These are required to mount the master PD.  We use the kubernetes
safe-format-and-mount implementation.
 2016-05-13 13:29:56 -04:00
Justin Santa Barbara caccb8953f UpUp: AWS support 
Adds AWS support for both cloudup & nodeup.
Also cleaning up things found along the way!
 2016-05-09 13:08:27 -04:00
Justin Santa Barbara d4c2cfaae7 Initial version of upup: cloudup & nodeup 
* GCE support only
* Key and secret generation
* "Direct mode" makes API calls
* "Dry run mode" previews the changes
* Terraform output (though key generation not working for master ip)
* cloud-init output (though debian image does not ship with cloud-init)
 2016-05-06 16:01:33 -04:00