Commit Graph

769 Commits

Author SHA1 Message Date
Chris Love 18d1dfde8b Merge pull request #1118 from justinsb/codegen_fixes
Improvements to codegen: header-check & golint
2016-12-11 18:22:14 -07:00
Justin Santa Barbara 4c02a28866 MockAWSCloud: Use new route53 test interface 2016-12-11 17:11:10 -05:00
Justin Santa Barbara afd0c25abe First model -> tf test 2016-12-11 17:11:10 -05:00
Chris Love dc63402649 Merge pull request #1125 from DualSpark/fix-log-about-issue-241
Removing workaround for #241
2016-12-11 14:04:01 -07:00
Kris Childress e9e5d82a91 Merge pull request #1121 from justinsb/fix_log
Improve log message when skipping log
2016-12-11 12:55:43 -07:00
icereval 0331f70f11 internal aws elb 2016-12-11 14:51:33 -05:00
Kris Childress 195007b7da Removing workaround for #241 2016-12-11 12:40:35 -07:00
Kris Childress d41b1f02d1 Merge pull request #1062 from icereval/feature/terraform-path-module
Add path.module to Terraform output
2016-12-11 12:20:14 -07:00
Kris Childress d195005e31 Merge pull request #961 from yancl/fix-aws-region-hardcoded
fix the aws region that hardcoded to "us-east-1"
2016-12-11 12:03:05 -07:00
Justin Santa Barbara f5441922d4 Improve log message when skipping log 2016-12-11 13:15:39 -05:00
Chris Love 704aaaeff2 Merge pull request #1089 from justinsb/kops_edit_cluster
Use versioned marshaling where we can
2016-12-11 10:09:53 -07:00
Justin Santa Barbara 14b396225b Updated codegen output 2016-12-11 11:31:48 -05:00
Justin Santa Barbara cc80e75e90 codegen template improvments
* Include the header so we generate code that passes the header-check
* Some tweaks to make golint happier
2016-12-11 11:31:40 -05:00
icereval ead86d03d1 terraform import, thanks vsc 2016-12-10 20:55:00 -05:00
icereval bf62eb7019 fill in RenderTerraform methods for private topology 2016-12-10 17:29:46 -05:00
Chris Love 9cf2826387 Merge pull request #1105 from justinsb/dryrun_use_name
dryrun output: include name instead of id:nil
2016-12-09 16:15:28 -07:00
Chris Love 9c87763f6f Merge pull request #1101 from justinsb/sorted_task_list
diagnostics: sort known tasks before printing
2016-12-09 16:14:02 -07:00
Chris Love 6424010210 Merge pull request #1096 from justinsb/test_elastic_ip
First test of a fi.Task
2016-12-08 20:26:26 -07:00
Justin Santa Barbara dbab97e508 Use versioned marshaling where we can
In particular during a kops edit
2016-12-08 22:17:26 -05:00
Justin Santa Barbara de15b6b562 Recognize ca-central-1 region 2016-12-08 16:30:27 -05:00
Justin Santa Barbara f4092d0113 First test of a fi.Task 2016-12-08 13:17:38 -05:00
Justin Santa Barbara 104332108f dryrun output: include name instead of id:nil
When we're creating objects, we tend not to have an id.  Rather than
drop them entirely though, we can print their name (where we have one)
2016-12-08 12:55:15 -05:00
Justin Santa Barbara dd76c0e233 DryRun output: Don't print task type twice
We were printing the task type, and then the task key, which includes
the task type.  So we were printing `TaskName TaskName/TaskKey`

Strip the task type out of the task key: `TaskName/TaskKey`
2016-12-08 10:33:58 -05:00
Justin Santa Barbara a66850e15a diagnostics: sort known tasks before printing 2016-12-08 10:26:50 -05:00
Chris Love 37b3499a20 Merge pull request #1092 from justinsb/spurious_privates
Cleanups to make kops update apply cleanly for private topologies
2016-12-07 22:31:17 -07:00
Justin Santa Barbara 1589e66c77 Fix error handling around listing ELBs 2016-12-07 18:42:39 -05:00
Chris Love a53fa418af Merge pull request #1086 from justinsb/fix_kubecfg_builder_output
Fix typo
2016-12-07 15:58:16 -07:00
Justin Santa Barbara 86622dad09 Cleanups to make kops update apply cleanly for private topologies
Remove a few spurious changes
2016-12-07 11:44:36 -05:00
Justin Santa Barbara 64749953a3 elastic ip cleanup 2016-12-07 11:43:39 -05:00
Justin Santa Barbara 0e7ab42eb7 Merge pull request #1084 from DualSpark/nat-gateway-sanity-check
Adding sanity check for natgateways
2016-12-07 11:05:15 -05:00
Justin Santa Barbara 3ec68efbe7 Fix typo 2016-12-07 08:55:52 -05:00
Kris Childress 7377250a13 logging 2016-12-07 00:13:26 -07:00
Kris Childress 0b941e8649 Merge pull request #1050 from DualSpark/kubectl-context-message
Kubectl context message
2016-12-06 23:58:36 -07:00
Kris Childress db11e113d9 Adding sanity check for natgateways 2016-12-06 23:49:08 -07:00
Justin Santa Barbara 68250e4b9a Fixes to existing NAT gateway detection 2016-12-06 13:28:10 -05:00
yancl aec534a93f change KOPS_AWS_REGION to AWS_REGION 2016-12-05 15:04:03 +00:00
Justin Santa Barbara ade674e4b4 Remove unused import
Fix mistake in rebase
2016-12-05 02:32:34 -05:00
Justin Santa Barbara d1ea4f969a Make sure we set APIServerCount
It looks like it got lost in a refactor.  Add a unit test, and move
initialization to code (and have the code self-check as well).

Also we can now have a fairly long code comment about the reasons why
this is such a mess...

Fix #371
2016-12-05 02:30:53 -05:00
icereval 2bea2987dd add path.module 2016-12-04 18:37:37 -05:00
chrislovecnm bad489cb75 message when we change kubectl context
fixing kubectl config path when set to multiple paths
2016-12-04 15:58:35 -05:00
Justin Santa Barbara df9e1e44db Hotfix for segfaults 2016-12-04 12:44:22 -05:00
alok87 99aa9d6490 Merge remote-tracking branch 'kopsrepo/master' into bastion_improvements
* kopsrepo/master: (29 commits)
  Add verify-boilerplate target
  Add logging of AWS retries
  adding hack/verify-boilerplate.sh to make ci target
  Print time remaining to succeed as a positive value
  adding hack/verify-boilerplate.sh to make ci target
  updating headers, OMG we need this in the ci
  Format resource diffs
  Include error in message when we fail to query AZs
  Import tidying
  Apply gofmt
  Update cmd/kops/validate_cluster for refactor
  Move to pkg/validation and tidy up
  Update command building pattern, a few tweaks
  adds more machine types
  fix path to adding feature doc
  Update dns-controller README
  bug in my fix header script
  updating header
  bumping weave version
  Remove old file
  ...
2016-12-04 17:24:29 +05:30
alok87 66d2e4791d IdleTimeout configurable from editcluster 2016-12-04 16:35:39 +05:30
alok87 0ab99a432f Headers updated for ci fix 2016-12-04 15:15:10 +05:30
alok87 91e332f759 Handled nil checks for attributes 2016-12-04 14:59:29 +05:30
Justin Santa Barbara 5a4cdceb56 Merge pull request #997 from justinsb/format_diff
Format resource diffs
2016-12-03 22:00:57 -05:00
Justin Santa Barbara a2832f8a20 Merge pull request #1045 from justinsb/print_ttl_as_positive
Print time remaining to succeed as a positive value
2016-12-03 18:38:30 -05:00
Justin Santa Barbara b65d9e3463 Add logging of AWS retries
Also of logging at v=2
2016-12-03 17:14:16 -05:00
Justin Santa Barbara 7a5ad979fb Print time remaining to succeed as a positive value 2016-12-03 16:29:26 -05:00
Justin Santa Barbara c8812ab563 Format resource diffs
Fix #973
2016-12-03 10:26:45 -05:00
Justin Santa Barbara 8d9d65d004 Include error in message when we fail to query AZs 2016-12-03 09:45:03 -05:00
alok87 ef73285659 Connection settings loadbalancer 2016-12-03 02:38:22 +05:30
alok87 fd51303622 Check for topology nil and bastionMachineType empty handle 2016-12-02 13:20:06 +05:30
Kris Childress c3d166805f Merge pull request #1021 from DualSpark/weave-bump-version
bumping weave version
2016-12-01 20:43:29 -07:00
Zachary Adam Kaplan 8b91b71e34 adds more machine types
adds the following machine types:

* r4.large
* r4.xlarge
* r4.2xlarge
* r4.4xlarge
* r4.8xlarge
* r4.16xlarge
* t2.xlarge
* t2.2xlarge
2016-12-02 00:23:49 +00:00
chrislovecnm 517a81bb05 bug in my fix header script 2016-11-30 22:52:04 -07:00
chrislovecnm 3f29bc982e updating header 2016-11-30 22:46:33 -07:00
chrislovecnm 1bbbe0b71d bumping weave version 2016-11-30 22:06:12 -07:00
alok87 d4eccb2688 Made bastion as part of TopologySpec 2016-12-01 09:56:45 +05:30
Kris Childress f452890a86 Merge pull request #995 from justinsb/messaging_waiting_for_nat_gateway
Refactor wait-for-NAT-gateway code & add log messages
2016-11-30 08:44:23 -07:00
alok87 fa18857b43 Auto generated fi tasks for loadbalancer attributes 2016-11-30 09:16:22 +05:30
alok87 179ef94174 Unmarshal json to go values for bastion 2016-11-30 08:37:56 +05:30
alok87 a413ea5ac3 Merge remote-tracking branch 'kopsrepo/master' into bastion_improvements
* kopsrepo/master:
  gcs-upload: Use a no-clobber copy instead
  gcs-upload: Fix cache-control on other files as well
  changes from code review
  doc updates
  unit tests with fakes
  it is working in alpha
  working on the start of validate
  Starting work on node lookup and validation
  starting porting node code
  Fix retries for AutoScalingGroup pending delete
  Apply gofmt to pkg directory
  Avoid tests hitting kubernetes stable.txt HTTP file
  Fix printing of max size on instance group
  Disable kubelet from starting until after volume mounts
  Fix Cluster parsing error message
  bumping stable channel to k8s 1.4.6
  support more zones(cn-north-1a/b) for cloud provider guess
2016-11-30 07:58:41 +05:30
alok87 f0b80503c3 Configure LoadBalancer Attributes 2016-11-30 07:35:45 +05:30
Kris Childress 72c5868966 Merge pull request #707 from DualSpark/node-validation
New feature - node validation
2016-11-29 16:47:44 -07:00
Chris Love 44555739aa Merge pull request #1004 from zmerlynn/wait-asg-pending
Fix retries for AutoScalingGroup pending delete
2016-11-28 19:19:04 -07:00
chrislovecnm 82cf2c2c0f working on the start of validate
cluster validation seem to be working.  Need to test more

documentation yo

refactoring to get rid of import cycle not allowed
2016-11-28 18:16:32 -07:00
chrislovecnm 3ffc4e91cb Starting work on node lookup and validation 2016-11-28 18:16:31 -07:00
chrislovecnm 89ac0d24f8 starting porting node code 2016-11-28 18:16:31 -07:00
Zach Loafman e60c36068f Fix retries for AutoScalingGroup pending delete
This:
- reworks how retries are handled in fi/executor.go to a time-based scheme
- changes the single-task limit to 10m (from about 30s of no-progress)
- eliminates the inner IAM propagation retry for LaunchConfigurations,
because the task itself will just be redriven for a while. This also
eliminates any long-pole delay caused by this error (since task Run()
should be 'fast').
2016-11-28 17:04:02 -08:00
Justin Santa Barbara fced564a8f Refactor wait-for-NAT-gateway code & add log messages
So users know what is happening
2016-11-28 10:23:47 -05:00
Justin Santa Barbara 4901573a0c Avoid tests hitting kubernetes stable.txt HTTP file
Fix #949
2016-11-28 02:09:04 -05:00
Justin Santa Barbara ddfa3e467e Merge pull request #963 from yancl/support-more-zones-for-cloud-provider-guess
support more zones(cn-north-1a/b) for cloud provider guess
2016-11-28 01:44:17 -05:00
Zach Loafman 1f657990b3 Disable kubelet from starting until after volume mounts
* Change protokube to do `systemctl start kubelet` every sync round
** .. which takes a change to the systemd unit for protokube to mount in D-Bus
* Don't start kubelet in nodeup
2016-11-23 11:30:19 -08:00
Zach Loafman 5b3da840d5 Fix Cluster parsing error message 2016-11-23 07:47:39 -08:00
alok87 edf22f3797 Bastion DNS as an option and not by default 2016-11-23 12:40:45 +05:30
alok87 6b17c27572 Bastion Improvements 2016-11-23 12:37:42 +05:30
alok87 6206bc711e Separate configuration for bastion 2016-11-23 12:35:10 +05:30
chrislovecnm a47e0ccc10 missed addons 2016-11-22 16:29:05 -05:00
chrislovecnm 6e9a88151a starting work on limits 2016-11-22 13:20:57 -07:00
Kris Childress 6e12fcf449 Merge pull request #944 from emerlinsky/patch-2
Wait for NatGateway creation to complete
2016-11-21 15:50:48 -07:00
yancl 927aa2ba69 support more zones(cn-north-1a/b) for cloud provider guess 2016-11-21 11:04:27 +00:00
yancl bc444fd535 fix the aws region that hardcoded to "us-east-1" which doesn't work in
some isolated regions(cn-north-1 and GovCloud, for example)
2016-11-21 08:53:45 +00:00
Justin Santa Barbara 6f20979790 Adapt unit tests to sets.String; convert node tags also 2016-11-19 16:23:51 -05:00
Justin Santa Barbara c3fd29a0e9 Logging cleanup 2016-11-19 16:23:51 -05:00
Justin Santa Barbara a3fa83ac34 Switch to use sets.String 2016-11-19 16:23:51 -05:00
Justin Santa Barbara 76dc86ff49 Apply gofmt 2016-11-19 15:01:06 -05:00
Justin Santa Barbara a34adc9be3 Update for new mount API 2016-11-19 15:01:00 -05:00
emerlinsky 0c21860e49 Wait for NatGateway creation to complete
NatGateway creation in AWS is a long procedure. It can take up to 10 min for NatGateway to go from Pending to Available state.
We have to use WaitUntilNatGatewayAvailable function to make sure that NatGateway is fully up before trying to use it.
Without this change all my tests attempts to create or update (add nodes to) kubernetis cluster with private topology in us-east-1 region failed with error:

```
W1117 12:14:08.719010   51863 executor.go:100] error running task "route/private-us-east-1c.kubpriv.pink-ptdevcloud.com": error creating Route: InvalidNatGatewayID.NotFound: The natGateway ID 'nat-08be6e70ddffd44d4' does not exist
	status code: 400, request id: 5adf5c0a-c12f-4d6b-8dfd-186c51efff9f
```
2016-11-18 10:18:15 -08:00
chrislovecnm 3cd825e1ab removing logging 2016-11-17 00:10:55 -05:00
chrislovecnm 19ea96e387 working on weave 2016-11-16 23:41:50 -05:00
chrislovecnm 4ae70c4fba why is this no worky 2016-11-16 21:11:12 -07:00
chrislovecnm 0657abfa2a adding a test 2016-11-16 22:41:18 -05:00
chrislovecnm a890390fbf updating bugs 2016-11-16 20:05:27 -05:00
chrislovecnm f8e10dc200 because these peeps do awesome code reviews 2016-11-16 16:39:35 -07:00
chrislovecnm 25ee1e4cdb adding weave support 2016-11-16 15:48:32 -07:00
Chris Love 4be4069d7f Merge pull request #898 from justinsb/krouton
Support for kopeio-networking
2016-11-16 12:43:55 -07:00
Justin Santa Barbara ee44353cde Add support for kopeio networking 2016-11-16 14:20:23 -05:00
Zach Loafman 6e96b10f37 Merge pull request #905 from zmerlynn/increase-client-retries
Bump retries up to ~6m for a request
2016-11-15 16:18:52 -08:00
Chris Love b5646694a6 Merge pull request #807 from justinsb/master_resources
Reduce CPURequests, so we can run on 1 core machine
2016-11-15 13:34:49 -07:00
Zach Loafman 64e64df5de Bump retries up to ~6m for a request 2016-11-15 11:09:38 -08:00
Justin Santa Barbara 551e7a1343 Remove dead code: upup/pkg/fi/options.go 2016-11-14 22:47:40 -05:00
Justin Santa Barbara f709f909e4 Options through code 2016-11-14 12:26:42 -05:00
Chris Love 42874db44f Merge pull request #888 from JuJu227/correct_import_for_asset
for whatever reason the import was missing for asset, which caused co…
2016-11-14 09:49:55 -07:00
Justin Santa Barbara ae84beddb7 Use more api machinery when writing 2016-11-13 22:11:36 -05:00
Justin Santa Barbara 933506bdbb Apply gofmt 2016-11-13 21:06:55 -05:00
Carl B. beethoven1110@gmail.com 2e91af9fbe for whatever reason the import was missing for asset, which caused compile failures for api and examples 2016-11-13 12:01:54 -05:00
Chris Love 18879f7094 Merge pull request #861 from DualSpark/patch-single-node
Fixing conditional for our single node friends
2016-11-10 11:11:28 -08:00
alok87 839707debe Comment should be inside the if block Fix for - https://github.com/kubernetes/kops/issues/862 2016-11-10 21:50:10 +05:30
Kris Childress 52f0190080 Fixing conditional for our single node friends 2016-11-09 15:28:09 -08:00
Kris Childress 8c41dad16d Unit Tests
- Adding populateClusterSpec unit tests
 - Topology happy/sad paths
 - Fleshing out topology in the buildMinimalCluster() function
2016-11-08 15:16:41 -08:00
Kris Childress b1febd9caf Stubbing out tests
- Fixing build errors
- Missed a privatemasters reference
- Fixing the nil pointer problem in SG awstask
2016-11-08 15:16:41 -08:00
Kris Childress 6f78e0ca18 Flipping associatePublicIP bool for nodes/bastion/master in private topology 2016-11-08 15:16:41 -08:00
Kris Childress 95a8c59fbf Fixing NP panic 2016-11-08 15:16:41 -08:00
Kris Childress 729598a4ae Updating error message for invalid topology. 2016-11-08 15:16:41 -08:00
Kris Childress b8d2301812 Header linter fix 2016-11-08 15:16:41 -08:00
Kris Childress 3c92c6a9de Fixing verbage for AWS route awstask 2016-11-08 15:16:41 -08:00
Kris Childress c1644cc4e7 Remove refs to `privatemasters` 2016-11-08 15:16:41 -08:00
Kris Childress 479c7782d6 Fixing DNS annotations on pods / Bumping failed iterations
- Adding template functions to nodeup
- Adding 30 -> 42 iteration boost for attempted iterations without progress
- Adding template to kube api manifest
2016-11-08 15:16:41 -08:00
Kris Childress cc2e920008 Fix for https://github.com/kubernetes/kops/pull/694#issuecomment-258308027 2016-11-08 15:16:41 -08:00
Kris Childress 712882f080 K8s API
- Fixing Kubernetes API forwarding in the ELB
 - Fixing DNS for kubectul
 - Fixing Suggestions: output for bastion
2016-11-08 15:16:41 -08:00
Kris Childress 78ecdb2165 Moar YAML cleanup and putting finishing touches on k8s debugging for tomorrow - Oh etcd... <3 2016-11-08 15:16:41 -08:00
Kris Childress 3f4bc39d52 Yaml Docs cleanup 2016-11-08 15:16:41 -08:00
Kris Childress 37f5bb7d57 Working networking commit!
- Stick bastion in ASG
 - ELBs for API and Bastion
2016-11-08 15:16:41 -08:00
Kris Childress 0857ed1732 Working Bastion with ELB - now time to start on the k8s API :) :) :) 2016-11-08 15:16:41 -08:00
Kris Childress 312621b0d0 Pushing up some last minute tweaks before asking for help and feedback from testing 2016-11-08 15:16:41 -08:00
Kris Childress e962f9c5fd Adding bastion support 2016-11-08 15:16:41 -08:00
Kris Childress cebdde3fb4 Woo! Time to start playing with private networks in AWS!! 2016-11-08 15:16:41 -08:00
Kris Childress cb3157935c Fixing CI build and cluster tests to work with topologies 2016-11-08 15:16:41 -08:00
Kris Childress 835e24f788 Working EIP and NGW CRUD for private networking..
Next step.. lets piece them all together
2016-11-08 15:16:41 -08:00
Kris Childress de79ca21b8 Now that is one fine elastic_ip.go file 2016-11-08 15:16:41 -08:00
Kris Childress a3dd1257ce Working ElasticIP associations on subnet. Delete and Create! 2016-11-08 15:16:41 -08:00
Kris Childress c1e8dbe9d6 More work on the network and EIP things 2016-11-08 15:16:41 -08:00
Kris Childress e90b5faf7e fitask header fix 2016-11-08 15:16:41 -08:00
Kris Childress d729596bca fitask header fix 2016-11-08 15:16:41 -08:00
Kris Childress 0f8f2d1ba1 fitask header fix 2016-11-08 15:16:41 -08:00
Kris Childress db693a6add fitask header fix 2016-11-08 15:16:41 -08:00
Kris Childress 9bd9e30bdd Adding another large commit after a make codegen 2016-11-08 15:16:41 -08:00
Kris Childress a1c5c77b23 docs 2016-11-08 15:16:41 -08:00
Kris Childress a1ca6b7a5b More progress - getting out to Github so I can switch laptops... will be needing 8 cores today :D 2016-11-08 15:16:41 -08:00
Kris Childress 8f30225b32 Switching over branches 2016-11-08 15:16:41 -08:00
Kris Childress 8fba14b85b Small refactor - getting ready to start YAML 2016-11-08 15:16:41 -08:00
Kris Childress 000e847af2 Topology Initial Commit
- Refactor private networking -> topology
- Define new topology models (no changes yet)
- Docs
- Create cluster --topology and -t
- New functions for topology templating
2016-11-08 15:16:41 -08:00
Chris Love 60054495b0 Merge pull request #823 from DualSpark/plane-docs
Work from 30k before kubecon - getting some notes into the tree for discussion points
2016-11-08 01:15:12 -08:00
Kris Childress 649aef3782 Merge pull request #825 from justinsb/fix_813
Fix bug in security group matching
2016-11-07 21:40:42 -08:00
Chris Love abb86c3234 Merge pull request #826 from justinsb/redhat_fixes
Recognize redhat.com image-owner-alias, fix docs
2016-11-06 20:04:00 -08:00
Justin Santa Barbara 4eb918a9ae apiserver needs /etc/pki/ca-trust, at least on RHEL
/etc/pki/ca-trust has the actual CA certs, on RHEL & derivatives, and
thus must be mounted by apiserver.  It is used in the volume admission
controller, for example.

Fix #668
2016-11-06 22:15:28 -05:00
Justin Santa Barbara 3e2d89c6b0 Recognize redhat.com image-owner-alias, fix docs 2016-11-06 21:38:21 -05:00
Justin Santa Barbara c02d5d38dc Fix bug in security group matching
We were matching only by the name tag, instead of using the group-name

If users had an SG with the same Name in different VPCs we could have
matched SGs in both VPCs.  This seems unlikely, and particularly tricky
to then get through the remaining sanity checks (even in the find
function itself).

Fix #813
2016-11-06 21:14:43 -05:00
Kris Childress 941ce272db Commit from 30k feet before kubecon, adding information on fi/cloudup/nodeup 2016-11-06 15:33:29 -07:00