Commit Graph

806 Commits

Author SHA1 Message Date
Kubernetes Submit Queue b02c3a269c
Merge pull request #3726 from justinsb/storage_acls
Automatic merge from submit-queue.

GCS: Use ACLs for GCE permissions

This needs less permissions, but also allows for more granular control over
access to files than whole-bucket permissions that IAM gives us.
2017-10-29 20:18:51 -07:00
Justin Santa Barbara b2bcba4a6d GCE: Use object-level permissions for files in GCS
This lets us configure cross-project permissions while ourselves needing
minimal permissions, but also gives us a nice hook for future lockdown
of object-level permissions.
2017-10-29 19:17:00 -04:00
Justin Santa Barbara 1f0601126c Refactor gce resources into pkg/resources/gce
Should be a no-op refactor
2017-10-29 18:37:23 -04:00
Kubernetes Submit Queue 322507ced3
Merge pull request #3720 from justinsb/toolbox_refactor_2
Automatic merge from submit-queue.

Move DeleteResources into pkg/resources/utils
2017-10-29 15:34:44 -07:00
Justin Santa Barbara d1ee8026ac GCE: Tasks for object & bucket level permissions
We also switch to setting a bucket-level ACL permission, as this
requires less permissions.
2017-10-29 18:08:08 -04:00
Justin Santa Barbara 66e74fac6a Rename pkg/resources/utils -> pkg/resources/ops
That seems to be what utils really is.
2017-10-29 16:32:48 -04:00
Kubernetes Submit Queue dd207dcd7c
Merge pull request #3719 from justinsb/tooldump_dump_instances
Automatic merge from submit-queue.

Refactor toolbox dump & dump structured instances
2017-10-29 10:25:43 -07:00
Justin Santa Barbara 1eb17ae6b9 Move DeleteResources into pkg/resources/utils 2017-10-29 12:26:07 -04:00
Kubernetes Submit Queue cc4d4c437b
Merge pull request #3692 from gambol99/audit_policy
Automatic merge from submit-queue.

Audit Policy File

The current implementation doesn't allow users to set the [advanced audit policy](https://v1-7.docs.kubernetes.io/docs/tasks/debug-application-cluster/audit/#advanced-audit) location. Note, the file contents can be pushed by a [FileAsset](https://v1-7.docs.kubernetes.io/docs/tasks/debug-application-cluster/audit/#advanced-audit) a sample given below .... Or do we want an explicit secret for this? .. 

```YAML
fileAssets:
- name: audit-policy.conf
  path: /srv/kubernetes/audit-policy.conf
  roles: [Master]
  content: |
    some_content
```
related to https://github.com/kubernetes/kops/issues/3672
2017-10-28 17:44:39 -07:00
Justin Santa Barbara a18363f059 Refactor toolbox dump & dump structured instances
This will enable log collection even if nodes don't register.

AWS: Dumps ids & addresses
GCE: Dumps names - addresses to follow
Others: Not yet!
2017-10-28 14:46:28 -04:00
Kubernetes Submit Queue 8885c6d77b
Merge pull request #3520 from chrislovecnm/api-docs-updates
Automatic merge from submit-queue.

API docs updates

- tweak to .gitignore
- updating api docs
- updated docs/apireference/README.md with how to generate docs

@justinsb any idea what happened to the hack script? `hack/make-gendocs.sh` did not work last time I tried to do this, but I updated the instructions in the `docs/apireference/README.md` for how to get it to work now.
2017-10-28 11:18:28 -07:00
Kubernetes Submit Queue 4162ae1e3b
Merge pull request #3712 from justinsb/storage_acls_minimal
Automatic merge from submit-queue.

GCE: Set up permissions for cross-project configurations
2017-10-28 07:15:19 -07:00
chrislovecnm b3a93bb2ee updating apireference docs 2017-10-28 07:00:24 -06:00
Justin Santa Barbara 15dedf8e79 Updates from running gazelle 2017-10-28 03:27:18 -04:00
Justin Santa Barbara dbbe3f373b GCE: Set up permissions for cross-project configurations
This ensures that the cluster can read the kops state store files, even
if the GCS bucket is in a different project.

We automatically set up an IAM access policy that grants access.
2017-10-28 03:24:18 -04:00
chrislovecnm 4711d1596e update to imports that apimachinery is doing now automatically 2017-10-27 14:28:48 -06:00
Kubernetes Submit Queue a4d6895472 Merge pull request #3707 from andrewsykim/droplet
Automatic merge from submit-queue.

Implement DigitalOcean Droplet FI Task

Implements cloudup fi tasks for DigitalOcean droplets. It makes a few assumptions to reduce the size of this PR, those will be addressed in future PRs. 

Also does some cleanup in the DigitalOcean `dns` package.
2017-10-27 08:30:57 -07:00
andrewsykim 45bf2cb3a7 Implement DigitalOcean Droplet FI Task 2017-10-27 09:48:42 -04:00
Kubernetes Submit Queue aab00d7dc3 Merge pull request #3699 from brdude/disable_kube-proxy
Automatic merge from submit-queue.

Allow disabling kube-proxy

This adds the ability to turn off kube-proxy.

My specific use case is the usage of a custom CNI.
2017-10-26 23:05:57 -07:00
Manatsawin Hanmongkolchai a708919bf4 Generate scheduler policy by dynamic cluster addons 2017-10-27 08:56:07 +07:00
Kubernetes Submit Queue 68c90369ba Merge pull request #3661 from chrislovecnm/etcd-container
Automatic merge from submit-queue.

etcd using asset builder

Updating etcd to use asset builder
2017-10-26 18:29:31 -07:00
chrislovecnm 4de78b0055 setting up etcd to use asset builder for its container 2017-10-26 17:25:50 -06:00
Rohith e995c7b98e Audit Policy File
Extending the KubeAPI component config to allow setting the audit-policy-file
2017-10-26 22:08:37 +00:00
chrislovecnm dc338c4829 Refactoring phases to work in sequence properly 2017-10-26 13:17:34 -06:00
Kubernetes Submit Queue b8a0d71081 Merge pull request #3686 from justinsb/ig_alias
Automatic merge from submit-queue.

kops-server: ig short-name
2017-10-25 19:38:47 -07:00
Justin Santa Barbara 8faebc67c0 Spelling fixes 2017-10-25 21:39:52 -04:00
Rodrigo Menezes 1edd99ccc1 Allow disabling kube-proxy 2017-10-25 14:23:58 -07:00
Rodrigo Menezes d803480485 Merge remote-tracking branch 'upstream/master' into extra_user-data 2017-10-24 12:52:19 -07:00
Kubernetes Submit Queue 8df13bd468 Merge pull request #3679 from justinsb/support_api_aggregation
Automatic merge from submit-queue.

Initial aggregation work

Create the keypairs, which are supposed to be signed by a different CA.
    
Set the `--requestheader-...` flags on apiserver.
    
Fix #3152
Fix #2691
2017-10-24 12:08:27 -07:00
Kashif Saadat 5bfb22ac92 Make the IAM ECR Permissions optional, can be specified within the Cluster Spec. 2017-10-24 09:20:17 +01:00
Justin Santa Barbara a7f82a6380 Fix shared subnet/vpc tags
* Stop setting the Name tag on a shared subnet/vpc

* Stop setting the legacy KubernetesCluster tag on a shared subnet/vpc
that is new enough (>=1.6); we rely on the shared tags instead

* Set tags on shared subnets; i.e. we _do_ set the shared tag on a
shared subnet; that is important for ELBs

* Set tags on shared VPCs; i.e. we _do_ set the shared tag on a shared
VPC; that is not used but consistent with subnets.

* Add tests for shared subnet
2017-10-23 11:30:25 -04:00
Kashif Saadat 28c4b7aca9 Add IAM Permissions so nodes can access AWS ECR 2017-10-23 10:11:27 +01:00
Justin Santa Barbara ed2dab1af5 kops-server: ig short-name
Also cleanup the shared REST subclass
2017-10-23 00:55:55 -04:00
Justin Santa Barbara 15d50aff48 Update tests; add to bazel 2017-10-23 00:42:56 -04:00
Kubernetes Submit Queue 0ae2bb835f Merge pull request #3684 from justinsb/delete_firewall_rules_gce
Automatic merge from submit-queue.

Delete firewall rules on GCE
2017-10-22 16:23:10 -07:00
Kubernetes Submit Queue a8cebe6679 Merge pull request #3680 from gambol99/missing_vars
Automatic merge from submit-queue.

Kops Toolbox Template Missing Variables

The current implementation fails on templates which reference unset variables, it is however useful at times to permit overriding this behavior and using sprig default() for example to handle defaults.

- added a new command line option --fail-on-missing (defaults to true, so keeps the current behaviour)
- updated the unit tests to reflect the changes
2017-10-22 14:51:26 -07:00
Justin Santa Barbara badef7b056 bazel: set test size on pkg/resources 2017-10-22 16:55:36 -04:00
Justin Santa Barbara 5b08c36cbb Delete firewall rules on GCE
As part of cluster delete on GCE, we now clean up our firewall rules.
2017-10-22 16:54:43 -04:00
Eric Hole 742278be01 First pass 2017-10-22 11:47:35 -07:00
Justin Santa Barbara a879521ba3 Initial aggregation support
Create the keypairs, which are supposed to be signed by a different CA.

Set the `--requestheader-...` flags on apiserver.

Fix #3152
Fix #2691
2017-10-22 14:41:38 -04:00
Rohith 9958d333ae Kops Toolbox Template
The current implementation fails on template which reference unset variables, it is however useful at times to permit overriding this behavior and using sprig default() for example methods to handle it

- added a new command line option --fail-on-missing (defaults to true, so keeps the current behaviour)
- updated the unit test to reflect the changes
- updated the cli docs
2017-10-22 14:29:23 +00:00
Kubernetes Submit Queue 794e12dc52 Merge pull request #3675 from mschurenko/add-etcd-settings-to-nodeup-and-protokube
Automatic merge from submit-queue.

Add etcd settings to nodeup and protokube

This adds etcd settings for leader timeout and heartbeat interval to protokube. It uses the additions to `EtcdClusterSpec` that were given the go ahead in https://github.com/kubernetes/kops/pull/3663

@chrislovecnm 
@robinpercy
2017-10-20 20:10:00 -07:00
Kubernetes Submit Queue 3315d29bdb Merge pull request #3630 from wlan0/api
Automatic merge from submit-queue.

Add Cloud Controller Manager addon

This adds the CCM addon for the Kubernetes cluster. 

This is a follow-up PR to https://github.com/kubernetes/kops/pull/3408. 

cc @chrislovecnm @andrewsykim
2017-10-20 19:07:50 -07:00
Mike Splain 8549e8e7ff Add HorizontalPodAutoscalerSyncPeriod for kubeControllerManager 2017-10-19 17:57:33 -04:00
wlan0 d3143ec3cc add Cloud Controller Manager addon 2017-10-19 12:33:44 -07:00
Matt Schurenko 90c7ccb6c8 adding etcd settings to EtcdClusterSpec 2017-10-18 18:29:00 -04:00
Justin Santa Barbara eec1141a41 Rationalize timeouts for rolling-update
The intervals remain the minimum time between instances; drain &
validate time is additional.
2017-10-17 11:44:46 -04:00
Rodrigo Menezes 75aa120f43 Merge remote-tracking branch 'upstream/master' into extra_user-data 2017-10-15 17:17:12 -07:00
Rodrigo Menezes e77cda1af1 Allow passing in extra user-data to cloud-init 2017-10-15 17:10:03 -07:00
Kubernetes Submit Queue 496575b1e4 Merge pull request #3616 from chrislovecnm/fixing-route-table-task
Automatic merge from submit-queue.

adding missing lifecycle to route table task

We missed one
2017-10-15 13:37:37 -07:00
Kashif Saadat c78790f902 Modified OS detection logic when updating http proxy settings. 2017-10-13 19:08:47 +01:00
chrislovecnm f00ee27dbe adding missing lifecycle to route table task 2017-10-12 11:55:50 -06:00
Kubernetes Submit Queue 5b6a8ecc05 Merge pull request #3593 from justinsb/limit_gce_task_length
Automatic merge from submit-queue.

GCE: Limit length of InstanceTemplate names
2017-10-10 21:49:23 -07:00
Justin Santa Barbara 19677523c0 Mock kops version in tests
This avoids having to update the tests every time our version changes.
2017-10-10 11:11:57 -04:00
Justin Santa Barbara d71bd09a6c GCE: Limit length of InstanceTemplate
We explicitly set a separate prefix for the names, and we ensure it is
not too long
2017-10-10 09:48:38 -04:00
Manatsawin Hanmongkolchai a06fbbac79 Added documentation that PolicyConfigMap should not be used during cluster creation 2017-10-10 09:33:48 +07:00
Manatsawin Hanmongkolchai c00f5ea9a7 Added error when trying to use kube-scheduler policyConfigMap before Kube 1.7.x 2017-10-10 09:10:05 +07:00
Manatsawin Hanmongkolchai 78023ba0df Updated PolicyConfigMap comment 2017-10-10 09:10:05 +07:00
Manatsawin Hanmongkolchai 0f2dde994c Regenerated API 2017-10-10 09:10:04 +07:00
Manatsawin Hanmongkolchai 091e87eb4f Added PolicyConfigMap and PolicyConfigMapNamespace to KubeSchedulerConfig 2017-10-10 09:10:04 +07:00
Kubernetes Submit Queue 1c9df0b269 Merge pull request #3524 from justinsb/gce_format_ig_zones
Automatic merge from submit-queue.

GCE: Fix subnets vs zones formatting of instance groups
2017-10-08 13:16:52 -07:00
Kubernetes Submit Queue b052495e3d Merge pull request #3448 from justinsb/unit_test_new_model_helpers
Automatic merge from submit-queue.

Add unit test for model helper functions
2017-10-08 12:42:30 -07:00
Kubernetes Submit Queue 14ba6d0414 Merge pull request #3474 from chrislovecnm/test-cleanup
Automatic merge from submit-queue.

Fixing stdout on tests - cleaning house

We are printing to STDOUT and logging in some tests.  Not needed.
2017-10-08 10:01:05 -07:00
Justin Santa Barbara 0356f45957 Add unit test for model helper functions 2017-10-08 11:49:39 -04:00
Justin Santa Barbara 8693683324 GCE: use recreateInstance when rolling a MIG 2017-10-04 02:54:40 -04:00
Kubernetes Submit Queue 35ead73978 Merge pull request #3462 from justinsb/map_docker_1_13_1_17_03_1
Automatic merge from submit-queue.

Map docker 1.13.1 & 17.03.1, default 1.8 to 1.13.1

Kubernetes 1.8 adds validation for 1.13.1 and 17.03.1 (17.03.1 being
essentially the same as 1.13.1).

For 1.8, the default should be 1.13.1
2017-10-03 18:10:52 -07:00
Justin Santa Barbara cc559dc373 Map docker 1.13.1 & 17.03.2, default 1.8 to 1.13.1
Kubernetes 1.8 is validated with 1.13.1 and 17.03.2.

For 1.8, the default should be 1.13.1
2017-10-03 19:04:20 -04:00
Justin Santa Barbara 38ea16a4ae GCE: Fix subnets vs zones formatting of instance groups 2017-10-03 13:42:14 -04:00
Justin Santa Barbara b63f4d2dbe Add data dependencies to go_test rules
This lets all the tests pass in bazel, other than create_cluster which
references files outside its tree.
2017-10-03 10:45:50 -04:00
Kubernetes Submit Queue 518e97d97b Merge pull request #3510 from justinsb/bazel
Automatic merge from submit-queue.

Initial bazel support

Builds on the 1.8 version bump

The "trick" is to strip the BUILD & BUILD.bazel files from the vendor-ed deps.

Will rebase after 1.8 version bump merges.
2017-10-03 01:19:27 -07:00
Kubernetes Submit Queue 1c81ec5e42 Merge pull request #3490 from justinsb/download_with_wget
Automatic merge from submit-queue.

Support wget for download, not just curl
2017-10-03 00:45:04 -07:00
Kubernetes Submit Queue 48e61b9523 Merge pull request #3507 from justinsb/gce_rolling_update
Automatic merge from submit-queue.

rolling-update - initial GCE support
2017-10-03 00:05:03 -07:00
Kubernetes Submit Queue 74d48eab46 Merge pull request #3496 from justinsb/using-same-disk-size-refactor
Automatic merge from submit-queue.

Tweak signature in #3478

Follow-on to #3478

Because the default doesn't depend on the user-specified value, it's
misleading to pass it in.
2017-10-02 23:29:40 -07:00
Justin Santa Barbara 737f2fcd80 rolling-update - initial GCE support 2017-10-02 23:07:35 -04:00
Justin Santa Barbara 0143be7c4f autogen: BUILD and BUILD.bazel 2017-10-02 14:27:21 -04:00
Justin Santa Barbara 544990842a More fixes for 1.8 API changes 2017-10-01 23:02:32 -04:00
Justin Santa Barbara 95d4f3eb59 More code updates for 1.8 2017-10-01 21:13:00 -04:00
Justin Santa Barbara 3478031533 API types changed package 2017-10-01 14:03:56 -04:00
Justin Santa Barbara 4b328e67a7 New apimachinery codegen 2017-10-01 13:53:42 -04:00
Justin Santa Barbara 2205e26257 Updates for new apimachinery 2017-10-01 13:53:26 -04:00
Kubernetes Submit Queue 0905e71741 Merge pull request #3411 from justinsb/mirror_stores
Automatic merge from submit-queue.

Mirror keystore & secretstore
2017-09-30 23:08:33 -07:00
Kubernetes Submit Queue 34bb6ec3cd Merge pull request #3494 from justinsb/pod_critical_annotations
Automatic merge from submit-queue.

Add critical pod annotations to our system pods
2017-09-30 22:32:08 -07:00
Justin Santa Barbara a4a58d277d Mirroring - Fixes from code review 2017-09-30 22:05:14 -04:00
Justin Santa Barbara 66b174321f Cleanup signature of default volume-size method
Because the default doesn't depend on the user-specified value, it's
misleading to pass it in.
2017-09-30 21:24:51 -04:00
Kubernetes Submit Queue 852455afed Merge pull request #3439 from justinsb/gce_zones_in_instancegroup
Automatic merge from submit-queue.

Add Zones field to InstanceGroup

The Zones field can specify zones where they are not specified on a
Subnet, for example on GCE where we have regional subnets.
2017-09-30 18:17:32 -07:00
Kubernetes Submit Queue 0ef1b697af Merge pull request #3493 from justinsb/dry_getgroups
Automatic merge from submit-queue.

Minor cleanups to #3446

Felt easier than iterating in PR comments!
2017-09-30 17:19:35 -07:00
Justin Santa Barbara 7fd1196708 Add Zones field to InstanceGroup
The Zones field can specify zones where they are not specified on a
Subnet, for example on GCE where we have regional subnets.
2017-09-30 19:44:35 -04:00
Justin Santa Barbara da99a7270d Subnets on GCE are regional, create one per region
We also make the subnet zone and CIDR optional on GCE
2017-09-30 19:22:14 -04:00
Justin Santa Barbara eb2cd45cdf Support wget for download, not just curl
Some images don't include curl but do have wget.  Prefer curl, but
fallback to wget.

(The official Debian 9 image does not have curl.)
2017-09-30 19:12:58 -04:00
Kubernetes Submit Queue eb110f5b6c Merge pull request #3478 from chrislovecnm/using-same-disk-size-gce
Automatic merge from submit-queue.

using same disk sizes for gce

Using the same disk size for gce and aws.  We need to override the disk sizes in the e2e tests, but that is another PR.

Closes: https://github.com/kubernetes/kops/issues/3294
2017-09-30 16:01:38 -07:00
Justin Santa Barbara abd48ee653 Name CloudInstanceGroupMember consistently
Keep the naming of the type consistent.
2017-09-30 17:39:53 -04:00
Justin Santa Barbara 8280c4f9d3 Remove MarkIsReady function, calculate on demand
Avoids setting the trap of failing to call MarkIsReady
2017-09-30 17:39:53 -04:00
Justin Santa Barbara a5d72ca190 DRY FindAutoscalingGroups by removing from resources pkg 2017-09-30 17:39:53 -04:00
Kubernetes Submit Queue c7b4f7e925 Merge pull request #3446 from chrislovecnm/aws-get-groups
Automatic merge from submit-queue.

Refactoring to use cloud method for getting cloud groups

This builds on various other PRs.  The only two SHAs to review are:

1. Moving delete instancesgroups into its own file d52d767508
2. Refactoring to use cloud based GetCloudGroups c33a078f52

AWS is the only one that has GetCloudGroups implemented at this point.  GCE is next.

TODO

- [x] e2e testing rolling-update
- [x] e2e testing rolling-update with only one instance group
- [x] e2e testing force
- [x] e2e testing cloud-only

Updates

I have moved more of the code into `/pkg/cloudinstances` per guidance from @justinsb!  I am liking it more!
2017-09-30 14:39:06 -07:00
Justin Santa Barbara 383194780a Create helper function for critical pod annotations
In particularly I think we want a toleration also; easiest to put the
code in one function.
2017-09-30 17:38:20 -04:00
chrislovecnm c9cc561885 refactoring FindInstanceTemplates into gce cloud interface 2017-09-30 14:02:58 -06:00
Justin Santa Barbara b32e6b569f baremetal: error if NetworkCIDR set 2017-09-30 15:10:10 -04:00
chrislovecnm c4c63b2b0c using same disk sizes for gce 2017-09-29 16:07:38 -06:00
chrislovecnm a431eb3e43 refactoring to use cloud based GetGroups 2017-09-29 12:29:07 -06:00
chrislovecnm 9bf197830f Fixing stdout on tests 2017-09-29 12:25:07 -06:00