Commit Graph

206 Commits

Author SHA1 Message Date
Justin SB b1f8f84306
Code changes for 1.15 2019-08-25 16:00:39 -04:00
Hippo 9326ff006e
Fix that the node of the instance group of the mixed instance type is in the "NeedsUpdate" state even after the rolling update 2019-08-22 09:56:52 +08:00
liranp 80020052d3
feat(spotinst): new instance group type: ocean 2019-06-03 12:38:46 +03:00
Justin SB 3a41588efd
Don't panic when deleting instancegroups
Issue #6999
2019-05-16 09:17:17 -07:00
Pavlo Kutishchev 7a40da5016 LaunchTemplate - support for deletion of instance group which using launch template 2019-05-13 16:24:43 +02:00
Justin SB 3e33ac7682
Change code from glog to klog
We don't call klog.InitFlags yet, because that will cause a flag
redefinition error until we get everyone to stop using glog.  That
will happen when we update to k8s 1.13.
2019-05-06 12:54:51 -04:00
Ryan Bonham eca2b0a27a User version number as well as name when checking LaunchTemplates 2019-04-08 14:11:02 -05:00
mikesplain 14cbad9bc0 Fix tagging and remove tagging elbs 2019-03-29 13:29:11 -04:00
Rohith ec8d1206cd Instance LaunchConfig/Template Bug Fix
On merge of https://github.com/kubernetes/kops/pull/6277 the launchconfiguration or template is evaluated; where as before LC was just taken at face value, now the LC/LT is checked for existence. This causes an issue on rolling nodes, nodes where the LC has dissappeared due to retention and terminating instances
2019-03-06 16:39:39 +00:00
Rohith ff25209381 - fixing up the terraform render to use the correct format for the device mappings (was using the launchconfiguration ones) 2019-02-04 12:00:04 +00:00
Rohith 098b4486f5 - fixing up the spelling mistake and the package update 2019-02-04 11:59:43 +00:00
Rohith 9f5ed3536c - updating the aws cloud group discovery to check against both launch configuration, launch templates and mixed instance polices when deriving the needs update 2019-02-04 11:59:42 +00:00
Derek Lemon -T (delemon - AEROTEK INC at Cisco) 4f0169bb79 codegen 2019-01-16 09:30:40 -07:00
Kierran McPherson 0be767a90a Request AWS ASGs in batches
Signed-off-by: Kierran McPherson <kierran.mcpherson@xero.com>
2018-11-10 11:10:54 +13:00
Liran Polak 9f94c06e67 fix: rename spotinst's feature flag 2018-10-14 11:37:31 +03:00
Liran Polak 7654a923f1 feature: new integration: spotinst 2018-10-14 11:37:31 +03:00
k8s-ci-robot 2ad8588e51
Merge pull request #5635 from nareshku/support-albs-nlbs
Support for deletion of aws resources albs nlbs during delete
2018-09-21 11:40:33 -07:00
Justin Santa Barbara 7861a056d8 Recognize ubuntu images in sshUser dumping 2018-09-16 21:56:44 -04:00
Naresh Kumar Amrutham 37651c3534 fixed merge conflicts 2018-09-10 11:45:47 -07:00
Thanabodee Charoenpiriyakij 49b203e126 Fixes go vet complains
When try `make govet` on `go1.11beta1`. It complains many things
related to invalid string formatting:

```
pkg/kubemanifest/visitor.go:35: Verbose.Infof format %s has arg v of wrong type bool
pkg/kubemanifest/visitor.go:40: Verbose.Infof format %s has arg v of wrong type float64
upup/pkg/fi/cloudup/alitasks/disk.go:76: Verbose.Info call has possible formatting directive %q
upup/pkg/fi/cloudup/alitasks/disk.go:91: Verbose.Info call has possible formatting directive %q
upup/pkg/fi/cloudup/alitasks/launchconfiguration.go:89: Verbose.Info call has possible formatting directive %q
upup/pkg/fi/cloudup/alitasks/loadbalancer.go:71: Verbose.Info call has possible formatting directive %q
upup/pkg/fi/cloudup/alitasks/loadbalancer.go:125: Verbose.Info call has possible formatting directive %q
upup/pkg/fi/cloudup/alitasks/scalinggroup.go:71: Verbose.Info call has possible formatting directive %q
dns-controller/pkg/dns/dnscontroller.go:603: Verbose.Infof format %s has arg records of wrong type []dns.Record
dns-controller/cmd/dns-controller/main.go:184: Verbose.Info call has possible formatting directive %q
pkg/acls/s3/storage.go:62: Verbose.Infof format %q arg u.String is a func value, not called
pkg/apis/kops/validation/validation_test.go:199: T.Fatalf format %q has arg config of wrong type *k8s.io/kops/pkg/apis/kops.DockerConfig
pkg/resources/aws/aws.go:1306: Warning call has possible formatting directive %q
pkg/resources/aws/aws.go:1313: Warning call has possible formatting directive %v
upup/pkg/fi/cloudup/aliup/ali_cloud.go:218: Verbose.Info call has possible formatting directive %q
upup/pkg/fi/cloudup/aliup/ali_cloud.go:290: Verbose.Info call has possible formatting directive %q
upup/pkg/fi/fitasks/keypair.go:266: Errorf format %q has arg e.Name of wrong type *string
upup/pkg/fi/files_owner.go:56: Infof format %s has arg group of wrong type *fi.Group
upup/pkg/fi/users.go:57: Warning call has possible formatting directive %q
upup/pkg/fi/users.go:63: Warning call has possible formatting directive %q
upup/pkg/fi/users.go:68: Warning call has possible formatting directive %q
upup/pkg/fi/users.go:129: Warning call has possible formatting directive %q
upup/pkg/fi/users.go:135: Warning call has possible formatting directive %q
upup/pkg/fi/nodeup/nodetasks/file.go:313: Errorf format %q has arg e.Mode of wrong type *string
upup/pkg/fi/cloudup/awsup/aws_cloud.go:1021: Warningf format %q reads arg #2, but call has 1 arg
upup/pkg/fi/cloudup/awsup/aws_cloud.go:1025: Warningf format %q reads arg #2, but call has 1 arg
```
2018-08-22 22:48:22 +07:00
Naresh Kumar Amrutham 504414dd1c include tests 2018-08-17 13:04:28 -07:00
Naresh Kumar Amrutham 4e9d6760b8 initialize elbv2 session 2018-08-15 10:51:55 -07:00
Naresh Kumar Amrutham 76a0018d53 add delete support for NLBs, ALBs and Target groups 2018-08-14 16:15:22 -07:00
Patrick Harböck 9e891656fa Add amazon.com image owner alias and Amazon Linux 2 documentation 2018-08-03 19:58:46 +02:00
Justin Santa Barbara 5933aed899 Add ssh user to kops toolbox dump
Where we can identify the SSH user to use, we can include it in kops
toolbox dump.  This is a precursor to trying to better understand
what's in an image (warnings about NVME or network drivers, or showing
the correct SSH username)
2018-07-24 17:18:40 -04:00
Zach Aller 0a456d3101 Fix formating 2018-05-16 14:30:10 -05:00
Zach Aller 4cb92b7c1e Kuberenets 1.11 has deprecated ExternalID this replaces it with ProviderID
Per https://github.com/kubernetes/kubernetes/pull/61877 ExternalID
is now removed from k8s.
2018-05-16 13:31:04 -05:00
Justin Santa Barbara f0f9b9a028 Ignore shared-ownership tags on volumes
We were otherwise logging a spurious warning message
2018-04-10 23:58:17 -04:00
Justin Santa Barbara 110c1a2aa7 Find RouteTable by cluster & role tag
Once we dropped the Name tag,we have no way to find RouteTables
otherwise.
2018-03-15 20:11:54 -04:00
Justin Santa Barbara ab7439b360 More mocks - sufficient for privatecalico to roundtrip 2018-03-11 23:12:22 -04:00
Justin Santa Barbara c121e7f14e Create lifecycle test against AWS mocks
We create a cluster, run kops update, then run kops update again.  We
expect no changes on the second update.
2018-03-11 17:04:30 -04:00
Dennis Webb 5855787574 returns latest image if more than 1 found 2018-01-25 10:04:12 -06:00
Kubernetes Submit Queue e88496a248
Merge pull request #4061 from meringu/create-subnets
Automatic merge from submit-queue.

Add --subnets and --utility-subnets to kops create cluster

This change adds two new options to `kops create cluster`

When specifying `--vpc`, `--subnets` can be specified as an unordered array of subnet ids. Kops will then look up the zones of the subnets to find which zone to add the subnet id to.

If `--topology private` is also specified, `--utility-subnets` can similarly be specified.

~If a zone was specified but a subnet wasn't given that matches the zone, then the subnet will be allocated a CIDR with the current behaviour.~ This case fails validation here 7bd0a6a703/pkg/apis/kops/validation/validation.go (L151)

I can add unit tests and docs changes if required, but I am keen to get feedback before I proceed much further.

I have only added support for AWS.

I have tested this by running a command similar to this:

```bash
kops create cluster \
  --zones=us-east-1a,us-east-1b,us-east-1c \
  --topology private \
  --master-zones=us-east-1a,us-east-1b,us-east-1c  \
  --vpc $vpc_id \
  --subnets subnet-111111,subnet-222222,subnet-333333 \
  --utility-subnets subnet-444444,subnet-555555,subnet-666666 \
  $cluster_hosted_zone_name
```

And the cluster spec was as expected.
2017-12-18 21:12:59 -08:00
Henry Muru Paenga afc1b58722 Tests for create cluster in shared vpc 2017-12-15 18:29:20 +13:00
Justin Santa Barbara f03cc5dbd9 Update dnsprovider imports to use our version 2017-12-13 20:43:01 -05:00
Arto Jantunen 53669b8217 Make ELB somewhat mockable 2017-11-28 11:48:57 +02:00
Justin Santa Barbara b05faa0068 Set SleepDelay function in AWS
Works around nil SleepDelay problem: latest aws-sdk-go (in k8s 1.9 and
kops 1.8) has updated SleepDelay logic; fix is in
https://github.com/kubernetes/kubernetes/pull/55307 but that is only in
1.9.

Set the SleepDelay to work around the problem.
2017-11-22 16:50:19 -05:00
chrislovecnm 6dc953c3d4 adding kubernetes core regional global rate limiter that spans all calls 2017-10-26 17:46:21 -06:00
Justin Santa Barbara 737f2fcd80 rolling-update - initial GCE support 2017-10-02 23:07:35 -04:00
Justin Santa Barbara 3478031533 API types changed package 2017-10-01 14:03:56 -04:00
Kubernetes Submit Queue 84891f39c7 Merge pull request #3492 from justinsb/fix_receiver_name
Automatic merge from submit-queue.

golint: rename two receievers for awsCloudImplementation
2017-09-30 20:44:55 -07:00
Kubernetes Submit Queue 852455afed Merge pull request #3439 from justinsb/gce_zones_in_instancegroup
Automatic merge from submit-queue.

Add Zones field to InstanceGroup

The Zones field can specify zones where they are not specified on a
Subnet, for example on GCE where we have regional subnets.
2017-09-30 18:17:32 -07:00
Justin Santa Barbara 7fd1196708 Add Zones field to InstanceGroup
The Zones field can specify zones where they are not specified on a
Subnet, for example on GCE where we have regional subnets.
2017-09-30 19:44:35 -04:00
Justin Santa Barbara abd48ee653 Name CloudInstanceGroupMember consistently
Keep the naming of the type consistent.
2017-09-30 17:39:53 -04:00
Justin Santa Barbara 8280c4f9d3 Remove MarkIsReady function, calculate on demand
Avoids setting the trap of failing to call MarkIsReady
2017-09-30 17:39:53 -04:00
Justin Santa Barbara a5d72ca190 DRY FindAutoscalingGroups by removing from resources pkg 2017-09-30 17:39:53 -04:00
Justin Santa Barbara 44e67c4f93 DRY mock-aws cloudinstances functions 2017-09-30 17:39:53 -04:00
Justin Santa Barbara cc97557096 golint: rename two receievers for awsCloudImplementation
Keep the receiver name the same
2017-09-30 12:21:46 -04:00
chrislovecnm a431eb3e43 refactoring to use cloud based GetGroups 2017-09-29 12:29:07 -06:00
chrislovecnm 93f3600f36 adding aws_cloud instancegroups delete and get methods 2017-09-28 15:52:50 -06:00
chrislovecnm db5b931579 Adding three new funcs for cloud providers, for refactoring delete and
get
2017-09-28 15:52:49 -06:00
Justin Santa Barbara e730e043ab Add function to get etcd status
This will allow us to permit changing the etcd configuration only if
etcd has not yet been configured.
2017-09-22 23:27:26 -04:00
Justin Santa Barbara a26b28576e AWS: Always use verbose errors
As otherwise very difficult to diagnose errors
2017-08-25 23:08:39 -04:00
Lars Lehtonen c26f4b31f0
Return existing struct in awsup error conditions 2017-08-25 13:14:58 -07:00
Lars Lehtonen afea9d05c5
Replace deprecated aws session.New() with session.NewSession() 2017-08-22 17:28:55 -07:00
Kashif Saadat 1574b19798 Correctly delete cloud labels that are no longer present in the expected resources (cloudLabels, labels, taints). 2017-08-19 09:19:02 +01:00
Justin Santa Barbara d21d81dc5b Initial experimental kops-apiserver 2017-07-17 00:51:20 -04:00
Justin Santa Barbara 9d099fb1ea Merge pull request #2837 from justinsb/query_for_available_instance_types
Query for available instance types
2017-07-15 12:31:08 -04:00
Justin Santa Barbara ef99e61f83 Query AWS to determine available instance types 2017-06-30 15:38:32 -04:00
Justin Santa Barbara 47d4f2643b Fix missing ids in a few log messages 2017-06-29 21:26:52 -04:00
Justin Santa Barbara d2df318ecc Move CloudProvider to kops API
This avoids a circular reference when breaking up the fi package
2017-06-17 16:27:07 -04:00
andrewsykim 976b54f398 unit tests for rolling updates 2017-03-24 20:23:55 -04:00
Justin Santa Barbara 645f330dad Re-enable GCE support
We move everything to the models.  We feature-flag it, because we
probably want to change the names etc, and we aren't going to be able to
offer smooth upgrades until that is done.
2017-02-28 20:08:03 -05:00
Kris Nova 52ba8e82c1 Initial commit for removing CloudFormation on kops delete cluster 2017-02-21 13:52:08 -07:00
Justin Santa Barbara 1bacf8271e Initial (experimental) CoreOS support
* Detect CoreOS
* Move key manifests to code, to tolerate read-only mounts
* Misc refactorings so more code can be shared
* Change lots of ints to int32s in the models
* Run nodeup as a oneshot systemd service, rather than relying on
cloud-init behaviour which varies across distros
2017-02-11 13:57:30 -03:00
Kris Nova 930acc3a0f Merge pull request #1540 from justinsb/existing_subnet
Automatically infer VPC & Subnet CIDRs
2017-01-21 11:44:54 -07:00
Justin Santa Barbara 94e54156b2 Fix segfault in NatGateway
Fix #1568
2017-01-20 10:05:22 -05:00
Justin Santa Barbara 30ae86bb86 Automatically infer VPC & Subnet CIDRs
Issue #1492
2017-01-20 02:04:36 -05:00
Justin Santa Barbara afd0c25abe First model -> tf test 2016-12-11 17:11:10 -05:00
Justin Santa Barbara f4092d0113 First test of a fi.Task 2016-12-08 13:17:38 -05:00
Justin Santa Barbara b65d9e3463 Add logging of AWS retries
Also of logging at v=2
2016-12-03 17:14:16 -05:00
Justin Santa Barbara 8d9d65d004 Include error in message when we fail to query AZs 2016-12-03 09:45:03 -05:00
Zach Loafman 64e64df5de Bump retries up to ~6m for a request 2016-11-15 11:09:38 -08:00
Justin Santa Barbara 3e2d89c6b0 Recognize redhat.com image-owner-alias, fix docs 2016-11-06 21:38:21 -05:00
Justin Santa Barbara 626106ffbc Move FindDNSHostedZone to utils 2016-10-19 00:16:38 -04:00
Justin Santa Barbara 6ec5da2827 Add unit test for deleting untagged route table 2016-10-18 22:16:05 -04:00
Justin Santa Barbara 5568568094 Wait longer for eventual consistency convergence
We don't have a lot of choice here; also try to provide more status
feedback.

This is not a true fix, it merely extends the timeouts.

Issue #597
2016-10-18 00:37:47 -04:00
chrislovecnm 8fa2aac99f fixing more headers 2016-10-15 19:20:56 -06:00
Justin Santa Barbara ecf0e94484 Turn on verbose credentials error messages
Issue #605
2016-10-06 13:10:15 -04:00
Justin Santa Barbara aebf5f1e03 Create stable channel for kops
The idea being we don't necessarily always want to pick up the latest
change immediately, without validating it first.

Also this is an easy way to recommend the latest AMI without having to
search for it.  We also create an alias to the owner id, so that it is
readable / verifiable.
2016-09-30 20:59:32 -04:00
Justin Santa Barbara 9ba847ed90 Fix variable masking issue 2016-09-24 18:45:05 -04:00
Justin Santa Barbara bb01cf6ddd Make AWSCloud an interface, mock it out in tests
Beginnings of a mock for the AWSCloud, so that hopefully we aren't
calling out to AWS at all in the tests.  We will likely start mocking
the actual EC2 APIs in future, but this seems a good starting point.

Fix #425
2016-09-24 15:52:52 -04:00
Justin Santa Barbara 3942ffa8fb Deeper DNS validation: don't create hosted zone
Avoids a problem with TF output, but also lays the groundwork for deeper
DNS validation, which is probably the #1 source of problems right now.
2016-09-13 12:45:12 -04:00
Justin Santa Barbara 62d5451b25 Initial (experimental) Ubuntu 16.04 support 2016-09-08 10:20:42 -04:00
Justin Santa Barbara 637a10be12 govet fixes 2016-08-20 01:09:19 -04:00
Justin Santa Barbara 93a3c344ed Add another eventual consistency error code 2016-08-11 02:45:02 -04:00
Justin Santa Barbara 1102e7818d Perform as much validation as possible before writing a cluster config
This prevents us getting into a situation where an invalid cluster is
saved, and then cannot be deleted.
2016-07-14 09:54:21 -04:00
Justin Santa Barbara cae256340b Find matching existing DNS hosted zone
We choose the longest matching existing hosted zone as the default,
rather than the previous heuristic which was overly simplistic.

We also require the hosted zone to exist now; it doesn't seem to really
cost us anything given that the user has to set up DNS delegation
anyway.

Fix #125
2016-07-12 22:15:20 -04:00
Justin Santa Barbara 935b08fba2 Tolerate eventual consistency error tagging gateways 2016-07-09 01:46:32 -04:00
Justin Santa Barbara bfdd976855 More diagnostics when an EC2 zone is not recognized 2016-07-01 10:00:19 -04:00
Justin Santa Barbara d8b661777b Merge cloudup tool into `kops create cluster` 2016-06-30 10:25:50 -04:00
Justin Santa Barbara d427858477 Rename to kops
The upup tool is now called kops, and we have moved repos
2016-06-30 09:25:25 -04:00
Justin Santa Barbara 506d0d555a Merge pull request #127 from justinsb/upup_upgrade_tooling
upup: improved delete; create upup export and upup upgrade
2016-06-23 10:27:32 -04:00
Justin Santa Barbara 90d7fb87ad upup: improved delete; create upup export and upup upgrade 2016-06-23 08:59:20 -04:00
Justin Santa Barbara 93f634b428 upup: use vfs for secretstore/keystore
This is needed so that we can have encrypted storage and complex keys
(e.g. multiple CA certs).  Multiple CA certs are needed for an in-place
upgrade from kube-up v1.
2016-06-23 08:58:54 -04:00
Justin Santa Barbara b52877e2ce upup: separate node & master zone configuration; validate
We allow --zones & --master-zones to be specified separately now, but we
validate for common errors (using a region where you meant a zone,
duplicating a zone, spanning regions, entering an invalid AZ etc)
2016-06-11 21:06:31 -04:00
Justin Santa Barbara 6e7c4209de upup: Simple tag fixes to avoid comparison failures
We populate the actual expected tags in-place now
2016-06-10 11:33:33 -04:00
Justin Santa Barbara 4c0f54f95d upup: run create & delete in parallel
It makes logging harder, but we want to start testing it!
2016-06-07 15:23:07 -04:00
Justin Santa Barbara 9d6847ff49 cloudup: better retrying around resource tagging 2016-06-04 16:14:00 -04:00
Justin Santa Barbara 4a85ef64fb Merge pull request #48 from justinsb/upup_elb
upup: simple ELB support
2016-06-04 15:56:57 -04:00
Justin Santa Barbara 3a660fe8fa upup: simple ELB support
Add support for ELB, though this doesn't wire it up yet
2016-05-30 17:53:05 -04:00
Justin Santa Barbara 3389323392 upup: Simple route53 (DNS) support 2016-05-30 17:50:55 -04:00
Mike Danese 19bb767767 Merge pull request #28 from justinsb/upup_aws_retry_tags_on_eventual_consistency_error
upup: tolerate aws eventual-consistency errors on tags
2016-05-17 09:34:53 -07:00
Justin Santa Barbara c68aeec94d upup: tolerate aws eventual-consistency errors on tags
AWS will sometimes return an error like "resource not found" when a
DescribeTags or CreateTags call immediately follows creation of the
resource.  Introduce a retry-loop when we get an error that is of the
appropriate type.
2016-05-16 12:08:28 -04:00
Justin Santa Barbara 14ac983241 upup: Map matching image ids to source name
This avoids spurious changes, and also is more intuitive for the user -
whatever name the user gave it, if it resolves to the same image, that
is the name we will use.
2016-05-16 11:20:24 -04:00
Justin Santa Barbara caccb8953f UpUp: AWS support
Adds AWS support for both cloudup & nodeup.
Also cleaning up things found along the way!
2016-05-09 13:08:27 -04:00