Commit Graph

91 Commits

Author SHA1 Message Date
Peter Rifel 0bbaaa3742
Kubetest2 - use same kops binary for all commands in upgrade scenario
This ensures that the same binary is used for kubetest2-kops commands as well as the kops commands invoked directly in the scenario script.

Periodic jobs will create a temp file that will be used to save the kops binary from the provided version marker.
non-periodic jobs (local invocation) will use the bazel build binary, preserving original behavior but using this same binary for kops commands rather than relying on PATH.
2021-03-11 16:47:29 -06:00
Peter Rifel 3fb7cbbf0d
Remove trailing newline from kubernetes version marker
Should fix the trailing "%0A" here: https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/e2e-kops-aws-misc-arm64-ci/1369566440318832640#1:build-log.txt%3A90
2021-03-10 07:52:00 -06:00
Kubernetes Prow Robot de22989eda
Merge pull request #11005 from rifelpet/upgrade-periodic
Don't build kops during periodic upgrade tests
2021-03-09 21:57:13 -08:00
Kubernetes Prow Robot e7ef671e5a
Merge pull request #11007 from rifelpet/kubetest2-ci-version
Kubetest2 - Fix kops' --kubernetes-version with k8s version markers
2021-03-09 21:15:14 -08:00
Peter Rifel a3f49646ac
actually build kops on kubetest2 presubmit 2021-03-09 21:28:04 -06:00
Peter Rifel 864039e606
Kubetest2 - Have kops' --kubernetes-version flag value include the bucket's url
This matches the kubetest1 behavior.
See an example kops create cluster command here: https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/e2e-kops-aws-misc-arm64-conformance/1369076996193128448#1:build-log.txt%3A174
The kubetest 1 logic is here: 37b80c5e3b/kubetest/kops.go (L390-L400)
2021-03-09 21:19:44 -06:00
Kubernetes Prow Robot 9eca4115a3
Merge pull request #11006 from rifelpet/kubetest2-publish
Kubetest2 - Add support for publishing the kops version marker
2021-03-09 19:15:13 -08:00
Peter Rifel 10b30eff0e
Kubetest2 - Add support for publishing the kops version marker
This will unblock the remaining periodic e2e jobs that havent been migrated yet.
They run a test with the kops version from "latest-ci.txt" as published by the "postsubmit-push-to-staging" postsubmit job,
and if the tests succeed then they get published to "latest-ci-updown-green.txt" which is what all of the other periodic jobs rely on.

example job that uses this functionality: 37b80c5e3b/config/jobs/kubernetes/kops/kops-pipeline.yaml (L46-L48)
2021-03-09 20:31:10 -06:00
Peter Rifel 208a4a2bf7
Don't build kops during periodic upgrade tests
This updates the upgrade scenario script to support building kops when ran locally, or using the version markers when ran in a periodic prow job.

hoping to fix the upgrade tests here: https://testgrid.k8s.io/kops-kubetest2#kops-aws-upgrade
2021-03-09 20:17:37 -06:00
Peter Rifel 594cdc6ed1
kubetest2 - don't overwrite create args that use equals signs
Previously we would incorrectly append create cluster arguments if they had already been specified and used --foo=bar notation.
This resulted in arguments being specified multiple times causing undesired behavior.
We now check for both `--foo bar` and `--foo=bar` when attempting to add a `--foo` argument.
2021-03-08 11:35:36 -06:00
Barry Melbourne 1a6ff4efbf Update Go to v1.16 2021-03-04 18:41:48 +00:00
Barry Melbourne a77233a843 Update Openstack Cloud Go module to v1.20.1 2021-02-21 12:19:05 +00:00
Peter Rifel d6f3d3c690
kubetest2 - support terraform with `kops create cluster`
Previously we were only recognizing it with `kops create --filename`
2021-02-17 23:25:35 -06:00
Peter Rifel 6424867614
use go install to install `kubetest2` 2021-02-17 19:20:50 -06:00
Peter Rifel 8a5e95be0a
Pass --debug to ginkgo to enable node log files in presubmit test 2021-02-17 19:13:52 -06:00
Peter Rifel fb0a29aaa6
Bump kubetest2 version to include --ginkgo-args 2021-02-17 19:13:52 -06:00
Peter Rifel 6e8a329f7d
Validate cluster 10 times to mimic kubetest1 and ensure nodes readiness is not flapping 2021-02-16 08:11:51 -06:00
Peter Rifel 68d2044b12
kubetest2 - add terraform support 2021-02-16 08:11:51 -06:00
Justin SB f4dfb43993 kubetest2: Call Test, not Execute
Execute will reparse the flags; we want to reuse the test execution
but not the flag setup.
2021-02-14 21:03:55 -05:00
Justin SB 9ab363dfb8 kubetest2: Add --host argument
This argument was required for the tests to pass prior to 1.18, so we
should still pass it if we want these tests to pass.

We can infer it from kubeconfig (that is the fix that was introduced in 1.18)
2021-02-13 18:16:13 -05:00
Peter Rifel 47b0551014
kubetest2 - Dump all pod logs in addition to host logs 2021-02-11 20:00:36 -06:00
Peter Rifel e325d8b72a
Kubetest2 - Fix splitting of --create-args 2021-02-09 07:07:25 -06:00
Peter Rifel 144d038b76
Kubetest2 - Use a shell lexer for passing extra args to `create cluster`
This way any spaces within a quoted value wont cause the value to be broken up into multiple arguments.
The CentOS image in AWS has spaces in its name which is what exposed this issue.
2021-02-08 22:12:30 -06:00
Peter Rifel 2787218fa6
Update the skipped tests in the upgrade job to help the test stage pass
We'll look into these failures separately but for now we just want to get to the upgrade process
2021-02-02 12:09:52 -06:00
Peter Rifel 1ce2352646
Fix create args for upgrade test 2021-02-02 09:16:13 -06:00
Peter Rifel dd41a2f8fe
Update upgrade test to use 1.18->1.19 2021-02-02 07:41:45 -06:00
Peter Rifel 72ae782c04
Dont overwrite default feature flags when appending more flags 2021-02-01 12:27:33 -06:00
Peter Rifel 044cba7a7a
Kubetest2 - refactor how `kops create cluster` arguments are set
This allows kubetest's --create-args to take precedence over arguments we were otherwise hardcoding.
Specifically this will be used to hardcode the list of zones for some prow jobs with `--create-args="--zones us-east-1a"`.
2021-02-01 12:27:19 -06:00
Peter Rifel 942c48c04b
Remove unnecessary --networking flag from kubetest2.
this can be specified in --create-args
2021-02-01 12:27:18 -06:00
Peter Rifel accbf7986d
kubetest2 - Use --ssh-user to dump logs 2021-01-28 19:45:06 -06:00
Peter Rifel cf12fe4efd
Support additional feature flags specified via --env=KOPS_FEATURE_FLAGS=foo 2021-01-25 21:50:30 -06:00
Peter Rifel 39537d7190
Add --create-args kubetest2 flag
This allows us to customize the images, instance types, etc.

kubetest (1) named this --kops-args, I think --create-args is more appropriate because its only passed to `create cluster`
2021-01-25 20:48:38 -06:00
Peter Rifel 6114c4fbfa
Use the same package marker for kubectl as for e2e binary 2021-01-23 19:21:47 -06:00
Kubernetes Prow Robot c3eda853a8
Merge pull request #10646 from rifelpet/kubetest-upgrade
Update kubetest2 library
2021-01-23 10:53:39 -08:00
Peter Rifel f492f3d82e
Update kubetest2 library
This picks up the --test-package-marker support used by the kops grid jobs
2021-01-23 12:10:53 -06:00
Kubernetes Prow Robot 3d03968083
Merge pull request #10622 from srikiz/DO-SupportCorrectRegions
[Digital Ocean] Add SFO3 region. Also update e2e tests to use full list of supported zones
2021-01-22 11:15:27 -08:00
srikiz dd5c386564 Fix trailing characters 2021-01-22 22:44:36 +05:30
srikiz bbe692486d Revert go.mod and go.sum 2021-01-22 21:19:30 +05:30
srikiz b04af61a4b Add SFO3 region for DO. Also update e2e tests to use the full list of supported zones 2021-01-22 21:19:30 +05:30
Peter Rifel 9e0bd463ad
Fix phony make target for setting up kubetest2 2021-01-22 07:04:23 -06:00
Kubernetes Prow Robot 7c93c3e7b1
Merge pull request #10632 from hakman/kubetest2-validation-timeout
kubetest2 - increase validation timeout for the upgrade scenario
2021-01-21 22:49:28 -08:00
Ciprian Hacman f124a19740 kubetest2 - increase validation timeout for the upgrade scenario 2021-01-22 07:35:19 +02:00
Peter Rifel 17eed0ead2
Fix unbound variable in upgrade scenario script 2021-01-21 14:58:02 -06:00
Peter Rifel aa47981f07
Ensure SpecOverrideFlag is set in upgrade test 2021-01-21 11:21:43 -06:00
Peter Rifel 127d3bb712
kubetest2 - update the skip regex for the upgrade scenario 2021-01-21 06:17:37 -06:00
Peter Rifel 6da7c2ac33
kubetest 2 - fix parsing of k8s version semver values
it turns out "v1.2.3" parses as a URL, so try parsing values as a semver first before URL
2021-01-20 21:46:27 -06:00
Peter Rifel 631e0fe235
Add support for specifying a kubernetes version marker file 2021-01-20 19:18:55 -06:00
Peter Rifel c5664f7363
kubetest2 upgrade script - PATH needs to be a directory
another attempt at fixing https://prow.k8s.io/view/gcs/kubernetes-jenkins/logs/e2e-kops-aws-misc-upgrade/1351855654456791040
2021-01-20 06:21:24 -06:00
Peter Rifel 90a15a7977
Set --ssh-{private,public}-key-path defaults in all kubetest2-kops commands
Previously the default was only used in invocations using --up.
2021-01-19 22:10:01 -06:00
Peter Rifel 9e6218bcb4
Use kops binary built by kubetest2-kops in upgrade script 2021-01-19 22:07:11 -06:00