From b117c6fe601ad93c7b9319f5b24e29469f55f22f Mon Sep 17 00:00:00 2001 From: eduartua Date: Mon, 28 Jan 2019 16:41:16 -0600 Subject: [PATCH] file cri-testing-policy.md was moved to the new sig-node folder - URLs in k/community were updated --- contributors/devel/e2e-node-tests.md | 230 +----------------- .../devel/sig-node/cri-testing-policy.md | 2 +- contributors/devel/sig-node/e2e-node-tests.md | 229 +++++++++++++++++ 3 files changed, 232 insertions(+), 229 deletions(-) create mode 100644 contributors/devel/sig-node/e2e-node-tests.md diff --git a/contributors/devel/e2e-node-tests.md b/contributors/devel/e2e-node-tests.md index 4f3327cba..f4809ab7a 100644 --- a/contributors/devel/e2e-node-tests.md +++ b/contributors/devel/e2e-node-tests.md @@ -1,229 +1,3 @@ -# Node End-To-End tests - -Node e2e tests are component tests meant for testing the Kubelet code on a custom host environment. - -Tests can be run either locally or against a host running on GCE. - -Node e2e tests are run as both pre- and post- submit tests by the Kubernetes project. - -*Note: Linux only. Mac and Windows unsupported.* - -*Note: There is no scheduler running. The e2e tests have to do manual scheduling, e.g. by using `framework.PodClient`.* - -# Running tests - -## Locally - -Why run tests *Locally*? Much faster than running tests Remotely. - -Prerequisites: -- [Install etcd](https://github.com/coreos/etcd/releases) on your PATH - - Verify etcd is installed correctly by running `which etcd` - - Or make etcd binary available and executable at `/tmp/etcd` -- [Install ginkgo](https://github.com/onsi/ginkgo) on your PATH - - Verify ginkgo is installed correctly by running `which ginkgo` - -From the Kubernetes base directory, run: - -```sh -make test-e2e-node -``` - -This will: run the *ginkgo* binary against the subdirectory *test/e2e_node*, which will in turn: -- Ask for sudo access (needed for running some of the processes) -- Build the Kubernetes source code -- Pre-pull docker images used by the tests -- Start a local instance of *etcd* -- Start a local instance of *kube-apiserver* -- Start a local instance of *kubelet* -- Run the test using the locally started processes -- Output the test results to STDOUT -- Stop *kubelet*, *kube-apiserver*, and *etcd* - -## Remotely - -Why Run tests *Remotely*? Tests will be run in a customized pristine environment. Closely mimics what will be done -as pre- and post- submit testing performed by the project. - -Prerequisites: -- [join the googlegroup](https://groups.google.com/forum/#!forum/kubernetes-dev) -`kubernetes-dev@googlegroups.com` - - *This provides read access to the node test images.* -- Setup a [Google Cloud Platform](https://cloud.google.com/) account and project with Google Compute Engine enabled -- Install and setup the [gcloud sdk](https://cloud.google.com/sdk/downloads) - - Verify the sdk is setup correctly by running `gcloud compute instances list` and `gcloud compute images list --project kubernetes-node-e2e-images` - -Run: - -```sh -make test-e2e-node REMOTE=true -``` - -This will: -- Build the Kubernetes source code -- Create a new GCE instance using the default test image - - Instance will be called **test-e2e-node-containervm-v20160321-image** -- Lookup the instance public ip address -- Copy a compressed archive file to the host containing the following binaries: - - ginkgo - - kubelet - - kube-apiserver - - e2e_node.test (this binary contains the actual tests to be run) -- Unzip the archive to a directory under **/tmp/gcloud** -- Run the tests using the `ginkgo` command - - Starts etcd, kube-apiserver, kubelet - - The ginkgo command is used because this supports more features than running the test binary directly -- Output the remote test results to STDOUT -- `scp` the log files back to the local host under /tmp/_artifacts/e2e-node-containervm-v20160321-image -- Stop the processes on the remote host -- **Leave the GCE instance running** - -**Note: Subsequent tests run using the same image will *reuse the existing host* instead of deleting it and -provisioning a new one. To delete the GCE instance after each test see -*[DELETE_INSTANCE](#delete-instance-after-tests-run)*.** - - -# Additional Remote Options - -## Run tests using different images - -This is useful if you want to run tests against a host using a different OS distro or container runtime than -provided by the default image. - -List the available test images using gcloud. - -```sh -make test-e2e-node LIST_IMAGES=true -``` - -This will output a list of the available images for the default image project. - -Then run: - -```sh -make test-e2e-node REMOTE=true IMAGES="" -``` - -## Run tests against a running GCE instance (not an image) - -This is useful if you have an host instance running already and want to run the tests there instead of on a new instance. - -```sh -make test-e2e-node REMOTE=true HOSTS="" -``` - -## Delete instance after tests run - -This is useful if you want recreate the instance for each test run to trigger flakes related to starting the instance. - -```sh -make test-e2e-node REMOTE=true DELETE_INSTANCES=true -``` - -## Keep instance, test binaries, and *processes* around after tests run - -This is useful if you want to manually inspect or debug the kubelet process run as part of the tests. - -```sh -make test-e2e-node REMOTE=true CLEANUP=false -``` - -## Run tests using an image in another project - -This is useful if you want to create your own host image in another project and use it for testing. - -```sh -make test-e2e-node REMOTE=true IMAGE_PROJECT="" IMAGES="" -``` - -Setting up your own host image may require additional steps such as installing etcd or docker. See -[setup_host.sh](https://git.k8s.io/kubernetes/test/e2e_node/environment/setup_host.sh) for common steps to setup hosts to run node tests. - -## Create instances using a different instance name prefix - -This is useful if you want to create instances using a different name so that you can run multiple copies of the -test in parallel against different instances of the same image. - -```sh -make test-e2e-node REMOTE=true INSTANCE_PREFIX="my-prefix" -``` - -# Additional Test Options for both Remote and Local execution - -## Only run a subset of the tests - -To run tests matching a regex: - -```sh -make test-e2e-node REMOTE=true FOCUS="" -``` - -To run tests NOT matching a regex: - -```sh -make test-e2e-node REMOTE=true SKIP="" -``` - -## Run tests continually until they fail - -This is useful if you are trying to debug a flaky test failure. This will cause ginkgo to continually -run the tests until they fail. **Note: this will only perform test setup once (e.g. creating the instance) and is -less useful for catching flakes related creating the instance from an image.** - -```sh -make test-e2e-node REMOTE=true RUN_UNTIL_FAILURE=true -``` - -## Run tests in parallel - -Running test in parallel can usually shorten the test duration. By default node -e2e test runs with`--nodes=8` (see ginkgo flag -[--nodes](https://onsi.github.io/ginkgo/#parallel-specs)). You can use the -`PARALLELISM` option to change the parallelism. - -```sh -make test-e2e-node PARALLELISM=4 # run test with 4 parallel nodes -make test-e2e-node PARALLELISM=1 # run test sequentially -``` - -## Run tests with kubenet network plugin - -[kubenet](http://kubernetes.io/docs/admin/network-plugins/#kubenet) is -the default network plugin used by kubelet since Kubernetes 1.3. The -plugin requires [CNI](https://github.com/containernetworking/cni) and -[nsenter](http://man7.org/linux/man-pages/man1/nsenter.1.html). - -Currently, kubenet is enabled by default for Remote execution `REMOTE=true`, -but disabled for Local execution. **Note: kubenet is not supported for -local execution currently. This may cause network related test result to be -different for Local and Remote execution. So if you want to run network -related test, Remote execution is recommended.** - -To enable/disable kubenet: - -```sh -# enable kubenet -make test-e2e-node TEST_ARGS='--kubelet-flags="--network-plugin=kubenet --network-plugin-dir=/opt/cni/bin"' -# disable kubenet -make test-e2e-node TEST_ARGS='--kubelet-flags="--network-plugin= --network-plugin-dir="' -``` - -## Additional QoS Cgroups Hierarchy level testing - -For testing with the QoS Cgroup Hierarchy enabled, you can pass --cgroups-per-qos flag as an argument into Ginkgo using TEST_ARGS - -```sh -make test_e2e_node TEST_ARGS="--cgroups-per-qos=true" -``` - -# Notes on tests run by the Kubernetes project during pre-, post- submit. - -The node e2e tests are run by the PR builder for each Pull Request and the results published at -the bottom of the comments section. To re-run just the node e2e tests from the PR builder add the comment -`@k8s-bot node e2e test this issue: #` and **include a link to the test -failure logs if caused by a flake.** - -The PR builder runs tests against the images listed in [jenkins-pull.properties](https://git.k8s.io/kubernetes/test/e2e_node/jenkins/jenkins-pull.properties) - -The post submit tests run against the images listed in [jenkins-ci.properties](https://git.k8s.io/kubernetes/test/e2e_node/jenkins/jenkins-ci.properties) +This file has moved to https://git.k8s.io/community/contributors/devel/sig-node/e2e-node-tests.md. +This file is a placeholder to preserve links. Please remove by April 28, 2019 or the release of kubernetes 1.13, whichever comes first. \ No newline at end of file diff --git a/contributors/devel/sig-node/cri-testing-policy.md b/contributors/devel/sig-node/cri-testing-policy.md index 73b48c5ee..d03716778 100644 --- a/contributors/devel/sig-node/cri-testing-policy.md +++ b/contributors/devel/sig-node/cri-testing-policy.md @@ -25,7 +25,7 @@ Runtime maintainers are **required** to submit the tests listed below. Node E2E tests qualify an OS image with a pre-installed CRI runtime. The runtime maintainers are free to choose any OS distribution, packaging, and deployment mechanism. Please see the -[tutorial](https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-node-tests.md) +[tutorial](e2e-node-tests.md) to know more about the Node E2E test framework and tests for validating a compatible OS image. diff --git a/contributors/devel/sig-node/e2e-node-tests.md b/contributors/devel/sig-node/e2e-node-tests.md new file mode 100644 index 000000000..4f3327cba --- /dev/null +++ b/contributors/devel/sig-node/e2e-node-tests.md @@ -0,0 +1,229 @@ +# Node End-To-End tests + +Node e2e tests are component tests meant for testing the Kubelet code on a custom host environment. + +Tests can be run either locally or against a host running on GCE. + +Node e2e tests are run as both pre- and post- submit tests by the Kubernetes project. + +*Note: Linux only. Mac and Windows unsupported.* + +*Note: There is no scheduler running. The e2e tests have to do manual scheduling, e.g. by using `framework.PodClient`.* + +# Running tests + +## Locally + +Why run tests *Locally*? Much faster than running tests Remotely. + +Prerequisites: +- [Install etcd](https://github.com/coreos/etcd/releases) on your PATH + - Verify etcd is installed correctly by running `which etcd` + - Or make etcd binary available and executable at `/tmp/etcd` +- [Install ginkgo](https://github.com/onsi/ginkgo) on your PATH + - Verify ginkgo is installed correctly by running `which ginkgo` + +From the Kubernetes base directory, run: + +```sh +make test-e2e-node +``` + +This will: run the *ginkgo* binary against the subdirectory *test/e2e_node*, which will in turn: +- Ask for sudo access (needed for running some of the processes) +- Build the Kubernetes source code +- Pre-pull docker images used by the tests +- Start a local instance of *etcd* +- Start a local instance of *kube-apiserver* +- Start a local instance of *kubelet* +- Run the test using the locally started processes +- Output the test results to STDOUT +- Stop *kubelet*, *kube-apiserver*, and *etcd* + +## Remotely + +Why Run tests *Remotely*? Tests will be run in a customized pristine environment. Closely mimics what will be done +as pre- and post- submit testing performed by the project. + +Prerequisites: +- [join the googlegroup](https://groups.google.com/forum/#!forum/kubernetes-dev) +`kubernetes-dev@googlegroups.com` + - *This provides read access to the node test images.* +- Setup a [Google Cloud Platform](https://cloud.google.com/) account and project with Google Compute Engine enabled +- Install and setup the [gcloud sdk](https://cloud.google.com/sdk/downloads) + - Verify the sdk is setup correctly by running `gcloud compute instances list` and `gcloud compute images list --project kubernetes-node-e2e-images` + +Run: + +```sh +make test-e2e-node REMOTE=true +``` + +This will: +- Build the Kubernetes source code +- Create a new GCE instance using the default test image + - Instance will be called **test-e2e-node-containervm-v20160321-image** +- Lookup the instance public ip address +- Copy a compressed archive file to the host containing the following binaries: + - ginkgo + - kubelet + - kube-apiserver + - e2e_node.test (this binary contains the actual tests to be run) +- Unzip the archive to a directory under **/tmp/gcloud** +- Run the tests using the `ginkgo` command + - Starts etcd, kube-apiserver, kubelet + - The ginkgo command is used because this supports more features than running the test binary directly +- Output the remote test results to STDOUT +- `scp` the log files back to the local host under /tmp/_artifacts/e2e-node-containervm-v20160321-image +- Stop the processes on the remote host +- **Leave the GCE instance running** + +**Note: Subsequent tests run using the same image will *reuse the existing host* instead of deleting it and +provisioning a new one. To delete the GCE instance after each test see +*[DELETE_INSTANCE](#delete-instance-after-tests-run)*.** + + +# Additional Remote Options + +## Run tests using different images + +This is useful if you want to run tests against a host using a different OS distro or container runtime than +provided by the default image. + +List the available test images using gcloud. + +```sh +make test-e2e-node LIST_IMAGES=true +``` + +This will output a list of the available images for the default image project. + +Then run: + +```sh +make test-e2e-node REMOTE=true IMAGES="" +``` + +## Run tests against a running GCE instance (not an image) + +This is useful if you have an host instance running already and want to run the tests there instead of on a new instance. + +```sh +make test-e2e-node REMOTE=true HOSTS="" +``` + +## Delete instance after tests run + +This is useful if you want recreate the instance for each test run to trigger flakes related to starting the instance. + +```sh +make test-e2e-node REMOTE=true DELETE_INSTANCES=true +``` + +## Keep instance, test binaries, and *processes* around after tests run + +This is useful if you want to manually inspect or debug the kubelet process run as part of the tests. + +```sh +make test-e2e-node REMOTE=true CLEANUP=false +``` + +## Run tests using an image in another project + +This is useful if you want to create your own host image in another project and use it for testing. + +```sh +make test-e2e-node REMOTE=true IMAGE_PROJECT="" IMAGES="" +``` + +Setting up your own host image may require additional steps such as installing etcd or docker. See +[setup_host.sh](https://git.k8s.io/kubernetes/test/e2e_node/environment/setup_host.sh) for common steps to setup hosts to run node tests. + +## Create instances using a different instance name prefix + +This is useful if you want to create instances using a different name so that you can run multiple copies of the +test in parallel against different instances of the same image. + +```sh +make test-e2e-node REMOTE=true INSTANCE_PREFIX="my-prefix" +``` + +# Additional Test Options for both Remote and Local execution + +## Only run a subset of the tests + +To run tests matching a regex: + +```sh +make test-e2e-node REMOTE=true FOCUS="" +``` + +To run tests NOT matching a regex: + +```sh +make test-e2e-node REMOTE=true SKIP="" +``` + +## Run tests continually until they fail + +This is useful if you are trying to debug a flaky test failure. This will cause ginkgo to continually +run the tests until they fail. **Note: this will only perform test setup once (e.g. creating the instance) and is +less useful for catching flakes related creating the instance from an image.** + +```sh +make test-e2e-node REMOTE=true RUN_UNTIL_FAILURE=true +``` + +## Run tests in parallel + +Running test in parallel can usually shorten the test duration. By default node +e2e test runs with`--nodes=8` (see ginkgo flag +[--nodes](https://onsi.github.io/ginkgo/#parallel-specs)). You can use the +`PARALLELISM` option to change the parallelism. + +```sh +make test-e2e-node PARALLELISM=4 # run test with 4 parallel nodes +make test-e2e-node PARALLELISM=1 # run test sequentially +``` + +## Run tests with kubenet network plugin + +[kubenet](http://kubernetes.io/docs/admin/network-plugins/#kubenet) is +the default network plugin used by kubelet since Kubernetes 1.3. The +plugin requires [CNI](https://github.com/containernetworking/cni) and +[nsenter](http://man7.org/linux/man-pages/man1/nsenter.1.html). + +Currently, kubenet is enabled by default for Remote execution `REMOTE=true`, +but disabled for Local execution. **Note: kubenet is not supported for +local execution currently. This may cause network related test result to be +different for Local and Remote execution. So if you want to run network +related test, Remote execution is recommended.** + +To enable/disable kubenet: + +```sh +# enable kubenet +make test-e2e-node TEST_ARGS='--kubelet-flags="--network-plugin=kubenet --network-plugin-dir=/opt/cni/bin"' +# disable kubenet +make test-e2e-node TEST_ARGS='--kubelet-flags="--network-plugin= --network-plugin-dir="' +``` + +## Additional QoS Cgroups Hierarchy level testing + +For testing with the QoS Cgroup Hierarchy enabled, you can pass --cgroups-per-qos flag as an argument into Ginkgo using TEST_ARGS + +```sh +make test_e2e_node TEST_ARGS="--cgroups-per-qos=true" +``` + +# Notes on tests run by the Kubernetes project during pre-, post- submit. + +The node e2e tests are run by the PR builder for each Pull Request and the results published at +the bottom of the comments section. To re-run just the node e2e tests from the PR builder add the comment +`@k8s-bot node e2e test this issue: #` and **include a link to the test +failure logs if caused by a flake.** + +The PR builder runs tests against the images listed in [jenkins-pull.properties](https://git.k8s.io/kubernetes/test/e2e_node/jenkins/jenkins-pull.properties) + +The post submit tests run against the images listed in [jenkins-ci.properties](https://git.k8s.io/kubernetes/test/e2e_node/jenkins/jenkins-ci.properties) +