rancher
/
rke2
mirror of https://github.com/rancher/rke2.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rke2/developer-docs/testing.md

8.5 KiB
Raw Permalink Blame History

Testing Standards in RKE2

Go testing in RKE2 comes in 4 forms:

This document will explain when each test should be written and how each test should be generated, formatted, and run.

Note: all shell commands given are relative to the root RKE2 repo directory.

Unit Tests

Unit tests should be written when a component or function of a package needs testing. Unit tests should be used for "white box" testing.

Framework

All unit tests in RKE2 follow a Table Driven Test style. Specifically, RKE2 unit tests are automatically generated using the gotests tool. This is built into the Go vscode extension, has documented integrations for other popular editors, or can be run via command line. Additionally, a set of custom templates are provided to extend the generated test's functionality. To use these templates, call:

gotests --template_dir=<PATH_TO_RKE2>/contrib/gotests_templates

Or in vscode, edit the Go extension setting Go: Generate Tests Flags
and add --template_dir=<PATH_TO_RKE2>/contrib/gotests_templates as an item.

Format

All unit tests should be placed within the package of the file they test.
All unit test files should be named: <FILE_UNDER_TEST>_test.go. All unit test functions should be named: Test_Unit<FUNCTION_TO_TEST> or Test_Unit<RECEIVER>_<METHOD_TO_TEST>. See the service account unit test as an example.

Running

go test ./pkg/... -run Unit

Note: As unit tests call functions directly, they are the primary drivers of RKE2's code coverage metric.

Integration Tests

Integration tests should be used to test a specific functionality of RKE2 that exists across multiple Go packages, either via exported function calls, or more often, CLI commands. Integration tests should be used for "black box" testing.

Framework

All integration tests in RKE2 follow a Behavior Driven Development (BDD) style. Specifically, RKE2 uses Ginkgo and Gomega to drive the tests.
To generate an initial test, the command ginkgo bootstrap can be used.

To facilitate RKE2 CLI testing, see tests/util/cmd.go helper functions.

Format

All integration tests should be places in tests directory. All integration test files should be named: <TEST_NAME>_int_test.go All integration test functions should be named: Test_Integration<Test_Name>. See the etcd snapshot test as an example.

Running

Integration tests must be with an existing single-node cluster, tests will skip if the server is not configured correctly.

make dev-shell
# Once in the dev-shell
# Start rke2 server with appropriate flags
./bin/rke2 server

Open another terminal

make dev-shell-enter
# once in the dev-shell
go test ./tests/ -run Integration

Installer Tests

Installer tests should be run anytime one makes a modification to the installer script(s). These can be run locally with Vagrant:

When adding new installer test(s) please copy the prevalent style for the Vagrantfile. Ideally, the boxes used for additional assertions will support the default virtualbox provider which enables them to be used by our Github Actions Workflow(s). See install.yaml.

Framework

If you are new to Vagrant, Hashicorp has written some pretty decent introductory tutorials and docs, see:

Plugins and Providers

The libvirt and vmware_desktop providers cannot be used without first installing the relevant plugins which are vagrant-libvirt and vagrant-vmware-desktop, respectively. Much like the default virtualbox provider these will do nothing useful without also installing the relevant server runtimes and/or client programs.

Environment Variables

These can be set on the CLI or exported before invoking Vagrant:

  • TEST_INSTALL_SH (default ➡️ ../../../../install.sh) The location of the installer script to be uploaded to the guest by the Vagrant File Provisioner before being invoked at /home/vagrant/install.sh.
  • TEST_VM_CPUS (default ➡️ 2) The number of vCPU for the guest to use.
  • TEST_VM_MEMORY (default ➡️ 2048) The number of megabytes of memory for the guest to use.
  • TEST_VM_BOOT_TIMEOUT (default ➡️ 600) The time in seconds that Vagrant will wait for the machine to boot and be accessible.
  • INSTALL_PACKAGES A space-delimited string of package names (or locations accessible to the guest's native package manager) that will be installed prior to invoking the RKE2 install script. Useful for overriding the rke2-selinux RPM via URL pointing at a specific Github release artifact.
  • INSTALL_RKE2_*, RKE2_* All values (other than those mentioned below) are passed through to the guest for capture by the installer script:
    • INSTALL_RKE2_TYPE=server (hard-coded)
    • RKE2_KUBECONFIG_MODE=0644 (hard-coded)
    • RKE2_TOKEN=vagrant (hard-coded)

End-to-End (E2E) Tests

E2E tests cover multi-node RKE2 configuration and administration: bringup, update, teardown etc. across a wide range of operating systems. E2E tests are run nightly as part of RKE2 quality assurance (QA).

Framework

End-to-end tests utilize Ginkgo and Gomega like the integration tests, but rely on Vagrant to provide the underlying cluster configuration.

Currently tested operating systems are:

Format

All E2E tests should be placed under tests/e2e/<TEST_NAME>.
All E2E test functions should be named: Test_E2E<TEST_NAME>.
A E2E test consists of two parts:

  1. Vagrantfile: a vagrant file which describes and configures the VMs upon which the cluster and test will run
  2. <TEST_NAME>.go: A go test file which calls vagrant up and controls the actual testing of the cluster

See the validate cluster test as an example.

Running

Generally, E2E tests are run as a nightly Jenkins job for QA. They can still be run locally but additional setup is required. By default, all E2E tests are designed with libvirt as the underlying VM provider. Instructions for installing libvirt and its associated vagrant plugin, vagrant-libvirt can be found here. VirtualBox is also supported as a backup VM provider.

Once setup is complete, all E2E tests can be run with:

go test -timeout=15m ./tests/e2e/... -run E2E

Tests can be run individually with:

go test -timeout=15m ./tests/e2e/validatecluster/... -run E2E
#or
go test -timeout=15m ./tests/e2e/... -run E2EClusterValidation

Additionally, to generate junit reporting for the tests, the Ginkgo CLI is used. Installation instructions can be found here.

To run the all E2E tests and generate JUnit testing reports:

ginkgo --junit-report=result.xml ./tests/e2e/...

Note: The go test default timeout is 10 minutes, thus the -timeout flag should be used. The ginkgo default timeout is 1 hour, no timeout flag is needed.

Contributing New Or Updated Tests

If you wish to create a new test or update an existing test, please submit a PR with a title that includes the words <NAME_OF_TEST> (Created/Updated).