community/contributors/devel/sig-testing/verify-tests.md

Verification Tests

Table of Contents

• Verification Tests
  • Overview
  • Note
  • verify-govet-levee
  • verify-api-groups

Overview

Verification tests for Kubernetes provide a mechanism to verify contributions for adherence to project conventions and best practices, and to validate generated build artifacts for soundness.

All blocking verification tests can be executed via make verify. Individual verification tests also can be found in vestigial shell scripts at hack/verify-*.sh.

Most verification tests are self-explanatory. verify-govet, for instance, performs go vet checks, which defends against common mistakes. The verification tests fails when go vet produces any findings. More complex verification tests are described below.

Note

This documentation is a work in progress. This listing is incomplete.

verify-govet-levee

Verification in verify-govet-levee.sh uses taint propagation analysis to defend against accidental logging of credentials. Struct fields which may contain credentials should be annotated as such using the datapolicy field tag. Field tagging was introduced by KEP-1753, and analysis was introduced by KEP-1993. Additional credential sources may be identified in analysis configuration (see below).

Taint propagation analysis defends against both direct and indirect logging of credentials. Consider the following hypothetical snippet.

// kubernetes/cmd/kubelet/app/server.go

// kubeConfigSpec struct holds info required to build a KubeConfig object
type kubeConfigSpec struct {
	CACert         *x509.Certificate
	APIServer      string
	ClientName     string
	TokenAuth      *tokenAuth      `datapolicy:"token"`
	ClientCertAuth *clientCertAuth `datapolicy:"security-key"`
}

func MyDangerousFunction(spec kubeConfigSpec) error {
	if spec.CACert == nil {
		err := fmt.Errorf("kubeConfigSpec missing expected CACert, got %#v", spec)  // Dangerous logging!
		klog.Error(err)
		return err
	}
	
	if err := DoSomethingElse(spec); err != nil {
		klog.Error(err)  // Dangerous logging!
	}

	return nil
}

In the above, when spec.CACert == nil, we log the spec. However, we know from the datapolicy field tags that the spec could contain one or more credentials and should not be logged. The analysis will detect this and cause the verification test to fail. The log call should be adjusted to extract exactly what information is relevant from the spec.

The second klog.Error call is also problematic. The error returned by DoSomethingElse could potentially encapsulate the credential passed in by spec, and so we must not log it. That is, we consider err to be "tainted" by the call which has access to the credentials. The analysis will detect this as well and call the verification test to fail.

When this analysis causes the verification test to fail, a developer has several options. In order of decreasing preference:

• Reconstruct logging calls such that only non-secret information is passed.
• Reconstruct a method which caused taint to spread to return indicators which are not logged directly, e.g. return value, ok rather than value, err.
• Write a sanitizer whose return value is guaranteed to be log-safe. Add this sanitizer to the analysis configuration (see below).
• Add the method where the log call occurs to the analysis configuration exclude-list.

Analysis configuration can be found at kubernetes/kubernetes/hack/testdata/levee/levee-config.yaml. Contact SIG-Security with any additional questions.

verify-api-groups

This verification script validates the different api-groups by reading the respective register.go file. Every register file must contain a GroupName. Another check which is performed when this script runs is to ensure that all types have client code generated for them, except types that belong to groups not served from the API server (defined in this script via the bash array groups_without_codegen).

Next, the script compares the GroupNames against import_known_versions to ensure the import packages will get installed. We list out packages which are required without installation along with importing known_version. Then we do a search for packages that reqiure installation on the basis of packages_without_installation. We verify if file is a known_version_file or not only if an expected_install_package is present in it.

Finally the script checks that all external group versions (e.g. foobar/v1) are defined in hack/lib/init.sh in either the KUBE_AVAILABLE_GROUP_VERSIONS or KUBE_NONSERVER_GROUP_VERSIONS bash variables.

verify-bazel

This verify-bazel script validates the removal of bazel related files. The script ensures no bazel related temporary, intermediate or output files remain as part of KEP-2420.

verify-boilerplate

This script checks for the license headers for all the files, whether the header is correct or wrong. The purpose of boilerplate headers is to identify license headers. The script collects all the file names generated by hack/boilerplate/boilerplate.py script and stores them into a list called files_need_boilerplate.

Once we collect all the file names, we run a check to identify the files with wrong header. This check will only run if the file exists in the list mentioned above.

verify-cli-conventions

This script checks whether the description format of help message of kubectl command is valid or not. This check is done for all the kubectl sub-commands as well. The script first checks whether the go command is available or not in the ${PATH}. And then the binary cmd/clicheck is checked if it exists or not in the well-known output locations. It runs command checks on all the kubectl commands or subcommands like,

kubectl version
kubectl uncordon
kubectl wait
kubectl top node

And if the output looks good i.e. CLI follows all tested conventions the test passes.

verify-codegen

This script verifies if the code update is needed or not for specific sub-projects. It first verifies the correct Go version and creates Go path. The script checks for the updated code for below subprojects,

k8s.io/code-generator
k8s.io/kube-aggregator
k8s.io/sample-apiserver
k8s.io/sample-controller
k8s.io/apiextensions-apiserver
k8s.io/metrics

Once it completes checking for code updates, later the script calls update-codegen.sh scripts.

verify-structured-logging

This script verifies if a package is properly migrated to structured logging or not. The script involves verification steps based on new klog methods which have few disallowed keywords like,

* klog.Infof, klog.Info, klog.Infoln
* klog.InfoDepth
* klog.WarningDepth
* klog.Error, klog.Errorf, klog.Errorln
* klog.ErrorDepth

More info is available here.

verify-gofmt

This script is used to check whether the go source code needs to be formatted or not using, the gofmt tool. Gofmt tool automatically formats the code and the formatted code is easier to read, write and maintain.

verify-spelling

This script uses client9/misspell package to search and correct commonly misspelled words as per the English language in all the files and directories under kubernetes/kubernetes.