kubernetes
/
community
mirror of https://github.com/kubernetes/community.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #4107 from ii/writing-good-conformance-tests 

New document "Writing Good Conformance Tests"
This commit is contained in:
Kubernetes Prow Robot 2019-09-27 10:17:35 -07:00 committed by GitHub
parent 0e96424dd9 7d1a5d22c5
commit 4414db8f8a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 53 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
contributors/devel/sig-testing/writing-good-conformance-tests.md Normal file
View File

@ -0,0 +1,53 @@
# Writing Good Conformance Tests for Kubernetes #
The overarching goal of conformance tests is to exercise core
Kubernetes functionality in (as much as possible) a provider-agnostic
way.
Conformance tests are promoted from existing e2e tests which have
been proven to be stable and "non-flaky".
Please read the the
["Writing good e2e
tests"](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/writing-good-e2e-tests.md)
guide as a supplement to this document.
### Requirements ###
In addition to the "Writing good e2e tests" guide, there are specific
requirements that all Conformance tests must adhere to. You can find
the definitive list of requirements in the [SIG-Arch community
docs](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md#conformance-test-requirements).
Generally speaking, tests should not target internal behaviors, but
instead target exposed system behaviors.
### Existing Tests ###
When looking for an existing e2e test which exercises a behaviour
which is useful for Conformance, you may find that the test does not
meet all of the [requirements](#Requirements). In these cases, you
might either modify the test to meet requirements, but sometimes doing
so will mean losing some of the intent of the test.
Instead, a better approach may be to duplicate the test, and make your
modifications, thus preserving the original test as-is.
Any existing e2e test should have gone through a standard review
process by a respective [SIG](https://github.com/ii/k8s-community/tree/writing-good-conformance-tests) in order to
have been merged in, which means that the test was most likely
reviewed by domain-specific experts. This can be useful to note when
modifying tests to better suit conformance concerns, or meet
[requirements](#Requirements).
### New Tests ###
New tests must go through the standard process for e2e tests even when
intended for Conformance from the onset. This means working with
SIG-specific reviewers in order to get your PR accepted. After which,
the test must be in the test cycle for 2 weeks, to prove its
stability. If the test is not slow or flaky, then you may begin the
promotion process by following the steps detailed in ["Promoting
Tests to
Conformance"](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md#promoting-tests-to-conformance)