doc: describe how to run conformance tests with kind
Signed-off-by: knight42 <anonymousknight96@gmail.com>
This commit is contained in:
parent
e08b0e8cc7
commit
1cf544cb29
|
|
@ -68,21 +68,21 @@ Examples of features which are not currently eligible for conformance tests:
|
||||||
- anything that requires a non-default admission plugin
|
- anything that requires a non-default admission plugin
|
||||||
- features that are pending deprecation, eg: componentstatus
|
- features that are pending deprecation, eg: componentstatus
|
||||||
|
|
||||||
Conformance tests are intended to be stable and backwards compatible according to
|
Conformance tests are intended to be stable and backwards compatible according to
|
||||||
the standard API deprecation policies. Therefore any test that relies on specific
|
the standard API deprecation policies. Therefore any test that relies on specific
|
||||||
output that is not subject to the deprecation policy cannot be promoted to conformance.
|
output that is not subject to the deprecation policy cannot be promoted to conformance.
|
||||||
Examples of tests which are not eligible to conformance:
|
Examples of tests which are not eligible to conformance:
|
||||||
- anything that checks specific Events are generated, as we make no guarantees
|
- anything that checks specific Events are generated, as we make no guarantees
|
||||||
about the contents of events, nor their delivery
|
about the contents of events, nor their delivery
|
||||||
- If a test depends on events it is recommended to change the test to
|
- If a test depends on events it is recommended to change the test to
|
||||||
use an informer pattern and watch specific resource changes instead.
|
use an informer pattern and watch specific resource changes instead.
|
||||||
- An exception to this is tests that generates synthetic events themselves
|
- An exception to this is tests that generates synthetic events themselves
|
||||||
to verify that the API is capable of being exercised
|
to verify that the API is capable of being exercised
|
||||||
- anything that checks optional Condition fields, such as Reason or Message, as
|
- anything that checks optional Condition fields, such as Reason or Message, as
|
||||||
these may change over time (however it is reasonable to verify these fields
|
these may change over time (however it is reasonable to verify these fields
|
||||||
exist or are non-empty)
|
exist or are non-empty)
|
||||||
- If the test is checking for specific conditions or reasons, it is considered
|
- If the test is checking for specific conditions or reasons, it is considered
|
||||||
overly specific and it is recommended to simply look for pass/failure criteria
|
overly specific and it is recommended to simply look for pass/failure criteria
|
||||||
where possible, and output the condition/reason for debugging purposes only.
|
where possible, and output the condition/reason for debugging purposes only.
|
||||||
|
|
||||||
Examples of areas we may want to relax these requirements once we have a
|
Examples of areas we may want to relax these requirements once we have a
|
||||||
|
|
@ -105,7 +105,7 @@ Generally speaking, the goals are to:
|
||||||
|
|
||||||
- Make sure tests that are already passing remain passing. If new OS-specific
|
- Make sure tests that are already passing remain passing. If new OS-specific
|
||||||
functionality is added, it should be in a new test.
|
functionality is added, it should be in a new test.
|
||||||
- Ensure that new tests covering Linux-specific functionality are tagged with `[LinuxOnly]`
|
- Ensure that new tests covering Linux-specific functionality are tagged with `[LinuxOnly]`
|
||||||
(see: [Kinds of Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md#kinds-of-tests),
|
(see: [Kinds of Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md#kinds-of-tests),
|
||||||
- Give future reviewers a reference to an active issue or documentation clarifying why a test
|
- Give future reviewers a reference to an active issue or documentation clarifying why a test
|
||||||
cannot run on Windows.
|
cannot run on Windows.
|
||||||
|
|
@ -116,7 +116,7 @@ The tests that are running today:
|
||||||
including Windows versions, or have been ported by SIG-Windows
|
including Windows versions, or have been ported by SIG-Windows
|
||||||
(see [kubernetes-sigs/windows-testing/images](https://github.com/kubernetes-sigs/windows-testing/tree/master/images)
|
(see [kubernetes-sigs/windows-testing/images](https://github.com/kubernetes-sigs/windows-testing/tree/master/images)
|
||||||
- Do not depend on any functionality that is different or not available on Windows. The full list
|
- Do not depend on any functionality that is different or not available on Windows. The full list
|
||||||
is available in the Windows Kubernetes docs under [api](https://kubernetes.io/docs/setup/windows/intro-windows-in-kubernetes/#api).
|
is available in the Windows Kubernetes docs under [api](https://kubernetes.io/docs/setup/windows/intro-windows-in-kubernetes/#api).
|
||||||
A brief summary is included here as a starting point. If the docs are insufficient
|
A brief summary is included here as a starting point. If the docs are insufficient
|
||||||
or there are more questions, please contact #SIG-Windows on Slack to get another
|
or there are more questions, please contact #SIG-Windows on Slack to get another
|
||||||
reviewer.
|
reviewer.
|
||||||
|
|
@ -139,13 +139,13 @@ implicitly by Docker or ContainerD, not by the kubelet. Do not check properties
|
||||||
- Networking
|
- Networking
|
||||||
- Pods set `HostNetwork=true`. Is not supported on Windows, and the Pod will not start.
|
- Pods set `HostNetwork=true`. Is not supported on Windows, and the Pod will not start.
|
||||||
- Network and DNS settings must be passed through CNI. Windows does not use `/etc/resolv.conf`, so tests should not rely on reading that file to check DNS settings.
|
- Network and DNS settings must be passed through CNI. Windows does not use `/etc/resolv.conf`, so tests should not rely on reading that file to check DNS settings.
|
||||||
- If you to check network settings such as dns search lists, please use [agnhost](https://github.com/kubernetes/kubernetes/tree/master/test/images/agnhost) to output needed data from the container.
|
- If you to check network settings such as dns search lists, please use [agnhost](https://github.com/kubernetes/kubernetes/tree/master/test/images/agnhost) to output needed data from the container.
|
||||||
- Windows treats all DNS lookups with a `.` to be FQDN, not PQDN. For example `kubernetes` will resolve as a PQDN,
|
- Windows treats all DNS lookups with a `.` to be FQDN, not PQDN. For example `kubernetes` will resolve as a PQDN,
|
||||||
but `kubernetes.default` will be resolved as a FQDN and fail.
|
but `kubernetes.default` will be resolved as a FQDN and fail.
|
||||||
- ICMP only works between pods on the same network, and are not routable to external networks. TCP/UDP are routable.
|
- ICMP only works between pods on the same network, and are not routable to external networks. TCP/UDP are routable.
|
||||||
- Windows containers do not support IPv6.
|
- Windows containers do not support IPv6.
|
||||||
|
|
||||||
The existing tests which are affected by one of those criteria are tagged with `[LinuxOnly]`
|
The existing tests which are affected by one of those criteria are tagged with `[LinuxOnly]`
|
||||||
(see: [Kinds of Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md#kinds-of-tests).
|
(see: [Kinds of Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md#kinds-of-tests).
|
||||||
|
|
||||||
## Conformance Test Version Skew Policy
|
## Conformance Test Version Skew Policy
|
||||||
|
|
@ -169,7 +169,7 @@ with the following versions must pass conformance tests built from the
|
||||||
following branches:
|
following branches:
|
||||||
|
|
||||||
| cluster version | master | release-1.3 | release-1.2 | release-1.1 |
|
| cluster version | master | release-1.3 | release-1.2 | release-1.1 |
|
||||||
| --------------- | ----- | ----------- | ----------- | ----------- |
|
| --------------- | ------ | ----------- | ----------- | ----------- |
|
||||||
| v1.3.0-alpha | yes | yes | yes | no |
|
| v1.3.0-alpha | yes | yes | yes | no |
|
||||||
| v1.2.x | no | no | yes | yes |
|
| v1.2.x | no | no | yes | yes |
|
||||||
| v1.1.x | no | no | no | yes |
|
| v1.1.x | no | no | no | yes |
|
||||||
|
|
@ -181,6 +181,69 @@ configured. Conformance tests must be able to be run against clusters that have
|
||||||
not been created with `test-infra/kubetest`, just provide a kubeconfig with the
|
not been created with `test-infra/kubetest`, just provide a kubeconfig with the
|
||||||
appropriate endpoint and credentials.
|
appropriate endpoint and credentials.
|
||||||
|
|
||||||
|
### Running Conformance Tests With [KinD](https://kind.sigs.k8s.io/)
|
||||||
|
|
||||||
|
1. Work in your kubernetes branch, preferably in the default go src location: `$GOPATH/src/k8s.io/kubernetes`
|
||||||
|
|
||||||
|
2. Create your kind node image:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Without bazel
|
||||||
|
kind build node-image
|
||||||
|
|
||||||
|
# Or with bazel
|
||||||
|
kind build node-image --type bazel
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Create your kind e2e cluster config kind-config-yaml:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# necessary for conformance
|
||||||
|
kind: Cluster
|
||||||
|
apiVersion: kind.x-k8s.io/v1alpha4
|
||||||
|
networking:
|
||||||
|
ipFamily: ipv4
|
||||||
|
nodes:
|
||||||
|
# the control plane node
|
||||||
|
- role: control-plane
|
||||||
|
- role: worker
|
||||||
|
- role: worker
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Set your KUBECONFIG env variable (KIND generates the conf based on it):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
export KUBECONFIG="${HOME}/.kube/kind-test-config"
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Use the previous config to create your cluster:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
kind create cluster --config kind-config.yaml --image kindest/node:latest -v4
|
||||||
|
```
|
||||||
|
|
||||||
|
6. Create your e2e Kubernetes binary (from your Kubernetes src code):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Without bazel
|
||||||
|
make WHAT="test/e2e/e2e.test"
|
||||||
|
|
||||||
|
# Or with bazel
|
||||||
|
bazel build //test/e2e:e2e.test
|
||||||
|
```
|
||||||
|
|
||||||
|
7. Execute your tests:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Without bazel
|
||||||
|
./_output/bin/e2e.test -context kind-kind -ginkgo.focus="\[sig-network\].*Conformance" -num-nodes 2
|
||||||
|
|
||||||
|
# Or with bazel
|
||||||
|
./bazel-bin/test/e2e/e2e.test -context kind-kind -ginkgo.focus="\[sig-network\].*Conformance" -num-nodes 2
|
||||||
|
```
|
||||||
|
|
||||||
|
### Running Conformance Tests With kubetest
|
||||||
|
|
||||||
These commands are intended to be run within a kubernetes directory, either
|
These commands are intended to be run within a kubernetes directory, either
|
||||||
cloned from source, or extracted from release artifacts such as
|
cloned from source, or extracted from release artifacts such as
|
||||||
`kubernetes.tar.gz`. They assume you have a valid golang installation.
|
`kubernetes.tar.gz`. They assume you have a valid golang installation.
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue