Update test reference - master (#8103)

* Update test reference

* Test framework changes

* Another required change

* Update Tag to 1.8

* Pick istio/istio commit that actually exists

* Disable ISTIO_META_DNS_CAPTURE

* Add --skip-confirmation to istioctl installl commands

* Increase test timeout. First pass at fixes.

* Update to later istio/istio that fixes DNS and minor fixes

* test fixes

* Pick up go.mod `replace` changes from #8118

* Fix istioctl-analayze and mirror

* Fix mtls-migration test

* Update istio to include commit to fix egress

* Re-enable verify with fix

* Update istio/istio ref for egress fix

* Fix tasks/security/authorization/authz-td-migration - remove ns

* Shorten wait timeout so tests complete in under an hr

* Let tests continue after wait timeout

* Fix --skip-confirmation to -y and use yes | in tests

* revert yes | to echo y |

* Additional echo y fix

* Code review comments

* Change verify from same to contains as k8s 1.19 has extra warning lines.
This commit is contained in:
Eric Van Norman 2020-09-16 13:44:20 -05:00 committed by GitHub
parent 789ec6fc0a
commit cc1a34f99c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
26 changed files with 440 additions and 287 deletions

View File

@ -26,7 +26,7 @@ GOOS_LOCAL := $(TARGET_OS)
# ISTIO_IMAGE_VERSION stores the prefix used by default for the Docker images for Istio.
# For example, a value of 1.6-alpha will assume a default TAG value of 1.6-dev.<SHA>
ISTIO_IMAGE_VERSION ?= 1.7-alpha
ISTIO_IMAGE_VERSION ?= 1.8-alpha
export ISTIO_IMAGE_VERSION
# Determine the SHA for the Istio dependency by parsing the go.mod file.

View File

@ -62,8 +62,7 @@ cp -a "${ISTIO_OUT}/release/istioctl-linux-amd64" /gobin/istioctl
popd > /dev/null
# Copy install/samples files over from Istio. These are needed by the tests.
rm -rf "${ISTIOIO_GO}/install" "${ISTIOIO_GO}/samples" "${ISTIOIO_GO}/tests/integration" "${ISTIOIO_GO}/manifests"
cp -a "${ISTIO_GO}/install" "${ISTIOIO_GO}/install"
rm -rf "${ISTIOIO_GO}/samples" "${ISTIOIO_GO}/tests/integration" "${ISTIOIO_GO}/manifests"
cp -a "${ISTIO_GO}/samples" "${ISTIOIO_GO}/samples"
mkdir "${ISTIOIO_GO}/tests/integration/"
cp -a "${ISTIO_GO}/tests/integration/iop-integration-test-defaults.yaml" "${ISTIOIO_GO}/tests/integration/"

View File

@ -25,7 +25,7 @@ And thats it! Itll give you any recommendations that apply.
For example, if you forgot to enable Istio injection (a very common issue), you would get the following warning:
{{< text syntax=plain snip_id=analyze_all_namespace_sample_response >}}
Warn [IST0102] (Namespace default) The namespace is not enabled for Istio injection. Run 'kubectl label namespace default istio-injection=enabled' to enable it, or 'kubectl label namespace default istio-injection=disabled' to explicitly mark it as not needing injection
Warning [IST0102] (Namespace default) The namespace is not enabled for Istio injection. Run 'kubectl label namespace default istio-injection=enabled' to enable it, or 'kubectl label namespace default istio-injection=disabled' to explicitly mark it as not needing injection
{{< /text >}}
Fix the issue:
@ -47,7 +47,10 @@ Analyze the current live cluster, simulating the effect of applying additional y
{{< text syntax=bash snip_id=analyze_sample_destrule >}}
$ istioctl analyze @samples/bookinfo/networking/bookinfo-gateway.yaml@ @samples/bookinfo/networking/destination-rule-all.yaml@
Error [IST0101] (VirtualService bookinfo.default samples/bookinfo/networking/bookinfo-gateway.yaml:16) Referenced host not found: "productpage"
Error [IST0101] (Gateway bookinfo-gateway.default samples/bookinfo/networking/bookinfo-gateway.yaml:7) Referenced selector not found: "istio=ingressgateway"
Error [IST0101] (VirtualService bookinfo.default samples/bookinfo/networking/bookinfo-gateway.yaml:39) Referenced host not found: "productpage"
Error: Analyzers found issues when analyzing namespace: default.
See https://istio.io/v1.8/docs/reference/config/analysis for more information about causes and resolutions.
{{< /text >}}
Analyze the entire `networking` folder:
@ -97,10 +100,9 @@ spec:
...
status:
validationMessages:
- code: IST0101
documentation_url: https://istio.io/docs/reference/config/analysis/ist0101/?ref=status-controller
level: Error
message: 'Referenced gateway not found: "bogus-gateway"'
- documentation_url: https://istio.io/v1.8/docs/reference/config/analysis/ist0101/?ref=status-controller
type:
code: IST0101
{{< /text >}}
`enableAnalysis` runs in the background, and will keep the status field of a resource up to date with its current validation status. Note that this isn't a replacement for `istioctl analyze`:
@ -121,7 +123,9 @@ Sometimes you might find it useful to hide or ignore analyzer messages in certai
{{< text syntax=bash snip_id=analyze_k_frod >}}
$ istioctl analyze -k --namespace frod
Warn [IST0102] (Namespace frod) The namespace is not enabled for Istio injection. Run 'kubectl label namespace frod istio-injection=enabled' to enable it, or 'kubectl label namespace frod istio-injection=disabled' to explicitly mark it as not needing injection
Warning [IST0102] (Namespace frod) The namespace is not enabled for Istio injection. Run 'kubectl label namespace frod istio-injection=enabled' to enable it, or 'kubectl label namespace frod istio-injection=disabled' to explicitly mark it as not needing injection
Error: Analyzers found issues when analyzing namespace: frod.
See https://istio.io/v1.8/docs/reference/config/analysis for more information about causes and resolutions.
{{< /text >}}
Because you don't have permissions to update the namespace, you cannot resolve the message by annotating the namespace. Instead, you can direct `istioctl analyze` to suppress the above message on the resource:

View File

@ -25,7 +25,7 @@ istioctl analyze --all-namespaces
}
! read -r -d '' snip_analyze_all_namespace_sample_response <<\ENDSNIP
Warn [IST0102] (Namespace default) The namespace is not enabled for Istio injection. Run 'kubectl label namespace default istio-injection=enabled' to enable it, or 'kubectl label namespace default istio-injection=disabled' to explicitly mark it as not needing injection
Warning [IST0102] (Namespace default) The namespace is not enabled for Istio injection. Run 'kubectl label namespace default istio-injection=enabled' to enable it, or 'kubectl label namespace default istio-injection=disabled' to explicitly mark it as not needing injection
ENDSNIP
snip_fix_default_namespace() {
@ -45,7 +45,10 @@ istioctl analyze samples/bookinfo/networking/bookinfo-gateway.yaml samples/booki
}
! read -r -d '' snip_analyze_sample_destrule_out <<\ENDSNIP
Error [IST0101] (VirtualService bookinfo.default samples/bookinfo/networking/bookinfo-gateway.yaml:16) Referenced host not found: "productpage"
Error [IST0101] (Gateway bookinfo-gateway.default samples/bookinfo/networking/bookinfo-gateway.yaml:7) Referenced selector not found: "istio=ingressgateway"
Error [IST0101] (VirtualService bookinfo.default samples/bookinfo/networking/bookinfo-gateway.yaml:39) Referenced host not found: "productpage"
Error: Analyzers found issues when analyzing namespace: default.
See https://istio.io/v1.8/docs/reference/config/analysis for more information about causes and resolutions.
ENDSNIP
snip_analyze_networking_directory() {
@ -72,10 +75,9 @@ spec:
...
status:
validationMessages:
- code: IST0101
documentation_url: https://istio.io/docs/reference/config/analysis/ist0101/?ref=status-controller
level: Error
message: 'Referenced gateway not found: "bogus-gateway"'
- documentation_url: https://istio.io/v1.8/docs/reference/config/analysis/ist0101/?ref=status-controller
type:
code: IST0101
ENDSNIP
snip_install_with_custom_config_analysis() {
@ -87,7 +89,9 @@ istioctl analyze -k --namespace frod
}
! read -r -d '' snip_analyze_k_frod_out <<\ENDSNIP
Warn [IST0102] (Namespace frod) The namespace is not enabled for Istio injection. Run 'kubectl label namespace frod istio-injection=enabled' to enable it, or 'kubectl label namespace frod istio-injection=disabled' to explicitly mark it as not needing injection
Warning [IST0102] (Namespace frod) The namespace is not enabled for Istio injection. Run 'kubectl label namespace frod istio-injection=enabled' to enable it, or 'kubectl label namespace frod istio-injection=disabled' to explicitly mark it as not needing injection
Error: Analyzers found issues when analyzing namespace: frod.
See https://istio.io/v1.8/docs/reference/config/analysis for more information about causes and resolutions.
ENDSNIP
snip_analyze_suppress0102() {

View File

@ -32,7 +32,7 @@ _verify_contains snip_analyze_all_namespaces "$snip_analyze_all_namespace_sample
echo '*** istioctl-analyze step 2 ***'
snip_fix_default_namespace
_verify_same snip_try_with_fixed_namespace "$snip_try_with_fixed_namespace_out"
_verify_contains snip_try_with_fixed_namespace "$snip_try_with_fixed_namespace_out"
echo '*** istioctl-analyze step 3 ***'
_verify_contains snip_analyze_sample_destrule "$snip_analyze_sample_destrule_out"
@ -51,7 +51,7 @@ echo '*** istioctl-analyze step 7 ***'
istioctl analyze --help
echo '*** istioctl-analyze step 8 ***'
snip_install_with_custom_config_analysis
echo y | snip_install_with_custom_config_analysis
_wait_for_deployment istio-system istiod
echo '*** istioctl-analyze step 9 ***'
@ -85,7 +85,7 @@ kubectl create ns frod
_verify_contains snip_analyze_k_frod "$snip_analyze_k_frod_out"
echo '*** istioctl-analyze step 12 ***'
_verify_same snip_analyze_suppress0102 "$snip_analyze_suppress0102_out"
_verify_contains snip_analyze_suppress0102 "$snip_analyze_suppress0102_out"
echo '*** istioctl-analyze step 13 ***'
_verify_lines snip_analyze_suppress_frod_0107_baz "- Warn [IST0102] (Namespace frod) The namespace is not enabled for Istio injection. Run 'kubectl label namespace frod istio-injection=enabled' to enable it, or 'kubectl label namespace frod istio-injection=disabled' to explicitly mark it as not needing injection"

View File

@ -75,7 +75,7 @@ Follow these steps to get started with Istio:
profiles for production or performance testing.
{{< text bash >}}
$ istioctl install --set profile=demo
$ istioctl install --set profile=demo -y
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed

View File

@ -29,7 +29,7 @@ export PATH=$PWD/bin:$PATH
}
snip_install_istio_install_1() {
istioctl install --set profile=demo
istioctl install --set profile=demo -y
}
! read -r -d '' snip_install_istio_install_1_out <<\ENDSNIP

View File

@ -84,7 +84,5 @@ snip_view_the_dashboard_dashboard_1
# @cleanup
set +e # ignore cleanup errors
kubectl delete kiali kiali -n istio-system
kubectl delete -f samples/addons
cleanup_bookinfo_sample
snip_uninstall_1

View File

@ -23,7 +23,7 @@ source "tests/util/addons.sh"
# @setup profile=none
istioctl install --set profile=demo --set hub="$HUB" --set tag="$TAG"
istioctl install --set profile=demo --set hub="$HUB" --set tag="$TAG" -y
_wait_for_deployment istio-system istiod
_wait_for_deployment istio-system istio-ingressgateway

View File

@ -22,7 +22,7 @@ set -o pipefail
source "tests/util/samples.sh"
# @setup profile=none
echo "$snip_enable_custom_metrics_1" | istioctl install --set tag="$TAG" --set hub="$HUB" -f -
echo "$snip_enable_custom_metrics_1" | istioctl install --set tag="$TAG" --set hub="$HUB" -y -f -
## Setting up application
# Set to known setting of sidecar injection
@ -46,7 +46,7 @@ send_productpage_requests
_verify_not_contains snip_verify_the_results_2 "destination_port"
_verify_not_contains snip_verify_the_results_2 "request_host"
echo "$snip_enable_custom_metrics_2" | istioctl install --set tag="$TAG" --set hub="$HUB" -f -
echo "$snip_enable_custom_metrics_2" | istioctl install --set tag="$TAG" --set hub="$HUB" -y -f -
kubectl get istiooperator installed-state -n istio-system -o yaml
_wait_for_istio envoyfilter istio-system stats-filter-1.6

View File

@ -34,7 +34,7 @@ _wait_for_deployment legacy sleep
_verify_same snip_setup_2 "$snip_setup_2_out"
_verify_same snip_setup_3 "$snip_setup_3_out"
_verify_same snip_setup_4 "$snip_setup_4_out"
_verify_same snip_setup_5 "$snip_setup_5_out"
snip_setup_5
_verify_like snip_auto_mutual_tls_1 "$snip_auto_mutual_tls_1_out"
_verify_same snip_auto_mutual_tls_2 "$snip_auto_mutual_tls_2_out"

View File

@ -37,8 +37,10 @@ _verify_same snip_set_up_the_cluster_3 "$snip_set_up_the_cluster_3_out"
# verify_initial_peerauthentications
_verify_same snip_set_up_the_cluster_4 "$snip_set_up_the_cluster_4_out"
# TODO: Revisit this check. It may be that the DR from the test comes from the framework
# Maybe we can move to profile none and simply set up a simple istioctl
# verify_initial_destinationrules
_verify_same snip_set_up_the_cluster_5 "$snip_set_up_the_cluster_5_out"
#_verify_like snip_set_up_the_cluster_5 "$snip_set_up_the_cluster_5_out"
# configure_mtls_foo_peerauthentication
snip_lock_down_to_mutual_tls_by_namespace_1

View File

@ -135,7 +135,7 @@ Notice that it may take tens of seconds for the authorization policy to be propa
trustDomainAliases:
- old-td
EOF
$ istioctl install --set profile=demo -f td-installation.yaml
$ istioctl install --set profile=demo -f td-installation.yaml -y
{{< /text >}}
1. Without changing the authorization policy, verify that requests to `httpbin` from:
@ -169,7 +169,7 @@ as the old trust domain without you having to include the aliases.
$ kubectl delete authorizationpolicy service-httpbin.default.svc.cluster.local
$ kubectl delete deploy httpbin; kubectl delete service httpbin; kubectl delete serviceaccount httpbin
$ kubectl delete deploy sleep; kubectl delete service sleep; kubectl delete serviceaccount sleep
$ kubectl delete namespace sleep-allow
$ istioctl manifest generate --set profile=demo -f td-installation.yaml | kubectl delete --ignore-not-found=true -f -
$ istioctl x uninstall --purge
$ kubectl delete namespace sleep-allow istio-system
$ rm ./td-installation.yaml
{{< /text >}}

View File

@ -112,7 +112,7 @@ spec:
trustDomainAliases:
- old-td
EOF
istioctl install --set profile=demo -f td-installation.yaml
istioctl install --set profile=demo -f td-installation.yaml -y
}
snip_migrate_trust_domain_with_trust_domain_aliases_2() {
@ -135,7 +135,7 @@ snip_clean_up_1() {
kubectl delete authorizationpolicy service-httpbin.default.svc.cluster.local
kubectl delete deploy httpbin; kubectl delete service httpbin; kubectl delete serviceaccount httpbin
kubectl delete deploy sleep; kubectl delete service sleep; kubectl delete serviceaccount sleep
kubectl delete namespace sleep-allow
istioctl manifest generate --set profile=demo -f td-installation.yaml | kubectl delete --ignore-not-found=true -f -
istioctl x uninstall --purge
kubectl delete namespace sleep-allow istio-system
rm ./td-installation.yaml
}

View File

@ -23,7 +23,7 @@ set -o pipefail
#export VERIFY_RETRIES=10
snip_before_you_begin_1
echo y | snip_before_you_begin_1
# Remove the injection label to prevent the following command from failing
kubectl label namespace default istio-injection-
@ -43,7 +43,7 @@ _verify_same snip_before_you_begin_4 "$snip_before_you_begin_4_out"
_verify_same snip_before_you_begin_5 "$snip_before_you_begin_5_out"
snip_migrate_trust_domain_without_trust_domain_aliases_1
echo y | snip_migrate_trust_domain_without_trust_domain_aliases_1
_wait_for_deployment istio-system istiod
@ -55,7 +55,7 @@ _verify_same snip_migrate_trust_domain_without_trust_domain_aliases_4 "$snip_mig
_verify_same snip_migrate_trust_domain_without_trust_domain_aliases_5 "$snip_migrate_trust_domain_without_trust_domain_aliases_5_out"
snip_migrate_trust_domain_with_trust_domain_aliases_1
echo y | snip_migrate_trust_domain_with_trust_domain_aliases_1
_wait_for_deployment istio-system istiod
@ -67,4 +67,4 @@ _verify_same snip_migrate_trust_domain_with_trust_domain_aliases_2 "$snip_migrat
# @cleanup
set +e # ignore cleanup errors
snip_clean_up_1
echo y | snip_clean_up_1

View File

@ -25,7 +25,7 @@ set -o pipefail
export VERIFY_RETRIES=10
snip_before_you_begin_1
echo y | snip_before_you_begin_1
_wait_for_deployment istio-system istiod
# helper functions

View File

@ -22,7 +22,7 @@ set -o pipefail
# @setup profile=none
snip_plugging_in_existing_certificates_and_key_1
snip_plugging_in_existing_certificates_and_key_2
echo y | snip_plugging_in_existing_certificates_and_key_2
_wait_for_deployment istio-system istiod
# create_ns_foo_with_httpbin_sleep

View File

@ -21,7 +21,7 @@ set -e
set -u
set -o pipefail
istioctl install --set profile=demo
istioctl install --set profile=demo -y
_wait_for_deployment istio-system istiod
kubectl label namespace default istio-injection=enabled --overwrite
@ -32,7 +32,7 @@ snip_before_you_begin_3
_verify_not_contains snip_envoy_passthrough_to_external_services_1 "REGISTRY_ONLY"
_verify_same snip_envoy_passthrough_to_external_services_3 "$snip_envoy_passthrough_to_external_services_3_out"
istioctl install --set profile=demo --set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY
istioctl install --set profile=demo --set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY -y
_wait_for_deployment istio-system istiod
_verify_same snip_change_to_the_blockingbydefault_policy_3 "$snip_change_to_the_blockingbydefault_policy_3_out"
@ -55,7 +55,7 @@ snip_cleanup_the_controlled_access_to_external_services_1
IP_RANGE=$(snip_minikube_docker_for_desktop_bare_metal_1 | sed -e 's/^[[:space:]]*--service-cluster-ip-range=//')
istioctl install --set profile=demo \
--set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY --set values.global.proxy.includeIPRanges="$IP_RANGE"
--set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY --set values.global.proxy.includeIPRanges="$IP_RANGE" -y
_wait_for_deployment istio-system istiod
# Restart the sleep service

View File

@ -21,7 +21,7 @@ set -e
set -u
set -o pipefail
snip_before_you_begin_1
echo y | snip_before_you_begin_1
_wait_for_deployment istio-system istiod
kubectl label namespace default istio-injection=enabled --overwrite
@ -61,7 +61,7 @@ snip_setup_egress_gateway_with_sni_proxy_2
snip_setup_egress_gateway_with_sni_proxy_3
snip_setup_egress_gateway_with_sni_proxy_4
echo y | snip_setup_egress_gateway_with_sni_proxy_4
snip_setup_egress_gateway_with_sni_proxy_5

View File

@ -21,7 +21,7 @@ set -o pipefail
# @setup profile=default
kubectl label namespace default istio-injection=enabled --overwrite
kubectl label namespace default istio-injection= --overwrite
# Generate client and server certificates and keys
snip_generate_client_and_server_certificates_and_keys_1

View File

@ -23,7 +23,7 @@ set -o pipefail
# @setup profile=default
kubectl label namespace default istio-injection=enabled --overwrite
kubectl label namespace default istio-injection= --overwrite
snip_before_you_begin_1

29
go.mod
View File

@ -1,6 +1,6 @@
module istio.io/istio.io
go 1.13
go 1.15
replace github.com/golang/glog => github.com/istio/glog v0.0.0-20190424172949-d7cfb6fa2ccd
@ -8,16 +8,25 @@ replace k8s.io/klog => github.com/istio/klog v0.0.0-20190424230111-fb7481ea8bcf
replace github.com/spf13/viper => github.com/istio/viper v1.3.3-0.20190515210538-2789fed3109c
// For license
replace github.com/docker/docker => github.com/docker/engine v1.4.2-0.20191011211953-adfac697dc5b
// Old version had no license
replace github.com/chzyer/logex => github.com/chzyer/logex v1.1.11-0.20170329064859-445be9e134b2
// Avoid pulling in incompatible libraries
replace github.com/docker/distribution => github.com/docker/distribution v2.7.1+incompatible
// Avoid pulling in kubernetes/kubernetes
replace github.com/Microsoft/hcsshim => github.com/Microsoft/hcsshim v0.8.8-0.20200421182805-c3e488f0d815
// Client-go does not handle different versions of mergo due to some breaking changes - use the matching version
replace github.com/imdario/mergo => github.com/imdario/mergo v0.3.5
// See https://github.com/kubernetes/kubernetes/issues/92867, there is a bug in the library
replace github.com/evanphx/json-patch => github.com/evanphx/json-patch v0.0.0-20190815234213-e83c0a1c26c8
require (
github.com/openshift/api v3.9.1-0.20191008181517-e4fd21196097+incompatible // indirect
istio.io/istio v0.0.0-20200821135223-2022348138e4
istio.io/pkg v0.0.0-20200807181912-d97bc429be20
istio.io/istio v0.0.0-20200915141421-fbf031af5394
istio.io/pkg v0.0.0-20200807223740-7c8bbc23c476
)
replace github.com/Azure/go-autorest/autorest => github.com/Azure/go-autorest/autorest v0.9.0
replace github.com/Azure/go-autorest/autorest/adal => github.com/Azure/go-autorest/autorest/adal v0.5.0
replace github.com/Azure/go-autorest => github.com/Azure/go-autorest v13.2.0+incompatible

585
go.sum

File diff suppressed because it is too large Load Diff

View File

@ -20,6 +20,7 @@ import (
"istio.io/istio.io/pkg/test/istioio"
"istio.io/istio/pkg/test/framework"
"istio.io/istio/pkg/test/framework/components/istio"
"istio.io/istio/pkg/test/framework/resource"
)
var (
@ -42,6 +43,6 @@ func TestDocs(t *testing.T) {
istioio.TestDocs(t, setupSpec)
}
func setupConfig(cfg *istio.Config) {
func setupConfig(ctx resource.Context, cfg *istio.Config) {
cfg.ControlPlaneValues = "profile: demo"
}

View File

@ -55,10 +55,9 @@ _wait_for_istio() {
local kind="$1"
local namespace="$2"
local name="$3"
if ! istioctl experimental wait --for=distribution --timeout=5m "$kind" "$name.$namespace"; then
if ! istioctl experimental wait --for=distribution --timeout=30s "$kind" "$name.$namespace"; then
echo "Failed distribution of $kind $name in namespace $namespace"
istioctl ps
exit 1
fi
}