From 874ccc7c3f47fa383a7af57af349d6bb9c16d782 Mon Sep 17 00:00:00 2001 From: Faseela K Date: Thu, 12 Jan 2023 15:21:14 +0100 Subject: [PATCH] Doc: multiple control planes using revisions and discoverySelectors (#12415) * Doc for installing multiple control planes using revisions and discoverySelectors Signed-off-by: Faseela K * make gen Signed-off-by: Faseela K * Fix test Signed-off-by: Faseela K * enable peer-auth Signed-off-by: Faseela K * Fix test Signed-off-by: Faseela K * Review comments Signed-off-by: Faseela K * More review comments Signed-off-by: Faseela K * Additional review comments Signed-off-by: Faseela K * Review comments Signed-off-by: Faseela K * Apply suggestions from code review Co-authored-by: Frank Budinsky * Remove duplicates that came from commit suggestion * Rerun make gen Signed-off-by: Faseela K Signed-off-by: Faseela K Co-authored-by: Frank Budinsky --- .spelling | 1 + .../install/multiple-controlplanes/index.md | 285 ++++++++++++++++++ .../single-cluster-multiple-istiods.svg | 1 + .../install/multiple-controlplanes/snips.sh | 255 ++++++++++++++++ .../install/multiple-controlplanes/test.sh | 56 ++++ 5 files changed, 598 insertions(+) create mode 100644 content/en/docs/setup/install/multiple-controlplanes/index.md create mode 100644 content/en/docs/setup/install/multiple-controlplanes/single-cluster-multiple-istiods.svg create mode 100644 content/en/docs/setup/install/multiple-controlplanes/snips.sh create mode 100644 content/en/docs/setup/install/multiple-controlplanes/test.sh diff --git a/.spelling b/.spelling index 559dc4eef2..8a08738e3c 100644 --- a/.spelling +++ b/.spelling @@ -982,6 +982,7 @@ URIs url user user1 +usergroup userId UTF-8 v0.14.0 diff --git a/content/en/docs/setup/install/multiple-controlplanes/index.md b/content/en/docs/setup/install/multiple-controlplanes/index.md new file mode 100644 index 0000000000..8acbd81ee3 --- /dev/null +++ b/content/en/docs/setup/install/multiple-controlplanes/index.md @@ -0,0 +1,285 @@ +--- +title: Install Multiple Istio Control Planes in a Single Cluster +description: Install multiple Istio control planes in a single cluster using revisions and discoverySelectors. +weight: 55 +keywords: [multiple,control,istiod,local] +owner: istio/wg-environments-maintainers +test: yes +--- + +{{< boilerplate experimental-feature-warning >}} + +This guide walks you through the process of installing multiple Istio control planes within a single cluster and then a way to scope workloads to specific control planes. This deployment model has a single Kubernetes control plane with multiple Istio control planes and meshes. The separation between the meshes is provided by Kubernetes namespaces and RBAC. + +{{< image width="90%" + link="single-cluster-multiple-istiods.svg" + caption="Multiple meshes in a single cluster" + >}} + +Using `discoverySelectors`, you can scope Kubernetes resources in a cluster to specific namespaces managed by an Istio control plane. This includes the Istio custom resources (e.g., Gateway, VirtualService, DestinationRule, etc.) used to configure the mesh. Furthermore, `discoverySelectors` can be used to configure which namespaces should include the `istio-ca-root-cert` config map for a particular Istio control plane. Together, these functions allow mesh operators to specify the namespaces for a given control plane, enabling soft multi-tenancy for multiple meshes based on the boundary of one or more namespaces. This guide uses `discoverySelectors`, along with the revisions capability of Istio, to demonstrate how two meshes can be deployed on a single cluster, each working with a properly scoped subset of the cluster's resources. + +## Before you begin + +This guide requires that you have a Kubernetes cluster with any of the +[supported Kubernetes versions:](/docs/releases/supported-releases#support-status-of-istio-releases) {{< supported_kubernetes_versions >}}. + +This cluster will host two control planes installed in two different system namespaces. The mesh application workloads will run in multiple application-specific namespaces, each namespace associated with one or the other control plane based on revision and discovery selector configurations. + +## Cluster configuration + +### Deploying multiple control planes + +Deploying multiple Istio control planes on a single cluster can be achieved by using different system namespaces for each control plane. +Istio revisions and `discoverySelectors` are then used to scope the resources and workloads that are managed by each control plane. +{{< warning >}} +By default, Istio only uses `discoverySelectors` to scope workload endpoints. To enable full resource scoping, including configuration resources, the feature flag `ENABLE_ENHANCED_RESOURCE_SCOPING` must be set to true. +{{< /warning >}} + +1. Create the first system namespace, `usergroup-1`, and deploy istiod in it: + + {{< text bash >}} + $ kubectl create ns usergroup-1 + $ kubectl label ns usergroup-1 usergroup=usergroup-1 + $ istioctl install -y -f - <}} + +1. Create the second system namespace, `usergroup-2`, and deploy istiod in it: + + {{< text bash >}} + $ kubectl create ns usergroup-2 + $ kubectl label ns usergroup-2 usergroup=usergroup-2 + $ istioctl install -y -f - <}} + +1. Deploy a policy for workloads in the `usergroup-1` namespace to only accept mutual TLS traffic: + + {{< text bash >}} + $ kubectl apply -f - <}} + +1. Deploy a policy for workloads in the `usergroup-2` namespace to only accept mutual TLS traffic: + + {{< text bash >}} + $ kubectl apply -f - <}} + +### Verify the multiple control plane creation + +1. Check the labels on the system namespaces for each control plane: + + {{< text bash >}} + $ kubectl get ns usergroup-1 usergroup2 --show-labels + NAME STATUS AGE LABELS + usergroup-1 Active 13m kubernetes.io/metadata.name=usergroup-1,usergroup=usergroup-1 + usergroup-2 Active 12m kubernetes.io/metadata.name=usergroup-2,usergroup=usergroup-2 + {{< /text >}} + +1. Verify the control planes are deployed and running: + + {{< text bash >}} + $ kubectl get pods -n usergroup-1 + NAMESPACE NAME READY STATUS RESTARTS AGE + usergroup-1 istiod-usergroup-1-5ccc849b5f-wnqd6 1/1 Running 0 12m + {{< /text >}} + + {{< text bash >}} + $ kubectl get pods -n usergroup-2 + NAMESPACE NAME READY STATUS RESTARTS AGE + usergroup-2 istiod-usergroup-2-658d6458f7-slpd9 1/1 Running 0 12m + {{< /text >}} + + You will notice that one istiod deployment per usergroup is created in the specified namespaces. + +1. Run the following commands to list the installed webhooks: + + {{< text bash >}} + $ kubectl get validatingwebhookconfiguration + NAME WEBHOOKS AGE + istio-validator-usergroup-1-usergroup-1 1 18m + istio-validator-usergroup-2-usergroup-2 1 18m + istiod-default-validator 1 18m + {{< /text >}} + + {{< text bash >}} + $ kubectl get mutatingwebhookconfiguration + NAME WEBHOOKS AGE + istio-revision-tag-default-usergroup-1 4 18m + istio-sidecar-injector-usergroup-1-usergroup-1 2 19m + istio-sidecar-injector-usergroup-2-usergroup-2 2 18m + {{< /text >}} + + Note that the output includes `istiod-default-validator` and `istio-revision-tag-default-usergroup-1`, which are the default webhook configurations used for handling requests coming from resources which are not associated with any revision. In a fully scoped environment where every control plane is associated with its resources through proper namespace labeling, there is no need for these default webhook configurations. They should never be invoked. + +### Deploy application workloads per usergroup + +1. Create three application namespaces: + + {{< text bash >}} + $ kubectl create ns app-ns-1 + $ kubectl create ns app-ns-2 + $ kubectl create ns app-ns-3 + {{< /text >}} + +1. Label each namespace to associate them with their respective control planes: + + {{< text bash >}} + $ kubectl label ns app-ns-1 usergroup=usergroup-1 istio.io/rev=usergroup-1 + $ kubectl label ns app-ns-2 usergroup=usergroup-2 istio.io/rev=usergroup-2 + $ kubectl label ns app-ns-3 usergroup=usergroup-2 istio.io/rev=usergroup-2 + {{< /text >}} + +1. Deploy one `sleep` and `httpbin` application per namespace: + + {{< text bash >}} + $ kubectl -n app-ns-1 apply -f samples/sleep/sleep.yaml + $ kubectl -n app-ns-1 apply -f samples/httpbin/httpbin.yaml + $ kubectl -n app-ns-2 apply -f samples/sleep/sleep.yaml + $ kubectl -n app-ns-2 apply -f samples/httpbin/httpbin.yaml + $ kubectl -n app-ns-3 apply -f samples/sleep/sleep.yaml + $ kubectl -n app-ns-3 apply -f samples/httpbin/httpbin.yaml + {{< /text >}} + +1. Wait a few seconds for the `httpbin` and `sleep` pods to be running with sidecars injected: + + {{< text bash >}} + $ kubectl get pods -n app-ns-1 + NAME READY STATUS RESTARTS AGE + httpbin-9dbd644c7-zc2v4 2/2 Running 0 115m + sleep-78ff5975c6-fml7c 2/2 Running 0 115m + {{< /text >}} + + {{< text bash >}} + $ kubectl get pods -n app-ns-2 + NAME READY STATUS RESTARTS AGE + httpbin-9dbd644c7-sd9ln 2/2 Running 0 115m + sleep-78ff5975c6-sz728 2/2 Running 0 115m + {{< /text >}} + + {{< text bash >}} + $ kubectl get pods -n app-ns-3 + NAME READY STATUS RESTARTS AGE + httpbin-9dbd644c7-8ll27 2/2 Running 0 115m + sleep-78ff5975c6-sg4tq 2/2 Running 0 115m + {{< /text >}} + +### Verify the application to control plane mapping + +Now that the applications are deployed, you can use the `istioctl ps` command to confirm that the application workloads are managed by their respective control plane, i.e., `app-ns-1` is managed by `usergroup-1`, `app-ns-2` and `app-ns-3` are managed by `usergroup-2`: + + {{< text bash >}} + $ istioctl ps -i usergroup-1 + NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION + httpbin-9dbd644c7-hccpf.app-ns-1 Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-usergroup-1-5ccc849b5f-wnqd6 1.17-alpha.f5212a6f7df61fd8156f3585154bed2f003c4117 + sleep-78ff5975c6-9zb77.app-ns-1 Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-usergroup-1-5ccc849b5f-wnqd6 1.17-alpha.f5212a6f7df61fd8156f3585154bed2f003c4117 + {{< /text >}} + + {{< text bash >}} + $ istioctl ps -i usergroup-2 + NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION + httpbin-9dbd644c7-vvcqj.app-ns-3 Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-usergroup-2-658d6458f7-slpd9 1.17-alpha.f5212a6f7df61fd8156f3585154bed2f003c4117 + httpbin-9dbd644c7-xzgfm.app-ns-2 Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-usergroup-2-658d6458f7-slpd9 1.17-alpha.f5212a6f7df61fd8156f3585154bed2f003c4117 + sleep-78ff5975c6-fthmt.app-ns-2 Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-usergroup-2-658d6458f7-slpd9 1.17-alpha.f5212a6f7df61fd8156f3585154bed2f003c4117 + sleep-78ff5975c6-nxtth.app-ns-3 Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-usergroup-2-658d6458f7-slpd9 1.17-alpha.f5212a6f7df61fd8156f3585154bed2f003c4117 + {{< /text >}} + +### Verify the application connectivity is ONLY within the respective usergroup + +1. Send a request from the `sleep` pod in `app-ns-1` in `usergroup-1` to the `httpbin` service in `app-ns-2` in `usergroup-2`. The communication should fail: + + {{< text bash >}} + $ kubectl -n app-ns-1 exec "$(kubectl -n app-ns-1 get pod -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- curl -sIL http://httpbin.app-ns-2.svc.cluster.local:8000 + HTTP/1.1 503 Service Unavailable + content-length: 95 + content-type: text/plain + date: Sat, 24 Dec 2022 06:54:54 GMT + server: envoy + {{< /text >}} + +1. Send a request from the `sleep` pod in `app-ns-2` in `usergroup-2` to the `httpbin` service in `app-ns-3` in `usergroup-2`. The communication should work: + + {{< text bash >}} + $ kubectl -n app-ns-2 exec "$(kubectl -n app-ns-2 get pod -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- curl -sIL http://httpbin.app-ns-3.svc.cluster.local:8000 + HTTP/1.1 200 OK + server: envoy + date: Thu, 22 Dec 2022 15:01:36 GMT + content-type: text/html; charset=utf-8 + content-length: 9593 + access-control-allow-origin: * + access-control-allow-credentials: true + x-envoy-upstream-service-time: 3 + {{< /text >}} + +## Cleanup + +1. Clean up the first usergroup: + + {{< text bash >}} + $ istioctl uninstall --revision usergroup-1 + $ kubectl delete ns app-ns-1 usergroup-1 + {{< /text >}} + +1. Clean up the second usergroup: + + {{< text bash >}} + $ istioctl uninstall --revision usergroup-2 + $ kubectl delete ns app-ns-2 app-ns-3 usergroup-2 + {{< /text >}} + +{{< warning >}} +A Cluster Administrator must make sure that Mesh Administrators DO NOT have permission to invoke the global `istioctl uninstall --purge` command, +because that would uninstall all control planes in the cluster. +{{< /warning >}} diff --git a/content/en/docs/setup/install/multiple-controlplanes/single-cluster-multiple-istiods.svg b/content/en/docs/setup/install/multiple-controlplanes/single-cluster-multiple-istiods.svg new file mode 100644 index 0000000000..03bb3d257f --- /dev/null +++ b/content/en/docs/setup/install/multiple-controlplanes/single-cluster-multiple-istiods.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/content/en/docs/setup/install/multiple-controlplanes/snips.sh b/content/en/docs/setup/install/multiple-controlplanes/snips.sh new file mode 100644 index 0000000000..5c72ec3304 --- /dev/null +++ b/content/en/docs/setup/install/multiple-controlplanes/snips.sh @@ -0,0 +1,255 @@ +#!/bin/bash +# shellcheck disable=SC2034,SC2153,SC2155,SC2164 + +# Copyright Istio Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +#################################################################################################### +# WARNING: THIS IS AN AUTO-GENERATED FILE, DO NOT EDIT. PLEASE MODIFY THE ORIGINAL MARKDOWN FILE: +# docs/setup/install/multiple-controlplanes/index.md +#################################################################################################### + +snip_deploying_multiple_control_planes_1() { +kubectl create ns usergroup-1 +kubectl label ns usergroup-1 usergroup=usergroup-1 +istioctl install -y -f - <