7.7 KiB

Raw Blame History

working-with-istio-on-non-flat-network

This document uses an example to demonstrate how to use Istio on Karmada when the clusters reside on the different networks.

Follow this guide to install the Istio control plane on member1 (the primary cluster) and configure member2 (the remote cluster) to use the control plane in member1. All clusters reside on the different network, meaning there is not direct connectivity between the pods in all clusters.

The reason for deploying istiod on the member1 is that kiali needs to be deployed on the same cluster as istiod . If istiod and kiali are deployed on the karmada-host,kiali will not find the namespace created by karmada. It cannot implement the function of service topology for application deployed by karmada. I will continue to provide a new solution later that deploys istiod on the karmada-host.

Install Karmada

Install karmada control plane

Following the steps Install karmada control plane in Quick Start, you can get a Karmada.

Deploy Istio

Install istioctl

Please refer to the istioctl Installation.

Prepare CA certificates

Following the steps plug-in-certificates-and-key-into-the-cluster to configure Istio CA.

Replace the cluster name cluster1 with primary, the output will looks like as following:

[root@vm1-su-001 istio-1.12.6]# tree certs/
certs/
├── primary
│   ├── ca-cert.pem
│   ├── ca-key.pem
│   ├── cert-chain.pem
│   └── root-cert.pem
├── root-ca.conf
├── root-cert.csr
├── root-cert.pem
├── root-cert.srl
└── root-key.pem

Install Istio on karmada-apiserver

Export KUBECONFIG and switch to karmada apiserver:

export KUBECONFIG=$HOME/.kube/karmada.config
kubectl config use-context karmada-apiserver

Create a secret cacerts in istio-system namespace:

kubectl create namespace istio-system
kubectl create secret generic cacerts -n istio-system \
    --from-file=certs/primary/ca-cert.pem \
    --from-file=certs/primary/ca-key.pem \
    --from-file=certs/primary/root-cert.pem \
    --from-file=certs/primary/cert-chain.pem

Create a propagation policy for cacerts secret:

cat <<EOF | kubectl apply -f -
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: cacerts-propagation
  namespace: istio-system
spec:
  resourceSelectors:
    - apiVersion: v1
      kind: Secret
      name: cacerts
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2
EOF

Run the following command to install istio CRDs on karmada apiserver:

istioctl install

Karmada apiserver will not deploy a real istiod pod, you should press ctrl+c to exit installation when Processing resources for Istiod.

✔ Istio core installed                                                           
- Processing resources for Istiod.

Install Istiod on member1

Disable Karmada's auto-sync feature when labeling a namespace, because I need to label the same namespace istio-system in the different clusters differently.

Export KUBECONFIG and switch to karmada host:

export KUBECONFIG=$HOME/.kube/karmada.config
kubectl config use-context karmada-host

Edit karmada-controller-managerdeployment

kubectl edit deployment karmada-controller-manager -n karmada-system

add --controllers=-namespace,* in start command

Set the network of member1 and member2

switch to karmada api-server and list work about namespace istio-system:

kubectl config use-context karmada-apiserver
kubectl get work -A | grep istio-system-

the output will looks like as following:

[root@vm1-su-001 istio-1.12.6]# kubectl get work -A | grep istio-system-
karmada-es-member1   istio-system-f854dc5d9            true     21m
karmada-es-member2   istio-system-f854dc5d9            true     21m

label namespace istio-system on member1 by using the output above

kubectl edit work istio-system-f854dc5d9 -n karmada-es-member1

add topology.istio.io/network: network1 in .spec.workload.manifests.metadata.labels

label namespace istio-system on member2 by using the output above

kubectl edit work istio-system-f854dc5d9 -n karmada-es-member2

add topology.istio.io/network: network2 in .spec.workload.manifests.metadata.labels

Install istio control plane

Export KUBECONFIG and switch to member1:

export KUBECONFIG="$HOME/.kube/members.config"
kubectl config use-context member1

cat <<EOF | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    accessLogFile: /dev/stdout
  values:
    global:
      meshID: mesh1
      multiCluster:
        clusterName: member1
      network: network1
EOF

Install the east-west gateway in member1

samples/multicluster/gen-eastwest-gateway.sh --mesh mesh1 --cluster member1 --network network1 | istioctl install -y -f -

Expose the control plane and service in member1

kubectl apply -f samples/multicluster/expose-istiod.yaml -n istio-system
kubectl apply -f samples/multicluster/expose-services.yaml -n istio-system

Configure `member2` as a remote cluster

Enable API ServerAccess to member2

switch to member2:

kubectl config use-context member2

Prepare member2 cluster secret

istioctl x create-remote-secret --name=member2 > istio-remote-secret-member2.yaml

Switch to member1:

kubectl config use-context member1

Apply istio remote secret

kubectl apply -f istio-remote-secret-member2.yaml

Configure member2 as a remote

Save the address of member1’s east-west gateway

export DISCOVERY_ADDRESS=$(kubectl -n istio-system get svc istio-eastwestgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

Create a remote configuration on member2.

Switch to member2:

kubectl config use-context member2

cat <<EOF | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  values:
    global:
      meshID: mesh1
      multiCluster:
        clusterName: member2
      network: network2
      remotePilotAddress: ${DISCOVERY_ADDRESS}
EOF

Install the east-west gateway in member2

samples/multicluster/gen-eastwest-gateway.sh --mesh mesh1 --cluster member2 --network network2 | istioctl install -y -f -

Expose service in member2

kubectl apply -f samples/multicluster/expose-services.yaml -n istio-system

Deploy bookinfo application

Enable Karmada's auto-sync feature