docs: document Azure Workload Identity in specs
Co-authored-by: Hidde Beydals <hidde@hhh.computer> Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
This commit is contained in:
parent
37e113a422
commit
9832331db3
|
@ -280,6 +280,7 @@ Without a [Secret reference](#secret-reference), authentication using a chain
|
||||||
with:
|
with:
|
||||||
|
|
||||||
- [Environment credentials](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#EnvironmentCredential)
|
- [Environment credentials](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#EnvironmentCredential)
|
||||||
|
- [Workload Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity@v1.3.0-beta.4#WorkloadIdentityCredential)
|
||||||
- [Managed Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#ManagedIdentityCredential)
|
- [Managed Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#ManagedIdentityCredential)
|
||||||
with the `AZURE_CLIENT_ID`
|
with the `AZURE_CLIENT_ID`
|
||||||
- Managed Identity with a system-assigned identity
|
- Managed Identity with a system-assigned identity
|
||||||
|
@ -436,22 +437,103 @@ data:
|
||||||
accountKey: <BASE64>
|
accountKey: <BASE64>
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Managed Identity with AAD Pod Identity
|
##### Workload Identity
|
||||||
|
|
||||||
If you are using [aad pod identity](https://azure.github.io/aad-pod-identity/docs), you can create an identity that has access to Azure Storage.
|
If you have [Workload Identity mutating webhook](https://azure.github.io/azure-workload-identity/docs/installation/managed-clusters.html)
|
||||||
|
installed on your cluster. You need to create an Azure Identity and give it
|
||||||
|
access to Azure Blob Storage.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
export IDENTITY_NAME="blob-access"
|
||||||
|
|
||||||
|
az role assignment create --role "Storage Blob Data Reader" \
|
||||||
|
--assignee-object-id "$(az identity show -n $IDENTITY_NAME -o tsv --query principalId -g $RESOURCE_GROUP)" \
|
||||||
|
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
|
||||||
|
```
|
||||||
|
|
||||||
|
Establish a federated identity between the Identity and the source-controller
|
||||||
|
ServiceAccount.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
export SERVICE_ACCOUNT_ISSUER="$(az aks show --resource-group <RESOURCE_GROUP> --name <CLUSTER-NAME> --query "oidcIssuerProfile.issuerUrl" -otsv)"
|
||||||
|
|
||||||
|
az identity federated-credential create \
|
||||||
|
--name "kubernetes-federated-credential" \
|
||||||
|
--identity-name "${IDENTITY_NAME}" \
|
||||||
|
--resource-group "${RESOURCE_GROUP}" \
|
||||||
|
--issuer "${SERVICE_ACCOUNT_ISSUER}" \
|
||||||
|
--subject "system:serviceaccount:flux-system:source-controller"
|
||||||
|
```
|
||||||
|
|
||||||
|
Add a patch to label and annotate the source-controller Pods and ServiceAccount
|
||||||
|
correctly so that it can match an identity binding:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
resources:
|
||||||
|
- gotk-components.yaml
|
||||||
|
- gotk-sync.yaml
|
||||||
|
patches:
|
||||||
|
- patch: |-
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ServiceAccount
|
||||||
|
metadata:
|
||||||
|
name: source-controller
|
||||||
|
namespace: flux-system
|
||||||
|
annotations:
|
||||||
|
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
- patch: |-
|
||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: source-controller
|
||||||
|
namespace: flux-system
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
```
|
||||||
|
|
||||||
|
If you have set up Workload Identity correctly and labeled the source-controller
|
||||||
|
Pod and ServiceAccount, then you don't need to reference a Secret. For more information,
|
||||||
|
please see [documentation](https://azure.github.io/azure-workload-identity/docs/quick-start.html).
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: source.toolkit.fluxcd.io/v1beta2
|
||||||
|
kind: Bucket
|
||||||
|
metadata:
|
||||||
|
name: azure-bucket
|
||||||
|
namespace: flux-system
|
||||||
|
spec:
|
||||||
|
interval: 5m0s
|
||||||
|
provider: azure
|
||||||
|
bucketName: testsas
|
||||||
|
endpoint: https://testfluxsas.blob.core.windows.net
|
||||||
|
```
|
||||||
|
|
||||||
|
##### Managed Identity with AAD Pod Identity
|
||||||
|
|
||||||
|
If you are using [aad pod identity](https://azure.github.io/aad-pod-identity/docs),
|
||||||
|
You need to create an Azure Identity and give it access to Azure Blob Storage.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
export IDENTITY_NAME="blob-access"
|
export IDENTITY_NAME="blob-access"
|
||||||
|
|
||||||
az role assignment create --role "Storage Blob Data Contributor" \
|
az role assignment create --role "Storage Blob Data Reader" \
|
||||||
--assignee-object-id "$(az identity show -n blob-access -o tsv --query principalId -g $RESOURCE_GROUP)" \
|
--assignee-object-id "$(az identity show -n $IDENTITY_NAME -o tsv --query principalId -g $RESOURCE_GROUP)" \
|
||||||
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/aks-somto/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
|
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
|
||||||
|
|
||||||
export IDENTITY_CLIENT_ID="$(az identity show -n ${IDENTITY_NAME} -g ${RESOURCE_GROUP} -otsv --query clientId)"
|
export IDENTITY_CLIENT_ID="$(az identity show -n ${IDENTITY_NAME} -g ${RESOURCE_GROUP} -otsv --query clientId)"
|
||||||
export IDENTITY_RESOURCE_ID="$(az identity show -n ${IDENTITY_NAME} -otsv --query id)"
|
export IDENTITY_RESOURCE_ID="$(az identity show -n ${IDENTITY_NAME} -otsv --query id)"
|
||||||
```
|
```
|
||||||
|
|
||||||
Create an `AzureIdentity` object that references the identity created above:
|
Create an AzureIdentity object that references the identity created above:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
---
|
---
|
||||||
|
@ -466,7 +548,8 @@ spec:
|
||||||
type: 0 # user-managed identity
|
type: 0 # user-managed identity
|
||||||
```
|
```
|
||||||
|
|
||||||
Create an `AzureIdentityBinding` object that binds pods with a specific selector with the `AzureIdentity` created:
|
Create an AzureIdentityBinding object that binds Pods with a specific selector
|
||||||
|
with the AzureIdentity created:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
apiVersion: "aadpodidentity.k8s.io/v1"
|
apiVersion: "aadpodidentity.k8s.io/v1"
|
||||||
|
@ -493,7 +576,8 @@ spec:
|
||||||
aadpodidbinding: ${IDENTITY_NAME} # match the AzureIdentity name
|
aadpodidbinding: ${IDENTITY_NAME} # match the AzureIdentity name
|
||||||
```
|
```
|
||||||
|
|
||||||
If you have set aad-pod-identity up correctly and labeled the source-controller pod, then you don't need to reference a secret.
|
If you have set up aad-pod-identity correctly and labeled the source-controller
|
||||||
|
Pod, then you don't need to reference a Secret.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
apiVersion: source.toolkit.fluxcd.io/v1beta2
|
apiVersion: source.toolkit.fluxcd.io/v1beta2
|
||||||
|
@ -535,13 +619,16 @@ data:
|
||||||
sasKey: <base64>
|
sasKey: <base64>
|
||||||
```
|
```
|
||||||
|
|
||||||
The sasKey only contains the SAS token e.g `?sv=2020-08-0&ss=bfqt&srt=co&sp=rwdlacupitfx&se=2022-05-26T21:55:35Z&st=2022-05...`.
|
The `sasKey` only contains the SAS token e.g
|
||||||
The leading question mark is optional.
|
`?sv=2020-08-0&ss=bfqt&srt=co&sp=rwdlacupitfx&se=2022-05-26T21:55:35Z&st=2022-05...`.
|
||||||
The query values from the `sasKey` data field in the Secrets gets merged with the ones in the `spec.endpoint` of the `Bucket`.
|
The leading question mark (`?`) is optional. The query values from the `sasKey`
|
||||||
If the same key is present in the both of them, the value in the `sasKey` takes precedence.
|
data field in the Secrets gets merged with the ones in the `.spec.endpoint` of
|
||||||
|
the Bucket. If the same key is present in the both of them, the value in the
|
||||||
|
`sasKey` takes precedence.
|
||||||
|
|
||||||
**Note:** The SAS token has an expiry date and it must be updated before it expires to allow Flux to
|
**Note:** The SAS token has an expiry date, and it must be updated before it
|
||||||
continue to access Azure Storage. It is allowed to use an account-level or container-level SAS token.
|
expires to allow Flux to continue to access Azure Storage. It is allowed to use
|
||||||
|
an account-level or container-level SAS token.
|
||||||
|
|
||||||
The minimum permissions for an account-level SAS token are:
|
The minimum permissions for an account-level SAS token are:
|
||||||
|
|
||||||
|
@ -756,7 +843,7 @@ spec:
|
||||||
|
|
||||||
### Triggering a reconcile
|
### Triggering a reconcile
|
||||||
|
|
||||||
To manually tell the source-controller to reconcile a Bucket outside of the
|
To manually tell the source-controller to reconcile a Bucket outside the
|
||||||
[specified interval window](#interval), a Bucket can be annotated with
|
[specified interval window](#interval), a Bucket can be annotated with
|
||||||
`reconcile.fluxcd.io/requestedAt: <arbitrary value>`. Annotating the resource
|
`reconcile.fluxcd.io/requestedAt: <arbitrary value>`. Annotating the resource
|
||||||
queues the Bucket for reconciliation if the `<arbitrary-value>` differs from
|
queues the Bucket for reconciliation if the `<arbitrary-value>` differs from
|
||||||
|
|
|
@ -190,9 +190,13 @@ The `aws` provider can be used to authenticate automatically using the EKS worke
|
||||||
node IAM role or IAM Role for Service Accounts (IRSA), and by extension gain access
|
node IAM role or IAM Role for Service Accounts (IRSA), and by extension gain access
|
||||||
to ECR.
|
to ECR.
|
||||||
|
|
||||||
|
##### EKS Worker Node IAM Role
|
||||||
|
|
||||||
When the worker node IAM role has access to ECR, source-controller running on it
|
When the worker node IAM role has access to ECR, source-controller running on it
|
||||||
will also have access to ECR.
|
will also have access to ECR.
|
||||||
|
|
||||||
|
##### IAM Role for Service Accounts (IRSA)
|
||||||
|
|
||||||
When using IRSA to enable access to ECR, add the following patch to your bootstrap
|
When using IRSA to enable access to ECR, add the following patch to your bootstrap
|
||||||
repository, in the `flux-system/kustomization.yaml` file:
|
repository, in the `flux-system/kustomization.yaml` file:
|
||||||
|
|
||||||
|
@ -224,9 +228,56 @@ The `azure` provider can be used to authenticate automatically using kubelet man
|
||||||
identity or Azure Active Directory pod-managed identity (aad-pod-identity), and
|
identity or Azure Active Directory pod-managed identity (aad-pod-identity), and
|
||||||
by extension gain access to ACR.
|
by extension gain access to ACR.
|
||||||
|
|
||||||
|
##### Kubelet Managed Identity
|
||||||
|
|
||||||
When the kubelet managed identity has access to ACR, source-controller running on
|
When the kubelet managed identity has access to ACR, source-controller running on
|
||||||
it will also have access to ACR.
|
it will also have access to ACR.
|
||||||
|
|
||||||
|
##### Azure Workload Identity
|
||||||
|
|
||||||
|
When using Workload Identity to enable access to ACR, add the following patch to
|
||||||
|
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
resources:
|
||||||
|
- gotk-components.yaml
|
||||||
|
- gotk-sync.yaml
|
||||||
|
patches:
|
||||||
|
- patch: |-
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ServiceAccount
|
||||||
|
metadata:
|
||||||
|
name: source-controller
|
||||||
|
namespace: flux-system
|
||||||
|
annotations:
|
||||||
|
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
- patch: |-
|
||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: source-controller
|
||||||
|
namespace: flux-system
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
```
|
||||||
|
|
||||||
|
To use Workload Identity, you have to install the Workload Identity
|
||||||
|
mutating webhook and create an identity that has access to ACR. Next, establish
|
||||||
|
a federated identity between the source-controller ServiceAccount and the
|
||||||
|
identity. Patch the source-controller Pod and ServiceAccount as shown in the patch
|
||||||
|
above. Please take a look at this [guide](https://azure.github.io/azure-workload-identity/docs/quick-start.html#6-establish-federated-identity-credential-between-the-identity-and-the-service-account-issuer--subject).
|
||||||
|
|
||||||
|
##### AAD Pod Identity
|
||||||
|
|
||||||
When using aad-pod-identity to enable access to ACR, add the following patch to
|
When using aad-pod-identity to enable access to ACR, add the following patch to
|
||||||
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
|
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
|
||||||
|
|
||||||
|
@ -261,9 +312,13 @@ if you want to use AKS pod-managed identities add-on that is in preview.
|
||||||
The `gcp` provider can be used to authenticate automatically using OAuth scopes or
|
The `gcp` provider can be used to authenticate automatically using OAuth scopes or
|
||||||
Workload Identity, and by extension gain access to GCR or Artifact Registry.
|
Workload Identity, and by extension gain access to GCR or Artifact Registry.
|
||||||
|
|
||||||
|
#### Access Scopes
|
||||||
|
|
||||||
When the GKE nodes have the appropriate OAuth scope for accessing GCR and Artifact Registry,
|
When the GKE nodes have the appropriate OAuth scope for accessing GCR and Artifact Registry,
|
||||||
source-controller running on it will also have access to them.
|
source-controller running on it will also have access to them.
|
||||||
|
|
||||||
|
#### GKE Workload Identity
|
||||||
|
|
||||||
When using Workload Identity to enable access to GCR or Artifact Registry, add the
|
When using Workload Identity to enable access to GCR or Artifact Registry, add the
|
||||||
following patch to your bootstrap repository, in the `flux-system/kustomization.yaml`
|
following patch to your bootstrap repository, in the `flux-system/kustomization.yaml`
|
||||||
file:
|
file:
|
||||||
|
|
|
@ -161,9 +161,55 @@ The `azure` provider can be used to authenticate automatically using kubelet
|
||||||
managed identity or Azure Active Directory pod-managed identity (aad-pod-identity),
|
managed identity or Azure Active Directory pod-managed identity (aad-pod-identity),
|
||||||
and by extension gain access to ACR.
|
and by extension gain access to ACR.
|
||||||
|
|
||||||
|
##### Kubelet Managed Identity
|
||||||
|
|
||||||
When the kubelet managed identity has access to ACR, source-controller running
|
When the kubelet managed identity has access to ACR, source-controller running
|
||||||
on it will also have access to ACR.
|
on it will also have access to ACR.
|
||||||
|
|
||||||
|
##### Workload Identity
|
||||||
|
|
||||||
|
When using Workload Identity to enable access to ACR, add the following patch to
|
||||||
|
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
resources:
|
||||||
|
- gotk-components.yaml
|
||||||
|
- gotk-sync.yaml
|
||||||
|
patches:
|
||||||
|
- patch: |-
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ServiceAccount
|
||||||
|
metadata:
|
||||||
|
name: source-controller
|
||||||
|
namespace: flux-system
|
||||||
|
annotations:
|
||||||
|
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
- patch: |-
|
||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: source-controller
|
||||||
|
namespace: flux-system
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
spec:
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
azure.workload.identity/use: "true"
|
||||||
|
```
|
||||||
|
|
||||||
|
To use Workload Identity, you have to install the Workload Identity
|
||||||
|
mutating webhook and create an identity that has access to ACR. Next, establish
|
||||||
|
a federated identity between the source-controller ServiceAccount and the
|
||||||
|
identity. Patch the source-controller Pod and ServiceAccount as shown in the patch
|
||||||
|
above. Please take a look at this [guide](https://azure.github.io/azure-workload-identity/docs/quick-start.html#6-establish-federated-identity-credential-between-the-identity-and-the-service-account-issuer--subject).
|
||||||
|
|
||||||
|
##### AAD Pod Identity
|
||||||
When using aad-pod-identity to enable access to ACR, add the following patch to
|
When using aad-pod-identity to enable access to ACR, add the following patch to
|
||||||
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
|
your bootstrap repository, in the `flux-system/kustomization.yaml` file:
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue