Document node selectors for sharding (#176)

This updates sharding documentation to take node selectors, and the new
sharding parameters syntax, into account.
This commit is contained in:
Corentin Néau 2024-09-02 10:04:37 +02:00 committed by GitHub
parent f5ebb07475
commit a0afc4df72
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 68 additions and 34 deletions

View File

@ -22,6 +22,7 @@ fleetcontroller gitjob [flags]
--listen string The port the webhook listens. (default ":8080") --listen string The port the webhook listens. (default ":8080")
--metrics-bind-address string The address the metric endpoint binds to. (default ":8081") --metrics-bind-address string The address the metric endpoint binds to. (default ":8081")
--namespace string namespace to watch (default "cattle-fleet-system") --namespace string namespace to watch (default "cattle-fleet-system")
--shard-node-selector string node selector to apply to jobs based on the shard ID, if any
``` ```
### Options inherited from parent commands ### Options inherited from parent commands

View File

@ -86,30 +86,46 @@ You can now [register some git repos](./gitrepo-add.md) in the `fleet-local` nam
### Deployment ### Deployment
From 0.10 onwards, Fleet supports static sharding. The Fleet controller chart can be installed with `--set From 0.10 onwards, Fleet supports static sharding.
shards={<comma-separated shard IDs>}`, which will result in: Each shard is defined by its shard ID.
* as many Fleet controller deployments as specified unique shard IDs, Optionally, a shard can have a [node
selector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector), instructing Fleet to
create all controller pods and jobs for that shard on nodes matching that selector.
The Fleet controller chart can be installed with the following arguments:
* `--set shards[$index].id=$shard_id`
* `--set shards[$index].nodeSelector.$key=$value`
This will result in:
* as many Fleet controller and gitjob deployments as specified unique shard IDs,
* plus the usual unsharded Fleet controller pod. That latter pod will be the only one containing agent management and * plus the usual unsharded Fleet controller pod. That latter pod will be the only one containing agent management and
cleanup containers. cleanup containers.
For instance: For instance:
```bash ```bash
$ helm -n cattle-fleet-system install --create-namespace --wait --set shards="{foo,bar,baz}" \ $ helm -n cattle-fleet-system install --create-namespace --wait fleet fleet/fleet \
fleet fleet/fleet --set shards[0].id=foo \
--set shards[0].nodeSelector."kubernetes\.io/hostname"=k3d-upstream-server-0 \
$ kubectl -n cattle-fleet-system get pods -l app=fleet-controller --set shards[1].id=bar \
NAME READY STATUS RESTARTS AGE --set shards[1].nodeSelector."kubernetes\.io/hostname"=k3d-upstream-server-1 \
fleet-controller-78c74fdb85-b6q64 3/3 Running 0 77s --set shards[2].id=baz \
fleet-controller-shard-bar-777d888865-w2dks 1/1 Running 0 77s --set shards[2].nodeSelector."kubernetes\.io/hostname"=k3d-upstream-server-2 \
fleet-controller-shard-baz-6595bd9cb9-27whg 1/1 Running 0 77s
fleet-controller-shard-foo-85d49b446f-pzxkw 1/1 Running 0 77s
$ kubectl -n cattle-fleet-system get pods -l app=fleet-controller \ $ kubectl -n cattle-fleet-system get pods -l app=fleet-controller \
-o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.labels.fleet\.cattle\.io/shard-id}{"\n"}{end}' -o=custom-columns='Name:.metadata.name,Shard-ID:.metadata.labels.fleet\.cattle\.io/shard-id,Node:spec.nodeName'
fleet-controller-78c74fdb85-b6q64 Name Shard-ID Node
fleet-controller-shard-bar-777d888865-w2dks bar fleet-controller-b4c469c85-rj2q8 k3d-upstream-server-2
fleet-controller-shard-baz-6595bd9cb9-27whg baz fleet-controller-shard-bar-5f5999958f-nt4bm bar k3d-upstream-server-1
fleet-controller-shard-foo-85d49b446f-pzxkw foo fleet-controller-shard-baz-75c8587898-2wkk9 baz k3d-upstream-server-2
fleet-controller-shard-foo-55478fb9d8-42q2f foo k3d-upstream-server-0
$ kubectl -n cattle-fleet-system get pods -l app=gitjob \
-o=custom-columns='Name:.metadata.name,Shard-ID:.metadata.labels.fleet\.cattle\.io/shard-id,Node:spec.nodeName'
Name Shard-ID Node
gitjob-8498c6d78b-mdhgh k3d-upstream-server-1
gitjob-shard-bar-8659ffc945-9vtlx bar k3d-upstream-server-1
gitjob-shard-baz-6d67f596dc-fsz9m baz k3d-upstream-server-2
gitjob-shard-foo-8697bb7f67-wzsfj foo k3d-upstream-server-0
``` ```
### How it works ### How it works

View File

@ -30,6 +30,7 @@ fleetcontroller gitjob [flags]
--disable-gitops disable gitops components --disable-gitops disable gitops components
--disable-metrics disable metrics --disable-metrics disable metrics
--shard-id string only manage resources labeled with a specific shard ID --shard-id string only manage resources labeled with a specific shard ID
--shard-node-selector string node selector to apply to jobs based on the shard ID, if any
``` ```
### SEE ALSO ### SEE ALSO

View File

@ -86,30 +86,46 @@ You can now [register some git repos](./gitrepo-add.md) in the `fleet-local` nam
### Deployment ### Deployment
From 0.10 onwards, Fleet supports static sharding. The Fleet controller chart can be installed with `--set From 0.10 onwards, Fleet supports static sharding.
shards={<comma-separated shard IDs>}`, which will result in: Each shard is defined by its shard ID.
* as many Fleet controller deployments as specified unique shard IDs, Optionally, a shard can have a [node
selector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector), instructing Fleet to
create all controller pods and jobs for that shard on nodes matching that selector.
The Fleet controller chart can be installed with the following arguments:
* `--set shards[$index].id=$shard_id`
* `--set shards[$index].nodeSelector.$key=$value`
This will result in:
* as many Fleet controller and gitjob deployments as specified unique shard IDs,
* plus the usual unsharded Fleet controller pod. That latter pod will be the only one containing agent management and * plus the usual unsharded Fleet controller pod. That latter pod will be the only one containing agent management and
cleanup containers. cleanup containers.
For instance: For instance:
```bash ```bash
$ helm -n cattle-fleet-system install --create-namespace --wait --set shards="{foo,bar,baz}" \ $ helm -n cattle-fleet-system install --create-namespace --wait fleet fleet/fleet \
fleet fleet/fleet --set shards[0].id=foo \
--set shards[0].nodeSelector."kubernetes\.io/hostname"=k3d-upstream-server-0 \
$ kubectl -n cattle-fleet-system get pods -l app=fleet-controller --set shards[1].id=bar \
NAME READY STATUS RESTARTS AGE --set shards[1].nodeSelector."kubernetes\.io/hostname"=k3d-upstream-server-1 \
fleet-controller-78c74fdb85-b6q64 3/3 Running 0 77s --set shards[2].id=baz \
fleet-controller-shard-bar-777d888865-w2dks 1/1 Running 0 77s --set shards[2].nodeSelector."kubernetes\.io/hostname"=k3d-upstream-server-2 \
fleet-controller-shard-baz-6595bd9cb9-27whg 1/1 Running 0 77s
fleet-controller-shard-foo-85d49b446f-pzxkw 1/1 Running 0 77s
$ kubectl -n cattle-fleet-system get pods -l app=fleet-controller \ $ kubectl -n cattle-fleet-system get pods -l app=fleet-controller \
-o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.labels.fleet\.cattle\.io/shard-id}{"\n"}{end}' -o=custom-columns='Name:.metadata.name,Shard-ID:.metadata.labels.fleet\.cattle\.io/shard-id,Node:spec.nodeName'
fleet-controller-78c74fdb85-b6q64 Name Shard-ID Node
fleet-controller-shard-bar-777d888865-w2dks bar fleet-controller-b4c469c85-rj2q8 k3d-upstream-server-2
fleet-controller-shard-baz-6595bd9cb9-27whg baz fleet-controller-shard-bar-5f5999958f-nt4bm bar k3d-upstream-server-1
fleet-controller-shard-foo-85d49b446f-pzxkw foo fleet-controller-shard-baz-75c8587898-2wkk9 baz k3d-upstream-server-2
fleet-controller-shard-foo-55478fb9d8-42q2f foo k3d-upstream-server-0
$ kubectl -n cattle-fleet-system get pods -l app=gitjob \
-o=custom-columns='Name:.metadata.name,Shard-ID:.metadata.labels.fleet\.cattle\.io/shard-id,Node:spec.nodeName'
Name Shard-ID Node
gitjob-8498c6d78b-mdhgh k3d-upstream-server-1
gitjob-shard-bar-8659ffc945-9vtlx bar k3d-upstream-server-1
gitjob-shard-baz-6d67f596dc-fsz9m baz k3d-upstream-server-2
gitjob-shard-foo-8697bb7f67-wzsfj foo k3d-upstream-server-0
``` ```
### How it works ### How it works