80 lines
2.9 KiB
Markdown
80 lines
2.9 KiB
Markdown
# Dynamic Plugins Configuration
|
|
|
|
## Table of Contents
|
|
|
|
* [Dynamic Plugins Configuration](#dynamic-plugins-configuration)
|
|
* [Table of Contents](#table-of-contents)
|
|
* [Motivation](#motivation)
|
|
* [Function Detail](#function-detail)
|
|
* [Feature Interaction](#feature-interaction)
|
|
* [ConfigMap](#configmap)
|
|
* [Reference](#reference)
|
|
|
|
Created by [gh-md-toc](https://github.com/ekalinin/github-markdown-toc)
|
|
|
|
## Motivation
|
|
|
|
There are several plugins and actions in `kube-batch` right now; the users may want to only enable part of plugins and actions. This document is going to introduce dynamic plugins configuration, so the users can configure `kube-batch` according to their
|
|
scenario on the fly.
|
|
|
|
## Function Detail
|
|
|
|
The following YAML format will be introduced for dynamic plugin configuration:
|
|
|
|
```yaml
|
|
actions: "list_of_action_in_order"
|
|
tiers:
|
|
- plugins:
|
|
- name: "plugin_1"
|
|
disableJobOrder: true
|
|
- name: "plugin_2"
|
|
- plugins:
|
|
- name: "plugin_3"
|
|
disableJobOrder: true
|
|
```
|
|
|
|
The `actions` is a list of actions that will be executed by `kube-batch` in order, separated
|
|
by commas. Refer to the [tutorial](https://github.com/kubernetes-sigs/kube-batch/issues/434) for
|
|
the list of supported actions in `kube-batch`. Those actions will be executed in order, although
|
|
the "order" maybe incorrect; the `kube-batch` does not enforce that.
|
|
|
|
The `tiers` is a list of plugins that will be used by related actions, e.g. `allocate`. It includes
|
|
several tiers of plugin list by `plugins`; if it fits plugins in high priority tier, the action will not
|
|
go through the plugins in lower priority tiers. In each tier, it's considered passed if all the plugins are
|
|
fitted in `plugins.names`.
|
|
|
|
The `options` defines the detail behaviour of each plugins, e.g. whether preemption is enabled. If not
|
|
specific, `true` is default value. For now, `preemptable`, `jobOrder`, `taskOrder` are supported.
|
|
|
|
Takes following example as demonstration:
|
|
|
|
1. The actions `"reclaim, allocate, backfill, preempt"` will be executed in order by `kube-batch`
|
|
1. `"priority"` has higher priority than `"gang, drf, predicates, proportion"`; a job with higher priority
|
|
will preempt other jobs, although it's already allocated "enough" resource according to `"drf"`
|
|
1. `"tiers.plugins.drf.disableTaskOrder"` is `true`, so `drf` will not impact task order phase/action
|
|
|
|
```yaml
|
|
actions: "reclaim, allocate, backfill, preempt"
|
|
tiers:
|
|
- plugins:
|
|
- name: "priority"
|
|
- name: "gang"
|
|
- plugins:
|
|
- name: "drf"
|
|
disableTaskOrder: true
|
|
- name: "predicates"
|
|
- name: "proportion"
|
|
```
|
|
|
|
## Feature Interaction
|
|
|
|
### ConfigMap
|
|
|
|
`kube-batch` will read the plugin configuration from command line argument `--scheduler-conf`; user can
|
|
use `ConfigMap` to acesss the volume of `kube-batch` pod during deployment.
|
|
|
|
## Reference
|
|
|
|
* [Add preemption by Job priority](https://github.com/kubernetes-sigs/kube-batch/issues/261)
|
|
* [Support multiple tiers for Plugins](https://github.com/kubernetes-sigs/kube-batch/issues/484)
|