Cherry-pick commits to v1.3-branch (#1849)
* chore: remove argo manifests from contrib (#1813) * Make note of knative-eventing being optional in README (#1826) Only Knative Serving is required for KFServing to work. Knative Eventing is used for specific logger/metrics tracking use cases, so a note is made in the README about eventing being optional. * README: Expand section for connecting to a Kubeflow cluster (#1841) The current section for connecting to a Kubeflow cluster only covers connecting with port-forward. Expand the section with instructions for NodePort / LoadBalancer / Ingress. Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Sync Katib manifests with Kustomize v4 support (#1840) * Kubeflow Tekton Pipelines: Sync manifests (#1843) Sync manifests for application "Kubeflow Tekton Pipelines". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kfp-tekton - Path: manifests/kustomize - Revision: v0.8.0-rc0 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Sync wg-notebooks apps to revision 1.3.0-rc.1 (#1846) * Notebook Controller: Sync manifests Sync manifests for application "Notebook Controller". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kubeflow - Path: components/notebook-controller/config - Revision: 8901e3af3b83f599b89562162cdb2854f57082d3 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Tensorboard Controller: Sync manifests Sync manifests for application "Tensorboard Controller". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kubeflow - Path: components/tensorboard-controller/config - Revision: 8901e3af3b83f599b89562162cdb2854f57082d3 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Central Dashboard: Sync manifests Sync manifests for application "Central Dashboard". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kubeflow - Path: components/centraldashboard/manifests - Revision: 8901e3af3b83f599b89562162cdb2854f57082d3 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Profiles + KFAM: Sync manifests Sync manifests for application "Profiles + KFAM". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kubeflow - Path: components/profile-controller/config - Revision: 8901e3af3b83f599b89562162cdb2854f57082d3 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * PodDefaults Webhook: Sync manifests Sync manifests for application "PodDefaults Webhook". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kubeflow - Path: components/admission-webhook/manifests - Revision: 8901e3af3b83f599b89562162cdb2854f57082d3 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Jupyter Web App: Sync manifests Sync manifests for application "Jupyter Web App". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kubeflow - Path: components/crud-web-apps/jupyter/manifests - Revision: 8901e3af3b83f599b89562162cdb2854f57082d3 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Tensorboards Web App: Sync manifests Sync manifests for application "Tensorboards Web App". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kubeflow - Path: components/crud-web-apps/tensorboards/manifests - Revision: 8901e3af3b83f599b89562162cdb2854f57082d3 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Volumes Web App: Sync manifests Sync manifests for application "Volumes Web App". Upstream manifests are copied from: - Repo: https://github.com/kubeflow/kubeflow - Path: components/crud-web-apps/volumes/manifests - Revision: 8901e3af3b83f599b89562162cdb2854f57082d3 Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * Remove load_restrictor kustomize flag and lint README markdown (#1844) * remove load-restrictor kustomize flag and lint README markdown * Set kustomize loader to RestrictionRootOnly in test_util.go * review: undo changes to test scripts * review: add note to reusing patches best practices * README: Add component version matrix (#1847) Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * README: Add FAQ, elaborate kustomize version, update version matrix (#1848) * README: Add FAQ section and justify kustomize version Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> * README: Update version matrix Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com> Co-authored-by: Yuan (Bob) Gong <4957653+Bobgy@users.noreply.github.com> Co-authored-by: Paul Van Eck <pvaneck@us.ibm.com> Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Co-authored-by: DavidSpek <vanderspek.david@gmail.com>
This commit is contained in:
Yannis Zarkadas
GitHub
parent
ef8e7e0c8c
commit
f4d4aac424
144
README.md
144
README.md
|
|
@ -5,12 +5,14 @@
|
|||
<!-- toc -->
|
||||
|
||||
- [Overview](#overview)
|
||||
- [Kubeflow components versions](#kubeflow-components-versions)
|
||||
- [Installation](#installation)
|
||||
* [Prerequisites](#prerequisites)
|
||||
* [Install with a single command](#install-with-a-single-command)
|
||||
* [Install individual components](#install-individual-components)
|
||||
* [Connect to your Kubeflow Cluster](#connect-to-your-kubeflow-cluster)
|
||||
* [Change default user password](#change-default-user-password)
|
||||
- [Frequently Asked Questions](#frequently-asked-questions)
|
||||
|
||||
<!-- tocstop -->
|
||||
|
||||
|
|
@ -33,6 +35,30 @@ The `docs`, `hack`, and `tests` directories will also be gradually phased out.
|
|||
|
||||
Starting Kubeflow 1.3, all components should be deployable using `kustomize` only. Any automation tooling for deployment on top of the manifests should be maintained externally by distribution owners.
|
||||
|
||||
## Kubeflow components versions
|
||||
|
||||
This repo periodically syncs all official Kubeflow components from their respective upstream repos. The following matrix shows the git version that we include for each component:
|
||||
|
||||
| Component | Local Manifests Path | Upstream Revision |
|
||||
| - | - | - |
|
||||
| TFJob Operator | apps/tf-training/upstream | [v1.1.0](https://github.com/kubeflow/tf-operator/tree/v1.1.0/manifests) |
|
||||
| PyTorch Operator | apps/pytorch-job/upstream | [v0.7.0](https://github.com/kubeflow/pytorch-operator/tree/v0.7.0/manifests) |
|
||||
| MPI Operator | apps/mpi-job/upstream | [b367aa55886d2b042f5089df359d8e067e49e8d1](https://github.com/kubeflow/mpi-operator/tree/b367aa55886d2b042f5089df359d8e067e49e8d1/manifests) |
|
||||
| MXNet Operator | apps/mxnet-job/upstream | [v1.1.0](https://github.com/kubeflow/mxnet-operator/v1.1.0/manifests) |
|
||||
| XGBoost Operator | apps/xgboost-job/upstream | [v0.2.0](https://github.com/kubeflow/xgboost-operator/tree/v0.2.0/manifests) |
|
||||
| Notebook Controller | apps/jupyter/notebook-controller/upstream | [v1.3.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.3.0-rc.1/components/notebook-controller/config) |
|
||||
| Tensorboard Controller | apps/tensorboard/tensorboard-controller/upstream | [v1.3.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.3.0-rc.1/components/tensorboard-controller/config) |
|
||||
| Central Dashboard | apps/centraldashboard/upstream | [v1.3.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.3.0-rc.1/components/centraldashboard/manifests) |
|
||||
| Profiles + KFAM | apps/profiles/upstream | [v1.3.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.3.0-rc.1/components/profile-controller/config) |
|
||||
| PodDefaults Webhook | apps/admission-webhook/upstream | [v1.3.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.3.0-rc.1/components/admission-webhook/manifests) |
|
||||
| Jupyter Web App | apps/jupyter/jupyter-web-app/upstream | [v1.3.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.3.0-rc.1/components/crud-web-apps/jupyter/manifests) |
|
||||
| Tensorboards Web App | apps/tensorboard/tensorboards-web-app/upstream | [v1.3.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.3.0-rc.1/components/crud-web-apps/tensorboards/manifests) |
|
||||
| Volumes Web App | apps/volumes-web-app/upstream | [v1.3.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.3.0-rc.1/components/crud-web-apps/volumes/manifests) |
|
||||
| Katib | apps/katib/upstream | [origin/release-0.11 (7d7c34c72ab8bce74262c7abbe55ef9312291219)](https://github.com/kubeflow/katib/tree/7d7c34c72ab8bce74262c7abbe55ef9312291219/manifests/v1beta1) |
|
||||
| KFServing | apps/kfserving/upstream | [origin/release-0.5 (e189a510121c09f764f749143b80f6ee6baaf48b)](https://github.com/kubeflow/kfserving/tree/e189a510121c09f764f749143b80f6ee6baaf48b/config) |
|
||||
| Kubeflow Pipelines | apps/pipeline/upstream | [1.5.0-rc.2](https://github.com/kubeflow/pipelines/tree/1.5.0-rc.2/manifests/kustomize) |
|
||||
| Kubeflow Tekton Pipelines | apps/kfp-tekton/upstream | [v0.8.0-rc0](https://github.com/kubeflow/kfp-tekton/tree/v0.8.0-rc0/manifests/kustomize) |
|
||||
|
||||
## Installation
|
||||
|
||||
Starting Kubeflow 1.3, the Manifests WG provides two options for installing Kubeflow official components and common services with kustomize. The aim is to help end users install easily and to help distribution owners build their opinionated distributions from a tested starting point:
|
||||
|
|
@ -49,8 +75,9 @@ The `example` directory contains an example kustomization for the single command
|
|||
|
||||
### Prerequisites
|
||||
|
||||
- `Kubernetes` (tested with version `1.17`)
|
||||
- `kustomize` (version `3.2.0`) ([link](https://github.com/kubernetes-sigs/kustomize/releases/tag/v3.2.0))
|
||||
- `Kubernetes` (tested with version `1.17`) with a default [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/)
|
||||
- `kustomize` (version `3.2.0`) ([download link](https://github.com/kubernetes-sigs/kustomize/releases/tag/v3.2.0))
|
||||
- :warning: Kubeflow 1.3.0 is not compatible with the latest versions of of kustomize 4.x. This is due to changes in the order resources are sorted and printed. Please see [kubernetes-sigs/kustomize#3794](https://github.com/kubernetes-sigs/kustomize/issues/3794) and [kubeflow/manifests#1797](https://github.com/kubeflow/manifests/issues/1797). We know this is not ideal and are working with the upstream kustomize team to add support for the latest versions of kustomize as soon as we can.
|
||||
- `kubectl`
|
||||
|
||||
---
|
||||
|
|
@ -64,9 +91,8 @@ The `example` directory contains an example kustomization for the single command
|
|||
|
||||
You can install all Kubeflow official components (residing under `apps`) and all common services (residing under `common`) using the following command:
|
||||
|
||||
|
||||
```sh
|
||||
while ! kustomize build --load_restrictor=none example | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done
|
||||
while ! kustomize build example | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done
|
||||
```
|
||||
|
||||
Once, everything is installed successfully, you can access the Kubeflow Central Dashboard [by logging in to your cluster](#connect-to-your-kubeflow-cluster).
|
||||
|
|
@ -88,10 +114,11 @@ cert-manager is used by many Kubeflow components to provide certificates for
|
|||
admission webhooks.
|
||||
|
||||
Install cert-manager:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/cert-manager/cert-manager-kube-system-resources/base | kubectl apply -f -
|
||||
kustomize build --load_restrictor=none common/cert-manager/cert-manager-crds/base | kubectl apply -f -
|
||||
kustomize build --load_restrictor=none common/cert-manager/cert-manager/overlays/self-signed | kubectl apply -f -
|
||||
kustomize build common/cert-manager/cert-manager-kube-system-resources/base | kubectl apply -f -
|
||||
kustomize build common/cert-manager/cert-manager-crds/base | kubectl apply -f -
|
||||
kustomize build common/cert-manager/cert-manager/overlays/self-signed | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Istio
|
||||
|
|
@ -100,10 +127,11 @@ Istio is used by many Kubeflow components to secure their traffic, enforce
|
|||
network authorization and implement routing policies.
|
||||
|
||||
Install Istio:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/istio-1-9-0/istio-crds/base | kubectl apply -f -
|
||||
kustomize build --load_restrictor=none common/istio-1-9-0/istio-namespace/base | kubectl apply -f -
|
||||
kustomize build --load_restrictor=none common/istio-1-9-0/istio-install/base | kubectl apply -f -
|
||||
kustomize build common/istio-1-9-0/istio-crds/base | kubectl apply -f -
|
||||
kustomize build common/istio-1-9-0/istio-namespace/base | kubectl apply -f -
|
||||
kustomize build common/istio-1-9-0/istio-install/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Dex
|
||||
|
|
@ -113,7 +141,7 @@ Dex is an OpenID Connect Identity (OIDC) with multiple authentication backends.
|
|||
Install Dex:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/dex/overlays/istio | kubectl apply -f -
|
||||
kustomize build common/dex/overlays/istio | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### OIDC AuthService
|
||||
|
|
@ -121,20 +149,26 @@ kustomize build --load_restrictor=none common/dex/overlays/istio | kubectl apply
|
|||
The OIDC AuthService extends your Istio Ingress-Gateway capabilities, to be able to function as an OIDC client:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/oidc-authservice/base | kubectl apply -f -
|
||||
kustomize build common/oidc-authservice/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Knative
|
||||
|
||||
Knative is used by the KFServing official Kubeflow component.
|
||||
|
||||
Install Knative:
|
||||
Install Knative Serving:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/knative/knative-serving-crds/base | kubectl apply -f -
|
||||
kustomize build --load_restrictor=none common/knative/knative-serving-install/base | kubectl apply -f -
|
||||
kustomize build --load_restrictor=none common/knative/knative-eventing-crds/base | kubectl apply -f -
|
||||
kustomize build --load_restrictor=none common/knative/knative-eventing-install/base | kubectl apply -f -
|
||||
kustomize build --load_restrictor=none common/istio-1-9-0/cluster-local-gateway/base | kubectl apply -f -
|
||||
kustomize build common/knative/knative-serving-crds/base | kubectl apply -f -
|
||||
kustomize build common/knative/knative-serving-install/base | kubectl apply -f -
|
||||
kustomize build common/istio-1-9-0/cluster-local-gateway/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
Optionally, you can install Knative Eventing which can be used for inference request logging:
|
||||
|
||||
```sh
|
||||
kustomize build common/knative/knative-eventing-crds/base | kubectl apply -f -
|
||||
kustomize build common/knative/knative-eventing-install/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Kubeflow Namespace
|
||||
|
|
@ -143,8 +177,9 @@ Create the namespace where the Kubeflow components will live in. This namespace
|
|||
is named `kubeflow`.
|
||||
|
||||
Install kubeflow namespace:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/kubeflow-namespace/base | kubectl apply -f -
|
||||
kustomize build common/kubeflow-namespace/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Kubeflow Roles
|
||||
|
|
@ -154,10 +189,10 @@ Create the Kubeflow ClusterRoles, `kubeflow-view`, `kubeflow-edit` and
|
|||
ClusterRoles.
|
||||
|
||||
Install kubeflow roles:
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/kubeflow-roles/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
```sh
|
||||
kustomize build common/kubeflow-roles/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Kubeflow Istio Resources
|
||||
|
||||
|
|
@ -167,8 +202,9 @@ If you want to install with your own Istio, then you need this kustomization as
|
|||
well.
|
||||
|
||||
Install istio resources:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/istio-1-9-0/kubeflow-istio-resources/base | kubectl apply -f -
|
||||
kustomize build common/istio-1-9-0/kubeflow-istio-resources/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Kubeflow Pipelines
|
||||
|
|
@ -176,7 +212,7 @@ kustomize build --load_restrictor=none common/istio-1-9-0/kubeflow-istio-resourc
|
|||
Install the Kubeflow Pipelines official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/pipeline/upstream/env/platform-agnostic-multi-user | kubectl apply -f -
|
||||
kustomize build apps/pipeline/upstream/env/platform-agnostic-multi-user | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### KFServing
|
||||
|
|
@ -184,7 +220,7 @@ kustomize build --load_restrictor=none apps/pipeline/upstream/env/platform-agnos
|
|||
Install the KFServing official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/kfserving/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/kfserving/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Katib
|
||||
|
|
@ -192,7 +228,7 @@ kustomize build --load_restrictor=none apps/kfserving/upstream/overlays/kubeflow
|
|||
Install the Katib official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/katib/upstream/installs/katib-with-kubeflow-cert-manager | kubectl apply -f -
|
||||
kustomize build apps/katib/upstream/installs/katib-with-kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Central Dashboard
|
||||
|
|
@ -200,14 +236,15 @@ kustomize build --load_restrictor=none apps/katib/upstream/installs/katib-with-k
|
|||
Install the Central Dashboard official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/centraldashboard/upstream/overlays/istio | kubectl apply -f -
|
||||
kustomize build apps/centraldashboard/upstream/overlays/istio | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Admission Webhook
|
||||
|
||||
Install the Admission Webhook for PodDefaults:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/admission-webhook/upstream/overlays/cert-manager | kubectl apply -f -
|
||||
kustomize build apps/admission-webhook/upstream/overlays/cert-manager | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Notebooks
|
||||
|
|
@ -215,13 +252,13 @@ kustomize build --load_restrictor=none apps/admission-webhook/upstream/overlays/
|
|||
Install the Notebook Controller official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/jupyter/notebook-controller/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/jupyter/notebook-controller/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
Install the Jupyter Web App official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/jupyter/jupyter-web-app/upstream/overlays/istio | kubectl apply -f -
|
||||
kustomize build apps/jupyter/jupyter-web-app/upstream/overlays/istio | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Profiles + KFAM
|
||||
|
|
@ -230,7 +267,7 @@ Install the Profile Controller and the Kubeflow Access-Management (KFAM) officia
|
|||
components:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/profiles/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/profiles/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Volumes Web App
|
||||
|
|
@ -238,7 +275,7 @@ kustomize build --load_restrictor=none apps/profiles/upstream/overlays/kubeflow
|
|||
Install the Volumes Web App official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/volumes-web-app/upstream/overlays/istio | kubectl apply -f -
|
||||
kustomize build apps/volumes-web-app/upstream/overlays/istio | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### Tensorboard
|
||||
|
|
@ -246,12 +283,13 @@ kustomize build --load_restrictor=none apps/volumes-web-app/upstream/overlays/is
|
|||
Install the Tensorboards Web App official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/tensorboard/tensorboards-web-app/upstream/overlays/istio | kubectl apply -f -
|
||||
kustomize build apps/tensorboard/tensorboards-web-app/upstream/overlays/istio | kubectl apply -f -
|
||||
```
|
||||
|
||||
Install the Tensorboard Controller official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/tensorboard/tensorboard-controller/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/tensorboard/tensorboard-controller/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### TFJob Operator
|
||||
|
|
@ -259,7 +297,7 @@ kustomize build --load_restrictor=none apps/tensorboard/tensorboard-controller/u
|
|||
Install the TFJob Operator official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/tf-training/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/tf-training/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### PyTorch Operator
|
||||
|
|
@ -267,7 +305,7 @@ kustomize build --load_restrictor=none apps/tf-training/upstream/overlays/kubefl
|
|||
Install the PyTorch Operator official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/pytorch-job/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/pytorch-job/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### MPI Operator
|
||||
|
|
@ -275,7 +313,7 @@ kustomize build --load_restrictor=none apps/pytorch-job/upstream/overlays/kubefl
|
|||
Install the MPI Operator official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/mpi-job/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/mpi-job/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### MXNet Operator
|
||||
|
|
@ -283,7 +321,7 @@ kustomize build --load_restrictor=none apps/mpi-job/upstream/overlays/kubeflow |
|
|||
Install the MXNet Operator official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/mxnet-job/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/mxnet-job/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### XGBoost Operator
|
||||
|
|
@ -291,7 +329,7 @@ kustomize build --load_restrictor=none apps/mxnet-job/upstream/overlays/kubeflow
|
|||
Install the XGBoost Operator official Kubeflow component:
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none apps/xgboost-job/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
kustomize build apps/xgboost-job/upstream/overlays/kubeflow | kubectl apply -f -
|
||||
```
|
||||
|
||||
#### User Namespace
|
||||
|
|
@ -299,7 +337,7 @@ kustomize build --load_restrictor=none apps/xgboost-job/upstream/overlays/kubefl
|
|||
Finally, create a new namespace for the the default user (named `user`).
|
||||
|
||||
```sh
|
||||
kustomize build --load_restrictor=none common/user-namespace/base | kubectl apply -f -
|
||||
kustomize build common/user-namespace/base | kubectl apply -f -
|
||||
```
|
||||
|
||||
### Connect to your Kubeflow Cluster
|
||||
|
|
@ -316,6 +354,8 @@ kubectl get pods -n kubeflow
|
|||
kubectl get pods -n kubeflow-user
|
||||
```
|
||||
|
||||
#### Port-Forward
|
||||
|
||||
The default way of accessing Kubeflow is via port-forward. This enables you to get started quickly without imposing any requirements on your environment. Run the following to port-forward Istio's Ingress-Gateway to local port `8080`:
|
||||
|
||||
```sh
|
||||
|
|
@ -323,22 +363,46 @@ kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80
|
|||
```
|
||||
|
||||
After running the command, you can access the Kubeflow Central Dashboard by doing the following:
|
||||
|
||||
1. Open your browser and visit `http://localhost:8080`. You should get the Dex login screen.
|
||||
2. Login with the default user's credential. The default username is `user` and the default password is `12341234`.
|
||||
|
||||
#### NodePort / LoadBalancer / Ingress
|
||||
|
||||
In order to connect to Kubeflow using NodePort / LoadBalancer / Ingress, you need to setup HTTPS. The reason is that many of our web apps (e.g., Tensorboard Web App, Jupyter Web App, Katib UI) use [Secure Cookies](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#restrict_access_to_cookies), so accessing Kubeflow with HTTP over a non-localhost domain does not work.
|
||||
|
||||
Exposing your Kubeflow cluster with proper HTTPS is a process heavily dependent on your environment. For this reason, please take a look at the available Kubeflow distributions, which are targeted to specific environments, and select the one that fits your needs.
|
||||
|
||||
---
|
||||
**NOTE**
|
||||
|
||||
If you absolutely need to expose Kubeflow over HTTP, you can disable the `Secure Cookies` feature by setting the `APP_SECURE_COOKIES` environment variable to `false` in every relevant web app. This is not recommended, as it poses security risks.
|
||||
|
||||
---
|
||||
|
||||
### Change default user password
|
||||
|
||||
For security reasons, we don't want to use the default password for the default Kubeflow user when installing in security-sensitive environments. Instead, you should define your own password before deploying. To define a password for the default user:
|
||||
|
||||
1. Pick a password for the default user, with handle `user`, and hash it using `bcrypt`:
|
||||
|
||||
```sh
|
||||
python3 -c 'from passlib.hash import bcrypt; import getpass; print(bcrypt.using(rounds=12, ident="2y").hash(getpass.getpass()))'
|
||||
```
|
||||
|
||||
2. Edit `dex/base/config-map.yaml` and fill the relevant field with the hash of the password you chose:
|
||||
|
||||
```yaml
|
||||
...
|
||||
staticPasswords:
|
||||
- email: user
|
||||
hash: <enter the generated hash here>
|
||||
```
|
||||
|
||||
## Frequently Asked Questions
|
||||
|
||||
- **Q:** What versions of Istio, Knative, Cert-Manager, Argo, ... are compatible with Kubeflow 1.3? \
|
||||
**A:** Please refer to each individual component's documentation for a dependency compatibility range. For Istio, Knative, Dex, Cert-Manager and OIDC-AuthService, the versions in `common` are the ones we have validated.
|
||||
|
||||
- **Q:** Can I use the latest Kustomize version (`v4.x`)? \
|
||||
**A:** Kubeflow 1.3.0 is not compatible with the latest versions of of kustomize 4.x. This is due to changes in the order resources are sorted and printed. Please see [kubernetes-sigs/kustomize#3794](https://github.com/kubernetes-sigs/kustomize/issues/3794) and [kubeflow/manifests#1797](https://github.com/kubeflow/manifests/issues/1797). We know this is not ideal and are working with the upstream kustomize team to add support for the latest versions of kustomize as soon as we can.
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ commonLabels:
|
|||
images:
|
||||
- name: public.ecr.aws/j1r0q0g6/notebooks/admission-webhook
|
||||
newName: public.ecr.aws/j1r0q0g6/notebooks/admission-webhook
|
||||
newTag: v1.3.0-rc.0
|
||||
newTag: v1.3.0-rc.1
|
||||
namespace: kubeflow
|
||||
generatorOptions:
|
||||
disableNameSuffixHash: true
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ commonLabels:
|
|||
images:
|
||||
- name: public.ecr.aws/j1r0q0g6/notebooks/central-dashboard
|
||||
newName: public.ecr.aws/j1r0q0g6/notebooks/central-dashboard
|
||||
newTag: v1.3.0-rc.0
|
||||
newTag: v1.3.0-rc.1
|
||||
configMapGenerator:
|
||||
- envs:
|
||||
- params.env
|
||||
|
|
|
|||
|
|
@ -0,0 +1,191 @@
|
|||
apiVersion: v1
|
||||
data:
|
||||
jupyter-icon.svg: |
|
||||
<svg width="44" height="51" viewBox="0 0 44 51" version="2.0" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:figma="http://www.figma.com/figma/ns">
|
||||
<desc>Created using Figma 0.90</desc>
|
||||
<g id="Canvas" transform="translate(-1640 -2453)" figma:type="canvas">
|
||||
<g id="Group" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="Group" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="Group" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="g" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path9 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path0_fill" transform="translate(1640.54 2474.36)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path10 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path1_fill" transform="translate(1645.68 2474.37)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path11 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path2_fill" transform="translate(1653.39 2474.26)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path12 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path3_fill" transform="translate(1660.43 2474.39)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path13 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path4_fill" transform="translate(1667.55 2472.54)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path14 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path5_fill" transform="translate(1672.47 2474.29)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path15 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path6_fill" transform="translate(1679.98 2474.24)" fill="#4E4E4E" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="g" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path16 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path7_fill" transform="translate(1673.48 2453.69)" fill="#767677" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path17 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path8_fill" transform="translate(1643.21 2484.27)" fill="#F37726" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path18 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path9_fill" transform="translate(1643.21 2457.88)" fill="#F37726" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path19 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path10_fill" transform="translate(1643.28 2496.09)" fill="#9E9E9E" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="path" style="mix-blend-mode:normal;" figma:type="group">
|
||||
<g id="path20 fill" style="mix-blend-mode:normal;" figma:type="vector">
|
||||
<use xlink:href="#path11_fill" transform="translate(1641.87 2458.43)" fill="#616262" style="mix-blend-mode:normal;"/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<path id="path0_fill" d="M 1.74498 5.47533C 1.74498 7.03335 1.62034 7.54082 1.29983 7.91474C 0.943119 8.23595 0.480024 8.41358 0 8.41331L 0.124642 9.3036C 0.86884 9.31366 1.59095 9.05078 2.15452 8.56466C 2.45775 8.19487 2.6834 7.76781 2.818 7.30893C 2.95261 6.85005 2.99341 6.36876 2.93798 5.89377L 2.93798 0L 1.74498 0L 1.74498 5.43972L 1.74498 5.47533Z"/>
|
||||
<path id="path1_fill" d="M 5.50204 4.76309C 5.50204 5.43081 5.50204 6.02731 5.55545 6.54368L 4.496 6.54368L 4.42478 5.48423C 4.20318 5.85909 3.88627 6.16858 3.50628 6.38125C 3.12628 6.59392 2.69675 6.70219 2.26135 6.69503C 1.22861 6.69503 0 6.13415 0 3.84608L 0 0.0445149L 1.193 0.0445149L 1.193 3.6057C 1.193 4.84322 1.57583 5.67119 2.65309 5.67119C 2.87472 5.67358 3.09459 5.63168 3.29982 5.54796C 3.50505 5.46424 3.69149 5.34039 3.84822 5.18366C 4.00494 5.02694 4.1288 4.84049 4.21252 4.63527C 4.29623 4.43004 4.33813 4.21016 4.33575 3.98853L 4.33575 0L 5.52874 0L 5.52874 4.72748L 5.50204 4.76309Z"/>
|
||||
<path id="path2_fill" d="M 0.0534178 2.27264C 0.0534178 1.44466 0.0534178 0.768036 0 0.153731L 1.06836 0.153731L 1.12177 1.2666C 1.3598 0.864535 1.70247 0.534594 2.11325 0.311954C 2.52404 0.0893145 2.98754 -0.0176786 3.45435 0.00238095C 5.03908 0.00238095 6.23208 1.32892 6.23208 3.30538C 6.23208 5.63796 4.7987 6.79535 3.24958 6.79535C 2.85309 6.81304 2.45874 6.7281 2.10469 6.54874C 1.75064 6.36937 1.44888 6.10166 1.22861 5.77151L 1.22861 5.77151L 1.22861 9.33269L 0.0534178 9.33269L 0.0534178 2.29935L 0.0534178 2.27264ZM 1.22861 4.00872C 1.23184 4.17026 1.24972 4.33117 1.28203 4.48948C 1.38304 4.88479 1.61299 5.23513 1.93548 5.48506C 2.25798 5.735 2.65461 5.87026 3.06262 5.86944C 4.31794 5.86944 5.05689 4.8456 5.05689 3.3588C 5.05689 2.05897 4.36246 0.946096 3.10714 0.946096C 2.61036 0.986777 2.14548 1.20726 1.79965 1.5662C 1.45382 1.92514 1.25079 2.3979 1.22861 2.89585L 1.22861 4.00872Z"/>
|
||||
<path id="path3_fill" d="M 1.31764 0.0178059L 2.75102 3.85499C 2.90237 4.28233 3.06262 4.7987 3.16946 5.18153C 3.2941 4.7898 3.42764 4.29123 3.5879 3.82828L 4.88773 0.0178059L 6.14305 0.0178059L 4.36246 4.64735C 3.47216 6.87309 2.92908 8.02158 2.11 8.71601C 1.69745 9.09283 1.19448 9.35658 0.649917 9.48166L 0.356119 8.48453C 0.736886 8.35942 1.09038 8.16304 1.39777 7.90584C 1.8321 7.55188 2.17678 7.10044 2.4038 6.5882C 2.45239 6.49949 2.48551 6.40314 2.50173 6.3033C 2.49161 6.19586 2.46457 6.0907 2.42161 5.9917L 0 0L 1.29983 0L 1.31764 0.0178059Z"/>
|
||||
<path id="path4_fill" d="M 2.19013 0L 2.19013 1.86962L 3.8995 1.86962L 3.8995 2.75992L 2.19013 2.75992L 2.19013 6.26769C 2.19013 7.06896 2.42161 7.53191 3.08043 7.53191C 3.31442 7.53574 3.54789 7.5088 3.77486 7.45179L 3.82828 8.34208C 3.48794 8.45999 3.12881 8.51431 2.76882 8.50234C 2.53042 8.51726 2.29161 8.48043 2.06878 8.39437C 1.84595 8.30831 1.64438 8.17506 1.47789 8.00377C 1.11525 7.51873 0.949826 6.91431 1.01494 6.31221L 1.01494 2.75102L 0 2.75102L 0 1.86072L 1.03274 1.86072L 1.03274 0.275992L 2.19013 0Z"/>
|
||||
<path id="path5_fill" d="M 1.17716 3.57899C 1.153 3.88093 1.19468 4.18451 1.29933 4.46876C 1.40398 4.75301 1.5691 5.01114 1.78329 5.22532C 1.99747 5.43951 2.2556 5.60463 2.53985 5.70928C 2.8241 5.81393 3.12768 5.85561 3.42962 5.83145C 4.04033 5.84511 4.64706 5.72983 5.21021 5.49313L 5.41498 6.38343C 4.72393 6.66809 3.98085 6.80458 3.23375 6.78406C 2.79821 6.81388 2.36138 6.74914 1.95322 6.59427C 1.54505 6.43941 1.17522 6.19809 0.869071 5.88688C 0.562928 5.57566 0.327723 5.2019 0.179591 4.79125C 0.0314584 4.38059 -0.0260962 3.94276 0.0108748 3.50777C 0.0108748 1.54912 1.17716 0 3.0824 0C 5.21911 0 5.75329 1.86962 5.75329 3.06262C 5.76471 3.24644 5.76471 3.43079 5.75329 3.61461L 1.15046 3.61461L 1.17716 3.57899ZM 4.66713 2.6887C 4.70149 2.45067 4.68443 2.20805 4.61709 1.97718C 4.54976 1.74631 4.43372 1.53255 4.2768 1.35031C 4.11987 1.16808 3.92571 1.0216 3.70739 0.920744C 3.48907 0.81989 3.25166 0.767006 3.01118 0.765656C 2.52201 0.801064 2.06371 1.01788 1.72609 1.37362C 1.38847 1.72935 1.19588 2.19835 1.18607 2.6887L 4.66713 2.6887Z"/>
|
||||
<path id="path6_fill" d="M 0.0534178 2.19228C 0.0534178 1.42663 0.0534178 0.767806 0 0.162404L 1.06836 0.162404L 1.06836 1.43553L 1.12177 1.43553C 1.23391 1.04259 1.4656 0.694314 1.78468 0.439049C 2.10376 0.183783 2.4944 0.034196 2.90237 0.0110538C 3.01466 -0.00368459 3.12839 -0.00368459 3.24068 0.0110538L 3.24068 1.12393C 3.10462 1.10817 2.9672 1.10817 2.83114 1.12393C 2.427 1.13958 2.04237 1.30182 1.7491 1.58035C 1.45583 1.85887 1.27398 2.23462 1.23751 2.63743C 1.20422 2.8196 1.18635 3.00425 1.1841 3.18941L 1.1841 6.65267L 0.00890297 6.65267L 0.00890297 2.20118L 0.0534178 2.19228Z"/>
|
||||
<path id="path7_fill" d="M 6.03059 2.83565C 6.06715 3.43376 5.92485 4.02921 5.6218 4.54615C 5.31875 5.0631 4.86869 5.47813 4.32893 5.73839C 3.78917 5.99864 3.18416 6.09233 2.59097 6.00753C 1.99778 5.92272 1.44326 5.66326 0.998048 5.26219C 0.552837 4.86113 0.23709 4.33661 0.0910307 3.75546C -0.0550287 3.17431 -0.0247891 2.56283 0.177897 1.99893C 0.380583 1.43503 0.746541 0.944221 1.22915 0.589037C 1.71176 0.233853 2.28918 0.0303686 2.88784 0.00450543C 3.28035 -0.0170932 3.67326 0.0391144 4.04396 0.169896C 4.41467 0.300677 4.75587 0.503453 5.04794 0.766561C 5.34 1.02967 5.57718 1.34792 5.74582 1.70301C 5.91446 2.0581 6.01124 2.44303 6.03059 2.83565L 6.03059 2.83565Z"/>
|
||||
<path id="path8_fill" d="M 18.6962 7.12238C 10.6836 7.12238 3.64131 4.24672 0 0C 1.41284 3.82041 3.96215 7.1163 7.30479 9.44404C 10.6474 11.7718 14.623 13.0196 18.6962 13.0196C 22.7695 13.0196 26.745 11.7718 30.0877 9.44404C 33.4303 7.1163 35.9796 3.82041 37.3925 4.0486e-13C 33.7601 4.24672 26.7445 7.12238 18.6962 7.12238Z"/>
|
||||
<path id="path9_fill" d="M 18.6962 5.89725C 26.7089 5.89725 33.7512 8.77291 37.3925 13.0196C 35.9796 9.19922 33.4303 5.90333 30.0877 3.57559C 26.745 1.24785 22.7695 4.0486e-13 18.6962 0C 14.623 4.0486e-13 10.6474 1.24785 7.30479 3.57559C 3.96215 5.90333 1.41284 9.19922 0 13.0196C 3.64131 8.76401 10.648 5.89725 18.6962 5.89725Z"/>
|
||||
<path id="path10_fill" d="M 7.59576 3.56656C 7.64276 4.31992 7.46442 5.07022 7.08347 5.72186C 6.70251 6.3735 6.13619 6.89698 5.45666 7.22561C 4.77713 7.55424 4.01515 7.67314 3.26781 7.56716C 2.52046 7.46117 1.82158 7.13511 1.26021 6.63051C 0.698839 6.12591 0.300394 5.46561 0.115637 4.73375C -0.0691191 4.00188 -0.0318219 3.23159 0.222777 2.52099C 0.477376 1.8104 0.93775 1.19169 1.54524 0.743685C 2.15274 0.295678 2.87985 0.0386595 3.63394 0.00537589C 4.12793 -0.0210471 4.62229 0.0501173 5.08878 0.214803C 5.55526 0.37949 5.98473 0.63447 6.35264 0.965179C 6.72055 1.29589 7.01971 1.69584 7.233 2.1422C 7.4463 2.58855 7.56957 3.07256 7.59576 3.56656L 7.59576 3.56656Z"/>
|
||||
<path id="path11_fill" d="M 2.25061 4.37943C 1.81886 4.39135 1.39322 4.27535 1.02722 4.04602C 0.661224 3.81668 0.371206 3.48424 0.193641 3.09052C 0.0160762 2.69679 -0.0411078 2.25935 0.0292804 1.83321C 0.0996686 1.40707 0.294486 1.01125 0.589233 0.695542C 0.883981 0.37983 1.2655 0.158316 1.68581 0.0588577C 2.10611 -0.0406005 2.54644 -0.0135622 2.95143 0.136572C 3.35641 0.286707 3.70796 0.553234 3.96186 0.902636C 4.21577 1.25204 4.3607 1.66872 4.37842 2.10027C 4.39529 2.6838 4.18131 3.25044 3.78293 3.67715C 3.38455 4.10387 2.83392 4.35623 2.25061 4.37943Z"/>
|
||||
</defs>
|
||||
</svg>
|
||||
jupyterlab-logo.svg: |
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="200" viewBox="0 0 1860.8 475">
|
||||
<g class="jp-icon2" fill="#4E4E4E" transform="translate(480.136401, 64.271493)">
|
||||
<g transform="translate(0.000000, 58.875566)">
|
||||
<g transform="translate(0.087603, 0.140294)">
|
||||
<path d="M-426.9,169.8c0,48.7-3.7,64.7-13.6,76.4c-10.8,10-25,15.5-39.7,15.5l3.7,29 c22.8,0.3,44.8-7.9,61.9-23.1c17.8-18.5,24-44.1,24-83.3V0H-427v170.1L-426.9,169.8L-426.9,169.8z"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(155.045296, 56.837104)">
|
||||
<g transform="translate(1.562453, 1.799842)">
|
||||
<path d="M-312,148c0,21,0,39.5,1.7,55.4h-31.8l-2.1-33.3h-0.8c-6.7,11.6-16.4,21.3-28,27.9 c-11.6,6.6-24.8,10-38.2,9.8c-31.4,0-69-17.7-69-89V0h36.4v112.7c0,38.7,11.6,64.7,44.6,64.7c10.3-0.2,20.4-3.5,28.9-9.4 c8.5-5.9,15.1-14.3,18.9-23.9c2.2-6.1,3.3-12.5,3.3-18.9V0.2h36.4V148H-312L-312,148z"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(390.013322, 53.479638)">
|
||||
<g transform="translate(1.706458, 0.231425)">
|
||||
<path d="M-478.6,71.4c0-26-0.8-47-1.7-66.7h32.7l1.7,34.8h0.8c7.1-12.5,17.5-22.8,30.1-29.7 c12.5-7,26.7-10.3,41-9.8c48.3,0,84.7,41.7,84.7,103.3c0,73.1-43.7,109.2-91,109.2c-12.1,0.5-24.2-2.2-35-7.8 c-10.8-5.6-19.9-13.9-26.6-24.2h-0.8V291h-36v-220L-478.6,71.4L-478.6,71.4z M-442.6,125.6c0.1,5.1,0.6,10.1,1.7,15.1 c3,12.3,9.9,23.3,19.8,31.1c9.9,7.8,22.1,12.1,34.7,12.1c38.5,0,60.7-31.9,60.7-78.5c0-40.7-21.1-75.6-59.5-75.6 c-12.9,0.4-25.3,5.1-35.3,13.4c-9.9,8.3-16.9,19.7-19.6,32.4c-1.5,4.9-2.3,10-2.5,15.1V125.6L-442.6,125.6L-442.6,125.6z"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(606.740726, 56.837104)">
|
||||
<g transform="translate(0.751226, 1.989299)">
|
||||
<path d="M-440.8,0l43.7,120.1c4.5,13.4,9.5,29.4,12.8,41.7h0.8c3.7-12.2,7.9-27.7,12.8-42.4 l39.7-119.2h38.5L-346.9,145c-26,69.7-43.7,105.4-68.6,127.2c-12.5,11.7-27.9,20-44.6,23.9l-9.1-31.1 c11.7-3.9,22.5-10.1,31.8-18.1c13.2-11.1,23.7-25.2,30.6-41.2c1.5-2.8,2.5-5.7,2.9-8.8c-0.3-3.3-1.2-6.6-2.5-9.7L-480.2,0.1 h39.7L-440.8,0L-440.8,0z"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(822.748104, 0.000000)">
|
||||
<g transform="translate(1.464050, 0.378914)">
|
||||
<path d="M-413.7,0v58.3h52v28.2h-52V196c0,25,7,39.5,27.3,39.5c7.1,0.1,14.2-0.7,21.1-2.5 l1.7,27.7c-10.3,3.7-21.3,5.4-32.2,5c-7.3,0.4-14.6-0.7-21.3-3.4c-6.8-2.7-12.9-6.8-17.9-12.1c-10.3-10.9-14.1-29-14.1-52.9 V86.5h-31V58.3h31V9.6L-413.7,0L-413.7,0z"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(974.433286, 53.479638)">
|
||||
<g transform="translate(0.990034, 0.610339)">
|
||||
<path d="M-445.8,113c0.8,50,32.2,70.6,68.6,70.6c19,0.6,37.9-3,55.3-10.5l6.2,26.4 c-20.9,8.9-43.5,13.1-66.2,12.6c-61.5,0-98.3-41.2-98.3-102.5C-480.2,48.2-444.7,0-386.5,0c65.2,0,82.7,58.3,82.7,95.7 c-0.1,5.8-0.5,11.5-1.2,17.2h-140.6H-445.8L-445.8,113z M-339.2,86.6c0.4-23.5-9.5-60.1-50.4-60.1 c-36.8,0-52.8,34.4-55.7,60.1H-339.2L-339.2,86.6L-339.2,86.6z"/>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(1201.961058, 53.479638)">
|
||||
<g transform="translate(1.179640, 0.705068)">
|
||||
<path d="M-478.6,68c0-23.9-0.4-44.5-1.7-63.4h31.8l1.2,39.9h1.7c9.1-27.3,31-44.5,55.3-44.5 c3.5-0.1,7,0.4,10.3,1.2v34.8c-4.1-0.9-8.2-1.3-12.4-1.2c-25.6,0-43.7,19.7-48.7,47.4c-1,5.7-1.6,11.5-1.7,17.2v108.3h-36V68 L-478.6,68z"/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
|
||||
<g class="jp-icon-warn0" fill="#F37726">
|
||||
<path d="M1352.3,326.2h37V28h-37V326.2z M1604.8,326.2c-2.5-13.9-3.4-31.1-3.4-48.7v-76 c0-40.7-15.1-83.1-77.3-83.1c-25.6,0-50,7.1-66.8,18.1l8.4,24.4c14.3-9.2,34-15.1,53-15.1c41.6,0,46.2,30.2,46.2,47v4.2 c-78.6-0.4-122.3,26.5-122.3,75.6c0,29.4,21,58.4,62.2,58.4c29,0,50.9-14.3,62.2-30.2h1.3l2.9,25.6H1604.8z M1565.7,257.7 c0,3.8-0.8,8-2.1,11.8c-5.9,17.2-22.7,34-49.2,34c-18.9,0-34.9-11.3-34.9-35.3c0-39.5,45.8-46.6,86.2-45.8V257.7z M1698.5,326.2 l1.7-33.6h1.3c15.1,26.9,38.7,38.2,68.1,38.2c45.4,0,91.2-36.1,91.2-108.8c0.4-61.7-35.3-103.7-85.7-103.7 c-32.8,0-56.3,14.7-69.3,37.4h-0.8V28h-36.6v245.7c0,18.1-0.8,38.6-1.7,52.5H1698.5z M1704.8,208.2c0-5.9,1.3-10.9,2.1-15.1 c7.6-28.1,31.1-45.4,56.3-45.4c39.5,0,60.5,34.9,60.5,75.6c0,46.6-23.1,78.1-61.8,78.1c-26.9,0-48.3-17.6-55.5-43.3 c-0.8-4.2-1.7-8.8-1.7-13.4V208.2z"/>
|
||||
</g>
|
||||
</svg>
|
||||
group-two-icon.svg: |-
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 13.0.2, SVG Export Plug-In . SVG Version: 6.00 Build 14948) -->
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg version="1.1" id="Ebene_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
width="14.35px" height="42.88px" viewBox="0 0 14.35 42.88" enable-background="new 0 0 14.35 42.88" xml:space="preserve">
|
||||
<g>
|
||||
<path d="M12.583,28.057v2.027H1.228c-0.016-0.508,0.066-0.996,0.246-1.465c0.289-0.773,0.752-1.535,1.389-2.285
|
||||
s1.557-1.617,2.76-2.602c1.867-1.531,3.129-2.744,3.785-3.639s0.984-1.74,0.984-2.537c0-0.836-0.299-1.541-0.896-2.115
|
||||
s-1.377-0.861-2.338-0.861c-1.016,0-1.828,0.305-2.438,0.914s-0.918,1.453-0.926,2.531l-2.168-0.223
|
||||
c0.148-1.617,0.707-2.85,1.676-3.697s2.27-1.271,3.902-1.271c1.648,0,2.953,0.457,3.914,1.371s1.441,2.047,1.441,3.398
|
||||
c0,0.688-0.141,1.363-0.422,2.027s-0.748,1.363-1.4,2.098s-1.736,1.742-3.252,3.023c-1.266,1.063-2.078,1.783-2.438,2.162
|
||||
s-0.656,0.76-0.891,1.143H12.583z"/>
|
||||
</g>
|
||||
</svg>
|
||||
group-two-logo.svg: |-
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 13.0.2, SVG Export Plug-In . SVG Version: 6.00 Build 14948) -->
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg version="1.1" id="Ebene_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
width="14.35px" height="42.88px" viewBox="0 0 14.35 42.88" enable-background="new 0 0 14.35 42.88" xml:space="preserve">
|
||||
<g>
|
||||
<path d="M12.583,28.057v2.027H1.228c-0.016-0.508,0.066-0.996,0.246-1.465c0.289-0.773,0.752-1.535,1.389-2.285
|
||||
s1.557-1.617,2.76-2.602c1.867-1.531,3.129-2.744,3.785-3.639s0.984-1.74,0.984-2.537c0-0.836-0.299-1.541-0.896-2.115
|
||||
s-1.377-0.861-2.338-0.861c-1.016,0-1.828,0.305-2.438,0.914s-0.918,1.453-0.926,2.531l-2.168-0.223
|
||||
c0.148-1.617,0.707-2.85,1.676-3.697s2.27-1.271,3.902-1.271c1.648,0,2.953,0.457,3.914,1.371s1.441,2.047,1.441,3.398
|
||||
c0,0.688-0.141,1.363-0.422,2.027s-0.748,1.363-1.4,2.098s-1.736,1.742-3.252,3.023c-1.266,1.063-2.078,1.783-2.438,2.162
|
||||
s-0.656,0.76-0.891,1.143H12.583z"/>
|
||||
</g>
|
||||
</svg>
|
||||
group-one-icon.svg: |-
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 13.0.2, SVG Export Plug-In . SVG Version: 6.00 Build 14948) -->
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg version="1.1" id="Ebene_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
width="14.35px" height="42.88px" viewBox="0 0 14.35 42.88" enable-background="new 0 0 14.35 42.88" xml:space="preserve">
|
||||
<g>
|
||||
<path d="M9.442,30.084H7.333V16.643c-0.508,0.484-1.174,0.969-1.998,1.453s-1.564,0.848-2.221,1.09v-2.039
|
||||
c1.18-0.555,2.211-1.227,3.094-2.016s1.508-1.555,1.875-2.297h1.359V30.084z"/>
|
||||
</g>
|
||||
</svg>
|
||||
group-one-logo.svg: |-
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 13.0.2, SVG Export Plug-In . SVG Version: 6.00 Build 14948) -->
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg version="1.1" id="Ebene_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
width="14.35px" height="42.88px" viewBox="0 0 14.35 42.88" enable-background="new 0 0 14.35 42.88" xml:space="preserve">
|
||||
<g>
|
||||
<path d="M9.442,30.084H7.333V16.643c-0.508,0.484-1.174,0.969-1.998,1.453s-1.564,0.848-2.221,1.09v-2.039
|
||||
c1.18-0.555,2.211-1.227,3.094-2.016s1.508-1.555,1.875-2.297h1.359V30.084z"/>
|
||||
</g>
|
||||
</svg>
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: logos
|
||||
|
|
@ -17,26 +17,35 @@
|
|||
spawnerFormDefaults:
|
||||
image:
|
||||
# The container Image for the user's Jupyter Notebook
|
||||
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.3.0-rc.0
|
||||
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.3.0-rc.1
|
||||
# The list of available standard container Images
|
||||
options:
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.3.0-rc.0
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full:v1.3.0-rc.0
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full:v1.3.0-rc.0
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full:v1.3.0-rc.0
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full:v1.3.0-rc.0
|
||||
imageVSCode:
|
||||
# The container Image for the user's VS-Code Server
|
||||
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.3.0-rc.0
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.3.0-rc.1
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full:v1.3.0-rc.1
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full:v1.3.0-rc.1
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full:v1.3.0-rc.1
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full:v1.3.0-rc.1
|
||||
imageGroupOne:
|
||||
# The container Image for the user's Group One Server
|
||||
# The annotation `notebooks.kubeflow.org/http-rewrite-uri: /`
|
||||
# is applied to notebook in this group, configuring
|
||||
# the Istio rewrite for containers that host their web UI at `/`
|
||||
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.3.0-rc.1
|
||||
# The list of available standard container Images
|
||||
options:
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.3.0-rc.0
|
||||
imageRStudio:
|
||||
# The container Image for the user's RStudio Server
|
||||
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.3.0-rc.0
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.3.0-rc.1
|
||||
imageGroupTwo:
|
||||
# The container Image for the user's Group Two Server
|
||||
# The annotation `notebooks.kubeflow.org/http-rewrite-uri: /`
|
||||
# is applied to notebook in this group, configuring
|
||||
# the Istio rewrite for containers that host their web UI at `/`
|
||||
# The annotation `notebooks.kubeflow.org/http-headers-request-set`
|
||||
# is applied to notebook in this group, configuring Istio
|
||||
# to add the `X-RStudio-Root-Path` header to requests
|
||||
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.3.0-rc.1
|
||||
# The list of available standard container Images
|
||||
options:
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.3.0-rc.0
|
||||
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.3.0-rc.1
|
||||
allowCustomImage: true
|
||||
imagePullPolicy:
|
||||
value: IfNotPresent
|
||||
|
|
@ -44,10 +53,16 @@ spawnerFormDefaults:
|
|||
cpu:
|
||||
# CPU for user's Notebook
|
||||
value: '0.5'
|
||||
# Factor by with to multiply request to calculate limit
|
||||
# if no limit is set, to disable set "none"
|
||||
limitFactor: "1.2"
|
||||
readOnly: false
|
||||
memory:
|
||||
# Memory for user's Notebook
|
||||
value: 1.0Gi
|
||||
# Factor by with to multiply request to calculate limit
|
||||
# if no limit is set, to disable set "none"
|
||||
limitFactor: "1.2"
|
||||
readOnly: false
|
||||
workspaceVolume:
|
||||
# Workspace Volume to be attached to user's Notebook
|
||||
|
|
|
|||
|
|
@ -17,6 +17,8 @@ spec:
|
|||
volumeMounts:
|
||||
- mountPath: /etc/config
|
||||
name: config-volume
|
||||
- mountPath: /src/apps/default/static/assets
|
||||
name: logos-volume
|
||||
env:
|
||||
- name: APP_PREFIX
|
||||
value: $(JWA_PREFIX)
|
||||
|
|
@ -31,3 +33,6 @@ spec:
|
|||
- configMap:
|
||||
name: config
|
||||
name: config-volume
|
||||
- configMap:
|
||||
name: jupyter-web-app-logos
|
||||
name: logos-volume
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ resources:
|
|||
- role.yaml
|
||||
- service-account.yaml
|
||||
- service.yaml
|
||||
- configs/logos-configmap.yaml
|
||||
namePrefix: jupyter-web-app-
|
||||
namespace: kubeflow
|
||||
commonLabels:
|
||||
|
|
@ -22,7 +23,7 @@ commonLabels:
|
|||
images:
|
||||
- name: public.ecr.aws/j1r0q0g6/notebooks/jupyter-web-app
|
||||
newName: public.ecr.aws/j1r0q0g6/notebooks/jupyter-web-app
|
||||
newTag: v1.3.0-rc.0
|
||||
newTag: v1.3.0-rc.1
|
||||
# We need the name to be unique without the suffix because the original name is what
|
||||
# gets used with patches
|
||||
configMapGenerator:
|
||||
|
|
|
|||
|
|
@ -5,4 +5,4 @@ resources:
|
|||
images:
|
||||
- name: public.ecr.aws/j1r0q0g6/notebooks/notebook-controller
|
||||
newName: public.ecr.aws/j1r0q0g6/notebooks/notebook-controller
|
||||
newTag: v1.3.0-rc.0
|
||||
newTag: v1.3.0-rc.1
|
||||
|
|
|
|||
|
|
@ -1,8 +1,6 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
bases:
|
||||
- ../../base
|
||||
|
||||
resources:
|
||||
- virtual-service.yaml
|
||||
configurations:
|
||||
- params.yaml
|
||||
- cert-generator.yaml
|
||||
- rbac.yaml
|
||||
|
|
@ -7,13 +7,13 @@ data:
|
|||
metrics-collector-sidecar: |-
|
||||
{
|
||||
"StdOut": {
|
||||
"image": "docker.io/kubeflowkatib/file-metrics-collector"
|
||||
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.11.0"
|
||||
},
|
||||
"File": {
|
||||
"image": "docker.io/kubeflowkatib/file-metrics-collector"
|
||||
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.11.0"
|
||||
},
|
||||
"TensorFlowEvent": {
|
||||
"image": "docker.io/kubeflowkatib/tfevent-metrics-collector",
|
||||
"image": "docker.io/kubeflowkatib/tfevent-metrics-collector:v0.11.0",
|
||||
"resources": {
|
||||
"limits": {
|
||||
"memory": "1Gi"
|
||||
|
|
@ -24,25 +24,25 @@ data:
|
|||
suggestion: |-
|
||||
{
|
||||
"random": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperopt"
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.11.0"
|
||||
},
|
||||
"tpe": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperopt"
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.11.0"
|
||||
},
|
||||
"grid": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-chocolate"
|
||||
"image": "docker.io/kubeflowkatib/suggestion-chocolate:v0.11.0"
|
||||
},
|
||||
"hyperband": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperband"
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperband:v0.11.0"
|
||||
},
|
||||
"bayesianoptimization": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-skopt"
|
||||
"image": "docker.io/kubeflowkatib/suggestion-skopt:v0.11.0"
|
||||
},
|
||||
"cmaes": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-goptuna"
|
||||
"image": "docker.io/kubeflowkatib/suggestion-goptuna:v0.11.0"
|
||||
},
|
||||
"enas": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-enas",
|
||||
"image": "docker.io/kubeflowkatib/suggestion-enas:v0.11.0",
|
||||
"resources": {
|
||||
"limits": {
|
||||
"memory": "200Mi"
|
||||
|
|
@ -50,12 +50,12 @@ data:
|
|||
}
|
||||
},
|
||||
"darts": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-darts"
|
||||
"image": "docker.io/kubeflowkatib/suggestion-darts:v0.11.0"
|
||||
}
|
||||
}
|
||||
early-stopping: |-
|
||||
{
|
||||
"medianstop": {
|
||||
"image": "docker.io/kubeflowkatib/earlystopping-medianstop"
|
||||
"image": "docker.io/kubeflowkatib/earlystopping-medianstop:v0.11.0"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,9 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- controller.yaml
|
||||
- katib-config.yaml
|
||||
- rbac.yaml
|
||||
- service.yaml
|
||||
- trial-templates.yaml
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- experiment.yaml
|
||||
- suggestion.yaml
|
||||
- trial.yaml
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- db-manager.yaml
|
||||
- service.yaml
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- mysql.yaml
|
||||
- pv.yaml
|
||||
- pvc.yaml
|
||||
- secret.yaml
|
||||
- service.yaml
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
resources:
|
||||
# Namespace.
|
||||
- namespace.yaml
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- rbac.yaml
|
||||
- service.yaml
|
||||
- ui.yaml
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- webhooks.yaml
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
resources:
|
||||
# Namespace.
|
||||
- ../../components/namespace
|
||||
# Katib controller.
|
||||
- ../../components/controller/
|
||||
# Katib CRDs.
|
||||
- ../../components/crd/
|
||||
# Katib DB manager.
|
||||
- ../../components/db-manager/
|
||||
# Katib DB mysql.
|
||||
- ../../components/mysql/
|
||||
# Katib UI.
|
||||
- ../../components/ui/
|
||||
# Katib webhooks.
|
||||
- ../../components/webhook/
|
||||
# Cert-manager certificate for webhooks
|
||||
- certificate.yaml
|
||||
images:
|
||||
- name: docker.io/kubeflowkatib/katib-controller
|
||||
newName: docker.io/kubeflowkatib/katib-controller
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/katib-db-manager
|
||||
newName: docker.io/kubeflowkatib/katib-db-manager
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/katib-ui
|
||||
newName: docker.io/kubeflowkatib/katib-ui
|
||||
newTag: v0.11.0
|
||||
|
||||
patchesStrategicMerge:
|
||||
- patches/katib-cert-injection.yaml
|
||||
|
||||
vars:
|
||||
- fieldref:
|
||||
fieldPath: metadata.namespace
|
||||
name: KATIB_NAMESPACE
|
||||
objref:
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
name: katib-controller
|
||||
- fieldref:
|
||||
fieldPath: metadata.name
|
||||
name: KATIB_SERVICE_NAME
|
||||
objref:
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
name: katib-controller
|
||||
- name: KATIB_CERT_NAME
|
||||
objref:
|
||||
kind: Certificate
|
||||
group: cert-manager.io
|
||||
version: v1alpha2
|
||||
name: katib-webhook-cert
|
||||
fieldref:
|
||||
fieldpath: metadata.name
|
||||
|
||||
configurations:
|
||||
- params.yaml
|
||||
|
|
@ -3,43 +3,33 @@ kind: Kustomization
|
|||
namespace: kubeflow
|
||||
resources:
|
||||
# Namespace.
|
||||
- ../../components/namespace.yaml
|
||||
- ../../components/namespace/
|
||||
# Katib controller.
|
||||
- ../../components/controller/controller.yaml
|
||||
- ../../components/controller/service.yaml
|
||||
- ../../components/controller/rbac.yaml
|
||||
- ../../components/controller/katib-config.yaml
|
||||
- ../../components/controller/trial-templates.yaml
|
||||
- ../../components/controller/
|
||||
# Katib CRDs.
|
||||
- ../../components/crd/experiment.yaml
|
||||
- ../../components/crd/suggestion.yaml
|
||||
- ../../components/crd/trial.yaml
|
||||
- ../../components/crd/
|
||||
# Katib DB manager.
|
||||
- ../../components/db-manager/db-manager.yaml
|
||||
- ../../components/db-manager/service.yaml
|
||||
- ../../components/db-manager/
|
||||
# Katib UI.
|
||||
- ../../components/ui/ui.yaml
|
||||
- ../../components/ui/service.yaml
|
||||
- ../../components/ui/rbac.yaml
|
||||
- ../../components/ui/
|
||||
# Katib Cert Generator
|
||||
- ../../components/cert-generator/
|
||||
# Katib webhooks.
|
||||
- ../../components/webhook/cert-generator.yaml
|
||||
- ../../components/webhook/rbac.yaml
|
||||
- ../../components/webhook/webhooks.yaml
|
||||
- ../../components/webhook/
|
||||
images:
|
||||
- name: docker.io/kubeflowkatib/katib-controller
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-controller
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/katib-db-manager
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-db-manager
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/katib-ui
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-ui
|
||||
- name: docker.io/kubeflowkatib/cert-generator
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/cert-generator
|
||||
newName: docker.io/kubeflowkatib/cert-generator
|
||||
newTag: v0.11.0
|
||||
patchesStrategicMerge:
|
||||
- ../katib-standalone/katib-config-patch.yaml
|
||||
- db-manager-patch.yaml
|
||||
# Modify katib-mysql-secrets with parameters for the DB.
|
||||
secretGenerator:
|
||||
|
|
|
|||
|
|
@ -1,61 +0,0 @@
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: katib-config
|
||||
namespace: kubeflow
|
||||
data:
|
||||
metrics-collector-sidecar: |-
|
||||
{
|
||||
"StdOut": {
|
||||
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.11.0"
|
||||
},
|
||||
"File": {
|
||||
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.11.0"
|
||||
},
|
||||
"TensorFlowEvent": {
|
||||
"image": "docker.io/kubeflowkatib/tfevent-metrics-collector:v0.11.0",
|
||||
"resources": {
|
||||
"limits": {
|
||||
"memory": "1Gi"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
suggestion: |-
|
||||
{
|
||||
"random": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.11.0"
|
||||
},
|
||||
"tpe": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.11.0"
|
||||
},
|
||||
"grid": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-chocolate:v0.11.0"
|
||||
},
|
||||
"hyperband": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-hyperband:v0.11.0"
|
||||
},
|
||||
"bayesianoptimization": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-skopt:v0.11.0"
|
||||
},
|
||||
"cmaes": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-goptuna:v0.11.0"
|
||||
},
|
||||
"enas": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-enas:v0.11.0",
|
||||
"resources": {
|
||||
"limits": {
|
||||
"memory": "200Mi"
|
||||
}
|
||||
}
|
||||
},
|
||||
"darts": {
|
||||
"image": "docker.io/kubeflowkatib/suggestion-darts:v0.11.0"
|
||||
}
|
||||
}
|
||||
early-stopping: |-
|
||||
{
|
||||
"medianstop": {
|
||||
"image": "docker.io/kubeflowkatib/earlystopping-medianstop:v0.11.0"
|
||||
}
|
||||
}
|
||||
|
|
@ -3,46 +3,31 @@ kind: Kustomization
|
|||
namespace: kubeflow
|
||||
resources:
|
||||
# Namespace.
|
||||
- ../../components/namespace.yaml
|
||||
- ../../components/namespace/
|
||||
# Katib controller.
|
||||
- ../../components/controller/controller.yaml
|
||||
- ../../components/controller/service.yaml
|
||||
- ../../components/controller/rbac.yaml
|
||||
- ../../components/controller/katib-config.yaml
|
||||
- ../../components/controller/trial-templates.yaml
|
||||
- ../../components/controller/
|
||||
# Katib CRDs.
|
||||
- ../../components/crd/experiment.yaml
|
||||
- ../../components/crd/suggestion.yaml
|
||||
- ../../components/crd/trial.yaml
|
||||
- ../../components/crd/
|
||||
# Katib DB manager.
|
||||
- ../../components/db-manager/db-manager.yaml
|
||||
- ../../components/db-manager/service.yaml
|
||||
- ../../components/db-manager/
|
||||
# Katib DB mysql.
|
||||
- ../../components/mysql/mysql.yaml
|
||||
- ../../components/mysql/service.yaml
|
||||
- ../../components/mysql/pv.yaml
|
||||
- ../../components/mysql/pvc.yaml
|
||||
- ../../components/mysql/secret.yaml
|
||||
- ../../components/mysql/
|
||||
# Katib UI.
|
||||
- ../../components/ui/ui.yaml
|
||||
- ../../components/ui/service.yaml
|
||||
- ../../components/ui/rbac.yaml
|
||||
- ../../components/ui/
|
||||
# Katib Cert Generator
|
||||
- ../../components/cert-generator/
|
||||
# Katib webhooks.
|
||||
- ../../components/webhook/cert-generator.yaml
|
||||
- ../../components/webhook/rbac.yaml
|
||||
- ../../components/webhook/webhooks.yaml
|
||||
- ../../components/webhook/
|
||||
images:
|
||||
- name: docker.io/kubeflowkatib/katib-controller
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-controller
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/katib-db-manager
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-db-manager
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/katib-ui
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-ui
|
||||
- name: docker.io/kubeflowkatib/cert-generator
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/cert-generator
|
||||
newName: docker.io/kubeflowkatib/cert-generator
|
||||
patchesStrategicMerge:
|
||||
- katib-config-patch.yaml
|
||||
newTag: v0.11.0
|
||||
|
|
|
|||
|
|
@ -1,92 +0,0 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
resources:
|
||||
# Katib controller.
|
||||
- ../../components/controller/controller.yaml
|
||||
- ../../components/controller/service.yaml
|
||||
- ../../components/controller/rbac.yaml
|
||||
- ../../components/controller/katib-config.yaml
|
||||
- ../../components/controller/trial-templates.yaml
|
||||
# Katib CRDs.
|
||||
- ../../components/crd/experiment.yaml
|
||||
- ../../components/crd/suggestion.yaml
|
||||
- ../../components/crd/trial.yaml
|
||||
# Katib DB manager.
|
||||
- ../../components/db-manager/db-manager.yaml
|
||||
- ../../components/db-manager/service.yaml
|
||||
# Katib DB mysql.
|
||||
- ../../components/mysql/mysql.yaml
|
||||
- ../../components/mysql/service.yaml
|
||||
- ../../components/mysql/pvc.yaml
|
||||
- ../../components/mysql/secret.yaml
|
||||
# Katib UI.
|
||||
- ../../components/ui/ui.yaml
|
||||
- ../../components/ui/service.yaml
|
||||
- ../../components/ui/rbac.yaml
|
||||
# Katib webhooks.
|
||||
- ../../components/webhook/webhooks.yaml
|
||||
# Cert-manager certificate for webhooks
|
||||
- certificate.yaml
|
||||
# Kubeflow Katib components.
|
||||
- ../katib-with-kubeflow/kubeflow-katib-roles.yaml
|
||||
- ../katib-with-kubeflow/ui-virtual-service.yaml
|
||||
images:
|
||||
- name: docker.io/kubeflowkatib/katib-controller
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-controller
|
||||
- name: docker.io/kubeflowkatib/katib-db-manager
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-db-manager
|
||||
- name: docker.io/kubeflowkatib/katib-ui
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-ui
|
||||
- name: docker.io/kubeflowkatib/cert-generator
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/cert-generator
|
||||
|
||||
patchesStrategicMerge:
|
||||
- ../katib-standalone/katib-config-patch.yaml
|
||||
- patches/katib-cert-injection.yaml
|
||||
|
||||
patchesJson6902:
|
||||
- path: ../katib-with-kubeflow/patches/mysql-pvc.yaml
|
||||
target:
|
||||
version: v1
|
||||
name: katib-mysql
|
||||
kind: PersistentVolumeClaim
|
||||
namespace: kubeflow
|
||||
|
||||
vars:
|
||||
- fieldref:
|
||||
fieldPath: metadata.namespace
|
||||
name: KATIB_UI_NAMESPACE
|
||||
objref:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: katib-ui
|
||||
- fieldref:
|
||||
fieldPath: metadata.namespace
|
||||
name: KATIB_NAMESPACE
|
||||
objref:
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
name: katib-controller
|
||||
- fieldref:
|
||||
fieldPath: metadata.name
|
||||
name: KATIB_SERVICE_NAME
|
||||
objref:
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
name: katib-controller
|
||||
- name: KATIB_CERT_NAME
|
||||
objref:
|
||||
kind: Certificate
|
||||
group: cert-manager.io
|
||||
version: v1alpha2
|
||||
name: katib-webhook-cert
|
||||
fieldref:
|
||||
fieldpath: metadata.name
|
||||
|
||||
configurations:
|
||||
- params.yaml
|
||||
|
|
@ -2,51 +2,23 @@ apiVersion: kustomize.config.k8s.io/v1beta1
|
|||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
resources:
|
||||
# Katib controller.
|
||||
- ../../components/controller/controller.yaml
|
||||
- ../../components/controller/service.yaml
|
||||
- ../../components/controller/rbac.yaml
|
||||
- ../../components/controller/katib-config.yaml
|
||||
- ../../components/controller/trial-templates.yaml
|
||||
# Katib CRDs.
|
||||
- ../../components/crd/experiment.yaml
|
||||
- ../../components/crd/suggestion.yaml
|
||||
- ../../components/crd/trial.yaml
|
||||
# Katib DB manager.
|
||||
- ../../components/db-manager/db-manager.yaml
|
||||
- ../../components/db-manager/service.yaml
|
||||
# Katib DB mysql.
|
||||
- ../../components/mysql/mysql.yaml
|
||||
- ../../components/mysql/service.yaml
|
||||
- ../../components/mysql/pvc.yaml
|
||||
- ../../components/mysql/secret.yaml
|
||||
# Katib UI.
|
||||
- ../../components/ui/ui.yaml
|
||||
- ../../components/ui/service.yaml
|
||||
- ../../components/ui/rbac.yaml
|
||||
# Katib webhooks.
|
||||
- ../../components/webhook/cert-generator.yaml
|
||||
- ../../components/webhook/rbac.yaml
|
||||
- ../../components/webhook/webhooks.yaml
|
||||
- ../katib-cert-manager
|
||||
# Kubeflow Katib components.
|
||||
- kubeflow-katib-roles.yaml
|
||||
- ui-virtual-service.yaml
|
||||
images:
|
||||
- name: docker.io/kubeflowkatib/katib-controller
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-controller
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/katib-db-manager
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-db-manager
|
||||
newTag: v0.11.0
|
||||
- name: docker.io/kubeflowkatib/katib-ui
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/katib-ui
|
||||
- name: docker.io/kubeflowkatib/cert-generator
|
||||
newTag: v0.11.0
|
||||
newName: docker.io/kubeflowkatib/cert-generator
|
||||
|
||||
patchesStrategicMerge:
|
||||
- ../katib-standalone/katib-config-patch.yaml
|
||||
- patches/remove-resources-patch.yaml
|
||||
|
||||
patchesJson6902:
|
||||
- path: patches/mysql-pvc.yaml
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
$patch: delete
|
||||
apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
name: katib-mysql
|
||||
---
|
||||
$patch: delete
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: kubeflow
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# This makefile is a quick test to verify all manifests can be hydrated.
|
||||
|
||||
test: aws azure dev gcp platform-agnostic platform-agnostic-multi-user plain plain-multi-user
|
||||
|
||||
aws: FORCE
|
||||
kubectl kustomize env/aws
|
||||
|
||||
azure: FORCE
|
||||
kubectl kustomize env/azure
|
||||
|
||||
dev: FORCE
|
||||
kubectl kustomize env/dev
|
||||
|
||||
gcp: FORCE
|
||||
kubectl kustomize env/gcp
|
||||
|
||||
platform-agnostic: FORCE
|
||||
kubectl kustomize env/platform-agnostic
|
||||
|
||||
platform-agnostic-multi-user: FORCE
|
||||
kustomize build --load_restrictor none env/platform-agnostic-multi-user
|
||||
|
||||
plain: FORCE
|
||||
kubectl kustomize env/plain
|
||||
|
||||
plain-multi-user: FORCE
|
||||
kustomize build --load_restrictor none env/plain-multi-user
|
||||
|
||||
FORCE: ;
|
||||
|
|
@ -0,0 +1,158 @@
|
|||
# Kubeflow Pipelines Kustomize Manifest Folder
|
||||
|
||||
## Install Kubeflow Pipelines
|
||||
|
||||
This folder contains Kubeflow Pipelines Kustomize manifests for a light weight
|
||||
deployment. You can follow the instruction and deploy Kubeflow Pipelines in an
|
||||
existing cluster.
|
||||
|
||||
To install Kubeflow Pipelines, you have several options.
|
||||
|
||||
- Via an upcoming commandline tool.
|
||||
- Via Kubectl with Kustomize, it's detailed here.
|
||||
|
||||
### Install via Kustomize
|
||||
|
||||
Deploy latest version of Kubeflow Pipelines.
|
||||
|
||||
It uses following default settings.
|
||||
|
||||
- image: latest released images
|
||||
- namespace: kubeflow
|
||||
- application name: pipeline
|
||||
|
||||
#### Option-1 Install it to any K8s cluster
|
||||
|
||||
It's based on in-cluster PersistentVolumeClaim storage.
|
||||
|
||||
```bash
|
||||
kubectl apply -k cluster-scoped-resources/
|
||||
kubectl wait crd/applications.app.k8s.io --for condition=established --timeout=60s
|
||||
kubectl apply -k env/platform-agnostic/
|
||||
kubectl wait pods -l application-crd-id=kubeflow-pipelines -n kubeflow --for condition=Ready --timeout=1800s
|
||||
kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8080:80
|
||||
```
|
||||
|
||||
Now you can access it via localhost:8080
|
||||
|
||||
#### Option-2 Install it to GCP with in-cluster PersistentVolumeClaim storage
|
||||
|
||||
It's based on in-cluster PersistentVolumeClaim storage.
|
||||
Additionally, it introduced a proxy in GCP to allow user easily access KFP safely.
|
||||
|
||||
```bash
|
||||
kubectl apply -k cluster-scoped-resources/
|
||||
kubectl wait crd/applications.app.k8s.io --for condition=established --timeout=60s
|
||||
|
||||
kubectl apply -k env/dev/
|
||||
kubectl wait applications/pipeline -n kubeflow --for condition=Ready --timeout=1800s
|
||||
|
||||
# Or visit http://console.cloud.google.com/ai-platform/pipelines
|
||||
kubectl describe configmap inverse-proxy-config -n kubeflow | grep googleusercontent.com
|
||||
```
|
||||
|
||||
#### Option-3 Install it to GCP with CloudSQL & GCS-Minio managed storage
|
||||
|
||||
Its storage is based on CloudSQL & GCS. It's better than others for production usage.
|
||||
|
||||
Please following [sample](sample/README.md) for a customized installation.
|
||||
|
||||
#### Option-4 Install it to AWS with S3 and RDS MySQL
|
||||
|
||||
Its storage is based on S3 & AWS RDS. It's more natural for AWS users to use this option.
|
||||
|
||||
Please following [AWS Instructions](env/aws/README.md) for installation.
|
||||
|
||||
Note: Community maintains a repo [e2fyi/kubeflow-aws](https://github.com/e2fyi/kubeflow-aws/tree/master/pipelines) for AWS.
|
||||
|
||||
#### Option-5 Install it to IBM Cloud with in-cluster PersistentVolumeClaim storage
|
||||
|
||||
It's based on in-cluster PersistentVolumeClaim storage.
|
||||
Additionally, it uses the ibm cloud NFS storage with UID support to make sure all pods can run as non-root users.
|
||||
|
||||
Please follow the [IKS group ID storage setup](https://www.kubeflow.org/docs/ibm/deploy/install-kubeflow-on-iks/#ibm-cloud-group-id-storage-setup)
|
||||
before running the below commands.
|
||||
|
||||
```bash
|
||||
kubectl apply -k cluster-scoped-resources/
|
||||
kubectl wait crd/applications.app.k8s.io --for condition=established --timeout=60s
|
||||
kubectl apply -k env/platform-agnostic/
|
||||
kubectl wait pods -l application-crd-id=kubeflow-pipelines -n kubeflow --for condition=Ready --timeout=1800s
|
||||
kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8080:80
|
||||
```
|
||||
|
||||
## Uninstall
|
||||
|
||||
If the installation is based on CloudSQL/GCS, after the uninstall, the data is still there,
|
||||
reinstall a newer version can reuse the data.
|
||||
|
||||
```bash
|
||||
### 1. namespace scoped
|
||||
# Depends on how you installed it:
|
||||
kubectl kustomize env/platform-agnostic/ | kubectl delete -f -
|
||||
# or
|
||||
kubectl kustomize env/dev | kubectl delete -f -
|
||||
# or
|
||||
kubectl kustomize env/gcp | kubectl delete -f -
|
||||
# or
|
||||
kubectl delete applications/pipeline -n kubeflow
|
||||
|
||||
### 2. cluster scoped
|
||||
kubectl delete -k cluster-scoped-resources/
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Permission error installing Kubeflow Pipelines to a cluster
|
||||
|
||||
Run
|
||||
|
||||
```bash
|
||||
kubectl create clusterrolebinding your-binding --clusterrole=cluster-admin --user=[your-user-name]
|
||||
```
|
||||
|
||||
### Samples requires "user-gcp-sa" secret
|
||||
|
||||
If sample code requires a "user-gcp-sa" secret, you could create one by
|
||||
|
||||
- First download the GCE VM service account token
|
||||
[Document](https://cloud.google.com/iam/docs/creating-managing-service-account-keys#creating_service_account_keys)
|
||||
|
||||
```bash
|
||||
gcloud iam service-accounts keys create application_default_credentials.json \
|
||||
--iam-account [SA-NAME]@[PROJECT-ID].iam.gserviceaccount.com
|
||||
```
|
||||
|
||||
- Run
|
||||
|
||||
```bash
|
||||
kubectl create secret -n [your-namespace] generic user-gcp-sa --from-file=user-gcp-sa.json=application_default_credentials.json`
|
||||
```
|
||||
|
||||
## Folder Structure
|
||||
|
||||
### Overview
|
||||
|
||||
- User facing manifest entrypoints are `cluster-scoped-resources` package and `env/<env-name>` package.
|
||||
- `cluster-scoped-resources` should collect all cluster-scoped resources.
|
||||
- `env/<env-name>` should collect env specific namespace-scoped resources.
|
||||
- Note, for multi-user envs, they already included cluster-scoped resources.
|
||||
- KFP core components live in `base/<component-name>` folders.
|
||||
- If a component requires cluster-scoped resources, it should have a folder inside named `cluster-scoped` with related resources, but note that `base/<component-name>/kustomization.yaml` shouldn't include the `cluster-scoped` folder. `cluster-scoped` folders should be collected by top level `cluster-scoped-resources` folder.
|
||||
- KFP core installations are in `base/installs/<install-type>`, they only include the core KFP components, not third party ones.
|
||||
- Third party components live in `third-party/<component-name>` folders.
|
||||
|
||||
### For direct deployments
|
||||
|
||||
Env specific overlays live in `env/<env-name>` folders, they compose above components to get ready for directly deploying.
|
||||
|
||||
### For downstream consumers
|
||||
|
||||
Please compose `base/installs/<install-type>` and third party dependencies based on your own requirements.
|
||||
|
||||
### Rationale
|
||||
|
||||
Constraints for namespaced installation we need to comply with (that drove above structure):
|
||||
|
||||
- CRDs must be applied separately, because if we apply CRs in the same `kubectl apply` command, the CRD may not have been accepted by k8s api server (e.g. Application CRD).
|
||||
- [A Kubeflow 1.0 constraint](https://github.com/kubeflow/pipelines/issues/2884#issuecomment-577158715) is that we should separate cluster scoped resources from namespace scoped resources, because sometimes different roles are required to deploy them. Cluster scoped resources usually need a cluster admin role, while namespaced resources can be deployed by individual teams managing a namespace.
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
# Note, this application.yaml is not included by default for most environments.
|
||||
|
||||
apiVersion: app.k8s.io/v1beta1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: $(kfp-app-name)
|
||||
annotations:
|
||||
kubernetes-engine.cloud.google.com/icon: >-
|
||||
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADMAAAAyCAYAAADx/eOPAAALuUlEQVRogd2afWxd9XnHP99bK4pS3yhiGUIRiiJUVVVqbq8ppdR20ibqpuIMtDRkUYERp29J57gMZVuxsrZiK7oZXVv5re1AiOuoG+N1DMkuytprsGPEVMouxqQZJVHEWIdQlGVxZlmZdb/747zcc869jpMO+seOdPz7nd/L83tev89zzjX8Bq795Rq9o17zXp+Tey+Ijkyboela29DRWkhffyT733pH/Z3este9F2cC6N0kNjxtjD+FdRD8UF9X7u97y7UbQFPAivC0BdllS381slun3s3z3xVhhqeds90tqR/oMB7u68z19ZZra0E/l1if3WOziPx3skrDPTr+bvDxfxImEIJbgX6gGBJ7EfHJX/ySReDHwO9KYAenyWCMFKw21GSeslwa2Z17+TcuzPBRr7B8m6Df5oOJqdPAR/u6cm/2lmv3At+IT3GiXZqbcaxSLsfRoTsvn7XL2jE87ZXGnwf+VGiDY86ETM1wU1+XjvSW/RlgTJADQ2QaCZKWcX1/aDIcjE8i3SdzZLjn0lm8pJXD02417BM+gLmq2Rqjr/d16Vu95dp6wc8Ra5O8NrPIcoZCvIR1H+KZkd2qLcfnRYUZOuorJO+3uQt0RerolGYZR7r5+C9ZATwPviGyQprd6Liszy3bnwVKwGMjPbnFyxJmeNpX2T4gaR/QmmSpyYZTho/2depMb9k/kNh3KawuJ1bWauHzUcyXRpZAv5Zmg7aHBLcmNN9ECAFeAO3s69KZ3nLtDuF9dnBs0IT9JO24rbPb0JfP2syCZpFfE5q1mRWcvlgMNcwMTRq9z/+OWXdx4AGjvX1deqC37DbwPwOrMgsufol5mWMWs1ivEbjTrOCtLNNb+udygqsNbUBtopR/NkuuwTJ6Hxsw67KSuvH5MPDA/nJttfGTdUFCMUlp/ALwOtIs9muBxpnFnBzuSQf21oP/BbXclVvumWuTaDN8WNBm2GizJkxPM0CDMA2WGZ72bbb/Njue2TRj9Il/PcG87SeBz4ZTNaSTsmctHcO8SqDp14d7dCFLZ2v/3OpQ023Ah4n65kohvETUCdcsfmuilD+bpNdgGZvOODuHqYGIVGCec9g7+7o031v2jaBTiD0ysxbHRnZrPktzyz1zK7f0z10nh5pWwLRhvZro1KqznVJhNB8UyDeSsU4zAOiIXV1OuEqQ2AR79nflXgcY6dGLwIvR8q39cy1b+uc2Emo6dI824BpMSxz8iVhy4m/2WiYHdV5UmOHp2mpwm52ESCdwRn+9v0tPAWzpn9sAFAQbMdc60PaHsFZEWd9uxk4z8G3seykECfObTEd2KmuZG4CWyLXkYLMwtiYt+hMsTUdAEZQzjs9apv66SHJRk73ZjBQ+iRu29s+1VEr5OImmXs4MHUahVoLWgK23wbv6OrU4OulcuHYehWsVHhpXwpE2FNRayTszX2cwDpQEzTB+QvrJHCXUaigk+c++aXZiE98YmUVgV19X7u3ypH/fgfUA5h2usY2jNjmWoGVn50nvC9T2NviA5OPBGPW91OlG+0Xa1WJhhqadk3WjpKCilQIQFP19XZocnfIHgIeFWyNh6goXyX6gdNWfU8aJ5tNjEheAHZVS/ruGj0s8k6VPhh6ms6kwgoLl1aGuCEuSpwXfHZ2qrTJ+HHkNCpOjmbdFcEcGUIhUSj/H65rPO6j+766U8i/QXV0z8cqJc4btwF8AtWgtMb1wj+j41Df/s1EYQwdEDiqM3hDes9quGY3IKoYOvCrU7HlCoZtEWapPkzEpsU8uq8b36a6uBqaBv5l45URLpZT/pmGH8LnkvlAdAOt1oeXqRsuYTjlEMJiXvWN/Z+5szfqioKcOKo7qr/nAEesKiOyv2A/q88rOx8+8bPhK5dUTAA8jbUT6MuKnbKteNVHKP23xCeD1LC0F2TWOmzoAKEiWxmC+sr8rN1OerF2HGaqXFcZhDWaYj11S4ZxcXxVqyKqPZOeNTwM7Jkr5BeDPQJ8NFQaoC/gZ26rXT5TyxxAfRx6P94d0gU0pYYama+tsbwix/AHM4fKUrwAeB68kRJ5AZsWWieGTjLipsVCgrKCwKHF7pZQ/RXf104j76i4ZMmquxkzRXb2zUsqfxdxsfCiA70hRjZbpCDHmJcRdeZPDHkVck0Ul5PeHZ81DgHxKtglXaHCxVN9fr5TyR9hW3QA8Amqp5526SyKtBEbZVv1eZeZkbqKU7xfsFJwPqRW29s+11oUxnUhnkHf2dWoB+R5Jv5dNaGHh1wog8d/ZAI+0GgVpFPTp4AfJT2Hup7u6EvMk0tpkboutEz0HMPzHyD+mu3pFpZR/Aug0Pgm0RLkvFzLWYfjDvs7cqfKUt2LuXTLhue5mdWhVDJdEzxDDcRKawceN9lRePVkDfgBcR/LKVqNpz/s08DO6q4VKKT8j8zHgJ1HyzA1P11YZjfV1arw85auBR4RalDB5lEjDKi0CgPPphKZ0QiNRwUQeg88B2ydKreew9yH1NCxe/r4GaZpt1Vsrh/JnDDcBLwPkbLVgf6s86RXYj4KvtJKJM8KsGLkSlsmUL6mSg1RJY1xD7KmU8sfprnYgBqJsGVsiEfupsca7FfMo26p/OfHKiVqllB8HyPV16VxfV66G/G1QBwY5xvCgTT7X3/MTaBbFVr0fJvqw2ASZ+yul/FN0V68CHsesiDl3UopM3CwhDZDD/Dnwj3S/sjoYAMqTtc1YX02jVqYOiuuqsAKIkqZCfFIz/IrfFY8gDrKt2gI8irSuwQezyTeNaOl+6qYb+fpYGKEXJE9GSTObK5ItrheaLHE5/XRKcHul+kYN8x2kzWlLNNuVtUqibzKW5CBjxUoszO7NWrS1E/xWvMeJjck2WQHEKJeMD+qH4gWCSvg00m3AVxv5TMRKsp9Cs0Q/Ka/1BOZQNBSXMz2b9Q5oO9JCKgkqg2aKofl8uvTPeE1w3t5KKf8y26pFxINhLRa5R9JV6huT/aZuFu7Ds+A9jBdj+VIvZz2b9BL2Xi5yJQEgUFqinI9SZBDx358o5Q/HiRGtquOEmxJu6DcbC/afQWxnvHg+Odrwm2bP5txh5OEYjOM3vaiu8qqHJw1mPmK/Xs7HJf0LRncDMF5cAL6NWUxDrX/duwbczljxjSzvTX+gtXU3MBlrRCltrsxBTgorACKrRGf5bczOiVLrhUL74B2F9oHVjBd/iLwTWEhr+CIWaLYumDjIWLHha+aSwvRs1iJmJ9Kb9ZJRETS3ACsMC8i1ZNwgXZDYWTmU/1WhfeAW8Cjo+UL7wDrGik8jfid0kYz/Z2ODepv+GPIY+FAznpcUJhAo9w5mh81CFtEsWieCTzwXkogmfKBSyh8ttA98EDPqoPouYqYLxYEPMVY8itmEeTM+KEaqZhVAkiPPIL6QDPhLFiYQSC9J7M3mGlF/24zWSvwIM1xoH2gF/sFiTcSPxQakqUJxsIPx4jGCr0AzCUYTbROJ7DPAdsbSAX9ZwgDs3qTDiMGUOxF/1DgfekLVsPf0sw8DPARsDNwy8iYBXov4p0L7wC2MF99CfBJ4rqmbJbO/qYE+x1jx5HK8Xtp/aFgHDM/FX+RM9FFjHjjj4NV3HvlPsP4g+SqQgm6zCuvJQnHgi4wVz2JuAj8RnLGEVaCf8Y8cuRQ2L0mYEBB2Gb8ZHKD4NQBx+0Qpf7LQPrAVVGqiiWTpCcEn4QcLxcF7C7+aXMDahT1YX5IS5DHE/ZfC4yULEwr0DtIOWwuuvwZ8rVLKP1soDqzHPGJoyRao9b4SXiQQ30A8eO1/PJ8D7gK+BtQSJcQM8AXGlg747LUkmi91lad8J3CuZ5OeBii0D64ET2FdH1N0omWJvgLPkvwM8LmZf7lrnm3VO4CHsM4DH2P8I8vGSfK67P9q8v9wWPAcQLH4PbBHbK6Pq+3M9+Ml+6FL2dyC+WmhOLiWseKPMDeDd12uIPBrWCZ5Xds++AHsAwGlBKnoB5747c2J+aSJEuvRL8CDv/2Zz+cqh/LL/gPD//vrfwFjcI5oX6jDBwAAAABJRU5ErkJggg==
|
||||
marketplace.cloud.google.com/deploy-info: '{"partner_id": "google-cloud-ai-platform", "product_id": "kubeflow-pipelines", "partner_name": "Google Cloud AI Platform"}'
|
||||
spec:
|
||||
addOwnerRef: true
|
||||
selector:
|
||||
matchLabels:
|
||||
application-crd-id: kubeflow-pipelines
|
||||
descriptor:
|
||||
version: $(kfp-app-version)
|
||||
type: Kubeflow Pipelines
|
||||
description: |-
|
||||
Reusable end-to-end ML workflow
|
||||
maintainers:
|
||||
- name: Google Cloud AI Platform
|
||||
url: https://cloud.google.com/ai-platform/
|
||||
- name: Kubeflow Pipelines
|
||||
url: https://github.com/kubeflow/pipelines
|
||||
links:
|
||||
- description: 'Kubeflow Pipelines Documentation'
|
||||
url: https://www.kubeflow.org/docs/pipelines/
|
||||
notes: |-
|
||||
Please go to [Hosted Kubeflow Pipelines Console](https://console.cloud.google.com/ai-platform/pipelines/clusters).
|
||||
|
||||
info:
|
||||
- name: Console
|
||||
value: 'https://console.cloud.google.com/ai-platform/pipelines/clusters'
|
||||
componentKinds:
|
||||
- group: v1
|
||||
kind: ServiceAccount
|
||||
- group: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
- group: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
- group: v1
|
||||
kind: Service
|
||||
- group: v1
|
||||
kind: ConfigMap
|
||||
- group: v1
|
||||
kind: Secret
|
||||
- group: apps/v1
|
||||
kind: Deployment
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- application.yaml
|
||||
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: cache-deployer-deployment
|
||||
labels:
|
||||
app: cache-deployer
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: cache-deployer
|
||||
strategy:
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: cache-deployer
|
||||
spec:
|
||||
containers:
|
||||
- name: main
|
||||
image: gcr.io/ml-pipeline/cache-deployer:dummy
|
||||
imagePullPolicy: Always
|
||||
env:
|
||||
- name: NAMESPACE_TO_WATCH
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
serviceAccountName: kubeflow-pipelines-cache-deployer-sa
|
||||
restartPolicy: Always
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
labels:
|
||||
app: kubeflow-pipelines-cache-deployer-role
|
||||
name: kubeflow-pipelines-cache-deployer-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- secrets
|
||||
verbs:
|
||||
- create
|
||||
- delete
|
||||
- get
|
||||
- patch
|
||||
- list
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: kubeflow-pipelines-cache-deployer-rolebinding
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: kubeflow-pipelines-cache-deployer-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kubeflow-pipelines-cache-deployer-sa
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
labels:
|
||||
app: kubeflow-pipelines-cache-deployer-clusterrole
|
||||
name: kubeflow-pipelines-cache-deployer-clusterrole
|
||||
rules:
|
||||
- apiGroups:
|
||||
- certificates.k8s.io
|
||||
resources:
|
||||
- certificatesigningrequests
|
||||
- certificatesigningrequests/approval
|
||||
verbs:
|
||||
- create
|
||||
- delete
|
||||
- get
|
||||
- update
|
||||
- apiGroups:
|
||||
- admissionregistration.k8s.io
|
||||
resources:
|
||||
- mutatingwebhookconfigurations
|
||||
verbs:
|
||||
- create
|
||||
- delete
|
||||
- get
|
||||
- list
|
||||
- patch
|
||||
- apiGroups:
|
||||
- certificates.k8s.io
|
||||
resources:
|
||||
- signers
|
||||
resourceNames:
|
||||
- kubernetes.io/*
|
||||
verbs:
|
||||
- approve
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: kubeflow-pipelines-cache-deployer-clusterrolebinding
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: kubeflow-pipelines-cache-deployer-clusterrole
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kubeflow-pipelines-cache-deployer-sa
|
||||
# namespace will be added by kustomize automatically according to the namespace field in kustomization.yaml
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: kubeflow-pipelines-cache-deployer-sa
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- cache-deployer-clusterrole.yaml
|
||||
- cache-deployer-clusterrolebinding.yaml
|
||||
# HACK: although a service account(SA) is not a cluster-scoped resource.
|
||||
# Presence of a SA referred by a clusterrolebinding allows kustomize to auto-add
|
||||
# namespace for the clusterrolebinding's SA ref.
|
||||
- cache-deployer-sa.yaml
|
||||
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
bases:
|
||||
- cluster-scoped
|
||||
resources:
|
||||
- cache-deployer-role.yaml
|
||||
- cache-deployer-rolebinding.yaml
|
||||
- cache-deployer-deployment.yaml
|
||||
commonLabels:
|
||||
app: cache-deployer
|
||||
images:
|
||||
- name: gcr.io/ml-pipeline/cache-deployer
|
||||
newTag: 1.5.0-rc.2
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: cache-server
|
||||
labels:
|
||||
app: cache-server
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: cache-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: cache-server
|
||||
spec:
|
||||
containers:
|
||||
- name: server
|
||||
image: gcr.io/ml-pipeline/cache-server:dummy
|
||||
env:
|
||||
- name: CACHE_IMAGE
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: pipeline-install-config
|
||||
key: cacheImage
|
||||
- name: DBCONFIG_DRIVER
|
||||
value: mysql
|
||||
- name: DBCONFIG_DB_NAME
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: pipeline-install-config
|
||||
key: cacheDb
|
||||
- name: DBCONFIG_HOST_NAME
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: pipeline-install-config
|
||||
key: dbHost
|
||||
- name: DBCONFIG_PORT
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: pipeline-install-config
|
||||
key: dbPort
|
||||
- name: DBCONFIG_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-secret
|
||||
key: username
|
||||
- name: DBCONFIG_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-secret
|
||||
key: password
|
||||
- name: NAMESPACE_TO_WATCH
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
args: ["--db_driver=$(DBCONFIG_DRIVER)",
|
||||
"--db_host=$(DBCONFIG_HOST_NAME)",
|
||||
"--db_port=$(DBCONFIG_PORT)",
|
||||
"--db_name=$(DBCONFIG_DB_NAME)",
|
||||
"--db_user=$(DBCONFIG_USER)",
|
||||
"--db_password=$(DBCONFIG_PASSWORD)",
|
||||
"--namespace_to_watch=$(NAMESPACE_TO_WATCH)",
|
||||
]
|
||||
imagePullPolicy: Always
|
||||
ports:
|
||||
- containerPort: 8443
|
||||
name: webhook-api
|
||||
volumeMounts:
|
||||
- name: webhook-tls-certs
|
||||
mountPath: /etc/webhook/certs
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: webhook-tls-certs
|
||||
secret:
|
||||
secretName: webhook-server-tls
|
||||
serviceAccountName: kubeflow-pipelines-cache
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
labels:
|
||||
app: kubeflow-pipelines-cache-role
|
||||
name: kubeflow-pipelines-cache-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- pods
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- argoproj.io
|
||||
resources:
|
||||
- workflows
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- apiGroups:
|
||||
- tekton.dev
|
||||
resources:
|
||||
- taskruns
|
||||
- taskruns/status
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: kubeflow-pipelines-cache-binding
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: kubeflow-pipelines-cache-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kubeflow-pipelines-cache
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: kubeflow-pipelines-cache
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: cache-server
|
||||
spec:
|
||||
selector:
|
||||
app: cache-server
|
||||
ports:
|
||||
- port: 443
|
||||
targetPort: webhook-api
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- cache-deployment.yaml
|
||||
- cache-service.yaml
|
||||
- cache-role.yaml
|
||||
- cache-rolebinding.yaml
|
||||
- cache-sa.yaml
|
||||
commonLabels:
|
||||
app: cache-server
|
||||
images:
|
||||
- name: gcr.io/ml-pipeline/cache-server
|
||||
newTag: 1.5.0-rc.2
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
bases:
|
||||
- ../../pipeline
|
||||
- ../../cache
|
||||
- ../../cache-deployer
|
||||
|
||||
resources:
|
||||
- pipeline-install-config.yaml
|
||||
- mysql-secret.yaml
|
||||
|
||||
|
||||
images:
|
||||
- name: gcr.io/ml-pipeline/api-server
|
||||
newName: docker.io/aipipeline/api-server
|
||||
newTag: latest
|
||||
- name: gcr.io/ml-pipeline/persistenceagent
|
||||
newName: docker.io/aipipeline/persistenceagent
|
||||
newTag: latest
|
||||
- name: gcr.io/ml-pipeline/frontend
|
||||
newName: docker.io/aipipeline/frontend
|
||||
newTag: latest
|
||||
- name: gcr.io/ml-pipeline/metadata-writer
|
||||
newName: docker.io/aipipeline/metadata-writer
|
||||
newTag: latest
|
||||
- name: gcr.io/ml-pipeline/scheduledworkflow
|
||||
newName: docker.io/aipipeline/scheduledworkflow
|
||||
newTag: latest
|
||||
- name: gcr.io/ml-pipeline/cache-server
|
||||
newName: docker.io/aipipeline/cache-server
|
||||
newTag: latest
|
||||
|
||||
# Used by Kustomize
|
||||
vars:
|
||||
- name: kfp-namespace
|
||||
objref:
|
||||
kind: Deployment
|
||||
apiVersion: apps/v1
|
||||
name: ml-pipeline
|
||||
fieldref:
|
||||
fieldpath: metadata.namespace
|
||||
- name: kfp-app-name
|
||||
objref:
|
||||
kind: ConfigMap
|
||||
name: pipeline-install-config
|
||||
apiVersion: v1
|
||||
fieldref:
|
||||
fieldpath: data.appName
|
||||
- name: kfp-app-version
|
||||
objref:
|
||||
kind: ConfigMap
|
||||
name: pipeline-install-config
|
||||
apiVersion: v1
|
||||
fieldref:
|
||||
fieldpath: data.appVersion
|
||||
- name: kfp-artifact-bucket-name
|
||||
objref:
|
||||
kind: ConfigMap
|
||||
name: pipeline-install-config
|
||||
apiVersion: v1
|
||||
fieldref:
|
||||
fieldpath: data.bucketName
|
||||
configurations:
|
||||
- params.yaml
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
kind: Secret
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: mysql-secret
|
||||
stringData:
|
||||
username: root
|
||||
password: ""
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
# Allow Kustomize var to replace following fields.
|
||||
varReference:
|
||||
- path: data/config
|
||||
kind: ConfigMap
|
||||
- path: metadata/name
|
||||
kind: Application
|
||||
- path: spec/descriptor/version
|
||||
kind: Application
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: pipeline-install-config
|
||||
data:
|
||||
appName: pipeline
|
||||
appVersion: 1.5.0-rc.2
|
||||
dbHost: mysql
|
||||
dbPort: "3306"
|
||||
mlmdDb: metadb
|
||||
cacheDb: cachedb
|
||||
pipelineDb: mlpipeline
|
||||
bucketName: mlpipeline
|
||||
## autoUpdatePipelineDefaultVersion: States if the pipeline version
|
||||
## should be updated by defult for a versioned pipeline or not when a new
|
||||
## version is uploaded. This sets the deployment wide definition.
|
||||
autoUpdatePipelineDefaultVersion: "true"
|
||||
## cronScheduleTimezone: States the timezone which should be used for
|
||||
## the cron scheduler. If not specified the local timezone of the
|
||||
## cluster will be used. Valid values are UTC, Local or values according to
|
||||
## the IANA Time Zone database, such as "America/New_York" and "Asia/Shanghai".
|
||||
## Feature stage:
|
||||
## [Alpha](https://github.com/kubeflow/pipelines/blob/07328e5094ac2981d3059314cc848fbb71437a76/docs/release/feature-stages.md#alpha)
|
||||
cronScheduleTimezone: "UTC"
|
||||
## cacheImage is the image that the mutating webhook will use to patch
|
||||
## cached steps with. Will be used to echo a message announcing that
|
||||
## the cached step result will be used. If not set it will default to
|
||||
## 'gcr.io/google-containers/busybox'
|
||||
cacheImage: "gcr.io/google-containers/busybox"
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: ml-pipeline
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: ml-pipeline
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: ml-pipeline
|
||||
|
|
@ -0,0 +1,63 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: ml-pipeline
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- pods
|
||||
- pods/log
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- delete
|
||||
- apiGroups:
|
||||
- argoproj.io
|
||||
resources:
|
||||
- workflows
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
- apiGroups:
|
||||
- kubeflow.org
|
||||
resources:
|
||||
- scheduledworkflows
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
- apiGroups:
|
||||
- authorization.k8s.io
|
||||
resources:
|
||||
- subjectaccessreviews
|
||||
verbs:
|
||||
- create
|
||||
- apiGroups:
|
||||
- authentication.k8s.io
|
||||
resources:
|
||||
- tokenreviews
|
||||
verbs:
|
||||
- create
|
||||
- apiGroups:
|
||||
- tekton.dev
|
||||
resources:
|
||||
- pipelineruns
|
||||
- taskruns
|
||||
- conditions
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ml-pipeline
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: ml-pipeline-api-server
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: pipeline-api-server-config
|
||||
env:
|
||||
- name: KUBEFLOW_USERID_HEADER
|
||||
value: kubeflow-userid
|
||||
- name: KUBEFLOW_USERID_PREFIX
|
||||
value: ""
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- cluster-role-binding.yaml
|
||||
- cluster-role.yaml
|
||||
configMapGenerator:
|
||||
- name: pipeline-api-server-config
|
||||
envs:
|
||||
- params.env
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
MULTIUSER=true
|
||||
DEFAULTPIPELINERUNNERSERVICEACCOUNT=default-editor
|
||||
VISUALIZATIONSERVICE_NAME=ml-pipeline-visualizationserver
|
||||
VISUALIZATIONSERVICE_PORT=8888
|
||||
11
apps/kfp-tekton/upstream/base/installs/multi-user/cache/cluster-role-binding.yaml
vendored
Normal file
11
apps/kfp-tekton/upstream/base/installs/multi-user/cache/cluster-role-binding.yaml
vendored
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: kubeflow-pipelines-cache-binding
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: kubeflow-pipelines-cache-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kubeflow-pipelines-cache
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: kubeflow-pipelines-cache-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- pods
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- argoproj.io
|
||||
resources:
|
||||
- workflows
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
13
apps/kfp-tekton/upstream/base/installs/multi-user/cache/deployment-patch.yaml
vendored
Normal file
13
apps/kfp-tekton/upstream/base/installs/multi-user/cache/deployment-patch.yaml
vendored
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: cache-server
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: server
|
||||
env:
|
||||
- name: NAMESPACE_TO_WATCH
|
||||
value: ''
|
||||
valueFrom: null
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
commonLabels:
|
||||
app: cache-server
|
||||
resources:
|
||||
- cluster-role.yaml
|
||||
- cluster-role-binding.yaml
|
||||
|
|
@ -0,0 +1,115 @@
|
|||
apiVersion: security.istio.io/v1beta1
|
||||
kind: AuthorizationPolicy
|
||||
metadata:
|
||||
name: ml-pipeline-ui
|
||||
namespace: kubeflow
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: ml-pipeline-ui
|
||||
rules:
|
||||
- from:
|
||||
- source:
|
||||
namespaces:
|
||||
- istio-system
|
||||
---
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: AuthorizationPolicy
|
||||
metadata:
|
||||
name: ml-pipeline
|
||||
namespace: kubeflow
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: ml-pipeline
|
||||
rules:
|
||||
- from:
|
||||
- source:
|
||||
principals:
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline-ui
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline-persistenceagent
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline-scheduledworkflow
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline-viewer-crd-service-account
|
||||
- cluster.local/ns/kubeflow/sa/kubeflow-pipelines-cache
|
||||
# For user workloads, which cannot user http headers for authentication
|
||||
- when:
|
||||
- key: request.headers[kubeflow-userid]
|
||||
notValues: ['*']
|
||||
---
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: AuthorizationPolicy
|
||||
metadata:
|
||||
name: ml-pipeline-visualizationserver
|
||||
namespace: kubeflow
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: ml-pipeline-visualizationserver
|
||||
rules:
|
||||
- from:
|
||||
- source:
|
||||
principals:
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline-ui
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline-persistenceagent
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline-scheduledworkflow
|
||||
- cluster.local/ns/kubeflow/sa/ml-pipeline-viewer-crd-service-account
|
||||
- cluster.local/ns/kubeflow/sa/kubeflow-pipelines-cache
|
||||
|
||||
---
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: AuthorizationPolicy
|
||||
metadata:
|
||||
name: service-cache-server
|
||||
namespace: kubeflow
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: cache-server
|
||||
rules:
|
||||
- {}
|
||||
|
||||
---
|
||||
apiVersion: security.istio.io/v1beta1
|
||||
kind: AuthorizationPolicy
|
||||
metadata:
|
||||
name: metadata-grpc-service
|
||||
spec:
|
||||
action: ALLOW
|
||||
selector:
|
||||
matchLabels:
|
||||
component: metadata-grpc-server
|
||||
rules:
|
||||
- {}
|
||||
|
||||
---
|
||||
apiVersion: "networking.istio.io/v1alpha3"
|
||||
kind: DestinationRule
|
||||
metadata:
|
||||
name: ml-pipeline-ui
|
||||
spec:
|
||||
host: ml-pipeline-ui.kubeflow.svc.cluster.local
|
||||
trafficPolicy:
|
||||
tls:
|
||||
mode: ISTIO_MUTUAL
|
||||
---
|
||||
apiVersion: "networking.istio.io/v1alpha3"
|
||||
kind: DestinationRule
|
||||
metadata:
|
||||
name: ml-pipeline
|
||||
spec:
|
||||
host: ml-pipeline.kubeflow.svc.cluster.local
|
||||
trafficPolicy:
|
||||
tls:
|
||||
mode: ISTIO_MUTUAL
|
||||
---
|
||||
apiVersion: "networking.istio.io/v1alpha3"
|
||||
kind: DestinationRule
|
||||
metadata:
|
||||
name: ml-pipeline-visualizationserver
|
||||
spec:
|
||||
host: ml-pipeline-visualizationserver.kubeflow.svc.cluster.local
|
||||
trafficPolicy:
|
||||
tls:
|
||||
mode: ISTIO_MUTUAL
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
commonLabels:
|
||||
app.kubernetes.io/name: kubeflow-pipelines
|
||||
app.kubernetes.io/component: ml-pipeline
|
||||
resources:
|
||||
- ../generic
|
||||
- view-edit-cluster-roles.yaml
|
||||
- api-service
|
||||
- pipelines-ui
|
||||
- pipelines-profile-controller
|
||||
- scheduled-workflow
|
||||
- viewer-controller
|
||||
- persistence-agent
|
||||
- cache
|
||||
- metadata-writer
|
||||
- istio-authorization-config.yaml
|
||||
- virtual-service.yaml
|
||||
patchesStrategicMerge:
|
||||
- api-service/deployment-patch.yaml
|
||||
- pipelines-ui/deployment-patch.yaml
|
||||
- pipelines-ui/configmap-patch.yaml
|
||||
- scheduled-workflow/deployment-patch.yaml
|
||||
- viewer-controller/deployment-patch.yaml
|
||||
- persistence-agent/deployment-patch.yaml
|
||||
- metadata-writer/deployment-patch.yaml
|
||||
- cache/deployment-patch.yaml
|
||||
|
||||
configurations:
|
||||
- params.yaml
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: kubeflow-pipelines-metadata-writer-binding
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: kubeflow-pipelines-metadata-writer-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kubeflow-pipelines-metadata-writer
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: kubeflow-pipelines-metadata-writer-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- pods
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- argoproj.io
|
||||
resources:
|
||||
- workflows
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- apiGroups:
|
||||
- tekton.dev
|
||||
resources:
|
||||
- pipelineruns
|
||||
- taskruns
|
||||
- conditions
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: metadata-writer
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: main
|
||||
env:
|
||||
- name: NAMESPACE_TO_WATCH
|
||||
value: ''
|
||||
valueFrom: null
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- cluster-role.yaml
|
||||
- cluster-role-binding.yaml
|
||||
|
|
@ -1,5 +1,4 @@
|
|||
# Allow Kustomize var to replace following fields.
|
||||
varReference:
|
||||
- path: data/config
|
||||
kind: ConfigMap
|
||||
- path: spec/http/route/destination/host
|
||||
kind: VirtualService
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: ml-pipeline-persistenceagent-binding
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: ml-pipeline-persistenceagent-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: ml-pipeline-persistenceagent
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: ml-pipeline-persistenceagent-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- argoproj.io
|
||||
resources:
|
||||
- workflows
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- kubeflow.org
|
||||
resources:
|
||||
- scheduledworkflows
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- tekton.dev
|
||||
resources:
|
||||
- pipelineruns
|
||||
- taskruns
|
||||
- conditions
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ml-pipeline-persistenceagent
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: ml-pipeline-persistenceagent
|
||||
env:
|
||||
- name: NAMESPACE
|
||||
value: ''
|
||||
valueFrom: null
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- cluster-role.yaml
|
||||
- cluster-role-binding.yaml
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
apiVersion: metacontroller.k8s.io/v1alpha1
|
||||
kind: CompositeController
|
||||
metadata:
|
||||
name: kubeflow-pipelines-profile-controller
|
||||
spec:
|
||||
generateSelector: true
|
||||
resyncPeriodSeconds: 10
|
||||
parentResource:
|
||||
apiVersion: v1
|
||||
resource: namespaces
|
||||
childResources:
|
||||
- apiVersion: v1
|
||||
resource: secrets
|
||||
updateStrategy:
|
||||
method: OnDelete
|
||||
- apiVersion: v1
|
||||
resource: configmaps
|
||||
updateStrategy:
|
||||
method: OnDelete
|
||||
- apiVersion: apps/v1
|
||||
resource: deployments
|
||||
updateStrategy:
|
||||
method: InPlace
|
||||
- apiVersion: v1
|
||||
resource: services
|
||||
updateStrategy:
|
||||
method: InPlace
|
||||
- apiVersion: networking.istio.io/v1alpha3
|
||||
resource: destinationrules
|
||||
updateStrategy:
|
||||
method: InPlace
|
||||
- apiVersion: security.istio.io/v1beta1
|
||||
resource: authorizationpolicies
|
||||
updateStrategy:
|
||||
method: InPlace
|
||||
hooks:
|
||||
sync:
|
||||
webhook:
|
||||
url: http://kubeflow-pipelines-profile-controller/sync
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: kubeflow-pipelines-profile-controller
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
sidecar.istio.io/inject: "false"
|
||||
spec:
|
||||
containers:
|
||||
- name: profile-controller
|
||||
image: python:3.7
|
||||
command: ["python", "/hooks/sync.py"]
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: kubeflow-pipelines-profile-controller-env
|
||||
env:
|
||||
- name: KFP_VERSION
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: pipeline-install-config
|
||||
key: appVersion
|
||||
- name: MINIO_ACCESS_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mlpipeline-minio-artifact
|
||||
key: accesskey
|
||||
- name: MINIO_SECRET_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mlpipeline-minio-artifact
|
||||
key: secretkey
|
||||
volumeMounts:
|
||||
- name: hooks
|
||||
mountPath: /hooks
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
volumes:
|
||||
- name: hooks
|
||||
configMap:
|
||||
name: kubeflow-pipelines-profile-controller-code
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
commonLabels:
|
||||
app: kubeflow-pipelines-profile-controller
|
||||
resources:
|
||||
- service.yaml
|
||||
- deployment.yaml
|
||||
- composite-controller.yaml
|
||||
configMapGenerator:
|
||||
- name: kubeflow-pipelines-profile-controller-code
|
||||
files:
|
||||
- sync.py
|
||||
- name: kubeflow-pipelines-profile-controller-env
|
||||
envs:
|
||||
- params.env
|
||||
|
|
@ -0,0 +1 @@
|
|||
DISABLE_ISTIO_SIDECAR=false
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: kubeflow-pipelines-profile-controller
|
||||
spec:
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
protocol: TCP
|
||||
targetPort: 8080
|
||||
|
|
@ -0,0 +1,285 @@
|
|||
# Copyright 2020-2021 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
import json
|
||||
import os
|
||||
import base64
|
||||
|
||||
kfp_version = os.environ["KFP_VERSION"]
|
||||
disable_istio_sidecar = os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
|
||||
mlpipeline_minio_access_key = base64.b64encode(
|
||||
bytes(os.environ.get("MINIO_ACCESS_KEY"), 'utf-8')).decode('utf-8')
|
||||
mlpipeline_minio_secret_key = base64.b64encode(
|
||||
bytes(os.environ.get("MINIO_SECRET_KEY"), 'utf-8')).decode('utf-8')
|
||||
|
||||
|
||||
class Controller(BaseHTTPRequestHandler):
|
||||
def sync(self, parent, children):
|
||||
pipeline_enabled = parent.get("metadata", {}).get(
|
||||
"labels", {}).get("pipelines.kubeflow.org/enabled")
|
||||
|
||||
if pipeline_enabled != "true":
|
||||
return {"status": {}, "children": []}
|
||||
|
||||
# Compute status based on observed state.
|
||||
desired_status = {
|
||||
"kubeflow-pipelines-ready": \
|
||||
len(children["Secret.v1"]) == 1 and \
|
||||
len(children["ConfigMap.v1"]) == 1 and \
|
||||
len(children["Deployment.apps/v1"]) == 2 and \
|
||||
len(children["Service.v1"]) == 2 and \
|
||||
len(children["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and \
|
||||
len(children["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and \
|
||||
"True" or "False"
|
||||
}
|
||||
|
||||
# Generate the desired child object(s).
|
||||
# parent is a namespace
|
||||
namespace = parent.get("metadata", {}).get("name")
|
||||
desired_resources = [
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "ConfigMap",
|
||||
"metadata": {
|
||||
"name": "metadata-grpc-configmap",
|
||||
"namespace": namespace,
|
||||
},
|
||||
"data": {
|
||||
"METADATA_GRPC_SERVICE_HOST":
|
||||
"metadata-grpc-service.kubeflow",
|
||||
"METADATA_GRPC_SERVICE_PORT": "8080",
|
||||
},
|
||||
},
|
||||
# Visualization server related manifests below
|
||||
{
|
||||
"apiVersion": "apps/v1",
|
||||
"kind": "Deployment",
|
||||
"metadata": {
|
||||
"labels": {
|
||||
"app": "ml-pipeline-visualizationserver"
|
||||
},
|
||||
"name": "ml-pipeline-visualizationserver",
|
||||
"namespace": namespace,
|
||||
},
|
||||
"spec": {
|
||||
"selector": {
|
||||
"matchLabels": {
|
||||
"app": "ml-pipeline-visualizationserver"
|
||||
},
|
||||
},
|
||||
"template": {
|
||||
"metadata": {
|
||||
"labels": {
|
||||
"app": "ml-pipeline-visualizationserver"
|
||||
},
|
||||
"annotations": disable_istio_sidecar and {
|
||||
"sidecar.istio.io/inject": "false"
|
||||
} or {},
|
||||
},
|
||||
"spec": {
|
||||
"containers": [{
|
||||
"image":
|
||||
"gcr.io/ml-pipeline/visualization-server:" +
|
||||
kfp_version,
|
||||
"imagePullPolicy":
|
||||
"IfNotPresent",
|
||||
"name":
|
||||
"ml-pipeline-visualizationserver",
|
||||
"ports": [{
|
||||
"containerPort": 8888
|
||||
}],
|
||||
"resources": {
|
||||
"requests": {
|
||||
"cpu": "50m",
|
||||
"memory": "200Mi"
|
||||
},
|
||||
"limits": {
|
||||
"cpu": "500m",
|
||||
"memory": "1Gi"
|
||||
},
|
||||
}
|
||||
}],
|
||||
"serviceAccountName":
|
||||
"default-editor",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
"apiVersion": "networking.istio.io/v1alpha3",
|
||||
"kind": "DestinationRule",
|
||||
"metadata": {
|
||||
"name": "ml-pipeline-visualizationserver",
|
||||
"namespace": namespace,
|
||||
},
|
||||
"spec": {
|
||||
"host": "ml-pipeline-visualizationserver",
|
||||
"trafficPolicy": {
|
||||
"tls": {
|
||||
"mode": "ISTIO_MUTUAL"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"apiVersion": "security.istio.io/v1beta1",
|
||||
"kind": "AuthorizationPolicy",
|
||||
"metadata": {
|
||||
"name": "ml-pipeline-visualizationserver",
|
||||
"namespace": namespace,
|
||||
},
|
||||
"spec": {
|
||||
"selector": {
|
||||
"matchLabels": {
|
||||
"app": "ml-pipeline-visualizationserver"
|
||||
}
|
||||
},
|
||||
"rules": [{
|
||||
"from": [{
|
||||
"source": {
|
||||
"principals": ["cluster.local/ns/kubeflow/sa/ml-pipeline"]
|
||||
}
|
||||
}]
|
||||
}]
|
||||
}
|
||||
},
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "Service",
|
||||
"metadata": {
|
||||
"name": "ml-pipeline-visualizationserver",
|
||||
"namespace": namespace,
|
||||
},
|
||||
"spec": {
|
||||
"ports": [{
|
||||
"name": "http",
|
||||
"port": 8888,
|
||||
"protocol": "TCP",
|
||||
"targetPort": 8888,
|
||||
}],
|
||||
"selector": {
|
||||
"app": "ml-pipeline-visualizationserver",
|
||||
},
|
||||
},
|
||||
},
|
||||
# Artifact fetcher related resources below.
|
||||
{
|
||||
"apiVersion": "apps/v1",
|
||||
"kind": "Deployment",
|
||||
"metadata": {
|
||||
"labels": {
|
||||
"app": "ml-pipeline-ui-artifact"
|
||||
},
|
||||
"name": "ml-pipeline-ui-artifact",
|
||||
"namespace": namespace,
|
||||
},
|
||||
"spec": {
|
||||
"selector": {
|
||||
"matchLabels": {
|
||||
"app": "ml-pipeline-ui-artifact"
|
||||
}
|
||||
},
|
||||
"template": {
|
||||
"metadata": {
|
||||
"labels": {
|
||||
"app": "ml-pipeline-ui-artifact"
|
||||
},
|
||||
"annotations": disable_istio_sidecar and {
|
||||
"sidecar.istio.io/inject": "false"
|
||||
} or {},
|
||||
},
|
||||
"spec": {
|
||||
"containers": [{
|
||||
"name":
|
||||
"ml-pipeline-ui-artifact",
|
||||
"image":
|
||||
"gcr.io/ml-pipeline/frontend:" + kfp_version,
|
||||
"imagePullPolicy":
|
||||
"IfNotPresent",
|
||||
"ports": [{
|
||||
"containerPort": 3000
|
||||
}],
|
||||
"resources": {
|
||||
"requests": {
|
||||
"cpu": "10m",
|
||||
"memory": "70Mi"
|
||||
},
|
||||
"limits": {
|
||||
"cpu": "100m",
|
||||
"memory": "500Mi"
|
||||
},
|
||||
}
|
||||
}],
|
||||
"serviceAccountName":
|
||||
"default-editor"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "Service",
|
||||
"metadata": {
|
||||
"name": "ml-pipeline-ui-artifact",
|
||||
"namespace": namespace,
|
||||
"labels": {
|
||||
"app": "ml-pipeline-ui-artifact"
|
||||
}
|
||||
},
|
||||
"spec": {
|
||||
"ports": [{
|
||||
"name":
|
||||
"http", # name is required to let istio understand request protocol
|
||||
"port": 80,
|
||||
"protocol": "TCP",
|
||||
"targetPort": 3000
|
||||
}],
|
||||
"selector": {
|
||||
"app": "ml-pipeline-ui-artifact"
|
||||
}
|
||||
}
|
||||
},
|
||||
]
|
||||
print('Received request:', parent)
|
||||
print('Desired resources except secrets:', desired_resources)
|
||||
# Moved after the print argument because this is sensitive data.
|
||||
desired_resources.append({
|
||||
"apiVersion": "v1",
|
||||
"kind": "Secret",
|
||||
"metadata": {
|
||||
"name": "mlpipeline-minio-artifact",
|
||||
"namespace": namespace,
|
||||
},
|
||||
"data": {
|
||||
"accesskey": mlpipeline_minio_access_key,
|
||||
"secretkey": mlpipeline_minio_secret_key,
|
||||
},
|
||||
})
|
||||
|
||||
return {"status": desired_status, "children": desired_resources}
|
||||
|
||||
def do_POST(self):
|
||||
# Serve the sync() function as a JSON webhook.
|
||||
observed = json.loads(
|
||||
self.rfile.read(int(self.headers.get("content-length"))))
|
||||
desired = self.sync(observed["parent"], observed["children"])
|
||||
|
||||
self.send_response(200)
|
||||
self.send_header("Content-type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(bytes(json.dumps(desired), 'utf-8'))
|
||||
|
||||
|
||||
HTTPServer(("", 8080), Controller).serve_forever()
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: ml-pipeline-ui
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: ml-pipeline-ui
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: ml-pipeline-ui
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: ml-pipeline-ui
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- pods
|
||||
- pods/log
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- events
|
||||
verbs:
|
||||
- list
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- secrets
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- apiGroups:
|
||||
- "kubeflow.org"
|
||||
resources:
|
||||
- viewers
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- delete
|
||||
- apiGroups:
|
||||
- "argoproj.io"
|
||||
resources:
|
||||
- workflows
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- apiGroups:
|
||||
- tekton.dev
|
||||
resources:
|
||||
- pipelineruns
|
||||
- taskruns
|
||||
- conditions
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: ml-pipeline-ui-configmap
|
||||
data:
|
||||
# Temporary workarounds:
|
||||
# 1. Using default-editor because default-viewer isn't bound to workload identity
|
||||
viewer-pod-template.json: |-
|
||||
{
|
||||
"spec": {
|
||||
"serviceAccountName": "default-editor"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ml-pipeline-ui
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
volumes:
|
||||
- name: config-volume
|
||||
configMap:
|
||||
name: ml-pipeline-ui-configmap
|
||||
containers:
|
||||
- name: ml-pipeline-ui
|
||||
env:
|
||||
- name: VIEWER_TENSORBOARD_POD_TEMPLATE_SPEC_PATH
|
||||
value: /etc/config/viewer-pod-template.json
|
||||
- name: DEPLOYMENT
|
||||
value: KUBEFLOW
|
||||
- name: ARTIFACTS_SERVICE_PROXY_NAME
|
||||
value: ml-pipeline-ui-artifact
|
||||
- name: ARTIFACTS_SERVICE_PROXY_PORT
|
||||
value: '80'
|
||||
- name: ARTIFACTS_SERVICE_PROXY_ENABLED
|
||||
value: 'true'
|
||||
- name: ENABLE_AUTHZ
|
||||
value: 'true'
|
||||
- name: KUBEFLOW_USERID_HEADER
|
||||
value: kubeflow-userid
|
||||
- name: KUBEFLOW_USERID_PREFIX
|
||||
value: ""
|
||||
volumeMounts:
|
||||
- name: config-volume
|
||||
mountPath: /etc/config
|
||||
readOnly: true
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
commonLabels:
|
||||
app: ml-pipeline-ui
|
||||
resources:
|
||||
- cluster-role.yaml
|
||||
- cluster-role-binding.yaml
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: ml-pipeline-scheduledworkflow-binding
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: ml-pipeline-scheduledworkflow-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: ml-pipeline-scheduledworkflow
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: ml-pipeline-scheduledworkflow-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- argoproj.io
|
||||
resources:
|
||||
- workflows
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
- apiGroups:
|
||||
- kubeflow.org
|
||||
resources:
|
||||
- scheduledworkflows
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
- apiGroups:
|
||||
- ''
|
||||
resources:
|
||||
- events
|
||||
verbs:
|
||||
- create
|
||||
- patch
|
||||
- apiGroups:
|
||||
- tekton.dev
|
||||
resources:
|
||||
- pipelineruns
|
||||
- taskruns
|
||||
- conditions
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ml-pipeline-scheduledworkflow
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: ml-pipeline-scheduledworkflow
|
||||
env:
|
||||
- name: NAMESPACE
|
||||
value: '' # Empty namespace let viewer controller watch all namespaces
|
||||
valueFrom: null # HACK: https://github.com/kubernetes-sigs/kustomize/issues/2606
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: kubeflow
|
||||
resources:
|
||||
- cluster-role.yaml
|
||||
- cluster-role-binding.yaml
|
||||
|
|
@ -0,0 +1,115 @@
|
|||
# NOTE: IMPORTANT
|
||||
# We need to separate out actual rules from aggregation rules due to
|
||||
# https://github.com/kubernetes/kubernetes/issues/65171
|
||||
# TL;DR: We can't have both aggregation and rules in a [Cluster]Role. When that
|
||||
# is the case, the rules get ignored.
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
labels:
|
||||
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-edit: "true"
|
||||
name: kubeflow-pipelines-edit
|
||||
aggregationRule:
|
||||
clusterRoleSelectors:
|
||||
- matchLabels:
|
||||
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-pipelines-edit: "true"
|
||||
rules: []
|
||||
|
||||
---
|
||||
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
labels:
|
||||
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-pipelines-edit: "true"
|
||||
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-view: "true"
|
||||
name: kubeflow-pipelines-view
|
||||
aggregationRule:
|
||||
clusterRoleSelectors:
|
||||
- matchLabels:
|
||||
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-pipelines-view: "true"
|
||||
rules: []
|
||||
|
||||
---
|
||||
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
labels:
|
||||
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-pipelines-edit: "true"
|
||||
name: aggregate-to-kubeflow-pipelines-edit
|
||||
rules:
|
||||
- apiGroups:
|
||||
- pipelines.kubeflow.org
|
||||
resources:
|
||||
- pipelines
|
||||
- pipelines/versions
|
||||
verbs:
|
||||
- create
|
||||
- delete
|
||||
- update
|
||||
- apiGroups:
|
||||
- pipelines.kubeflow.org
|
||||
resources:
|
||||
- experiments
|
||||
verbs:
|
||||
- archive
|
||||
- create
|
||||
- delete
|
||||
- unarchive
|
||||
- apiGroups:
|
||||
- pipelines.kubeflow.org
|
||||
resources:
|
||||
- runs
|
||||
verbs:
|
||||
- archive
|
||||
- create
|
||||
- delete
|
||||
- retry
|
||||
- terminate
|
||||
- unarchive
|
||||
- apiGroups:
|
||||
- pipelines.kubeflow.org
|
||||
resources:
|
||||
- jobs
|
||||
verbs:
|
||||
- create
|
||||
- delete
|
||||
- disable
|
||||
- enable
|
||||
|
||||
---
|
||||
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
labels:
|
||||
rbac.authorization.kubeflow.org/aggregate-to-kubeflow-pipelines-view: "true"
|
||||
name: aggregate-to-kubeflow-pipelines-view
|
||||
rules:
|
||||
- apiGroups:
|
||||
- pipelines.kubeflow.org
|
||||
resources:
|
||||
- pipelines
|
||||
- pipelines/versions
|
||||
- experiments
|
||||
- runs
|
||||
- jobs
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- apiGroups:
|
||||
- kubeflow.org
|
||||
resources:
|
||||
- viewers
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- delete
|
||||
- apiGroups:
|
||||
- pipelines.kubeflow.org
|
||||
resources:
|
||||
- visualizations
|
||||
verbs:
|
||||
- create
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: ml-pipeline-viewer-crd-binding
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: ml-pipeline-viewer-controller-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: ml-pipeline-viewer-crd-service-account
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: ml-pipeline-viewer-controller-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- '*'
|
||||
resources:
|
||||
- deployments
|
||||
- services
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
- apiGroups:
|
||||
- kubeflow.org
|
||||
resources:
|
||||
- viewers
|
||||
verbs:
|
||||
- create
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- update
|
||||
- patch
|
||||
- delete
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ml-pipeline-viewer-crd
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: ml-pipeline-viewer-crd
|
||||
env:
|
||||
- name: NAMESPACE
|
||||
value: '' # Empty namespace let viewer controller watch all namespaces
|
||||
valueFrom: null
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- cluster-role.yaml
|
||||
- cluster-role-binding.yaml
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
apiVersion: networking.istio.io/v1alpha3
|
||||
kind: VirtualService
|
||||
metadata:
|
||||
name: ml-pipeline-ui
|
||||
spec:
|
||||
gateways:
|
||||
- kubeflow-gateway
|
||||
hosts:
|
||||
- '*'
|
||||
http:
|
||||
- match:
|
||||
- uri:
|
||||
prefix: /pipeline
|
||||
rewrite:
|
||||
uri: /pipeline
|
||||
route:
|
||||
- destination:
|
||||
host: ml-pipeline-ui.$(kfp-namespace).svc.cluster.local
|
||||
port:
|
||||
number: 80
|
||||
timeout: 300s
|
||||
|
||||
---
|
||||
|
||||
apiVersion: networking.istio.io/v1alpha3
|
||||
kind: VirtualService
|
||||
metadata:
|
||||
name: metadata-grpc
|
||||
namespace: kubeflow
|
||||
spec:
|
||||
gateways:
|
||||
- kubeflow-gateway
|
||||
hosts:
|
||||
- '*'
|
||||
http:
|
||||
- match:
|
||||
- uri:
|
||||
prefix: /ml_metadata
|
||||
rewrite:
|
||||
uri: /ml_metadata
|
||||
route:
|
||||
- destination:
|
||||
host: ml-pipeline-ui.$(kfp-namespace).svc.cluster.local
|
||||
port:
|
||||
number: 80
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- metadata-grpc-configmap.yaml
|
||||
- metadata-grpc-deployment.yaml
|
||||
- metadata-grpc-service.yaml
|
||||
- metadata-envoy-deployment.yaml
|
||||
- metadata-envoy-service.yaml
|
||||
- metadata-grpc-sa.yaml
|
||||
images:
|
||||
- name: gcr.io/ml-pipeline/metadata-envoy
|
||||
newTag: 1.5.0-rc.2
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: metadata-envoy-deployment
|
||||
labels:
|
||||
component: metadata-envoy
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
component: metadata-envoy
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
component: metadata-envoy
|
||||
annotations:
|
||||
sidecar.istio.io/inject: "false"
|
||||
spec:
|
||||
containers:
|
||||
- name: container
|
||||
image: gcr.io/ml-pipeline/metadata-envoy:dummy
|
||||
ports:
|
||||
- name: md-envoy
|
||||
containerPort: 9090
|
||||
- name: envoy-admin
|
||||
containerPort: 9901
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
kind: Service
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
labels:
|
||||
app: metadata-envoy
|
||||
name: metadata-envoy-service
|
||||
spec:
|
||||
selector:
|
||||
component: metadata-envoy
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 9090
|
||||
protocol: TCP
|
||||
name: md-envoy
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: metadata-grpc-configmap
|
||||
labels:
|
||||
component: metadata-grpc-server
|
||||
data:
|
||||
METADATA_GRPC_SERVICE_HOST: "metadata-grpc-service"
|
||||
METADATA_GRPC_SERVICE_PORT: "8080"
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: metadata-grpc-deployment
|
||||
labels:
|
||||
component: metadata-grpc-server
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
component: metadata-grpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
component: metadata-grpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: container
|
||||
# ! Sync to the same MLMD version:
|
||||
# * backend/metadata_writer/requirements.in and requirements.txt
|
||||
# * @kubeflow/frontend/src/mlmd/generated
|
||||
# * .cloudbuild.yaml and .release.cloudbuild.yaml
|
||||
# * manifests/kustomize/base/metadata/base/metadata-grpc-deployment.yaml
|
||||
# * test/tag_for_hosted.sh
|
||||
image: gcr.io/tfx-oss-public/ml_metadata_store_server:0.25.1
|
||||
env:
|
||||
- name: DBCONFIG_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-secret
|
||||
key: username
|
||||
- name: DBCONFIG_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-secret
|
||||
key: password
|
||||
- name: MYSQL_DATABASE
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: pipeline-install-config
|
||||
key: mlmdDb
|
||||
- name: MYSQL_HOST
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: pipeline-install-config
|
||||
key: dbHost
|
||||
- name: MYSQL_PORT
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: pipeline-install-config
|
||||
key: dbPort
|
||||
command: ["/bin/metadata_store_server"]
|
||||
args: ["--grpc_port=8080",
|
||||
"--mysql_config_database=$(MYSQL_DATABASE)",
|
||||
"--mysql_config_host=$(MYSQL_HOST)",
|
||||
"--mysql_config_port=$(MYSQL_PORT)",
|
||||
"--mysql_config_user=$(DBCONFIG_USER)",
|
||||
"--mysql_config_password=$(DBCONFIG_PASSWORD)",
|
||||
"--enable_database_upgrade=true"
|
||||
]
|
||||
ports:
|
||||
- name: grpc-api
|
||||
containerPort: 8080
|
||||
livenessProbe:
|
||||
tcpSocket:
|
||||
port: grpc-api
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 2
|
||||
readinessProbe:
|
||||
tcpSocket:
|
||||
port: grpc-api
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 2
|
||||
serviceAccountName: metadata-grpc-server
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: metadata-grpc-server
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
kind: Service
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
labels:
|
||||
app: metadata
|
||||
name: metadata-grpc-service
|
||||
spec:
|
||||
selector:
|
||||
component: metadata-grpc-server
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 8080
|
||||
protocol: TCP
|
||||
name: grpc-api
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue