kubeflow
/
manifests
mirror of https://github.com/kubeflow/manifests.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Cherry-picks for v1.5-branch before cutting RC1 (#2138) 

* tests: Scripts for e2e tests (#2128)

* remove old test files

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* gitignore: Don't track pyc files

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* flake8: Introduce linting file

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* hack: Introduce scripts for cluster manipulation

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* tests: Add e2e test

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* GH action for running e2e test

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* Reduce the installed components and system reqs

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* kserve: Add simple kustomization file

To avoid having to use --load_restrictor none we'll need to wrap the
KServe manifests inside a kustomization.yaml file.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* unittests: Fix unit tests

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* gh: Remove action for e2e tests

We should use prow instead to trigger our e2e tests.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* Add networkpolicies under /contrib/networkpolicies (#2121)

* Create .gitkeep

* Add files via upload

* Create OWNERS

* Create README.md

* Delete default-deny-not-istio-system.yaml

* Create default-allow-same-namespace.yaml

* Create centraldashboard.yaml

* Create jupyter-web-app.yaml

* Create katib-ui.yaml

* Create kfserving-models-web-app.yaml

* Create ml-pipeline-ui.yaml

* Update ml-pipeline.yaml

* Create volumes-web-app.yaml

* Update kustomization.yaml

* Update OWNERS

* Sync kubeflow pipelines manifests 1.8.0 rc.2 (#2131)

* hack: Update pipelines sync script to change README

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* Update kubeflow/pipelines manifests from 1.8.0-rc.2

* Sync kubeflow kubeflow manifests v1.5.0 rc.1 (#2134)

* hack: Sync README for kubeflow/kubeflow sync-script

Extend the sync-script for kubeflow/kubeflow to also update the
components versions in the readme.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* Update kubeflow/kubeflow manifests from v1.5.0-rc.1

* Sync kserve/models-web-app manifests (#2135)

* kserve: Rename from upstream to kserve

We will be including both kserve/kserve and kserve/models-web-app into
the manifests, so the names will need to reflect this.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* kserve: Add manifests for the models-web-app

Include the MWA manifests from the v0.7.0 tag.
https://github.com/kserve/models-web-app/tree/v0.7.0

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* kserve: Include both kserve and mwa manifests

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* Update kubeflow/kfp-tekton manifests from v1.1.1 (#2141)

* hack: Update tekton script to edit README

The hack script for updating the kfp-tekton manifests should also be
updating the README file as well.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* Update kubeflow/kfp-tekton manifests from v1.1.1

* Update manifests for Katib v0.13.0-rc.1 release (#2139)

* Update manifests for Katib v0.13.0-rc.1 release

* Change README

* readme: Remove MPI reference and add ingress distributions link (#2143)

* Closes #1963
* Remove unused MPI reference (PR #2119)

* Update kubeflow/pipelines manifests from 1.8.0 (#2144)

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* hack: Don't error if namespace kubeflow exists (#2140)

The helper setup scripts should not error when the namespaces already
exist.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

Co-authored-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Co-authored-by: a9p <58503488+a9p@users.noreply.github.com>
This commit is contained in:
Kimonas Sotirchos 2022-02-16 17:10:07 +02:00 committed by GitHub
parent 7bd34debf4
commit 9195ebfb34
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5868 changed files with 2757 additions and 337166 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.flake8 Normal file
View File

@ -0,0 +1,4 @@
[flake8]
docstring_convention = google
exclude = assets,__init__.py,__pycache__
ignore = D100,D103,D104,D107,W503

11
.github/workflows/manifests_unittests.yaml vendored
View File

@ -13,9 +13,14 @@ jobs:
- name: Check out repo
uses: actions/checkout@v2
- name: Install Kustomize
working-directory: ./tests/e2e
run: |
curl -Lo ./kustomize https://github.com/kubernetes-sigs/kustomize/releases/download/v3.2.0/kustomize_3.2.0_linux_amd64
chmod +x ./kustomize
sudo mv kustomize /usr/local/bin
- name: Unit Test
run: |
cd tests
make test
kustomize build example

2
.gitignore vendored
View File

@ -2,6 +2,8 @@
.vscode
.DS_Store
**/*.pyc
# Swap
[._]*.s[a-v][a-z]
!*.svg # comment out if you don't need vector files

32
README.md
View File

@ -42,19 +42,19 @@ This repo periodically syncs all official Kubeflow components from their respect
| Component | Local Manifests Path | Upstream Revision |
| - | - | - |
| Training Operator | apps/training-operator/upstream | [v1.4.0-rc.0](https://github.com/kubeflow/tf-operator/tree/v1.4.0-rc.0/manifests) |
| Notebook Controller | apps/jupyter/notebook-controller/upstream | [v1.5.0-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.0/components/notebook-controller/config) |
| Tensorboard Controller | apps/tensorboard/tensorboard-controller/upstream | [v1.5.0-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.0/components/tensorboard-controller/config) |
| Central Dashboard | apps/centraldashboard/upstream | [v1.5.0-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.0/components/centraldashboard/manifests) |
| Profiles + KFAM | apps/profiles/upstream | [v1.5.0-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.0/components/profile-controller/config) |
| PodDefaults Webhook | apps/admission-webhook/upstream | [v1.5.0-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.0/components/admission-webhook/manifests) |
| Jupyter Web App | apps/jupyter/jupyter-web-app/upstream | [v1.5.0-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.0/components/crud-web-apps/jupyter/manifests) |
| Tensorboards Web App | apps/tensorboard/tensorboards-web-app/upstream | [v1.5.0-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.0/components/crud-web-apps/tensorboards/manifests) |
| Volumes Web App | apps/volumes-web-app/upstream | [v1.5.0-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.0/components/crud-web-apps/volumes/manifests) |
| Katib | apps/katib/upstream | [v0.13.0-rc.0](https://github.com/kubeflow/katib/tree/v0.12.0-rc.0/manifests/v1beta1) |
| Notebook Controller | apps/jupyter/notebook-controller/upstream | [v1.5.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.1/components/notebook-controller/config) |
| Tensorboard Controller | apps/tensorboard/tensorboard-controller/upstream | [v1.5.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.1/components/tensorboard-controller/config) |
| Central Dashboard | apps/centraldashboard/upstream | [v1.5.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.1/components/centraldashboard/manifests) |
| Profiles + KFAM | apps/profiles/upstream | [v1.5.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.1/components/profile-controller/config) |
| PodDefaults Webhook | apps/admission-webhook/upstream | [v1.5.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.1/components/admission-webhook/manifests) |
| Jupyter Web App | apps/jupyter/jupyter-web-app/upstream | [v1.5.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.1/components/crud-web-apps/jupyter/manifests) |
| Tensorboards Web App | apps/tensorboard/tensorboards-web-app/upstream | [v1.5.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.1/components/crud-web-apps/tensorboards/manifests) |
| Volumes Web App | apps/volumes-web-app/upstream | [v1.5.0-rc.1](https://github.com/kubeflow/kubeflow/tree/v1.5.0-rc.1/components/crud-web-apps/volumes/manifests) |
| Katib | apps/katib/upstream | [v0.13.0-rc.1](https://github.com/kubeflow/katib/tree/v0.13.0-rc.1/manifests/v1beta1) |
| KFServing | apps/kfserving/upstream | [v0.6.1](https://github.com/kubeflow/kfserving/releases/tag/v0.6.1) |
| KServe | contrib/kserve/upstream | [v0.7.0](https://github.com/kserve/kserve/tree/v0.7.0) |
| Kubeflow Pipelines | apps/pipeline/upstream | [1.8.0-rc.1](https://github.com/kubeflow/pipelines/tree/1.8.0-rc.1/manifests/kustomize) |
| Kubeflow Tekton Pipelines | apps/kfp-tekton/upstream | [v1.1.0](https://github.com/kubeflow/kfp-tekton/tree/v1.1.0/manifests/kustomize) |
| Kubeflow Pipelines | apps/pipeline/upstream | [1.8.0](https://github.com/kubeflow/pipelines/tree/1.8.0/manifests/kustomize) |
| Kubeflow Tekton Pipelines | apps/kfp-tekton/upstream | [v1.1.1](https://github.com/kubeflow/kfp-tekton/tree/v1.1.1/manifests/kustomize) |
The following is also a matrix with versions from common components that are
used from the different projects of Kubeflow:
@ -329,14 +329,6 @@ Install the Training Operator official Kubeflow component:
kustomize build apps/training-operator/upstream/overlays/kubeflow | kubectl apply -f -
```
#### MPI Operator
Install the MPI Operator official Kubeflow component:
```sh
kustomize build apps/mpi-job/upstream/overlays/kubeflow | kubectl apply -f -
```
#### User Namespace
Finally, create a new namespace for the the default user (named `kubeflow-user-example-com`).
@ -376,7 +368,7 @@ After running the command, you can access the Kubeflow Central Dashboard by doin
In order to connect to Kubeflow using NodePort / LoadBalancer / Ingress, you need to setup HTTPS. The reason is that many of our web apps (e.g., Tensorboard Web App, Jupyter Web App, Katib UI) use [Secure Cookies](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#restrict_access_to_cookies), so accessing Kubeflow with HTTP over a non-localhost domain does not work.
Exposing your Kubeflow cluster with proper HTTPS is a process heavily dependent on your environment. For this reason, please take a look at the available Kubeflow distributions, which are targeted to specific environments, and select the one that fits your needs.
Exposing your Kubeflow cluster with proper HTTPS is a process heavily dependent on your environment. For this reason, please take a look at the available [Kubeflow distributions](https://www.kubeflow.org/docs/started/installing-kubeflow/#install-a-packaged-kubeflow-distribution), which are targeted to specific environments, and select the one that fits your needs.
---
**NOTE**

2
apps/admission-webhook/upstream/base/kustomization.yaml
View File

@ -16,7 +16,7 @@ commonLabels:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/admission-webhook
newName: public.ecr.aws/j1r0q0g6/notebooks/admission-webhook
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1
namespace: kubeflow
generatorOptions:
disableNameSuffixHash: true

2
apps/centraldashboard/upstream/base/kustomization.yaml
View File

@ -18,7 +18,7 @@ commonLabels:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/central-dashboard
newName: public.ecr.aws/j1r0q0g6/notebooks/central-dashboard
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1
configMapGenerator:
- envs:
- params.env

9
apps/jupyter/jupyter-web-app/upstream/base/cluster-role.yaml
View File

@ -3,15 +3,6 @@ kind: ClusterRole
metadata:
name: cluster-role
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- create
- delete
- apiGroups:
- authorization.k8s.io
resources:

103
apps/jupyter/jupyter-web-app/upstream/base/configs/spawner_ui_config.yaml
View File

@ -17,23 +17,23 @@
spawnerFormDefaults:
image:
# The container Image for the user's Jupyter Notebook
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.5.0-rc.0
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.5.0-rc.1
# The list of available standard container Images
options:
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.5.0-rc.0
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full:v1.5.0-rc.0
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full:v1.5.0-rc.0
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full:v1.5.0-rc.0
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full:v1.5.0-rc.0
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.5.0-rc.1
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full:v1.5.0-rc.1
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full:v1.5.0-rc.1
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full:v1.5.0-rc.1
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full:v1.5.0-rc.1
imageGroupOne:
# The container Image for the user's Group One Server
# The annotation `notebooks.kubeflow.org/http-rewrite-uri: /`
# is applied to notebook in this group, configuring
# the Istio rewrite for containers that host their web UI at `/`
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.5.0-rc.0
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.5.0-rc.1
# The list of available standard container Images
options:
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.5.0-rc.0
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.5.0-rc.1
imageGroupTwo:
# The container Image for the user's Group Two Server
# The annotation `notebooks.kubeflow.org/http-rewrite-uri: /`
@ -42,10 +42,10 @@ spawnerFormDefaults:
# The annotation `notebooks.kubeflow.org/http-headers-request-set`
# is applied to notebook in this group, configuring Istio
# to add the `X-RStudio-Root-Path` header to requests
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.5.0-rc.0
value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.5.0-rc.1
# The list of available standard container Images
options:
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.5.0-rc.0
- public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.5.0-rc.1
# If true, hide registry and/or tag name in the image selection dropdown
hideRegistry: true
hideTag: false
@ -75,71 +75,38 @@ spawnerFormDefaults:
readOnly: false
workspaceVolume:
# Workspace Volume to be attached to user's Notebook
# Each Workspace Volume is declared with the following attributes:
# Type, Name, Size, MountPath and Access Mode
# If you don't want a workspace volume then delete the 'value' key
value:
type:
# The Type of the Workspace Volume
# Supported values: 'New', 'Existing'
value: New
name:
# The Name of the Workspace Volume
# Note that this is a templated value. Special values:
# {notebook-name}: Replaced with the name of the Notebook. The frontend
# will replace this value as the user types the name
value: 'workspace-{notebook-name}'
size:
# The Size of the Workspace Volume (in Gi)
value: '5Gi'
mountPath:
# The Path that the Workspace Volume will be mounted
value: /home/jovyan
accessModes:
# The Access Mode of the Workspace Volume
# Supported values: 'ReadWriteOnce', 'ReadWriteMany', 'ReadOnlyMany'
value: ReadWriteOnce
class:
# The StrageClass the PVC will use if type is New. Special values are:
# {none}: default StorageClass
# {empty}: empty string ""
value: '{none}'
mount: /home/jovyan
newPvc:
metadata:
name: '{notebook-name}-workspace'
spec:
resources:
requests:
storage: 10Gi
accessModes:
- ReadWriteOnce
readOnly: false
dataVolumes:
# List of additional Data Volumes to be attached to the user's Notebook
value: []
# Each Data Volume is declared with the following attributes:
# Type, Name, Size, MountPath and Access Mode
#
# For example, a list with 2 Data Volumes:
# value:
# - value:
# type:
# value: New
# name:
# value: '{notebook-name}-vol-1'
# size:
# value: '10Gi'
# class:
# value: standard
# mountPath:
# value: /home/jovyan/vol-1
# accessModes:
# value: ReadWriteOnce
# class:
# value: {none}
# - value:
# type:
# value: New
# name:
# value: '{notebook-name}-vol-2'
# size:
# value: '10Gi'
# mountPath:
# value: /home/jovyan/vol-2
# accessModes:
# value: ReadWriteMany
# class:
# value: {none}
# - mount: /home/jovyan/datavol-1
# newPvc:
# metadata:
# name: '{notebook-name}-datavol-1'
# spec:
# resources:
# requests:
# storage: 5Gi
# accessModes:
# - ReadWriteOnce
# - mount: /home/jovyan/datavol-1
# existingSource:
# persistentVolumeClaim:
# claimName: test-pvc
readOnly: false
gpus:
# Number of GPUs to be assigned to the Notebook Container

2
apps/jupyter/jupyter-web-app/upstream/base/kustomization.yaml
View File

@ -23,7 +23,7 @@ commonLabels:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/jupyter-web-app
newName: public.ecr.aws/j1r0q0g6/notebooks/jupyter-web-app
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1
# We need the name to be unique without the suffix because the original name is what
# gets used with patches
configMapGenerator:

2
apps/jupyter/jupyter-web-app/upstream/base/role-binding.yaml
View File

@ -1,4 +1,4 @@
apiVersion: rbac.authorization.k8s.io/v1beta1
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: jupyter-notebook-role-binding

55
apps/jupyter/jupyter-web-app/upstream/base/role.yaml
View File

@ -1,35 +1,48 @@
apiVersion: rbac.authorization.k8s.io/v1beta1
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: jupyter-notebook-role
rules:
- apiGroups:
- ""
- authorization.k8s.io
resources:
- pods
- pods/log
- secrets
- services
- subjectaccessreviews
verbs:
- '*'
- apiGroups:
- ""
- apps
- extensions
resources:
- deployments
- replicasets
verbs:
- '*'
- create
- apiGroups:
- kubeflow.org
resources:
- '*'
- notebooks
- notebooks/finalizers
- poddefaults
verbs:
- '*'
- get
- list
- create
- delete
- patch
- update
- apiGroups:
- batch
- ""
resources:
- jobs
- persistentvolumeclaims
verbs:
- '*'
- create
- delete
- get
- list
- apiGroups:
- ""
resources:
- events
- nodes
verbs:
- list
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch

2
apps/jupyter/notebook-controller/upstream/base/kustomization.yaml
View File

@ -5,4 +5,4 @@ resources:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/notebook-controller
newName: public.ecr.aws/j1r0q0g6/notebooks/notebook-controller
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1

28
apps/katib/upstream/components/controller/katib-config.yaml
View File

@ -7,13 +7,13 @@ data:
metrics-collector-sidecar: |-
{
"StdOut": {
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.13.0-rc.1"
},
"File": {
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/file-metrics-collector:v0.13.0-rc.1"
},
"TensorFlowEvent": {
"image": "docker.io/kubeflowkatib/tfevent-metrics-collector:v0.13.0-rc.0",
"image": "docker.io/kubeflowkatib/tfevent-metrics-collector:v0.13.0-rc.1",
"resources": {
"limits": {
"memory": "1Gi"
@ -24,31 +24,31 @@ data:
suggestion: |-
{
"random": {
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.13.0-rc.1"
},
"tpe": {
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-hyperopt:v0.13.0-rc.1"
},
"grid": {
"image": "docker.io/kubeflowkatib/suggestion-chocolate:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-chocolate:v0.13.0-rc.1"
},
"hyperband": {
"image": "docker.io/kubeflowkatib/suggestion-hyperband:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-hyperband:v0.13.0-rc.1"
},
"bayesianoptimization": {
"image": "docker.io/kubeflowkatib/suggestion-skopt:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-skopt:v0.13.0-rc.1"
},
"cmaes": {
"image": "docker.io/kubeflowkatib/suggestion-goptuna:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-goptuna:v0.13.0-rc.1"
},
"sobol": {
"image": "docker.io/kubeflowkatib/suggestion-goptuna:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-goptuna:v0.13.0-rc.1"
},
"multivariate-tpe": {
"image": "docker.io/kubeflowkatib/suggestion-optuna:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-optuna:v0.13.0-rc.1"
},
"enas": {
"image": "docker.io/kubeflowkatib/suggestion-enas:v0.13.0-rc.0",
"image": "docker.io/kubeflowkatib/suggestion-enas:v0.13.0-rc.1",
"resources": {
"limits": {
"memory": "200Mi"
@ -56,12 +56,12 @@ data:
}
},
"darts": {
"image": "docker.io/kubeflowkatib/suggestion-darts:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/suggestion-darts:v0.13.0-rc.1"
}
}
early-stopping: |-
{
"medianstop": {
"image": "docker.io/kubeflowkatib/earlystopping-medianstop:v0.13.0-rc.0"
"image": "docker.io/kubeflowkatib/earlystopping-medianstop:v0.13.0-rc.1"
}
}

8
apps/katib/upstream/components/controller/trial-templates.yaml
View File

@ -14,7 +14,7 @@ data:
spec:
containers:
- name: training-container
image: docker.io/kubeflowkatib/mxnet-mnist:v0.13.0-rc.0
image: docker.io/kubeflowkatib/mxnet-mnist:v0.13.0-rc.1
command:
- "python3"
- "/opt/mxnet-mnist/mnist.py"
@ -32,7 +32,7 @@ data:
spec:
containers:
- name: training-container
image: docker.io/kubeflowkatib/enas-cnn-cifar10-cpu:v0.13.0-rc.0
image: docker.io/kubeflowkatib/enas-cnn-cifar10-cpu:v0.13.0-rc.1
command:
- python3
- -u
@ -53,7 +53,7 @@ data:
spec:
containers:
- name: pytorch
image: docker.io/kubeflowkatib/pytorch-mnist:v0.13.0-rc.0
image: docker.io/kubeflowkatib/pytorch-mnist:v0.13.0-rc.1
imagePullPolicy: Always
command:
- "python3"
@ -68,7 +68,7 @@ data:
spec:
containers:
- name: pytorch
image: docker.io/kubeflowkatib/pytorch-mnist:v0.13.0-rc.0
image: docker.io/kubeflowkatib/pytorch-mnist:v0.13.0-rc.1
imagePullPolicy: Always
command:
- "python3"

6
apps/katib/upstream/installs/katib-cert-manager/kustomization.yaml
View File

@ -21,13 +21,13 @@ resources:
images:
- name: docker.io/kubeflowkatib/katib-controller
newName: docker.io/kubeflowkatib/katib-controller
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-db-manager
newName: docker.io/kubeflowkatib/katib-db-manager
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-ui
newName: docker.io/kubeflowkatib/katib-ui
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
patchesStrategicMerge:
- patches/katib-cert-injection.yaml

8
apps/katib/upstream/installs/katib-external-db/kustomization.yaml
View File

@ -19,16 +19,16 @@ resources:
images:
- name: docker.io/kubeflowkatib/katib-controller
newName: docker.io/kubeflowkatib/katib-controller
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-db-manager
newName: docker.io/kubeflowkatib/katib-db-manager
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-ui
newName: docker.io/kubeflowkatib/katib-ui
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/cert-generator
newName: docker.io/kubeflowkatib/cert-generator
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
patchesStrategicMerge:
- patches/db-manager.yaml
# Modify katib-mysql-secrets with parameters for the DB.

6
apps/katib/upstream/installs/katib-openshift/kustomization.yaml
View File

@ -30,13 +30,13 @@ resources:
images:
- name: docker.io/kubeflowkatib/katib-controller
newName: docker.io/kubeflowkatib/katib-controller
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-db-manager
newName: docker.io/kubeflowkatib/katib-db-manager
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-ui
newName: docker.io/kubeflowkatib/katib-ui
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
patchesJson6902:
# Annotate Service to delegate TLS-secret generation to OpenShift service controller

8
apps/katib/upstream/installs/katib-standalone/kustomization.yaml
View File

@ -21,13 +21,13 @@ resources:
images:
- name: docker.io/kubeflowkatib/katib-controller
newName: docker.io/kubeflowkatib/katib-controller
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-db-manager
newName: docker.io/kubeflowkatib/katib-db-manager
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-ui
newName: docker.io/kubeflowkatib/katib-ui
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/cert-generator
newName: docker.io/kubeflowkatib/cert-generator
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1

6
apps/katib/upstream/installs/katib-with-kubeflow/kustomization.yaml
View File

@ -9,13 +9,13 @@ resources:
images:
- name: docker.io/kubeflowkatib/katib-controller
newName: docker.io/kubeflowkatib/katib-controller
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-db-manager
newName: docker.io/kubeflowkatib/katib-db-manager
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
- name: docker.io/kubeflowkatib/katib-ui
newName: docker.io/kubeflowkatib/katib-ui
newTag: v0.13.0-rc.0
newTag: v0.13.0-rc.1
patchesStrategicMerge:
- patches/remove-namespace.yaml

2
apps/kfp-tekton/upstream/base/cache-deployer/kustomization.yaml
View File

@ -10,4 +10,4 @@ commonLabels:
app: cache-deployer
images:
- name: gcr.io/ml-pipeline/cache-deployer
newTag: 1.7.0
newTag: 1.8.0-rc.3

2
apps/kfp-tekton/upstream/base/cache/kustomization.yaml vendored
View File

@ -11,4 +11,4 @@ commonLabels:
images:
- name: gcr.io/ml-pipeline/cache-server
newName: docker.io/aipipeline/cache-server
newTag: 1.1.0
newTag: 1.1.1

1
apps/kfp-tekton/upstream/base/installs/multi-user/scheduled-workflow/cluster-role.yaml
View File

@ -19,6 +19,7 @@ rules:
- kubeflow.org
resources:
- scheduledworkflows
- scheduledworkflows/finalizers
verbs:
- create
- get

10
apps/kfp-tekton/upstream/base/pipeline/kustomization.yaml
View File

@ -43,20 +43,20 @@ patchesStrategicMerge:
images:
- name: gcr.io/ml-pipeline/api-server
newName: docker.io/aipipeline/api-server
newTag: 1.1.0
newTag: 1.1.1
- name: gcr.io/ml-pipeline/persistenceagent
newName: docker.io/aipipeline/persistenceagent
newTag: 1.1.0
newTag: 1.1.1
- name: gcr.io/ml-pipeline/scheduledworkflow
newName: docker.io/aipipeline/scheduledworkflow
newTag: 1.1.0
newTag: 1.1.1
- name: gcr.io/ml-pipeline/frontend
newName: docker.io/aipipeline/frontend
newTag: 1.1.0
newTag: 1.1.1
- name: gcr.io/ml-pipeline/viewer-crd-controller
newTag: 1.7.0
- name: gcr.io/ml-pipeline/visualization-server
newTag: 1.7.0
- name: gcr.io/ml-pipeline/metadata-writer
newName: docker.io/aipipeline/metadata-writer
newTag: 1.1.0
newTag: 1.1.1

1
apps/kfp-tekton/upstream/base/pipeline/ml-pipeline-scheduledworkflow-role.yaml
View File

@ -21,6 +21,7 @@ rules:
- kubeflow.org
resources:
- scheduledworkflows
- scheduledworkflows/finalizers
verbs:
- create
- get

727
apps/kfp-tekton/upstream/third-party/application/cluster-scoped/application-crd.yaml vendored
View File

@ -1,234 +1,531 @@
apiVersion: apiextensions.k8s.io/v1beta1
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
labels:
controller-tools.k8s.io: "1.0"
annotations:
api-approved.kubernetes.io: https://github.com/kubernetes-sigs/application/pull/2
controller-gen.kubebuilder.io/version: v0.4.0
creationTimestamp: null
name: applications.app.k8s.io
spec:
group: app.k8s.io
names:
categories:
- all
kind: Application
listKind: ApplicationList
plural: applications
shortNames:
- app
singular: application
scope: Namespaced
validation:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
addOwnerRef:
type: boolean
assemblyPhase:
type: string
componentKinds:
items:
type: object
type: array
descriptor:
properties:
description:
type: string
icons:
items:
properties:
size:
type: string
src:
type: string
type:
type: string
required:
versions:
- additionalPrinterColumns:
- description: The type of the application
jsonPath: .spec.descriptor.type
name: Type
type: string
- description: The creation date
jsonPath: .spec.descriptor.version
name: Version
type: string
- description: The application object owns the matched resources
jsonPath: .spec.addOwnerRef
name: Owner
type: boolean
- description: Numbers of components ready
jsonPath: .status.componentsReady
name: Ready
type: string
- description: The creation date
jsonPath: .metadata.creationTimestamp
name: Age
type: date
name: v1beta1
schema:
openAPIV3Schema:
description: Application is the Schema for the applications API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ApplicationSpec defines the specification for an Application.
properties:
addOwnerRef:
description: AddOwnerRef objects - flag to indicate if we need to
add OwnerRefs to matching objects Matching is done by using Selector
to query all ComponentGroupKinds
type: boolean
assemblyPhase:
description: AssemblyPhase represents the current phase of the application's
assembly. An empty value is equivalent to "Succeeded".
type: string
componentKinds:
description: ComponentGroupKinds is a list of Kinds for Application's
components (e.g. Deployments, Pods, Services, CRDs). It can be used
in conjunction with the Application's Selector to list or watch
the Applications components.
items:
description: GroupKind specifies a Group and a Kind, but does not
force a version. This is useful for identifying concepts during
lookup stages without having partially valid types
properties:
group:
type: string
kind:
type: string
required:
- group
- kind
type: object
type: array
descriptor:
description: Descriptor regroups information and metadata about an
application.
properties:
description:
description: Description is a brief string description of the
Application.
type: string
icons:
description: Icons is an optional list of icons for an application.
Icon information includes the source, size, and mime type.
items:
description: ImageSpec contains information about an image used
as an icon.
properties:
size:
description: (optional) The size of the image in pixels
(e.g., 25x25).
type: string
src:
description: The source for image represented as either
an absolute URL to the image or a Data URL containing
the image. Data URLs are defined in RFC 2397.
type: string
type:
description: (optional) The mine type of the image (e.g.,
"image/png").
type: string
required:
- src
type: object
type: array
keywords:
items:
type: string
type: array
links:
items:
properties:
description:
type: string
url:
type: string
type: object
type: array
maintainers:
items:
properties:
email:
type: string
name:
type: string
url:
type: string
type: object
type: array
notes:
type: string
owners:
items:
properties:
email:
type: string
name:
type: string
url:
type: string
type: object
type: array
type:
type: string
version:
type: string
type: object
info:
items:
properties:
name:
type: object
type: array
keywords:
description: Keywords is an optional list of key words associated
with the application (e.g. MySQL, RDBMS, database).
items:
type: string
type: array
links:
description: Links are a list of descriptive URLs intended to
be used to surface additional documentation, dashboards, etc.
items:
description: Link contains information about an URL to surface
documentation, dashboards, etc.
properties:
description:
description: Description is human readable content explaining
the purpose of the link.
type: string
url:
description: Url typically points at a website address.
type: string
type: object
type: array
maintainers:
description: Maintainers is an optional list of maintainers of
the application. The maintainers in this list maintain the the
source code, images, and package for the application.
items:
description: ContactData contains information about an individual
or organization.
properties:
email:
description: Email is the email address.
type: string
name:
description: Name is the descriptive name.
type: string
url:
description: Url could typically be a website address.
type: string
type: object
type: array
notes:
description: Notes contain a human readable snippets intended
as a quick start for the users of the Application. CommonMark
markdown syntax may be used for rich text representation.
type: string
owners:
description: Owners is an optional list of the owners of the installed
application. The owners of the application should be contacted
in the event of a planned or unplanned disruption affecting
the application.
items:
description: ContactData contains information about an individual
or organization.
properties:
email:
description: Email is the email address.
type: string
name:
description: Name is the descriptive name.
type: string
url:
description: Url could typically be a website address.
type: string
type: object
type: array
type:
description: Type is the type of the application (e.g. WordPress,
MySQL, Cassandra).
type: string
value:
type: string
valueFrom:
properties:
configMapKeyRef:
properties:
apiVersion:
type: string
fieldPath:
type: string
key:
type: string
kind:
type: string
name:
type: string
namespace:
type: string
resourceVersion:
type: string
uid:
type: string
type: object
ingressRef:
properties:
apiVersion:
type: string
fieldPath:
type: string
host:
type: string
kind:
type: string
name:
type: string
namespace:
type: string
path:
type: string
resourceVersion:
type: string
uid:
type: string
type: object
secretKeyRef:
properties:
apiVersion:
type: string
fieldPath:
type: string
key:
type: string
kind:
type: string
name:
type: string
namespace:
type: string
resourceVersion:
type: string
uid:
type: string
type: object
serviceRef:
properties:
apiVersion:
type: string
fieldPath:
type: string
kind:
type: string
name:
type: string
namespace:
type: string
path:
type: string
port:
format: int32
type: integer
resourceVersion:
type: string
uid:
type: string
type: object
type:
type: string
type: object
type: object
type: array
selector:
type: object
type: object
status:
properties:
components:
items:
properties:
group:
type: string
kind:
type: string
link:
type: string
name:
type: string
status:
version:
description: Version is an optional version indicator for the
Application.
type: string
type: object
type: array
conditions:
items:
info:
description: Info contains human readable key,value pairs for the
Application.
items:
description: InfoItem is a human readable key,value pair containing
important information about how to access the Application.
properties:
name:
description: Name is a human readable title for this piece of
information.
type: string
type:
description: Type of the value for this InfoItem.
type: string
value:
description: Value is human readable content.
type: string
valueFrom:
description: ValueFrom defines a reference to derive the value
from another source.
properties:
configMapKeyRef:
description: Selects a key of a ConfigMap.
properties:
apiVersion:
description: API version of the referent.
type: string
fieldPath:
description: 'If referring to a piece of an object instead
of an entire object, this string should contain a
valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container
within a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container
that triggered the event) or if no container name
is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to
have some well-defined way of referencing a part of
an object. TODO: this design is not final and this
field is subject to change in the future.'
type: string
key:
description: The key to select.
type: string
kind:
description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type: string
namespace:
description: 'Namespace of the referent. More info:
https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type: string
resourceVersion:
description: 'Specific resourceVersion to which this
reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type: string
uid:
description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type: string
type: object
ingressRef:
description: Select an Ingress.
properties:
apiVersion:
description: API version of the referent.
type: string
fieldPath:
description: 'If referring to a piece of an object instead
of an entire object, this string should contain a
valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container
within a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container
that triggered the event) or if no container name
is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to
have some well-defined way of referencing a part of
an object. TODO: this design is not final and this
field is subject to change in the future.'
type: string
host:
description: The optional host to select.
type: string
kind:
description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type: string
namespace:
description: 'Namespace of the referent. More info:
https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type: string
path:
description: The optional HTTP path.
type: string
protocol:
description: Protocol for the ingress
type: string
resourceVersion:
description: 'Specific resourceVersion to which this
reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type: string
uid:
description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type: string
type: object
secretKeyRef:
description: Selects a key of a Secret.
properties:
apiVersion:
description: API version of the referent.
type: string
fieldPath:
description: 'If referring to a piece of an object instead
of an entire object, this string should contain a
valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container
within a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container
that triggered the event) or if no container name
is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to
have some well-defined way of referencing a part of
an object. TODO: this design is not final and this
field is subject to change in the future.'
type: string
key:
description: The key to select.
type: string
kind:
description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type: string
namespace:
description: 'Namespace of the referent. More info:
https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type: string
resourceVersion:
description: 'Specific resourceVersion to which this
reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type: string
uid:
description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type: string
type: object
serviceRef:
description: Select a Service.
properties:
apiVersion:
description: API version of the referent.
type: string
fieldPath:
description: 'If referring to a piece of an object instead
of an entire object, this string should contain a
valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container
within a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container
that triggered the event) or if no container name
is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to
have some well-defined way of referencing a part of
an object. TODO: this design is not final and this
field is subject to change in the future.'
type: string
kind:
description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type: string
namespace:
description: 'Namespace of the referent. More info:
https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type: string
path:
description: The optional HTTP path.
type: string
port:
description: The optional port to select.
format: int32
type: integer
protocol:
description: Protocol for the service
type: string
resourceVersion:
description: 'Specific resourceVersion to which this
reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type: string
uid:
description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type: string
type: object
type:
description: Type of source.
type: string
type: object
type: object
type: array
selector:
description: 'Selector is a label query over kinds that created by
the application. It must match the component objects'' labels. More
info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors'
properties:
lastTransitionTime:
format: date-time
type: string
lastUpdateTime:
format: date-time
type: string
message:
type: string
reason:
type: string
status:
type: string
type:
type: string
required:
- type
matchExpressions:
description: matchExpressions is a list of label selector requirements.
The requirements are ANDed.
items:
description: A label selector requirement is a selector that
contains values, a key, and an operator that relates the key
and values.
properties:
key:
description: key is the label key that the selector applies
to.
type: string
operator:
description: operator represents a key's relationship to
a set of values. Valid operators are In, NotIn, Exists
and DoesNotExist.
type: string
values:
description: values is an array of string values. If the
operator is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single
{key,value} in the matchLabels map is equivalent to an element
of matchExpressions, whose key field is "key", the operator
is "In", and the values array contains only "value". The requirements
are ANDed.
type: object
type: object
type: object
status:
description: ApplicationStatus defines controller's the observed state
of Application
properties:
components:
description: Object status array for all matching objects
items:
description: ObjectStatus is a generic status holder for objects
properties:
group:
description: Object group
type: string
kind:
description: Kind of object
type: string
link:
description: Link to object
type: string
name:
description: Name of object
type: string
status:
description: 'Status. Values: InProgress, Ready, Unknown'
type: string
type: object
type: array
componentsReady:
description: 'ComponentsReady: status of the components in the format
ready/total'
type: string
conditions:
description: Conditions represents the latest state of the object
items:
description: Condition describes the state of an object at a certain
point.
properties:
lastTransitionTime:
description: Last time the condition transitioned from one status
to another.
format: date-time
type: string
lastUpdateTime:
description: Last time the condition was probed
format: date-time
type: string
message:
description: A human readable message indicating details about
the transition.
type: string
reason:
description: The reason for the condition's last transition.
type: string
status:
description: Status of the condition, one of True, False, Unknown.
type: string
type:
description: Type of condition.
type: string
required:
- status
type: object
type: array
observedGeneration:
format: int64
type: integer
type: object
version: v1beta1
- type
type: object
type: array
observedGeneration:
description: ObservedGeneration is the most recent generation observed.
It corresponds to the Object's generation, which is updated on mutation
by the API Server.
format: int64
type: integer
type: object
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

500
apps/kfp-tekton/upstream/third-party/metacontroller/base/crd.yaml vendored
View File

@ -1,45 +1,533 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
"api-approved.kubernetes.io": "unapproved, request not yet submitted"
name: compositecontrollers.metacontroller.k8s.io
spec:
group: metacontroller.k8s.io
names:
kind: CompositeController
listKind: CompositeControllerList
plural: compositecontrollers
shortNames:
- cc
- cctl
singular: compositecontroller
scope: Cluster
version: v1alpha1
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
childResources:
items:
properties:
apiVersion:
type: string
resource:
type: string
updateStrategy:
properties:
method:
type: string
statusChecks:
properties:
conditions:
items:
properties:
reason:
type: string
status:
type: string
type:
type: string
required:
- type
type: object
type: array
type: object
type: object
required:
- apiVersion
- resource
type: object
type: array
generateSelector:
type: boolean
hooks:
properties:
customize:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
finalize:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
postUpdateChild:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
preUpdateChild:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
sync:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
type: object
parentResource:
properties:
apiVersion:
type: string
resource:
type: string
revisionHistory:
properties:
fieldPaths:
items:
type: string
type: array
type: object
required:
- apiVersion
- resource
type: object
resyncPeriodSeconds:
format: int32
type: integer
required:
- parentResource
type: object
status:
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
"api-approved.kubernetes.io": "unapproved, request not yet submitted"
name: controllerrevisions.metacontroller.k8s.io
spec:
group: metacontroller.k8s.io
names:
kind: ControllerRevision
listKind: ControllerRevisionList
plural: controllerrevisions
singular: controllerrevision
scope: Namespaced
version: v1alpha1
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
children:
items:
properties:
apiGroup:
type: string
kind:
type: string
names:
items:
type: string
type: array
required:
- apiGroup
- kind
- names
type: object
type: array
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
parentPatch:
type: object
required:
- metadata
- parentPatch
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
"api-approved.kubernetes.io": "unapproved, request not yet submitted"
name: decoratorcontrollers.metacontroller.k8s.io
spec:
group: metacontroller.k8s.io
names:
kind: DecoratorController
listKind: DecoratorControllerList
plural: decoratorcontrollers
shortNames:
- dec
- decorators
singular: decoratorcontroller
scope: Cluster
version: v1alpha1
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
attachments:
items:
properties:
apiVersion:
type: string
resource:
type: string
updateStrategy:
properties:
method:
type: string
type: object
required:
- apiVersion
- resource
type: object
type: array
hooks:
properties:
customize:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
finalize:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
sync:
properties:
webhook:
properties:
path:
type: string
service:
properties:
name:
type: string
namespace:
type: string
port:
format: int32
type: integer
protocol:
type: string
required:
- name
- namespace
type: object
timeout:
type: string
url:
type: string
type: object
type: object
type: object
resources:
items:
properties:
annotationSelector:
properties:
matchAnnotations:
additionalProperties:
type: string
type: object
matchExpressions:
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
type: object
apiVersion:
type: string
labelSelector:
description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies to.
type: string
operator:
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
resource:
type: string
required:
- apiVersion
- resource
type: object
type: array
resyncPeriodSeconds:
format: int32
type: integer
required:
- resources
type: object
status:
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

4
apps/kfp-tekton/upstream/third-party/tekton-custom-task/kustomization.yaml vendored
View File

@ -8,6 +8,6 @@ namespace: tekton-pipelines
images:
- name: docker.io/aipipeline/pipelineloop-controller
newTag: 1.1.0
newTag: 1.1.1
- name: docker.io/aipipeline/pipelineloop-webhook
newTag: 1.1.0
newTag: 1.1.1

2
apps/pipeline/upstream/base/cache-deployer/kustomization.yaml
View File

@ -8,4 +8,4 @@ commonLabels:
app: cache-deployer
images:
- name: gcr.io/ml-pipeline/cache-deployer
newTag: 1.8.0-rc.1
newTag: 1.8.0

2
apps/pipeline/upstream/base/cache/kustomization.yaml vendored
View File

@ -10,4 +10,4 @@ commonLabels:
app: cache-server
images:
- name: gcr.io/ml-pipeline/cache-server
newTag: 1.8.0-rc.1
newTag: 1.8.0

2
apps/pipeline/upstream/base/installs/generic/pipeline-install-config.yaml
View File

@ -11,7 +11,7 @@ data:
until the changes take effect. A quick way to restart all deployments in a
namespace: `kubectl rollout restart deployment -n <your-namespace>`.
appName: pipeline
appVersion: 1.8.0-rc.1
appVersion: 1.8.0
dbHost: mysql
dbPort: "3306"
mlmdDb: metadb

3
apps/pipeline/upstream/base/installs/multi-user/pipelines-profile-controller/composite-controller.yaml
View File

@ -10,9 +10,6 @@ spec:
parentResource:
apiVersion: v1
resource: namespaces
labelSelector:
matchLabels:
pipelines.kubeflow.org/enabled = "true"
childResources:
- apiVersion: v1
resource: secrets

1
apps/pipeline/upstream/base/installs/multi-user/scheduled-workflow/cluster-role.yaml
View File

@ -19,6 +19,7 @@ rules:
- kubeflow.org
resources:
- scheduledworkflows
- scheduledworkflows/finalizers
verbs:
- create
- get

2
apps/pipeline/upstream/base/metadata/base/kustomization.yaml
View File

@ -9,4 +9,4 @@ resources:
- metadata-grpc-sa.yaml
images:
- name: gcr.io/ml-pipeline/metadata-envoy
newTag: 1.8.0-rc.1
newTag: 1.8.0

12
apps/pipeline/upstream/base/pipeline/kustomization.yaml
View File

@ -37,14 +37,14 @@ resources:
- kfp-launcher-configmap.yaml
images:
- name: gcr.io/ml-pipeline/api-server
newTag: 1.8.0-rc.1
newTag: 1.8.0
- name: gcr.io/ml-pipeline/persistenceagent
newTag: 1.8.0-rc.1
newTag: 1.8.0
- name: gcr.io/ml-pipeline/scheduledworkflow
newTag: 1.8.0-rc.1
newTag: 1.8.0
- name: gcr.io/ml-pipeline/frontend
newTag: 1.8.0-rc.1
newTag: 1.8.0
- name: gcr.io/ml-pipeline/viewer-crd-controller
newTag: 1.8.0-rc.1
newTag: 1.8.0
- name: gcr.io/ml-pipeline/visualization-server
newTag: 1.8.0-rc.1
newTag: 1.8.0

2
apps/pipeline/upstream/base/pipeline/metadata-writer/kustomization.yaml
View File

@ -7,4 +7,4 @@ resources:
- metadata-writer-sa.yaml
images:
- name: gcr.io/ml-pipeline/metadata-writer
newTag: 1.8.0-rc.1
newTag: 1.8.0

1
apps/pipeline/upstream/base/pipeline/ml-pipeline-scheduledworkflow-role.yaml
View File

@ -21,6 +21,7 @@ rules:
- kubeflow.org
resources:
- scheduledworkflows
- scheduledworkflows/finalizers
verbs:
- create
- get

2
apps/pipeline/upstream/env/gcp/cloudsql-proxy/cloudsql-proxy-deployment.yaml vendored
View File

@ -43,7 +43,7 @@ spec:
livenessProbe:
httpGet:
path: /liveness
port: '8090'
port: 8090
# Number of seconds after the container has started before the first probe is scheduled. Defaults to 0.
# Not necessary when the startup probe is in use.
initialDelaySeconds: 0

2
apps/pipeline/upstream/env/gcp/inverse-proxy/kustomization.yaml vendored
View File

@ -2,7 +2,7 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
images:
- name: gcr.io/ml-pipeline/inverse-proxy-agent
newTag: 1.8.0-rc.1
newTag: 1.8.0
resources:
- proxy-configmap.yaml
- proxy-deployment.yaml

2
apps/profiles/upstream/base/kustomization.yaml
View File

@ -12,7 +12,7 @@ patchesStrategicMerge:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/profile-controller
newName: public.ecr.aws/j1r0q0g6/notebooks/profile-controller
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1
configMapGenerator:
- name: namespace-labels-data

2
apps/profiles/upstream/overlays/kubeflow/kustomization.yaml
View File

@ -29,4 +29,4 @@ vars:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/access-management
newName: public.ecr.aws/j1r0q0g6/notebooks/access-management
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1

6
apps/tensorboard/tensorboard-controller/upstream/base/kustomization.yaml
View File

@ -12,7 +12,7 @@ patchesStrategicMerge:
- patches/add_service_account.yaml
patches:
- patch: |
apiVersion: rbac.authorization.k8s.io/v1beta1
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: "*"
@ -23,7 +23,7 @@ patches:
kind: "RoleBinding"
group: "rbac.authorization.k8s.io"
- patch: |
apiVersion: rbac.authorization.k8s.io/v1beta1
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: "*"
@ -36,4 +36,4 @@ patches:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/tensorboard-controller
newName: public.ecr.aws/j1r0q0g6/notebooks/tensorboard-controller
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1

2
apps/tensorboard/tensorboards-web-app/upstream/base/kustomization.yaml
View File

@ -14,7 +14,7 @@ commonLabels:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/tensorboards-web-app
newName: public.ecr.aws/j1r0q0g6/notebooks/tensorboards-web-app
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1
# We need the name to be unique without the suffix because the original name is what
# gets used with patches
configMapGenerator:

2
apps/volumes-web-app/upstream/base/kustomization.yaml
View File

@ -14,7 +14,7 @@ commonLabels:
images:
- name: public.ecr.aws/j1r0q0g6/notebooks/volumes-web-app
newName: public.ecr.aws/j1r0q0g6/notebooks/volumes-web-app
newTag: v1.5.0-rc.0
newTag: v1.5.0-rc.1
# We need the name to be unique without the suffix because the original name is what
# gets used with patches
configMapGenerator:

0
contrib/kserve/upstream/kserve.yaml → contrib/kserve/kserve/kserve.yaml
View File

0
contrib/kserve/upstream/kserve_kubeflow.yaml → contrib/kserve/kserve/kserve_kubeflow.yaml
View File

7
contrib/kserve/kserve/kustomization.yaml Normal file
View File

@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- kserve.yaml
# For KF 1.5 we are including both KFServing and KServe. Thus we install the
# standalone kserve manifests, to avoid conflicts with 0.6.1 KFServing.
#- kserve_kubeflow.yaml

25
contrib/kserve/models-web-app/base/deployment.yaml Normal file
View File

@ -0,0 +1,25 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: kserve-models-web-app
namespace: kserve
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/component: kserve-models-web-app
template:
metadata:
labels:
app.kubernetes.io/component: kserve-models-web-app
spec:
containers:
- image: kserve/models-web-app:latest
imagePullPolicy: Always
name: kserve-models-web-app
envFrom:
- configMapRef:
name: kserve-models-web-app-config
ports:
- containerPort: 5000
serviceAccountName: kserve-models-web-app

21
contrib/kserve/models-web-app/base/istio.yaml Normal file
View File

@ -0,0 +1,21 @@
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: kserve-models-web-app
namespace: kserve
spec:
gateways:
- $(ingressGateway)
hosts:
- '*'
http:
- match:
- uri:
prefix: /kserve-endpoints/
rewrite:
uri: /
route:
- destination:
host: kserve-models-web-app.kserve.svc.cluster.local
port:
number: 80

16
contrib/kserve/models-web-app/base/kustomization.yaml Normal file
View File

@ -0,0 +1,16 @@
resources:
- rbac.yaml
- service.yaml
- deployment.yaml
- istio.yaml
commonLabels:
kustomize.component: kserve-models-web-app
app.kubernetes.io/component: kserve-models-web-app
images:
- name: kserve/models-web-app
newName: kserve/models-web-app
newTag: v0.7.0
configMapGenerator:
- name: kserve-models-web-app-config
literals:
- APP_DISABLE_AUTH="True"

68
contrib/kserve/models-web-app/base/rbac.yaml Normal file
View File

@ -0,0 +1,68 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: kserve-models-web-app
namespace: kserve
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kserve-models-web-app-cluster-role
rules:
- apiGroups:
- authorization.k8s.io
resources:
- subjectaccessreviews
verbs:
- create
- apiGroups:
- ""
resources:
- namespaces
- pods
- pods/log
- events
verbs:
- get
- list
- apiGroups:
- serving.kserve.io
resources:
- inferenceservices
- inferenceservices/status
verbs:
- get
- list
- watch
- create
- delete
- deletecollection
- patch
- update
- apiGroups:
- serving.knative.dev
resources:
- services
- services/status
- routes
- routes/status
- configurations
- configurations/status
- revisions
- revisions/status
verbs:
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kserve-models-web-app-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kserve-models-web-app-cluster-role
subjects:
- kind: ServiceAccount
name: kserve-models-web-app
namespace: kserve

14
contrib/kserve/models-web-app/base/service.yaml Normal file
View File

@ -0,0 +1,14 @@
apiVersion: v1
kind: Service
metadata:
name: kserve-models-web-app
namespace: kserve
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 5000
selector:
app.kubernetes.io/component: kserve-models-web-app
type: ClusterIP

41
contrib/kserve/models-web-app/overlays/kubeflow/kustomization.yaml Normal file
View File

@ -0,0 +1,41 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# Adds namespace to all resources.
namespace: kubeflow
# Labels to add to all resources and selectors.
commonLabels:
app: kserve
app.kubernetes.io/name: kserve
bases:
- ../../base
- web-app-authorization-policy.yaml
patchesStrategicMerge:
- patches/web-app-sidecar.yaml
patchesJson6902:
- target:
group: networking.istio.io
version: v1beta1
kind: VirtualService
name: kserve-models-web-app
namespace: kserve
path: patches/web-app-vsvc.yaml
generatorOptions:
disableNameSuffixHash: true
# To make namespace for standalone installation kustomizable,
# variabalize ingress gateway, webhook service name and
# kserve namespace in webhook configurations
configMapGenerator:
- name: kserve-models-web-app-config
behavior: replace
literals:
- USERID_HEADER=kubeflow-userid
configurations:
- params.yaml

4
contrib/kserve/models-web-app/overlays/kubeflow/params.yaml Normal file
View File

@ -0,0 +1,4 @@
commonLabels:
- kind: AuthorizationPolicy
path: spec/selector/matchLabels
create: true

10
contrib/kserve/models-web-app/overlays/kubeflow/patches/web-app-sidecar.yaml Normal file
View File

@ -0,0 +1,10 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: kserve-models-web-app
namespace: kserve
spec:
template:
metadata:
annotations:
sidecar.istio.io/inject: "true"

6
contrib/kserve/models-web-app/overlays/kubeflow/patches/web-app-vsvc.yaml Normal file
View File

@ -0,0 +1,6 @@
- op: replace
path: /spec/http/0/route/0/destination
value:
host: kserve-models-web-app.kubeflow.svc.cluster.local
port:
number: 80

18
contrib/kserve/models-web-app/overlays/kubeflow/web-app-authorization-policy.yaml Normal file
View File

@ -0,0 +1,18 @@
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
labels:
kustomize.component: kserve-models-web-app
app.kubernetes.io/component: kserve-models-web-app
name: kserve-models-web-app
spec:
action: ALLOW
selector:
matchLabels:
kustomize.component: kserve-models-web-app
app.kubernetes.io/component: kserve-models-web-app
rules:
- from:
- source:
principals:
- cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account

1
contrib/networkpolicies/.gitkeep Normal file
View File

@ -0,0 +1 @@

5
contrib/networkpolicies/OWNERS Normal file
View File

@ -0,0 +1,5 @@
approvers:
- juliusvonkohout
reviewers:
- juliusvonkohout
- kimwnasptd

8
contrib/networkpolicies/README.md Normal file
View File

@ -0,0 +1,8 @@
### 1. Why would a user apply the extra policies?
It is a second line of defence after Istio autorization policies and it protects pods and services that are not protected by Istio
### 2. Effects they will have in the cluster
Please consult the name of and comments in each networkpolicy for further information.
### 3. We should achieve the same with AuthorizationPolicies
But there are components, e.g. Katib that are not secured by istio

21
contrib/networkpolicies/cache-server.yaml Normal file
View File

@ -0,0 +1,21 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: cache-server
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app
operator: In
values:
- cache-server # mutating webhook
# https://www.elastic.co/guide/en/cloud-on-k8s/1.1/k8s-webhook-network-policies.html
# The kubernetes api server must reach the webhook
ingress:
- ports:
- protocol: TCP
port: 8443
policyTypes:
- Ingress

23
contrib/networkpolicies/centraldashboard.yaml Normal file
View File

@ -0,0 +1,23 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: centraldashboard
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app
operator: In
values:
- centraldashboard
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- istio-system
- podSelector: {}
policyTypes:
- Ingress

12
contrib/networkpolicies/default-allow-same-namespace.yaml Normal file
View File

@ -0,0 +1,12 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: default-allow-same-namespace
namespace: kubeflow
spec:
podSelector: {}
ingress:
- from:
- podSelector: {}
policyTypes:
- Ingress

23
contrib/networkpolicies/jupyter-web-app.yaml Normal file
View File

@ -0,0 +1,23 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: jupyter-web-app
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app
operator: In
values:
- jupyter-web-app
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- istio-system
- podSelector: {}
policyTypes:
- Ingress

23
contrib/networkpolicies/katib-controller.yaml Normal file
View File

@ -0,0 +1,23 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: katib-controller
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: katib.kubeflow.org/component
operator: In
values:
- controller # katib mutating webhook to add metrics logger
# https://www.elastic.co/guide/en/cloud-on-k8s/1.1/k8s-webhook-network-policies.html
# The kubernetes api server must reach the webhook
ingress:
- ports: # webhook
- protocol: TCP
port: 8443
# - ports: # metrics
# - protocol: TCP
# port: 8080
policyTypes:
- Ingress

23
contrib/networkpolicies/katib-db-manager.yaml Normal file
View File

@ -0,0 +1,23 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: katib-db-manager
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: katib.kubeflow.org/component
operator: In
values:
- db-manager # the metrics loggers write directly to this database
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: app.kubernetes.io/part-of
operator: In
values:
- kubeflow-profile
- podSelector: {} # allow all pods from the same namespace
policyTypes:
- Ingress

22
contrib/networkpolicies/katib-ui.yaml Normal file
View File

@ -0,0 +1,22 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: katib-ui
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: katib.kubeflow.org/component
operator: In
values:
- ui
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- istio-system
policyTypes:
- Ingress

22
contrib/networkpolicies/kfserving-models-web-app.yaml Normal file
View File

@ -0,0 +1,22 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: kfserving-models-web-app
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- kfserving-models-web-app
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- istio-system
policyTypes:
- Ingress

21
contrib/networkpolicies/kfserving.yaml Normal file
View File

@ -0,0 +1,21 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: kfserving
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: control-plane
operator: In
values:
- kfserving-controller-manager # mutating webhook
# https://www.elastic.co/guide/en/cloud-on-k8s/1.1/k8s-webhook-network-policies.html
# The kubernetes api server must reach the webhook
ingress:
- ports:
- protocol: TCP
port: 9443
policyTypes:
- Ingress

20
contrib/networkpolicies/kustomization.yaml Normal file
View File

@ -0,0 +1,20 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kubeflow
resources:
- cache-server.yaml
- centraldashboard.yaml
- default-allow-same-namespace.yaml
- jupyter-web-app.yaml
- katib-controller.yaml
- katib-db-manager.yaml
- katib-ui.yaml
- kfserving-models-web-app.yaml
- kfserving.yaml
- metadata-grpc-server.yaml
- minio.yaml
- ml-pipeline-ui.yaml
- ml-pipeline.yaml
- poddefaults.yaml
- seldon.yaml
- volumes-web-app.yaml

24
contrib/networkpolicies/metadata-grpc-server.yaml Normal file
View File

@ -0,0 +1,24 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: metadata-grpc-server
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: component
operator: In
values:
- metadata-grpc-server # metadata server
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: app.kubernetes.io/part-of
operator: In
values:
- kubeflow-profile
- podSelector: {} # allow all pods from the same namespace
policyTypes:
- Ingress

23
contrib/networkpolicies/minio.yaml Normal file
View File

@ -0,0 +1,23 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: minio
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app
operator: In
values:
- minio # artifact storage
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: app.kubernetes.io/part-of
operator: In
values:
- kubeflow-profile
- podSelector: {} # allow all pods from the same namespace
policyTypes:
- Ingress

22
contrib/networkpolicies/ml-pipeline-ui.yaml Normal file
View File

@ -0,0 +1,22 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: ml-pipeline-ui
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app
operator: In
values:
- ml-pipeline-ui
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- istio-system
policyTypes:
- Ingress

28
contrib/networkpolicies/ml-pipeline.yaml Normal file
View File

@ -0,0 +1,28 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: ml-pipeline
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app
operator: In
values:
- ml-pipeline # just the apiserver
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: app.kubernetes.io/part-of
operator: In
values:
- kubeflow-profile
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- istio-system
policyTypes:
- Ingress

20
contrib/networkpolicies/poddefaults.yaml Normal file
View File

@ -0,0 +1,20 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: poddefaults
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app
operator: In
values:
- poddefaults # mutating webhook
# https://www.elastic.co/guide/en/cloud-on-k8s/1.1/k8s-webhook-network-policies.html
# The kubernetes api server must reach the webhook
ingress:
- ports:
- protocol: TCP
port: 4443
policyTypes:
- Ingress

21
contrib/networkpolicies/seldon.yaml Normal file
View File

@ -0,0 +1,21 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: seldon
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: control-plane
operator: In
values:
- seldon-controller-manager # validating webhook
# https://www.elastic.co/guide/en/cloud-on-k8s/1.1/k8s-webhook-network-policies.html
# The kubernetes api server must reach the webhook
ingress:
- ports:
- protocol: TCP
port: 4443
policyTypes:
- Ingress

23
contrib/networkpolicies/volumes-web-app.yaml Normal file
View File

@ -0,0 +1,23 @@
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: volumes-web-app
namespace: kubeflow
spec:
podSelector:
matchExpressions:
- key: app
operator: In
values:
- volumes-web-app
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- istio-system
- podSelector: {}
policyTypes:
- Ingress

4
example/kustomization.yaml
View File

@ -53,5 +53,5 @@ resources:
- ../common/user-namespace/base
# KServe
#- ../contrib/kserve/upstream/kserve_kubeflow.yaml
- ../contrib/kserve/upstream/kserve.yaml
- ../contrib/kserve/kserve
- ../contrib/kserve/models-web-app/base

23
hack/kind-cluster-1-20.yaml Normal file
View File

@ -0,0 +1,23 @@
# https://github.com/kubernetes-sigs/kind/issues/1954#issuecomment-737775492
# https://github.com/istio/istio/blob/e02690fbfb8bda564582b27d22d9e8e6e00422a5/prow/config/trustworthy-jwt.yaml#L1-L13
# This configs KinD to spin up a k8s cluster with trustworthy jwt (Service Account Token Volume Projection) feature.
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: manifests-1-20
nodes:
- role: control-plane
image: kindest/node:1.20.7@sha256:688fba5ce6b825be62a7c7fe1415b35da2bdfbb5a69227c499ea4cc0008661ca
- role: worker
image: kindest/node:1.20.7@sha256:688fba5ce6b825be62a7c7fe1415b35da2bdfbb5a69227c499ea4cc0008661ca
- role: worker
image: kindest/node:1.20.7@sha256:688fba5ce6b825be62a7c7fe1415b35da2bdfbb5a69227c499ea4cc0008661ca
kubeadmConfigPatches:
- |
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
metadata:
name: config
apiServer:
extraArgs:
"service-account-issuer": "kubernetes.default.svc"
"service-account-signing-key-file": "/etc/kubernetes/pki/sa.key"

23
hack/kind-cluster-1-21.yaml Normal file
View File

@ -0,0 +1,23 @@
# https://github.com/kubernetes-sigs/kind/issues/1954#issuecomment-737775492
# https://github.com/istio/istio/blob/e02690fbfb8bda564582b27d22d9e8e6e00422a5/prow/config/trustworthy-jwt.yaml#L1-L13
# This configs KinD to spin up a k8s cluster with trustworthy jwt (Service Account Token Volume Projection) feature.
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: manifests-1-21
nodes:
- role: control-plane
image: kindest/node:1.21.2@sha256:19c2315068fd5951aa478ef7b9d1771572c8ea58fbfbf7bc81f7b153679d7a6c
- role: worker
image: kindest/node:1.21.2@sha256:19c2315068fd5951aa478ef7b9d1771572c8ea58fbfbf7bc81f7b153679d7a6c
- role: worker
image: kindest/node:1.21.2@sha256:19c2315068fd5951aa478ef7b9d1771572c8ea58fbfbf7bc81f7b153679d7a6c
kubeadmConfigPatches:
- |
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
metadata:
name: config
apiServer:
extraArgs:
"service-account-issuer": "kubernetes.default.svc"
"service-account-signing-key-file": "/etc/kubernetes/pki/sa.key"

105
hack/setup-kubeflow-light.sh Executable file
View File

@ -0,0 +1,105 @@
#!/usr/bin/env bash
# Copyright 2021 The Kubeflow Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This shell script is used to setup Katib deployment.
set -euo pipefail
TIMEOUT=600s # 10mins
echo "Creating Kubeflow namespace..."
kubectl create namespace kubeflow --dry-run=client -o yaml | kubectl apply -f -
echo "Deploying Cert-Manager."
kustomize build common/cert-manager/cert-manager/base | kubectl apply -f -
echo "Waiting for Cert Manager pods to become ready..."
sleep 5
kubectl wait --timeout=${TIMEOUT} -n cert-manager --all --for=condition=Ready pod
echo "Deploying Istio."
kustomize build common/istio-1-11/istio-crds/base | kubectl apply -f -
kustomize build common/istio-1-11/istio-namespace/base | kubectl apply -f -
kustomize build common/istio-1-11/istio-install/base | kubectl apply -f -
echo "Waiting for istio-system Pods to become ready..."
sleep 5
kubectl wait --timeout=${TIMEOUT} -n istio-system --all --for=condition=Ready pod
echo "Deploying Knative."
function install_knative {
kustomize build common/knative/knative-serving/overlays/gateways | kubectl apply -f -
}
while ! install_knative;
do
echo "Retrying to install knative..."
sleep 10
done
kustomize build common/knative/knative-eventing/base | kubectl apply -f -
kustomize build common/istio-1-11/cluster-local-gateway/base | kubectl apply -f -
echo "Waiting for knative-serving Pods to become ready..."
sleep 5
kubectl wait --timeout=${TIMEOUT} -n knative-serving --all --for=condition=Ready pod
echo "Deploying KFP."
function install_kfp {
kustomize build apps/pipeline/upstream/env/platform-agnostic-multi-user | kubectl apply -f - --validate=false
}
while ! install_kfp;
do
echo "Retrying to install kfp..."
sleep 10
done
echo "Waiting for kubeflow/ml-pipelines to become ready..."
sleep 5
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l app=ml-pipeline --for=condition=Ready pod
echo "Deploying KFServing."
kustomize build apps/kfserving/upstream/overlays/kubeflow | kubectl apply -f -
echo "Waiting for kubeflow/kfserving to become ready..."
sleep 5
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l app=kfserving --for=condition=Ready pod
echo "Deploying Katib."
kustomize build apps/katib/upstream/installs/katib-with-kubeflow | kubectl apply -f -
echo "Waiting for kubeflow/katib to become ready..."
sleep 5
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l katib.kubeflow.org/component=controller --for=condition=Ready pod
echo "Deploying Training Operator."
kustomize build apps/training-operator/upstream/overlays/kubeflow | kubectl apply -f -
echo "Waiting for kubeflow/training-operator to become ready..."
sleep 5
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l control-plane=kubeflow-training-operator --for=condition=Ready pod
echo "Installing Profiles Controller."
kustomize build apps/profiles/upstream/overlays/kubeflow | kubectl apply -f -
echo "Waiting for kubeflow/profiles-controller to become ready..."
sleep 5
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l kustomize.component=profiles --for=condition=Ready pod
echo "Creating user resources."
kustomize build common/user-namespace/base | kubectl apply -f -
kustomize build common/cert-manager/kubeflow-issuer/base | kubectl apply -f -

58
hack/setup-kubeflow.sh Executable file
View File

@ -0,0 +1,58 @@
#!/usr/bin/env bash
# Copyright 2021 The Kubeflow Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This shell script is used to setup Katib deployment.
set -euo pipefail
TIMEOUT=600s # 10mins
echo "Creating Kubeflow namespace..."
kubectl create namespace kubeflow --dry-run=client -o yaml | kubectl apply -f -
echo "Deploying all Kubeflow components..."
function install_kubeflow {
kustomize build example --load_restrictor none | kubectl apply -f - --validate=false
}
while ! install_kubeflow;
do
echo "Retrying to apply resources"
sleep 10
done
echo "---"
echo "Waiting for all Kubeflow components to become ready."
echo "Waiting for Cert Manager pods to become ready..."
kubectl wait --timeout=${TIMEOUT} -n cert-manager --all --for=condition=Ready pod
echo "Waiting for istio-system Pods to become ready..."
kubectl wait --timeout=${TIMEOUT} -n istio-system --all --for=condition=Ready pod
echo "Waiting for knative-serving Pods to become ready..."
kubectl wait --timeout=${TIMEOUT} -n knative-serving --all --for=condition=Ready pod
echo "Waiting for kubeflow/ml-pipelines to become ready..."
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l app=ml-pipeline --for=condition=Ready pod
echo "Waiting for kubeflow/kfserving to become ready..."
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l app=kfserving --for=condition=Ready pod
echo "Waiting for kubeflow/katib to become ready..."
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l katib.kubeflow.org/component=controller --for=condition=Ready pod
echo "Waiting for kubeflow/training-operator to become ready..."
kubectl wait --timeout=${TIMEOUT} -n kubeflow -l control-plane=kubeflow-training-operator --for=condition=Ready pod

6
hack/sync-kfp-tekton-manifests.sh
View File

@ -52,6 +52,11 @@ DST_DIR=$MANIFESTS_DIR/apps/kfp-tekton/upstream
rm -r $DST_DIR
cp $SRC_DIR/manifests/kustomize $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kfp-tekton/tree/.*/manifests/kustomize)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kfp-tekton/tree/$COMMIT/manifests/kustomize)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Successfully copied all manifests."
@ -59,4 +64,5 @@ echo "Successfully copied all manifests."
echo "Committing the changes..."
cd $MANIFESTS_DIR
git add apps
git add README.md
git commit -m "Update kubeflow/kfp-tekton manifests from ${COMMIT}"

41
hack/sync-kubeflow-manifests.sh
View File

@ -52,45 +52,86 @@ DST_DIR=$MANIFESTS_DIR/apps/admission-webhook/upstream
rm -r $DST_DIR
cp $SRC_DIR/components/admission-webhook/manifests $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kubeflow/tree/.*/components/admission-webhook/manifests)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kubeflow/tree/$COMMIT/components/admission-webhook/manifests)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Copying centraldashboard manifests..."
DST_DIR=$MANIFESTS_DIR/apps/centraldashboard/upstream
rm -r $DST_DIR
cp $SRC_DIR/components/centraldashboard/manifests $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kubeflow/tree/.*/components/centraldashboard/manifests)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kubeflow/tree/$COMMIT/components/centraldashboard/manifests)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Copying jupyter-web-app manifests..."
DST_DIR=$MANIFESTS_DIR/apps/jupyter/jupyter-web-app/upstream
rm -r $DST_DIR
cp $SRC_DIR/components/crud-web-apps/jupyter/manifests $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kubeflow/tree/.*/components/crud-web-apps/jupyter/manifests)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kubeflow/tree/$COMMIT/components/crud-web-apps/jupyter/manifests)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Copying volumes-web-app manifests..."
DST_DIR=$MANIFESTS_DIR/apps/volumes-web-app/upstream
rm -r $DST_DIR
cp $SRC_DIR/components/crud-web-apps/volumes/manifests $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kubeflow/tree/.*/components/crud-web-apps/volumes/manifests)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kubeflow/tree/$COMMIT/components/crud-web-apps/volumes/manifests)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Copying tensorboards-web-app manifests..."
DST_DIR=$MANIFESTS_DIR/apps/tensorboard/tensorboards-web-app/upstream
rm -r $DST_DIR
cp $SRC_DIR/components/crud-web-apps/tensorboards/manifests $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kubeflow/tree/.*/components/crud-web-apps/tensorboards/manifests)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kubeflow/tree/$COMMIT/components/crud-web-apps/tensorboards/manifests)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Copying profile-controller manifests..."
DST_DIR=$MANIFESTS_DIR/apps/profiles/upstream
rm -r $DST_DIR
cp $SRC_DIR/components/profile-controller/config $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kubeflow/tree/.*/components/profile-controller/config)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kubeflow/tree/$COMMIT/components/profile-controller/config)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Copying notebook-controller manifests..."
DST_DIR=$MANIFESTS_DIR/apps/jupyter/notebook-controller/upstream
rm -r $DST_DIR
cp $SRC_DIR/components/notebook-controller/config $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kubeflow/tree/.*/components/notebook-controller/config)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kubeflow/tree/$COMMIT/components/notebook-controller/config)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Copying tensorboard-controller manifests..."
DST_DIR=$MANIFESTS_DIR/apps/tensorboard/tensorboard-controller/upstream
rm -r $DST_DIR
cp $SRC_DIR/components/tensorboard-controller/config $DST_DIR -r
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/kubeflow/tree/.*/components/tensorboard-controller/config)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/kubeflow/tree/$COMMIT/components/tensorboard-controller/config)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
echo "Successfully copied all manifests."
# DEV: Comment out these commands when local testing
echo "Committing the changes..."
cd $MANIFESTS_DIR
git add apps
git add README.md
git commit -m "Update kubeflow/kubeflow manifests from ${COMMIT}"

7
hack/sync-pipelines-manifests.sh
View File

@ -55,8 +55,15 @@ cp $SRC_DIR/manifests/kustomize $DST_DIR -r
echo "Successfully copied all manifests."
echo "Updating README..."
SRC_TXT="\[.*\](https://github.com/kubeflow/pipelines/tree/.*/manifests/kustomize)"
DST_TXT="\[$COMMIT\](https://github.com/kubeflow/pipelines/tree/$COMMIT/manifests/kustomize)"
sed -i "s|$SRC_TXT|$DST_TXT|g" ${MANIFESTS_DIR}/README.md
# DEV: Comment out these commands when local testing
echo "Committing the changes..."
cd $MANIFESTS_DIR
git add apps
git add README.md
git commit -m "Update kubeflow/pipelines manifests from ${COMMIT}"

72
tests/Makefile
View File

@ -1,72 +0,0 @@
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
GOLANG_VERSION ?= 1.12.4
GOPATH ?= $(HOME)/go
PYTHON_BIN ?= python
export KUSTOMIZE_BIN ?= kustomize
# Comma seperated items within {} for more than one file
# EXCLUDE ?= istio-install-base_test.go
export GO111MODULE = on
export GO = go
all: test
# Reset various kpt values to default values
# TODO(jlewi): We should add a test to make sure changed values don't get checked in
# We don't run it in generate because we don't want to force all developers to install kpt
# TODO(jlewi): This ends up setting "isSet" to true for the setters in the KptFile.
# Does this prevent gcloud from automatically filling in the values zone and
# region? If so we may need to use yq to set that back to false.
gcp-reset:
kpt cfg set ../gcp/ gcloud.core.project project-id
kpt cfg set ../gcp/ gcloud.project.projectNumber projectNumber
kpt cfg set ../gcp/ gcloud.compute.zone ZONE
kpt cfg set ../gcp/ gcloud.compute.region REGION
kpt cfg set ../gcp/ name name
kpt cfg set ../gcp/ location location
yq d -i ../gcp/Kptfile openAPI.definitions["io.k8s.cli.setters.gcloud.core.project"].["x-k8s-cli"].["setter"].["isSet"]
yq d -i ../gcp/Kptfile openAPI.definitions["io.k8s.cli.setters.gcloud.project.projectNumber"].["x-k8s-cli"].["setter"].["isSet"]
yq d -i ../gcp/Kptfile openAPI.definitions["io.k8s.cli.setters.gcloud.compute.zone"].["x-k8s-cli"].["setter"].["isSet"]
yq d -i ../gcp/Kptfile openAPI.definitions["io.k8s.cli.setters.gcloud.compute.region"].["x-k8s-cli"].["setter"].["isSet"]
yq d -i ../gcp/Kptfile openAPI.definitions["io.k8s.cli.setters.location"].["x-k8s-cli"].["setter"].["isSet"]
yq d -i ../gcp/Kptfile openAPI.definitions["io.k8s.cli.setters.name"].["x-k8s-cli"].["setter"].["isSet"]
generate:
$(PYTHON_BIN) ../hack/generate_tests.py --all
$(GO) fmt ./...
generate-changed-only:
$(PYTHON_BIN) ../hack/generate_tests.py
$(GO) fmt ./...
modules:
@GO111MODULE=on $(GO) mod download
test: modules
# Temporarily disable the autogenerated tests. Re-enable them once stacks
# are buildable again, before the release. In addition, evaluate if we need
# the legacy kustomizations tests. Perhaps a simpler sanity test of building
# all kustomizations in the repo would be preferrable.
# @GO111MODULE=on $(GO) test -v github.com/kubeflow/manifests/tests/...
@GO111MODULE=on $(GO) test -run TestCheckWebhookSelector -v github.com/kubeflow/manifests/tests/.
@GO111MODULE=on $(GO) test -run TestKustomizationHasDeprecatedEnv -v github.com/kubeflow/manifests/tests/.
run-unittest-plugin:
cd .. && XDG_CONFIG_HOME=$$(pwd)/plugins kustomize build profiles/overlays/test --enable_alpha_plugins
run-application-plugin:
cd .. && XDG_CONFIG_HOME=$$(pwd)/plugins kustomize build jupyter/jupyter-web-app/overlays/application --enable_alpha_plugins

9
tests/OWNERS
View File

@ -1,9 +0,0 @@
approvers:
- andreyvelich
- gaocegege
- jeffwan
- johnugeorge
- krishnadurai
- PatrickXYS
- kimwnasptd
- thesuperzapper

39
tests/README.md
View File

@ -1,30 +1,25 @@
# Kustomize Manifest Tests
## Unittests
## E2E
There are two goals for the unittests:
There are 2 goals for the e2e tests:
1. Verify that core components can be applied and become Ready
2. Virify that core CRDs can be created and succeed
1. Verify that kustomizations are applied correctly (kubeflow/manifests#1014)
1. Verify that various conventions are enforced (kubeflow/manifests#1015)
### Components
### Verifying Kustomizations Are Applied Correctly
The e2e are installing everything by using the [single-install
command](../README.md#install-with-a-single-command). This means that all of
the core and common (Istio, Knativ, Cert Manager) are being installed and
included in the tests.
Examples of kustomizations that we would like to verify are applied and generate the expected output
### Test Suite
* Patches
* Variable substitution
* Composition of resources
The e2e tests are completely independent of the underlying K8s cluster, as well
as the platform of the cluster. These tests should be able to run in real
world clusters, as well as ephemeral ones like KinD.
The general approach to doing this is
1. Check in one more "kustomization.yaml" files corresponding to test cases
1. Run "kustomize build -o ..." and check in the output as the expected test output
* Reviewers can verify changes to the expected output to ensure changes have the desired effect on the expected output
1. Unittests run "kustomize build" and compare output to expected output to ensure kustomize packages are in sync with the expected output
1. Make commands make it easy to regenerate the expected output as part of a change.
```
cd tests
make generate-changed-only
```
The tests are also explicitly bypassing any authentication system of the
installation. The goal of these tests are to ensure that a common use case that
deploys all of Kubeflow's components can succeed. Thus we only want to test
that the core CRDs can be successfully applied and complete.

2
tests/doc.go
View File

@ -1,2 +0,0 @@
// tests package contains unittests for kustomize manifests
package tests

1
tests/e2e/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
**/pids.env

27
tests/e2e/README.md Normal file
View File

@ -0,0 +1,27 @@
# E2E Mnist
We've converted the community's [E2E Notebook](https://github.com/kubeflow/pipelines/blob/master/samples/contrib/kubeflow-e2e-mnist/kubeflow-e2e-mnist.ipynb) into a python script. This test can be used to ensure the core Kubeflow CRDs can be applied and complete.
This test is using the following Kubeflow CRDs:
1. Kubeflow Pipelines
2. Katib Experiments
3. TFJobs
4. KFServing InferenceServices
## How to run
The heart of this test is the `mnist.py` python script, which applies and waits
for the CRDs to complete. The python scripts are all expecting that
1. `kubectl` is configured with access to a Kubeflow cluster
2. `kustomize` 3.2.0 is available
3. The KFP backend is proxied to localhost
While the `mnist.py` is used for running the test, it is advised to use the
`runner.sh` script instead. The `runner.sh` script will be running the python
script, but also ensure the KFP backend is port-forwarded and will clean up
afterwards.
## Failures
Both the python and the bash scripts are designed to be failing early. If any
intermediate command fails, then the whole test will fail.

10
tests/e2e/hack/cleanup_proxies.sh Executable file
View File

@ -0,0 +1,10 @@
#!/usr/bin/env bash
source pids.env
echo "Killing background jobs..."
kill -KILL $ISTIO_PID
echo "Killed istio port-forward."
kill -KILL $PIPELINES_PID
echo "Killed pipelines port-forward."

16
tests/e2e/hack/cleanup_yamls.sh Executable file
View File

@ -0,0 +1,16 @@
#!/usr/bin/env bash
source pids.env
echo "Killing background jobs..."
kill -KILL $ISTIO_PID
echo "Killed istio port-forward."
kill -KILL $PIPELINES_PID
echo "Killed pipelines port-forward."
kubectl delete experiments.kubeflow.org -n kubeflow-user-example-com mnist-e2e
kubectl delete tfjobs.kubeflow.org -n kubeflow-user-example-com mnist-e2e
kubectl delete inferenceservices.serving.kubeflow.org -n kubeflow-user-example-com mnist-e2e

18
tests/e2e/hack/proxy_istio.sh Executable file
View File

@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -euo pipefail
# stop all port-forward processes
trap ctrl_c INT
function ctrl_c() {
echo "Stopping port-forward processes..."
echo "Killing process $ISTIO_PID..."
kill -KILL $ISTIO_PID
}
kubectl port-forward -n istio-system svc/istio-ingressgateway 8080:80 &
ISTIO_PID=$!
echo "Started Istio port-forward, pid: $ISTIO_PID"
echo ISTIO_PID=$ISTIO_PID >> pids.env
sleep 1

10
tests/e2e/hack/proxy_pipelines.sh Executable file
View File

@ -0,0 +1,10 @@
#!/usr/bin/env bash
set -euo pipefail
kubectl port-forward -n kubeflow svc/ml-pipeline-ui 3000:80 &
PIPELINES_PID=$!
echo "Started Pipelines port-forward, pid: $PIPELINES_PID"
echo PIPELINES_PID=$PIPELINES_PID >> pids.env
sleep 1

83
tests/e2e/mnist.py Normal file
View File

@ -0,0 +1,83 @@
"""E2E Kubeflow test that tesst Pipelines, Katib, TFJobs and KFServing.
Requires:
pip install kfp==1.8.4
pip install kubeflow-katib==0.12.0
"""
import kfp
import kfp.dsl as dsl
from kubernetes import config
import settings
from utils import isvc, katib, kfserving, tfjob
config.load_kube_config()
@dsl.pipeline(
name="End to End Pipeline",
description="An end to end mnist example including hyperparameter tuning, "
"train and inference",
)
def mnist_pipeline(name=settings.PIPELINE_NAME,
namespace=settings.NAMESPACE,
training_steps=settings.TRAINING_STEPS):
# Run the hyperparameter tuning with Katib.
katib_op = katib.create_katib_experiment_task(
name, namespace, training_steps)
# Create volume to train and serve the model.
model_volume_op = dsl.VolumeOp(
name="model-volume",
resource_name="model-volume",
size="1Gi",
modes=dsl.VOLUME_MODE_RWO,
)
# Run the distributive training with TFJob.
tfjob_op = tfjob.create_tfjob_task(name, namespace, training_steps,
katib_op, model_volume_op)
# Create the KFServing inference.
kfserving.create_kfserving_task(name, namespace, tfjob_op,
model_volume_op)
if __name__ == "__main__":
# Run the Kubeflow Pipeline in the user's namespace.
kfp_client = kfp.Client(host="http://localhost:3000",
namespace="kubeflow-user-example-com")
kfp_client.runs.api_client.default_headers.update(
{"kubeflow-userid": "kubeflow-user-example-com"})
# create the KFP run
run_id = kfp_client.create_run_from_pipeline_func(
mnist_pipeline,
namespace=settings.NAMESPACE,
arguments={},
).run_id
print("Run ID: ", run_id)
katib.wait_to_create(name=settings.EXPERIMENT_NAME,
namespace=settings.NAMESPACE,
timeout=settings.TIMEOUT)
tfjob.wait_to_create(name=settings.EXPERIMENT_NAME,
namespace=settings.NAMESPACE,
timeout=settings.TIMEOUT)
tfjob.wait_to_succeed(name=settings.TFJOB_NAME,
namespace=settings.NAMESPACE,
timeout=settings.TIMEOUT)
katib.wait_to_succeed(name=settings.EXPERIMENT_NAME,
namespace=settings.NAMESPACE,
timeout=settings.TIMEOUT)
isvc.wait_to_create(settings.ISVC_NAME,
namespace=settings.NAMESPACE,
timeout=settings.TIMEOUT)
isvc.wait_to_succeed(settings.ISVC_NAME,
namespace=settings.NAMESPACE,
timeout=settings.TIMEOUT)

3
tests/e2e/requirements.txt Normal file
View File

@ -0,0 +1,3 @@
kubernetes==21.7.0
kfp==1.8.4
kubeflow-katib==0.12.0

17
tests/e2e/runner.sh Executable file
View File

@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail
echo "Installing necessary RBAC."""
kubectl apply -f yamls
echo "Setting up port-forward..."
./hack/proxy_istio.sh
./hack/proxy_pipelines.sh
echo "Running the tests."""
python3 mnist.py
echo "Cleaning up opened processes."""
./hack/cleanup_proxies.sh
echo "Leaving the cluster as is for further inspection."

9
tests/e2e/settings.py Normal file
View File

@ -0,0 +1,9 @@
NAMESPACE = "kubeflow-user-example-com"
TIMEOUT = 300
PIPELINE_NAME = "mnist-e2e"
EXPERIMENT_NAME = "mnist-e2e"
TFJOB_NAME = "mnist-e2e"
ISVC_NAME = "mnist-e2e"
TRAINING_STEPS = "1"

0
tests/e2e/utils/__init__.py Normal file
View File

20
tests/e2e/utils/isvc.py Normal file
View File

@ -0,0 +1,20 @@
from . import watch
GROUP = "serving.kubeflow.org"
PLURAL = "inferenceservices"
VERSION = "v1beta1"
# wait_for_ready(name, namespace, timeout):
def wait_to_create(name, namespace, timeout):
"""Wait until the specified InferenceService gets created."""
return watch.wait_created_cr(name, namespace,
timeout=timeout, group=GROUP, plural=PLURAL,
version=VERSION)
def wait_to_succeed(name, namespace, timeout):
"""Wait until the specified InferenceService succeeds."""
return watch.wait_to_succeed(name=name, namespace=namespace,
timeout=timeout, group=GROUP, plural=PLURAL,
version=VERSION)

Some files were not shown because too many files have changed in this diff Show More