tenzen-y steps down from Katib approver role (#2561)

Signed-off-by: Yuki Iwai <yuki.iwai.tz@gmail.com>

.dockerignore

@@ -1,9 +1,6 @@
 .git
 .gitignore
 docs
-examples
-!examples/v1alpha3/nas
-!examples/v1beta1/nas
 manifests
 pkg/ui/*/frontend/node_modules
 pkg/ui/*/frontend/build

.flake8

@@ -0,0 +1,4 @@
+[flake8]
+max-line-length = 100
+# E203 is ignored to avoid conflicts with Black's formatting, as it's not PEP 8 compliant
+extend-ignore = W503, E203

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,25 +0,0 @@
----
-name: Bug report
-about: Tell us about a problem you are experiencing
-
----
-
-/kind bug
-
-**What steps did you take and what happened:**
-[A clear and concise description of what the bug is.]
-
-
-**What did you expect to happen:**
-
-
-**Anything else you would like to add:**
-[Miscellaneous information that will assist in solving the issue.]
-
-
-**Environment:**
-
-- Kubeflow version (`kfctl version`):
-- Minikube version (`minikube version`):
-- Kubernetes version: (use `kubectl version`):
-- OS (e.g. from `/etc/os-release`):

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -0,0 +1,50 @@
+name: Bug Report
+description: Tell us about a problem you are experiencing with Katib
+labels: ["kind/bug", "lifecycle/needs-triage"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to fill out this Katib bug report!
+  - type: textarea
+    id: problem
+    attributes:
+      label: What happened?
+      description: |
+        Please provide as much info as possible. Not doing so may result in your bug not being
+        addressed in a timely manner.
+    validations:
+      required: true
+  - type: textarea
+    id: expected
+    attributes:
+      label: What did you expect to happen?
+    validations:
+      required: true
+  - type: textarea
+    id: environment
+    attributes:
+      label: Environment
+      value: |
+        Kubernetes version:
+        ```bash
+        $ kubectl version
+
+        ```
+        Katib controller version:
+        ```bash
+        $ kubectl get pods -n kubeflow -l katib.kubeflow.org/component=controller -o jsonpath="{.items[*].spec.containers[*].image}"
+
+        ```
+        Katib Python SDK version:
+        ```bash
+        $ pip show kubeflow-katib
+
+        ```
+    validations:
+      required: true
+  - type: input
+    id: votes
+    attributes:
+      label: Impacted by this bug?
+      value: Give it a 👍 We prioritize the issues with most 👍

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,12 @@
+blank_issues_enabled: true
+
+contact_links:
+  - name: Katib Documentation
+    url: https://www.kubeflow.org/docs/components/katib/
+    about: Much help can be found in the docs
+  - name: Kubeflow Katib Slack Channel
+    url: https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels
+    about: Ask the Katib community on CNCF Slack
+  - name: Kubeflow Katib Community Meeting
+    url: https://bit.ly/2PWVCkV
+    about: Join the Kubeflow AutoML working group meeting

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,14 +0,0 @@
----
-name: Feature enhancement request
-about: Suggest an idea for this project
-
----
-
-/kind feature
-
-**Describe the solution you'd like**
-[A clear and concise description of what you want to happen.]
-
-
-**Anything else you would like to add:**
-[Miscellaneous information that will assist in solving the issue.]

.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -0,0 +1,28 @@
+name: Feature Request
+description: Suggest an idea for Katib
+labels: ["kind/feature", "lifecycle/needs-triage"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to fill out this Katib feature request!
+  - type: textarea
+    id: feature
+    attributes:
+      label: What you would like to be added?
+      description: |
+        A clear and concise description of what you want to add to Katib.
+        Please consider to write Katib enhancement proposal if it is a large feature request.
+    validations:
+      required: true
+  - type: textarea
+    id: rationale
+    attributes:
+      label: Why is this needed?
+    validations:
+      required: true
+  - type: input
+    id: votes
+    attributes:
+      label: Love this feature?
+      value: Give it a 👍 We prioritize the features with most 👍

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,25 +1,14 @@
-<!--  Thanks for sending a pull request!  Here are some tips for you:
-1. If this is your first time, read our contributor guidelines https://git.k8s.io/community/contributors/guide/pull-requests.md#the-pull-request-submit-process and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
-2. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
-3. Follow the instructions for writing a release note: https://git.k8s.io/community/contributors/guide/release-notes.md
-4. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-5. If this PR changes image versions, please title this PR "Bump <image name> from x.x.x to y.y.y."
+<!--  Thanks for sending a pull request! Here are some tips for you:
+1. If this is your first time, check our contributor guidelines https://www.kubeflow.org/docs/about/contributing
+2. To know more about Katib components, check developer guide https://github.com/kubeflow/katib/blob/master/CONTRIBUTING.md
+3. If you want *faster* PR reviews, check how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
 -->
 
 **What this PR does / why we need it**:
 
-**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
+**Which issue(s) this PR fixes** _(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)_:
 Fixes #
 
-**Special notes for your reviewer**:
+**Checklist:**
 
-1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.
-
-**Release note**:
-<!--  Write your release note:
-1. Enter your extended release note in the below block. If the PR requires additional action from users switching to the new release, include the string "action required".
-2. If no release note is required, just write "NONE".
--->
-```release-note
-
-```
+- [ ] [Docs](https://www.kubeflow.org/docs/components/katib/) included if any changes are user facing

.github/workflows/build-and-publish-images.yaml

@@ -0,0 +1,81 @@
+# Reusable workflows for publishing Katib images.
+name: Build and Publish Images
+
+on:
+  workflow_call:
+    inputs:
+      component-name:
+        required: true
+        type: string
+      platforms:
+        required: true
+        type: string
+      dockerfile:
+        required: true
+        type: string
+    secrets:
+      DOCKERHUB_USERNAME:
+        required: false
+      DOCKERHUB_TOKEN:
+        required: false
+
+jobs:
+  build-and-publish:
+    name: Build and Publish Images
+    runs-on: ubuntu-22.04
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Set Publish Condition
+        id: publish-condition
+        shell: bash
+        run: |
+          if [[ "${{ github.repository }}" == 'kubeflow/katib' && \
+                ( "${{ github.ref }}" == 'refs/heads/master' || \
+                  "${{ github.ref }}" =~ ^refs/heads/release- || \
+                  "${{ github.ref }}" =~ ^refs/tags/v ) ]]; then
+            echo "should_publish=true" >> $GITHUB_OUTPUT
+          else
+            echo "should_publish=false" >> $GITHUB_OUTPUT
+          fi
+
+      - name: GHCR Login
+        if: steps.publish-condition.outputs.should_publish == 'true'
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: DockerHub Login
+        if: steps.publish-condition.outputs.should_publish == 'true'
+        uses: docker/login-action@v3
+        with:
+          registry: docker.io
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Publish Component ${{ inputs.component-name }}
+        if: steps.publish-condition.outputs.should_publish == 'true'
+        id: publish
+        uses: ./.github/workflows/template-publish-image
+        with:
+          image: |
+            ghcr.io/kubeflow/katib/${{ inputs.component-name }}
+            docker.io/kubeflowkatib/${{ inputs.component-name }}
+          dockerfile: ${{ inputs.dockerfile }}
+          platforms: ${{ inputs.platforms }}
+          push: true
+
+      - name: Test Build For Component ${{ inputs.component-name }}
+        if: steps.publish.outcome == 'skipped'
+        uses: ./.github/workflows/template-publish-image
+        with:
+          image: |
+            ghcr.io/kubeflow/katib/${{ inputs.component-name }}
+            docker.io/kubeflowkatib/${{ inputs.component-name }}
+          dockerfile: ${{ inputs.dockerfile }}
+          platforms: ${{ inputs.platforms }}
+          push: false

.github/workflows/e2e-test-darts-cifar10.yaml

@@ -0,0 +1,38 @@
+name: E2E Test with darts-cnn-cifar10
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  e2e:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 120
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Test Env
+        uses: ./.github/workflows/template-setup-e2e-test
+        with:
+          kubernetes-version: ${{ matrix.kubernetes-version }}
+          python-version: "3.11"
+
+      - name: Run e2e test with ${{ matrix.experiments }} experiments
+        uses: ./.github/workflows/template-e2e-test
+        with:
+          experiments: ${{ matrix.experiments }}
+          # Comma Delimited
+          trial-images: darts-cnn-cifar10-cpu
+
+    strategy:
+      fail-fast: false
+      matrix:
+        kubernetes-version: ["v1.29.2", "v1.30.7", "v1.31.3"]
+        # Comma Delimited
+        experiments: ["darts-cpu"]

.github/workflows/e2e-test-enas-cifar10.yaml

@@ -0,0 +1,38 @@
+name: E2E Test with enas-cnn-cifar10
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  e2e:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 120
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Test Env
+        uses: ./.github/workflows/template-setup-e2e-test
+        with:
+          kubernetes-version: ${{ matrix.kubernetes-version }}
+          python-version: "3.8"
+
+      - name: Run e2e test with ${{ matrix.experiments }} experiments
+        uses: ./.github/workflows/template-e2e-test
+        with:
+          experiments: ${{ matrix.experiments }}
+          # Comma Delimited
+          trial-images: enas-cnn-cifar10-cpu
+
+    strategy:
+      fail-fast: false
+      matrix:
+        kubernetes-version: ["v1.29.2", "v1.30.7", "v1.31.3"]
+        # Comma Delimited
+        experiments: ["enas-cpu"]

.github/workflows/e2e-test-pytorch-mnist.yaml

@@ -0,0 +1,46 @@
+name: E2E Test with pytorch-mnist
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  e2e:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 120
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Test Env
+        uses: ./.github/workflows/template-setup-e2e-test
+        with:
+          kubernetes-version: ${{ matrix.kubernetes-version }}
+          python-version: "3.10"
+
+      - name: Run e2e test with ${{ matrix.experiments }} experiments
+        uses: ./.github/workflows/template-e2e-test
+        with:
+          experiments: ${{ matrix.experiments }}
+          training-operator: true
+          # Comma Delimited
+          trial-images: pytorch-mnist-cpu
+
+    strategy:
+      fail-fast: false
+      matrix:
+        kubernetes-version: ["v1.29.2", "v1.30.7", "v1.31.3"]
+        # Comma Delimited
+        experiments:
+          # suggestion-hyperopt
+          - "long-running-resume,from-volume-resume,median-stop"
+          # others
+          - "grid,bayesian-optimization,tpe,multivariate-tpe,cma-es,hyperband"
+          - "hyperopt-distribution,optuna-distribution"
+          - "file-metrics-collector,pytorchjob-mnist"
+          - "median-stop-with-json-format,file-metrics-collector-with-json-format"

.github/workflows/e2e-test-simple-pbt.yaml

@@ -0,0 +1,38 @@
+name: E2E Test with simple-pbt
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  e2e:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 120
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Test Env
+        uses: ./.github/workflows/template-setup-e2e-test
+        with:
+          kubernetes-version: ${{ matrix.kubernetes-version }}
+
+      - name: Run e2e test with ${{ matrix.experiments }} experiments
+        uses: ./.github/workflows/template-e2e-test
+        with:
+          experiments: ${{ matrix.experiments }}
+          # Comma Delimited
+          trial-images: simple-pbt
+
+    strategy:
+      fail-fast: false
+      matrix:
+        # Detail: https://hub.docker.com/r/kindest/node
+        kubernetes-version: ["v1.29.2", "v1.30.7", "v1.31.3"]
+        # Comma Delimited
+        experiments: ["simple-pbt"]

.github/workflows/e2e-test-tf-mnist-with-summaries.yaml

@@ -0,0 +1,38 @@
+name: E2E Test with tf-mnist-with-summaries
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  e2e:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 120
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Test Env
+        uses: ./.github/workflows/template-setup-e2e-test
+        with:
+          kubernetes-version: ${{ matrix.kubernetes-version }}
+
+      - name: Run e2e test with ${{ matrix.experiments }} experiments
+        uses: ./.github/workflows/template-e2e-test
+        with:
+          experiments: ${{ matrix.experiments }}
+          training-operator: true
+          # Comma Delimited
+          trial-images: tf-mnist-with-summaries
+
+    strategy:
+      fail-fast: false
+      matrix:
+        kubernetes-version: ["v1.29.2", "v1.30.7", "v1.31.3"]
+        # Comma Delimited
+        experiments: ["tfjob-mnist-with-summaries"]

.github/workflows/e2e-test-tune-api.yaml

@@ -0,0 +1,40 @@
+name: E2E Test with tune API
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  e2e:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 120
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Test Env
+        uses: ./.github/workflows/template-setup-e2e-test
+        with:
+          kubernetes-version: ${{ matrix.kubernetes-version }}
+      
+      - name: Install Katib SDK with extra requires
+        shell: bash
+        run: |
+          pip install --prefer-binary -e 'sdk/python/v1beta1[huggingface]'
+      
+      - name: Run e2e test with tune API
+        uses: ./.github/workflows/template-e2e-test
+        with:
+          tune-api: true
+          training-operator: true
+
+    strategy:
+      fail-fast: false
+      matrix:
+        # Detail: https://hub.docker.com/r/kindest/node
+        kubernetes-version: ["v1.29.2", "v1.30.7", "v1.31.3"]

.github/workflows/e2e-test-ui-random-search-postgres.yaml

@@ -0,0 +1,35 @@
+name: E2E Test with Katib UI, random search, and postgres
+
+on:
+  - pull_request
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  e2e:
+    runs-on: ubuntu-22.04
+    timeout-minutes: 120
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Test Env
+        uses: ./.github/workflows/template-setup-e2e-test
+        with:
+          kubernetes-version: ${{ matrix.kubernetes-version }}
+
+      - name: Run e2e test with ${{ matrix.experiments }} experiments
+        uses: ./.github/workflows/template-e2e-test
+        with:
+          experiments: random
+          # Comma Delimited
+          trial-images: pytorch-mnist-cpu
+          katib-ui: true
+          database-type: postgres
+
+    strategy:
+      fail-fast: false
+      matrix:
+        kubernetes-version: ["v1.29.2", "v1.30.7", "v1.31.3"]

.github/workflows/free-up-disk-space/action.yaml

@@ -0,0 +1,49 @@
+name: Free-Up Disk Space
+description: Remove Non-Essential Tools And Move Docker Data Directory to /mnt/docker
+
+runs:
+  using: composite
+  steps:
+    # This step is a Workaround to avoid the "No space left on device" error.
+    # ref: https://github.com/actions/runner-images/issues/2840
+    - name: Remove unnecessary files
+      shell: bash
+      run: |
+        echo "Disk usage before cleanup:"
+        df -hT
+
+        sudo rm -rf /usr/share/dotnet
+        sudo rm -rf /opt/ghc
+        sudo rm -rf /usr/local/share/boost
+        sudo rm -rf "$AGENT_TOOLSDIRECTORY"
+        sudo rm -rf /usr/local/lib/android
+        sudo rm -rf /usr/local/share/powershell
+        sudo rm -rf /usr/share/swift
+
+        echo "Disk usage after cleanup:"
+        df -hT
+
+    - name: Prune docker images
+      shell: bash
+      run: |
+        docker image prune -a -f
+        docker system df
+        df -hT
+
+    - name: Move docker data directory
+      shell: bash
+      run: |
+        echo "Stopping docker service ..."
+        sudo systemctl stop docker
+        DOCKER_DEFAULT_ROOT_DIR=/var/lib/docker
+        DOCKER_ROOT_DIR=/mnt/docker
+        echo "Moving ${DOCKER_DEFAULT_ROOT_DIR} -> ${DOCKER_ROOT_DIR}"
+        sudo mv ${DOCKER_DEFAULT_ROOT_DIR} ${DOCKER_ROOT_DIR}
+        echo "Creating symlink ${DOCKER_DEFAULT_ROOT_DIR} -> ${DOCKER_ROOT_DIR}"
+        sudo ln -s ${DOCKER_ROOT_DIR} ${DOCKER_DEFAULT_ROOT_DIR}
+        echo "$(sudo ls -l ${DOCKER_DEFAULT_ROOT_DIR})"
+        echo "Starting docker service ..."
+        sudo systemctl daemon-reload
+        sudo systemctl start docker
+        echo "Docker service status:"
+        sudo systemctl --no-pager -l -o short status docker

.github/workflows/publish-algorithm-images.yaml

@@ -0,0 +1,42 @@
+name: Publish AutoML Algorithm Images
+
+on:
+  push:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+jobs:
+  algorithm:
+    name: Publish Image
+    uses: ./.github/workflows/build-and-publish-images.yaml
+    with:
+      component-name: ${{ matrix.component-name }}
+      platforms: linux/amd64,linux/arm64
+      dockerfile: ${{ matrix.dockerfile }}
+    secrets:
+      DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
+      DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - component-name: suggestion-hyperopt
+            dockerfile: cmd/suggestion/hyperopt/v1beta1/Dockerfile
+          - component-name: suggestion-hyperband
+            dockerfile: cmd/suggestion/hyperband/v1beta1/Dockerfile
+          - component-name: suggestion-skopt
+            dockerfile: cmd/suggestion/skopt/v1beta1/Dockerfile
+          - component-name: suggestion-goptuna
+            dockerfile: cmd/suggestion/goptuna/v1beta1/Dockerfile
+          - component-name: suggestion-optuna
+            dockerfile: cmd/suggestion/optuna/v1beta1/Dockerfile
+          - component-name: suggestion-pbt
+            dockerfile: cmd/suggestion/pbt/v1beta1/Dockerfile
+          - component-name: suggestion-enas
+            dockerfile: cmd/suggestion/nas/enas/v1beta1/Dockerfile
+          - component-name: suggestion-darts
+            dockerfile: cmd/suggestion/nas/darts/v1beta1/Dockerfile
+          - component-name: earlystopping-medianstop
+            dockerfile: cmd/earlystopping/medianstop/v1beta1/Dockerfile

.github/workflows/publish-conformance-images.yaml

@@ -0,0 +1,24 @@
+name: Publish Katib Conformance Test Images
+
+on:
+  - push
+  - pull_request
+
+jobs:
+  core:
+    name: Publish Image
+    uses: ./.github/workflows/build-and-publish-images.yaml
+    with:
+      component-name: ${{ matrix.component-name }}
+      platforms: linux/amd64,linux/arm64
+      dockerfile: ${{ matrix.dockerfile }}
+    secrets:
+      DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
+      DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - component-name: katib-conformance
+            dockerfile: Dockerfile.conformance

.github/workflows/publish-core-images.yaml

@@ -0,0 +1,32 @@
+name: Publish Katib Core Images
+
+on:
+  - push
+  - pull_request
+
+jobs:
+  core:
+    name: Publish Image
+    uses: ./.github/workflows/build-and-publish-images.yaml
+    with:
+      component-name: ${{ matrix.component-name }}
+      platforms: linux/amd64,linux/arm64
+      dockerfile: ${{ matrix.dockerfile }}
+    secrets:
+      DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
+      DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - component-name: katib-controller
+            dockerfile: cmd/katib-controller/v1beta1/Dockerfile
+          - component-name: katib-db-manager
+            dockerfile: cmd/db-manager/v1beta1/Dockerfile
+          - component-name: katib-ui
+            dockerfile: cmd/ui/v1beta1/Dockerfile
+          - component-name: file-metrics-collector
+            dockerfile: cmd/metricscollector/v1beta1/file-metricscollector/Dockerfile
+          - component-name: tfevent-metrics-collector
+            dockerfile: cmd/metricscollector/v1beta1/tfevent-metricscollector/Dockerfile

.github/workflows/publish-trial-images.yaml

@@ -0,0 +1,48 @@
+name: Publish Trial Images
+
+on:
+  push:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+jobs:
+  trial:
+    name: Publish Image
+    uses: ./.github/workflows/build-and-publish-images.yaml
+    with:
+      component-name: ${{ matrix.trial-name }}
+      platforms: ${{ matrix.platforms }}
+      dockerfile: ${{ matrix.dockerfile }}
+    secrets:
+      DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
+      DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - trial-name: pytorch-mnist-cpu
+            platforms: linux/amd64,linux/arm64
+            dockerfile: examples/v1beta1/trial-images/pytorch-mnist/Dockerfile.cpu
+          - trial-name: pytorch-mnist-gpu
+            platforms: linux/amd64
+            dockerfile: examples/v1beta1/trial-images/pytorch-mnist/Dockerfile.gpu
+          - trial-name: tf-mnist-with-summaries
+            platforms: linux/amd64,linux/arm64
+            dockerfile: examples/v1beta1/trial-images/tf-mnist-with-summaries/Dockerfile
+          - trial-name: enas-cnn-cifar10-gpu
+            platforms: linux/amd64
+            dockerfile: examples/v1beta1/trial-images/enas-cnn-cifar10/Dockerfile.gpu
+          - trial-name: enas-cnn-cifar10-cpu
+            platforms: linux/amd64,linux/arm64
+            dockerfile: examples/v1beta1/trial-images/enas-cnn-cifar10/Dockerfile.cpu
+          - trial-name: darts-cnn-cifar10-cpu
+            platforms: linux/amd64,linux/arm64
+            dockerfile: examples/v1beta1/trial-images/darts-cnn-cifar10/Dockerfile.cpu
+          - trial-name: darts-cnn-cifar10-gpu
+            platforms: linux/amd64
+            dockerfile: examples/v1beta1/trial-images/darts-cnn-cifar10/Dockerfile.gpu
+          - trial-name: simple-pbt
+            platforms: linux/amd64,linux/arm64
+            dockerfile: examples/v1beta1/trial-images/simple-pbt/Dockerfile

.github/workflows/stale.yaml

@@ -0,0 +1,42 @@
+# This workflow warns and then closes issues and PRs that have had no activity for a specified amount of time.
+#
+# You can adjust the behavior by modifying this file.
+# For more information, see:
+# https://github.com/actions/stale
+name: Mark stale issues and pull requests
+
+on:
+  schedule:
+    - cron: "0 */5 * * *"
+
+jobs:
+  stale:
+    runs-on: ubuntu-22.04
+    permissions:
+      issues: write
+      pull-requests: write
+
+    steps:
+      - uses: actions/stale@v5
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          days-before-stale: 90
+          days-before-close: 20
+          stale-issue-message: >
+            This issue has been automatically marked as stale because it has not had
+            recent activity. It will be closed if no further activity occurs. Thank you
+            for your contributions.
+          close-issue-message: >
+            This issue has been automatically closed because it has not had recent
+            activity. Please comment "/reopen" to reopen it.
+          stale-issue-label: lifecycle/stale
+          exempt-issue-labels: lifecycle/frozen
+          stale-pr-message: >
+            This pull request has been automatically marked as stale because it has not had
+            recent activity. It will be closed if no further activity occurs. Thank you
+            for your contributions.
+          close-pr-message: >
+            This pull request has been automatically closed because it has not had recent
+            activity. Please comment "/reopen" to reopen it.
+          stale-pr-label: lifecycle/stale
+          exempt-pr-labels: lifecycle/frozen

.github/workflows/template-e2e-test/action.yaml

@@ -0,0 +1,49 @@
+# Composite action for e2e tests.
+name: Run E2E Test
+description: Run e2e test using the minikube cluster
+
+inputs:
+  experiments:
+    required: false
+    description: comma delimited experiment name
+    default: ""
+  training-operator:
+    required: false
+    description: whether to deploy training-operator or not
+    default: false
+  trial-images:
+    required: false
+    description: comma delimited trial image name
+    default: ""
+  katib-ui:
+    required: true
+    description: whether to deploy katib-ui or not
+    default: false
+  database-type:
+    required: false
+    description: mysql or postgres
+    default: mysql
+  tune-api:
+    required: true
+    description: whether to execute tune-api test or not
+    default: false
+
+runs:
+  using: composite
+  steps:
+    - name: Setup Minikube Cluster
+      shell: bash
+      run: ./test/e2e/v1beta1/scripts/gh-actions/setup-minikube.sh ${{ inputs.katib-ui }} ${{ inputs.tune-api }} ${{ inputs.trial-images }} ${{ inputs.experiments }}
+
+    - name: Setup Katib
+      shell: bash
+      run: ./test/e2e/v1beta1/scripts/gh-actions/setup-katib.sh ${{ inputs.katib-ui }} ${{ inputs.training-operator }} ${{ inputs.database-type }}
+
+    - name: Run E2E Experiment
+      shell: bash
+      run: |
+        if "${{ inputs.tune-api }}"; then
+          ./test/e2e/v1beta1/scripts/gh-actions/run-e2e-tune-api.sh
+        else
+          ./test/e2e/v1beta1/scripts/gh-actions/run-e2e-experiment.sh ${{ inputs.experiments }}
+        fi

.github/workflows/template-publish-image/action.yaml

@@ -0,0 +1,62 @@
+# Composite action for publishing Katib images.
+name: Build And Publish Container Images
+description: Build MultiPlatform Supporting Container Images
+
+inputs:
+  image:
+    required: true
+    description: image tag
+  dockerfile:
+    required: true
+    description: path for dockerfile
+  platforms:
+    required: true
+    description: linux/amd64 or linux/amd64,linux/arm64
+  push:
+    required: true
+    description: whether to push container images or not
+
+runs:
+  using: composite
+  steps:
+      # This step is a Workaround to avoid the "No space left on device" error.
+      # ref: https://github.com/actions/runner-images/issues/2840
+    - name: Remove unnecessary files
+      shell: bash
+      run: |
+        sudo rm -rf /usr/share/dotnet
+        sudo rm -rf /opt/ghc
+        sudo rm -rf "/usr/local/share/boost"
+        sudo rm -rf "$AGENT_TOOLSDIRECTORY"
+        sudo rm -rf /usr/local/lib/android
+        sudo rm -rf /usr/local/share/powershell
+        sudo rm -rf /usr/share/swift
+
+        echo "Disk usage after cleanup:"
+        df -h
+
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+
+    - name: Set Up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+
+    - name: Add Docker Tags
+      id: meta
+      uses: docker/metadata-action@v5
+      with:
+        images: ${{ inputs.image }}
+        tags: |
+          type=raw,latest
+          type=sha,prefix=v1beta1-
+
+    - name: Build and Push
+      uses: docker/build-push-action@v5
+      with:
+        context: .
+        file: ${{ inputs.dockerfile }}
+        push: ${{ inputs.push }}
+        tags: ${{ steps.meta.outputs.tags }}
+        cache-from: type=gha
+        cache-to: type=gha,mode=max,ignore-error=true
+        platforms: ${{ inputs.platforms }}

.github/workflows/template-setup-e2e-test/action.yaml

@@ -0,0 +1,48 @@
+# Composite action to setup e2e tests.
+name: Setup E2E Test
+description: setup env for e2e test using the minikube cluster
+
+inputs:
+  kubernetes-version:
+    required: true
+    description: kubernetes version
+  python-version:
+    required: false
+    description: Python version
+    # Most latest supporting version
+    default: "3.10"
+
+runs:
+  using: composite
+  steps:
+    # This step is a Workaround to avoid the "No space left on device" error.
+    # ref: https://github.com/actions/runner-images/issues/2840
+    - name: Free-Up Disk Space
+      uses: ./.github/workflows/free-up-disk-space
+
+    - name: Setup kubectl
+      uses: azure/setup-kubectl@v4
+      with:
+        version: ${{ inputs.kubernetes-version }}
+
+    - name: Setup Minikube Cluster
+      uses: medyagh/setup-minikube@v0.0.18
+      with:
+        network-plugin: cni
+        cni: flannel
+        driver: none
+        kubernetes-version: ${{ inputs.kubernetes-version }}
+        minikube-version: 1.34.0
+        start-args: --wait-timeout=120s
+
+    - name: Setup Docker Buildx
+      uses: docker/setup-buildx-action@v3
+
+    - name: Setup Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ inputs.python-version }}
+
+    - name: Install Katib SDK
+      shell: bash
+      run: pip install --prefer-binary -e sdk/python/v1beta1

.github/workflows/test-go.yaml

@@ -0,0 +1,79 @@
+name: Go Test
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  generatetests:
+    name: Generate And Format Test
+    runs-on: ubuntu-22.04
+    env:
+      GOPATH: ${{ github.workspace }}/go
+    defaults:
+      run:
+        working-directory: ${{ env.GOPATH }}/src/github.com/kubeflow/katib
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+        with:
+          path: ${{ env.GOPATH }}/src/github.com/kubeflow/katib
+
+      - name: Setup Go
+        uses: actions/setup-go@v5
+        with:
+          go-version-file: ${{ env.GOPATH }}/src/github.com/kubeflow/katib/go.mod
+          cache-dependency-path: ${{ env.GOPATH }}/src/github.com/kubeflow/katib/go.sum
+
+      - name: Check Go Modules, Generated Go/Python codes, and Format
+        run: make check
+
+  unittests:
+    name: Unit Test
+    runs-on: ubuntu-22.04
+    env:
+      GOPATH: ${{ github.workspace }}/go
+    defaults:
+      run:
+        working-directory: ${{ env.GOPATH }}/src/github.com/kubeflow/katib
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+        with:
+          path: ${{ env.GOPATH }}/src/github.com/kubeflow/katib
+
+      - name: Setup Go
+        uses: actions/setup-go@v5
+        with:
+          go-version-file: ${{ env.GOPATH }}/src/github.com/kubeflow/katib/go.mod
+          cache-dependency-path: ${{ env.GOPATH }}/src/github.com/kubeflow/katib/go.sum
+
+      - name: Run Go test
+        run: go mod download && make test ENVTEST_K8S_VERSION=${{ matrix.kubernetes-version }}
+
+      - name: Coveralls report
+        uses: shogo82148/actions-goveralls@v1
+        with:
+          path-to-profile: coverage.out
+          working-directory: ${{ env.GOPATH }}/src/github.com/kubeflow/katib
+          parallel: true
+
+    strategy:
+      fail-fast: false
+      matrix:
+        # Detail: `setup-envtest list`
+        kubernetes-version: ["1.29.3", "1.30.0", "1.31.0"]
+
+  # notifies that all test jobs are finished.
+  finish:
+    needs: unittests
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: shogo82148/actions-goveralls@v1
+        with:
+          parallel-finished: true

.github/workflows/test-lint.yaml

@@ -0,0 +1,30 @@
+name: Lint Files
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  lint:
+    name: Lint
+    runs-on: ubuntu-22.04
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.9
+
+      - name: Check shell scripts
+        run: make shellcheck
+
+      - name: Run pre-commit
+        uses: pre-commit/action@v3.0.1

.github/workflows/test-node.yaml

@@ -0,0 +1,101 @@
+name: Frontend Test
+
+on:
+  pull_request:
+    paths:
+      - pkg/ui/v1beta1/frontend/**
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    name: Code format and lint
+    runs-on: ubuntu-22.04
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Setup Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: 16.20.2
+
+      - name: Format katib code
+        run: |
+          npm install prettier --prefix ./pkg/ui/v1beta1/frontend
+          make prettier-check
+
+      - name: Lint katib code
+        run: |
+          cd pkg/ui/v1beta1/frontend
+          npm run lint-check
+
+  frontend-unit-tests:
+    name: Frontend Unit Tests
+    runs-on: ubuntu-22.04
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Setup Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: 16.20.2
+
+      - name: Fetch Kubeflow and install common code dependencies
+        run: |
+          COMMIT=$(cat pkg/ui/v1beta1/frontend/COMMIT)
+          cd /tmp && git clone https://github.com/kubeflow/kubeflow.git
+          cd kubeflow
+          git checkout $COMMIT
+          cd components/crud-web-apps/common/frontend/kubeflow-common-lib
+          npm i
+          npm run build
+          npm link ./dist/kubeflow
+
+      - name: Install KWA dependencies
+        run: |
+          cd pkg/ui/v1beta1/frontend
+          npm i
+          npm link kubeflow
+
+      - name: Run unit tests
+        run: |
+          cd pkg/ui/v1beta1/frontend
+          npm run test:prod
+
+  frontend-ui-tests:
+    name: UI tests with Cypress
+    runs-on: ubuntu-22.04
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Setup node version to 16
+        uses: actions/setup-node@v4
+        with:
+          node-version: 16
+
+      - name: Fetch Kubeflow and install common code dependencies
+        run: |
+          COMMIT=$(cat pkg/ui/v1beta1/frontend/COMMIT)
+          cd /tmp && git clone https://github.com/kubeflow/kubeflow.git
+          cd kubeflow
+          git checkout $COMMIT
+          cd components/crud-web-apps/common/frontend/kubeflow-common-lib
+          npm i
+          npm run build
+          npm link ./dist/kubeflow
+      - name: Install KWA dependencies
+        run: |
+          cd pkg/ui/v1beta1/frontend
+          npm i
+          npm link kubeflow
+      - name: Serve UI & run Cypress tests in Chrome and Firefox
+        run: |
+          cd pkg/ui/v1beta1/frontend
+          npm run start & npx wait-on http://localhost:4200
+          npm run ui-test-ci-all

.github/workflows/test-python.yaml

@@ -0,0 +1,47 @@
+name: Python Test
+
+on:
+  pull_request:
+    paths-ignore:
+      - "pkg/ui/v1beta1/frontend/**"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    name: Test
+    runs-on: ubuntu-22.04
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.11
+
+      - name: Run Python test
+        run: make pytest
+
+  # The skopt service doesn't work appropriately with Python 3.11.
+  # So, we need to run the test with Python 3.9.
+  # TODO (tenzen-y): Once we stop to support skopt, we can remove this test.
+  # REF: https://github.com/kubeflow/katib/issues/2280
+  test-skopt:
+    name: Test Skopt
+    runs-on: ubuntu-22.04
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.9
+
+      - name: Run Python test
+        run: make pytest-skopt

.gitignore

@@ -4,6 +4,12 @@ __pycache__/
 .coverage
 .pytest_cache
 *.egg-info
+build/
+*.charm
+test/unit/v1beta1/metricscollector/testdata
+
+# SDK generator JAR file
+hack/gen-python-sdk/openapi-generator-cli.jar
 
 # Project specific ignore files
 *.swp
@@ -16,6 +22,7 @@ bin
 *.dll
 *.so
 *.dylib
+pkg/metricscollector/v1beta1/file-metricscollector/testdata
 
 ## Test binary, build with `go test -c`
 *.test
@@ -68,3 +75,9 @@ $RECYCLE.BIN/
 /katib-controller
 /katib-db-manager
 /katib-ui
+
+## Vendor dir
+vendor
+
+# Jupyter Notebooks.
+**/.ipynb_checkpoints

.pre-commit-config.yaml

@@ -0,0 +1,38 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v2.3.0
+    hooks:
+      - id: check-yaml
+        args: [--allow-multiple-documents]
+      - id: check-json
+  - repo: https://github.com/pycqa/isort
+    rev: 5.11.5
+    hooks:
+      - id: isort
+        name: isort
+        entry: isort --profile black
+  - repo: https://github.com/psf/black
+    rev: 24.2.0
+    hooks:
+      - id: black
+        files: (sdk|examples|pkg)/.*
+  - repo: https://github.com/pycqa/flake8
+    rev: 7.1.1
+    hooks:
+      - id: flake8
+        files: (sdk|examples|pkg)/.*
+exclude: |
+  (?x)^(
+    .*zz_generated.deepcopy.*|
+    .*pb.go|
+    pkg/apis/manager/.*pb2(?:_grpc)?.py(?:i)?|
+    pkg/apis/v1beta1/openapi_generated.go|
+    pkg/mock/.*|
+    pkg/client/controller/.*|
+    sdk/python/v1beta1/kubeflow/katib/configuration.py|
+    sdk/python/v1beta1/kubeflow/katib/rest.py|
+    sdk/python/v1beta1/kubeflow/katib/__init__.py|
+    sdk/python/v1beta1/kubeflow/katib/exceptions.py|
+    sdk/python/v1beta1/kubeflow/katib/api_client.py|
+    sdk/python/v1beta1/kubeflow/katib/models/.*
+  )$

.travis.yml

@@ -1,27 +0,0 @@
-jobs:
-  include:
-    - name: "Go unit tests, gofmt, golint and coveralls"
-      language: go
-      go: "1.14.2"
-      go_import_path: github.com/kubeflow/katib
-      install:
-        - curl -L -O "https://github.com/kubernetes-sigs/kubebuilder/releases/download/v1.0.7/kubebuilder_1.0.7_linux_amd64.tar.gz"
-        -  # extract the archive
-        - tar -zxvf kubebuilder_1.0.7_linux_amd64.tar.gz
-        - sudo mv kubebuilder_1.0.7_linux_amd64 /usr/local/kubebuilder
-        - export PATH=$PATH:/usr/local/kubebuilder/bin
-        # get coveralls.io support
-        - go get github.com/mattn/goveralls
-      script:
-        - make check
-        - make test
-      after_success:
-        - goveralls -coverprofile=coverage.out
-    - name: "Prettier frontend check"
-      language: node_js
-      node_js: "12.18.1"
-      install:
-        - npm install --global prettier@1.19.1
-      script:
-        - make prettier-check-v1alpha3
-        - make prettier-check

ADOPTERS.md

@@ -0,0 +1,20 @@
+# Adopters of Kubeflow Katib
+
+Below are the adopters of project Katib. If you are using Katib
+please add yourself into the following list by a pull request.
+Please keep the list in alphabetical order.
+
+| Organization                                     | Contact                                              | Description of Use                                                   |
+|--------------------------------------------------|------------------------------------------------------|----------------------------------------------------------------------|
+| [Akuity](https://akuity.io/)                     | [@terrytangyuan](https://github.com/terrytangyuan)   |                                                                      |
+| [Ant Group](https://www.antgroup.com/)           | [@ohmystack](https://github.com/ohmystack)           | Automatic training in Ant Group internal AI Platform                 |
+| [babylon health](https://www.babylonhealth.com/) | [@jeremievallee](https://github.com/jeremievallee)   | Hyperparameter tuning for AIR internal AI Platform                   |
+| [caicloud](https://caicloud.io/)                 | [@gaocegege](https://github.com/gaocegege)           | Hyperparameter tuning in Caicloud Cloud-Native AI Platform           |
+| [canonical](https://ubuntu.com/)                 | [@RFMVasconcelos](https://github.com/rfmvasconcelos) | Hyperparameter tuning for customer projects in Defense and Fintech   |
+| [CERN](https://home.cern/)                       | [@d-gol](https://github.com/d-gol)                   | Hyperparameter tuning within the ML platform on private cloud   |
+| [cisco](https://cisco.com/)                      | [@ramdootp](https://github.com/ramdootp)             | Hyperparameter tuning for conversational AI interface using Rasa     |
+| [cubonacci](https://www.cubonacci.com)           | [@janvdvegt](https://github.com/janvdvegt)           | Hyperparameter tuning within the Cubonacci machine learning platform |
+| [CyberAgent](https://www.cyberagent.co.jp/en/)   | [@tenzen-y](https://github.com/tenzen-y)             | Experiment in CyberAgent internal ML Platform on Private Cloud       |
+| [fuzhi](http://www.fuzhi.ai/)                    | [@planck0591](https://github.com/planck0591)         | Experiment and Trial in autoML Platform                              |
+| [karrot](https://uk.karrotmarket.com/)           | [@muik](https://github.com/muik)                     | Hyperparameter tuning in Karrot ML Platform                          |
+| [PITS Global Data Recovery Services](https://www.pitsdatarecovery.net/) | [@pheianox](https://github.com/pheianox) | CyberAgent and ML Platform |

CITATION.cff

@@ -0,0 +1,43 @@
+cff-version: 1.2.0
+message: "If you use Katib in your scientific publication, please cite it as below."
+authors:
+  - family-names: "George"
+    given-names: "Johnu"
+  - family-names: "Gao"
+    given-names: "Ce"
+  - family-names: "Liu"
+    given-names: "Richard"
+  - family-names: "Liu"
+    given-names: "Hou Gang"
+  - family-names: "Tang"
+    given-names: "Yuan"
+  - family-names: "Pydipaty"
+    given-names: "Ramdoot"
+  - family-names: "Saha"
+    given-names: "Amit Kumar"
+title: "Katib"
+type: software
+repository-code: "https://github.com/kubeflow/katib"
+preferred-citation:
+  type: misc
+  title: "A Scalable and Cloud-Native Hyperparameter Tuning System"
+  authors:
+    - family-names: "George"
+      given-names: "Johnu"
+    - family-names: "Gao"
+      given-names: "Ce"
+    - family-names: "Liu"
+      given-names: "Richard"
+    - family-names: "Liu"
+      given-names: "Hou Gang"
+    - family-names: "Tang"
+      given-names: "Yuan"
+    - family-names: "Pydipaty"
+      given-names: "Ramdoot"
+    - family-names: "Saha"
+      given-names: "Amit Kumar"
+  year: 2020
+  url: "https://arxiv.org/abs/2006.02085"
+  identifiers:
+    - type: "other"
+      value: "arXiv:2006.02085"

CONTRIBUTING.md

@@ -0,0 +1,167 @@
+# Developer Guide
+
+This developer guide is for people who want to contribute to the Katib project.
+If you're interesting in using Katib in your machine learning project,
+see the following guides:
+
+- [Getting started with Katib](https://kubeflow.org/docs/components/katib/hyperparameter/).
+- [How to configure Katib Experiment](https://kubeflow.org/docs/components/katib/experiment/).
+- [Katib architecture and concepts](https://www.kubeflow.org/docs/components/katib/reference/architecture/)
+  for hyperparameter tuning and neural architecture search.
+
+## Requirements
+
+- [Go](https://golang.org/) (1.22 or later)
+- [Docker](https://docs.docker.com/) (24.0 or later)
+- [Docker Buildx](https://docs.docker.com/build/buildx/) (0.8.0 or later)
+- [Java](https://docs.oracle.com/javase/8/docs/technotes/guides/install/install_overview.html) (8 or later)
+- [Python](https://www.python.org/) (3.11 or later)
+- [kustomize](https://kustomize.io/) (4.0.5 or later)
+- [pre-commit](https://pre-commit.com/)
+
+## Build from source code
+
+**Note** that your Docker Desktop should
+[enable containerd image store](https://docs.docker.com/desktop/containerd/#enable-the-containerd-image-store)
+to build multi-arch images. Check source code as follows:
+
+```bash
+make build REGISTRY=<image-registry> TAG=<image-tag>
+```
+
+If you are using an Apple Silicon machine and encounter the "rosetta error: bss_size overflow," go to Docker Desktop -> General and uncheck "Use Rosetta for x86_64/amd64 emulation on Apple Silicon."
+
+To use your custom images for the Katib components, modify
+[Kustomization file](https://github.com/kubeflow/katib/blob/master/manifests/v1beta1/installs/katib-standalone/kustomization.yaml)
+and [Katib Config](https://github.com/kubeflow/katib/blob/master/manifests/v1beta1/installs/katib-standalone/katib-config.yaml)
+
+You can deploy Katib v1beta1 manifests into a Kubernetes cluster as follows:
+
+```bash
+make deploy
+```
+
+You can undeploy Katib v1beta1 manifests from a Kubernetes cluster as follows:
+
+```bash
+make undeploy
+```
+
+## Technical and style guide
+
+The following guidelines apply primarily to Katib,
+but other projects like [Training Operator](https://github.com/kubeflow/training-operator) might also adhere to them.
+
+## Go Development
+
+When coding:
+
+- Follow [effective go](https://go.dev/doc/effective_go) guidelines.
+- Run locally [`make check`](https://github.com/kubeflow/katib/blob/46173463027e4fd2e604e25d7075b2b31a702049/Makefile#L31)
+  to verify if changes follow best practices before submitting PRs.
+
+Testing:
+
+- Use [`cmp.Diff`](https://pkg.go.dev/github.com/google/go-cmp/cmp#Diff) instead of `reflect.Equal`, to provide useful comparisons.
+- Define test cases as maps instead of slices to avoid dependencies on the running order.
+  Map key should be equal to the test case name.
+
+## Modify controller APIs
+
+If you want to modify Katib controller APIs, you have to
+generate deepcopy, clientset, listers, informers, open-api and Python SDK with the changed APIs.
+You can update the necessary files as follows:
+
+```bash
+make generate
+```
+
+## Controller Flags
+
+Below is a list of command-line flags accepted by Katib controller:
+
+| Name         | Type   | Default | Description                                                                                                                      |
+| ------------ | ------ | ------- | -------------------------------------------------------------------------------------------------------------------------------- |
+| katib-config | string | ""      | The katib-controller will load its initial configuration from this file. Omit this flag to use the default configuration values. |
+
+## DB Manager Flags
+
+Below is a list of command-line flags accepted by Katib DB Manager:
+
+| Name            | Type          | Default      | Description                                                         |
+| --------------- | ------------- | -------------| ------------------------------------------------------------------- |
+| connect-timeout | time.Duration | 60s          | Timeout before calling error during database connection             |
+| listen-address  | string        | 0.0.0.0:6789 | The network interface or IP address to receive incoming connections |
+
+## Katib admission webhooks
+
+Katib uses three [Kubernetes admission webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/).
+
+1. `validator.experiment.katib.kubeflow.org` -
+   [Validating admission webhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook)
+   to validate the Katib Experiment before the creation.
+
+1. `defaulter.experiment.katib.kubeflow.org` -
+   [Mutating admission webhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook)
+   to set the [default values](../pkg/apis/controller/experiments/v1beta1/experiment_defaults.go)
+   in the Katib Experiment before the creation.
+
+1. `mutator.pod.katib.kubeflow.org` - Mutating admission webhook to inject the metrics
+   collector sidecar container to the training pod. Learn more about the Katib's
+   metrics collector in the
+   [Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/user-guides/metrics-collector/).
+
+You can find the YAMLs for the Katib webhooks
+[here](../manifests/v1beta1/components/webhook/webhooks.yaml).
+
+**Note:** If you are using a private Kubernetes cluster, you have to allow traffic
+via `TCP:8443` by specifying the firewall rule and you have to update the master
+plane CIDR source range to use the Katib webhooks
+
+### Katib cert generator
+
+Katib Controller has the internal `cert-generator` to generate certificates for the webhooks.
+
+Once Katib is deployed in the Kubernetes cluster, the `cert-generator` follows these steps:
+
+- Generate the self-signed certificate and private key.
+
+- Update a Kubernetes Secret with the self-signed TLS certificate and private key.
+- Patch the webhooks with the `CABundle`.
+
+Once the `cert-generator` finished, the Katib controller starts to register controllers such as `experiment-controller` to the manager.
+
+You can find the `cert-generator` source code [here](../pkg/certgenerator/v1beta1).
+
+NOTE: the Katib also supports the [cert-manager](https://cert-manager.io/) to generate certs for the admission webhooks instead of using cert-generator.
+You can find the installation with the cert-manager [here](../manifests/v1beta1/installs/katib-cert-manager).
+
+## Implement a new algorithm and use it in Katib
+
+Please see [new-algorithm-service.md](./new-algorithm-service.md).
+
+## Katib UI documentation
+
+Please see [Katib UI README](../pkg/ui/v1beta1).
+
+## Design proposals
+
+Please see [proposals](./proposals).
+
+## Code Style
+
+### pre-commit
+
+Make sure to install [pre-commit](https://pre-commit.com/) (`pip install
+pre-commit`) and run `pre-commit install` from the root of the repository at
+least once before creating git commits.
+
+The pre-commit [hooks](../.pre-commit-config.yaml) ensure code quality and
+consistency. They are executed in CI. PRs that fail to comply with the hooks
+will not be able to pass the corresponding CI gate. The hooks are only executed
+against staged files unless you run `pre-commit run --all`, in which case,
+they'll be executed against every file in the repository.
+
+Specific programmatically generated files listed in the `exclude` field in
+[.pre-commit-config.yaml](../.pre-commit-config.yaml) are deliberately excluded
+from the hooks.

Dockerfile.conformance

@@ -0,0 +1,32 @@
+# Copyright 2023 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Dockerfile for building the source code of conformance tests
+FROM python:3.10-slim
+
+WORKDIR /kubeflow/katib
+
+COPY sdk/ /kubeflow/katib/sdk/
+COPY examples/ /kubeflow/katib/examples/
+COPY test/ /kubeflow/katib/test/
+COPY pkg/ /kubeflow/katib/pkg/
+
+COPY conformance/run.sh .
+
+# Add test script.
+RUN chmod +x run.sh
+
+RUN pip install --prefer-binary -e sdk/python/v1beta1
+
+ENTRYPOINT [ "./run.sh" ]

Gopkg.toml

@@ -1,129 +0,0 @@
-required = [
-    "github.com/emicklei/go-restful",
-    "github.com/onsi/ginkgo", # for test framework
-    "github.com/onsi/gomega", # for test matchers
-    "k8s.io/client-go/plugin/pkg/client/auth/gcp", # for development against gcp
-    "k8s.io/code-generator/cmd/deepcopy-gen", # for deepcopy generation
-    "k8s.io/code-generator/cmd/openapi-gen", # for openapi generation
-    "sigs.k8s.io/controller-tools/cmd/controller-gen", # for crd/rbac generation
-    "sigs.k8s.io/controller-runtime/pkg/client/config",
-    "sigs.k8s.io/controller-runtime/pkg/controller",
-    "sigs.k8s.io/controller-runtime/pkg/handler",
-    "sigs.k8s.io/controller-runtime/pkg/manager",
-    "sigs.k8s.io/controller-runtime/pkg/runtime/signals",
-    "sigs.k8s.io/controller-runtime/pkg/source",
-    "sigs.k8s.io/testing_frameworks/integration", # for integration testing
-    "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1",
-    "github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway",
-    "github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger",
-    "github.com/golang/protobuf/protoc-gen-go",
-    ]
-
-[prune]
-  go-tests = true
-  unused-packages = true
-  non-go = true
-
-[[constraint]]
-  name = "github.com/go-sql-driver/mysql"
-  version = "1.4.0"
-
-[[constraint]]
-  name = "github.com/golang/mock"
-  version = "1.4.3"
-
-[[override]]
-  name = "github.com/google/go-containerregistry"
-  # HEAD as of 2019-03-20
-  revision = "8d4083db9aa0d2fae6588c1acdbe6a1f5db461e3"
-
-[[constraint]]
-  name = "github.com/golang/protobuf"
-  version = "1.2.0"
-
-[[constraint]]
-  name = "github.com/spf13/viper"
-  version = "1.2.0"
-
-[[constraint]]
-  name = "golang.org/x/net"
-  revision="640f4622ab692b87c2f3a94265e6f579fe38263d"
-
-[[constraint]]
-  name = "gopkg.in/DATA-DOG/go-sqlmock.v1"
-  version = "1.3.0"
-
-[[constraint]]
-  name = "sigs.k8s.io/controller-runtime"
-  version = "0.1.9"
-
-[[override]]
-  name = "k8s.io/api"
-  version = "kubernetes-1.12.9"
-
-[[override]]
-  name = "k8s.io/apimachinery"
-  version = "kubernetes-1.12.9"
-
-[[override]]
-  name = "k8s.io/code-generator"
-  version = "kubernetes-1.12.9"
-
-[[override]]
-  name = "k8s.io/client-go"
-  version = "kubernetes-1.12.9"
-
-[[override]]
-  name = "k8s.io/apiextensions-apiserver"
-  version = "kubernetes-1.12.9"
-
-[[override]]
-  name = "k8s.io/kubernetes"
-  version = "v1.13.3"
-
-[[override]]
-  name = "gopkg.in/fsnotify.v1"
-  source = "https://github.com/fsnotify/fsnotify.git"
-  version="v1.4.7"
-
-[[constraint]]
-  name = "github.com/kubeflow/tf-operator"
-  branch = "v0.7-branch"
-
-[[constraint]]
-  name = "github.com/kubeflow/pytorch-operator"
-  branch = "v0.7-branch"
-
-[[constraint]]
-  name = "github.com/awalterschulze/gographviz"
-  branch = "master"
-
-[[constraint]]
-  name = "github.com/c-bata/goptuna"
-  version = "v0.5.1"
-
-[[prune.project]]
-  name = "github.com/kubeflow/katib"
-  unused-packages = false
-  non-go = false
-
-[[prune.project]]
-  name = "k8s.io/code-generator"
-  unused-packages = false
-  non-go = false
-
-[[prune.project]]
-  name = "k8s.io/gengo"
-  unused-packages = false
-
-[[constraint]]
-  name = "github.com/grpc-ecosystem/go-grpc-middleware"
-  version = "1.2.0"
-
-[[constraint]]
-  name = "github.com/tidwall/gjson"
-  version = "1.6.0"
-
-[[constraint]]
-  name = "github.com/shirou/gopsutil"
-  version = "2.20.7"

Makefile

@@ -1,75 +1,193 @@
-HAS_DEP := $(shell command -v dep;)
-HAS_LINT := $(shell command -v golint;)
+HAS_LINT := $(shell command -v golangci-lint;)
+HAS_YAMLLINT := $(shell command -v yamllint;)
+HAS_SHELLCHECK := $(shell command -v shellcheck;)
+HAS_SETUP_ENVTEST := $(shell command -v setup-envtest;)
+HAS_MOCKGEN := $(shell command -v mockgen;)
+
+COMMIT := v1beta1-$(shell git rev-parse --short=7 HEAD)
+KATIB_REGISTRY := ghcr.io/kubeflow/katib
+CPU_ARCH ?= linux/amd64,linux/arm64
+ENVTEST_K8S_VERSION ?= 1.31
+MOCKGEN_VERSION ?= $(shell grep 'go.uber.org/mock' go.mod | cut -d ' ' -f 2)
+GO_VERSION=$(shell grep '^go' go.mod | cut -d ' ' -f 2)
+GOPATH ?= $(shell go env GOPATH)
+
+TEST_TENSORFLOW_EVENT_FILE_PATH ?= $(CURDIR)/test/unit/v1beta1/metricscollector/testdata/tfevent-metricscollector/logs
 
 # Run tests
 .PHONY: test
-test:
-	go test ./pkg/... ./cmd/... -coverprofile coverage.out
+test: envtest
+	KUBEBUILDER_ASSETS="$(shell setup-envtest use $(ENVTEST_K8S_VERSION) -p path)" go test ./pkg/... ./cmd/... -coverprofile coverage.out
 
-depend:
-ifndef HAS_DEP
-	curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
+envtest:
+ifndef HAS_SETUP_ENVTEST
+	go install sigs.k8s.io/controller-runtime/tools/setup-envtest@release-0.19
+	$(info "setup-envtest has been installed")
 endif
-	dep ensure -v
+	$(info "setup-envtest has already installed")
 
-check: depend generate fmt vet lint
+check: generated-codes go-mod fmt vet lint
 
 fmt:
 	hack/verify-gofmt.sh
 
 lint:
 ifndef HAS_LINT
-	go get -u golang.org/x/lint/golint
-	echo "installing golint"
+	go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.64.7
+	$(info "golangci-lint has been installed")
 endif
-	hack/verify-golint.sh
+	hack/verify-golangci-lint.sh
+
+yamllint:
+ifndef HAS_YAMLLINT
+	pip install --prefer-binary yamllint
+	$(info "yamllint has been installed")
+endif
+	hack/verify-yamllint.sh
 
 vet:
 	go vet ./pkg/... ./cmd/...
 
+shellcheck:
+ifndef HAS_SHELLCHECK
+	bash hack/install-shellcheck.sh
+	$(info "shellcheck has been installed")
+endif
+	hack/verify-shellcheck.sh
+
 update:
 	hack/update-gofmt.sh
 
-# Deploy Katib v1alpha3 manifests into a k8s cluster
-deployv1alpha3:
-	bash scripts/v1alpha3/deploy.sh
-
-# Deploy Katib v1beta1 manifests into a k8s cluster
+# Deploy Katib v1beta1 manifests using Kustomize into a k8s cluster.
 deploy:
-	bash scripts/v1beta1/deploy.sh
+	bash scripts/v1beta1/deploy.sh $(WITH_DATABASE_TYPE)
 
-# Undeploy Katib v1alpha3 manifests from a k8s cluster
-undeployv1alpha3:
-	bash scripts/v1alpha3/undeploy.sh
-
-# Undeploy Katib v1beta1 manifests from a k8s cluster
+# Undeploy Katib v1beta1 manifests using Kustomize from a k8s cluster
 undeploy:
 	bash scripts/v1beta1/undeploy.sh
 
-# Generate deepcopy, clientset, listers, informers, open-api and python SDK for APIs.
+generated-codes: generate
+ifneq ($(shell bash hack/verify-generated-codes.sh '.'; echo $$?),0)
+	$(error 'Please run "make generate" to generate codes')
+endif
+
+go-mod: sync-go-mod
+ifneq ($(shell bash hack/verify-generated-codes.sh 'go.*'; echo $$?),0)
+	$(error 'Please run "go mod tidy -go $(GO_VERSION)" to sync Go modules')
+endif
+
+sync-go-mod:
+	go mod tidy -go $(GO_VERSION)
+
+.PHONY: go-mod-download
+go-mod-download:
+	go mod download
+
+CONTROLLER_GEN = $(shell pwd)/bin/controller-gen
+.PHONY: controller-gen
+controller-gen:
+	@GOBIN=$(shell pwd)/bin GO111MODULE=on go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.5
+
 # Run this if you update any existing controller APIs.
-generate:
-ifndef GOPATH
-	$(error GOPATH not defined, please define GOPATH. Run "go help gopath" to learn more about GOPATH)
+# 1. Generate deepcopy, clientset, listers, informers for the APIs (hack/update-codegen.sh)
+# 2. Generate open-api for the APIs (hack/update-openapigen)
+# 3. Generate Python SDK for Katib (hack/gen-python-sdk/gen-sdk.sh)
+# 4. Generate gRPC manager APIs (pkg/apis/manager/v1beta1/build.sh and pkg/apis/manager/health/build.sh)
+# 5. Generate Go mock codes
+generate: go-mod-download controller-gen
+ifndef HAS_MOCKGEN
+	go install go.uber.org/mock/mockgen@$(MOCKGEN_VERSION)
+	$(info "mockgen has been installed")
 endif
 	go generate ./pkg/... ./cmd/...
 	hack/gen-python-sdk/gen-sdk.sh
+	hack/update-proto.sh
+	hack/update-mockgen.sh
 
-# Build images for Katib v1alpha3 components
-buildv1alpha3: depend generate
-	bash scripts/v1alpha3/build.sh
-
-# Build images for Katib v1beta1 components
-build: depend generate
-ifeq ($(and $(REGISTRY),$(TAG)),)
-	$(error REGISTRY and TAG must be set. Usage make build REGISTRY=<registry> TAG=<TAG>)
+# Build images for the Katib v1beta1 components.
+build: generate
+ifeq ($(and $(REGISTRY),$(TAG),$(CPU_ARCH)),)
+	$(error REGISTRY and TAG must be set. Usage: make build REGISTRY=<registry> TAG=<tag> CPU_ARCH=<cpu-architecture>)
 endif
-	bash scripts/v1beta1/build.sh -r $(REGISTRY) -t $(TAG)
+	bash scripts/v1beta1/build.sh $(REGISTRY) $(TAG) $(CPU_ARCH)
 
-# Prettier UI format check for Katib v1alpha3
-prettier-check-v1alpha3:
-	npm run format:check --prefix pkg/ui/v1alpha3/frontend
+# Build and push Katib images from the latest master commit.
+push-latest: generate
+	bash scripts/v1beta1/build.sh $(KATIB_REGISTRY) latest $(CPU_ARCH)
+	bash scripts/v1beta1/build.sh $(KATIB_REGISTRY) $(COMMIT) $(CPU_ARCH)
+	bash scripts/v1beta1/push.sh $(KATIB_REGISTRY) latest
+	bash scripts/v1beta1/push.sh $(KATIB_REGISTRY) $(COMMIT)
 
-# Prettier UI format check for Katib v1beta1
+# Build and push Katib images for the given tag.
+push-tag:
+ifeq ($(TAG),)
+	$(error TAG must be set. Usage: make push-tag TAG=<release-tag>)
+endif
+	bash scripts/v1beta1/build.sh $(KATIB_REGISTRY) $(TAG) $(CPU_ARCH)
+	bash scripts/v1beta1/push.sh $(KATIB_REGISTRY) $(TAG)
+
+# Release a new version of Katib.
+release:
+ifeq ($(and $(BRANCH),$(TAG)),)
+	$(error BRANCH and TAG must be set. Usage: make release BRANCH=<branch> TAG=<tag>)
+endif
+	bash scripts/v1beta1/release.sh $(BRANCH) $(TAG)
+
+# Update all Katib images.
+update-images:
+ifeq ($(and $(OLD_PREFIX),$(NEW_PREFIX),$(TAG)),)
+	$(error OLD_PREFIX, NEW_PREFIX, and TAG must be set. \
+	Usage: make update-images OLD_PREFIX=<old-prefix> NEW_PREFIX=<new-prefix> TAG=<tag> \
+	For more information, check this file: scripts/v1beta1/update-images.sh)
+endif
+	bash scripts/v1beta1/update-images.sh $(OLD_PREFIX) $(NEW_PREFIX) $(TAG)
+
+# Prettier UI format check for Katib v1beta1.
 prettier-check:
 	npm run format:check --prefix pkg/ui/v1beta1/frontend
+
+# Update boilerplate for the source code.
+update-boilerplate:
+	./hack/boilerplate/update-boilerplate.sh
+
+prepare-pytest:
+	pip install --prefer-binary -r test/unit/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/suggestion/hyperopt/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/suggestion/optuna/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/suggestion/hyperband/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/suggestion/nas/enas/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/suggestion/nas/darts/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/suggestion/pbt/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/earlystopping/medianstop/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/metricscollector/v1beta1/tfevent-metricscollector/requirements.txt
+	# `TypeIs` was introduced in typing-extensions 4.10.0, and torch 2.6.0 requires typing-extensions>=4.10.0.
+	# REF: https://github.com/kubeflow/katib/pull/2504
+	# TODO (tenzen-y): Once we upgrade libraries depended on typing-extensions==4.5.0, we can remove this line.
+	pip install typing-extensions==4.10.0
+
+prepare-pytest-testdata:
+ifeq ("$(wildcard $(TEST_TENSORFLOW_EVENT_FILE_PATH))", "")
+	python examples/v1beta1/trial-images/tf-mnist-with-summaries/mnist.py --epochs 5 --batch-size 200 --log-path $(TEST_TENSORFLOW_EVENT_FILE_PATH)
+endif
+
+# TODO(Electronic-Waste): Remove the import rewrite when protobuf supports `python_package` option.
+# REF: https://github.com/protocolbuffers/protobuf/issues/7061
+pytest: prepare-pytest prepare-pytest-testdata
+	pytest ./test/unit/v1beta1/suggestion --ignore=./test/unit/v1beta1/suggestion/test_skopt_service.py
+	pytest ./test/unit/v1beta1/earlystopping
+	pytest ./test/unit/v1beta1/metricscollector
+	cp ./pkg/apis/manager/v1beta1/python/api_pb2.py ./sdk/python/v1beta1/kubeflow/katib/katib_api_pb2.py
+	cp ./pkg/apis/manager/v1beta1/python/api_pb2_grpc.py ./sdk/python/v1beta1/kubeflow/katib/katib_api_pb2_grpc.py
+	sed -i "s/api_pb2/kubeflow\.katib\.katib_api_pb2/g" ./sdk/python/v1beta1/kubeflow/katib/katib_api_pb2_grpc.py
+	pytest ./sdk/python/v1beta1/kubeflow/katib
+	rm ./sdk/python/v1beta1/kubeflow/katib/katib_api_pb2.py ./sdk/python/v1beta1/kubeflow/katib/katib_api_pb2_grpc.py
+
+# The skopt service doesn't work appropriately with Python 3.11.
+# So, we need to run the test with Python 3.9.
+# TODO (tenzen-y): Once we stop to support skopt, we can remove this test.
+# REF: https://github.com/kubeflow/katib/issues/2280
+pytest-skopt:
+	pip install six
+	pip install --prefer-binary -r test/unit/v1beta1/requirements.txt
+	pip install --prefer-binary -r cmd/suggestion/skopt/v1beta1/requirements.txt
+	pytest ./test/unit/v1beta1/suggestion/test_skopt_service.py

OWNERS

@@ -1,8 +1,10 @@
 approvers:
-    - gaocegege
-    - hougangliu
-    - johnugeorge
-    - andreyvelich
+  - andreyvelich
+  - gaocegege
+  - johnugeorge
 reviewers:
-    - sperlingxx
-    - c-bata
+  - anencore94
+  - c-bata
+  - Electronic-Waste
+emeritus_approvers:
+  - tenzen-y

PROJECT

@@ -1,3 +1,3 @@
-version: "1"
+version: "3"
 domain: kubeflow.org
 repo: github.com/kubeflow/katib

README.md

@@ -1,439 +1,213 @@
+# Kubeflow Katib
+
+[![Build Status](https://github.com/kubeflow/katib/actions/workflows/test-go.yaml/badge.svg?branch=master)](https://github.com/kubeflow/katib/actions/workflows/test-go.yaml?branch=master)
+[![Coverage Status](https://coveralls.io/repos/github/kubeflow/katib/badge.svg?branch=master)](https://coveralls.io/github/kubeflow/katib?branch=master)
+[![Go Report Card](https://goreportcard.com/badge/github.com/kubeflow/katib)](https://goreportcard.com/report/github.com/kubeflow/katib)
+[![Releases](https://img.shields.io/github/release-pre/kubeflow/katib.svg?sort=semver)](https://github.com/kubeflow/katib/releases)
+[![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels)
+[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/9941/badge)](https://www.bestpractices.dev/projects/9941)
+
 <h1 align="center">
-    <img src="./docs/images/Katib_Logo.png" alt="logo" width="200">
+    <img src="./docs/images/logo-title.png" alt="logo" width="200">
   <br>
 </h1>
 
-[![Build Status](https://travis-ci.org/kubeflow/katib.svg?branch=master)](https://travis-ci.org/kubeflow/katib)
-[![Coverage Status](https://coveralls.io/repos/github/kubeflow/katib/badge.svg?branch=master)](https://coveralls.io/github/kubeflow/katib?branch=master)
-[![Go Report Card](https://goreportcard.com/badge/github.com/kubeflow/katib)](https://goreportcard.com/report/github.com/kubeflow/katib)
+Kubeflow Katib is a Kubernetes-native project for automated machine learning (AutoML).
+Katib supports
+[Hyperparameter Tuning](https://en.wikipedia.org/wiki/Hyperparameter_optimization),
+[Early Stopping](https://en.wikipedia.org/wiki/Early_stopping) and
+[Neural Architecture Search](https://en.wikipedia.org/wiki/Neural_architecture_search).
 
-Katib is a Kubernetes-based system for [Hyperparameter Tuning][1] and [Neural Architecture Search][2]. Katib supports a number of ML frameworks, including TensorFlow, Apache MXNet, PyTorch, XGBoost, and others.
+Katib is the project which is agnostic to machine learning (ML) frameworks.
+It can tune hyperparameters of applications written in any language of the
+users’ choice and natively supports many ML frameworks, such as
+[TensorFlow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), [XGBoost](https://xgboost.readthedocs.io/en/latest/), and others.
 
-Table of Contents
-=================
-
-  * [Getting Started](#getting-started)
-  * [Name](#name)
-  * [Concepts in Katib](#concepts-in-katib)
-      * [Experiment](#experiment)
-      * [Suggestion](#suggestion)
-      * [Trial](#trial)
-      * [Worker Job](#worker-job)
-  * [Components in Katib](#components-in-katib)
-  * [Web UI](#web-ui)
-  * [API documentation](#api-documentation)
-  * [Installation](#installation)
-      * [TF operator](#tf-operator)
-      * [PyTorch operator](#pytorch-operator)
-      * [Katib](#katib)
-      * [Running examples](#running-examples)
-      * [Cleanups](#cleanups)
-  * [Katib SDK](#katib-sdk)      
-  * [Quick Start](#quick-start)
-  * [Who are using Katib?](#who-are-using-katib)
-  * [Citation](#citation)
-  * [CONTRIBUTING](#contributing)
-
-Created by [gh-md-toc](https://github.com/ekalinin/github-markdown-toc)
-
-## Getting Started
-
-See the [getting-started 
-guide](https://www.kubeflow.org/docs/components/hyperparameter-tuning/hyperparameter/)
-on the Kubeflow website.
-
-## Name
+Katib can perform training jobs using any Kubernetes
+[Custom Resources](https://www.kubeflow.org/docs/components/katib/trial-template/)
+with out of the box support for [Kubeflow Training Operator](https://github.com/kubeflow/training-operator),
+[Argo Workflows](https://github.com/argoproj/argo-workflows), [Tekton Pipelines](https://github.com/tektoncd/pipeline)
+and many more.
 
 Katib stands for `secretary` in Arabic.
 
-## Concepts in Katib
+## Search Algorithms
 
-For a detailed description of the concepts in Katib, hyperparameter tuning, and
-neural architecture search, see the [Kubeflow 
-documentation](https://www.kubeflow.org/docs/components/hyperparameter-tuning/overview/).
+Katib supports several search algorithms. Follow the
+[Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/user-guides/hp-tuning/configure-algorithm/#hp-tuning-algorithms)
+to know more about each algorithm and check the
+[this guide](https://www.kubeflow.org/docs/components/katib/user-guides/hp-tuning/configure-algorithm/#use-custom-algorithm-in-katib)
+to implement your custom algorithm.
 
-Katib has the concepts of Experiment, Trial, Job and Suggestion.
+<table>
+  <tbody>
+    <tr align="center">
+      <td>
+        <b>Hyperparameter Tuning</b>
+      </td>
+      <td>
+        <b>Neural Architecture Search</b>
+      </td>
+      <td>
+        <b>Early Stopping</b>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#random-search">Random Search</a>
+      </td>
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#neural-architecture-search-based-on-enas">ENAS</a>
+      </td>
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/early-stopping/#median-stopping-rule">Median Stop</a>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#grid-search">Grid Search</a>
+      </td>
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#differentiable-architecture-search-darts">DARTS</a>
+      </td>
+      <td>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#bayesian-optimization">Bayesian Optimization</a>
+      </td>
+      <td>
+      </td>
+      <td>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#tree-of-parzen-estimators-tpe">TPE</a>
+      </td>
+      <td>
+      </td>
+      <td>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#multivariate-tpe">Multivariate TPE</a>
+      </td>
+      <td>
+      </td>
+      <td>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#covariance-matrix-adaptation-evolution-strategy-cma-es">CMA-ES</a>
+      </td>
+      <td>
+      </td>
+      <td>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#sobols-quasirandom-sequence">Sobol's Quasirandom Sequence</a>
+      </td>
+      <td>
+      </td>
+      <td>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#hyperband">HyperBand</a>
+      </td>
+      <td>
+      </td>
+      <td>
+      </td>
+    </tr>
+    <tr align="center">
+      <td>
+        <a href="https://www.kubeflow.org/docs/components/katib/experiment/#pbt">Population Based Training</a>
+      </td>
+      <td>
+      </td>
+      <td>
+      </td>
+    </tr>
+  </tbody>
+</table>
 
-### Experiment
+To perform the above algorithms Katib supports the following frameworks:
 
-`Experiment` represents a single optimization run over a feasible space.
-Each `Experiment` contains a configuration:
+- [Goptuna](https://github.com/c-bata/goptuna)
+- [Hyperopt](https://github.com/hyperopt/hyperopt)
+- [Optuna](https://github.com/optuna/optuna)
+- [Scikit Optimize](https://github.com/scikit-optimize/scikit-optimize)
 
-1. Objective: What we are trying to optimize.
-2. Search Space: Constraints for configurations describing the feasible space.
-3. Search Algorithm: How to find the optimal configurations.
+## Prerequisites
 
-`Experiment` is defined as a CRD. See the detailed guide to [configuring and running a Katib 
-experiment](https://kubeflow.org/docs/components/hyperparameter-tuning/experiment/)
-in the Kubeflow docs.
-
-### Suggestion
-
-A Suggestion is a proposed solution to the optimization problem which is one set of hyperparameter values or a list of parameter assignments. Then a `Trial` will be created to evaluate the parameter assignments.
-
-`Suggestion` is defined as a CRD.
-
-### Trial
-
-A `Trial` is one iteration of the optimization process, which is one `worker job` instance with a list of parameter assignments(corresponding to a suggestion).
-
-`Trial` is defined as a CRD.
-
-### Worker Job 
-
-A `Worker Job` refers to a process responsible for evaluating a `Trial` and calculating its objective value. 
-
-The worker kind can be [Kubernetes Job](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/) which is a non distributed execution, [Kubeflow TFJob](https://www.kubeflow.org/docs/guides/components/tftraining/) or [Kubeflow PyTorchJob](https://www.kubeflow.org/docs/guides/components/pytorch/) which are distributed executions.
-Thus, Katib supports multiple frameworks with the help of different job kinds. 
-
-Currently Katib supports the following exploration algorithms:
-
-#### Hyperparameter Tuning
-
-* [Random Search](https://en.wikipedia.org/wiki/Hyperparameter_optimization#Random_search)
-* [Tree of Parzen Estimators (TPE)](https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf)
-* [Grid Search](https://en.wikipedia.org/wiki/Hyperparameter_optimization#Grid_search)
-* [Hyperband](https://arxiv.org/pdf/1603.06560.pdf)
-* [Bayesian Optimization](https://arxiv.org/pdf/1012.2599.pdf)
-* [CMA Evolution Strategy](https://arxiv.org/abs/1604.00772)
-
-#### Neural Architecture Search
-
-* [Efficient Neural Architecture Search (ENAS)](https://github.com/kubeflow/katib/tree/master/pkg/suggestion/v1beta1/nas/enas)
-* [Differentiable Architecture Search (DARTS)](https://github.com/kubeflow/katib/tree/master/pkg/suggestion/v1beta1/nas/darts)
-
-
-## Components in Katib
-
-Katib consists of several components as shown below. Each component is running on k8s as a deployment.
-Each component communicates with others via GRPC and the API is defined at `pkg/apis/manager/v1beta1/api.proto`
-for v1beta1 version and `pkg/apis/manager/v1alpha3/api.proto` for v1alpha3 version.
-
-- Katib main components:
-  - katib-db-manager: GRPC API server of Katib which is the DB Interface.
-  - katib-mysql: Data storage backend of Katib using mysql.
-  - katib-ui: User interface of Katib.
-  - katib-controller: Controller for Katib CRDs in Kubernetes.
-
-## Web UI
-
-Katib provides a Web UI.
-You can visualize general trend of Hyper parameter space and each training history. You can use
-[random-example](https://github.com/kubeflow/katib/blob/master/examples/v1beta1/random-example.yaml) or
-[other examples](https://github.com/kubeflow/katib/blob/master/examples/v1beta1) to generate a similar UI.
-![katibui](./docs/images/katib-ui.png)
-
-## GRPC API documentation
-
-See the [Katib v1beta1 API reference docs](https://github.com/kubeflow/katib/blob/master/pkg/apis/manager/v1beta1/gen-doc/api.md).
-
-See the [Katib v1alpha3 API reference docs](https://www.kubeflow.org/docs/reference/katib/).
+Please check [the official Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/installation/#prerequisites)
+for prerequisites to install Katib.
 
 ## Installation
 
-For standard installation of Katib with support for all job operators, 
-install Kubeflow. Current official Katib version in Kubeflow latest release is v1alpha3.
-See the documentation:
+Please follow [the Kubeflow Katib guide](https://www.kubeflow.org/docs/components/katib/installation/#installing-katib)
+for the detailed instructions on how to install Katib.
 
-* [Kubeflow installation 
-guide](https://www.kubeflow.org/docs/started/getting-started/)
-* [Kubeflow hyperparameter tuning 
-guides](https://www.kubeflow.org/docs/components/hyperparameter-tuning/).
+### Installing the Control Plane
 
-If you install Katib with other Kubeflow components, you can't submit Katib jobs in Kubeflow namespace.
-
-Alternatively, if you want to install Katib manually with TF and PyTorch operators support, follow these steps:
-
-Create Kubeflow namespace:
+Run the following command to install the latest stable release of Katib control plane:
 
 ```
-kubectl create namespace kubeflow
+kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=v0.17.0"
 ```
 
-Clone Kubeflow manifest repository:
+Run the following command to install the latest changes of Katib control plane:
 
 ```
-git clone git@github.com:kubeflow/manifests.git
-Set `MANIFESTS_DIR` to the cloned folder.
-export MANIFESTS_DIR=<cloned-folder>
+kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=master"
 ```
 
-### TF operator
+For the Katib Experiments check the [complete examples list](./examples/v1beta1).
 
-For installing TF operator, run the following:
+### Installing the Python SDK
 
-```
-cd "${MANIFESTS_DIR}/tf-training/tf-job-crds/base"
-kustomize build . | kubectl apply -f -
-cd "${MANIFESTS_DIR}/tf-training/tf-job-operator/base"
-kustomize build . | kubectl apply -n kubeflow -f -
+Katib implements [a Python SDK](https://pypi.org/project/kubeflow-katib/) to simplify creation of
+hyperparameter tuning jobs for Data Scientists.
+
+Run the following command to install the latest stable release of Katib SDK:
+
+```sh
+pip install -U kubeflow-katib
 ```
 
-### PyTorch operator
+## Getting Started
 
-For installing PyTorch operator, run the following:
+Please refer to [the getting started guide](https://www.kubeflow.org/docs/components/katib/getting-started/#getting-started-with-katib-python-sdk)
+to quickly create your first hyperparameter tuning Experiment using the Python SDK.
 
-```
-cd "${MANIFESTS_DIR}/pytorch-job/pytorch-job-crds/base"
-kustomize build . | kubectl apply -f -
-cd "${MANIFESTS_DIR}/pytorch-job/pytorch-operator/base/"
-kustomize build . | kubectl apply -n kubeflow -f -
-```
+## Community
 
-### Katib
+The following links provide information on how to get involved in the community:
 
-Finally, you can install Katib.
+- Attend [the bi-weekly AutoML and Training Working Group](https://bit.ly/2PWVCkV)
+  community meeting.
+- Join our [`#kubeflow-katib`](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels)
+  Slack channel.
+- Check out [who is using Katib](ADOPTERS.md) and [presentations about Katib project](docs/presentations.md).
 
-For v1beta1 version, run the following:
+## Contributing
 
-```
-git clone git@github.com:kubeflow/katib.git
-bash katib/scripts/v1beta1/deploy.sh
-```
-
-For v1alpha3 version, run the following:
-
-```
-cd "${MANIFESTS_DIR}/katib/katib-crds/base"
-kustomize build . | kubectl apply -f -
-cd "${MANIFESTS_DIR}/katib/katib-controller/base"
-kustomize build . | kubectl apply -f -
-
-```
-
-If you install Katib from Kubeflow manifest repository and you want to use Katib in a cluster that doesn't have a StorageClass for dynamic volume provisioning, you have to create persistent volume manually to bound your persistent volume claim.
-
-This is sample yaml file for creating a persistent volume with local storage:
-
-```yaml
-apiVersion: v1
-kind: PersistentVolume
-metadata:
-  name: katib-mysql
-  labels:
-    type: local
-    app: katib
-spec:
-  storageClassName: katib
-  capacity:
-    storage: 10Gi
-  accessModes:
-    - ReadWriteOnce
-  hostPath:
-    path: /tmp/katib
-```
-
-Create this PV after deploying Katib package
-
-Check if all components are running successfully:
-
-```
-kubectl get pods -n kubeflow
-```
-
-Expected output:
-
-```
-NAME                                READY   STATUS    RESTARTS   AGE
-katib-controller-858d6cc48c-df9jc   1/1     Running   1          20m
-katib-db-manager-7966fbdf9b-w2tn8   1/1     Running   0          20m
-katib-mysql-7f8bc6956f-898f9        1/1     Running   0          20m
-katib-ui-7cf9f967bf-nm72p           1/1     Running   0          20m
-pytorch-operator-55f966b548-9gq9v   1/1     Running   0          20m
-tf-job-operator-796b4747d8-4fh82    1/1     Running   0          21m
-```
-
-### Running examples
-
-After deploy everything, you can run examples to verify the installation.
-Examples bellow are for v1beta1 version.
-
-This is an example for TF operator:
-
-```
-kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/v1beta1/tfjob-example.yaml
-```
-
-This is an example for PyTorch operator:
-
-```
-kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/v1beta1/pytorchjob-example.yaml
-```
-
-You can check status of experiment 
-
-```yaml
-$ kubectl describe experiment tfjob-example -n kubeflow
-
-Name:         tfjob-example
-Namespace:    kubeflow
-Labels:       <none>
-Annotations:  <none>
-API Version:  kubeflow.org/v1beta1
-Kind:         Experiment
-Metadata:
-  Creation Timestamp:  2020-07-15T14:27:53Z
-  Finalizers:
-    update-prometheus-metrics
-  Generation:        1
-  Resource Version:  100380029
-  Self Link:         /apis/kubeflow.org/v1beta1/namespaces/kubeflow/experiments/tfjob-example
-  UID:               5e3cf1f5-c6a7-11ea-90dd-42010a9a0020
-Spec:
-  Algorithm:
-    Algorithm Name:        random
-  Max Failed Trial Count:  3
-  Max Trial Count:         12
-  Metrics Collector Spec:
-    Collector:
-      Kind:  TensorFlowEvent
-    Source:
-      File System Path:
-        Kind:  Directory
-        Path:  /train
-  Objective:
-    Goal:  0.99
-    Metric Strategies:
-      Name:                 accuracy_1
-      Value:                max
-    Objective Metric Name:  accuracy_1
-    Type:                   maximize
-  Parallel Trial Count:     3
-  Parameters:
-    Feasible Space:
-      Max:           0.05
-      Min:           0.01
-    Name:            learning_rate
-    Parameter Type:  double
-    Feasible Space:
-      Max:           200
-      Min:           100
-    Name:            batch_size
-    Parameter Type:  int
-  Resume Policy:     LongRunning
-  Trial Template:
-    Trial Parameters:
-      Description:  Learning rate for the training model
-      Name:         learningRate
-      Reference:    learning_rate
-      Description:  Batch Size
-      Name:         batchSize
-      Reference:    batch_size
-    Trial Spec:
-      API Version:  kubeflow.org/v1
-      Kind:         TFJob
-      Spec:
-        Tf Replica Specs:
-          Worker:
-            Replicas:        2
-            Restart Policy:  OnFailure
-            Template:
-              Spec:
-                Containers:
-                  Command:
-                    python
-                    /var/tf_mnist/mnist_with_summaries.py
-                    --log_dir=/train/metrics
-                    --learning_rate=${trialParameters.learningRate}
-                    --batch_size=${trialParameters.batchSize}
-                  Image:              gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0
-                  Image Pull Policy:  Always
-                  Name:               tensorflow
-Status:
-  Completion Time:  2020-07-15T14:30:52Z
-  Conditions:
-    Last Transition Time:  2020-07-15T14:27:53Z
-    Last Update Time:      2020-07-15T14:27:53Z
-    Message:               Experiment is created
-    Reason:                ExperimentCreated
-    Status:                True
-    Type:                  Created
-    Last Transition Time:  2020-07-15T14:30:52Z
-    Last Update Time:      2020-07-15T14:30:52Z
-    Message:               Experiment is running
-    Reason:                ExperimentRunning
-    Status:                False
-    Type:                  Running
-    Last Transition Time:  2020-07-15T14:30:52Z
-    Last Update Time:      2020-07-15T14:30:52Z
-    Message:               Experiment has succeeded because Objective goal has reached
-    Reason:                ExperimentGoalReached
-    Status:                True
-    Type:                  Succeeded
-  Current Optimal Trial:
-    Best Trial Name:  tfjob-example-gjxn54vl
-    Observation:
-      Metrics:
-        Latest:  0.966300010681
-        Max:     1.0
-        Min:     0.103260867298
-        Name:    accuracy_1
-    Parameter Assignments:
-      Name:    learning_rate
-      Value:   0.015945204040626416
-      Name:    batch_size
-      Value:   184
-  Start Time:  2020-07-15T14:27:53Z
-  Succeeded Trial List:
-    tfjob-example-5jd8nnjg
-    tfjob-example-bgjfpd5t
-    tfjob-example-gjxn54vl
-    tfjob-example-vpdqxkch
-    tfjob-example-wvptx7gt
-  Trials:            5
-  Trials Succeeded:  5
-Events:              <none>
-```
-
-When the spec.Status.Condition becomes ```Succeeded```, the experiment is finished.
-
-You can monitor your results in Katib UI. 
-Access Katib UI via Kubeflow dashboard if you have used standard installation or port-forward the `katib-ui` service if you have installed manually.
-
-```
-kubectl -n kubeflow port-forward svc/katib-ui 8080:80
-```
-
-You can access the Katib UI using this URL: ```http://localhost:8080/katib/```.
-
-### Katib SDK
-
-Katib supports Python SDK for v1beta1 and v1alpha3 version.
-
-* See the [Katib v1beta1 SDK documentation](https://github.com/kubeflow/katib/tree/master/sdk/python/v1beta1).
-
-* See the [Katib v1alpha3 SDK documentation](https://github.com/kubeflow/katib/tree/master/sdk/python/v1alpha3).
-
-Run [`gen-sdk.sh`](https://github.com/kubeflow/katib/blob/master/hack/gen-python-sdk/gen-sdk.sh) to update SDK.
-
-### Cleanups
-
-To delete installed TF and PyTorch operator run `kubectl delete -f` on the respective folders. 
-
-To delete Katib for v1beta1 version run `bash katib/scripts/v1beta1/undeploy.sh`.
-
-## Quick Start
-
-Please see [Quick Start Guide](./docs/quick-start.md).
-
-## Who are using Katib?
-
-Please see [adopters.md](./docs/community/adopters.md).
-
-## CONTRIBUTING
-
-Please feel free to test the system! [developer-guide.md](./docs/developer-guide.md) is a good starting point for developers.
-
-[1]: https://en.wikipedia.org/wiki/Hyperparameter_optimization
-[2]: https://en.wikipedia.org/wiki/Neural_architecture_search
-[3]: https://static.googleusercontent.com/media/research.google.com/ja//pubs/archive/bcb15507f4b52991a0783013df4222240e942381.pdf
+Please refer to the [CONTRIBUTING guide](CONTRIBUTING.md).
 
 ## Citation
 
 If you use Katib in a scientific publication, we would appreciate
 citations to the following paper:
 
-[A Scalable and Cloud-Native Hyperparameter Tuning System](https://arxiv.org/abs/2006.02085), George *et al.*, arXiv:2006.02085, 2020.
+[A Scalable and Cloud-Native Hyperparameter Tuning System](https://arxiv.org/abs/2006.02085), George _et al._, arXiv:2006.02085, 2020.
 
 Bibtex entry:
 

ROADMAP.md

@@ -1,3 +1,71 @@
+# Katib 2022/2023 Roadmap
+
+## AutoML Features
+
+- Support advance HyperParameter tuning algorithms:
+
+  - Population Based Training (PBT) - [#1382](https://github.com/kubeflow/katib/issues/1382)
+  - Tree of Parzen Estimators (TPE)
+  - Multivariate TPE
+  - Sobol’s Quasirandom Sequence
+  - Asynchronous Successive Halving - [ASHA](https://arxiv.org/pdf/1810.05934.pdf)
+
+- Support multi-objective optimization - [#1549](https://github.com/kubeflow/katib/issues/1549)
+- Support various HP distributions (log-uniform, uniform, normal) - [#1207](https://github.com/kubeflow/katib/issues/1207)
+- Support Auto Model Compression - [#460](https://github.com/kubeflow/katib/issues/460)
+- Support Auto Feature Engineering - [#475](https://github.com/kubeflow/katib/issues/475)
+- Improve Neural Architecture Search design
+
+## Backend and API Enhancements
+
+- Conformance tests for Katib - [#2044](https://github.com/kubeflow/katib/issues/2044)
+- Support push-based metrics collection in Katib - [#577](https://github.com/kubeflow/katib/issues/577)
+- Support PostgreSQL as a Katib DB - [#915](https://github.com/kubeflow/katib/issues/915)
+- Improve Katib scalability - [#1847](https://github.com/kubeflow/katib/issues/1847)
+- Promote Katib APIs to the `v1` version
+- Support multiple CRD versions (`v1beta1`, `v1`) with conversion webhook
+
+## Improve Katib User Experience
+
+- Simplify Katib Experiment creation with Katib SDK - [#1951](https://github.com/kubeflow/katib/pull/1951)
+- Fully migrate to a new Katib UI - [Project 1](https://github.com/kubeflow/katib/projects/1)
+- Expose Trial logs in Katib UI - [#971](https://github.com/kubeflow/katib/issues/971)
+- Enhance Katib UI visualization metrics for AutoML Experiments
+- Improve Katib Config UX - [#2150](https://github.com/kubeflow/katib/issues/2150)
+
+## Integration with Kubeflow Components
+
+- Kubeflow Pipeline as a Katib Trial target - [#1914](https://github.com/kubeflow/katib/issues/1914)
+- Improve data passing when Katib Experiment is part of Kubeflow Pipeline - [#1846](https://github.com/kubeflow/katib/issues/1846)
+
+# History
+
+# Katib 2021 Roadmap
+
+## New Features
+
+### AutoML
+
+- Support Population Based Training [#1382](https://github.com/kubeflow/katib/issues/1382)
+- Support [ASHA](https://arxiv.org/pdf/1810.05934.pdf)
+- Support Auto Model Compression [#460](https://github.com/kubeflow/katib/issues/460)
+- Support Auto Feature Engineering [#475](https://github.com/kubeflow/katib/issues/475)
+- Various CRDs for HP, NAS and other AutoML techniques.
+
+### UI
+
+- Migrate to the new Katib UI [Project 1](https://github.com/kubeflow/katib/projects/1)
+- Hyperparameter importances visualization with fANOVA algorithm
+
+## Enhancements
+
+- Finish AWS CI/CD migration
+- Support various parameter distribution [#1207](https://github.com/kubeflow/katib/issues/1207)
+- Finish validation for Algorithms [#1126](https://github.com/kubeflow/katib/issues/1126)
+- Refactor Hyperband [#1389](https://github.com/kubeflow/katib/issues/1389)
+- Support multiple CRD version with conversion webhook
+- MLMD integration with Katib Experiments
+
 # Katib 2020 Roadmap
 
 ## New Features

SECURITY.md

@@ -0,0 +1,64 @@
+# Security Policy
+
+## Supported Versions
+
+Kubeflow Katib versions are expressed as `vX.Y.Z`, where X is the major version,
+Y is the minor version, and Z is the patch version, following the
+[Semantic Versioning](https://semver.org/) terminology.
+
+The Kubeflow Katib project maintains release branches for the most recent two minor releases.
+Applicable fixes, including security fixes, may be backported to those two release branches,
+depending on severity and feasibility.
+
+Users are encouraged to stay updated with the latest releases to benefit from security patches and
+improvements.
+
+## Reporting a Vulnerability
+
+We're extremely grateful for security researchers and users that report vulnerabilities to the
+Kubeflow Open Source Community. All reports are thoroughly investigated by Kubeflow projects owners.
+
+You can use the following ways to report security vulnerabilities privately:
+
+- Using the Kubeflow Katib repository [GitHub Security Advisory](https://github.com/kubeflow/katib/security/advisories/new).
+- Using our private Kubeflow Steering Committee mailing list: ksc@kubeflow.org.
+
+Please provide detailed information to help us understand and address the issue promptly.
+
+## Disclosure Process
+
+**Acknowledgment**: We will acknowledge receipt of your report within 10 business days.
+
+**Assessment**: The Kubeflow projects owners will investigate the reported issue to determine its
+validity and severity.
+
+**Resolution**: If the issue is confirmed, we will work on a fix and prepare a release.
+
+**Notification**: Once a fix is available, we will notify the reporter and coordinate a public
+disclosure.
+
+**Public Disclosure**: Details of the vulnerability and the fix will be published in the project's
+release notes and communicated through appropriate channels.
+
+## Prevention Mechanisms
+
+Kubeflow Katib employs several measures to prevent security issues:
+
+**Code Reviews**: All code changes are reviewed by maintainers to ensure code quality and security.
+
+**Dependency Management**: Regular updates and monitoring of dependencies (e.g. Dependabot) to
+address known vulnerabilities.
+
+**Continuous Integration**: Automated testing and security checks are integrated into the CI/CD pipeline.
+
+**Image Scanning**: Container images are scanned for vulnerabilities.
+
+## Communication Channels
+
+For the general questions please join the following resources:
+
+- Kubeflow [Slack channels](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels).
+
+- Kubeflow discuss [mailing list](https://www.kubeflow.org/docs/about/community/#kubeflow-mailing-list).
+
+Please **do not report** security vulnerabilities through public channels.

cmd/db-manager/v1alpha3/Dockerfile

@@ -1,26 +0,0 @@
-FROM golang:alpine AS build-env
-# The GOPATH in the image is /go.
-ADD . /go/src/github.com/kubeflow/katib
-WORKDIR /go/src/github.com/kubeflow/katib/cmd/db-manager
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-        apk --update add git gcc musl-dev && \
-        go build -o katib-db-manager ./v1alpha3; \
-    else \
-        go build -o katib-db-manager ./v1alpha3; \
-    fi
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
-FROM alpine:3.7
-WORKDIR /app
-COPY --from=build-env /bin/grpc_health_probe /bin/
-COPY --from=build-env /go/src/github.com/kubeflow/katib/cmd/db-manager/katib-db-manager /app/
-ENTRYPOINT ["./katib-db-manager"]
-CMD ["-w", "kubernetes"]

cmd/db-manager/v1alpha3/main.go

@@ -1,100 +0,0 @@
-package main
-
-import (
-	"context"
-	"flag"
-	"fmt"
-	"net"
-	"os"
-
-	health_pb "github.com/kubeflow/katib/pkg/apis/manager/health"
-	api_pb "github.com/kubeflow/katib/pkg/apis/manager/v1alpha3"
-	db "github.com/kubeflow/katib/pkg/db/v1alpha3"
-	"github.com/kubeflow/katib/pkg/db/v1alpha3/common"
-	"k8s.io/klog"
-
-	"google.golang.org/grpc"
-	"google.golang.org/grpc/reflection"
-)
-
-const (
-	port = "0.0.0.0:6789"
-)
-
-var dbIf common.KatibDBInterface
-
-type server struct {
-}
-
-// Report a log of Observations for a Trial.
-// The log consists of timestamp and value of metric.
-// Katib store every log of metrics.
-// You can see accuracy curve or other metric logs on UI.
-func (s *server) ReportObservationLog(ctx context.Context, in *api_pb.ReportObservationLogRequest) (*api_pb.ReportObservationLogReply, error) {
-	err := dbIf.RegisterObservationLog(in.TrialName, in.ObservationLog)
-	return &api_pb.ReportObservationLogReply{}, err
-}
-
-// Get all log of Observations for a Trial.
-func (s *server) GetObservationLog(ctx context.Context, in *api_pb.GetObservationLogRequest) (*api_pb.GetObservationLogReply, error) {
-	ol, err := dbIf.GetObservationLog(in.TrialName, in.MetricName, in.StartTime, in.EndTime)
-	return &api_pb.GetObservationLogReply{
-		ObservationLog: ol,
-	}, err
-}
-
-// Delete all log of Observations for a Trial.
-func (s *server) DeleteObservationLog(ctx context.Context, in *api_pb.DeleteObservationLogRequest) (*api_pb.DeleteObservationLogReply, error) {
-	err := dbIf.DeleteObservationLog(in.TrialName)
-	return &api_pb.DeleteObservationLogReply{}, err
-}
-
-func (s *server) Check(ctx context.Context, in *health_pb.HealthCheckRequest) (*health_pb.HealthCheckResponse, error) {
-	resp := health_pb.HealthCheckResponse{
-		Status: health_pb.HealthCheckResponse_SERVING,
-	}
-
-	// We only accept optional service name only if it's set to suggested format.
-	if in != nil && in.Service != "" && in.Service != "grpc.health.v1.Health" {
-		resp.Status = health_pb.HealthCheckResponse_UNKNOWN
-		return &resp, fmt.Errorf("grpc.health.v1.Health can only be accepted if you specify service name.")
-	}
-
-	// Check if connection to katib db driver is okay since otherwise manager could not serve most of its methods.
-	err := dbIf.SelectOne()
-	if err != nil {
-		resp.Status = health_pb.HealthCheckResponse_NOT_SERVING
-		return &resp, fmt.Errorf("Failed to execute `SELECT 1` probe: %v", err)
-	}
-
-	return &resp, nil
-}
-
-func main() {
-	flag.Parse()
-	var err error
-	dbNameEnvName := common.DBNameEnvName
-	dbName := os.Getenv(dbNameEnvName)
-	if dbName == "" {
-		klog.Fatal("DB_NAME env is not set. Exiting")
-	}
-	dbIf, err = db.NewKatibDBInterface(dbName)
-	if err != nil {
-		klog.Fatalf("Failed to open db connection: %v", err)
-	}
-	dbIf.DBInit()
-	listener, err := net.Listen("tcp", port)
-	if err != nil {
-		klog.Fatalf("Failed to listen: %v", err)
-	}
-
-	size := 1<<31 - 1
-	klog.Infof("Start Katib manager: %s", port)
-	s := grpc.NewServer(grpc.MaxRecvMsgSize(size), grpc.MaxSendMsgSize(size))
-	api_pb.RegisterManagerServer(s, &server{})
-	health_pb.RegisterHealthServer(s, &server{})
-	reflection.Register(s)
-	if err = s.Serve(listener); err != nil {
-		klog.Fatalf("Failed to serve: %v", err)
-	}
-}

cmd/db-manager/v1alpha3/main_test.go

@@ -1,171 +0,0 @@
-package main
-
-import (
-	"context"
-	"testing"
-
-	"github.com/golang/mock/gomock"
-
-	health_pb "github.com/kubeflow/katib/pkg/apis/manager/health"
-	api_pb "github.com/kubeflow/katib/pkg/apis/manager/v1alpha3"
-	mockdb "github.com/kubeflow/katib/pkg/mock/v1alpha3/db"
-)
-
-func TestReportObservationLog(t *testing.T) {
-	ctrl := gomock.NewController(t)
-	defer ctrl.Finish()
-	s := &server{}
-	mockDB := mockdb.NewMockKatibDBInterface(ctrl)
-	dbIf = mockDB
-
-	req := &api_pb.ReportObservationLogRequest{
-		TrialName: "test1-trial1",
-		ObservationLog: &api_pb.ObservationLog{
-			MetricLogs: []*api_pb.MetricLog{
-				{
-					TimeStamp: "2019-02-03T04:05:06+09:00",
-					Metric: &api_pb.Metric{
-						Name:  "f1_score",
-						Value: "88.95",
-					},
-				},
-				{
-					TimeStamp: "2019-02-03T04:05:06+09:00",
-					Metric: &api_pb.Metric{
-						Name:  "loss",
-						Value: "0.5",
-					},
-				},
-				{
-					TimeStamp: "2019-02-03T04:05:06+09:00",
-					Metric: &api_pb.Metric{
-						Name:  "precision",
-						Value: "88.7",
-					},
-				},
-				{
-					TimeStamp: "2019-02-03T04:05:06+09:00",
-					Metric: &api_pb.Metric{
-						Name:  "recall",
-						Value: "89.2",
-					},
-				},
-			},
-		},
-	}
-	mockDB.EXPECT().RegisterObservationLog(req.TrialName, req.ObservationLog).Return(nil)
-	_, err := s.ReportObservationLog(context.Background(), req)
-	if err != nil {
-		t.Fatalf("ReportObservationLog Error %v", err)
-	}
-}
-
-func TestGetObservationLog(t *testing.T) {
-	ctrl := gomock.NewController(t)
-	defer ctrl.Finish()
-	s := &server{}
-	mockDB := mockdb.NewMockKatibDBInterface(ctrl)
-	dbIf = mockDB
-
-	req := &api_pb.GetObservationLogRequest{
-		TrialName: "test1-trial1",
-		StartTime: "2019-02-03T03:05:06+09:00",
-		EndTime:   "2019-02-03T05:05:06+09:00",
-	}
-
-	obs := &api_pb.ObservationLog{
-		MetricLogs: []*api_pb.MetricLog{
-			{
-				TimeStamp: "2019-02-03T04:05:06+09:00",
-				Metric: &api_pb.Metric{
-					Name:  "f1_score",
-					Value: "88.95",
-				},
-			},
-			{
-				TimeStamp: "2019-02-03T04:05:06+09:00",
-				Metric: &api_pb.Metric{
-					Name:  "loss",
-					Value: "0.5",
-				},
-			},
-			{
-				TimeStamp: "2019-02-03T04:05:06+09:00",
-				Metric: &api_pb.Metric{
-					Name:  "precision",
-					Value: "88.7",
-				},
-			},
-			{
-				TimeStamp: "2019-02-03T04:05:06+09:00",
-				Metric: &api_pb.Metric{
-					Name:  "recall",
-					Value: "89.2",
-				},
-			},
-		},
-	}
-
-	mockDB.EXPECT().GetObservationLog(req.TrialName, req.MetricName, req.StartTime, req.EndTime).Return(obs, nil)
-	ret, err := s.GetObservationLog(context.Background(), req)
-	if err != nil {
-		t.Fatalf("GetObservationLog Error %v", err)
-	}
-	if len(obs.MetricLogs) != len(ret.ObservationLog.MetricLogs) {
-		t.Fatalf("GetObservationLog Test fail expect metrics number %d got %d", len(obs.MetricLogs), len(ret.ObservationLog.MetricLogs))
-	}
-}
-
-func TestDeleteObservationLog(t *testing.T) {
-	ctrl := gomock.NewController(t)
-	defer ctrl.Finish()
-	s := &server{}
-	mockDB := mockdb.NewMockKatibDBInterface(ctrl)
-	dbIf = mockDB
-
-	req := &api_pb.DeleteObservationLogRequest{
-		TrialName: "test1-trial1",
-	}
-	mockDB.EXPECT().DeleteObservationLog(req.TrialName).Return(nil)
-	_, err := s.DeleteObservationLog(context.Background(), req)
-	if err != nil {
-		t.Fatalf("DeleteExperiment Error %v", err)
-	}
-}
-
-func TestCheck(t *testing.T) {
-	ctrl := gomock.NewController(t)
-	defer ctrl.Finish()
-	s := &server{}
-	mockDB := mockdb.NewMockKatibDBInterface(ctrl)
-	dbIf = mockDB
-	testCases := []struct {
-		Request        *health_pb.HealthCheckRequest
-		ExpectedStatus health_pb.HealthCheckResponse_ServingStatus
-		Name           string
-	}{
-		{
-			Request: &health_pb.HealthCheckRequest{
-				Service: "grpc.health.v1.Health",
-			},
-			ExpectedStatus: health_pb.HealthCheckResponse_SERVING,
-			Name:           "Valid Request",
-		},
-		{
-			Request: &health_pb.HealthCheckRequest{
-				Service: "grpc.health.v1.1.Health",
-			},
-			ExpectedStatus: health_pb.HealthCheckResponse_UNKNOWN,
-			Name:           "Invalid service name",
-		},
-	}
-
-	mockDB.EXPECT().SelectOne().Return(nil)
-
-	for _, tc := range testCases {
-		response, _ := s.Check(context.Background(), tc.Request)
-		if response.Status != tc.ExpectedStatus {
-			t.Fatalf("Case %v failed. ExpectedStatus %v, got %v", tc.Name, tc.ExpectedStatus, response.Status)
-		}
-	}
-}

cmd/db-manager/v1beta1/Dockerfile

@@ -1,26 +1,24 @@
+# Build the Katib DB manager.
 FROM golang:alpine AS build-env
-# The GOPATH in the image is /go.
-ADD . /go/src/github.com/kubeflow/katib
-WORKDIR /go/src/github.com/kubeflow/katib/cmd/db-manager
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-    apk --update add git gcc musl-dev && \
-    go build -o katib-db-manager ./v1beta1; \
-    else \
-    go build -o katib-db-manager ./v1beta1; \
-    fi
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
 
-FROM alpine:3.7
+ARG TARGETARCH
+
+WORKDIR /go/src/github.com/kubeflow/katib
+
+# Download packages.
+COPY go.mod .
+COPY go.sum .
+RUN go mod download -x
+
+# Copy sources.
+COPY cmd/ cmd/
+COPY pkg/ pkg/
+
+# Build the binary.
+RUN CGO_ENABLED=0 GOOS=linux GOARCH="${TARGETARCH}" go build -a -o katib-db-manager ./cmd/db-manager/v1beta1
+
+# Copy the db-manager into a thin image.
+FROM alpine:3.15
 WORKDIR /app
-COPY --from=build-env /bin/grpc_health_probe /bin/
-COPY --from=build-env /go/src/github.com/kubeflow/katib/cmd/db-manager/katib-db-manager /app/
+COPY --from=build-env /go/src/github.com/kubeflow/katib/katib-db-manager /app/
 ENTRYPOINT ["./katib-db-manager"]
-CMD ["-w", "kubernetes"]

cmd/db-manager/v1beta1/main.go

@@ -1,3 +1,19 @@
+/*
+Copyright 2022 The Kubeflow Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
 package main
 
 import (
@@ -6,19 +22,21 @@ import (
 	"fmt"
 	"net"
 	"os"
+	"time"
 
 	health_pb "github.com/kubeflow/katib/pkg/apis/manager/health"
 	api_pb "github.com/kubeflow/katib/pkg/apis/manager/v1beta1"
 	db "github.com/kubeflow/katib/pkg/db/v1beta1"
 	"github.com/kubeflow/katib/pkg/db/v1beta1/common"
-	"k8s.io/klog"
+	"k8s.io/klog/v2"
 
 	"google.golang.org/grpc"
 	"google.golang.org/grpc/reflection"
 )
 
 const (
-	port = "0.0.0.0:6789"
+	defaultListenAddress  = "0.0.0.0:6789"
+	defaultConnectTimeout = time.Second * 60
 )
 
 var dbIf common.KatibDBInterface
@@ -71,25 +89,30 @@ func (s *server) Check(ctx context.Context, in *health_pb.HealthCheckRequest) (*
 }
 
 func main() {
+	var connectTimeout time.Duration
+	var listenAddress string
+	flag.DurationVar(&connectTimeout, "connect-timeout", defaultConnectTimeout, "Timeout before calling error during database connection. (e.g. 120s)")
+	flag.StringVar(&listenAddress, "listen-address", defaultListenAddress, "The network interface or IP address to receive incoming connections. (e.g. 0.0.0.0:6789)")
 	flag.Parse()
+
 	var err error
 	dbNameEnvName := common.DBNameEnvName
 	dbName := os.Getenv(dbNameEnvName)
 	if dbName == "" {
 		klog.Fatal("DB_NAME env is not set. Exiting")
 	}
-	dbIf, err = db.NewKatibDBInterface(dbName)
+	dbIf, err = db.NewKatibDBInterface(dbName, connectTimeout)
 	if err != nil {
 		klog.Fatalf("Failed to open db connection: %v", err)
 	}
 	dbIf.DBInit()
-	listener, err := net.Listen("tcp", port)
+	listener, err := net.Listen("tcp", listenAddress)
 	if err != nil {
 		klog.Fatalf("Failed to listen: %v", err)
 	}
 
 	size := 1<<31 - 1
-	klog.Infof("Start Katib manager: %s", port)
+	klog.Infof("Start Katib manager: %s", listenAddress)
 	s := grpc.NewServer(grpc.MaxRecvMsgSize(size), grpc.MaxSendMsgSize(size))
 	api_pb.RegisterDBManagerServer(s, &server{})
 	health_pb.RegisterHealthServer(s, &server{})

cmd/db-manager/v1beta1/main_test.go

@@ -1,10 +1,26 @@
+/*
+Copyright 2022 The Kubeflow Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
 package main
 
 import (
 	"context"
 	"testing"
 
-	"github.com/golang/mock/gomock"
+	"go.uber.org/mock/gomock"
 
 	health_pb "github.com/kubeflow/katib/pkg/apis/manager/health"
 	api_pb "github.com/kubeflow/katib/pkg/apis/manager/v1beta1"

cmd/earlystopping/medianstop/v1beta1/Dockerfile

@@ -1,22 +1,24 @@
-FROM python:3.6
+FROM python:3.11-slim
 
+ARG TARGETARCH
 ENV TARGET_DIR /opt/katib
 ENV EARLY_STOPPING_DIR cmd/earlystopping/medianstop/v1beta1
+ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python
 
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-  apt-get -y update && \
-  apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-  pip install cython; \
+RUN if [ "${TARGETARCH}" = "ppc64le" ] || [ "${TARGETARCH}" = "arm64" ]; then \
+    apt-get -y update && \
+    apt-get -y install gfortran libopenblas-dev liblapack-dev && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*; \
   fi
 
 ADD ./pkg/ ${TARGET_DIR}/pkg/
 ADD ./${EARLY_STOPPING_DIR}/ ${TARGET_DIR}/${EARLY_STOPPING_DIR}/
-WORKDIR  ${TARGET_DIR}/${EARLY_STOPPING_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
 
+WORKDIR  ${TARGET_DIR}/${EARLY_STOPPING_DIR}
+
+RUN pip install --prefer-binary --no-cache-dir -r requirements.txt
 RUN chgrp -R 0 ${TARGET_DIR} \
   && chmod -R g+rwX ${TARGET_DIR}
 
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python
-
 ENTRYPOINT ["python", "main.py"]

cmd/earlystopping/medianstop/v1beta1/main.py

@@ -1,9 +1,25 @@
-import grpc
-import time
+# Copyright 2022 The Kubeflow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import logging
+import time
+from concurrent import futures
+
+import grpc
+
 from pkg.apis.manager.v1beta1.python import api_pb2_grpc
 from pkg.earlystopping.v1beta1.medianstop.service import MedianStopService
-from concurrent import futures
 
 _ONE_DAY_IN_SECONDS = 60 * 60 * 24
 DEFAULT_PORT = "0.0.0.0:6788"

cmd/earlystopping/medianstop/v1beta1/requirements.txt

@@ -1,4 +1,5 @@
-grpcio==1.23.0
-protobuf==3.9.1
+grpcio>=1.64.1
+protobuf>=4.21.12,<5
 googleapis-common-protos==1.6.0
-kubernetes==11.0.0
+kubernetes==22.6.0
+cython>=0.29.24

cmd/katib-controller/v1alpha3/Dockerfile

@@ -1,22 +0,0 @@
-# Build the manager binary
-FROM golang:alpine AS build-env
-
-# Copy in the go src
-ADD . /go/src/github.com/kubeflow/katib
-
-WORKDIR /go/src/github.com/kubeflow/katib/cmd/katib-controller
-# Build
-RUN if [ "$(uname -m)" = "ppc64le" ]; then \
-        CGO_ENABLED=0 GOOS=linux GOARCH=ppc64le go build -a -o katib-controller  ./v1alpha3; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-        CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -o katib-controller  ./v1alpha3; \
-    else \
-        CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o katib-controller  ./v1alpha3; \
-    fi
-# Copy the controller-manager into a thin image
-FROM alpine:3.7
-WORKDIR /app
-RUN apk update && apk add ca-certificates
-COPY --from=build-env /go/src/github.com/kubeflow/katib/cmd/katib-controller/katib-controller .
-USER 1000
-ENTRYPOINT ["./katib-controller"]

cmd/katib-controller/v1alpha3/main.go

@@ -1,123 +0,0 @@
-/*
-Copyright 2018 The Kubeflow Authors
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-    http://www.apache.org/licenses/LICENSE-2.0
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-/*
- Katib-controller is a controller (operator) for Experiments and Trials
-*/
-package main
-
-import (
-	"flag"
-	"os"
-
-	"github.com/spf13/viper"
-	_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
-	"sigs.k8s.io/controller-runtime/pkg/client/config"
-	"sigs.k8s.io/controller-runtime/pkg/manager"
-	logf "sigs.k8s.io/controller-runtime/pkg/runtime/log"
-	"sigs.k8s.io/controller-runtime/pkg/runtime/signals"
-
-	apis "github.com/kubeflow/katib/pkg/apis/controller"
-	controller "github.com/kubeflow/katib/pkg/controller.v1alpha3"
-	"github.com/kubeflow/katib/pkg/controller.v1alpha3/consts"
-	webhook "github.com/kubeflow/katib/pkg/webhook/v1alpha3"
-)
-
-func main() {
-	logf.SetLogger(logf.ZapLogger(false))
-	log := logf.Log.WithName("entrypoint")
-
-	var experimentSuggestionName string
-	var metricsAddr string
-	var webhookPort int
-	var certLocalFS bool
-	var injectSecurityContext bool
-	var serviceName string
-	var enableGRPCProbeInSuggestion bool
-
-	flag.StringVar(&experimentSuggestionName, "experiment-suggestion-name",
-		"default", "The implementation of suggestion interface in experiment controller (default|fake)")
-	flag.StringVar(&metricsAddr, "metrics-addr", ":8080", "The address the metric endpoint binds to.")
-	flag.IntVar(&webhookPort, "webhook-port", 8443, "The port number to be used for admission webhook server.")
-	flag.BoolVar(&certLocalFS, "cert-localfs", false, "Store the webhook cert in local file system")
-	flag.BoolVar(&injectSecurityContext, "webhook-inject-securitycontext", false, "Inject the securityContext of container[0] in the sidecar")
-	flag.StringVar(&serviceName, "webhook-service-name", "katib-controller", "The service name which will be used in webhook")
-	flag.BoolVar(&enableGRPCProbeInSuggestion, "enable-grpc-probe-in-suggestion", true, "enable grpc probe in suggestions")
-
-	flag.Parse()
-
-	// Set the config in viper.
-	viper.Set(consts.ConfigExperimentSuggestionName, experimentSuggestionName)
-	viper.Set(consts.ConfigCertLocalFS, certLocalFS)
-	viper.Set(consts.ConfigInjectSecurityContext, injectSecurityContext)
-	viper.Set(consts.ConfigEnableGRPCProbeInSuggestion, enableGRPCProbeInSuggestion)
-
-	log.Info("Config:",
-		consts.ConfigExperimentSuggestionName,
-		viper.GetString(consts.ConfigExperimentSuggestionName),
-		consts.ConfigCertLocalFS,
-		viper.GetBool(consts.ConfigCertLocalFS),
-		"webhook-port",
-		webhookPort,
-		"metrics-addr",
-		metricsAddr,
-		consts.ConfigInjectSecurityContext,
-		viper.GetBool(consts.ConfigInjectSecurityContext),
-		consts.ConfigEnableGRPCProbeInSuggestion,
-		viper.GetBool(consts.ConfigEnableGRPCProbeInSuggestion),
-	)
-
-	// Get a config to talk to the apiserver
-	cfg, err := config.GetConfig()
-	if err != nil {
-		log.Error(err, "Fail to get the config")
-		os.Exit(1)
-	}
-
-	// Create a new katib controller to provide shared dependencies and start components
-	mgr, err := manager.New(cfg, manager.Options{
-		MetricsBindAddress: metricsAddr,
-	})
-	if err != nil {
-		log.Error(err, "unable add APIs to scheme")
-		os.Exit(1)
-	}
-
-	log.Info("Registering Components.")
-
-	// Setup Scheme for all resources
-	if err := apis.AddToScheme(mgr.GetScheme()); err != nil {
-		log.Error(err, "Fail to create the manager")
-		os.Exit(1)
-	}
-
-	// Setup all Controllers
-	log.Info("Setting up controller")
-	if err := controller.AddToManager(mgr); err != nil {
-		log.Error(err, "unable to register controllers to the manager")
-		os.Exit(1)
-	}
-
-	log.Info("Setting up webhooks")
-	if err := webhook.AddToManager(mgr, int32(webhookPort), serviceName); err != nil {
-		log.Error(err, "unable to register webhooks to the manager")
-		os.Exit(1)
-	}
-
-	// Start the Cmd
-	log.Info("Starting the Cmd.")
-	if err := mgr.Start(signals.SetupSignalHandler()); err != nil {
-		log.Error(err, "unable to run the manager")
-		os.Exit(1)
-	}
-}

cmd/katib-controller/v1beta1/Dockerfile

@@ -1,22 +1,24 @@
-# Build the manager binary
+# Build the Katib controller.
 FROM golang:alpine AS build-env
 
-# Copy in the go src
-ADD . /go/src/github.com/kubeflow/katib
+ARG TARGETARCH
 
-WORKDIR /go/src/github.com/kubeflow/katib/cmd/katib-controller
-# Build
-RUN if [ "$(uname -m)" = "ppc64le" ]; then \
-    CGO_ENABLED=0 GOOS=linux GOARCH=ppc64le go build -a -o katib-controller  ./v1beta1; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -o katib-controller  ./v1beta1; \
-    else \
-    CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o katib-controller  ./v1beta1; \
-    fi
-# Copy the controller-manager into a thin image
-FROM alpine:3.7
+WORKDIR /go/src/github.com/kubeflow/katib
+
+# Download packages.
+COPY go.mod .
+COPY go.sum .
+RUN go mod download -x
+
+# Copy sources.
+COPY cmd/ cmd/
+COPY pkg/ pkg/
+
+# Build the binary.
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} go build -a -o katib-controller ./cmd/katib-controller/v1beta1
+
+# Copy the controller-manager into a thin image.
+FROM alpine:3.15
 WORKDIR /app
-RUN apk update && apk add ca-certificates
-COPY --from=build-env /go/src/github.com/kubeflow/katib/cmd/katib-controller/katib-controller .
-USER 1000
+COPY --from=build-env /go/src/github.com/kubeflow/katib/katib-controller .
 ENTRYPOINT ["./katib-controller"]

cmd/katib-controller/v1beta1/main.go

@@ -1,9 +1,12 @@
 /*
-Copyright 2018 The Kubeflow Authors
+Copyright 2022 The Kubeflow Authors.
+
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
+
     http://www.apache.org/licenses/LICENSE-2.0
+
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -12,7 +15,7 @@ limitations under the License.
 */
 
 /*
- Katib-controller is a controller (operator) for Experiments and Trials
+Katib-controller is a controller (operator) for Experiments and Trials
 */
 package main
 
@@ -21,60 +24,75 @@ import (
 	"os"
 
 	"github.com/spf13/viper"
+	"k8s.io/apimachinery/pkg/runtime"
 	_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
 	"sigs.k8s.io/controller-runtime/pkg/client/config"
+	"sigs.k8s.io/controller-runtime/pkg/healthz"
+	logf "sigs.k8s.io/controller-runtime/pkg/log"
+	"sigs.k8s.io/controller-runtime/pkg/log/zap"
 	"sigs.k8s.io/controller-runtime/pkg/manager"
-	logf "sigs.k8s.io/controller-runtime/pkg/runtime/log"
-	"sigs.k8s.io/controller-runtime/pkg/runtime/signals"
+	"sigs.k8s.io/controller-runtime/pkg/manager/signals"
+	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
+	"sigs.k8s.io/controller-runtime/pkg/webhook"
 
+	configv1beta1 "github.com/kubeflow/katib/pkg/apis/config/v1beta1"
 	apis "github.com/kubeflow/katib/pkg/apis/controller"
-	controller "github.com/kubeflow/katib/pkg/controller.v1beta1"
+	cert "github.com/kubeflow/katib/pkg/certgenerator/v1beta1"
+	"github.com/kubeflow/katib/pkg/controller.v1beta1"
 	"github.com/kubeflow/katib/pkg/controller.v1beta1/consts"
-	trialutil "github.com/kubeflow/katib/pkg/controller.v1beta1/trial/util"
-	webhook "github.com/kubeflow/katib/pkg/webhook/v1beta1"
+	"github.com/kubeflow/katib/pkg/util/v1beta1/katibconfig"
+	webhookv1beta1 "github.com/kubeflow/katib/pkg/webhook/v1beta1"
+	utilruntime "k8s.io/apimachinery/pkg/util/runtime"
+	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
 )
 
+var (
+	scheme = runtime.NewScheme()
+	log    = logf.Log.WithName("entrypoint")
+)
+
+func init() {
+	utilruntime.Must(apis.AddToScheme(scheme))
+	utilruntime.Must(configv1beta1.AddToScheme(scheme))
+	utilruntime.Must(clientgoscheme.AddToScheme(scheme))
+}
+
 func main() {
-	logf.SetLogger(logf.ZapLogger(false))
-	log := logf.Log.WithName("entrypoint")
-
-	var experimentSuggestionName string
-	var metricsAddr string
-	var webhookPort int
-	var certLocalFS bool
-	var injectSecurityContext bool
-	var serviceName string
-	var enableGRPCProbeInSuggestion bool
-	var trialResources trialutil.GvkListFlag
-
-	flag.StringVar(&experimentSuggestionName, "experiment-suggestion-name",
-		"default", "The implementation of suggestion interface in experiment controller (default)")
-	flag.StringVar(&metricsAddr, "metrics-addr", ":8080", "The address the metric endpoint binds to.")
-	flag.IntVar(&webhookPort, "webhook-port", 8443, "The port number to be used for admission webhook server.")
-	flag.BoolVar(&certLocalFS, "cert-localfs", false, "Store the webhook cert in local file system")
-	flag.BoolVar(&injectSecurityContext, "webhook-inject-securitycontext", false, "Inject the securityContext of container[0] in the sidecar")
-	flag.StringVar(&serviceName, "webhook-service-name", "katib-controller", "The service name which will be used in webhook")
-	flag.BoolVar(&enableGRPCProbeInSuggestion, "enable-grpc-probe-in-suggestion", true, "enable grpc probe in suggestions")
-	flag.Var(&trialResources, "trial-resources", "The list of resources that can be used as trial template, in the form: Kind.version.group (e.g. TFJob.v1.kubeflow.org)")
+	logf.SetLogger(zap.New())
 
+	var katibConfigFile string
+	flag.StringVar(&katibConfigFile, "katib-config", "",
+		"The katib-controller will load its initial configuration from this file. "+
+			"Omit this flag to use the default configuration values. ")
 	flag.Parse()
 
+	initConfig, err := katibconfig.GetInitConfigData(scheme, katibConfigFile)
+	if err != nil {
+		log.Error(err, "Failed to get KatibConfig")
+		os.Exit(1)
+	}
+
 	// Set the config in viper.
-	viper.Set(consts.ConfigExperimentSuggestionName, experimentSuggestionName)
-	viper.Set(consts.ConfigCertLocalFS, certLocalFS)
-	viper.Set(consts.ConfigInjectSecurityContext, injectSecurityContext)
-	viper.Set(consts.ConfigEnableGRPCProbeInSuggestion, enableGRPCProbeInSuggestion)
-	viper.Set(consts.ConfigTrialResources, trialResources)
+	viper.Set(consts.ConfigExperimentSuggestionName, initConfig.ControllerConfig.ExperimentSuggestionName)
+	viper.Set(consts.ConfigInjectSecurityContext, initConfig.ControllerConfig.InjectSecurityContext)
+	viper.Set(consts.ConfigEnableGRPCProbeInSuggestion, initConfig.ControllerConfig.EnableGRPCProbeInSuggestion)
+
+	trialGVKs, err := katibconfig.TrialResourcesToGVKs(initConfig.ControllerConfig.TrialResources)
+	if err != nil {
+		log.Error(err, "Failed to parse trialResources")
+		os.Exit(1)
+	}
+	viper.Set(consts.ConfigTrialResources, trialGVKs)
 
 	log.Info("Config:",
 		consts.ConfigExperimentSuggestionName,
 		viper.GetString(consts.ConfigExperimentSuggestionName),
-		consts.ConfigCertLocalFS,
-		viper.GetBool(consts.ConfigCertLocalFS),
 		"webhook-port",
-		webhookPort,
+		initConfig.ControllerConfig.WebhookPort,
 		"metrics-addr",
-		metricsAddr,
+		initConfig.ControllerConfig.MetricsAddr,
+		"healthz-addr",
+		initConfig.ControllerConfig.HealthzAddr,
 		consts.ConfigInjectSecurityContext,
 		viper.GetBool(consts.ConfigInjectSecurityContext),
 		consts.ConfigEnableGRPCProbeInSuggestion,
@@ -92,38 +110,76 @@ func main() {
 
 	// Create a new katib controller to provide shared dependencies and start components
 	mgr, err := manager.New(cfg, manager.Options{
-		MetricsBindAddress: metricsAddr,
+		Metrics: metricsserver.Options{
+			BindAddress: initConfig.ControllerConfig.MetricsAddr,
+		},
+		HealthProbeBindAddress: initConfig.ControllerConfig.HealthzAddr,
+		LeaderElection:         initConfig.ControllerConfig.EnableLeaderElection,
+		LeaderElectionID:       initConfig.ControllerConfig.LeaderElectionID,
+		Scheme:                 scheme,
 	})
 	if err != nil {
-		log.Error(err, "unable add APIs to scheme")
+		log.Error(err, "Failed to create the manager")
 		os.Exit(1)
 	}
 
 	log.Info("Registering Components.")
 
-	// Setup Scheme for all resources
-	if err := apis.AddToScheme(mgr.GetScheme()); err != nil {
-		log.Error(err, "Fail to create the manager")
-		os.Exit(1)
+	// Create a webhook server.
+	hookServer := webhook.NewServer(webhook.Options{
+		Port:    *initConfig.ControllerConfig.WebhookPort,
+		CertDir: consts.CertDir,
+	})
+
+	ctx := signals.SetupSignalHandler()
+	certsReady := make(chan struct{})
+	defer close(certsReady)
+
+	// The setupControllers will register controllers to the manager
+	// after generated certs for the admission webhooks.
+	go setupControllers(mgr, certsReady, hookServer)
+
+	if initConfig.CertGeneratorConfig.Enable {
+		if err = cert.AddToManager(mgr, initConfig.CertGeneratorConfig, certsReady); err != nil {
+			log.Error(err, "Failed to set up cert-generator")
+		}
+	} else {
+		certsReady <- struct{}{}
 	}
 
-	// Setup all Controllers
-	log.Info("Setting up controller")
-	if err := controller.AddToManager(mgr); err != nil {
-		log.Error(err, "unable to register controllers to the manager")
+	log.Info("Setting up health checker.")
+	if err := mgr.AddReadyzCheck("readyz", hookServer.StartedChecker()); err != nil {
+		log.Error(err, "Unable to add readyz endpoint to the manager")
 		os.Exit(1)
 	}
-
-	log.Info("Setting up webhooks")
-	if err := webhook.AddToManager(mgr, int32(webhookPort), serviceName); err != nil {
-		log.Error(err, "unable to register webhooks to the manager")
+	if err = mgr.AddHealthzCheck("healthz", healthz.Ping); err != nil {
+		log.Error(err, "Add webhook server health checker to the manager failed")
 		os.Exit(1)
 	}
 
 	// Start the Cmd
-	log.Info("Starting the Cmd.")
-	if err := mgr.Start(signals.SetupSignalHandler()); err != nil {
-		log.Error(err, "unable to run the manager")
+	log.Info("Starting the manager.")
+	if err = mgr.Start(ctx); err != nil {
+		log.Error(err, "Unable to run the manager")
+		os.Exit(1)
+	}
+}
+
+func setupControllers(mgr manager.Manager, certsReady chan struct{}, hookServer webhook.Server) {
+	// The certsReady blocks to register controllers until generated certs.
+	<-certsReady
+	log.Info("Certs ready")
+
+	// Setup all Controllers
+	log.Info("Setting up controller.")
+	if err := controller.AddToManager(mgr); err != nil {
+		log.Error(err, "Unable to register controllers to the manager")
+		os.Exit(1)
+	}
+
+	log.Info("Setting up webhooks.")
+	if err := webhookv1beta1.AddToManager(mgr, hookServer); err != nil {
+		log.Error(err, "Unable to register webhooks to the manager")
 		os.Exit(1)
 	}
 }

cmd/metricscollector/v1alpha3/file-metricscollector/Dockerfile

@@ -1,22 +0,0 @@
-# Build the manager binary
-FROM golang:alpine AS build-env
-
-# Copy in the go src
-ADD . /go/src/github.com/kubeflow/katib
-
-WORKDIR /go/src/github.com/kubeflow/katib/cmd/metricscollector/v1alpha3/file-metricscollector/
-
-# Build
-RUN if [ "$(uname -m)" = "ppc64le" ]; then \
-        CGO_ENABLED=0 GOOS=linux GOARCH=ppc64le go build -a -o file-metricscollector ./; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-        CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -o file-metricscollector ./; \
-    else \
-        CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o file-metricscollector ./; \
-    fi
-
-# Copy the controller-manager into a thin image
-FROM alpine:3.7
-WORKDIR /app
-COPY --from=build-env /go/src/github.com/kubeflow/katib/cmd/metricscollector/v1alpha3/file-metricscollector/file-metricscollector .
-ENTRYPOINT ["./file-metricscollector"]

cmd/metricscollector/v1alpha3/file-metricscollector/main.go

@@ -1,128 +0,0 @@
-/*
-Copyright 2018 The Kubeflow Authors
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-/*
-MetricsCollector is a default metricscollector for worker.
-It will collect metrics from pod log.
-You should print metrics in {{MetricsName}}={{MetricsValue}} format.
-For example, the objective value name is F1 and the metrics are loss, your training code should print like below.
-     ---
-     epoch 1:
-     batch1 loss=0.8
-     batch2 loss=0.6
-
-     F1=0.4
-
-     epoch 2:
-     batch1 loss=0.4
-     batch2 loss=0.2
-
-     F1=0.7
-     ---
-The metrics collector will collect all logs of metrics.
-*/
-
-package main
-
-import (
-	"context"
-	"flag"
-	"os"
-	"path/filepath"
-	"strings"
-
-	"github.com/hpcloud/tail"
-	"google.golang.org/grpc"
-	"k8s.io/klog"
-
-	api "github.com/kubeflow/katib/pkg/apis/manager/v1alpha3"
-	"github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/common"
-	filemc "github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/file-metricscollector"
-)
-
-var (
-	metricsFileName = flag.String("path", "", "Metrics File Path")
-	trialName       = flag.String("t", "", "Trial Name")
-	managerService  = flag.String("s", "", "Katib Manager service")
-	metricNames     = flag.String("m", "", "Metric names")
-	filters         = flag.String("f", "", "Metric filters")
-	pollInterval    = flag.Duration("p", common.DefaultPollInterval, "Poll interval to check if main process of worker container exit")
-	timeout         = flag.Duration("timeout", common.DefaultTimeout, "Timeout to check if main process of worker container exit")
-	waitAll         = flag.Bool("w", common.DefaultWaitAll, "Whether wait for all other main process of container exiting")
-)
-
-func printMetricsFile(mFile string) {
-	for {
-		_, err := os.Stat(mFile)
-		if err == nil {
-			break
-		} else if os.IsNotExist(err) {
-			continue
-		} else {
-			klog.Fatalf("could not watch metrics file: %v", err)
-		}
-	}
-
-	t, _ := tail.TailFile(mFile, tail.Config{Follow: true})
-	for line := range t.Lines {
-		klog.Info(line.Text)
-	}
-}
-
-func main() {
-	flag.Parse()
-	klog.Infof("Trial Name: %s", *trialName)
-
-	go printMetricsFile(*metricsFileName)
-	wopts := common.WaitPidsOpts{
-		PollInterval:           *pollInterval,
-		Timeout:                *timeout,
-		WaitAll:                *waitAll,
-		CompletedMarkedDirPath: filepath.Dir(*metricsFileName),
-	}
-	if err := common.Wait(wopts); err != nil {
-		klog.Fatalf("Failed to wait for worker container: %v", err)
-	}
-
-	conn, err := grpc.Dial(*managerService, grpc.WithInsecure())
-	if err != nil {
-		klog.Fatalf("could not connect: %v", err)
-	}
-	defer conn.Close()
-	c := api.NewManagerClient(conn)
-	ctx := context.Background()
-	var metricList []string
-	if len(*metricNames) != 0 {
-		metricList = strings.Split(*metricNames, ";")
-	}
-	var filterList []string
-	if len(*filters) != 0 {
-		filterList = strings.Split(*filters, ";")
-	}
-	olog, err := filemc.CollectObservationLog(*metricsFileName, metricList, filterList)
-	if err != nil {
-		klog.Fatalf("Failed to collect logs: %v", err)
-	}
-	reportreq := &api.ReportObservationLogRequest{
-		TrialName:      *trialName,
-		ObservationLog: olog,
-	}
-	_, err = c.ReportObservationLog(ctx, reportreq)
-	if err != nil {
-		klog.Fatalf("Failed to Report logs: %v", err)
-	}
-	klog.Infof("Metrics reported. :\n%v", olog)
-}

cmd/metricscollector/v1alpha3/tfevent-metricscollector/Dockerfile

@@ -1,7 +0,0 @@
-FROM tensorflow/tensorflow:1.11.0
-RUN pip install rfc3339 grpcio googleapis-common-protos
-ADD . /usr/src/app/github.com/kubeflow/katib
-WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/metricscollector/v1alpha3/tfevent-metricscollector/
-RUN pip install --no-cache-dir -r requirements.txt
-ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha3/python:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/tfevent-metricscollector/:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/common/
-ENTRYPOINT ["python", "main.py"]

cmd/metricscollector/v1alpha3/tfevent-metricscollector/Dockerfile.aarch64

@@ -1,28 +0,0 @@
-FROM ubuntu:18.04
-
-RUN apt-get update \
-    && apt-get -y install software-properties-common \
-        autoconf \
-        automake \
-        build-essential \
-        cmake \
-        pkg-config \
-        wget \
-        python-pip \
-        libhdf5-dev \
-        libhdf5-serial-dev \
-        hdf5-tools\
-    && apt-get clean \
-    && rm -rf /var/lib/apt/lists/*
-
-RUN wget https://github.com/lhelontra/tensorflow-on-arm/releases/download/v1.11.0/tensorflow-1.11.0-cp27-none-linux_aarch64.whl \
-    && pip install tensorflow-1.11.0-cp27-none-linux_aarch64.whl \
-    && rm tensorflow-1.11.0-cp27-none-linux_aarch64.whl \
-    && rm -rf .cache
-
-RUN pip install rfc3339 grpcio googleapis-common-protos jupyter
-ADD . /usr/src/app/github.com/kubeflow/katib
-WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/metricscollector/v1alpha3/tfevent-metricscollector/
-RUN pip install --no-cache-dir -r requirements.txt
-ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha3/python:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/tfevent-metricscollector/:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/common/
-ENTRYPOINT ["python", "main.py"]

cmd/metricscollector/v1alpha3/tfevent-metricscollector/Dockerfile.ppc64le

@@ -1,7 +0,0 @@
-FROM ibmcom/tensorflow-ppc64le:1.14.0-py3
-RUN pip install rfc3339 grpcio googleapis-common-protos
-ADD . /usr/src/app/github.com/kubeflow/katib
-WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/metricscollector/v1alpha3/tfevent-metricscollector/
-RUN pip install --no-cache-dir -r requirements.txt
-ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha3/python:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/tfevent-metricscollector/:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/common/
-ENTRYPOINT ["python", "main.py"]

cmd/metricscollector/v1alpha3/tfevent-metricscollector/main.py

@@ -1,54 +0,0 @@
-import grpc
-import argparse
-import api_pb2
-import api_pb2_grpc
-from pns import WaitOtherMainProcesses
-from tfevent_loader import MetricsCollector
-from logging import getLogger, StreamHandler, INFO
-
-timeout_in_seconds = 60
-
-
-def parse_options():
-    parser = argparse.ArgumentParser(
-        description='TF-Event MetricsCollector',
-        add_help=True
-    )
-    parser.add_argument("-s", "--manager_server_addr",
-                        type=str, default="katib-db-manager:6789")
-    parser.add_argument("-t", "--trial_name", type=str, default="")
-    parser.add_argument("-path", "--dir_path", type=str, default="/log")
-    parser.add_argument("-m", "--metric_names", type=str, default="")
-    parser.add_argument("-f", "--metric_filters", type=str, default="")
-    opt = parser.parse_args()
-    return opt
-
-
-if __name__ == '__main__':
-    logger = getLogger(__name__)
-    handler = StreamHandler()
-    handler.setLevel(INFO)
-    logger.setLevel(INFO)
-    logger.addHandler(handler)
-    logger.propagate = False
-    opt = parse_options()
-    manager_server = opt.manager_server_addr.split(':')
-    if len(manager_server) != 2:
-        raise Exception("Invalid katib manager service address: %s" %
-                        opt.manager_server_addr)
-
-    WaitOtherMainProcesses(completed_marked_dir=opt.dir_path)
-
-    mc = MetricsCollector(opt.metric_names.split(';'))
-    observation_log = mc.parse_file(opt.dir_path)
-
-    channel = grpc.beta.implementations.insecure_channel(
-        manager_server[0], int(manager_server[1]))
-
-    with api_pb2.beta_create_Manager_stub(channel) as client:
-        logger.info("In " + opt.trial_name + " " +
-                    str(len(observation_log.metric_logs)) + " metrics will be reported.")
-        client.ReportObservationLog(api_pb2.ReportObservationLogRequest(
-            trial_name=opt.trial_name,
-            observation_log=observation_log
-        ), timeout=timeout_in_seconds)

cmd/metricscollector/v1alpha3/tfevent-metricscollector/requirements.txt

@@ -1 +0,0 @@
-psutil==5.6.6

cmd/metricscollector/v1beta1/file-metricscollector/Dockerfile

@@ -1,22 +1,24 @@
-# Build the manager binary
+# Build the Katib file metrics collector.
 FROM golang:alpine AS build-env
 
-# Copy in the go src
-ADD . /go/src/github.com/kubeflow/katib
+ARG TARGETARCH
 
-WORKDIR /go/src/github.com/kubeflow/katib/cmd/metricscollector/v1beta1/file-metricscollector/
+WORKDIR /go/src/github.com/kubeflow/katib
 
-# Build
-RUN if [ "$(uname -m)" = "ppc64le" ]; then \
-    CGO_ENABLED=0 GOOS=linux GOARCH=ppc64le go build -a -o file-metricscollector ./; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -o file-metricscollector ./; \
-    else \
-    CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o file-metricscollector ./; \
-    fi
+# Download packages.
+COPY go.mod .
+COPY go.sum .
+RUN go mod download -x
 
-# Copy the controller-manager into a thin image
-FROM alpine:3.7
+# Copy sources.
+COPY cmd/ cmd/
+COPY pkg/ pkg/
+
+# Build the binary.
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} go build -a -o file-metricscollector ./cmd/metricscollector/v1beta1/file-metricscollector
+
+# Copy the file metrics collector into a thin image.
+FROM alpine:3.15
 WORKDIR /app
-COPY --from=build-env /go/src/github.com/kubeflow/katib/cmd/metricscollector/v1beta1/file-metricscollector/file-metricscollector .
+COPY --from=build-env /go/src/github.com/kubeflow/katib/file-metricscollector .
 ENTRYPOINT ["./file-metricscollector"]

cmd/metricscollector/v1beta1/file-metricscollector/main.go

@@ -1,5 +1,5 @@
 /*
-Copyright 2018 The Kubeflow Authors
+Copyright 2022 The Kubeflow Authors.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -39,19 +39,21 @@ package main
 
 import (
 	"context"
+	"encoding/json"
 	"flag"
 	"fmt"
-	"io/ioutil"
 	"os"
 	"path/filepath"
+	"regexp"
 	"strconv"
 	"strings"
 	"time"
 
-	"github.com/hpcloud/tail"
-	psutil "github.com/shirou/gopsutil/process"
+	"github.com/nxadm/tail"
+	psutil "github.com/shirou/gopsutil/v3/process"
 	"google.golang.org/grpc"
-	"k8s.io/klog"
+	"google.golang.org/grpc/credentials/insecure"
+	"k8s.io/klog/v2"
 
 	commonv1beta1 "github.com/kubeflow/katib/pkg/apis/controller/common/v1beta1"
 	api "github.com/kubeflow/katib/pkg/apis/manager/v1beta1"
@@ -102,12 +104,13 @@ var (
 	earlyStopServiceAddr = flag.String("s-earlystop", "", "Katib Early Stopping service endpoint")
 	trialName            = flag.String("t", "", "Trial Name")
 	metricsFilePath      = flag.String("path", "", "Metrics File Path")
+	metricsFileFormat    = flag.String("format", "", "Metrics File Format")
 	metricNames          = flag.String("m", "", "Metric names")
 	objectiveType        = flag.String("o-type", "", "Objective type")
 	metricFilters        = flag.String("f", "", "Metric filters")
 	pollInterval         = flag.Duration("p", common.DefaultPollInterval, "Poll interval between running processes check")
 	timeout              = flag.Duration("timeout", common.DefaultTimeout, "Timeout before invoke error during running processes check")
-	waitAll              = flag.Bool("w", common.DefaultWaitAll, "Whether wait for all other main process of container exiting")
+	waitAllProcesses     = flag.String("w", common.DefaultWaitAllProcesses, "Whether wait for all other main process of container exiting")
 	stopRules            stopRulesFlag
 	isEarlyStopped       = false
 )
@@ -131,13 +134,17 @@ func printMetricsFile(mFile string) {
 	checkMetricFile(mFile)
 
 	// Print lines from metrics file.
-	t, _ := tail.TailFile(mFile, tail.Config{Follow: true})
+	t, err := tail.TailFile(mFile, tail.Config{Follow: true, ReOpen: true})
+	if err != nil {
+		klog.Errorf("Failed to open metrics file: %v", err)
+	}
+
 	for line := range t.Lines {
 		klog.Info(line.Text)
 	}
 }
 
-func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
+func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string, fileFormat commonv1beta1.FileFormat) {
 
 	// metricStartStep is the dict where key = metric name, value = start step.
 	// We should apply early stopping rule only if metric is reported at least "start_step" times.
@@ -148,9 +155,6 @@ func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
 		}
 	}
 
-	// First metric is objective in metricNames array.
-	objMetric := strings.Split(*metricNames, ";")[0]
-	objType := commonv1beta1.ObjectiveType(*objectiveType)
 	// For objective metric we calculate best optimal value from the recorded metrics.
 	// This is workaround for Median Stop algorithm.
 	// TODO (andreyvelich): Think about it, maybe define latest, max or min strategy type in stop-rule as well ?
@@ -159,8 +163,10 @@ func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
 	// Check that metric file exists.
 	checkMetricFile(mFile)
 
-	// Get Main proccess.
-	_, mainProcPid, err := common.GetMainProcesses(mFile)
+	// Get Main process.
+	// Extract the metric file dir path based on the file name.
+	mDirPath, _ := filepath.Split(mFile)
+	_, mainProcPid, err := common.GetMainProcesses(mDirPath)
 	if err != nil {
 		klog.Fatalf("GetMainProcesses failed: %v", err)
 	}
@@ -169,9 +175,6 @@ func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
 		klog.Fatalf("Failed to create new Process from pid %v, error: %v", mainProcPid, err)
 	}
 
-	// Get list of regural expressions from filters.
-	metricRegList := filemc.GetFilterRegexpList(filters)
-
 	// Start watch log lines.
 	t, _ := tail.TailFile(mFile, tail.Config{Follow: true})
 	for line := range t.Lines {
@@ -179,78 +182,82 @@ func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
 		// Print log line
 		klog.Info(logText)
 
-		// Check if log line contains metric from stop rules.
-		isRuleLine := false
-		for _, rule := range stopRules {
-			if strings.Contains(logText, rule.Name) {
-				isRuleLine = true
-				break
+		switch fileFormat {
+		case commonv1beta1.TextFormat:
+			// Get list of regural expressions from filters.
+			var metricRegList []*regexp.Regexp
+			metricRegList = filemc.GetFilterRegexpList(filters)
+
+			// Check if log line contains metric from stop rules.
+			isRuleLine := false
+			for _, rule := range stopRules {
+				if strings.Contains(logText, rule.Name) {
+					isRuleLine = true
+					break
+				}
+			}
+			// If log line doesn't contain appropriate metric, continue track file.
+			if !isRuleLine {
+				continue
 			}
-		}
-		// If log line doesn't contain appropriate metric, continue track file.
-		if !isRuleLine {
-			continue
-		}
 
-		// If log line contains appropriate metric, find all submatches from metric filters.
-		for _, metricReg := range metricRegList {
-			matchStrings := metricReg.FindAllStringSubmatch(logText, -1)
-			for _, subMatchList := range matchStrings {
-				if len(subMatchList) < 3 {
-					continue
-				}
-				// Submatch must have metric name and float value
-				metricName := strings.TrimSpace(subMatchList[1])
-				metricValue, err := strconv.ParseFloat(strings.TrimSpace(subMatchList[2]), 64)
-				if err != nil {
-					klog.Fatalf("Unable to parse value %v to float for metric %v", metricValue, metricName)
-				}
-
-				// stopRules contains array of EarlyStoppingRules that has not been reached yet.
-				// After rule is reached we delete appropriate element from the array.
-				for idx, rule := range stopRules {
-					if metricName != rule.Name {
+			// If log line contains appropriate metric, find all submatches from metric filters.
+			for _, metricReg := range metricRegList {
+				matchStrings := metricReg.FindAllStringSubmatch(logText, -1)
+				for _, subMatchList := range matchStrings {
+					if len(subMatchList) < 3 {
 						continue
 					}
-
-					// Calculate optimalObjValue.
-					if metricName == objMetric {
-						if optimalObjValue == nil {
-							optimalObjValue = &metricValue
-						} else if objType == commonv1beta1.ObjectiveTypeMaximize && metricValue > *optimalObjValue {
-							optimalObjValue = &metricValue
-						} else if objType == commonv1beta1.ObjectiveTypeMinimize && metricValue < *optimalObjValue {
-							optimalObjValue = &metricValue
-						}
-						// Assign best optimal value to metric value.
-						metricValue = *optimalObjValue
+					// Submatch must have metric name and float value
+					metricName := strings.TrimSpace(subMatchList[1])
+					metricValue, err := strconv.ParseFloat(strings.TrimSpace(subMatchList[2]), 64)
+					if err != nil {
+						klog.Fatalf("Unable to parse value %v to float for metric %v", metricValue, metricName)
 					}
 
-					// Reduce steps if appropriate metric is reported.
-					// Once rest steps are empty we apply early stopping rule.
-					if restSteps, ok := metricStartStep[metricName]; ok {
-						metricStartStep[metricName]--
-						if restSteps != 0 {
+					// stopRules contains array of EarlyStoppingRules that has not been reached yet.
+					// After rule is reached we delete appropriate element from the array.
+					for idx, rule := range stopRules {
+						if metricName != rule.Name {
 							continue
 						}
-					}
-
-					ruleValue, err := strconv.ParseFloat(rule.Value, 64)
-					if err != nil {
-						klog.Fatalf("Unable to parse value %v to float for rule metric %v", rule.Value, rule.Name)
-					}
-
-					// Metric value can be equal, less or greater than stop rule.
-					// Deleting suitable stop rule from the array.
-					if rule.Comparison == commonv1beta1.ComparisonTypeEqual && metricValue == ruleValue {
-						stopRules = deleteStopRule(stopRules, idx)
-					} else if rule.Comparison == commonv1beta1.ComparisonTypeLess && metricValue < ruleValue {
-						stopRules = deleteStopRule(stopRules, idx)
-					} else if rule.Comparison == commonv1beta1.ComparisonTypeGreater && metricValue > ruleValue {
-						stopRules = deleteStopRule(stopRules, idx)
+						stopRules, optimalObjValue = updateStopRules(stopRules, optimalObjValue, metricValue, metricStartStep, rule, idx)
 					}
 				}
 			}
+		case commonv1beta1.JsonFormat:
+			var logJsonObj map[string]interface{}
+			if err = json.Unmarshal([]byte(logText), &logJsonObj); err != nil {
+				klog.Fatalf("Failed to unmarshal logs in %v format, log: %s, error: %v", commonv1beta1.JsonFormat, logText, err)
+			}
+			// Check if log line contains metric from stop rules.
+			isRuleLine := false
+			for _, rule := range stopRules {
+				if _, exist := logJsonObj[rule.Name]; exist {
+					isRuleLine = true
+					break
+				}
+			}
+			// If log line doesn't contain appropriate metric, continue track file.
+			if !isRuleLine {
+				continue
+			}
+
+			// stopRules contains array of EarlyStoppingRules that has not been reached yet.
+			// After rule is reached we delete appropriate element from the array.
+			for idx, rule := range stopRules {
+				value, exist := logJsonObj[rule.Name].(string)
+				if !exist {
+					continue
+				}
+				metricValue, err := strconv.ParseFloat(strings.TrimSpace(value), 64)
+				if err != nil {
+					klog.Fatalf("Unable to parse value %v to float for metric %v", metricValue, rule.Name)
+				}
+				stopRules, optimalObjValue = updateStopRules(stopRules, optimalObjValue, metricValue, metricStartStep, rule, idx)
+			}
+		default:
+			klog.Fatalf("Format must be set to %v or %v", commonv1beta1.TextFormat, commonv1beta1.JsonFormat)
 		}
 
 		// If stopRules array is empty, Trial is early stopped.
@@ -266,12 +273,12 @@ func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
 				klog.Fatalf("Create mark file %v error: %v", markFile, err)
 			}
 
-			err = ioutil.WriteFile(markFile, []byte(common.TrainingEarlyStopped), 0)
+			err = os.WriteFile(markFile, []byte(common.TrainingEarlyStopped), 0)
 			if err != nil {
 				klog.Fatalf("Write to file %v error: %v", markFile, err)
 			}
 
-			// Get child proccess from main PID.
+			// Get child process from main PID.
 			childProc, err := mainProc.Children()
 			if err != nil {
 				klog.Fatalf("Get children proceses for main PID: %v failed: %v", mainProcPid, err)
@@ -289,9 +296,9 @@ func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
 			}
 
 			// Report metrics to DB.
-			reportMetrics(filters)
+			reportMetrics(filters, fileFormat)
 
-			// Wait until main proccess is completed.
+			// Wait until main process is completed.
 			timeout := 60 * time.Second
 			endTime := time.Now().Add(timeout)
 			isProcRunning := true
@@ -304,11 +311,10 @@ func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
 			}
 
 			// Create connection and client for Early Stopping service.
-			conn, err := grpc.Dial(*earlyStopServiceAddr, grpc.WithInsecure())
+			conn, err := grpc.NewClient(*earlyStopServiceAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
 			if err != nil {
 				klog.Fatalf("Could not connect to Early Stopping service, error: %v", err)
 			}
-			defer conn.Close()
 			c := api.NewEarlyStoppingClient(conn)
 
 			setTrialStatusReq := &api.SetTrialStatusRequest{
@@ -320,13 +326,65 @@ func watchMetricsFile(mFile string, stopRules stopRulesFlag, filters []string) {
 			if err != nil {
 				klog.Fatalf("Set Trial status error: %v", err)
 			}
+			conn.Close()
 
 			klog.Infof("Trial status is successfully updated")
-
 		}
 	}
 }
 
+func updateStopRules(
+	stopRules []commonv1beta1.EarlyStoppingRule,
+	optimalObjValue *float64,
+	metricValue float64,
+	metricStartStep map[string]int,
+	rule commonv1beta1.EarlyStoppingRule,
+	ruleIdx int,
+) ([]commonv1beta1.EarlyStoppingRule, *float64) {
+
+	// First metric is objective in metricNames array.
+	objMetric := strings.Split(*metricNames, ";")[0]
+	objType := commonv1beta1.ObjectiveType(*objectiveType)
+
+	// Calculate optimalObjValue.
+	if rule.Name == objMetric {
+		if optimalObjValue == nil {
+			optimalObjValue = &metricValue
+		} else if objType == commonv1beta1.ObjectiveTypeMaximize && metricValue > *optimalObjValue {
+			optimalObjValue = &metricValue
+		} else if objType == commonv1beta1.ObjectiveTypeMinimize && metricValue < *optimalObjValue {
+			optimalObjValue = &metricValue
+		}
+		// Assign best optimal value to metric value.
+		metricValue = *optimalObjValue
+	}
+
+	// Reduce steps if appropriate metric is reported.
+	// Once rest steps are empty we apply early stopping rule.
+	if _, ok := metricStartStep[rule.Name]; ok {
+		metricStartStep[rule.Name]--
+		if metricStartStep[rule.Name] != 0 {
+			return stopRules, optimalObjValue
+		}
+	}
+
+	ruleValue, err := strconv.ParseFloat(rule.Value, 64)
+	if err != nil {
+		klog.Fatalf("Unable to parse value %v to float for rule metric %v", rule.Value, rule.Name)
+	}
+
+	// Metric value can be equal, less or greater than stop rule.
+	// Deleting suitable stop rule from the array.
+	if rule.Comparison == commonv1beta1.ComparisonTypeEqual && metricValue == ruleValue {
+		return deleteStopRule(stopRules, ruleIdx), optimalObjValue
+	} else if rule.Comparison == commonv1beta1.ComparisonTypeLess && metricValue < ruleValue {
+		return deleteStopRule(stopRules, ruleIdx), optimalObjValue
+	} else if rule.Comparison == commonv1beta1.ComparisonTypeGreater && metricValue > ruleValue {
+		return deleteStopRule(stopRules, ruleIdx), optimalObjValue
+	}
+	return stopRules, optimalObjValue
+}
+
 func deleteStopRule(stopRules []commonv1beta1.EarlyStoppingRule, idx int) []commonv1beta1.EarlyStoppingRule {
 	if idx >= len(stopRules) {
 		klog.Fatalf("Index %v out of range stopRules: %v", idx, stopRules)
@@ -346,17 +404,21 @@ func main() {
 		filters = strings.Split(*metricFilters, ";")
 	}
 
+	fileFormat := commonv1beta1.FileFormat(*metricsFileFormat)
+
 	// If stop rule is set we need to parse metrics during run.
 	if len(stopRules) != 0 {
-		go watchMetricsFile(*metricsFilePath, stopRules, filters)
+		go watchMetricsFile(*metricsFilePath, stopRules, filters, fileFormat)
 	} else {
 		go printMetricsFile(*metricsFilePath)
 	}
 
+	waitAll, _ := strconv.ParseBool(*waitAllProcesses)
+
 	wopts := common.WaitPidsOpts{
 		PollInterval:           *pollInterval,
 		Timeout:                *timeout,
-		WaitAll:                *waitAll,
+		WaitAll:                waitAll,
 		CompletedMarkedDirPath: filepath.Dir(*metricsFilePath),
 	}
 	if err := common.WaitMainProcesses(wopts); err != nil {
@@ -365,13 +427,13 @@ func main() {
 
 	// If training was not early stopped, report the metrics.
 	if !isEarlyStopped {
-		reportMetrics(filters)
+		reportMetrics(filters, fileFormat)
 	}
 }
 
-func reportMetrics(filters []string) {
+func reportMetrics(filters []string, fileFormat commonv1beta1.FileFormat) {
 
-	conn, err := grpc.Dial(*dbManagerServiceAddr, grpc.WithInsecure())
+	conn, err := grpc.NewClient(*dbManagerServiceAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
 	if err != nil {
 		klog.Fatalf("Could not connect to DB manager service, error: %v", err)
 	}
@@ -382,7 +444,7 @@ func reportMetrics(filters []string) {
 	if len(*metricNames) != 0 {
 		metricList = strings.Split(*metricNames, ";")
 	}
-	olog, err := filemc.CollectObservationLog(*metricsFilePath, metricList, filters)
+	olog, err := filemc.CollectObservationLog(*metricsFilePath, metricList, filters, fileFormat)
 	if err != nil {
 		klog.Fatalf("Failed to collect logs: %v", err)
 	}

cmd/metricscollector/v1beta1/tfevent-metricscollector/Dockerfile

@@ -1,7 +1,24 @@
-FROM tensorflow/tensorflow:1.11.0
-RUN pip install rfc3339 grpcio googleapis-common-protos
-ADD . /usr/src/app/github.com/kubeflow/katib
-WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/metricscollector/v1beta1/tfevent-metricscollector/
-RUN pip install --no-cache-dir -r requirements.txt
-ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1beta1/python:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1beta1/tfevent-metricscollector/:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1beta1/common/
+FROM python:3.11-slim
+
+ARG TARGETARCH
+ENV TARGET_DIR /opt/katib
+ENV METRICS_COLLECTOR_DIR cmd/metricscollector/v1beta1/tfevent-metricscollector
+ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/metricscollector/v1beta1/tfevent-metricscollector/::${TARGET_DIR}/pkg/metricscollector/v1beta1/common/
+
+ADD ./pkg/ ${TARGET_DIR}/pkg/
+ADD ./${METRICS_COLLECTOR_DIR}/ ${TARGET_DIR}/${METRICS_COLLECTOR_DIR}/
+
+WORKDIR  ${TARGET_DIR}/${METRICS_COLLECTOR_DIR}
+
+RUN if [ "${TARGETARCH}" = "arm64" ]; then \
+    apt-get -y update && \
+    apt-get -y install gfortran libpcre3 libpcre3-dev && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*; \
+    fi
+
+RUN pip install --prefer-binary --no-cache-dir -r requirements.txt
+RUN chgrp -R 0 ${TARGET_DIR} \
+  && chmod -R g+rwX ${TARGET_DIR}
+
 ENTRYPOINT ["python", "main.py"]

cmd/metricscollector/v1beta1/tfevent-metricscollector/Dockerfile.aarch64

@@ -1,28 +0,0 @@
-FROM ubuntu:18.04
-
-RUN apt-get update \
-    && apt-get -y install software-properties-common \
-    autoconf \
-    automake \
-    build-essential \
-    cmake \
-    pkg-config \
-    wget \
-    python-pip \
-    libhdf5-dev \
-    libhdf5-serial-dev \
-    hdf5-tools\
-    && apt-get clean \
-    && rm -rf /var/lib/apt/lists/*
-
-RUN wget https://github.com/lhelontra/tensorflow-on-arm/releases/download/v1.11.0/tensorflow-1.11.0-cp27-none-linux_aarch64.whl \
-    && pip install tensorflow-1.11.0-cp27-none-linux_aarch64.whl \
-    && rm tensorflow-1.11.0-cp27-none-linux_aarch64.whl \
-    && rm -rf .cache
-
-RUN pip install rfc3339 grpcio googleapis-common-protos jupyter
-ADD . /usr/src/app/github.com/kubeflow/katib
-WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/metricscollector/v1beta1/tfevent-metricscollector/
-RUN pip install --no-cache-dir -r requirements.txt
-ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1beta1/python:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1beta1/tfevent-metricscollector/:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1beta1/common/
-ENTRYPOINT ["python", "main.py"]

cmd/metricscollector/v1beta1/tfevent-metricscollector/Dockerfile.ppc64le

@@ -1,7 +1,6 @@
-FROM ibmcom/tensorflow-ppc64le:1.14.0-py3
-RUN pip install rfc3339 grpcio googleapis-common-protos
+FROM ibmcom/tensorflow-ppc64le:2.2.0-py3
 ADD . /usr/src/app/github.com/kubeflow/katib
 WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/metricscollector/v1beta1/tfevent-metricscollector/
-RUN pip install --no-cache-dir -r requirements.txt
+RUN pip install --prefer-binary --no-cache-dir -r requirements.txt
 ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1beta1/python:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1beta1/tfevent-metricscollector/:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1beta1/common/
 ENTRYPOINT ["python", "main.py"]

cmd/metricscollector/v1beta1/tfevent-metricscollector/main.py

@@ -1,10 +1,26 @@
-import grpc
+# Copyright 2022 The Kubeflow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import argparse
+from logging import INFO, StreamHandler, getLogger
+
 import api_pb2
-from pns import WaitMainProcesses
+import api_pb2_grpc
 import const
+import grpc
+from pns import WaitMainProcesses
 from tfevent_loader import MetricsCollector
-from logging import getLogger, StreamHandler, INFO
 
 timeout_in_seconds = 60
 
@@ -24,7 +40,7 @@ def parse_options():
     parser.add_argument("-f", "--metric_filters", type=str, default="")
     parser.add_argument("-p", "--poll_interval", type=int, default=const.DEFAULT_POLL_INTERVAL)
     parser.add_argument("-timeout", "--timeout", type=int, default=const.DEFAULT_TIMEOUT)
-    parser.add_argument("-w", "--wait_all", type=bool, default=const.DEFAULT_WAIT_ALL)
+    parser.add_argument("-w", "--wait_all_processes", type=str, default=const.DEFAULT_WAIT_ALL_PROCESSES)
 
     opt = parser.parse_args()
     return opt
@@ -38,27 +54,31 @@ if __name__ == '__main__':
     logger.addHandler(handler)
     logger.propagate = False
     opt = parse_options()
+    wait_all_processes = opt.wait_all_processes.lower() == "true"
     db_manager_server = opt.db_manager_server_addr.split(':')
     if len(db_manager_server) != 2:
-        raise Exception("Invalid Katib DB manager service address: %s" %
-                        opt.db_manager_server_addr)
+        raise Exception(
+            f"Invalid Katib DB manager service address: {opt.db_manager_server_addr}"
+        )
 
     WaitMainProcesses(
         pool_interval=opt.poll_interval,
         timout=opt.timeout,
-        wait_all=opt.wait_all,
-        completed_marked_dir=opt.metrics_file_dir)
+        wait_all=wait_all_processes,
+        completed_marked_dir=opt.metrics_file_dir,
+    )
 
-    mc = MetricsCollector(opt.metric_names.split(';'))
+    mc = MetricsCollector(opt.metric_names.split(";"))
     observation_log = mc.parse_file(opt.metrics_file_dir)
 
-    channel = grpc.beta.implementations.insecure_channel(
-        db_manager_server[0], int(db_manager_server[1]))
-
-    with api_pb2.beta_create_DBManager_stub(channel) as client:
-        logger.info("In " + opt.trial_name + " " +
-                    str(len(observation_log.metric_logs)) + " metrics will be reported.")
-        client.ReportObservationLog(api_pb2.ReportObservationLogRequest(
-            trial_name=opt.trial_name,
-            observation_log=observation_log
-        ), timeout=timeout_in_seconds)
+    with grpc.insecure_channel(opt.db_manager_server_addr) as channel:
+        stub = api_pb2_grpc.DBManagerStub(channel)
+        logger.info(
+            f"In {opt.trial_name} {str(len(observation_log.metric_logs))} metrics will be reported."
+        )
+        stub.ReportObservationLog(
+            api_pb2.ReportObservationLogRequest(
+                trial_name=opt.trial_name, observation_log=observation_log
+            ),
+            timeout=timeout_in_seconds,
+        )

cmd/metricscollector/v1beta1/tfevent-metricscollector/requirements.txt

@@ -1 +1,6 @@
-psutil==5.6.6
+psutil==5.9.4
+rfc3339>=6.2
+grpcio>=1.64.1
+googleapis-common-protos==1.6.0
+tensorflow==2.16.1
+protobuf>=4.21.12,<5

cmd/suggestion/chocolate/v1alpha3/Dockerfile

@@ -1,31 +0,0 @@
-FROM python:3.6
-
-ENV TARGET_DIR /opt/katib
-ENV SUGGESTION_DIR cmd/suggestion/chocolate/v1alpha3
-
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-        apt-get -y update && \
-        apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-        pip install cython 'numpy>=1.13.3'; \
-    fi
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-        wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
-ADD ./pkg/ ${TARGET_DIR}/pkg/
-ADD ./${SUGGESTION_DIR}/ ${TARGET_DIR}/${SUGGESTION_DIR}/
-WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
-
-RUN chgrp -R 0 ${TARGET_DIR} \
-  && chmod -R g+rwX ${TARGET_DIR}
-
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1alpha3/python:${TARGET_DIR}/pkg/apis/manager/health/python
-
-ENTRYPOINT ["python", "main.py"]

cmd/suggestion/chocolate/v1alpha3/main.py

@@ -1,28 +0,0 @@
-import grpc
-import time
-from pkg.apis.manager.v1alpha3.python import api_pb2_grpc
-from pkg.apis.manager.health.python import health_pb2_grpc
-from pkg.suggestion.v1alpha3.chocolate.service import ChocolateService
-from concurrent import futures
-
-_ONE_DAY_IN_SECONDS = 60 * 60 * 24
-DEFAULT_PORT = "0.0.0.0:6789"
-
-
-def serve():
-    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
-    service = ChocolateService()
-    api_pb2_grpc.add_SuggestionServicer_to_server(service, server)
-    health_pb2_grpc.add_HealthServicer_to_server(service, server)
-    server.add_insecure_port(DEFAULT_PORT)
-    print("Listening...")
-    server.start()
-    try:
-        while True:
-            time.sleep(_ONE_DAY_IN_SECONDS)
-    except KeyboardInterrupt:
-        server.stop(0)
-
-
-if __name__ == "__main__":
-    serve()

cmd/suggestion/chocolate/v1alpha3/requirements.txt

@@ -1,12 +0,0 @@
-grpcio==1.23.0
-duecredit===0.7.0
-cloudpickle==0.5.6
-numpy>=1.13.3
-scikit-learn>=0.19.0
-scipy>=0.19.1
-forestci==0.3
-protobuf==3.9.1
-googleapis-common-protos==1.6.0
-SQLAlchemy==1.3.8
-git+https://github.com/AIworx-Labs/chocolate@master
-ghalton>=0.6

cmd/suggestion/chocolate/v1beta1/Dockerfile

@@ -1,31 +0,0 @@
-FROM python:3.6
-
-ENV TARGET_DIR /opt/katib
-ENV SUGGESTION_DIR cmd/suggestion/chocolate/v1beta1
-
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-    apt-get -y update && \
-    apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-    pip install cython 'numpy>=1.13.3'; \
-    fi
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
-ADD ./pkg/ ${TARGET_DIR}/pkg/
-ADD ./${SUGGESTION_DIR}/ ${TARGET_DIR}/${SUGGESTION_DIR}/
-WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
-
-RUN chgrp -R 0 ${TARGET_DIR} \
-  && chmod -R g+rwX ${TARGET_DIR}
-
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python
-
-ENTRYPOINT ["python", "main.py"]

cmd/suggestion/chocolate/v1beta1/main.py

@@ -1,28 +0,0 @@
-import grpc
-import time
-from pkg.apis.manager.v1beta1.python import api_pb2_grpc
-from pkg.apis.manager.health.python import health_pb2_grpc
-from pkg.suggestion.v1beta1.chocolate.service import ChocolateService
-from concurrent import futures
-
-_ONE_DAY_IN_SECONDS = 60 * 60 * 24
-DEFAULT_PORT = "0.0.0.0:6789"
-
-
-def serve():
-    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
-    service = ChocolateService()
-    api_pb2_grpc.add_SuggestionServicer_to_server(service, server)
-    health_pb2_grpc.add_HealthServicer_to_server(service, server)
-    server.add_insecure_port(DEFAULT_PORT)
-    print("Listening...")
-    server.start()
-    try:
-        while True:
-            time.sleep(_ONE_DAY_IN_SECONDS)
-    except KeyboardInterrupt:
-        server.stop(0)
-
-
-if __name__ == "__main__":
-    serve()

cmd/suggestion/chocolate/v1beta1/requirements.txt

@@ -1,12 +0,0 @@
-grpcio==1.23.0
-duecredit===0.7.0
-cloudpickle==0.5.6
-numpy>=1.13.3
-scikit-learn>=0.19.0
-scipy>=0.19.1
-forestci==0.3
-protobuf==3.9.1
-googleapis-common-protos==1.6.0
-SQLAlchemy==1.3.8
-git+https://github.com/AIworx-Labs/chocolate@master
-ghalton>=0.6

cmd/suggestion/goptuna/v1alpha3/Dockerfile

@@ -1,33 +0,0 @@
-FROM golang:alpine AS go-build
-# The GOPATH in the image is /go.
-ADD . /go/src/github.com/kubeflow/katib
-WORKDIR /go/src/github.com/kubeflow/katib/cmd/suggestion/goptuna
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-        apk --update add gcc musl-dev  && \
-        go build -o goptuna-suggestion ./v1alpha3; \
-    else \
-        go build -o goptuna-suggestion ./v1alpha3; \
-    fi
-
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-        wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-        wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-        wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
-FROM alpine:3.7
-
-ENV TARGET_DIR /opt/katib
-
-WORKDIR ${TARGET_DIR}
-COPY --from=go-build /bin/grpc_health_probe /bin/
-COPY --from=go-build /go/src/github.com/kubeflow/katib/cmd/suggestion/goptuna/goptuna-suggestion ${TARGET_DIR}/
-
-RUN chgrp -R 0 ${TARGET_DIR} \
-  && chmod -R g+rwX ${TARGET_DIR}
-
-ENTRYPOINT ["./goptuna-suggestion"]

cmd/suggestion/goptuna/v1alpha3/main.go

@@ -1,41 +0,0 @@
-package main
-
-import (
-	"context"
-	"net"
-
-	health_pb "github.com/kubeflow/katib/pkg/apis/manager/health"
-	"github.com/kubeflow/katib/pkg/apis/manager/v1alpha3"
-	suggestion "github.com/kubeflow/katib/pkg/suggestion/v1alpha3/goptuna"
-	"google.golang.org/grpc"
-	"k8s.io/klog"
-)
-
-const (
-	address = "0.0.0.0:6789"
-)
-
-type healthService struct {
-}
-
-func (s *healthService) Check(ctx context.Context, in *health_pb.HealthCheckRequest) (*health_pb.HealthCheckResponse, error) {
-	return &health_pb.HealthCheckResponse{
-		Status: health_pb.HealthCheckResponse_SERVING,
-	}, nil
-}
-
-func main() {
-	l, err := net.Listen("tcp", address)
-	if err != nil {
-		klog.Fatalf("Failed to listen: %v", err)
-	}
-	srv := grpc.NewServer()
-	api_v1_alpha3.RegisterSuggestionServer(srv, suggestion.NewSuggestionService())
-	health_pb.RegisterHealthServer(srv, &healthService{})
-
-	klog.Infof("Start Goptuna suggestion service: %s", address)
-	err = srv.Serve(l)
-	if err != nil {
-		klog.Fatalf("Failed to serve: %v", err)
-	}
-}

cmd/suggestion/goptuna/v1beta1/Dockerfile

@@ -1,31 +1,30 @@
-FROM golang:alpine AS go-build
-# The GOPATH in the image is /go.
-ADD . /go/src/github.com/kubeflow/katib
-WORKDIR /go/src/github.com/kubeflow/katib/cmd/suggestion/goptuna
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-    apk --update add gcc musl-dev  && \
-    go build -o goptuna-suggestion ./v1beta1; \
-    else \
-    go build -o goptuna-suggestion ./v1beta1; \
-    fi
+# Build the Goptuna Suggestion.
+FROM golang:alpine AS build-env
 
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
+ARG TARGETARCH
 
-FROM alpine:3.7
+WORKDIR /go/src/github.com/kubeflow/katib
+
+# Download packages.
+COPY go.mod .
+COPY go.sum .
+RUN go mod download -x
+
+# Copy sources.
+COPY cmd/ cmd/
+COPY pkg/ pkg/
+
+# Build the binary.
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} go build -a -o goptuna-suggestion ./cmd/suggestion/goptuna/v1beta1
+
+# Copy the Goptuna suggestion into a thin image.
+FROM alpine:3.15
 
 ENV TARGET_DIR /opt/katib
 
 WORKDIR ${TARGET_DIR}
-COPY --from=go-build /bin/grpc_health_probe /bin/
-COPY --from=go-build /go/src/github.com/kubeflow/katib/cmd/suggestion/goptuna/goptuna-suggestion ${TARGET_DIR}/
+
+COPY --from=build-env /go/src/github.com/kubeflow/katib/goptuna-suggestion ${TARGET_DIR}/
 
 RUN chgrp -R 0 ${TARGET_DIR} \
   && chmod -R g+rwX ${TARGET_DIR}

cmd/suggestion/goptuna/v1beta1/main.go

@@ -1,3 +1,19 @@
+/*
+Copyright 2022 The Kubeflow Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
 package main
 
 import (
@@ -8,7 +24,7 @@ import (
 	api_v1_beta1 "github.com/kubeflow/katib/pkg/apis/manager/v1beta1"
 	suggestion "github.com/kubeflow/katib/pkg/suggestion/v1beta1/goptuna"
 	"google.golang.org/grpc"
-	"k8s.io/klog"
+	"k8s.io/klog/v2"
 )
 
 const (

cmd/suggestion/hyperband/v1alpha3/Dockerfile

@@ -1,32 +0,0 @@
-FROM python:3.6
-
-ENV TARGET_DIR /opt/katib
-ENV SUGGESTION_DIR cmd/suggestion/hyperband/v1alpha3
-
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-        apt-get -y update && \
-        apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-        pip install cython; \
-    fi
-
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-        wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
-ADD ./pkg/ ${TARGET_DIR}/pkg/
-ADD ./${SUGGESTION_DIR}/ ${TARGET_DIR}/${SUGGESTION_DIR}/
-WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
-
-RUN chgrp -R 0 ${TARGET_DIR} \
-  && chmod -R g+rwX ${TARGET_DIR}
-
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1alpha3/python:${TARGET_DIR}/pkg/apis/manager/health/python
-
-ENTRYPOINT ["python", "main.py"]

cmd/suggestion/hyperband/v1alpha3/main.py

@@ -1,29 +0,0 @@
-import grpc
-import time
-from pkg.apis.manager.v1alpha3.python import api_pb2_grpc
-from pkg.apis.manager.health.python import health_pb2_grpc
-from pkg.suggestion.v1alpha3.hyperband.service import HyperbandService
-from concurrent import futures
-
-_ONE_DAY_IN_SECONDS = 60 * 60 * 24
-DEFAULT_PORT = "0.0.0.0:6789"
-
-
-def serve():
-    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
-    service = HyperbandService()
-    api_pb2_grpc.add_SuggestionServicer_to_server(service, server)
-    health_pb2_grpc.add_HealthServicer_to_server(service, server)
-
-    server.add_insecure_port(DEFAULT_PORT)
-    print("Listening...")
-    server.start()
-    try:
-        while True:
-            time.sleep(_ONE_DAY_IN_SECONDS)
-    except KeyboardInterrupt:
-        server.stop(0)
-
-
-if __name__ == "__main__":
-    serve()

cmd/suggestion/hyperband/v1alpha3/requirements.txt

@@ -1,9 +0,0 @@
-grpcio==1.23.0
-duecredit===0.7.0
-cloudpickle==0.5.6
-numpy>=1.13.3
-scikit-learn>=0.19.0
-scipy>=0.19.1
-forestci==0.3
-protobuf==3.9.1
-googleapis-common-protos==1.6.0

cmd/suggestion/hyperband/v1beta1/Dockerfile

@@ -1,32 +1,24 @@
-FROM python:3.6
+FROM python:3.11-slim
 
+ARG TARGETARCH
 ENV TARGET_DIR /opt/katib
 ENV SUGGESTION_DIR cmd/suggestion/hyperband/v1beta1
+ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python
 
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
+RUN if [ "${TARGETARCH}" = "ppc64le" ] || [ "${TARGETARCH}" = "arm64" ]; then \
     apt-get -y update && \
     apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-    pip install cython; \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*; \
     fi
 
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
 ADD ./pkg/ ${TARGET_DIR}/pkg/
 ADD ./${SUGGESTION_DIR}/ ${TARGET_DIR}/${SUGGESTION_DIR}/
-WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
 
+WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
+
+RUN pip install --prefer-binary --no-cache-dir -r requirements.txt
 RUN chgrp -R 0 ${TARGET_DIR} \
   && chmod -R g+rwX ${TARGET_DIR}
 
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python
-
 ENTRYPOINT ["python", "main.py"]

cmd/suggestion/hyperband/v1beta1/main.py

@@ -1,10 +1,26 @@
-import grpc
+# Copyright 2022 The Kubeflow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import time
-from pkg.apis.manager.v1beta1.python import api_pb2_grpc
-from pkg.apis.manager.health.python import health_pb2_grpc
-from pkg.suggestion.v1beta1.hyperband.service import HyperbandService
 from concurrent import futures
 
+import grpc
+
+from pkg.apis.manager.health.python import health_pb2_grpc
+from pkg.apis.manager.v1beta1.python import api_pb2_grpc
+from pkg.suggestion.v1beta1.hyperband.service import HyperbandService
+
 _ONE_DAY_IN_SECONDS = 60 * 60 * 24
 DEFAULT_PORT = "0.0.0.0:6789"
 

cmd/suggestion/hyperband/v1beta1/requirements.txt

@@ -1,9 +1,9 @@
-grpcio==1.23.0
-duecredit===0.7.0
+grpcio>=1.64.1
 cloudpickle==0.5.6
-numpy>=1.13.3
-scikit-learn>=0.19.0
-scipy>=0.19.1
+numpy>=1.25.2
+scikit-learn>=0.24.0
+scipy>=1.5.4
 forestci==0.3
-protobuf==3.9.1
+protobuf>=4.21.12,<5
 googleapis-common-protos==1.6.0
+cython>=0.29.24

cmd/suggestion/hyperopt/v1alpha3/Dockerfile

@@ -1,33 +0,0 @@
-FROM python:3.6
-
-ENV TARGET_DIR /opt/katib
-ENV SUGGESTION_DIR cmd/suggestion/hyperopt/v1alpha3
-
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-        apt-get -y update && \
-        apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-        pip install cython; \
-    fi
-
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-        wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-	wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
-ADD ./pkg/ ${TARGET_DIR}/pkg/
-ADD ./${SUGGESTION_DIR}/ ${TARGET_DIR}/${SUGGESTION_DIR}/
-WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
-
-RUN chgrp -R 0 ${TARGET_DIR} \
-  && chmod -R g+rwX ${TARGET_DIR}
-
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1alpha3/python:${TARGET_DIR}/pkg/apis/manager/health/python
-
-ENTRYPOINT ["python", "main.py"]
-

cmd/suggestion/hyperopt/v1alpha3/main.py

@@ -1,28 +0,0 @@
-import grpc
-import time
-from pkg.apis.manager.v1alpha3.python import api_pb2_grpc
-from pkg.apis.manager.health.python import health_pb2_grpc
-from pkg.suggestion.v1alpha3.hyperopt.service import HyperoptService
-from concurrent import futures
-
-_ONE_DAY_IN_SECONDS = 60 * 60 * 24
-DEFAULT_PORT = "0.0.0.0:6789"
-
-
-def serve():
-    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
-    service = HyperoptService()
-    api_pb2_grpc.add_SuggestionServicer_to_server(service, server)
-    health_pb2_grpc.add_HealthServicer_to_server(service, server)
-    server.add_insecure_port(DEFAULT_PORT)
-    print("Listening...")
-    server.start()
-    try:
-        while True:
-            time.sleep(_ONE_DAY_IN_SECONDS)
-    except KeyboardInterrupt:
-        server.stop(0)
-
-
-if __name__ == "__main__":
-    serve()

cmd/suggestion/hyperopt/v1alpha3/requirements.txt

@@ -1,10 +0,0 @@
-grpcio==1.23.0
-duecredit===0.7.0
-cloudpickle==0.5.6
-numpy>=1.13.3
-scikit-learn>=0.19.0
-scipy>=0.19.1
-forestci==0.3
-protobuf==3.9.1
-googleapis-common-protos==1.6.0
-hyperopt==0.2.3

cmd/suggestion/hyperopt/v1beta1/Dockerfile

@@ -1,33 +1,24 @@
-FROM python:3.6
+FROM python:3.11-slim
 
+ARG TARGETARCH
 ENV TARGET_DIR /opt/katib
 ENV SUGGESTION_DIR cmd/suggestion/hyperopt/v1beta1
+ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python
 
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
+RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
     apt-get -y update && \
     apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-    pip install cython; \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*; \
     fi
 
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
 ADD ./pkg/ ${TARGET_DIR}/pkg/
 ADD ./${SUGGESTION_DIR}/ ${TARGET_DIR}/${SUGGESTION_DIR}/
-WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
 
+WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
+
+RUN pip install --prefer-binary --no-cache-dir -r requirements.txt
 RUN chgrp -R 0 ${TARGET_DIR} \
   && chmod -R g+rwX ${TARGET_DIR}
 
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python
-
 ENTRYPOINT ["python", "main.py"]
-

cmd/suggestion/hyperopt/v1beta1/main.py

@@ -1,10 +1,26 @@
-import grpc
+# Copyright 2022 The Kubeflow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import time
-from pkg.apis.manager.v1beta1.python import api_pb2_grpc
-from pkg.apis.manager.health.python import health_pb2_grpc
-from pkg.suggestion.v1beta1.hyperopt.service import HyperoptService
 from concurrent import futures
 
+import grpc
+
+from pkg.apis.manager.health.python import health_pb2_grpc
+from pkg.apis.manager.v1beta1.python import api_pb2_grpc
+from pkg.suggestion.v1beta1.hyperopt.service import HyperoptService
+
 _ONE_DAY_IN_SECONDS = 60 * 60 * 24
 DEFAULT_PORT = "0.0.0.0:6789"
 

cmd/suggestion/hyperopt/v1beta1/requirements.txt

@@ -1,10 +1,10 @@
-grpcio==1.23.0
-duecredit===0.7.0
+grpcio>=1.64.1
 cloudpickle==0.5.6
-numpy>=1.13.3
-scikit-learn>=0.19.0
-scipy>=0.19.1
+numpy>=1.25.2
+scikit-learn>=0.24.0
+scipy>=1.5.4
 forestci==0.3
-protobuf==3.9.1
+protobuf>=4.21.12,<5
 googleapis-common-protos==1.6.0
-hyperopt==0.2.3
+hyperopt==0.2.5
+cython>=0.29.24

cmd/suggestion/nas/darts/v1alpha3/Dockerfile

@@ -1,33 +0,0 @@
-FROM python:3.6
-
-ENV TARGET_DIR /opt/katib
-ENV SUGGESTION_DIR cmd/suggestion/nas/darts/v1alpha3
-
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
-    apt-get -y update && \
-    apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-    pip install cython; \
-    fi
-
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
-ADD ./pkg/ ${TARGET_DIR}/pkg/
-ADD ./${SUGGESTION_DIR}/ ${TARGET_DIR}/${SUGGESTION_DIR}/
-WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
-
-RUN chgrp -R 0 ${TARGET_DIR} \
-  && chmod -R g+rwX ${TARGET_DIR}
-
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1alpha3/python:${TARGET_DIR}/pkg/apis/manager/health/python
-
-ENTRYPOINT ["python", "main.py"]
-

cmd/suggestion/nas/darts/v1alpha3/main.py

@@ -1,30 +0,0 @@
-import grpc
-from concurrent import futures
-import time
-from pkg.apis.manager.v1alpha3.python import api_pb2_grpc
-from pkg.apis.manager.health.python import health_pb2_grpc
-from pkg.suggestion.v1alpha3.nas.darts.service import DartsService
-
-
-_ONE_DAY_IN_SECONDS = 60 * 60 * 24
-DEFAULT_PORT = "0.0.0.0:6789"
-
-
-def serve():
-    print("Darts Suggestion Service")
-    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
-    service = DartsService()
-    api_pb2_grpc.add_SuggestionServicer_to_server(service, server)
-    health_pb2_grpc.add_HealthServicer_to_server(service, server)
-    server.add_insecure_port(DEFAULT_PORT)
-    print("Listening...")
-    server.start()
-    try:
-        while True:
-            time.sleep(_ONE_DAY_IN_SECONDS)
-    except KeyboardInterrupt:
-        server.stop(0)
-
-
-if __name__ == "__main__":
-    serve()

cmd/suggestion/nas/darts/v1alpha3/requirements.txt

@@ -1,3 +0,0 @@
-grpcio==1.23.0
-protobuf==3.9.1
-googleapis-common-protos==1.6.0

cmd/suggestion/nas/darts/v1beta1/Dockerfile

@@ -1,33 +1,24 @@
-FROM python:3.6
+FROM python:3.11-slim
 
+ARG TARGETARCH
 ENV TARGET_DIR /opt/katib
 ENV SUGGESTION_DIR cmd/suggestion/nas/darts/v1beta1
+ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python
 
-RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \
+RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
     apt-get -y update && \
     apt-get -y install gfortran libopenblas-dev liblapack-dev && \
-    pip install cython; \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*; \
     fi
 
-RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
-    if [ "$(uname -m)" = "ppc64le" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
-    elif [ "$(uname -m)" = "aarch64" ]; then \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64; \
-    else \
-    wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
-    fi && \
-    chmod +x /bin/grpc_health_probe
-
 ADD ./pkg/ ${TARGET_DIR}/pkg/
 ADD ./${SUGGESTION_DIR}/ ${TARGET_DIR}/${SUGGESTION_DIR}/
-WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
-RUN pip install --no-cache-dir -r requirements.txt
 
+WORKDIR  ${TARGET_DIR}/${SUGGESTION_DIR}
+
+RUN pip install --prefer-binary --no-cache-dir -r requirements.txt
 RUN chgrp -R 0 ${TARGET_DIR} \
   && chmod -R g+rwX ${TARGET_DIR}
 
-ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python
-
 ENTRYPOINT ["python", "main.py"]
-

cmd/suggestion/nas/darts/v1beta1/main.py

@@ -1,10 +1,25 @@
-import grpc
-from concurrent import futures
-import time
-from pkg.apis.manager.v1beta1.python import api_pb2_grpc
-from pkg.apis.manager.health.python import health_pb2_grpc
-from pkg.suggestion.v1beta1.nas.darts.service import DartsService
+# Copyright 2022 The Kubeflow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
 
+import time
+from concurrent import futures
+
+import grpc
+
+from pkg.apis.manager.health.python import health_pb2_grpc
+from pkg.apis.manager.v1beta1.python import api_pb2_grpc
+from pkg.suggestion.v1beta1.nas.darts.service import DartsService
 
 _ONE_DAY_IN_SECONDS = 60 * 60 * 24
 DEFAULT_PORT = "0.0.0.0:6789"

cmd/suggestion/nas/darts/v1beta1/requirements.txt

@@ -1,3 +1,4 @@
-grpcio==1.23.0
-protobuf==3.9.1
+grpcio>=1.64.1
+protobuf>=4.21.12,<5
 googleapis-common-protos==1.6.0
+cython>=0.29.24