Machine Learning Pipelines for Kubeflow
Go to file
Yihong Wang bb9dc67572
feat(frontend): make container name configurable (#9433)
* feat(frontend): make container name configurable

Add an UIConfig param to specify the container name in a pod
where logs are retrieved. The default container name for
the Argo is `main`. However, in Tekton, the main container name
contains the `step-` prefix. Make this configurable to support
other orchestrations.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* fix test case error

add `undefined` as the expected `logContainerName` param

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* address the comment

use default value `main` in `getPodLogsStreamFromK8s`
same as `getPodLogs`

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* only pass podLogContainerName

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

---------

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
2023-05-17 23:37:06 +00:00
.github fix: Renovate json format. Fix #6772 (#8107) 2022-08-04 23:59:14 +00:00
api chore: release kfp-pipeline-spec 0.2.2 (#9120) 2023-04-07 21:24:51 +00:00
backend chore(backend): Add golangci-lint and pre-commit config (#9428) 2023-05-17 18:23:06 +00:00
components chore(components): Update KServe component to 0.10.1 for kubeflow 1.7 support (#9446) 2023-05-17 19:45:06 +00:00
contrib chore(deps): bump jinja2 from 2.10.1 to 2.11.3 in /contrib/components/openvino/ovms-deployer/containers (#5341) 2021-03-19 19:26:16 -07:00
docs docs(sdk): update dropdown for v1 KFP SDK versions (#9445) 2023-05-17 17:11:06 +00:00
frontend feat(frontend): make container name configurable (#9433) 2023-05-17 23:37:06 +00:00
hack chore: use the upstream go-licenses tool (#7408) 2022-03-16 02:10:34 +00:00
kubernetes_platform chore: recompile/update v2 samples to avoid possible runtime breakage (#9333) 2023-05-05 17:28:24 +00:00
manifests chore(release): bumped version to 2.0.0-rc.1 2023-05-12 21:36:20 +00:00
proxy chore: upgrade proxy agent image (#9307) 2023-05-04 05:49:45 +00:00
samples chore(components): Add sample for Sagemaker Model Monitor component (#9405) 2023-05-15 23:22:59 +00:00
sdk chore(sdk): release KFP SDK 2.0.0-rc.1 (#9429) 2023-05-16 18:23:00 +00:00
test chore: upgrade to Argo v3.3.10 (#9336) 2023-05-08 08:38:55 +00:00
third_party chore: Add more licenses for cache-deployer image (#9432) 2023-05-16 17:42:00 +00:00
tools feat(backend): upgrade argo go module to V3. Part of #5718 (#5792) 2021-06-08 05:04:45 -07:00
.cloudbuild.yaml chore: upgrade to Argo v3.3.10 (#9336) 2023-05-08 08:38:55 +00:00
.dockerignore refactor: migrate to api/v2alpha1/go/pipelinespec (#6016) 2021-07-13 00:05:17 -07:00
.gitattributes Support filtering on storage state (#629) 2019-01-11 11:01:01 -08:00
.gitignore chore: add kfp-kubernetes docs and update process infrastructure (#8976) 2023-03-15 18:23:12 +00:00
.golangci.yaml chore(backend): Add golangci-lint and pre-commit config (#9428) 2023-05-17 18:23:06 +00:00
.isort.cfg chore: add the missing `.isort.cfg` file (#8045) 2022-07-19 16:56:48 +00:00
.pre-commit-config.yaml chore(backend): Add golangci-lint and pre-commit config (#9428) 2023-05-17 18:23:06 +00:00
.pylintrc chore(sdk): make kfp v2 hermetic (#7428) 2022-03-18 03:00:40 +00:00
.readthedocs.yml fix(sdk): fixes requirements.txt discovery for sdk api reference docs (#8048) 2022-07-20 22:57:17 +00:00
.release.cloudbuild.yaml chore: upgrade to Argo v3.3.10 (#9336) 2023-05-08 08:38:55 +00:00
.style.yapf chore: consistent yapf style config for the entire repo (#6963) 2021-11-30 20:38:30 +00:00
AUTHORS chore: set up AUTHORS file. Fixes #5470 (#5766) 2021-06-01 02:49:05 -07:00
CHANGELOG.md chore(release): bumped version to 2.0.0-rc.1 2023-05-12 21:36:20 +00:00
CONTRIBUTING.md chore: update contributing guide (#8489) 2023-03-17 23:07:08 +00:00
LICENSE Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00
OWNERS chore: remove Bobgy from OWNERS (#7195) 2022-01-25 08:11:28 +08:00
README.md doc: Update README to reflect switch to Emissary Executor as default from Kubeflow Pipelines 1.8 (#8922) 2023-03-04 06:13:32 +00:00
RELEASE.md chore: Update RELEASE.md (#9138) 2023-04-17 21:13:53 +00:00
ROADMAP.md chore: Update ROADMAP.md (#7752) 2022-05-18 17:48:26 +00:00
SECURITY.md chore(repo): Add SECURITY.md. Fixes #9319. (#9341) 2023-05-05 23:29:52 +00:00
VERSION chore(release): bumped version to 2.0.0-rc.1 2023-05-12 21:36:20 +00:00
developer_guide.md chore(visualization): Revert to tensorflow image because tfx image is too big. Fix #6053 (#6061) 2021-07-15 20:16:38 -07:00
go.mod chore: upgrade to Argo v3.3.10 (#9336) 2023-05-08 08:38:55 +00:00
go.sum chore: upgrade to Argo v3.3.10 (#9336) 2023-05-08 08:38:55 +00:00
mypy.ini chore(sdk): clean up v2 CLI (#7558) 2022-04-20 10:40:38 -06:00
package-lock.json chore(release): set up conventional commit changelog tool. Part of #3920 (#4033) 2020-06-23 03:51:40 -07:00
package.json chore(release): set up conventional commit changelog tool. Part of #3920 (#4033) 2020-06-23 03:51:40 -07:00
release-status-check.png docs(release): update RELEASE.md (#4832) 2020-11-29 23:30:50 -08:00
requirements.txt feat: upgrade TFX to 1.2.0 (#6375) 2021-08-18 01:50:37 -07:00
retry-release-on-tag.png chore(doc): update release doc with caveats on `release-on-tag` retry (#4917) 2021-02-01 02:22:02 -08:00
verify-retry-the-right-build.png chore(doc): update release doc with caveats on `release-on-tag` retry (#4917) 2021-02-01 02:22:02 -08:00

README.md

Coverage Status SDK Documentation Status SDK Package version SDK Supported Python versions

Overview of the Kubeflow pipelines service

Kubeflow is a machine learning (ML) toolkit that is dedicated to making deployments of ML workflows on Kubernetes simple, portable, and scalable.

Kubeflow pipelines are reusable end-to-end ML workflows built using the Kubeflow Pipelines SDK.

The Kubeflow pipelines service has the following goals:

  • End to end orchestration: enabling and simplifying the orchestration of end to end machine learning pipelines
  • Easy experimentation: making it easy for you to try numerous ideas and techniques, and manage your various trials/experiments.
  • Easy re-use: enabling you to re-use components and pipelines to quickly cobble together end to end solutions, without having to re-build each time.

Installation

  • Install Kubeflow Pipelines from choices described in Installation Options for Kubeflow Pipelines.

  • The Docker container runtime has been deprecated on Kubernetes 1.20+. Kubeflow Pipelines has switched to use Emissary Executor by default from Kubeflow Pipelines 1.8. Emissary executor is Container runtime agnostic, meaning you are able to run Kubeflow Pipelines on Kubernetes cluster with any Container runtimes.

Documentation

Get started with your first pipeline and read further information in the Kubeflow Pipelines overview.

See the various ways you can use the Kubeflow Pipelines SDK.

See the Kubeflow Pipelines API doc for API specification.

Consult the Python SDK reference docs when writing pipelines using the Python SDK.

Refer to the versioning policy and feature stages documentation for more information about how we manage versions and feature stages (such as Alpha, Beta, and Stable).

Contributing to Kubeflow Pipelines

Before you start contributing to Kubeflow Pipelines, read the guidelines in How to Contribute. To learn how to build and deploy Kubeflow Pipelines from source code, read the developer guide.

Kubeflow Pipelines Community Meeting

The meeting is happening every other Wed 10-11AM (PST) Calendar Invite or Join Meeting Directly

Meeting notes

Kubeflow Pipelines Slack Channel

#kubeflow-pipelines

Blog posts

Acknowledgments

Kubeflow pipelines uses Argo Workflows by default under the hood to orchestrate Kubernetes resources. The Argo community has been very supportive and we are very grateful. Additionally there is Tekton backend available as well. To access it, please refer to Kubeflow Pipelines with Tekton repository.