Machine Learning Pipelines for Kubeflow
Go to file
Joe Li a45d72d75d
chore(frontend): Generate TypeScript v2 API client (#8834)
* Add v2apis script in package.json

* Generate typescript v2 api client.

* Change api client codegen section in README
Make it more clear: put the jar file in frontend folder.

* Add v2 api codegen script in README.

* Remove beta1 text to avoid confusion in README.

* Format

* Rename folder as apisv2beta1.
Add apisv2beta1 in .eslintrc.yaml file

* Change script from v2apis to apis:v2beta1
Remove unnecessary folder v2generated (move files to
generated/apisv2beta1/

* Add the command for exporting jar file path to $PATH in README file.
2023-02-10 22:22:29 +00:00
.github fix: Renovate json format. Fix #6772 (#8107) 2022-08-04 23:59:14 +00:00
api chore(sdk): bump kfp-pipeline-spec (#8744) 2023-01-20 12:41:16 -08:00
backend Update API references and readme. Closes #8723 (#8828) 2023-02-09 23:41:01 +00:00
components feat(component): Create component definition for GetVertexDataset 2023-02-09 16:06:11 -08:00
contrib chore(deps): bump jinja2 from 2.10.1 to 2.11.3 in /contrib/components/openvino/ovms-deployer/containers (#5341) 2021-03-19 19:26:16 -07:00
docs chore(sdk): release KFP SDK 2.0.0b12 (#8823) 2023-02-08 09:46:17 -08:00
frontend chore(frontend): Generate TypeScript v2 API client (#8834) 2023-02-10 22:22:29 +00:00
hack chore: use the upstream go-licenses tool (#7408) 2022-03-16 02:10:34 +00:00
manifests chore(release): bumped version to 2.0.0-beta.0 2023-02-07 23:41:29 +00:00
proxy fix: Update proxy agent image to fix CVE-2022-1292 (#8019) 2022-07-13 09:38:02 +00:00
samples chore(components): Update aws owners files to reflect current owners (#8622) 2022-12-22 22:29:14 +00:00
sdk feat(sdk): deprecate .set_gpu_limit in favor of .set_accelerator_limit (#8836) 2023-02-09 15:46:10 -08:00
test test: install compatible `kfp-server-api` and `kfp` packages (#8829) 2023-02-09 20:30:01 +00:00
third_party feat: Upgrade argo-workflow to v3.3.8 (#8009) 2022-07-12 19:22:31 +00:00
tools feat(backend): upgrade argo go module to V3. Part of #5718 (#5792) 2021-06-08 05:04:45 -07:00
.cloudbuild.yaml fix(backend): Upgrade mysql to 8.0.26 (#8351) 2022-10-14 07:34:04 +00:00
.dockerignore refactor: migrate to api/v2alpha1/go/pipelinespec (#6016) 2021-07-13 00:05:17 -07:00
.gitattributes Support filtering on storage state (#629) 2019-01-11 11:01:01 -08:00
.gitignore feat(components): Adds Notebooks Executor API in the experimental components (#6630) 2021-10-01 09:11:25 -07:00
.isort.cfg chore: add the missing `.isort.cfg` file (#8045) 2022-07-19 16:56:48 +00:00
.pre-commit-config.yaml chore(sdk): fix isort version in pre-commit (#8809) 2023-02-06 15:13:59 -08:00
.pylintrc chore(sdk): make kfp v2 hermetic (#7428) 2022-03-18 03:00:40 +00:00
.readthedocs.yml fix(sdk): fixes requirements.txt discovery for sdk api reference docs (#8048) 2022-07-20 22:57:17 +00:00
.release.cloudbuild.yaml fix(backend): Upgrade mysql to 8.0.26 (#8351) 2022-10-14 07:34:04 +00:00
.style.yapf chore: consistent yapf style config for the entire repo (#6963) 2021-11-30 20:38:30 +00:00
AUTHORS chore: set up AUTHORS file. Fixes #5470 (#5766) 2021-06-01 02:49:05 -07:00
CHANGELOG.md chore(release): bumped version to 2.0.0-beta.0 2023-02-07 23:41:29 +00:00
CONTRIBUTING.md chore: update contributing guideline (#8482) 2022-11-21 22:52:41 +00:00
LICENSE Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00
OWNERS chore: remove Bobgy from OWNERS (#7195) 2022-01-25 08:11:28 +08:00
README.md chore: fix broken link in README.md (#7284) 2022-02-10 10:55:15 +00:00
RELEASE.md chore: Update RELEASE.md (#8028) 2022-07-13 19:59:03 +00:00
ROADMAP.md chore: Update ROADMAP.md (#7752) 2022-05-18 17:48:26 +00:00
VERSION chore(release): bumped version to 2.0.0-beta.0 2023-02-07 23:41:29 +00:00
developer_guide.md chore(visualization): Revert to tensorflow image because tfx image is too big. Fix #6053 (#6061) 2021-07-15 20:16:38 -07:00
go.mod feat(backend): Refactor apiserver, add Pipeline and other v2beta services (#8633) 2023-02-03 20:17:01 +00:00
go.sum feat(backend): Refactor apiserver, add Pipeline and other v2beta services (#8633) 2023-02-03 20:17:01 +00:00
mypy.ini chore(sdk): clean up v2 CLI (#7558) 2022-04-20 10:40:38 -06:00
package-lock.json chore(release): set up conventional commit changelog tool. Part of #3920 (#4033) 2020-06-23 03:51:40 -07:00
package.json chore(release): set up conventional commit changelog tool. Part of #3920 (#4033) 2020-06-23 03:51:40 -07:00
release-status-check.png docs(release): update RELEASE.md (#4832) 2020-11-29 23:30:50 -08:00
requirements.txt feat: upgrade TFX to 1.2.0 (#6375) 2021-08-18 01:50:37 -07:00
retry-release-on-tag.png chore(doc): update release doc with caveats on `release-on-tag` retry (#4917) 2021-02-01 02:22:02 -08:00
verify-retry-the-right-build.png chore(doc): update release doc with caveats on `release-on-tag` retry (#4917) 2021-02-01 02:22:02 -08:00

README.md

Coverage Status SDK Documentation Status SDK Package version SDK Supported Python versions

Overview of the Kubeflow pipelines service

Kubeflow is a machine learning (ML) toolkit that is dedicated to making deployments of ML workflows on Kubernetes simple, portable, and scalable.

Kubeflow pipelines are reusable end-to-end ML workflows built using the Kubeflow Pipelines SDK.

The Kubeflow pipelines service has the following goals:

  • End to end orchestration: enabling and simplifying the orchestration of end to end machine learning pipelines
  • Easy experimentation: making it easy for you to try numerous ideas and techniques, and manage your various trials/experiments.
  • Easy re-use: enabling you to re-use components and pipelines to quickly cobble together end to end solutions, without having to re-build each time.

Installation

  • Install Kubeflow Pipelines from choices described in Installation Options for Kubeflow Pipelines.

  • [Alpha] Starting from Kubeflow Pipelines 1.7, try out Emissary Executor. Emissary executor is Container runtime agnostic meaning you are able to run Kubeflow Pipelines on Kubernetes cluster with any Container runtimes. The default Docker executor depends on Docker container runtime, which will be deprecated on Kubernetes 1.20+.

Documentation

Get started with your first pipeline and read further information in the Kubeflow Pipelines overview.

See the various ways you can use the Kubeflow Pipelines SDK.

See the Kubeflow Pipelines API doc for API specification.

Consult the Python SDK reference docs when writing pipelines using the Python SDK.

Refer to the versioning policy and feature stages documentation for more information about how we manage versions and feature stages (such as Alpha, Beta, and Stable).

Contributing to Kubeflow Pipelines

Before you start contributing to Kubeflow Pipelines, read the guidelines in How to Contribute. To learn how to build and deploy Kubeflow Pipelines from source code, read the developer guide.

Kubeflow Pipelines Community Meeting

The meeting is happening every other Wed 10-11AM (PST) Calendar Invite or Join Meeting Directly

Meeting notes

Kubeflow Pipelines Slack Channel

#kubeflow-pipelines

Blog posts

Acknowledgments

Kubeflow pipelines uses Argo Workflows by default under the hood to orchestrate Kubernetes resources. The Argo community has been very supportive and we are very grateful. Additionally there is Tekton backend available as well. To access it, please refer to Kubeflow Pipelines with Tekton repository.