f2a860b84c
* # This is a combination of 5 commits. # This is the 1st commit message: Add initial scripts # This is the commit message #2: Add working pytest script # This is the commit message #3: Add initial scripts # This is the commit message #4: Add environment variable files # This is the commit message #5: Remove old cluster script * Add initial scripts Add working pytest script Add initial scripts Add environment variable files Remove old cluster script Update pipeline credentials to OIDC Add initial scripts Add working pytest script Add initial scripts Add working pytest script * Remove debugging mark * Update example EKS cluster name * Remove quiet from Docker build * Manually pass env * Update env list vars as string * Update use array directly * Update variable array to export * Update to using read for splitting * Move to helper script * Update export from CodeBuild * Add wait for minio * Update kubectl wait timeout * Update minor changes for PR * Update integration test buildspec to quiet build * Add region to delete EKS * Add wait for pods * Updated README * Add fixed interval wait * Fix CodeBuild step order * Add file lock for experiment ID * Fix missing pytest parameter * Update run create only once * Add filelock to conda env * Update experiment name ensuring creation each time * Add try/catch with create experiment * Remove caching from KFP deployment * Remove disable KFP caching * Move .gitignore changes to inside component * Add blank line to default .gitignore |
||
|---|---|---|
| .github/ISSUE_TEMPLATE | ||
| backend | ||
| components | ||
| contrib | ||
| docs | ||
| frontend | ||
| manifests | ||
| proxy | ||
| release | ||
| samples | ||
| sdk | ||
| test | ||
| third_party | ||
| tools | ||
| .cloudbuild.yaml | ||
| .dockerignore | ||
| .gitattributes | ||
| .gitignore | ||
| .pylintrc | ||
| .release.cloudbuild.yaml | ||
| .style.yapf | ||
| .travis.yml | ||
| BUILD.bazel | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| Makefile | ||
| OWNERS | ||
| README.md | ||
| ROADMAP.md | ||
| VERSION | ||
| WORKSPACE | ||
| developer_guide.md | ||
| go.mod | ||
| go.sum | ||
README.md
SDK:
Overview of the Kubeflow pipelines service
Kubeflow is a machine learning (ML) toolkit that is dedicated to making deployments of ML workflows on Kubernetes simple, portable, and scalable.
Kubeflow pipelines are reusable end-to-end ML workflows built using the Kubeflow Pipelines SDK.
The Kubeflow pipelines service has the following goals:
- End to end orchestration: enabling and simplifying the orchestration of end to end machine learning pipelines
- Easy experimentation: making it easy for you to try numerous ideas and techniques, and manage your various trials/experiments.
- Easy re-use: enabling you to re-use components and pipelines to quickly cobble together end to end solutions, without having to re-build each time.
Documentation
Get started with your first pipeline and read further information in the Kubeflow Pipelines overview.
See the various ways you can use the Kubeflow Pipelines SDK.
See the Kubeflow Pipelines API doc for API specification.
Consult the Python SDK reference docs when writing pipelines using the Python SDK.
Kubeflow Pipelines Community Meeting
The meeting is happening every other Wed 10-11AM (PST) Calendar Invite or Join Meeting Directly
Kubeflow Pipelines Slack Channel
Blog posts
- Getting started with Kubeflow Pipelines (By Amy Unruh)
- How to create and deploy a Kubeflow Machine Learning Pipeline (By Lak Lakshmanan)
Acknowledgments
Kubeflow pipelines uses Argo under the hood to orchestrate Kubernetes resources. The Argo community has been very supportive and we are very grateful.