mirror of https://github.com/kubeflow/examples.git
767ecd240d
* use gcs client libs to copy checkpoint dir * more minor cleanup, use tagged image, use newer pipeline param spec. syntax. pylint cleanup. added set_memory_limit() to notebook pipeline training steps. modified the pipelines definitions to use the user-defined params as defaults. * put a retry loop around the copy_blob |
||
|---|---|---|
| .. | ||
| Pachyderm_Example | ||
| demo | ||
| docker | ||
| ks_app | ||
| notebooks | ||
| pipelines | ||
| sql | ||
| testing | ||
| workflow | ||
| .gitignore | ||
| 01_setup_a_kubeflow_cluster.md | ||
| 02_distributed_training.md | ||
| 02_training_the_model.md | ||
| 02_training_the_model_tfjob.md | ||
| 03_serving_the_model.md | ||
| 04_querying_the_model.md | ||
| 05_teardown.md | ||
| Makefile | ||
| README.md | ||
| image_build.jsonnet | ||
| requirements.txt | ||
README.md
End-to-End kubeflow tutorial using a Sequence-to-Sequence model
This example demonstrates how you can use kubeflow end-to-end to train and
serve a Sequence-to-Sequence model on an existing kubernetes cluster. This
tutorial is based upon @hamelsmu's article "How To Create Data Products That
Are Magical Using Sequence-to-Sequence
Models".
Goals
There are two primary goals for this tutorial:
- Demonstrate an End-to-End kubeflow example
- Present an End-to-End Sequence-to-Sequence model
By the end of this tutorial, you should learn how to:
- Setup a Kubeflow cluster on an existing Kubernetes deployment
- Spawn a Jupyter Notebook on the cluster
- Spawn a shared-persistent storage across the cluster to store large datasets
- Train a Sequence-to-Sequence model using TensorFlow and GPUs on the cluster
- Serve the model using Seldon Core
- Query the model from a simple front-end application
Steps:
- Setup a Kubeflow cluster
- Training the model. You can train the model using any of the following methods using Jupyter Notebook or using TFJob:
- Serving the model
- Querying the model
- Teardown