mirror of https://github.com/kubeflow/website.git
update `Kubeflow Notebooks` docs (#3003)
This commit is contained in:
Mathew Wicks
GitHub
parent
dbf5350c18
commit
5ad6019a72
|
|
@ -161,6 +161,12 @@ docs/started/requirements/ /docs/started/getting-started/
|
|||
/docs/reference/pytorchjob/* /docs/components/training
|
||||
/docs/reference/tfjob/* /docs/components/training
|
||||
|
||||
# Cleanup of Notebooks Docs
|
||||
/docs/components/notebooks/custom-notebook /docs/components/notebooks/container-images
|
||||
/docs/components/notebooks/setup /docs/components/notebooks/quickstart-guide
|
||||
/docs/components/notebooks/troubleshoot /docs/components/notebooks/troubleshooting
|
||||
/docs/components/notebooks/why-use-jupyter-notebook /docs/components/notebooks/overview
|
||||
|
||||
# ===============
|
||||
# IMPORTANT NOTE:
|
||||
# Catch-all redirects should be added at the end of this file as redirects happen from top to bottom
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
+++
|
||||
title = "Notebook Servers"
|
||||
description = "Using Jupyter notebooks in Kubeflow"
|
||||
title = "Kubeflow Notebooks"
|
||||
description = "Documentation for Kubeflow Notebooks"
|
||||
weight = 10
|
||||
+++
|
||||
|
|
|
|||
|
|
@ -0,0 +1,75 @@
|
|||
+++
|
||||
title = "Container Images"
|
||||
description = "About Container Images for Kubeflow Notebooks"
|
||||
weight = 30
|
||||
|
||||
+++
|
||||
Kubeflow Notebooks natively supports three types of notebooks, [JupyterLab](https://github.com/jupyterlab/jupyterlab), [RStudio](https://github.com/rstudio/rstudio), and [Visual Studio Code (code-server)](https://github.com/cdr/code-server), but any web-based IDE should work.
|
||||
Notebook servers run as containers inside a Kubernetes Pod, which means the type of IDE (and which packages are installed) is determined by the Docker image you pick for your server.
|
||||
|
||||
## Images
|
||||
|
||||
We provide a number of [example container images](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers) to get you started.
|
||||
|
||||
### Base Images
|
||||
|
||||
These images provide a common starting point for Kubeflow Notebook containers.
|
||||
See [custom images](#custom-images) to learn how to extend them with your own packages.
|
||||
|
||||
Dockerfile | Registry | Notes
|
||||
--- | --- | ---
|
||||
[base](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/base) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/base:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/base) | common base image
|
||||
[codeserver](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/codeserver) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver) | base [code-server](https://github.com/cdr/code-server) (Visual Studio Code) image
|
||||
[jupyter](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter) | base [JupyterLab](https://github.com/jupyterlab/jupyterlab) image
|
||||
[rstudio](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/rstudio) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio) | base [RStudio](https://github.com/rstudio/rstudio) image
|
||||
|
||||
### Full Images
|
||||
|
||||
These images extend the [base images](#base-images) with common packages used by Data Scientists and ML Engineers.
|
||||
|
||||
Dockerfile | Registry | Notes
|
||||
--- | --- | ---
|
||||
[codeserver-python](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/codeserver-python) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python) | code-server (Visual Studio Code) + Conda Python
|
||||
[jupyter-pytorch (CPU)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch) | JupyterLab + PyTorch (CPU)
|
||||
[jupyter-pytorch (CUDA)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda) | JupyterLab + PyTorch (CUDA)
|
||||
[jupyter-pytorch-full (CPU)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch-full) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full) | JupyterLab + PyTorch (CPU) + [common](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch-full/requirements.txt) packages
|
||||
[jupyter-pytorch-full (CUDA)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch-full) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full) | JupyterLab + PyTorch (CUDA) + [common](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch-full/requirements.txt) packages
|
||||
[jupyter-scipy](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-scipy) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy) | JupyterLab + [SciPy](https://www.scipy.org/) packages
|
||||
[jupyter-tensorflow (CPU)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow) | JupyterLab + TensorFlow (CPU)
|
||||
[jupyter-tensorflow (CUDA)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda) | JupyterLab + TensorFlow (CUDA)
|
||||
[jupyter-tensorflow-full (CPU)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow-full) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full) | JupyterLab + TensorFlow (CPU) + [common](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow-full/requirements.txt) packages
|
||||
[jupyter-tensorflow-full (CUDA)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow-full) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full) | JupyterLab + TensorFlow (CUDA) + [common](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow-full/requirements.txt) packages
|
||||
[rstudio-tidyverse](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/rstudio-tidyverse) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse) | RStudio + [Tidyverse](https://www.tidyverse.org/) packages
|
||||
|
||||
### Image Dependency Chart
|
||||
|
||||
This flow-chart shows how our notebook container images depend on each other.
|
||||
|
||||
<img src="/docs/images/notebook-container-image-chart.png"
|
||||
alt="A flow-chart showing how notebook container images depend on each other"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
## Custom Images
|
||||
|
||||
Packages installed by users __after spawning__ a Kubeflow Notebook will only last the lifetime of the pod (unless installed into a PVC-backed directory).
|
||||
|
||||
To ensure packages are preserved throughout Pod restarts users will need to either:
|
||||
1. [Build custom images that include them](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers#custom-images), or
|
||||
2. Ensure they are installed in a PVC-backed directory
|
||||
|
||||
### Image Requirements
|
||||
|
||||
For Kubeflow Notebooks to work with a container image, the image must:
|
||||
- expose an HTTP interface on port `8888`:
|
||||
- kubeflow sets an environment variable `NB_PREFIX` at runtime with the URL path we expect the container be listening under
|
||||
- kubeflow uses IFrames, so ensure your application sets `Access-Control-Allow-Origin: *` in HTTP response headers
|
||||
- run as a user called `jovyan`:
|
||||
- the home directory of `jovyan` should be `/home/jovyan`
|
||||
- the UID of `jovyan` should be `1000`
|
||||
- start successfully with an empty PVC mounted at `/home/jovyan`:
|
||||
- kubeflow mounts a PVC at `/home/jovyan` to keep state across Pod restarts
|
||||
|
||||
## Next steps
|
||||
|
||||
- Use your container image by specifying it when spawning your notebook server.
|
||||
(See the [quickstart guide](/docs/components/notebooks/quickstart-guide/).)
|
||||
|
|
@ -1,85 +0,0 @@
|
|||
+++
|
||||
title = "Create a Custom Jupyter Image"
|
||||
description = "Creating a custom Docker image for your Jupyter notebook"
|
||||
weight = 30
|
||||
|
||||
+++
|
||||
This guide tells you how to configure a custom Docker image for your Jupyter
|
||||
notebook server in Kubeflow.
|
||||
|
||||
Your custom image must meet the requirements of the Kubeflow notebook
|
||||
controller which manages the life cycle of notebooks. The Kubeflow UI expects
|
||||
Jupyter to start after launching the Docker image with `docker run`. You must
|
||||
therefore set the default command of your Docker image to launch Jupyter.
|
||||
|
||||
Follow these steps to configure the launch command (`CMD`) in your Docker image:
|
||||
|
||||
* Set the working directory:
|
||||
|
||||
```
|
||||
--notebook-dir=/home/jovyan
|
||||
```
|
||||
|
||||
The `/home/jovyan` directory is backed by a
|
||||
[Kubernetes persistent volume (PV)](https://kubernetes.io/docs/concepts/storage/persistent-volumes/).
|
||||
|
||||
* Allow Jupyter to listen on all IP addresses:
|
||||
|
||||
```
|
||||
--ip=0.0.0.0
|
||||
```
|
||||
|
||||
* Allow the user to run the notebook as root:
|
||||
|
||||
```
|
||||
--allow-root
|
||||
```
|
||||
|
||||
* Set the port:
|
||||
|
||||
```
|
||||
--port=8888
|
||||
```
|
||||
|
||||
* Disable authentication. (Kubeflow takes care of authentication based on
|
||||
the type of authentication selected during deployment of Kubeflow. After
|
||||
authentication to Kubeflow, users can access all Kubeflow components
|
||||
from the UI, including notebooks.)
|
||||
|
||||
Use the following setting to allow passwordless access to your Jupyter
|
||||
notebook servers:
|
||||
|
||||
```
|
||||
--NotebookApp.token='' --NotebookApp.password=''
|
||||
```
|
||||
|
||||
* Allow any origin to access your Jupyter notebook server:
|
||||
|
||||
|
||||
```
|
||||
--NotebookApp.allow_origin='*'
|
||||
```
|
||||
|
||||
* Set the base URL. The Kubeflow notebook controller manages the base URL for
|
||||
the notebook server using the environment variable called `NB_PREFIX`. Your
|
||||
Docker image should define the variable and set the value of `base_url` as
|
||||
follows:
|
||||
|
||||
```
|
||||
--NotebookApp.base_url=NB_PREFIX
|
||||
```
|
||||
|
||||
Below is an example of what your Dockerfile should contain:
|
||||
|
||||
|
||||
```
|
||||
ENV NB_PREFIX /
|
||||
|
||||
CMD ["sh","-c", "jupyter notebook --notebook-dir=/home/jovyan --ip=0.0.0.0 --no-browser --allow-root --port=8888 --NotebookApp.token='' --NotebookApp.password='' --NotebookApp.allow_origin='*' --NotebookApp.base_url=${NB_PREFIX}"]
|
||||
```
|
||||
|
||||
## Next steps
|
||||
|
||||
When starting a Jupyter notebook server from the
|
||||
Kubeflow UI, specify your custom Docker image. See the guide to [setting up
|
||||
your Jupyter notebooks](/docs/components/notebooks/setup/).
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
+++
|
||||
title = "Jupyter TensorFlow Examples"
|
||||
description = "Examples using Jupyter and TensorFlow in Kubeflow Notebooks"
|
||||
weight = 40
|
||||
|
||||
+++
|
||||
|
||||
## Mnist Example
|
||||
|
||||
(adapted from [tensorflow/tensorflow - mnist_softmax.py](https://github.com/tensorflow/tensorflow/blob/r1.4/tensorflow/examples/tutorials/mnist/mnist_softmax.py))
|
||||
|
||||
1. When creating your notebook server choose a [container image](/docs/components/notebooks/container-images/) which has Jupyter and TensorFlow installed.
|
||||
|
||||
2. Use Jupyter's interface to create a new **Python 3** notebook.
|
||||
|
||||
3. Copy the following code and paste it into your notebook:
|
||||
|
||||
```python
|
||||
from tensorflow.examples.tutorials.mnist import input_data
|
||||
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
|
||||
|
||||
import tensorflow as tf
|
||||
|
||||
x = tf.placeholder(tf.float32, [None, 784])
|
||||
|
||||
W = tf.Variable(tf.zeros([784, 10]))
|
||||
b = tf.Variable(tf.zeros([10]))
|
||||
|
||||
y = tf.nn.softmax(tf.matmul(x, W) + b)
|
||||
|
||||
y_ = tf.placeholder(tf.float32, [None, 10])
|
||||
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
|
||||
|
||||
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(cross_entropy)
|
||||
|
||||
sess = tf.InteractiveSession()
|
||||
tf.global_variables_initializer().run()
|
||||
|
||||
for _ in range(1000):
|
||||
batch_xs, batch_ys = mnist.train.next_batch(100)
|
||||
sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
|
||||
|
||||
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
|
||||
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
|
||||
print("Accuracy: ", sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))
|
||||
```
|
||||
|
||||
4. Run the code. You should see a number of `WARNING` messages from TensorFlow, followed by a line showing a training accuracy something like this:
|
||||
|
||||
```
|
||||
Accuracy: 0.9012
|
||||
```
|
||||
|
||||
## Next steps
|
||||
|
||||
- See a [simple example](https://github.com/kubeflow/examples/tree/master/pipelines/simple-notebook-pipeline) of creating Kubeflow pipelines in a Jupyter notebook.
|
||||
- Build machine-learning pipelines with the [Kubeflow Pipelines SDK](/docs/components/pipelines/sdk/sdk-overview/).
|
||||
- Learn the advanced features available from a Kubeflow notebook, such as [submitting Kubernetes resources](/docs/components/notebooks/submit-kubernetes/) or [building Docker images](/docs/components/notebooks/custom-notebook/).
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
+++
|
||||
title = "Overview"
|
||||
description = "An overview of Kubeflow Notebooks"
|
||||
weight = 5
|
||||
|
||||
+++
|
||||
{{% stable-status %}}
|
||||
|
||||
## What is Kubeflow Notebooks?
|
||||
|
||||
Kubeflow Notebooks provides a way to run web-based development environments inside your Kubernetes cluster by running them inside Pods.
|
||||
|
||||
Some key features include:
|
||||
- Native support for [JupyterLab](https://github.com/jupyterlab/jupyterlab), [RStudio](https://github.com/jupyterlab/jupyterlab), and [Visual Studio Code (code-server)](https://github.com/cdr/code-server).
|
||||
- Users can create notebook containers directly in the cluster, rather than locally on their workstations.
|
||||
- Admins can provide standard notebook images for their organization with required packages pre-installed.
|
||||
- Access control is managed by Kubeflow's RBAC, enabling easier notebook sharing across the organization.
|
||||
|
||||
## Next steps
|
||||
|
||||
- Get started with Kubeflow Notebooks using the [quickstart guide](/docs/components/notebooks/quickstart-guide/).
|
||||
- Learn how to create your own [container images](/docs/components/notebooks/container-images/).
|
||||
|
|
@ -0,0 +1,113 @@
|
|||
+++
|
||||
title = "Quickstart Guide"
|
||||
description = "Getting started with Kubeflow Notebooks"
|
||||
weight = 10
|
||||
|
||||
+++
|
||||
|
||||
## Summary
|
||||
|
||||
1. Install Kubeflow by following [Getting Started - Installing Kubeflow](/docs/started/installing-kubeflow/).
|
||||
2. Open the Kubeflow [Central Dashboard](/docs/components/central-dash/) in your browser.
|
||||
3. Click __"Notebooks"__ in the left-hand panel.
|
||||
4. Click __"New Server"__ to create a new notebook server.
|
||||
5. Specify the configs for your notebook server.
|
||||
6. Click __"CONNECT"__ once the notebook has been provisioned
|
||||
|
||||
## Detailed Steps
|
||||
|
||||
1. Open the Kubeflow [Central Dashboard](/docs/components/central-dash/) in your browser.
|
||||
|
||||
2. Select a Namespace:
|
||||
- Click the namespace dropdown to see the list of available namespaces.
|
||||
- Choose the namespace that corresponds to your Kubeflow Profile.
|
||||
(See the page on [multi-user isolation](/docs/components/multi-tenancy/) for more information about Profiles.)
|
||||
|
||||
<img src="/docs/images/notebooks-namespace.png"
|
||||
alt="Selecting a Kubeflow namespace"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
3. Click __"Notebook Servers"__ in the left-hand panel:
|
||||
|
||||
<img src="/docs/images/jupyterlink.png"
|
||||
alt="Opening notebooks from the Kubeflow UI"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
4. Click __"New Server"__ on the __"Notebook Servers"__ page:
|
||||
|
||||
<img src="/docs/images/add-notebook-server.png"
|
||||
alt="The Kubeflow notebook servers page"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
5. Enter a __"Name"__ for your notebook server.
|
||||
- The name can include letters and numbers, but no spaces.
|
||||
- For example, `my-first-notebook`.
|
||||
|
||||
<img src="/docs/images/new-notebook-server.png"
|
||||
alt="Form for adding a Kubeflow notebook server"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
6. Select a Docker __"Image"__ for your notebook server
|
||||
- __Custom image__: If you select the custom option, you must specify a Docker image in the form `registry/image:tag`.
|
||||
(See the guide on [container images](/docs/components/notebooks/container-images/).)
|
||||
- __Standard image__: Click the __"Image"__ dropdown menu to see the list of available images.
|
||||
(You can choose from the list configured by your Kubeflow administrator)
|
||||
|
||||
7. Specify the amount of __"CPU"__ that your notebook server will request.
|
||||
|
||||
8. Specify the amount of __"RAM"__ that your notebook server will request.
|
||||
|
||||
9. Specify a __"workspace volume"__ to be mounted as a PVC Volume on your home folder.
|
||||
|
||||
10. *(Optional)* Specify one or more __"data volumes"__ to be mounted as a PVC Volumes.
|
||||
|
||||
11. *(Optional)* Specify one or more additional __"configurations"__
|
||||
- These correspond to [PodDefault resources](https://github.com/kubeflow/kubeflow/blob/master/components/admission-webhook/README.md) which exit in your profile namespace.
|
||||
- Kubeflow matches the labels in the __"configurations"__ field against the properties specified in the PodDefault manifest.
|
||||
- For example, select the label `add-gcp-secret` in the __"configurations"__ field to match to a PodDefault manifest containing the following configuration:
|
||||
```yaml
|
||||
apiVersion: kubeflow.org/v1alpha1
|
||||
kind: PodDefault
|
||||
metadata:
|
||||
name: add-gcp-secret
|
||||
namespace: MY_PROFILE_NAMESPACE
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
add-gcp-secret: "true"
|
||||
desc: "add gcp credential"
|
||||
volumeMounts:
|
||||
- name: secret-volume
|
||||
mountPath: /secret/gcp
|
||||
volumes:
|
||||
- name: secret-volume
|
||||
secret:
|
||||
secretName: gcp-secret
|
||||
```
|
||||
|
||||
12. *(Optional)* Specify any __"GPUs"__ that your notebook server will request.
|
||||
- Kubeflow uses "limits" in Pod requests to provision GPUs onto the notebook Pods
|
||||
(Details about scheduling GPUs can be found in the [Kubernetes Documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/).)
|
||||
|
||||
13. *(Optional)* Specify the setting for __"enable shared memory"__.
|
||||
- Some libraries like PyTorch use shared memory for multiprocessing.
|
||||
- Currently, there is no implementation in Kubernetes to activate shared memory.
|
||||
- As a workaround, Kubeflow mounts an empty directory volume at `/dev/shm`.
|
||||
|
||||
14. Click __"LAUNCH"__ to create a new Notebook CRD with your specified settings.
|
||||
- You should see an entry for your new notebook server on the __"Notebook Servers"__ page
|
||||
- There should be a spinning indicator in the __"Status"__ column.
|
||||
- It can take a few minutes for kubernetes to provision the notebook server pod.
|
||||
- You can check the status of your Pod by hovering your mouse cursor over the icon in the __"Status"__ column.
|
||||
|
||||
15. Click __"CONNECT"__ to view the web interface exposed by your notebook server.
|
||||
|
||||
<img src="/docs/images/notebook-servers.png"
|
||||
alt="Opening notebooks from the Kubeflow UI"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
## Next steps
|
||||
|
||||
- Learn how to create your own [container images](/docs/components/notebooks/container-images/).
|
||||
- Review examples of using [jupyter and tensorflow](/docs/components/notebooks/jupyter-tensorflow-examples/).
|
||||
- Visit the [troubleshooting guide](/docs/components/notebooks/troubleshooting) to fix common errors.
|
||||
|
|
@ -1,311 +0,0 @@
|
|||
+++
|
||||
title = "Set Up Your Notebooks"
|
||||
description = "Getting started with Jupyter notebooks on Kubeflow"
|
||||
weight = 10
|
||||
|
||||
+++
|
||||
Your Kubeflow deployment includes services for spawning and managing Jupyter
|
||||
notebooks.
|
||||
|
||||
You can set up multiple *notebook servers* per Kubeflow deployment. Each
|
||||
notebook server can include multiple *notebooks*. Each notebook server belongs
|
||||
to a single *namespace*, which corresponds to the project group or team for that
|
||||
server.
|
||||
|
||||
This guide shows you how to set up a notebook server for your Jupyter notebooks
|
||||
in Kubeflow.
|
||||
|
||||
## Quick guide
|
||||
|
||||
Summary of steps:
|
||||
|
||||
1. Follow the [Kubeflow getting-started guide](/docs/started/getting-started/)
|
||||
to set up your Kubeflow deployment and open the Kubeflow UI.
|
||||
|
||||
1. Click **Notebook Servers** in the left-hand panel of the Kubeflow UI.
|
||||
1. Choose the **namespace** corresponding to your Kubeflow profile.
|
||||
1. Click **NEW SERVER** to create a notebook server.
|
||||
1. When the notebook server provisioning is complete, click **CONNECT**.
|
||||
1. Click **Upload** to upload an existing notebook, or click **New** to
|
||||
create an empty notebook.
|
||||
|
||||
The rest of this page contains details of the above steps.
|
||||
|
||||
## Install Kubeflow and open the Kubeflow UI
|
||||
|
||||
Follow the [Kubeflow getting-started guide](/docs/started/getting-started/) to
|
||||
set up your Kubeflow deployment in your environment of choice (locally, on
|
||||
premises, or in the cloud).
|
||||
|
||||
When Kubeflow is running, you can access the Kubeflow user interface (UI). If
|
||||
the getting-started guide for your chosen environment has instructions on
|
||||
accessing the UI, follow those instructions. Alternatively, see the generic
|
||||
guide to [accessing the Kubeflow
|
||||
central dashboard](/docs/components/central-dash/overview/).
|
||||
|
||||
## Create a Jupyter notebook server and add a notebook
|
||||
|
||||
1. Click **Notebook Servers** in the left-hand panel of the Kubeflow UI to
|
||||
access the Jupyter notebook services deployed with Kubeflow:
|
||||
<img src="/docs/images/jupyterlink.png"
|
||||
alt="Opening notebooks from the Kubeflow UI"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
1. Sign in:
|
||||
* On GCP, sign in using your Google Account. (If you have already logged in
|
||||
to your Google Account you may not need to log in again.)
|
||||
* On all other platforms, sign in using any username and password.
|
||||
|
||||
1. Select a namespace:
|
||||
* Click the namespace dropdown to see the list of available namespaces.
|
||||
* Choose the namespace that corresponds to your Kubeflow profile. (See
|
||||
the page on [multi-user isolation](/docs/components/multi-tenancy/)
|
||||
for more information about namespaces.)
|
||||
|
||||
<img src="/docs/images/notebooks-namespace.png"
|
||||
alt="Selecting a Kubeflow namespace"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
1. Click **NEW SERVER** on the **Notebook Servers** page:
|
||||
|
||||
<img src="/docs/images/add-notebook-server.png"
|
||||
alt="The Kubeflow notebook servers page"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
You should see a page for entering details of your new server. Here is a
|
||||
partial screenshot of the page:
|
||||
|
||||
<img src="/docs/images/new-notebook-server.png"
|
||||
alt="Form for adding a Kubeflow notebook server"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
1. Enter a **name** of your choice for the notebook server. The name can
|
||||
include letters and numbers, but no spaces. For example, `my-first-notebook`.
|
||||
1. Kubeflow automatically updates the value in the **namespace** field to
|
||||
be the same as the namespace that you selected in a previous step. This
|
||||
ensures that the new notebook server is in a namespace that you can access.
|
||||
|
||||
1. <a id="docker-image"></a> Select a Docker **image** for the baseline
|
||||
deployment of your notebook server. You can specify a custom image or
|
||||
choose from a range of standard images:
|
||||
|
||||
* **Custom image**: If you select the custom option, you must specify a Docker
|
||||
image in the form `registry/image:tag`. For guidelines on creating a Docker
|
||||
image for your notebook, see the guide to
|
||||
[creating a custom Jupyter image](/docs/components/notebooks/custom-notebook/).
|
||||
|
||||
* **Standard image**: Click the **Image** dropdown menu to see the list of
|
||||
available images. The standard Docker images include typical machine learning
|
||||
(ML) packages that you can use within your Jupyter notebooks on
|
||||
this notebook server. Click one of the images to select it.
|
||||
|
||||
The image names indicate the following choices:
|
||||
|
||||
* A TensorFlow version (for example, `tensorflow-1.15.2`).
|
||||
* `cpu` or `gpu`, depending on whether you want to train your model on a CPU
|
||||
or a GPU.
|
||||
|
||||
* If you choose a GPU image, make sure that you have GPUs
|
||||
available in your Kubeflow cluster. Run the following command to check
|
||||
if there are any GPUs available:
|
||||
```
|
||||
kubectl get nodes "-o=custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu"
|
||||
```
|
||||
* If you have GPUs available, you can schedule your server on a GPU
|
||||
node by specifying a number of GPUs to be attached to the server
|
||||
under the **GPUs** section at the bottom of the form.
|
||||
* Kubeflow version (for example, `1.0.0`).
|
||||
|
||||
*Hint:* If you're not sure which image you need, choose a *standard* image
|
||||
running TensorFlow on a CPU. You need a TensorFlow image to run the example
|
||||
code in the [experiment section](#experiment) below.
|
||||
|
||||
1. Specify the total amount of **CPU** that your notebook server should reserve.
|
||||
The default is `0.5`. For CPU-intensive jobs, you can choose more than one CPU
|
||||
(for example, `1.5`).
|
||||
|
||||
1. Specify the total amount of **memory** (RAM) that your notebook server should
|
||||
reserve. The default is `1.0Gi`.
|
||||
|
||||
1. Specify a **workspace volume** to hold your personal workspace for this
|
||||
notebook server. Kubeflow provisions a
|
||||
[Kubernetes persistent volume (PV)](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) for your workspace volume. The PV ensures that you can
|
||||
retain data even if you destroy your notebook server.
|
||||
|
||||
* The default is to create a new volume for your workspace with the
|
||||
following configuration:
|
||||
|
||||
* Name: The volume name is synced with the name of the notebook server,
|
||||
and has the form `workspace-<server-name>`.
|
||||
When you start typing the notebook server name, the volume name appears.
|
||||
You can edit the volume name, but if you later edit the
|
||||
notebook server name, the volume name changes to match the notebook
|
||||
server name.
|
||||
* Size: `10Gi`
|
||||
* Access mode: `ReadWriteOnce`. This setting means that the volume can be
|
||||
mounted as read-write by a single node. See the
|
||||
[Kubernetes documentation](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) for more details about access modes.
|
||||
* Mount point: `/home/jovyan`
|
||||
|
||||
* Alternatively, you can point the notebook server at an existing volume by
|
||||
specifying the name of the existing volume.
|
||||
|
||||
1. *(Optional)* Specify one or more **data volumes** if you want to store and
|
||||
access data from the notebooks on this notebook server. You can add new
|
||||
volumes or specify existing volumes. Kubeflow provisions a
|
||||
[Kubernetes persistent volume (PV)](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) for each of your data volumes.
|
||||
|
||||
1. *(Optional)* Specify one or more additional **configurations** as a list of
|
||||
PodDefault labels. To make use of this option, you must create a
|
||||
[PodDefault manifest](https://github.com/kubeflow/kubeflow/blob/master/components/admission-webhook/README.md). In the PodDefault manifest, you can specify configurations
|
||||
including volumes, secrets, and environment variables.
|
||||
Kubeflow matches the labels in the **configurations** field against
|
||||
the properties specified in the PodDefault manifest. Kubeflow then injects
|
||||
these configurations into all the notebook Pods on this notebook server.
|
||||
|
||||
For example, enter the label `addgcpsecret` in the **configurations** field
|
||||
to match to a PodDefault manifest containing the following configuration:
|
||||
|
||||
```
|
||||
matchLabels:
|
||||
addgcpsecret: "true"
|
||||
```
|
||||
|
||||
For indepth information on PodDefault usage, see the [admission-webhook
|
||||
README](https://github.com/kubeflow/kubeflow/blob/master/components/admission-webhook/README.md).
|
||||
|
||||
1. *(Optional)* Schedule one or more **GPUs** for your notebook server, as
|
||||
discussed in the section on [specifying your Docker image](#docker-image).
|
||||
|
||||
You can find more details about scheduling GPUs in the [Kubernetes
|
||||
documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/).
|
||||
|
||||
1. *(Optional)* Change the setting for **enable shared memory**. The default is
|
||||
that shared memory is enabled. Some libraries like PyTorch use shared memory
|
||||
for multiprocessing. Currently there is no implementation in Kubernetes to
|
||||
activate shared memory. As a workaround, Kubeflow creates an empty directory
|
||||
at `/dev/shm`.
|
||||
|
||||
1. Click **LAUNCH**. You should see an entry for your new
|
||||
notebook server on the **Notebook Servers** page, with a spinning indicator in
|
||||
the **Status** column. It can take a few minutes to set up
|
||||
the notebook server.
|
||||
|
||||
* You can check the status of your Pod by hovering your mouse cursor over
|
||||
the icon in the **Status** column next to the entry for your notebook
|
||||
server. For example, if the image is downloading then the status spinner
|
||||
has a tooltip that says `ContainerCreating`.
|
||||
|
||||
Alternatively, you can check the Pod status by entering the following
|
||||
command:
|
||||
|
||||
```
|
||||
kubectl -n <NAMESPACE> describe pods jupyter-<USERNAME>
|
||||
```
|
||||
|
||||
Where `<NAMESPACE>` is the namespace you specified earlier
|
||||
(default `kubeflow`) and `<USERNAME>` is the name you used to log in.
|
||||
**A note for GCP users:** If you have IAP turned on, the Pod has
|
||||
a different name. For example, if you signed in as `USER@DOMAIN.EXT`
|
||||
the Pod has a name of the following form:
|
||||
|
||||
```
|
||||
jupyter-accounts-2egoogle-2ecom-3USER-40DOMAIN-2eEXT
|
||||
```
|
||||
|
||||
1. When the notebook server provisioning is complete, you should see an entry
|
||||
for your server on the **Notebook Servers** page, with a check mark in the
|
||||
**Status** column:
|
||||
|
||||
<img src="/docs/images/notebook-servers.png"
|
||||
alt="Opening notebooks from the Kubeflow UI"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
1. Click **CONNECT** to start the notebook server.
|
||||
|
||||
1. When the notebook server is running, you should see the Jupyter dashboard
|
||||
interface. If you requested a new workspace, the dashboard should be empty
|
||||
of notebooks:
|
||||
|
||||
<img src="/docs/images/jupyter-dashboard.png"
|
||||
alt="Jupyter dashboard with no notebooks"
|
||||
class="mt-3 mb-3 border border-info rounded">
|
||||
|
||||
1. Click **Upload** to upload an existing notebook, or click **New** to
|
||||
create an empty notebook. You can read about using notebooks in the
|
||||
[Jupyter documentation](https://jupyter-notebook.readthedocs.io/en/latest/notebook.html#notebook-user-interface).
|
||||
|
||||
<a id="experiment"></a>
|
||||
## Experiment with your notebook
|
||||
|
||||
The standard notebook images include all the plugins that you need to train a
|
||||
TensorFlow model with Jupyter, including
|
||||
[Tensorboard](https://www.tensorflow.org/get_started/summaries_and_tensorboard)
|
||||
for rich visualizations and insights into your model.
|
||||
|
||||
To test your Jupyter installation, run the following basic 'hello world' program
|
||||
(adapted from
|
||||
[mnist_softmax.py](https://github.com/tensorflow/tensorflow/blob/r1.4/tensorflow/examples/tutorials/mnist/mnist_softmax.py)) as follows:
|
||||
|
||||
1. When adding the notebook server, make sure that the base image for your
|
||||
notebook server includes TensorFlow. To ensure this, you can select one of the
|
||||
standard images from the **Image** dropdown menu.
|
||||
|
||||
1. Use the Jupyter dashboard to create a new **Python 3** notebook.
|
||||
|
||||
1. Copy the following code and paste it into a code block in your notebook:
|
||||
|
||||
```
|
||||
from tensorflow.examples.tutorials.mnist import input_data
|
||||
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
|
||||
|
||||
import tensorflow as tf
|
||||
|
||||
x = tf.placeholder(tf.float32, [None, 784])
|
||||
|
||||
W = tf.Variable(tf.zeros([784, 10]))
|
||||
b = tf.Variable(tf.zeros([10]))
|
||||
|
||||
y = tf.nn.softmax(tf.matmul(x, W) + b)
|
||||
|
||||
y_ = tf.placeholder(tf.float32, [None, 10])
|
||||
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
|
||||
|
||||
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(cross_entropy)
|
||||
|
||||
sess = tf.InteractiveSession()
|
||||
tf.global_variables_initializer().run()
|
||||
|
||||
for _ in range(1000):
|
||||
batch_xs, batch_ys = mnist.train.next_batch(100)
|
||||
sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
|
||||
|
||||
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
|
||||
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
|
||||
print("Accuracy: ", sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))
|
||||
```
|
||||
|
||||
1. Run the code. You should see a number of `WARNING` messages from TensorFlow,
|
||||
followed by a line showing a training accuracy something like this:
|
||||
|
||||
```
|
||||
Accuracy: 0.9012
|
||||
```
|
||||
|
||||
Please note that when running on most cloud providers, the public IP address is
|
||||
exposed to the internet and is an unsecured endpoint by default.
|
||||
|
||||
## Next steps
|
||||
|
||||
* See a [simple example](https://github.com/kubeflow/examples/tree/master/pipelines/simple-notebook-pipeline) of creating Kubeflow pipelines in a Jupyter notebook on GCP.
|
||||
* Build machine-learning pipelines with the [Kubeflow Pipelines
|
||||
SDK](/docs/components/pipelines/sdk/sdk-overview/).
|
||||
* Explore [Kubeflow Fairing](/docs/external-add-ons/fairing/) for a complete solution to
|
||||
building, training, and deploying an ML model from a notebook.
|
||||
* See how to configure [multi-user isolation](/docs/components/multi-tenancy/) in Kubeflow, to separate the notebooks for each user in a shared Kubeflow deployment.
|
||||
* Learn the advanced features available from a Kubeflow notebook, such as
|
||||
[submitting Kubernetes resources](/docs/components/notebooks/submit-kubernetes/) or
|
||||
[building Docker images](/docs/components/notebooks/custom-notebook/).
|
||||
* Visit the [troubleshooting guide](/docs/components/notebooks/troubleshoot) for fixing common
|
||||
errors in creating Jupyter notebooks in Kubeflow
|
||||
|
||||
|
|
@ -1,43 +1,31 @@
|
|||
+++
|
||||
title = "Submit Kubernetes Resources"
|
||||
description = "Submitting Kubernetes resources from a Jupyter notebook"
|
||||
description = "Submitting Kubernetes resources from a Notebook"
|
||||
weight = 40
|
||||
|
||||
+++
|
||||
Kubeflow assigns the `default-editor` service account to the Jupyter notebook
|
||||
Pods. This service account is bound to the `kubeflow-edit` ClusterRole, which has namespace-scoped permissions to the many Kubernetes resources including:
|
||||
|
||||
* Pods
|
||||
* Deployments
|
||||
* Services
|
||||
* Jobs
|
||||
* TFJobs
|
||||
* PyTorchJobs
|
||||
## Notebook Pod ServiceAccount
|
||||
|
||||
You can get a full list of these permissions using:
|
||||
Kubeflow assigns the `default-editor` Kubernetes ServiceAccount to the Notebook Pods.
|
||||
The Kubernetes `default-editor` ServiceAccount is bound to the `kubeflow-edit` ClusterRole, which has namespace-scoped permissions to many Kubernetes resources.
|
||||
|
||||
You can get the full list of RBAC for `ClusterRole/kubeflow-edit` using:
|
||||
```
|
||||
kubectl describe clusterrole kubeflow-edit
|
||||
```
|
||||
|
||||
You can therefore create the above Kubernetes resources directly from your
|
||||
Jupyter notebook in Kubeflow. The Kubernetes
|
||||
[`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
|
||||
command-line tool is pre-installed in the notebook.
|
||||
## Kubectl in Notebook Pod
|
||||
|
||||
Run the following command in a Jupyter notebook cell to create Kubernetes
|
||||
resources:
|
||||
Because every Notebook Pod has the highly-privileged `default-editor` Kubernetes ServiceAccount bound to it, you can run `kubectl` inside it without providing additional authentication.
|
||||
|
||||
For example, the following command will create the resources defined in `test.yaml`:
|
||||
|
||||
```shell
|
||||
kubectl create -f "test.yaml" --namespace "MY_PROFILE_NAMESPACE"
|
||||
```
|
||||
!kubectl create -f myspec.yaml
|
||||
```
|
||||
|
||||
The `myspec.yaml` file should describe one of the above Kubernetes resources.
|
||||
For information about the format of the YAML file, see the
|
||||
[Kubernetes object guide](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/).
|
||||
|
||||
## Next steps
|
||||
|
||||
* See the guide to [setting up
|
||||
your Jupyter notebooks in Kubeflow](/docs/components/notebooks/setup/).
|
||||
* Explore the [components of Kubeflow](/docs/components/), including custom
|
||||
Kubernetes resources.
|
||||
- See the Kubeflow Notebook [quickstart guide](/docs/components/notebooks/quickstart-guide/).
|
||||
- Explore the other [components of Kubeflow](/docs/components/).
|
||||
|
|
|
|||
|
|
@ -1,95 +0,0 @@
|
|||
+++
|
||||
title = "Troubleshooting Guide for Jupyter Notebooks"
|
||||
description = "Fixing common problems of Jupyter notebook deployments on Kubeflow"
|
||||
weight = 50
|
||||
|
||||
+++
|
||||
## Persistent Volumes and Persistent Volumes Claims
|
||||
|
||||
First, make sure that [Persistent Volumes Claims (PVCs)](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) are bounded when using Jupyter notebooks. This should
|
||||
not be a problem when using managed Kubernetes. But if you are using Kubernetes
|
||||
on-prem, check out the guide to [Kubeflow on-prem in a multi-node Kubernetes cluster](/docs/other-guides/kubeflow-on-multinode-cluster/) if you are running Kubeflow in multi-node on-prem environment. Otherwise, look at the [Pods stuck in Pending State](/docs/other-guides/troubleshooting/#pods-stuck-in-pending-state) guide to troubleshoot this problem.
|
||||
|
||||
## Check the status of notebooks
|
||||
|
||||
Run the following commands replacing `${NOTEBOOK}` with your notebook file name:
|
||||
|
||||
```shell
|
||||
kubectl get notebooks -o yaml ${NOTEBOOK}
|
||||
kubectl describe notebooks ${NOTEBOOK}
|
||||
```
|
||||
|
||||
Check the `events` section to make sure that there are no errors.
|
||||
|
||||
## Check the status of statefulsets
|
||||
|
||||
Make sure that the number of `statefulsets` equals the desired number. If it is
|
||||
not the case, check for errors using the `kubectl describe`.
|
||||
|
||||
```shell
|
||||
kubectl get statefulsets -o yaml ${NOTEBOOK}
|
||||
kubectl describe statefulsets ${NOTEBOOK}
|
||||
```
|
||||
|
||||
The output should look like below:
|
||||
|
||||
```
|
||||
NAME DESIRED CURRENT AGE
|
||||
your-notebook 1 1 9m4s
|
||||
```
|
||||
|
||||
## Check the status of Pods
|
||||
|
||||
If the number of statefulsets didn't match the desired number, make sure that
|
||||
the number of Pods match the number of desired Pods in the first command.
|
||||
In case it didn't match, follow the steps below to further investigate the issue.
|
||||
|
||||
```shell
|
||||
kubectl get pod -o yaml ${NOTEBOOK}-0
|
||||
```
|
||||
|
||||
* The name of the Pod should start with `jupyter`.
|
||||
* If you are using username/password auth with Jupyter the pod will be named `jupyter-${USERNAME}`.
|
||||
* If you are using [Identity-Aware Proxy (IAP)](https://cloud.google.com/iap/docs/concepts-overview) on [GKE](https://cloud.google.com/iap/docs/enabling-kubernetes-howto), the pod will be named as follows: `jupyter-accounts-2egoogle-2ecom-3USER-40DOMAIN-2eEXT`, where `USER@DOMAIN.EXT` is the Google account you used with IAP.
|
||||
|
||||
Once you know the name of the pod, run:
|
||||
|
||||
```shell
|
||||
kubectl describe pod ${NOTEBOOK}-0
|
||||
```
|
||||
|
||||
* Check the `events` for any errors trying to schedule the pod.
|
||||
* One common error is not being able to schedule the pod because there aren't enough resources in the cluster.
|
||||
|
||||
If the error still persists, check for the errors in the logs of containers.
|
||||
|
||||
```shell
|
||||
kubectl logs ${NOTEBOOK}-0
|
||||
```
|
||||
|
||||
## Delete notebooks manually
|
||||
|
||||
It is possible to delete notebooks manually with the following command:
|
||||
|
||||
```shell
|
||||
kubectl delete notebook ${NOTEBOOK}
|
||||
```
|
||||
|
||||
Note that deleting the `statefulset` is not enough, it's necessary to delete the `notebook` resource.
|
||||
|
||||
## Note for GCP Users
|
||||
|
||||
If you're using Google Cloud, you may encounter an error, such as:
|
||||
|
||||
```
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Warning FailedCreate 2m19s (x26 over 7m39s) statefulset-controller create Pod test1-0 in StatefulSet test1 failed error: pods "test1-0" is forbidden: error looking up service account kubeflow/default-editor: serviceaccount "default-editor" not found
|
||||
```
|
||||
|
||||
To fix this problem, create a service account named `default-editor` with cluster-admin role.
|
||||
|
||||
```shell
|
||||
kubectl create sa default-editor
|
||||
kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user default-editor
|
||||
```
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
+++
|
||||
title = "Troubleshooting"
|
||||
description = "Problems and solutions for common problems with Kubeflow Notebooks"
|
||||
weight = 100
|
||||
|
||||
+++
|
||||
|
||||
## ISSUE: notebook not starting
|
||||
|
||||
### SOLUTION: check events of Notebook
|
||||
|
||||
Run the following command then check the `events` section to make sure that there are no errors:
|
||||
|
||||
```shell
|
||||
kubectl describe notebooks "${MY_NOTEBOOK_NAME}" --namespace "${MY_PROFILE_NAMESPACE}"
|
||||
```
|
||||
|
||||
### SOLUTION: check events of Pod
|
||||
|
||||
Run the following command then check the `events` section to make sure that there are no errors:
|
||||
|
||||
```shell
|
||||
kubectl describe pod "${MY_NOTEBOOK_NAME}-0" --namespace "${MY_PROFILE_NAMESPACE}"
|
||||
```
|
||||
|
||||
### SOLUTION: check YAML of Pod
|
||||
|
||||
Run the following command and check the Pod YAML looks as expected:
|
||||
|
||||
```shell
|
||||
kubectl get pod "${MY_NOTEBOOK_NAME}-0" --namespace "${MY_PROFILE_NAMESPACE}" -o yaml
|
||||
```
|
||||
|
||||
### SOLUTION: check logs of Pod
|
||||
|
||||
Run the following command to get the logs from the Pod:
|
||||
|
||||
```shell
|
||||
kubectl logs "${MY_NOTEBOOK_NAME}-0" --namespace "${MY_PROFILE_NAMESPACE}"
|
||||
```
|
||||
|
||||
## ISSUE: manually delete notebook
|
||||
|
||||
### SOLUTION: use kubectl to delete Notebook resource
|
||||
|
||||
Run the following command to delete a Notebook resource manually:
|
||||
|
||||
```shell
|
||||
kubectl delete notebook "${MY_NOTEBOOK_NAME}" --namespace "${MY_PROFILE_NAMESPACE}"
|
||||
```
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
+++
|
||||
title = "Overview of Jupyter Notebooks in Kubeflow"
|
||||
description = "Why use Jupyter notebooks in Kubeflow"
|
||||
weight = 5
|
||||
|
||||
+++
|
||||
{{% stable-status %}}
|
||||
|
||||
There are multiple benefits of integrating Jupyter notebooks in Kubeflow for enterprise environments. These benefits include:
|
||||
|
||||
* Integrating well with the rest of the infrastructure with respect to authentication and access control.
|
||||
* Enabling easier notebook sharing across the organization. Users can create notebook containers or pods directly in the cluster, rather than locally on their workstations. Admins can provide standard notebook images for their organization, and set up role-based access control (RBAC), Secrets and Credentials to manage which teams and individuals can access the notebooks.
|
||||
|
||||
When you bundle Jupyter notebooks in Kubeflow, you can use the Fairing library to submit training jobs using TFJob. The training job can run single node or distributed on the same Kubernetes cluster, but not inside the notebook pod itself. Submitting the job with the Fairing library makes processes like Docker containerization and pod allocation clear for data scientists.
|
||||
|
||||
Overall, Kubeflow-hosted notebooks are better integrated with other components while providing extensibility for notebook images.
|
||||
|
||||
## Next steps
|
||||
|
||||
* Learn more about [setting up notebooks](/docs/components/notebooks/setup/).
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
Loading…
Reference in New Issue