notebooks/components/notebook-controller

Notebook Controller

The controller allows users to create a custom resource "Notebook" (jupyter notebook). We originally wrote the controller using jsonnet and metacontroller, but are migrating to golang and Kubebuilder here. See discussion.

Spec

The user needs to specify the PodSpec for the jupyter notebook. For example:

apiVersion: kubeflow.org/v1alpha1
kind: Notebook
metadata:
  name: my-notebook
  namespace: test
spec:
  template:
    spec:  # Your PodSpec here
      containers:
      - image: gcr.io/kubeflow-images-public/tensorflow-1.10.1-notebook-cpu:v0.3.0
        args: ["start.sh", "lab", "--LabApp.token=''", "--LabApp.allow_remote_access='True'",
               "--LabApp.allow_root='True'", "--LabApp.ip='*'",
               "--LabApp.base_url=/test/my-notebook/",
               "--port=8888", "--no-browser"]
        name: notebook
      ...

The required fields are containers[0].image and (containers[0].command and/or containers[0].args). That is, the user should specify what and how to run.

All other fields will be filled in with default value if not specified.

Environment parameters

Parameter	Description
ADD_FSGROUP	If the value is true or unset, fsGroup: 100 will be included in the pod's security context. If this value is present and set to false, it will suppress the automatic addition of fsGroup: 100 to the security context of the pod.
DEV	If the value is false or unset, then the default implementation of the Notebook Controller will be used. If the admins want to use a custom implementation from their local machine, they should set this value to true.

Commandline parameters

metrics-addr: The address the metric endpoint binds to. The default value is :8080.

enable-leader-election: Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager. The default value is false.

Implementation detail

This part is WIP as we are still developing.

Under the hood, the controller creates a StatefulSet to run the notebook instance, and a Service for it.

Contributing

https://www.kubeflow.org/docs/about/contributing/

Development Environment

To develop on notebook-controller, your environment must have the following:

• go version v1.15+.
• docker version 17.03+.
• kubectl version v1.11.3+.
• kustomize v3.1.0+
• Access to a Kubernetes v1.11.3+ cluster.
• kubebuilder

In order for the custom Notebook Controller to be functional from your local machine, the admins must:

1. Set the number of replicas to zero:

   kubectl edit deployment notebook-controller-deployment -n=kubeflow
   

2. Allow the controller to proxy the traffic to the Notebook Services by executing on your local machine:

   kubectl proxy
   

TODO

• e2e test (we have one testing the jsonnet-metacontroller one, we should make it run on this one)
• status field should reflect the error if there is any. See #2269.
• Istio integration (controller will generate istio resources to secure each user's notebook)
• CRD validation
• ttlSecondsAfterFinished: This is in the original jsonnet controller spec, but not being used yet. I think we want to cleanup the notebook after idle?
• Add more instructions on contributing like build,deploy and test locally.
• A script for installing all deps.