* build: Update components makefiles for building
We'll create a top-level Makefile under components/ dir
that has the following rules:
* build-all:
To build all images locally
* push-all:
* We can use a specific REGISTRY and retag the images
* Push all the images
This top-level Makefile will run the sub-Makefiles that every
component has for building and pushing the images.
We modified every sub-Makefile as follows:
* We don't use a registry in images by default
* Removed unused rules and vars
* Use the --dirty flag of git describe in TAG
--dirty[=<mark>]
Describe the working tree. It means describe HEAD and
appends <mark> (-dirty by default) if the working tree
is dirty.
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Create makefiles for notebook servers
The common starting point of building the notebook-server
images are the following 4 *base* images:
- base
- codeserver
- jupyter
- rstudio
All other server images need to build on top of them. We'll
dynamically pass the base images in every Dockerfile by
using an ARG IMG. We can set the value of this ARG during
docker build with the --build-arg CLI argument.
This way we build both the base images with a tag locally,
and then we pass that image as arg via the Makefile and build the rest
So we modified our building procedure as follows:
1. Build the base image since everything starts from there
2. Pass the base image as an ARG in the Dockerfiles of
jupyter, codeserver, rstudio images and build on top
3. Pass the base images in all other server images and build
on top
For that we will:
1. Create a Makefile for each of the notebook servers, in each folder
a. Each makefile will be responsible for building the bases and use args for passing them on
2. Use the central Makefile to call each Makefile from above
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* cherry-pick: Notebook server upstream fixes
Relevant upstream PR: https://github.com/kubeflow/kubeflow/pull/6466/files
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Fix Makefiles
- Remove build-gcp and build-gcr rules as we don't use them anywhere in
the project
- Fix code conficts
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Fix Dockerfiles for notbook-server images
We remove the previous logic of using already built images as bases.
The users must use only the Makefiles to pass the appropriate BASE_IMG
and build the images correctly.
Thus, we have Makefiles everywhere that:
- Can build any base image
- If an image requires another notebook base, then we first build that one using its makefile,
and then use it as docker ARG for building the next one
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Fix notebook-controller Makefile
Removed a misplaced "|" char that breaks the Makefile
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Update GH action workflows
* Update workflow for notebook-server images:
- Add a step for building all images by using the
central-Makefile under components/example-notebook-servers/ dir.
- Add a step for pushing all images by using the
central-Makefile under components/example-notebook-servers/ dir.
* Update workflow for all Kubeflow images:
- Add a step for building & pushing all images by using the
top-level Makefile under components/ dir.
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Remove completely ECR references from images
Replace everywhere the "public.ecr.aws/j1r0q0g6/notebooks/notebook-servers"
prefix with "kubeflownotebookswg"
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Fix GH actions for Kubeflow components
Fix GH actions to use the updated make rules
when building the Kubeflow component images.
Remove the "docker.io" prefix when building with
GH action workflows
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* Cherry-pick latest commits to the 1.5-branch (kubeflow/kubeflow#6354)
* Fix(manifests): Upgrade rbac.authorization.k8s.io from v1beta1 to v1 (kubeflow/kubeflow#6261)
* proposal: Extend Notebook Controller to expose idleness for Jupyter (kubeflow/kubeflow#6295)
* proposal: Extend Notebook Controller to expose idleness for Jupyter (kubeflow/kubeflow#6270)
Provide a design doc as a proposal for extending Notebook Controller to
expose idleness for Jupyter. Our proposal is in markdown format and follows
the guidelines of the kubeflow/components/proposal/README.md guide.
You can view the #6270 issue in the following link:
https://github.com/kubeflow/kubeflow/issues/6270
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
* review: change the title of the proposal
Change the title of the proposal to only include the
proposed new feature. The new title of the proposal
will now be "Expose Idleness Information for Jupyter
Notebooks".
* review: rename the proposal markdown file
We want to give a more specific name to the markdown
which contains the proposal. Since this proposal
emphasizes on a feature regarding the Jupyter Notebooks,
the new name will be:
20220121-jupyter-notebook-idleness.md
* Synchronize jupyter-web-application role with clusterrole (kubeflow/kubeflow#6241)
* Update role.yaml
* Update role.yaml
* Update cluster-role.yaml
* Kubeflow Roadmap update - with 1.5 details (kubeflow/kubeflow#6266)
* Kubeflow Roadmap update - with 1.5 details
These proposed changes include: identifying that 1.4.1 has been delivered, provides themes for 1.5 and provides details of major features in 1.5 by working group. This is an initial proposal that needs review by the working group leads.
* correct formatting in KFP features
Moved KFP features under KFP Control Flow doc
* updating KFP section
updating KFP references with updates from KFP team
* Updated the 1.5 release date to March
updated the 1.5 release date to March
* Update ROADMAP.md
change Hyperparameter leader election to Katib leader election
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Update ROADMAP.md
improve description and details of feature for metrics collector
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Update Katib description for Early stopping in 1.5
updating with Andrey's suggestion (but without the world proper). * Validation for Early Stopping algorithm settings helps users to proper reduce model overfitting
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* notebooks: Extend Notebook Controller to expose idleness for Jupyter (kubeflow/kubeflow#6297)
* notebooks: Update image's tag in make
Modify Makefile to update properly the TAG
based on the git TAG.
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
Reviewed-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Expose last-activity
Extend the notebook-controller to:
* cull idle Notebook Servers based on their new `last-activity`
annotation
* expose the last activity of each Notebook Server as an annotation
on the metadata of the corresponding CR object
Modify notebook_controller.go to:
* update the Last Activity of each Notebook Server that has a
Running pod
* delete the Last Activity Annotation for every Notebook Server
that does not have a Running pod
Extend culler.go to:
* perform culling based on the new `last-activity` annotation and
not based on the `/api/status` endpoint.
* update the last activity of a Notebook Server, based on the
kernels' execution states.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Athanasios Markou <athamark@arrikto.com>
* notebooks: Introduce a DEV env var
We introduce a DEV ENV var to allow admins
develop and test on their local machine their
custom Notebook Controller.
We provide information and instructions inside
the components/notebook-controller/README.md.
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
Reviewed-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Add unit tests for last-activity
* Introduce new tests for allKernelsAreIdle()
* Extend the tests for NotebookIsIdle() and for
NotebookNeedsCulling().
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
Reviewed-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* review: UpdateNotebookLastActivityAnnotation()
Ensure that UpdateNotebookLastActivityAnnotation() does not return
"true". This function should not return any value.
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
* jwa: Rework the Storage API of the web app (kubeflow/kubeflow#6321)
* wa(back): Add helper for deserializing JSON obj
In some cases we might need to construct Python k8s lib objects from the
JSONs that are provided by clients. I.e. the UI will be sending a PVC
object in json format, so the backend will need to create the
corresponding client.V1PersistentVolumeClaim object and submit it.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Serialization helper
Add helper function for converting a k8s-client object into a dict that
can be sent as an HTTP response.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Add dry run to Notebooks and PVCs
The backend will need to be able to create objects with dry-run, in
order to ensure they are valid. The backend will need to check that both
the Notebook and the PVCs can be created beforehand.
This way we avoid the scenario where we create PVCs but the Notebook
fails to be created, and the PVCs are never garbage collected.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Update kubernetes to 0.17
In order to support dry-run we must use the 0.17 version of the Python
k8s client.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Extend api module to patch pvcs
The backend will need to be able to PATCH PVCs in order to set the
ownerReference to the Notebook that mounts the PVCs.
Ref: arrikto/dev/issues/386#issuecomment-856700392
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Work with new Volumes API
The backend API should not add any more layers of abstractions on top of
the K8s API. The backend should expect the client/UI to be sending the
entire PVC spec of a new PVC.
Refs: arrikto/dev/issues/386
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Add unittests for new volumes API
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Extend the PVC info returned
We want to show both the access mode and size of the existing PVCs, when
a user clicks on the dropdown to select which PVC to mount.
The backend will need to provide this information to the frontend. We
don't want to send the K8s list of PVCs since this will result in a lot
of unnecessary data to be sent.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(front): Add proxy config for Rok
When developing the Rok flavor locally we will need to be able to open
the Rok chooser. This can be done by using Angular/webpack proxy to
bring the exposed rok service and the app under the same domain.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Remove card from form
The form of the app should not be a big card, but a normal form.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Install AceModule for yaml editing
Install AceModule to allow users to edit yamls of objects.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* wa(front): Change the styling of form sections
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Create common volume components
Component for:
* New PVC and configuring its spec
* Attaching an existing PVC in a Notebook
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Update Rok form for new Volume API
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Mark inputs as dirty when restoring Lab
When the UI autofills the form with values from a JupyterLab snapshot
then it should mark the touched fields as dirty. This way if a field has
errors the UI will make that input red.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa: Update ConfigMap in manifests
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Fix format errors
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* profiles: Update the permissions for notebook idleness (kubeflow/kubeflow#6335)
Extend the Profiles Controller to give permissions to Notebooks
controller for making GET requests to notebook's /api/kernels endpoint.
Refs https://github.com/kubeflow/kubeflow/blob/master/components/proposals/20220121-jupyter-notebook-idleness.md
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Graceful handling of events (kubeflow/kubeflow#6338)
* notebooks: Handle events gracefully
The controller is not exiting the reconciliation loop after it has
re-emitted a Pod/STS Event as a Notebook Event. This results in the
controller to later on try and GET a Notebook with the name of the Event
that triggered the reconciliation loop.
The controller should exit the reconciliation function once it has
emitted the event.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Don't reconcile on deleted events
We don't want to trigger the reconciliation function when an event gets
deleted.
If a Notebook would be deleted then the underlying events would
be deleted as well, which results in the reconcile function to get
triggered and try to GET Events and Notebooks with the name of the
deleted event.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Fix endless restarts (kubeflow/kubeflow#6341)
* notebooks: Update notebook if timestamp changed
We don't want to be updating the spec of the notebook if the timestamp
hasn't changed, since this will lead to constant updates and
reconciliation loops.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Use a deep-copy of the notebook spec
The controller should use a deep-copy of the notebook spec when
calculating the spec for the StatefulSet. If not then we could
update the notebook object without wanting it, since the spec could have
been changed when calculating the STS spec.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Add prefix env var only if missing
The controller should be setting OR updating the NB_PREFIX env var.
Previously it would always blindly append it to the spec, which could
result in double entries for the same env var.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* releasing: Update tags for v1.5.0-rc.1 (kubeflow/kubeflow#6343)
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* fix error comparison between pointer and pointer in "CopyStatefulSetFields" (kubeflow/kubeflow#6195)
error comparison between pointer and pointer in "CopyStatefulSetFields"
* [bugfix]: update unsupported links to latest pages (kubeflow/kubeflow#6192)
* centraldashboard: Add KServe overlay (kubeflow/kubeflow#6383)
Configure the dashboard to use the KServe app instead of the KFServing
0.6.1 one.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Don't reconcile on Events deletion (kubeflow/kubeflow#6391)
The controller should not trigger the reconcile loop when an Event is
deleted. Previously the controller would run the reconciliation loop on
any event deletion.
This commit updates it to not run the loop for ANY event.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* releasing: Create v1.5.0-rc.2 images (kubeflow/kubeflow#6394)
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* fixed typo
* fixed typo
* changed remove rc2
Co-authored-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Co-authored-by: mofanke <54242816+mofanke@users.noreply.github.com>
Co-authored-by: Jaeyeon Kim <anencore94@gmail.com>
Fix https://github.com/kubeflow/kubeflow/issues/6366
Migrating to Kubebuilder v3 leads to the following changes:
- Add .dockerignore file.
- Upgrade Go version from v1.15 to v1.17.
- Adapt Makefile.
- Add image (build + push) target to makefile.
- Upgrade EnvTest to use K8s v1.22.
- Update PROJECT template.
- Migrate CRD apiVersion from v1beta to v1.
- Add livenessProbe and readinessProbe to controller manager.
- Upgrade controller-runtime from v0.2.0 to v0.11.0.
Other changes:
- Build image using public.ecr.aws registry instead of gcr.io.
- Update README.md documentation.
- Update 3rd party licences.
- Fix notebook.spec description.
- Add 3 sample notebooks (v1, v1alpha1 and v1beta1).
Signed-off-by: Samuel Veloso <svelosol@redhat.com>
The controller should not trigger the reconcile loop when an Event is
deleted. Previously the controller would run the reconciliation loop on
any event deletion.
This commit updates it to not run the loop for ANY event.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Update notebook if timestamp changed
We don't want to be updating the spec of the notebook if the timestamp
hasn't changed, since this will lead to constant updates and
reconciliation loops.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Use a deep-copy of the notebook spec
The controller should use a deep-copy of the notebook spec when
calculating the spec for the StatefulSet. If not then we could
update the notebook object without wanting it, since the spec could have
been changed when calculating the STS spec.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Add prefix env var only if missing
The controller should be setting OR updating the NB_PREFIX env var.
Previously it would always blindly append it to the spec, which could
result in double entries for the same env var.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Handle events gracefully
The controller is not exiting the reconciliation loop after it has
re-emitted a Pod/STS Event as a Notebook Event. This results in the
controller to later on try and GET a Notebook with the name of the Event
that triggered the reconciliation loop.
The controller should exit the reconciliation function once it has
emitted the event.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Don't reconcile on deleted events
We don't want to trigger the reconciliation function when an event gets
deleted.
If a Notebook would be deleted then the underlying events would
be deleted as well, which results in the reconcile function to get
triggered and try to GET Events and Notebooks with the name of the
deleted event.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* wa(back): Add helper for deserializing JSON obj
In some cases we might need to construct Python k8s lib objects from the
JSONs that are provided by clients. I.e. the UI will be sending a PVC
object in json format, so the backend will need to create the
corresponding client.V1PersistentVolumeClaim object and submit it.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Serialization helper
Add helper function for converting a k8s-client object into a dict that
can be sent as an HTTP response.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Add dry run to Notebooks and PVCs
The backend will need to be able to create objects with dry-run, in
order to ensure they are valid. The backend will need to check that both
the Notebook and the PVCs can be created beforehand.
This way we avoid the scenario where we create PVCs but the Notebook
fails to be created, and the PVCs are never garbage collected.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Update kubernetes to 0.17
In order to support dry-run we must use the 0.17 version of the Python
k8s client.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Extend api module to patch pvcs
The backend will need to be able to PATCH PVCs in order to set the
ownerReference to the Notebook that mounts the PVCs.
Ref: arrikto/dev/issues/386#issuecomment-856700392
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Work with new Volumes API
The backend API should not add any more layers of abstractions on top of
the K8s API. The backend should expect the client/UI to be sending the
entire PVC spec of a new PVC.
Refs: arrikto/dev/issues/386
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Add unittests for new volumes API
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Extend the PVC info returned
We want to show both the access mode and size of the existing PVCs, when
a user clicks on the dropdown to select which PVC to mount.
The backend will need to provide this information to the frontend. We
don't want to send the K8s list of PVCs since this will result in a lot
of unnecessary data to be sent.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(front): Add proxy config for Rok
When developing the Rok flavor locally we will need to be able to open
the Rok chooser. This can be done by using Angular/webpack proxy to
bring the exposed rok service and the app under the same domain.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Remove card from form
The form of the app should not be a big card, but a normal form.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Install AceModule for yaml editing
Install AceModule to allow users to edit yamls of objects.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* wa(front): Change the styling of form sections
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Create common volume components
Component for:
* New PVC and configuring its spec
* Attaching an existing PVC in a Notebook
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Update Rok form for new Volume API
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Mark inputs as dirty when restoring Lab
When the UI autofills the form with values from a JupyterLab snapshot
then it should mark the touched fields as dirty. This way if a field has
errors the UI will make that input red.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa: Update ConfigMap in manifests
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Fix format errors
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Update image's tag in make
Modify Makefile to update properly the TAG
based on the git TAG.
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
Reviewed-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Expose last-activity
Extend the notebook-controller to:
* cull idle Notebook Servers based on their new `last-activity`
annotation
* expose the last activity of each Notebook Server as an annotation
on the metadata of the corresponding CR object
Modify notebook_controller.go to:
* update the Last Activity of each Notebook Server that has a
Running pod
* delete the Last Activity Annotation for every Notebook Server
that does not have a Running pod
Extend culler.go to:
* perform culling based on the new `last-activity` annotation and
not based on the `/api/status` endpoint.
* update the last activity of a Notebook Server, based on the
kernels' execution states.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Athanasios Markou <athamark@arrikto.com>
* notebooks: Introduce a DEV env var
We introduce a DEV ENV var to allow admins
develop and test on their local machine their
custom Notebook Controller.
We provide information and instructions inside
the components/notebook-controller/README.md.
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
Reviewed-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* notebooks: Add unit tests for last-activity
* Introduce new tests for allKernelsAreIdle()
* Extend the tests for NotebookIsIdle() and for
NotebookNeedsCulling().
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
Reviewed-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* review: UpdateNotebookLastActivityAnnotation()
Ensure that UpdateNotebookLastActivityAnnotation() does not return
"true". This function should not return any value.
Signed-off-by: Athanasios Markou <athamark@arrikto.com>
* Update the releasing version tag
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Run automated script for updating versions
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* wa(docker): Don't copy node_modules
To ensure Kaniko is not copying stale node_moduels folders, even though
we have a dockerignore file, we are explicitly only copying the source
code.
We have seen the build system with Kaniko to fail, due to NFS stale
instances with files in node_modules and expect that this is the root
cause.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* twa: Update makefile to use global dockerignore
The dockerfile for TWA was copying over the local dockerignore. This was
overriding the global one we had for all the web apps.
This commit updates the Makefile of the app to use the global
dockerignore that all the apps should use.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Make ng lint work by ignoring e2e/tsconfig.json
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* web-apps(front): Make the table responsive
We introduced all of our changes in the underlying TableComponent. This
component can then be either used independently or inside a material
card.
The changes we did in that component are:
1. Create Output() emitters, since this component can be used directly
2. The config object for a Table row now supports a `style` prop for
defining the list of CSS styles to be applied
2. Remove the truncate classes (small, medium, large) and only have
a boolean value. The user can define the width directly now via the
`style` property in the row's config
3. Modify the classes for aligning contents right and left
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* web-apps(front): Add table paginator
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* web-apps(front): Add padding to titlebar text
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* web-apps(front): Use bigger font for toolbar title
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Use responsive table and toolbar
We refactor both the index and form pages of the app to:
1. Add a top row toolbar with the title of the app and the button to
create a new Notebook
2. Replace the card with a responsive table that shows the items. The
component also has a paginator
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* vwa(front): Use responsive table and toolbar
This commit:
1. Adds a toolbar at the top of the index page with the title of the app
and the button to create a new volume
2. Replaces the card with a responsive table
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* twa(front): Use responsive table and toolbar
This commit:
1. Adds a toolbar at the top of the index page with the title of the app
and a button to create a new TensorBoard instance
2. Replaces the card with a resopnsive table
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* fix the format
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* wa(front): Add npm script for running unit tests in docker
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* fix unit tests failing
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* kfam: Upgrade go to 1.17
Update to a more recent docker image that has a newer version of
openssl.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* poddefaults: Upgrade go to 1.17
Update to a more recent docker image that has a newer version of
openssl.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* tensorboards: Upgrade go to 1.17
Update to a more recent docker image that has a newer version of
openssl.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
JWA should not block users from selecting GPUs if the current cluster
nodes do not have any GPUs attached to them. We've seen users that have
autoscaled nodegroups for GPUs, so a GPU node will be added to the
cluster once a Pod has requested it.
Refs: arrikto/dev#1484
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
In the existing version, the 'timeout: 300s' added to the notebook's virtual service would cause websockets to disconnect at the 5 minute mark, causing the Jupyter Notebook web terminal function to hang. This is described in https://github.com/kubeflow/kubeflow/issues/6124.
* jwa(front): Don't allow NaN values in limits
The UI should always catch a NaN value and don't add it in the form.
Currently this is the case for the cpu/memory limits.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Limits should not be changed if dirty
If the user has manually edited the limits fields then the UI should not
try to automatically calculate them again, using the limitFactors.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Update dockerfiles and make compatible with Renovate
* Set memory for jupyter pytorch to the same as jupyter tensorflow
* Update protobuf
* Remove conda version and use substring expansion
* Update SQLAlchemy
* Update dill
* [fix]: Make jupyter-web-app parse workspace volume MountPath
- workspace volume path was fixed with "/home/jovyan"
- it should be enable to parse from jupyter-web-app-config's data
* change parsing key correctly
* cwa(front): Ignore font files in assets
* feat(jupyter): add fonts as assets to service
* CRUD: fonts in common
* CWA: Remove link to css file
* jwa(front): Remove font assets from jupyter
Co-authored-by: Wendy Gaultier <wvgaultier@gmail.com>
* jwa(front): Add static logos in the app
The app does not contain the logos' svgs in its source code/static
files. This results in the icons to not show when developing locally.
This commit adds the svgs found in the logos ConfigMap to the static
files of the app as well.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Change logos fetch url
Change the URLs of the logos from `static/assets/*` to
`static/assets/logos`.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(manifests): Don't override assets with logos
Mount the ConfigMap under the `static/assets/logos` directory to not
override the contents of the entire assets dir.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* vwa(front): Add npm script to check the formatting
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* vwa(front): Update the package-lock.json
Run `npm install` to bring the package-lock.json up to date
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* vwa(front): Fix formatting
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* vwa(back): Fix formatting
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* vwa(docker): Remove unused dockerignore file
We have created a global dockerignore file for all the web apps in the
parent dir.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* vwa(docker): Don't copy node_modules in dockerfile
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* vwa(make): Don't include dockerignore
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* ci(vwa): Add format check tasks
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(docker): Copy only necessary files for build
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* review: Use docker context instead of cd ..
Don't use a `cd ..` and copy dockerignore files back and forth. Instead
we should use the Docker context and the global dockerignore file we
have for all the web apps.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Make notebook limits configurable with a multiplication factor
* Make limits configurable under advanced section
* run prettier to format frontend code
* fix formatting and add rounding in backend
* Return error if limit is smaller than request
* Allow disabling limitFactor by setting it to none
* review: remove camelCase in python backend
* fix: update spawner_ui_config.yaml in manifests directory
* review: fix setting limits backend
* review: remove unnecessary check from backend
* rebase: Make logos configurable in configmap and remove trademark references
Rebased to remove the changes to the package-lock.json
* review: add suggested changes and add image group section to README
When the TB controller attempts to schedule a RWO PVC it checks its
accessModes in the PVC status. The controller panics if the list is
empty.
This commit adds a check to ensure the list is not empty.
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* web-apps(back): Introduce APP_SECURE_COOKIES var
Expose a new APP_SECURE_COOKIES env variable that will configure whether
the web apps should set Secure cookies or not.
This will allow the admins to configure the web apps to work when
Kubeflow is exposed over localhost/http.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Switch CSRF checks order
The order the backend makes the CSRF checks should be the following:
1. check if the CSRF cookie is present
2. check if the CSRF header is present
3. check if the CSRF cookie and header have the same value
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps: Set APP_SECURE_COOKIES to false in dev
When running the web apps via the makefiles in dev mode we will need to
explicitly set the APP_SECURE_COOKIES env var to False, since the app
will be served over http.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Add RStudio trademark statement
* move tooltip position to the right of the RStudio button
* fix labels of icons
* RStudio trademark tooltip on index page, remove title jupyter-icon
* jwa(back): ability to setting annotations on NB resource
* jwa(back): update spanwer yaml, dump logo from yaml if file doesn't exist
* jwa(front): add annotations and VSCode/RStudio image types/config
* jwa(front): add server type toggle to UI
* jwa(front): set annotations in notebook request based on server-type
* jwa(front): add server type column to index page
* review: improve button toggle formatting
* jwa(back): set rstudio-tidyverse image in spawner_ui_config
* review: move rewrite and headers to backend
* review: add logo SVGs and set them in environment*.ts
* review: fix how allowing custom images works
* review: add server type logo to index
* Add base dockerfile for all jupyter based images
* cleanup jupyter notebook image
Co-authored-by: Mathew Wicks <thesuperzapper@users.noreply.github.com>
* Add base dockerfile for all Web-IDE images (jupyter, r-studio, vs-code)
remove sudo from image
Add S6-overlay
Change naming and add CD
Add ci build test
change naming in prow config to avoid character limit
Add OWNERS file
rename folder
rename folder (again)
remove labels
Rename to the final folder
* cleanup base notebook image Dockerfile
Co-authored-by: Mathew Wicks <thesuperzapper@users.noreply.github.com>
mofanke
Xin Hao
Xiangquan Xiao
Hyunwoo Kim
apoger
Anna
Kimonas Sotirchos
Jeongwook Park
Mathew Wicks
Alex Lembiyeuski
David Lee
Samu
Benjamin Tan Wei Hao
dependabot[bot]
Younghwi Kim
Athanasios Markou
juliusvonkohout
Tobia De Koninck
Jaeyeon Kim
Rohan Katkar
James Liu
David van der Spek
Abe Sharp
Abhishek Vilas Munagekar
Filinto Duran
Wendy Gaultier
Saffa Alvi
Ilias Katsakioris
Yannis Zarkadas