* web-apps(back): Create list_events function
Create list_events(namespace, field_selector) function for listing
events of each resource.
Signed-off-by: Elena Zioga <elena@arrikto.com>
* web-apps(back): Utilize list_events for listing notebook events
Utilize list_events function for listing notebook events in notebook.py
file.
Signed-off-by: Elena Zioga <elena@arrikto.com>
* web-apps(back): Utilize list_events for listing pvc events
Utilize list_events function for listing pvc events in pvc.py file.
Signed-off-by: Elena Zioga <elena@arrikto.com>
---------
Signed-off-by: Elena Zioga <elena@arrikto.com>
* web-apps(front): Remove rok references
Signed-off-by: Elena Zioga <elena@arrikto.com>
* web-apps(back): Remove rok references
Signed-off-by: Elena Zioga <elena@arrikto.com>
---------
Signed-off-by: Elena Zioga <elena@arrikto.com>
* vwa(front): Fix tests symLinks to Kubeflow common code
Add field preserveSymLinks in angular.json in order to link to
Kubeflow common library during tests.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* vwa(front): Fix unit tests failing
Fix unit tests failing for the following components: app, index,
index-default, index-rok, form-default, form-rok
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* vwa(front): Add delete button to Volume details page
In this commit:
- Add button for Delete action.
- Create Actions Service to handle delete action and implement it in
all components that use it.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* vwa(back): Get a PVC's events
Add logic in the backend to fetch the events of a PVC.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* vwa(front): Add EVENTS tab to Volume details page
Add Events tab to show the available events of the pvc in the form
of a table, in order to help a user with debugging the pvc.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* vwa(front): fix linting errors
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* vwa(back): Get a single PVC and the pods using it
Add logic in the backend to fetch a single PVC and the pods using
it.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* vwa(front): Create Volume details page with OVERVIEW tab
Create OVERVIEW tab to show key information about the PVC.
* vwa(front): Format files
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* jwa(back): Get a notebook's events
- Add logic in the backend to fetch the events of a notebook.
- Fix the authorization parameters in the get_notebook_events
function.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* jwa(front): Add EVENTS tab to Notebook details page
Add Events tab to show the available events of the notebook in the
form of a table, in order to help a user with debugging the
notebook.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* jwa(back): Get a notebook's logs
Add logic in the backend to fetch the logs of a notebook's
underlying pod.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* jwa(front): Add LOGS tab to Notebook details page
- Create Logs tab to show the logs of the notebook's underlying
pod.
- Create Logs Viewer component in Kubeflow common library exactly
like the one we have in MWA.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* jwa(back): Get a single notebook and its pod
In this commit:
- Add logic in the backend to fetch a single notebook and its
underlying pod.
- Make list_pods function in Kubeflow common code to accept a
label_selector parameter and use it to filter out results when
available.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* jwa(front): Create Notebook details page with OVERVIEW tab
In this commit:
- Create OVERVIEW tab to show key information about the notebook
and its underlying pod.
- Create Content List Item component in Kubeflow commonn library to
be able to encapsulate any content/component in the form of a
list item.
- Create Variables Group component in Kubeflow common library to
show groups of variables in the form of chips.
- Small UI tweak in Details list Item component from Kubeflow
common library in order to stop chips from overlapping with each
other.
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* Fix linting errors
* Include Kubeflow common library's new package-lock.json
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* jwa(front): Replace pod conditions with notebook status conditions
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* web-apps(back): Backend fetch pods fix
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
* jwa(back): Use gevent for gunicorn's workers
Since the backend only serves simple HTTP requests that need some K8s
data, instead of only have working processes we will also use greenlets.
This way we can have multiple user-space threads that will handle the
http requests.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Use cli to continuously build the UI
We extend the current package.json file to allow us to build and serve
the UI via Angular's dev server.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* wa(front): Handle all-namespaces
The common library will add an event handler to dashboard's lib when it
is emitting all the namespaces. By extension, the common namespace
service will be exposing a new observable that will be emitting the list
of all the namespaces, when the user selects this from the dashboard.
This way current apps won't break, since they will simply not handle
this new observable.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* wa(front): Handle dynamic table colums
The UIs will be showing a new column for the namespace only when the
user selects all-namespaces from the dashboard. The common code should
expose handlers for modifying the tables' config columns accordingly.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Make requests for all-namespaces
As a first iteration in order for the JWA to support showing Notebooks
from all namespaces it will be polling with batch requests. After all
the requests, for all the namespaces, succeed then the JWA will sort
them based on their name and show them in the table.
While this approach will be generating a significantly bigger amount of
requests, we chose this approach because:
1. It was simple to implement, since we only need to modify the frontend
2. We don't add any extra abstractions on the backend and keep the 1-1
relationship with the k8s api endpoints
3. To further optimize we now only look on K8s concepts and won't have
to adjust them to a custom backend implementation [2]
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* wa(back): Add helper for deserializing JSON obj
In some cases we might need to construct Python k8s lib objects from the
JSONs that are provided by clients. I.e. the UI will be sending a PVC
object in json format, so the backend will need to create the
corresponding client.V1PersistentVolumeClaim object and submit it.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Serialization helper
Add helper function for converting a k8s-client object into a dict that
can be sent as an HTTP response.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Add dry run to Notebooks and PVCs
The backend will need to be able to create objects with dry-run, in
order to ensure they are valid. The backend will need to check that both
the Notebook and the PVCs can be created beforehand.
This way we avoid the scenario where we create PVCs but the Notebook
fails to be created, and the PVCs are never garbage collected.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Update kubernetes to 0.17
In order to support dry-run we must use the 0.17 version of the Python
k8s client.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* wa(back): Extend api module to patch pvcs
The backend will need to be able to PATCH PVCs in order to set the
ownerReference to the Notebook that mounts the PVCs.
Ref: arrikto/dev/issues/386#issuecomment-856700392
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Work with new Volumes API
The backend API should not add any more layers of abstractions on top of
the K8s API. The backend should expect the client/UI to be sending the
entire PVC spec of a new PVC.
Refs: arrikto/dev/issues/386
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Add unittests for new volumes API
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(back): Extend the PVC info returned
We want to show both the access mode and size of the existing PVCs, when
a user clicks on the dropdown to select which PVC to mount.
The backend will need to provide this information to the frontend. We
don't want to send the K8s list of PVCs since this will result in a lot
of unnecessary data to be sent.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* jwa(front): Add proxy config for Rok
When developing the Rok flavor locally we will need to be able to open
the Rok chooser. This can be done by using Angular/webpack proxy to
bring the exposed rok service and the app under the same domain.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Remove card from form
The form of the app should not be a big card, but a normal form.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Install AceModule for yaml editing
Install AceModule to allow users to edit yamls of objects.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* wa(front): Change the styling of form sections
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Create common volume components
Component for:
* New PVC and configuring its spec
* Attaching an existing PVC in a Notebook
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Update Rok form for new Volume API
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa(front): Mark inputs as dirty when restoring Lab
When the UI autofills the form with values from a JupyterLab snapshot
then it should mark the touched fields as dirty. This way if a field has
errors the UI will make that input red.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Tasos Alexiou <tasos@arrikto.com>
* jwa: Update ConfigMap in manifests
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Fix format errors
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Introduce APP_SECURE_COOKIES var
Expose a new APP_SECURE_COOKIES env variable that will configure whether
the web apps should set Secure cookies or not.
This will allow the admins to configure the web apps to work when
Kubeflow is exposed over localhost/http.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Switch CSRF checks order
The order the backend makes the CSRF checks should be the following:
1. check if the CSRF cookie is present
2. check if the CSRF header is present
3. check if the CSRF cookie and header have the same value
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps: Set APP_SECURE_COOKIES to false in dev
When running the web apps via the makefiles in dev mode we will need to
explicitly set the APP_SECURE_COOKIES env var to False, since the app
will be served over http.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Introduce an APP_NO_AUTHNZ env var
The admin can use the APP_NO_AUTHNZ={True,False} to configure if the
application should perform authnz checks or not.
In case of False, then the app will not be expecting a logged in user, in the
`kubeflow-userid` header, and will not perform authorization checks using
SubjectAccessReviews.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Expose if Dashboard is connected
The NamespaceService will also provide an observable that informs
different parts of the app if the CentralDashboard is present.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* review: Use enumeration for Dashboard state
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* review: Move common vars to a settings module
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Export package code for wheel
When building the wheel we need to define Python packages
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Don't include init in root dir
We don't want to have an init file in the setup directory
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Fetch the correct default SC
The backend would only check for the annotation of the default
StorageClass but not if it's value would be true/false.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Fetch Pod logs
Extend the common backend libraries to fetch pod logs
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Add global variables.scss
Since we will need the page padding in multiple places we'll create a
scss file to store these values.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Export the Dialog module
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Rework the Toolbar component
* Add option to make button to be stroked
* Don't emit an event when a button is clicked. Expect a function which
will be executed from this component.
* Put all the buttons on the right to mimic Pipelines UI
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Make the app's background white
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Export commonly used modules
The mat-divider and mat-icon modules are oftenly used so we could
include them in the exports of the KubeflowModule.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Key-val should be on top in list
In the list that shows key-valu pairs in lines we want the key to remain
on top if the value has a big height. Previously the key-title would be
in the middle height of the value, which looked weird.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Add a Comment component for forms
This is used to explain different sections in the form.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Bar at the bottom for every form
We want our forms to have a bar on the bottom with CREATE CANCEL buttons
and the ability to view the yaml contents of a CR.
This component is only used for visualization. The logic component
should be handling this component via this component's inputs/outputs
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): General purpose css
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(front): Make k8s fields optional
We might need to instantiate a k8s objecti with only some of the
subfields, like only the spec and not metadata.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* remove redundant link class
Co-authored-by: Tasos Alexiou <tasos@arrikto.com>
* support fit-content for firefox as well
Co-authored-by: Tasos Alexiou <tasos@arrikto.com>
* firefox fit-content support
Co-authored-by: Tasos Alexiou <tasos@arrikto.com>
* use button instead of span
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Co-authored-by: Tasos Alexiou <tasos@arrikto.com>
* web-apps(back): Helper config functions
Introduce helper function for creating the config object for an app.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* twa: Update the build process
Update both the frontend and the backend of the Tensorboards web app to
follow the build/run process of the other web apps as well.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* twa: Update the README
Restructure the README to look like the JWA one. Also update the
instructions with the latest process for running the web app.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Add CSRF protection to the backend
The server of each crud-web-app will be setting an XSRF-TOKEN cookie to
the frontend. On each unsafe method (POST, PATCH etc) the backend will
check to make sure that the request:
* Contains an XSRF-TOKEN cookie
* Contains an X-XSRF-TOKEN header
* The value of the above values are the same
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* web-apps(back): Document the CSRF_SAMESITE env var
Add a new table in the README of the common code to include the ENV vars
that a user can set in any web app. In the future we should also extend
the README of every app with the supported ENV vars.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Update the backend
For the frontend to work properly we will need to add the following
changes to jupyter web app's backend as well as to the common backend
code:
* rename the references from `flask_rest_backend` to `crud_backend` in
the web app's backend code
* add a route for exposing GPU info. This way the UI will block users
from creating Notebooks with a GPU type that is not installed at all
in the cluster
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Update the common frontend library
New functionality added:
* An `Advanced Settings` button that can expand and shrink to
expose/hide more options in the form
* All validators will have a debounce time to make the input of
characters smoother
* Extend the Status types to allow start/stopped resources
* Extend the main table config to support a button [ ex CONNECT for
jupyter web app ]
* The http services should use relative URLs
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Update the frontend to utilize the common code
The bulk of the new frontend code. The folder structure is changed to
make it more clear what pages are used from the page.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* {Make,Docker}files
Add Makefile and Dockerfiles. Note that GCB build process needs to be
updated.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* README.md
Add a readme that explains how to build the app and have a development
environment.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* WA: Backend common: update the library
Update the common python wheel wrt:
* How to distinguish between dev and prod mode
* Extra routes for handling Notebooks
* Serving the index.html for every non api route (SPA)
* Add a STOPPED state to the possible Status values
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* JWA: Add the refactored backend
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* JWA: Backend: Add support for Affinity/Tolerations
* Extend the configuration yaml with default form values for the
affinity/tolerations
* Set them accordingly when the user submits a notebook
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Reviewed-by: Ilias Katsakioris <elikatsis@arrikto.com>
* Create Tensorboard web-app backend
Create the code for the Tensorboard web-app backend which
includes routes for GET, POST and DELETE requests.
The backend is created with Python/Flask, so it also uses
the common code from 'kubeflow.kubeflow.crud_backend'.
* Add 'get_age(k8s_object)' function to 'crud_backend' common code
It would be useful for all web apps of the 'crud-web-apps' folder
to return age information to their frontends.
As a result, 'get_age(k8s_object)' was added to the common code,
so that all web apps can use it.
Create a python module under the kubeflow.kubeflow package that will
be exposing common code and a base app the takes care of:
* Exceptions handling
* Common routes for serving static files and their cache control policy
* Authorization checks with SubjectAccessReview
* Authentication checks on the Kubeflow headers
* Common helper functions for dates, yaml parsing etc
* health/liveness probes
Backends that are written with Python/Flask should use this common code
in order for us to reduce code duplication and have our backends align
with our accepted practices.
Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
Pranav Pandit
Kimonas Sotirchos
Elena Zioga
Orfeas Kourkakis
Xiangquan Xiao
Benjamin Tan Wei Hao
Wendy Gaultier
Konstantinos Andriopoulos