939e1e22ca
* build: Update components makefiles for building
We'll create a top-level Makefile under components/ dir
that has the following rules:
* build-all:
To build all images locally
* push-all:
* We can use a specific REGISTRY and retag the images
* Push all the images
This top-level Makefile will run the sub-Makefiles that every
component has for building and pushing the images.
We modified every sub-Makefile as follows:
* We don't use a registry in images by default
* Removed unused rules and vars
* Use the --dirty flag of git describe in TAG
--dirty[=<mark>]
Describe the working tree. It means describe HEAD and
appends <mark> (-dirty by default) if the working tree
is dirty.
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Create makefiles for notebook servers
The common starting point of building the notebook-server
images are the following 4 *base* images:
- base
- codeserver
- jupyter
- rstudio
All other server images need to build on top of them. We'll
dynamically pass the base images in every Dockerfile by
using an ARG IMG. We can set the value of this ARG during
docker build with the --build-arg CLI argument.
This way we build both the base images with a tag locally,
and then we pass that image as arg via the Makefile and build the rest
So we modified our building procedure as follows:
1. Build the base image since everything starts from there
2. Pass the base image as an ARG in the Dockerfiles of
jupyter, codeserver, rstudio images and build on top
3. Pass the base images in all other server images and build
on top
For that we will:
1. Create a Makefile for each of the notebook servers, in each folder
a. Each makefile will be responsible for building the bases and use args for passing them on
2. Use the central Makefile to call each Makefile from above
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* cherry-pick: Notebook server upstream fixes
Relevant upstream PR: https://github.com/kubeflow/kubeflow/pull/6466/files
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Fix Makefiles
- Remove build-gcp and build-gcr rules as we don't use them anywhere in
the project
- Fix code conficts
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Fix Dockerfiles for notbook-server images
We remove the previous logic of using already built images as bases.
The users must use only the Makefiles to pass the appropriate BASE_IMG
and build the images correctly.
Thus, we have Makefiles everywhere that:
- Can build any base image
- If an image requires another notebook base, then we first build that one using its makefile,
and then use it as docker ARG for building the next one
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Fix notebook-controller Makefile
Removed a misplaced "|" char that breaks the Makefile
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Update GH action workflows
* Update workflow for notebook-server images:
- Add a step for building all images by using the
central-Makefile under components/example-notebook-servers/ dir.
- Add a step for pushing all images by using the
central-Makefile under components/example-notebook-servers/ dir.
* Update workflow for all Kubeflow images:
- Add a step for building & pushing all images by using the
top-level Makefile under components/ dir.
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Remove completely ECR references from images
Replace everywhere the "public.ecr.aws/j1r0q0g6/notebooks/notebook-servers"
prefix with "kubeflownotebookswg"
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
* build: Fix GH actions for Kubeflow components
Fix GH actions to use the updated make rules
when building the Kubeflow component images.
Remove the "docker.io" prefix when building with
GH action workflows
Signed-off-by: Apotolos Gerakaris <apoger@arrikto.com>
|
||
|---|---|---|
| .. | ||
| api/v1alpha1 | ||
| config | ||
| controllers | ||
| hack | ||
| .dockerignore | ||
| .gitignore | ||
| Dockerfile | ||
| Makefile | ||
| OWNERS | ||
| PROJECT | ||
| README.md | ||
| go.mod | ||
| go.sum | ||
| main.go | ||
| skaffold.yaml | ||
README.md
GSoC 2020 - TENSORBOARD CONTROLLER
Related Closed Issues
-
Tensorboard controller creates servers that always mount user-gcp-sa secret #5065
-
Tensorboard CR doesn't contain information about the Tensorboard Server being ready or not #5166
Related Pull Requests
-
Add scheduling functionality for Tensorboard servers that use RWO PVCs as log storages #5218
-
Add functionality to inform TWA frontend about the status of Tensorboard servers #5259
Prequisites to build and run the controller:
-
GO
-
Docker
-
kustomize
-
kubectl
RUN TENSORBOARD CONTROLLER LOCALLY
Steps:
-
Clone the repository
-
Change directories to
components/tensorboard-controller -
Generate and install manifests and build the controller:
make install -
Run the controller locally:
make run
If you want to enable the scheduling functionality for Tensorboard servers that use ReadWriteOnce PVCs as log storages, then set the RWO_PVC_SCHEDULING to true and run: RWO_PVC_SCHEDULING="true" make run
BUILD TENSORBOARD CONTROLLER IMAGE AND DEPLOY TO CLUSTER
-
Clone the repository
-
Change directories to
components/tensorboard-controller -
Generate and install manifests and build the controller:
make manifests -
Build and push the docker image:
make docker-build docker-push IMG=YOUR_IMAGE_NAME -
Deploy the Tensorboard controller:
make deploy IMG=YOUR_IMAGE_NAME
If you want to enable the scheduling functionality for Tensorboard servers that use ReadWriteOnce PVCs as log storages, then:
-
Change directories to
components/tensorboard-controller/config/manager -
Modify the
manager.yamlfile by navigating to thedeployment.spec.template.specfield and manually setting the value of theRWO_PVC_SCHEDULINGenv var to"true"in the manager container. -
Run:
make deploy IMG=YOUR_IMAGE_NAME