5399585b6a
|
||
|---|---|---|
| .. | ||
| api | ||
| conformance | ||
| metadata_writer | ||
| src | ||
| test | ||
| third_party_licenses | ||
| Dockerfile | ||
| Dockerfile.cacheserver | ||
| Dockerfile.conformance | ||
| Dockerfile.driver | ||
| Dockerfile.launcher | ||
| Dockerfile.persistenceagent | ||
| Dockerfile.scheduledworkflow | ||
| Dockerfile.viewercontroller | ||
| Dockerfile.visualization | ||
| Makefile | ||
| OWNERS | ||
| README.md | ||
| requirements.in | ||
| requirements.txt | ||
| update_requirements.sh | ||
README.md
This directory contains code for the components that comprise the Kubeflow Pipelines backend.
Building & Testing
To run all unittests for backend:
go test -v -cover ./backend/...
The API server itself can be built using:
go build -o /tmp/apiserver backend/src/apiserver/*.go
Code Style
Backend codebase follows the Google's Go Style Guide. Please, take time to get familiar with the best practices. It is not intended to be exhaustive, but it often helps minimizing guesswork among developers and keep codebase uniform and consistent.
We use golangci-lint tool that can catch common mistakes locally (see detailed configuration here). It can be conveniently integrated with multiple popular IDEs such as VS Code or Vim.
Finally, it is advised to install pre-commit in order to automate linter checks (see configuration here)
Building APIServer image locally
The API server image can be built from the root folder of the repo using:
export API_SERVER_IMAGE=api_server
docker build -f backend/Dockerfile . --tag $API_SERVER_IMAGE
Deploy APIServer with the image you own build
Run
kubectl edit deployment.v1.apps/ml-pipeline -n kubeflow
You'll see the field reference the api server docker image. Change it to point to your own build, after saving and closing the file, apiserver will restart with your change.
Building client library and swagger files
After making changes to proto files, the Go client libraries, Python client libraries and swagger files need to be regenerated and checked-in. Refer to backend/api for details.
Updating licenses info
-
Install go-licenses tool and refer to its documentation for how to use it.
-
Run the tool to update all licenses:
make all
Updating python dependencies
pip-tools is used to manage python
dependencies. To update dependencies, edit requirements.in
and run ./update_requirements.sh to update and pin the transitive
dependencies.
Visualization Server Instructions
Updating python dependencies
pip-tools is used to manage python
dependencies. To update dependencies, edit requirements.in
and run ./update_requirements.sh to update and pin the transitive
dependencies.
Building conformance tests (WIP)
Run
docker build . -f backend/Dockerfile.conformance -t <tag>