pipelines/backend
Lingqing Gan 6c6b1e3a94
chore(backend): move v1 api to v1beta1 package and folder (#8376)
* move v1 api to v1beta1 package and folder

* fix test

* update README.md

* update README.md

* update readme
2022-10-26 07:11:30 +00:00
..
api chore(backend): move v1 api to v1beta1 package and folder (#8376) 2022-10-26 07:11:30 +00:00
conformance feat(conformance): containerize KFP conformance test. (#7738) 2022-05-23 19:31:01 +00:00
metadata_writer fix(backend): AIP-5634: Add in LRU cache and temp file rotation for metadata-writer. Relates to #4347 (#7199) 2022-02-15 21:42:06 +00:00
src chore(backend): move v1 api to v1beta1 package and folder (#8376) 2022-10-26 07:11:30 +00:00
test chore(backend): move v1 api to v1beta1 package and folder (#8376) 2022-10-26 07:11:30 +00:00
third_party_licenses feat(backend): Add ExecutionClient to access ExecutionSpec (#8050) 2022-08-06 08:05:50 +00:00
Dockerfile Use STABLE base image to fix CVE-2022-2068 2022-08-05 06:07:49 +00:00
Dockerfile.cacheserver chore(backend): Check for forbidden license during image build and in presubmit (#7210) 2022-01-26 00:30:43 +00:00
Dockerfile.conformance feat(conformance): containerize KFP conformance test. (#7738) 2022-05-23 19:31:01 +00:00
Dockerfile.persistenceagent chore(backend): Check for forbidden license during image build and in presubmit (#7210) 2022-01-26 00:30:43 +00:00
Dockerfile.scheduledworkflow chore(backend): Check for forbidden license during image build and in presubmit (#7210) 2022-01-26 00:30:43 +00:00
Dockerfile.viewercontroller chore(backend): Check for forbidden license during image build and in presubmit (#7210) 2022-01-26 00:30:43 +00:00
Dockerfile.visualization feat: upgrade TFX to 1.2.0 (#6375) 2021-08-18 01:50:37 -07:00
Makefile chore(backend): Merge v2 code into backend/src/ (#7173) 2022-01-25 01:21:24 -08:00
OWNERS chore: Update backend/OWNERS (#8369) 2022-10-14 20:42:04 +00:00
README.md chore(backend): format backend code and add style guide (#8140) 2022-08-17 12:50:52 -07:00
requirements.in feat(backend): Upgrade MLMD to 1.4.0 (#6910) 2021-11-26 20:42:07 +08:00
requirements.txt test: fix test failing to find kfp (#7108) 2021-12-23 12:37:43 +00:00
update_requirements.sh feat: upgrade TFX to 1.2.0 (#6375) 2021-08-18 01:50:37 -07:00

README.md

This directory contains code for the components that comprise the Kubeflow Pipelines backend.

Building & Testing

To run all unittests for backend:

go test -v -cover ./backend/...

The API server itself can be built using:

go build -o /tmp/apiserver backend/src/apiserver/*.go

Code Style

Use gofmt package to format your .go source files. There is no need to format the swagger generated go clients, so only run the following command in ./backend/src and ./backend/test folder.

go fmt ./...

For more information, see this blog.

Building APIServer image locally

The API server image can be built from the root folder of the repo using:

export API_SERVER_IMAGE=api_server
docker build -f backend/Dockerfile . --tag $API_SERVER_IMAGE

Deploy APIServer with the image you own build

Run

kubectl edit deployment.v1.apps/ml-pipeline -n kubeflow

You'll see the field reference the api server docker image. Change it to point to your own build, after saving and closing the file, apiserver will restart with your change.

Building client library and swagger files

After making changes to proto files, the Go client libraries, Python client libraries and swagger files need to be regenerated and checked-in. Refer to backend/api for details.

Updating licenses info

  1. Install go-licenses tool and refer to its documentation for how to use it.

  2. Run the tool to update all licenses:

    make all
    

Visualization Server Instructions

Updating python dependencies

pip-tools is used to manage python dependencies. To update dependencies, edit requirements.in and run ./update_requirements.sh to update and pin the transitive dependencies.

Building conformance tests (WIP)

Run

docker build . -f backend/Dockerfile.conformance -t <tag>