d482698e41
* Make pipelineFolder a configurable param * Can config the secure flag and region config for api-server object store * Updated minio client to use a chained credential provider so that IAM can be used. * fix bug in passing secure flag to minio * accesskey and secretkey for minio client is now optional. * Expose GetPipelineKey instead of giving objectStore a default folder path because there are eother functions that need to access the store directly. * fix fake bad object store. * region shld be provided when attempting to create a bucket. * Rename pipelineFolder to pipelinePath. |
||
|---|---|---|
| .. | ||
| api | ||
| metadata_writer | ||
| src | ||
| test | ||
| Dockerfile | ||
| Dockerfile.bazel | ||
| Dockerfile.persistenceagent | ||
| Dockerfile.scheduledworkflow | ||
| Dockerfile.viewercontroller | ||
| Dockerfile.visualization | ||
| OWNERS | ||
| README.md | ||
| build_api_server.sh | ||
README.md
This directory contains code for the components that comprise the Kubeflow Pipelines backend.
Building & Testing
All components can be built using Bazel. To build
everything under backend, run: bazel build --action_env=PATH --define=grpc_no_ares=true //backend/...
To run all tests: bazel test --action_env=PATH --define=grpc_no_ares=true //backend/...
The API server itself can only be built/tested using Bazel. The following
commands target building and testing just the API server. bazel build --action_env=PATH --define=grpc_no_ares=true backend/src/apiserver/... bazel test --action_env=PATH --define=grpc_no_ares=true backend/src/apiserver/...
Building APIServer Image using Remote Build Execution
If you are a dev in the Kubeflow Pipelines team, you can use Remote Build Execution Service to build the API Server image using Bazel with use of a shared cache for speeding up the build. To do so, execute the following command:
./build_api_server.sh -i gcr.io/cloud-ml-pipelines-test/api-server:dev
Building Go client library and swagger files
After making changes to proto files, the Go client libraries and swagger files need to be regenerated and checked-in. The backend/api/generate_api.sh script takes care of this.
Updating BUILD files
As the backend is written in Go, the BUILD files can be updated automatically
using Gazelle. Whenever a Go file
is added or updated, run the following to ensure the BUILD files are updated as
well: bazel run //:gazelle
If a new external Go dependency is added, or an existing one has its version
bumped in the go.mod file, ensure the BUILD files pick this up by updating the
WORKSPACE go_repository rules using the following command: bazel run //:gazelle -- update-repos --from_file=go.mod