Commit Graph

15 Commits

Author SHA1 Message Date
Niklas Hansson 2a8333619e fix(backend): remove Bazel from building the API. Part of #3250 (#4906)
* updated to remove bazel

* rename@

* script to run in container

* update the generation

* updated docker image

* changed name

* updated the code

* regenerated API

* fix env variables

* updated version of go swagger

* rerun generate with updated package

* added healthz

* typo with folders fixed

* changed version

* set version of protoc compiler

* test if version is correct

* test version

* changed version

* updated version agian

* test version

* the latest test

* updated docker image

* fixed some stuff

* new test

* tested other version

* new tests

* changed swagger

* new test

* updated versions

* missed docker file

* updated files@

* change back

* updated after feedback@

* clean up

* remove license and install binary

* update and check differences

* completed rebase

* go mod tidy

* updated based upon work of boby

* remove code commited by misstake

* added by misstake

* updated after feedback

* futher updates after feedback

* final updates from feedback

* switch to Makefile

* update

* fix non-root execution

* clean up, fix release script

* fix swagger

* fix healthz endpoint & regenerate python client

* Delete sample.py.tar.gz

Co-authored-by: Yuan Gong <gongyuan94@gmail.com>
Co-authored-by: Yuan (Bob) Gong <4957653+Bobgy@users.noreply.github.com>
2021-02-25 19:31:01 +08:00
Yuan (Bob) Gong 3d40bba9a3
chore(backend): clean up backend code generation (#5116)
* chore(backend): tidy go.mod and update tools.go

* go install, instead of go get

* fix problems reported by go vet

* simplify some ide reported redundant syntax

* license is not required for generated code

* remove licenses for generated code

* cleanup

* remove license more

* rm unused BUILD.bazel files

* fixed generate_api.sh

* reimport error.proto
2021-02-09 07:10:57 -08:00
Yang Pan c484cfa46c chore(release): bumped version to 1.3.0 2021-01-07 00:39:26 -08:00
jingzhang36 286492591b
More documentation on backend API methods (#3758)
* list experiment desc

* changes should be made in proto

* add comments and descriptions

* comments/descriptions in run.proto

* comments in job.proto and pipeline.proto

* try starting a new line

* newline doesnt help

* add swagger gen'ed file

* address comments

* regenerate json and client via swagger

* address comments

* regenerate go_http_client and swagger from proto

* two periods

* re-generate
2020-06-01 12:26:25 +08:00
jingzhang36 56d519e9f7
Add archive experiment feature in backend (#3359)
* add new field in db schema and api schema

* auto genereted types for experiment storage state

* add archive and unarchive methods to backend for experiments.

* auto generated archive/unarchive methods for epxeriments

* add archive and unarchive to client

* set proper storage state when creating experiment

* retrieve storage state when we get/list epxeriment(s)

* change expection in test to have storage state

* add storage state in resource manager test

* revise experiemnt server test

* revise api converter test

* integration test of experiment archive

* archive/unarchive experiment affect the storage state of runs in it

* test all the runs in archive/unarchive experiment

* test all runs are archived/unarchived with their experiment in experiment server

* integration test

* integration test: value type mismatch in assertion

* unused import; default value for storage state

* autogen code for frontend

* reorder the fields in api experiment schema

* switch the position of the two enum to verify a hypothesis

* Put a place hodler to prevent any valid item to take the value 0

* Get rid of the place holder since the cause of issue related to value 0 is found and fixed.

* The returned api experiment now has storage state field

* create experiment return doesn't contain storege state

* Cleanup needs to clean runs and pipeliens now

* a missing client

* use resource reference as fileter instead of experiment uuid

* use same namespace in archive unit test

* Leave archive/unarchive experiment integration test to a separate PR

* also need to update jobs when experiments are archived

* Change of unarchiving logic. When experiment is unarchived, jobs/runs in
it stay archived

* add unit test for the job status in archived/unarchived experiment

* change archive state to 3 value enum; add experiment integration test

* make archive state 3 value enum to avoid 0 value mapped to available; add integration test

* run swagger autogen

* fix an expected value

* fix experiment server test

* add job check in experiment server test

* update job crds

* fix a typo

* remove accidentally included irrelevant changes
2020-04-16 07:31:09 +08:00
Chen Sun 2f1dbbedd2
Experiment API change (#3198) 2020-03-06 09:17:23 -08:00
jingzhang36 4c491823a0
Update auto-generated files' license date (generated by swagger-gen) (#2998) 2020-02-07 14:44:50 +08:00
IronPan 06e4dc6604
Add doc for API (#1406)
* initial update

* initial update

* update

* add doc

* update
2019-05-30 17:48:50 -07:00
Ajay Gopinathan 5f1b41171f Fix API package names and regenerate checked-in proto files. (#1404)
* Fix API package names and regenerate checked-in proto files. Also bump version of GRPC gateway used.

* Fix BUILD.bazel file for api as well.

* Update Bazel version
2019-05-30 11:26:28 -07:00
Yasser Elsayed ec4d7e8bd3 Return resource count from ListXXX calls (#595)
* add count to protos and libs

* close db rows before second query

* count -> total_size

* int32 -> int

* move scan count row to util

* add comments

* add logs when transactions fail

* dedup from and where clauses

* simplify job count query

* job count queries

* run count queries

* add job_store total size test

* added tests for list util

* pr comments

* list_utils -> list

* fix clients and fake clients to support TotalSize

* added TotalSize checks in api integration tests
2019-01-31 11:15:54 -08:00
Ajay Gopinathan 163545b370 Use Bazel to build the entire backend and perform API code generation (#609)
* Use Bazel to build the entire backend.

This also uses Bazel to generate code from the API definition in the
proto files.

The Makefile is replaced with a script that uses Bazel to first generate
the code, and then copy them back into the source tree.

Most of the BUILD files were generated automatically using Gazelle.

* Fix indentation in generate_api.sh

* Clean up WORKSPACE

* Add README for building/testing backend.

Also fix the missing licenses in the generated proto files.

* Add license to files under go_http_client
2019-01-04 17:17:20 -08:00
Ajay Gopinathan 8616398602 Encode filter parameter as a base64-encoded JSON string in List requests (#563)
* Make all ListXXX operations use POST instead of GET.

Generate new swagger definitions and use these to generate the frontend
APIs using `npm run apis`.

This is to support filtering in List requests, as the current
grpc-gateway swagger generator tool does not support repeated fields in
requests used in GET endpoints.

* Use base64-encoded JSON-stringified version of Filter instead.

This lets us keep filter as a simple parameter in the ListXXX requests,
and gets around having to use POST for List requests.

* refactor filter parsing to parseAPIFilter and add tests

* Hack to ensure correct Swagger definitions are generated for Filter.

* Fix merge conflicts with master after rebase

* fix indentation

* Fix hack so frontend apis compile.

* print failing experiments

* try print again.

* revert experiment_api_test

* Use StdEncoding for base64 encoding

* Fix nil pointer dereference error caused err variable shadowing
2019-01-02 13:03:14 -08:00
Ajay Gopinathan 0c0d8a2b91 Add filtering ability for all backend API ListXXX requests (#537)
* WIP: Add filter package with tests.

* Add tests for IN predicate.

* Add listing functions

* Try updating list experiments

* Cleanup and finalize list API.

Add tests for list package, and let ExperimentStore use this new API.
Update tests for the latter as well.

* Add comments. BuildSQL -> AddToSelect for flexibility

* Run dep ensure

* Add filter proto to all other resources

* Add filtering for pipeline server

* Add filtering for job server

* Add filtering for run server

* Try to fix integration tests
2018-12-14 00:18:31 -08:00
Yang Pan b494e3daa0 Add integration tests for API servers (#112)
* add pipeline e2e test back

* delete samples

* more changes to pipeline test

* add delete run/experiment

* update test name

* stage

* add api

* fii

* add tensorboard routing rule (#143)

* add tensorboard routing rule

* rename tb routing rule

* address comments

* remove debugger

* fix url prefix

* more fixes

* update tests

* revert debugging

* update

* update test

* fix

* update test

* fix

* fix pipeline tests

* exp

* update run

* update jobs
2018-11-09 20:39:15 -08:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00