Commit Graph

17 Commits

Author SHA1 Message Date
Niklas Hansson 2a8333619e fix(backend): remove Bazel from building the API. Part of #3250 (#4906)
* updated to remove bazel

* rename@

* script to run in container

* update the generation

* updated docker image

* changed name

* updated the code

* regenerated API

* fix env variables

* updated version of go swagger

* rerun generate with updated package

* added healthz

* typo with folders fixed

* changed version

* set version of protoc compiler

* test if version is correct

* test version

* changed version

* updated version agian

* test version

* the latest test

* updated docker image

* fixed some stuff

* new test

* tested other version

* new tests

* changed swagger

* new test

* updated versions

* missed docker file

* updated files@

* change back

* updated after feedback@

* clean up

* remove license and install binary

* update and check differences

* completed rebase

* go mod tidy

* updated based upon work of boby

* remove code commited by misstake

* added by misstake

* updated after feedback

* futher updates after feedback

* final updates from feedback

* switch to Makefile

* update

* fix non-root execution

* clean up, fix release script

* fix swagger

* fix healthz endpoint & regenerate python client

* Delete sample.py.tar.gz

Co-authored-by: Yuan Gong <gongyuan94@gmail.com>
Co-authored-by: Yuan (Bob) Gong <4957653+Bobgy@users.noreply.github.com>
2021-02-25 19:31:01 +08:00
Yuan (Bob) Gong 3d40bba9a3
chore(backend): clean up backend code generation (#5116)
* chore(backend): tidy go.mod and update tools.go

* go install, instead of go get

* fix problems reported by go vet

* simplify some ide reported redundant syntax

* license is not required for generated code

* remove licenses for generated code

* cleanup

* remove license more

* rm unused BUILD.bazel files

* fixed generate_api.sh

* reimport error.proto
2021-02-09 07:10:57 -08:00
Yang Pan c484cfa46c chore(release): bumped version to 1.3.0 2021-01-07 00:39:26 -08:00
Yuan (Bob) Gong c38f36cdc1
[Backend] Add service account field to run and job api objects (#3649)
* Add service account field to run and job api objects

* Update description

* Fix field casing

* Add comment about the next field number
2020-04-29 22:58:17 -07:00
jingzhang36 4c491823a0
Update auto-generated files' license date (generated by swagger-gen) (#2998) 2020-02-07 14:44:50 +08:00
IronPan a9602fbc3f
Add API to rerun the pipeline (#1720)
* add resubmit proto

* add compiled code

* fix

* add resubmit proto

* add

* refactor

* update builder

* refactor

* refactor

* refactor

* refactor

* refactor

* refactor

* add test

* add test

* add test

* add test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* address comments

* add comments

* change request body def

* recompile api

* retry instead of resubmit

* update test

* update test

* fix tests

* fix tests

* fix tests

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* add error handling

* reorder the call

* remove logic to update the database entry

* add mock

* add tests for rerousrce manager

* update error handling logic

* fix tests

* address comments
2019-08-07 13:59:06 -07:00
Ajay Gopinathan 5f1b41171f Fix API package names and regenerate checked-in proto files. (#1404)
* Fix API package names and regenerate checked-in proto files. Also bump version of GRPC gateway used.

* Fix BUILD.bazel file for api as well.

* Update Bazel version
2019-05-30 11:26:28 -07:00
IronPan 086d4763d9 Surface workflow finished time in list run API (#1122)
* add finished time for list runs

* add finished time for list runs

* fix tests

* add finished time for list runs

* Update run.proto

* address comments

* make query more robust

* fix e2e test

* fix e2e test
2019-04-10 23:02:10 -07:00
Alexey Volkov f98ec68488 Added the ability to terminate a run (#528)
* Added the terminate run command to backend and CLI.
No generated files for now.

* Added the generated files.

* Moved the code to run_store.go
Now the call chain is run_client->run_server->resource_manager->run_store

* Using the backoff package for retries.

* Trying to update run status in the DB to "Terminating" before patching the workflow.

* Stopped using the Argo errors module.

* Fixed the compilation errors due to recent backend changes.

* RILEY - WIP - Implementation of workflow_fake.go and first test

Added successful test in resource_manager_test.go and completed, barring nits and conventions, the implementation of Patch() and isTerimated() within workflow_fake.go

Additional tests and lots of clean-up are still necessary

* Adds a few more tests to resource_manager_test and cleans up a bit

* Further clean up. Stopped using squirrel for UPDATE query. Added run_store_tests

* Adds terminate run integration test

* Undo changes to go.sum

* Fixes path to long-running.yaml in integration test

* Allow runs with no Conditions to be terminated
2019-03-21 18:00:18 -07:00
Yasser Elsayed ec4d7e8bd3 Return resource count from ListXXX calls (#595)
* add count to protos and libs

* close db rows before second query

* count -> total_size

* int32 -> int

* move scan count row to util

* add comments

* add logs when transactions fail

* dedup from and where clauses

* simplify job count query

* job count queries

* run count queries

* add job_store total size test

* added tests for list util

* pr comments

* list_utils -> list

* fix clients and fake clients to support TotalSize

* added TotalSize checks in api integration tests
2019-01-31 11:15:54 -08:00
Yasser Elsayed df635af409 Support filtering on storage state (#629)
* filter on storage state

* fix case

* Update run.go

* storageState -> storage_state

* ignore generated ts client libs

* remove url import to make tslint happy
2019-01-11 11:01:01 -08:00
Ajay Gopinathan 163545b370 Use Bazel to build the entire backend and perform API code generation (#609)
* Use Bazel to build the entire backend.

This also uses Bazel to generate code from the API definition in the
proto files.

The Makefile is replaced with a script that uses Bazel to first generate
the code, and then copy them back into the source tree.

Most of the BUILD files were generated automatically using Gazelle.

* Fix indentation in generate_api.sh

* Clean up WORKSPACE

* Add README for building/testing backend.

Also fix the missing licenses in the generated proto files.

* Add license to files under go_http_client
2019-01-04 17:17:20 -08:00
Ajay Gopinathan 8616398602 Encode filter parameter as a base64-encoded JSON string in List requests (#563)
* Make all ListXXX operations use POST instead of GET.

Generate new swagger definitions and use these to generate the frontend
APIs using `npm run apis`.

This is to support filtering in List requests, as the current
grpc-gateway swagger generator tool does not support repeated fields in
requests used in GET endpoints.

* Use base64-encoded JSON-stringified version of Filter instead.

This lets us keep filter as a simple parameter in the ListXXX requests,
and gets around having to use POST for List requests.

* refactor filter parsing to parseAPIFilter and add tests

* Hack to ensure correct Swagger definitions are generated for Filter.

* Fix merge conflicts with master after rebase

* fix indentation

* Fix hack so frontend apis compile.

* print failing experiments

* try print again.

* revert experiment_api_test

* Use StdEncoding for base64 encoding

* Fix nil pointer dereference error caused err variable shadowing
2019-01-02 13:03:14 -08:00
Yasser Elsayed 549a366c39 Support archiving/unarchiving runs on the backend (#552)
* skip integration tests when unit test flag is set to true

* wip

* add StorageState enum to proto

* add StorageState to model

* archive proto/model changes

* wip archive endpoint

* wip adding tests

* archive test

* unarchive proto and implementation

* cleanup

* make storage state required, with a default value

* remove unspecified value from storage state enum

* pr comments

* pr comments

* fix archive/unarchive endpoints, add api integration test

* typo
2018-12-19 14:01:06 -08:00
Ajay Gopinathan 0c0d8a2b91 Add filtering ability for all backend API ListXXX requests (#537)
* WIP: Add filter package with tests.

* Add tests for IN predicate.

* Add listing functions

* Try updating list experiments

* Cleanup and finalize list API.

Add tests for list package, and let ExperimentStore use this new API.
Update tests for the latter as well.

* Add comments. BuildSQL -> AddToSelect for flexibility

* Run dep ensure

* Add filter proto to all other resources

* Add filtering for pipeline server

* Add filtering for job server

* Add filtering for run server

* Try to fix integration tests
2018-12-14 00:18:31 -08:00
Yang Pan b494e3daa0 Add integration tests for API servers (#112)
* add pipeline e2e test back

* delete samples

* more changes to pipeline test

* add delete run/experiment

* update test name

* stage

* add api

* fii

* add tensorboard routing rule (#143)

* add tensorboard routing rule

* rename tb routing rule

* address comments

* remove debugger

* fix url prefix

* more fixes

* update tests

* revert debugging

* update

* update test

* fix

* update test

* fix

* fix pipeline tests

* exp

* update run

* update jobs
2018-11-09 20:39:15 -08:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00