Commit Graph

172 Commits

Author SHA1 Message Date
IronPan 36338774f1
update the API doc description (#1410)
* add single json file

* update doc

* update doc
2019-05-31 13:31:33 -07:00
IronPan 14c66f0174
add single json file (#1408) 2019-05-30 18:02:29 -07:00
IronPan 75cfab3d2a
Delete kfp_api.html 2019-05-30 17:58:51 -07:00
IronPan 89954f831e
Delete kfp_api_single_file.swagger.json 2019-05-30 17:58:02 -07:00
IronPan 06e4dc6604
Add doc for API (#1406)
* initial update

* initial update

* update

* add doc

* update
2019-05-30 17:48:50 -07:00
Ajay Gopinathan 5f1b41171f Fix API package names and regenerate checked-in proto files. (#1404)
* Fix API package names and regenerate checked-in proto files. Also bump version of GRPC gateway used.

* Fix BUILD.bazel file for api as well.

* Update Bazel version
2019-05-30 11:26:28 -07:00
Alexey Volkov 467adb5a3b SDK - Separated the generated api client package (#1214)
* SDK - Separated the generated api client package

* Splitting the package build scripts

* Pinning the API client package version

* Moved import kfp_server_api to the top of the file

* Added the Mac OS X prerequisite install instructions

* Moved the build_kfp_server_api_python_package.sh script to the backend dir

* Updated the dependency version span
2019-04-29 15:49:37 -07:00
Alexey Volkov b795a9a899 Swagger - Specifying content types on the route level (#1225) 2019-04-25 14:00:32 -07:00
IronPan 086d4763d9 Surface workflow finished time in list run API (#1122)
* add finished time for list runs

* add finished time for list runs

* fix tests

* add finished time for list runs

* Update run.proto

* address comments

* make query more robust

* fix e2e test

* fix e2e test
2019-04-10 23:02:10 -07:00
Alexey Volkov f98ec68488 Added the ability to terminate a run (#528)
* Added the terminate run command to backend and CLI.
No generated files for now.

* Added the generated files.

* Moved the code to run_store.go
Now the call chain is run_client->run_server->resource_manager->run_store

* Using the backoff package for retries.

* Trying to update run status in the DB to "Terminating" before patching the workflow.

* Stopped using the Argo errors module.

* Fixed the compilation errors due to recent backend changes.

* RILEY - WIP - Implementation of workflow_fake.go and first test

Added successful test in resource_manager_test.go and completed, barring nits and conventions, the implementation of Patch() and isTerimated() within workflow_fake.go

Additional tests and lots of clean-up are still necessary

* Adds a few more tests to resource_manager_test and cleans up a bit

* Further clean up. Stopped using squirrel for UPDATE query. Added run_store_tests

* Adds terminate run integration test

* Undo changes to go.sum

* Fixes path to long-running.yaml in integration test

* Allow runs with no Conditions to be terminated
2019-03-21 18:00:18 -07:00
Yasser Elsayed ec4d7e8bd3 Return resource count from ListXXX calls (#595)
* add count to protos and libs

* close db rows before second query

* count -> total_size

* int32 -> int

* move scan count row to util

* add comments

* add logs when transactions fail

* dedup from and where clauses

* simplify job count query

* job count queries

* run count queries

* add job_store total size test

* added tests for list util

* pr comments

* list_utils -> list

* fix clients and fake clients to support TotalSize

* added TotalSize checks in api integration tests
2019-01-31 11:15:54 -08:00
Yasser Elsayed df635af409 Support filtering on storage state (#629)
* filter on storage state

* fix case

* Update run.go

* storageState -> storage_state

* ignore generated ts client libs

* remove url import to make tslint happy
2019-01-11 11:01:01 -08:00
Ajay Gopinathan 5a9e3ff14b Add IS_SUBSTRING operator for use in API resource filtering. (#645)
* Add IS_SUBSTRING operator for use in API resource filtering.

This should allow substring matches on fields like names and labels and
so on.

Also bump the version of Mastermind/squirrel so we get the new 'like'
operator for use when building the SQL query.

Additionally, I also had to fix the generate_api.sh script which had a
bug (it modified the wrong file permissions before), and add a dummy
service to generate Swagger definitions for the Filter itself (this was
a hack in the previous Makefile that I lost when we moved to Bazel).

* Add comments for DummyFilterService

* Add more comments

* change errors returned

* fix import
2019-01-08 18:16:12 -08:00
Ajay Gopinathan 163545b370 Use Bazel to build the entire backend and perform API code generation (#609)
* Use Bazel to build the entire backend.

This also uses Bazel to generate code from the API definition in the
proto files.

The Makefile is replaced with a script that uses Bazel to first generate
the code, and then copy them back into the source tree.

Most of the BUILD files were generated automatically using Gazelle.

* Fix indentation in generate_api.sh

* Clean up WORKSPACE

* Add README for building/testing backend.

Also fix the missing licenses in the generated proto files.

* Add license to files under go_http_client
2019-01-04 17:17:20 -08:00
Ajay Gopinathan 8616398602 Encode filter parameter as a base64-encoded JSON string in List requests (#563)
* Make all ListXXX operations use POST instead of GET.

Generate new swagger definitions and use these to generate the frontend
APIs using `npm run apis`.

This is to support filtering in List requests, as the current
grpc-gateway swagger generator tool does not support repeated fields in
requests used in GET endpoints.

* Use base64-encoded JSON-stringified version of Filter instead.

This lets us keep filter as a simple parameter in the ListXXX requests,
and gets around having to use POST for List requests.

* refactor filter parsing to parseAPIFilter and add tests

* Hack to ensure correct Swagger definitions are generated for Filter.

* Fix merge conflicts with master after rebase

* fix indentation

* Fix hack so frontend apis compile.

* print failing experiments

* try print again.

* revert experiment_api_test

* Use StdEncoding for base64 encoding

* Fix nil pointer dereference error caused err variable shadowing
2019-01-02 13:03:14 -08:00
Yasser Elsayed 549a366c39 Support archiving/unarchiving runs on the backend (#552)
* skip integration tests when unit test flag is set to true

* wip

* add StorageState enum to proto

* add StorageState to model

* archive proto/model changes

* wip archive endpoint

* wip adding tests

* archive test

* unarchive proto and implementation

* cleanup

* make storage state required, with a default value

* remove unspecified value from storage state enum

* pr comments

* pr comments

* fix archive/unarchive endpoints, add api integration test

* typo
2018-12-19 14:01:06 -08:00
Ajay Gopinathan 0c0d8a2b91 Add filtering ability for all backend API ListXXX requests (#537)
* WIP: Add filter package with tests.

* Add tests for IN predicate.

* Add listing functions

* Try updating list experiments

* Cleanup and finalize list API.

Add tests for list package, and let ExperimentStore use this new API.
Update tests for the latter as well.

* Add comments. BuildSQL -> AddToSelect for flexibility

* Run dep ensure

* Add filter proto to all other resources

* Add filtering for pipeline server

* Add filtering for job server

* Add filtering for run server

* Try to fix integration tests
2018-12-14 00:18:31 -08:00
IronPan 9eee2bbb47 move name (#498) 2018-12-07 16:41:15 -08:00
Ajay Gopinathan d3a30889bc Pin versions of libraries and tools required for proto generation. (#492)
This change pins the versions of the libraries that were used to
generate the proto definitions using dep. The Makefile is then modified
so that the tool and library versions used to build the proto generated
files are from the vendor directory. This is a hacky, short-term
solution to ensure a reproducible build while we work on switching to
bazel.

The versions in the Gopkg.toml file were chosen based on my experiments
that generated proto files that did not change from what is already
checked in.
2018-12-07 15:08:46 -08:00
Yang Pan b494e3daa0 Add integration tests for API servers (#112)
* add pipeline e2e test back

* delete samples

* more changes to pipeline test

* add delete run/experiment

* update test name

* stage

* add api

* fii

* add tensorboard routing rule (#143)

* add tensorboard routing rule

* rename tb routing rule

* address comments

* remove debugger

* fix url prefix

* more fixes

* update tests

* revert debugging

* update

* update test

* fix

* update test

* fix

* fix pipeline tests

* exp

* update run

* update jobs
2018-11-09 20:39:15 -08:00
Pascal Vicaire 1bab424abf Fixing the GO import paths to reference the kubeflow/pipelines repository. 2018-11-02 14:53:42 -07:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00