Commit Graph

66 Commits

Author SHA1 Message Date
Yuan (Bob) Gong 04e23d2a05
[Release] Regenerate api clients on release (#3910)
* [Release] Regenerate api clients on release

* Fix hack/release.sh

* Add a script to check required tools

* update

* Update

* Update clean up
2020-06-04 13:24:16 +08:00
Yuan (Bob) Gong 4f34be6f01
[API] Add license header to python api client files (#3897)
* Regenerate clients after proto description fix

* Add LICENSE to api python client

* Add license header to python files

* Revert unintended changes
2020-06-04 10:23:24 +08:00
Yuan (Bob) Gong 7dfb34edc8
Regenerate clients after proto description fix (#3895) 2020-06-03 01:52:17 -07:00
Yuan (Bob) Gong 36eba13efb
[Python Client] Use openapi generator instead of swagger (#3872)
* Generate python client package into repo using kfp_api_single_file.swagger.json

* Commit python client generated by swagger

* Use openapi-generator instead

* Regenerate using openapi-generator

* Add extra info into swagger single file

* Update more info

* update

* Move python http client to upper folder

* Clean up build script

* Update kfp_server_api from new swagger.json
2020-06-02 21:18:17 -07:00
Amy 5d302b6edb
fix typo on sort order: 'des' --> 'desc' (#3893) 2020-06-02 18:00:17 -07:00
jingzhang36 286492591b
More documentation on backend API methods (#3758)
* list experiment desc

* changes should be made in proto

* add comments and descriptions

* comments/descriptions in run.proto

* comments in job.proto and pipeline.proto

* try starting a new line

* newline doesnt help

* add swagger gen'ed file

* address comments

* regenerate json and client via swagger

* address comments

* regenerate go_http_client and swagger from proto

* two periods

* re-generate
2020-06-01 12:26:25 +08:00
Yuan (Bob) Gong c38f36cdc1
[Backend] Add service account field to run and job api objects (#3649)
* Add service account field to run and job api objects

* Update description

* Fix field casing

* Add comment about the next field number
2020-04-29 22:58:17 -07:00
Yuan (Bob) Gong b61944458a
[Backend] Authorization service (#3627)
* Authorization service proto

* implement auth service

* Add unit tests
2020-04-27 16:42:04 -07:00
Chen Sun c4fb79447c
[API] Include namespace in visualization.swagger.json (#3588)
* include namespace in CreateVisualization

* include namespace in post body

* put namespace in the path and in front of visualization

* post /apis/v1beta1/visualizations/{namespace}
2020-04-23 13:44:06 -07:00
Chen Sun c3a0230747
[Backend][Multi-user] Support creating visualization in user namespace (#3495)
* Add namespace field to CreateVisualizationRequest

* Support getting visualization service URL with namespace

* fix typo

* Add auth checking & allow empty namespace
2020-04-17 19:11:37 -07:00
jingzhang36 60f896bacc
Upload pipeline/pipeline version with a description (#3511)
* add desscription to client interface

* autogen

* version doesn't have description field

* swagger autogen

* remove two accidentally committed local python package
2020-04-17 15:36:04 +08:00
jingzhang36 56d519e9f7
Add archive experiment feature in backend (#3359)
* add new field in db schema and api schema

* auto genereted types for experiment storage state

* add archive and unarchive methods to backend for experiments.

* auto generated archive/unarchive methods for epxeriments

* add archive and unarchive to client

* set proper storage state when creating experiment

* retrieve storage state when we get/list epxeriment(s)

* change expection in test to have storage state

* add storage state in resource manager test

* revise experiemnt server test

* revise api converter test

* integration test of experiment archive

* archive/unarchive experiment affect the storage state of runs in it

* test all the runs in archive/unarchive experiment

* test all runs are archived/unarchived with their experiment in experiment server

* integration test

* integration test: value type mismatch in assertion

* unused import; default value for storage state

* autogen code for frontend

* reorder the fields in api experiment schema

* switch the position of the two enum to verify a hypothesis

* Put a place hodler to prevent any valid item to take the value 0

* Get rid of the place holder since the cause of issue related to value 0 is found and fixed.

* The returned api experiment now has storage state field

* create experiment return doesn't contain storege state

* Cleanup needs to clean runs and pipeliens now

* a missing client

* use resource reference as fileter instead of experiment uuid

* use same namespace in archive unit test

* Leave archive/unarchive experiment integration test to a separate PR

* also need to update jobs when experiments are archived

* Change of unarchiving logic. When experiment is unarchived, jobs/runs in
it stay archived

* add unit test for the job status in archived/unarchived experiment

* change archive state to 3 value enum; add experiment integration test

* make archive state 3 value enum to avoid 0 value mapped to available; add integration test

* run swagger autogen

* fix an expected value

* fix experiment server test

* add job check in experiment server test

* update job crds

* fix a typo

* remove accidentally included irrelevant changes
2020-04-16 07:31:09 +08:00
Chen Sun 2f1dbbedd2
Experiment API change (#3198) 2020-03-06 09:17:23 -08:00
Yuan (Bob) Gong 688aa08d42
Release 0.2.5 - e66dcb1860 (#3179)
* Updated component images to version e66dcb1860

* Updated components to version 3f4b80127f

* Update setup.py

* Fix swagger codegen download url

* Update versions
2020-02-26 23:59:19 -08:00
jingzhang36 80fc767eb8
Fix a missing query parameter in swagger json file for version upload api (#3160)
* add pipelineid to api parameter

* Backend swagger autogent

* Reverse a local change

* Reverse another local change
2020-02-24 19:44:15 +08:00
Yuan (Bob) Gong 74a8178e1d
[Controller] Scheduled workflow supports catch up false (#3073)
* GetNextScheduledEpochNoCatchup implementation

* Add tests for cron schedule nocatchup

* Add tests for periodic schedule and fix a corner case

* Integrate no catchup behavior in swf controller

* Update job api proto

* Regenerate backend client

* Pass catchup parameter in backend api

* Rename proto field to no_catchup, so that it has backward compatible default value

* Update generated backend api

* Use no_catchup field instead

* Add some comments
2020-02-19 19:13:02 -08:00
Chen Sun 06c03f95db
Fix api generation script (#3090) 2020-02-14 15:53:28 -08:00
Michael Benavidez 811af3811d
Updated the API reference comments (#2916)
* Update comments with requested edits

* Update comments with requested edits
2020-02-12 01:34:52 -08:00
jingzhang36 96f1e4dedb
Swagger generated code for UploadPipelineVersion method (#3010) 2020-02-07 20:23:53 -08:00
jingzhang36 d1b72df5fb
Upload local file as new pipeline version Step 1 (#3001)
* add upload pipeline version to upload_pipeline_server and http main

* add apiPipelineVersion to pipeline upload swagger json

* add apiResourceReference to pipeline upload swagger json

* Add yet more types to pipeline upload swagger json

* Unit tests
2020-02-07 16:13:36 +08:00
jingzhang36 4c491823a0
Update auto-generated files' license date (generated by swagger-gen) (#2998) 2020-02-07 14:44:50 +08:00
jingzhang36 cc0b324281
Update version in generate_api.sh and run generate_api.sh to update (#2770)
Update version in generate_api.sh and run generate_api.sh to update swagger json.
2019-12-24 14:22:07 +08:00
Ning 1b85a614bf add namespace to the resourcereference (#2651)
* add namespace to some run APIs

* update only the create run api

* add resourcereference for namespace runs

* add variables in const

* add types to toModel func

* bug fix

* strip the namespace resource reference when mapping to the db model

* add unit tests

* use gofmt

* replace belonging relationshipreference to owner

* put a todo for further investigation of using namespace or uuid

* apply gofmt

* revert minor change

* Update model_converter.go
2019-12-03 22:06:57 -08:00
jingzhang36 6c1c641e40 Open version api in BE proto and auto-generate BE/FE api methods (#2620)
* Open the version api in BE for later FE PR to use. Including
auto-generated BE and FE code.

* format FE

* re-generate

* remove an unnecessary auto-generated file

* format
2019-11-26 19:35:19 -08:00
Michael Benavidez 452c360473 MDez patch 2 (#2635)
* Update job.proto

Added comments to describe endpoints.

* Adding endpoint comments for swagger.

* Spacing fix

* Spacing fix, comment consistency

* Add endpoint comments for swagger descriptions.

* Update experiment.proto

Add spacing

* Update experiment.proto

* Update job.proto

* Update pipeline.proto

* Update run.proto
2019-11-21 11:23:29 -08:00
Michael Benavidez e2c83759a2 MDez patch 1 (#2550)
* Add descriptive comments to all endpoints

* Add link to filter.proto page

* Update job.proto

* Add link to filter.proto page

* Update run.proto
2019-11-07 14:48:27 -08:00
jingzhang36 af090bffa2 Regenerate api since https://github.com/kubeflow/pipelines/pull/2445 changed api proto (#2506)
* re-generate api since #2445 changed api proto

* Fix integration test for pipeline client

* use generated constants
2019-10-29 23:54:52 -07:00
jingzhang36 47aaf08280 Runs and jobs can be created from pipeline version (#2445)
* add version api

* unit tests

* remove debug fmt

* remove unused func

* remove another unused method

* formatting

* remove unused consts

* some comments

* build

* unit tests

* unit tests

* formatting

* unit tests

* run from pipeline version

* pipeline version as resource type

* run store and resource reference store

* formatting and removing debug traces

* run server test

* job created from pipeline version

* variable names

* address comments

* Get pipeline version template is used on pipeline details page, which fetches pipelien version file.

* a temp revert

* address comment

* address comment

* add comment

* get pipeline version template

* verify pipeline version in resource reference

* add unit test for create run from pipeline version

* unit test for create job from pipeline version

* remove some comments

* reformat

* reformat again

* Remove an unrelated change

* change method name
2019-10-28 11:02:42 -07:00
Ajay Gopinathan 26f8e628b4 Fix documentation for filter.proto (#2447)
* Fix documentation for filter.proto

* Fix file generation
2019-10-25 02:35:38 -07:00
jingzhang36 f66af1f0ee Add pipeline version api methods (#2338)
* add version api

* unit tests

* remove debug fmt

* remove unused func

* remove another unused method

* formatting

* remove unused consts

* some comments

* build

* unit tests

* unit tests

* formatting

* unit tests

* address comments

* address comment

* address comment

* add comment
2019-10-17 01:34:39 -07:00
jingzhang36 7aaecb1501 Add necessary data types to api and database to support pipeline version. (#1873)
* Add necessary data types/tables for pipeline version. Mostly based
on Yang's branch at https://github.com/IronPan/pipelines/tree/kfpci/.
Backward compatible.

* Modified comment

* Modify api converter according with new pipeline (version) definition

* Change pipeline_store for DefaultVersionId field

* Add pipeline spec to pipeline version

* fix model converter

* fix a comment

* Add foreign key, pagination of list request, refactor code source

* Refactor code source

* Foreign key

* Change code source and package source type

* Fix ; separator

* Add versions table and modify existing pipeline apis

* Remove api pipeline defintiion change and leave it for later PR

* Add comment

* Make schema changing and data backfilling a single transaction

* Tolerate null default version id in code

* fix status

* Revise delete pipeline func

* Use raw query to migrate data

* No need to update versions status

* rename and minor changes

* accidentally removed a where clause

* Fix a model name prefix

* Refine comments

* Revise if condition

* Address comments

* address more comments

* Rearrange pipeline and version related parts inside CreatePipeline, to make them more separate.

* Add package url to pipeline version. Required when calling CreatePipelineVersionRequest

* Single code source url; remove pipeline id as sorting field; reformat

* resolve remote branch and local branch diff

* remove unused func

* Remove an empty line
2019-09-25 23:59:07 -07:00
Alexey Volkov 570141d7c9 Make wget quieter (#2069) 2019-09-09 14:32:54 -07:00
Yannis Zarkadas c781b30f7b api: fix generate_api.sh script with bash shebang (#2017)
Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
2019-09-03 13:28:58 -07:00
Kirin Patel cd2a684f7c Add generated client from visualization swagger (#1979)
* Update backend/api/generate_api.sh to build visualization swagger file

* Built visualization client
2019-08-30 09:35:54 -07:00
Kirin Patel 3d69062e13 Add custom visualization support for Python based visualizations (#1929)
* Added custom visualization type

* Added support for custom visualizations to the VisualizationCreator component

* Re-generated API

* Updated VisualizationCreator.test.tsx.snap

* Updated VisualizationCreator.test.tsx tohave new and more specific tests

* Added tests to ensure Editor component is visible when specifying visualization type
* Updated test to properly validate provided source is rendered

* Added unit test to ensure that an argument placeholder is provided for every visualization type

* Fixed linting error

* Simplified canGenerate logic
2019-08-25 15:10:20 -07:00
Kirin Patel 3c8952e6bf Add TFDV, TFMA, and Table visualization support for Python based visualizations (#1898)
* Added table and tfdv visualization

Also fixed issue surrounding ApiVisualizationType enum

* Fixed table visualization

* Removed byte limit
* Fixed issue where headers would not properly be applied
* Fixed issue where table would not be intractable

* Updated table visualizaiton to reflect changes made to dependency injection

* Fixed bug where checking if headers is provided to table visualizations could crash visualization

* Added TFMA visualization

* Updated new visualizations to match syntax of #1878

* Updated test snapshots to account for TFMA visualization

* Small if statement synax changes

* Add flake8 noqa comments to table.py and tfma.py
2019-08-22 13:57:20 -07:00
IronPan f1c9594f7d Propagate pipeline name in pipeline spec (#1842)
* update

* add pipeline name

* propagate name

* propagate name

* fix

* update test
2019-08-14 23:30:32 -07:00
IronPan 4e43750c9d
add reference name to resource reference API proto (#1781)
* add resource reference

* fix tests

* fix tests

* fix e2e test

* fix e2e

* fix test

* update api requirement

* fix tests

* Update job_api_test.go

* Update run_api_test.go

* Update setup.py

* Update deploy-kubeflow.sh

* fix tests

* Update deploy-kubeflow.sh
2019-08-13 03:15:11 -07:00
IronPan 9835b5b0ba
Update kfp_api_single_file.swagger.json 2019-08-09 01:18:33 -07:00
IronPan a9602fbc3f
Add API to rerun the pipeline (#1720)
* add resubmit proto

* add compiled code

* fix

* add resubmit proto

* add

* refactor

* update builder

* refactor

* refactor

* refactor

* refactor

* refactor

* refactor

* add test

* add test

* add test

* add test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* address comments

* add comments

* change request body def

* recompile api

* retry instead of resubmit

* update test

* update test

* fix tests

* fix tests

* fix tests

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* robust retry

* add error handling

* reorder the call

* remove logic to update the database entry

* add mock

* add tests for rerousrce manager

* update error handling logic

* fix tests

* address comments
2019-08-07 13:59:06 -07:00
Kirin Patel fa1abde7f6 Rename InputPath -> Source for Visualization API definition (#1717)
* InputPath -> Source

* Changed name of data path/pattern variable from InputPath to Source to improve consistency with current visualization method
* Updated unit tests to reflect name change
* Regenerated swagger definitions to reflect name change

* Readded test that was removed with previous commit

It was deleted by mistake
2019-08-05 10:55:50 -07:00
Alexey Volkov f8e06c325c API - Updated swagger-codegen-cli version (#1686)
Fixes https://github.com/kubeflow/pipelines/issues/1685
2019-07-31 18:54:12 -07:00
Kirin Patel 5838e2c9d0 Changed arguments parameter type and built visualization pb and swagger files (#1636)
* String array -> string for arguments parameter in visualization.proto

Switching from a repeated string to a string was done to allow stringified JSON to be used for specifying arguments for visualizations. By doing this, a more generic approach for passing arguments can be taken within python, rather than using argparser, json can be used to decode arguments without having to specify them beforehand.

* Ran generate_api.sh
2019-07-19 19:22:55 -07:00
Kirin Patel 377464f2f4 Add Visualization API endpoint (#1565)
* Created visualization.proto

* Addressed most of PR feedback

* Fixed comments

* Addressed additional PR feedback

* Changed output from path to html

* Removed id parameter from visualization and changed inputPaths to inputPath

* Added support for command line arguments to be passed via the API

These are required for the new roc curve and will become import for passing any parameters form a user to the visualization.

* Fixed typo
2019-07-10 12:30:26 -07:00
IronPan 36338774f1
update the API doc description (#1410)
* add single json file

* update doc

* update doc
2019-05-31 13:31:33 -07:00
IronPan 14c66f0174
add single json file (#1408) 2019-05-30 18:02:29 -07:00
IronPan 75cfab3d2a
Delete kfp_api.html 2019-05-30 17:58:51 -07:00
IronPan 89954f831e
Delete kfp_api_single_file.swagger.json 2019-05-30 17:58:02 -07:00
IronPan 06e4dc6604
Add doc for API (#1406)
* initial update

* initial update

* update

* add doc

* update
2019-05-30 17:48:50 -07:00
Ajay Gopinathan 5f1b41171f Fix API package names and regenerate checked-in proto files. (#1404)
* Fix API package names and regenerate checked-in proto files. Also bump version of GRPC gateway used.

* Fix BUILD.bazel file for api as well.

* Update Bazel version
2019-05-30 11:26:28 -07:00