Commit Graph

9 Commits

Author SHA1 Message Date
gkcalat 135dc64854
feat(backend): Refactor apiserver, add Pipeline and other v2beta services (#8633)
* Staging WIP

* Fix recurring run, switch to pipeline_version_id

* Regenerate python client

* Refactor packages for pipeline service v2

* Stage progress

* Add support for pipeline service v2beta1

* Go mod tidy

* Update licenses

* Set default namespace to be empty

* Separate deletion of pipeline versions

* Change report ReportRunMetrics to obey AIP

* Use pipeline version id in run proto

* Update run.proto

* Upgrade storage

* Remove dependencies on api

* Stage refactoring progress

* Refactor apiserver

* Update v1beta1 unit test

* Update dependencies

* Fix integration test

* Revert visualization server changes (workaround)

* Address comments

* Basic lint the api server

* Update licenses file

* Adjust intergtaion tests

* Fix typos in the tests

* Update tests

* Update test

* Add support v1 delepe pipeline

* Remove gorm.Model to split into a different PR

* Lint api server

* Fix default experiment in upgrade test

* Update storage states

* Testing upgrade test run

* Add copying pod logs to the GCS bucket

* Fix upgrade test

* Adjust backend test for list

* Fix the tests

* Fix workflow manifest discrepancy at create

* Add a few unit-tests

* Add misc service and fix persistence agent

* Remove default values from TEXT columns

* Address comments. Fixes #8702.

* Address comment, reorder resource, server, storage

* Fix front end integration test

* Fix FE integration test

* Fix tabs, add pauses, make more explicit FE test

* Revert FE test changes

* Enable default experiment and disable its deletion

* Replace empty namespace with POD_NAMESPACE

* Default experiment is now the oldest

* Emplace empty with POD_NAMESPACE for argo only

* Fix api integration test

* Revert changes to default experiment's description

* Improve FE test flakiness

* Force refresh in the FE test

* Fix FE integration test

* Add delay in FE test

* Add local debug to FE test

* Update tests

* Fix typo

* Address comments

* Enforce template version verification

* Fix workflow manifest discrepancy with v1

* Fix FE test

* Revert changes in persistence agent

* Limit to one resource owner

* Change default run state to unknown

* Revert changes to FE integration test

* Make states consistent

* Remove links from error messages

* Address comments

* Address comments

* Change the order for teardown

---------

Co-authored-by: Chen Sun <chensun@users.noreply.github.com>
2023-02-03 20:17:01 +00:00
Lingqing Gan d0f0ab309a
feat(backend): Add support for recurring run v2 api (#8634)
* add wiring

* added create recur run and wirings

* added get recurring run and test

* added enable disable delete rec run and tests

* revert namespace change

* fix typo

* use v1 counters for recurring runs

* fix bug

* change recurring run package name and regen

* merge recurring run and job server
2022-12-29 01:26:19 +00:00
Lingqing Gan 7e7e1afcb0
feat(backend): Add support for converting v2 recurring run to scheduled workflow in template (#8521)
* add template support for v2 api

* copyright

* add v2 tests

* add proto files and generate clients

* update dockerfile

* update script

* refactor rec_run_id

* copyright info

* move string definition

* small fix to proto

* small refactor

* template takes model instead of api

* template changes

* convert to model in resource manager

* resolve conflicts

* fix integration test typos

* test

* test

* test

* testing

* temp change for debug

* temp change for debug 2

* temp change for debug 3

* temp change for debug 4

* temp change for debug 5

* temp change for debug 6

* revert resource manager change

* add print

* bring back integration tests

* folder name

* delete a file

* remove package debug

* fix bug

* print more variables

* add log

* add log

* small fix

* add log

* more logs

* add test

* add  logs

* test

* add another test

* use old function for Trigger convertion

* change modeltoCRDTrigger function

* removed V1 at the end of function name

* add comments

* removed unnecessary variable

* typo

* small change

* test

* remove logs for debugging

* small bug

* remove logs for debug

* nit and removed unused functions

* remove unused functions  and move var declararion
2022-12-21 19:43:13 +00:00
gkcalat 8e02cc5c5a
chore(backend): Add pipeline and runtime protos (#8533)
* Adjust generator.sh

* Format the existing protos

* Add pipeline protos

* Add runtime protos

* Generate client libs

* Fix typos, adjust readme

* Add python client

* Update package name

* Replace error.proto with google.rps.Status
2022-12-09 01:46:11 +00:00
Lingqing Gan 6e364d35b1
feat(backend): Add v2 api for experiments (#8480)
* v2 exp proto and generate clients

* server changes

* move v1 and v2 protos into api folder

* regenerate v1 packages

* generate v2 package

* add v2 funcs.

* added v2 create experiment

* integration and initialization tests

* createexperiment works and rest can be pinged

* add create exp tests

* add get experiment

* add get experiment tests

* add list experiment and one test

* add list exp tests

* add TestValidateCreateExperimentRequest_Multiuser

* add delete exp tests

* add archive/unarchive exp and tests

* refactor storage and resource layers

* change api converter

* small fix

* small fixes

* move proto files outside and name v1 package api

* refactor

* refactor
2022-11-29 01:21:43 +00:00
Lingqing Gan 3f2e20d14f
chore(backend): regenerate v1beta1 api clients (#8403)
* add v2 experiment.proto

* regenerate v1beta1 client

* remove api version from combined swagger file

* Revert "regenerate v1beta1 client"

This reverts commit 02769d779b.

* regenerate v1beta1 clients

* regenerate python packages without api version

* remove regenerated v1 python packages&change proto

* refactor v1 exp apis and functions

* remove v2 api package and refactor v1 proto

* regenerate v1 go clients

* regenerate v1 python package

* refactor and update the human maintained swagger

* renew swagger file

* regenerate v1 packages

* regenerate v1 python package

* refactor

* regenerate v1 go clients

* regenerate v1 python package

* recover dockerfile and add s in python

* fix func name

* use commit for sdk execution test

* change sdk execution test

* update test script

* add v1 to methods

* proto update v1 method names

* regenerate v1 go packages

* regenerate python client

* update server methods

* refactor

* refactor

* refactor

* refactor python client

* fix get_healthz

* update sample-test.sh in v2 backend

* sample test script

* change v2 sample test

* change package install order

* Revert "change package install order"

This reverts commit c24fcfbed9.

* use KFP server api from commit for v2 sample tests

* small fix
2022-11-10 19:18:00 +00:00
Lingqing Gan 6c6b1e3a94
chore(backend): move v1 api to v1beta1 package and folder (#8376)
* move v1 api to v1beta1 package and folder

* fix test

* update README.md

* update README.md

* update readme
2022-10-26 07:11:30 +00:00
Alexey Volkov cc83e1089b
Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
Niklas Hansson e669b22954
fix(backend): remove Bazel from building the API. Part of #3250 (#4906)
* updated to remove bazel

* rename@

* script to run in container

* update the generation

* updated docker image

* changed name

* updated the code

* regenerated API

* fix env variables

* updated version of go swagger

* rerun generate with updated package

* added healthz

* typo with folders fixed

* changed version

* set version of protoc compiler

* test if version is correct

* test version

* changed version

* updated version agian

* test version

* the latest test

* updated docker image

* fixed some stuff

* new test

* tested other version

* new tests

* changed swagger

* new test

* updated versions

* missed docker file

* updated files@

* change back

* updated after feedback@

* clean up

* remove license and install binary

* update and check differences

* completed rebase

* go mod tidy

* updated based upon work of boby

* remove code commited by misstake

* added by misstake

* updated after feedback

* futher updates after feedback

* final updates from feedback

* switch to Makefile

* update

* fix non-root execution

* clean up, fix release script

* fix swagger

* fix healthz endpoint & regenerate python client

* Delete sample.py.tar.gz

Co-authored-by: Yuan Gong <gongyuan94@gmail.com>
Co-authored-by: Yuan (Bob) Gong <4957653+Bobgy@users.noreply.github.com>
2021-02-16 16:58:30 -08:00