* added draft of create v2 pipeline run
* fixed broken UT and added UT for parsing template
* modified run apis to support v2 IR spec
* remove temporary patch
* fixed dependency
* fixed build failure
* finished draft
* finished create job and run
* refactor template and fixed broken UT
* updated go license
* fixed build failure
* fixed build
* added UT
* modified UT
* fixed build failure
* fixed license
* add description to upload_pipeline_description http endpoint
* add test for pipeline upload server description
* add description to new pipeline version
* show pipeline version in pipeline detail page
* make description optional on UI + show version descriptio or pipeline description
* remove unused reference
* revert wrong change of ref
* updated on should be pipeline version
* add more versions to mock-backend
* show pipeline and version description at the same time
* add tests for UI
* show pipeline version always
* start adding description to version
* add description to resource manager
* remove description from pipeline default version
* update pipeline_store
* add some tests to ensure its being passed around
* update frontend generated api
* run auto formatter
* add comment to frontend readme about running npm run format
* add missing api converter step
* added model and storage layer for task
* added create task api
* added api to list tasks
* modified task proto and fixed nits
* renamed variable
* fixed ut
* fixed UT
* added UT for api_converter and resource manager
* added UT for api_converter and resource manager
* fixed BE UT
* added task storage layer UT
* changed UT
* fixed foreign key typo
* fixed typo and added comments
* Added multi-user pipelines backend
corrected typo
updating code based on review
fixes for pipelines server
reverting this back
* removing unnecessary info logging
* updated to remove bazel
* rename@
* script to run in container
* update the generation
* updated docker image
* changed name
* updated the code
* regenerated API
* fix env variables
* updated version of go swagger
* rerun generate with updated package
* added healthz
* typo with folders fixed
* changed version
* set version of protoc compiler
* test if version is correct
* test version
* changed version
* updated version agian
* test version
* the latest test
* updated docker image
* fixed some stuff
* new test
* tested other version
* new tests
* changed swagger
* new test
* updated versions
* missed docker file
* updated files@
* change back
* updated after feedback@
* clean up
* remove license and install binary
* update and check differences
* completed rebase
* go mod tidy
* updated based upon work of boby
* remove code commited by misstake
* added by misstake
* updated after feedback
* futher updates after feedback
* final updates from feedback
* switch to Makefile
* update
* fix non-root execution
* clean up, fix release script
* fix swagger
* fix healthz endpoint & regenerate python client
* Delete sample.py.tar.gz
Co-authored-by: Yuan Gong <gongyuan94@gmail.com>
Co-authored-by: Yuan (Bob) Gong <4957653+Bobgy@users.noreply.github.com>
* chore(backend): tidy go.mod and update tools.go
* go install, instead of go get
* fix problems reported by go vet
* simplify some ide reported redundant syntax
* license is not required for generated code
* remove licenses for generated code
* cleanup
* remove license more
* rm unused BUILD.bazel files
* fixed generate_api.sh
* reimport error.proto
* chore(backend): tidy go.mod and update tools.go
* go install, instead of go get
* fix problems reported by go vet
* simplify some ide reported redundant syntax
* Set current namespace in local KFP context if running from notebook
* Create "~/.config/kfp/" instead of ".config/kfp/"
At first it was assumed the `get_user_namespace` command would be executed from the home directory.
* Create local context file if it doesn't exist during set_user_namespace
* Grab path from LOCAL_KFP_CONTEXT when creating folder
Instead of harcoding the os.mkdirs path to `~/.config/kfp` it now grabs it from the LOCAL_KFP_CONTEXT. Also, removed path creation in `get_user_namespace` as that is now handled in `set_user_namespace`. Also, it now checks if the path exists rather than the local_context_file to remove the situation where it tries to create ~/.config/kfp/ because the context.json doesn't exist when the path does.
* add multi-user setting to healthz api
* Add http prefix to health api url
* move healtz api call to own function and fix multi_user boolean
* Fix HEALTH_PATH declaration
* Move check to Client __init__ and change get_kfp_healthz to avoid breaking in case of old apiserver image
* Add multi_user to frontend healthz
* Expose multi_user in frontend and add integration test
* Fix integration test
* Fix host hardcoding and error handling
* Handle empty API response, check if API up to date
* Fix response return
* remove API check due to empty response
* retry API call if first response empty
* retry getting healthz api if no response
* change health_api to https
The healthz_api has been returning empty responses which might be caused by sending an http request to an https endpoint. Although requests handles redirects, this commit is to test if this solves the issue.
* Add some debug info to healthz exception
* add url to debug and lower retries to 1
* Use api_client to get healthz data
* Debug info for API response
* Follow API redirect history
* Fix indentation
* Add healthz proto
* Try getting healthz api with new python backend
* Add installation of kfp_server_api in tests
* Fix incorrect setup location
* Replace old .get with new http backend .multi_user
* Code clean up
* Small fixes and TimeOutError for retries healthz api
* Remove changes to go dependencies
* Send empty proto request and fix exception client
* Remove unused commit_sha and tag_name
* initial work on exposing the default version of pipeline
* update description
* added missing files
* updated api build, unsure if this is correct ...
* updated after feedback
* clean up
* remove empty line
* started to make the integration test
* added integreation test
* fixed build and feedback
* updated the tests
* Updated the test
* new test
* typo
* updated the pipeline default
* updated the pipeline version
* formatting
* error in comparison
* enable pagination when expanding experiment in both the home page and the archive page
* Revert "enable pagination when expanding experiment in both the home page and the archive page"
This reverts commit 5b672739dd.
* add a quick guide on how to generate api reference from kfp api definition
* remove trailing lines
* Generate python client package into repo using kfp_api_single_file.swagger.json
* Commit python client generated by swagger
* Use openapi-generator instead
* Regenerate using openapi-generator
* Add extra info into swagger single file
* Update more info
* update
* Move python http client to upper folder
* Clean up build script
* Update kfp_server_api from new swagger.json
* list experiment desc
* changes should be made in proto
* add comments and descriptions
* comments/descriptions in run.proto
* comments in job.proto and pipeline.proto
* try starting a new line
* newline doesnt help
* add swagger gen'ed file
* address comments
* regenerate json and client via swagger
* address comments
* regenerate go_http_client and swagger from proto
* two periods
* re-generate
* include namespace in CreateVisualization
* include namespace in post body
* put namespace in the path and in front of visualization
* post /apis/v1beta1/visualizations/{namespace}
* Add namespace field to CreateVisualizationRequest
* Support getting visualization service URL with namespace
* fix typo
* Add auth checking & allow empty namespace
* add desscription to client interface
* autogen
* version doesn't have description field
* swagger autogen
* remove two accidentally committed local python package
* add new field in db schema and api schema
* auto genereted types for experiment storage state
* add archive and unarchive methods to backend for experiments.
* auto generated archive/unarchive methods for epxeriments
* add archive and unarchive to client
* set proper storage state when creating experiment
* retrieve storage state when we get/list epxeriment(s)
* change expection in test to have storage state
* add storage state in resource manager test
* revise experiemnt server test
* revise api converter test
* integration test of experiment archive
* archive/unarchive experiment affect the storage state of runs in it
* test all the runs in archive/unarchive experiment
* test all runs are archived/unarchived with their experiment in experiment server
* integration test
* integration test: value type mismatch in assertion
* unused import; default value for storage state
* autogen code for frontend
* reorder the fields in api experiment schema
* switch the position of the two enum to verify a hypothesis
* Put a place hodler to prevent any valid item to take the value 0
* Get rid of the place holder since the cause of issue related to value 0 is found and fixed.
* The returned api experiment now has storage state field
* create experiment return doesn't contain storege state
* Cleanup needs to clean runs and pipeliens now
* a missing client
* use resource reference as fileter instead of experiment uuid
* use same namespace in archive unit test
* Leave archive/unarchive experiment integration test to a separate PR
* also need to update jobs when experiments are archived
* Change of unarchiving logic. When experiment is unarchived, jobs/runs in
it stay archived
* add unit test for the job status in archived/unarchived experiment
* change archive state to 3 value enum; add experiment integration test
* make archive state 3 value enum to avoid 0 value mapped to available; add integration test
* run swagger autogen
* fix an expected value
* fix experiment server test
* add job check in experiment server test
* update job crds
* fix a typo
* remove accidentally included irrelevant changes
* GetNextScheduledEpochNoCatchup implementation
* Add tests for cron schedule nocatchup
* Add tests for periodic schedule and fix a corner case
* Integrate no catchup behavior in swf controller
* Update job api proto
* Regenerate backend client
* Pass catchup parameter in backend api
* Rename proto field to no_catchup, so that it has backward compatible default value
* Update generated backend api
* Use no_catchup field instead
* Add some comments
* add upload pipeline version to upload_pipeline_server and http main
* add apiPipelineVersion to pipeline upload swagger json
* add apiResourceReference to pipeline upload swagger json
* Add yet more types to pipeline upload swagger json
* Unit tests
* add namespace to some run APIs
* update only the create run api
* add resourcereference for namespace runs
* add variables in const
* add types to toModel func
* bug fix
* strip the namespace resource reference when mapping to the db model
* add unit tests
* use gofmt
* replace belonging relationshipreference to owner
* put a todo for further investigation of using namespace or uuid
* apply gofmt
* revert minor change
* Update model_converter.go
* Open the version api in BE for later FE PR to use. Including
auto-generated BE and FE code.
* format FE
* re-generate
* remove an unnecessary auto-generated file
* format
* add version api
* unit tests
* remove debug fmt
* remove unused func
* remove another unused method
* formatting
* remove unused consts
* some comments
* build
* unit tests
* unit tests
* formatting
* unit tests
* run from pipeline version
* pipeline version as resource type
* run store and resource reference store
* formatting and removing debug traces
* run server test
* job created from pipeline version
* variable names
* address comments
* Get pipeline version template is used on pipeline details page, which fetches pipelien version file.
* a temp revert
* address comment
* address comment
* add comment
* get pipeline version template
* verify pipeline version in resource reference
* add unit test for create run from pipeline version
* unit test for create job from pipeline version
* remove some comments
* reformat
* reformat again
* Remove an unrelated change
* change method name
* Add necessary data types/tables for pipeline version. Mostly based
on Yang's branch at https://github.com/IronPan/pipelines/tree/kfpci/.
Backward compatible.
* Modified comment
* Modify api converter according with new pipeline (version) definition
* Change pipeline_store for DefaultVersionId field
* Add pipeline spec to pipeline version
* fix model converter
* fix a comment
* Add foreign key, pagination of list request, refactor code source
* Refactor code source
* Foreign key
* Change code source and package source type
* Fix ; separator
* Add versions table and modify existing pipeline apis
* Remove api pipeline defintiion change and leave it for later PR
* Add comment
* Make schema changing and data backfilling a single transaction
* Tolerate null default version id in code
* fix status
* Revise delete pipeline func
* Use raw query to migrate data
* No need to update versions status
* rename and minor changes
* accidentally removed a where clause
* Fix a model name prefix
* Refine comments
* Revise if condition
* Address comments
* address more comments
* Rearrange pipeline and version related parts inside CreatePipeline, to make them more separate.
* Add package url to pipeline version. Required when calling CreatePipelineVersionRequest
* Single code source url; remove pipeline id as sorting field; reformat
* resolve remote branch and local branch diff
* remove unused func
* Remove an empty line
* Added custom visualization type
* Added support for custom visualizations to the VisualizationCreator component
* Re-generated API
* Updated VisualizationCreator.test.tsx.snap
* Updated VisualizationCreator.test.tsx tohave new and more specific tests
* Added tests to ensure Editor component is visible when specifying visualization type
* Updated test to properly validate provided source is rendered
* Added unit test to ensure that an argument placeholder is provided for every visualization type
* Fixed linting error
* Simplified canGenerate logic
* Added table and tfdv visualization
Also fixed issue surrounding ApiVisualizationType enum
* Fixed table visualization
* Removed byte limit
* Fixed issue where headers would not properly be applied
* Fixed issue where table would not be intractable
* Updated table visualizaiton to reflect changes made to dependency injection
* Fixed bug where checking if headers is provided to table visualizations could crash visualization
* Added TFMA visualization
* Updated new visualizations to match syntax of #1878
* Updated test snapshots to account for TFMA visualization
* Small if statement synax changes
* Add flake8 noqa comments to table.py and tfma.py
* InputPath -> Source
* Changed name of data path/pattern variable from InputPath to Source to improve consistency with current visualization method
* Updated unit tests to reflect name change
* Regenerated swagger definitions to reflect name change
* Readded test that was removed with previous commit
It was deleted by mistake
* String array -> string for arguments parameter in visualization.proto
Switching from a repeated string to a string was done to allow stringified JSON to be used for specifying arguments for visualizations. By doing this, a more generic approach for passing arguments can be taken within python, rather than using argparser, json can be used to decode arguments without having to specify them beforehand.
* Ran generate_api.sh
* Created visualization.proto
* Addressed most of PR feedback
* Fixed comments
* Addressed additional PR feedback
* Changed output from path to html
* Removed id parameter from visualization and changed inputPaths to inputPath
* Added support for command line arguments to be passed via the API
These are required for the new roc curve and will become import for passing any parameters form a user to the visualization.
* Fixed typo
Chen Sun
capri-xiyue
Yuan Gong
James Liu
Gerard Casas Saez
Joshua Carp
Yuan (Bob) Gong
XiyueYu
Alexey Volkov
Luis Magana
Niklas Hansson
Rui Fang
Yang Pan
numerology
hilcj
hilcj
DavidSpek
Yihong Wang
jingzhang36
Amy
Michael Benavidez
Ning
Ajay Gopinathan
Alexey Volkov
Yannis Zarkadas
Kirin Patel