* Add pipeline_id to run and recurring run protos
* Add pipeline_id to run and recurring_run
* Set status when creating a new job. Closes#9125.
* stage progress
* Fix timestamps in tasks and add unit test
* Add upsert to sql dialects. Closes#8851.
* Remove dead code
* Fix unit test
* Switch to upsert
* Add unit tests
* Allow empty namespace
* Cleanup
* Fix tests
* Fix inconsistencies
* Fix old tests
* Differentiate list requests in multi-user mode
* Refactor create run and create job paths
* Revert default experiment change
* Add pipeline_id to run and recurring run protos
* Add pipeline_id to run and recurring_run
* Set status when creating a new job. Closes#9125.
* Add service unavialbe error to the BE
* Fix a unit test
Limit the direct Workflow data struct access to common.util
and use ExecutionSpec and ExecutionStatus in other places.
Create a data struct to represent the node status information
in ExecutionStatus and add an API to access this information.
* Enable healthz test in full Kubeflow.
This change will also be ported to release-2.0-alpha branch to support
conformance tests for KF 1.7, which is using alpha.7 version of KFP.
* Fix build break due to recent refactoring.
* Add pipeline_id to run and recurring run protos
* Add pipeline_id to run and recurring_run
* Set status when creating a new job. Closes#9125.
* Fix timestamps in tasks and add unit test
* Add upsert to sql dialects. Closes#8851.
* Remove dead code
* Fix unit test
* Add unit tests
* Remove dead code
* Add pipeline_id to run and recurring run protos
* Add pipeline_id to run and recurring_run
* Set status when creating a new job. Closes#9125.
* Add delete create pipeline and improve errors
* Add single-transaction pipeline upload
* Revert previous changes
* Resolve inconsistency
* Add a unit test
* Stage progress
* support pvc creation, deletion and mount
* replace image address for unit tests
* go mod tidy
* address comments
* Update go.mod
nit: fix format
* go mod tidy
* fix merge error
* update driver image and nit
* Rebase
* Add node selector to driver
* Fix input parameter's name
* Add a comment on container order
* Update driver and launcher images
* Update tests with hard-coded v2 images
* Address comments
* Fix test
* Update error message
* Update error message
---------
Co-authored-by: Linchin <lingqing.gan@gmail.com>
* support pvc creation, deletion and mount
* replace image address for unit tests
* go mod tidy
* address comments
* Update go.mod
nit: fix format
* go mod tidy
* fix merge error
* update driver image and nit
* Add pipeline_id to run and recurring run protos
* Add pipeline_id to run and recurring_run
* Fix params in recurring run
* Add support for v1 and v2 parameters in runs and jobs
* Fix convertion issues
* Update a unit-test
* Revert change of the order in sql
* Address comments
* Add pipeline_id to run and recurring run protos
* Add pipeline_id to run and recurring_run
* Remove unused commented code
* Add pipeline id verification
* chore: replace `github.com/ghodss/yaml` with `sigs.k8s.io/yaml`
At the time of making this commit, the package `github.com/ghodss/yaml`
is no longer actively maintained.
`sigs.k8s.io/yaml` is a permanent fork of `ghodss/yaml` and is actively
maintained by Kubernetes SIG.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Update licenses CSV
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
---------
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* support yaml with platform-specific info
* run go mod tidy and fix comments
* license, test and small fixes
* fix integration test and add test
* address comments
* address comments
* Stage progress
* Fix comments in the filter.proto.
* Remove api dep from filter and list. Fixes#8934.
* Fix pipeline joining logic
* Update sqlite to support window ops
* Add subquery to lis pipelines. Closes#8954
* Creanup
* Update licenses
* Deprecate experiment_id in Run API v2
* Allow runs and recurring runs without pipeline version id
* Adjust the logic for retrieving existing pipeline name
* Remove pipeline parsing logic and tests
* Simplify the logic in fetching a template
* Add nodeSelector and res to podSpec
* Add sample test draft
* Add accelerator-type logic to driver pod
* Remove accelerator sample. Closes#7043
* Add unit-tests and remove type handling
* support optional and default parameters
* add sample test
* change assert value
* update test
* update test
* fix sample test
* fix sample test
* fix sample test
* test fix
* change parameter logic
* test
* test
* change logic handling not provided parameter
* remove unnecessary code and add more sample case
* fix sample test bug
* fix sample test
* update logic
* change parameter logic
* update logic
* update logic handling when parameter not provided
* fix sample test
* add error message
* fix samples test
* samples test
* remove unused code
* Add children to PipelineTaskDetail
* Add state and state history to task
* Add state and state history to run
* Add pod_name to PipelineTaskDetail
* Add PodName and Payload to tasks table
* Add upsert to task store
* Add create or update task report wf
* Wire the report server
* Remove v1 and v2 specific functions
* Revert Run proto order
* Stage progress
* Add run id to exec spec
* Disable unit test due to #8851
* Address comments
* Address some comment
* Remove empty lines
* Refactor toModelTasks
* Address comments
* Address comments
* Staging WIP
* Fix recurring run, switch to pipeline_version_id
* Regenerate python client
* Refactor packages for pipeline service v2
* Stage progress
* Add support for pipeline service v2beta1
* Go mod tidy
* Update licenses
* Set default namespace to be empty
* Separate deletion of pipeline versions
* Change report ReportRunMetrics to obey AIP
* Use pipeline version id in run proto
* Update run.proto
* Upgrade storage
* Remove dependencies on api
* Stage refactoring progress
* Refactor apiserver
* Update v1beta1 unit test
* Update dependencies
* Fix integration test
* Revert visualization server changes (workaround)
* Address comments
* Basic lint the api server
* Update licenses file
* Adjust intergtaion tests
* Fix typos in the tests
* Update tests
* Update test
* Add support v1 delepe pipeline
* Remove gorm.Model to split into a different PR
* Lint api server
* Fix default experiment in upgrade test
* Update storage states
* Testing upgrade test run
* Add copying pod logs to the GCS bucket
* Fix upgrade test
* Adjust backend test for list
* Fix the tests
* Fix workflow manifest discrepancy at create
* Add a few unit-tests
* Add misc service and fix persistence agent
* Remove default values from TEXT columns
* Address comments. Fixes#8702.
* Address comment, reorder resource, server, storage
* Fix front end integration test
* Fix FE integration test
* Fix tabs, add pauses, make more explicit FE test
* Revert FE test changes
* Enable default experiment and disable its deletion
* Replace empty namespace with POD_NAMESPACE
* Default experiment is now the oldest
* Emplace empty with POD_NAMESPACE for argo only
* Fix api integration test
* Revert changes to default experiment's description
* Improve FE test flakiness
* Force refresh in the FE test
* Fix FE integration test
* Add delay in FE test
* Add local debug to FE test
* Update tests
* Fix typo
* Address comments
* Enforce template version verification
* Fix workflow manifest discrepancy with v1
* Fix FE test
* Revert changes in persistence agent
* Limit to one resource owner
* Change default run state to unknown
* Revert changes to FE integration test
* Make states consistent
* Remove links from error messages
* Address comments
* Address comments
* Change the order for teardown
---------
Co-authored-by: Chen Sun <chensun@users.noreply.github.com>
* add support for "finished_at" and other in filter
* update job map
* add UTs
* add integration test for filter
* update integration test
* fix integration test
* add more info
* fixed integration test
* fix small bug
* fix flakiness
* Make cache-server webhook port number configurable
Previously the webhook listened on a fixed port, 8443, which can clash
with other services when the webhook is run on the host network in
Kubernetes, which is required when using some CNI implementations,
notably Calico on EKS [1].
Enable configuration of the webhook listen port via the program flags.
[1] https://projectcalico.docs.tigera.io/getting-started/kubernetes/managed-public-cloud/eks#install-eks-with-calico-networking
Signed-off-by: Steve Larkin <steve.larkin@gmail.com>
* fixup! Make cache-server webhook port number configurable
Signed-off-by: Steve Larkin <steve.larkin@gmail.com>
* fixup! fixup! Make cache-server webhook port number configurable
Signed-off-by: Steve Larkin <steve.larkin@gmail.com>
Signed-off-by: Steve Larkin <steve.larkin@gmail.com>
* remove template dependency on api
* add create run
* update run metrics proto
* add run methods
* small changes
* pass asserted types in model converter
* add wiring
* added create recur run and wirings
* added get recurring run and test
* added enable disable delete rec run and tests
* revert namespace change
* fix typo
* use v1 counters for recurring runs
* fix bug
* change recurring run package name and regen
* merge recurring run and job server
* add template support for v2 api
* copyright
* add v2 tests
* add proto files and generate clients
* update dockerfile
* update script
* refactor rec_run_id
* copyright info
* move string definition
* small fix to proto
* small refactor
* template takes model instead of api
* template changes
* convert to model in resource manager
* resolve conflicts
* fix integration test typos
* test
* test
* test
* testing
* temp change for debug
* temp change for debug 2
* temp change for debug 3
* temp change for debug 4
* temp change for debug 5
* temp change for debug 6
* revert resource manager change
* add print
* bring back integration tests
* folder name
* delete a file
* remove package debug
* fix bug
* print more variables
* add log
* add log
* small fix
* add log
* more logs
* add test
* add logs
* test
* add another test
* use old function for Trigger convertion
* change modeltoCRDTrigger function
* removed V1 at the end of function name
* add comments
* removed unnecessary variable
* typo
* small change
* test
* remove logs for debugging
* small bug
* remove logs for debug
* nit and removed unused functions
* remove unused functions and move var declararion
* feat(backend) Fix authentication in upload requests
Fix the way KFP API server authenticates pipeline upload requests.
We leverage 'isAuthenticated()` function which requires proper
initialization of the context object to include user identity.
* feat(backend): Add namespace field in pipeline upload swagger definition
Extend swagger defintion of the pipeline upload API with a namespace
parameter in order to support uploading namespaced pipelines.
* chore(backend): Generate Go & Python clients
Autogenerate the Go and Python clients after extending the swagger
definitions of upload pipeline APIs with a namespace field.
* v2 exp proto and generate clients
* server changes
* move v1 and v2 protos into api folder
* regenerate v1 packages
* generate v2 package
* add v2 funcs.
* added v2 create experiment
* integration and initialization tests
* createexperiment works and rest can be pinged
* add create exp tests
* add get experiment
* add get experiment tests
* add list experiment and one test
* add list exp tests
* add TestValidateCreateExperimentRequest_Multiuser
* add delete exp tests
* add archive/unarchive exp and tests
* refactor storage and resource layers
* change api converter
* small fix
* small fixes
* move proto files outside and name v1 package api
* refactor
* refactor
Instead of changing the WorkflowResource.Spec to string,
use interface{} to allow backward compatibility.
Add type casting and marshal/unmarshal to properly
handle old swf CRs.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
gkcalat
Chen Sun
Lingqing Gan
b4sus
James Wu
Yihong Wang
deepk2u
Eng Zer Jun
Ablai Akhazhanov
Connor McCarthy
Steve Larkin
Ilias Katsakioris
juliusvonkohout