Commit Graph

21 Commits

Author SHA1 Message Date
Chen Sun cd7e316db6
test: upgrade e2e sample tests to using sdk v2 (#9885)
* install kfp sdk 2.0 in sample test

* remove reference to GCSHelper from deprecated module

* exclude obsolete samples

* update test script

* update client usage

* revert recent changes to exit_handler.py
2023-08-17 09:33:38 +00:00
Ryan Jaemun Jung 1002e0cf8c
fix(samples): update samples to v2 pipelines (#9851)
* Update samples to v2 pipelines

* lint
2023-08-11 19:53:33 +00:00
gkcalat f626629f79
chore(testing): Enable samples in e2e tests with v1 kfp SDK (#9169)
* Add pipeline_id to run and recurring run protos

* Add pipeline_id to run and recurring_run

* Set status when creating a new job. Closes #9125.

* Enable sample tests
2023-04-19 04:55:53 +00:00
Chen Sun d25bbc88b1
fix(test): Make exit handler test install KFP SDK from source (#7785)
* Update exit_handler_v2.py

* Update exit_handler_v2.py

* Update exit_handler_v2.py
2022-06-01 23:30:51 +00:00
Lingqing Gan 98a58c7a0c
Pipeline level exit handler and v2 test (#7642)
* support IR YAML format in API

* Check the error message and return false if it is not nil

* update error message

* fixed simple loop but need cleaning up

* Deleted debug logs

* remove logs and fix some format

* fix static_loop_arguments

* change the driver image 

change the driver image back to the kfp container registry.

* change variable declaration

* remove logs

* remove log

* changed argo template

* change argocompiler dependency to my own repo

* Update go.mod

* Update VERSION

* v2 pipeline level exit handler; add test case

* Update VERSION

* remove logs and change temp dependency

* remove logs

* Update argo.go

* Update argo_test.go

* Update driver.go

* Update driver.go

* change VERSION

* Update VERSION

* Modifyb PR: remove loop fix and some changes

* update dsl compile command

* revert VERSION

* Update Makefile

* Update Makefile
2022-05-05 02:37:12 +00:00
Yaqi Ji b693909a08
feat(sdk)!: move v1 to deprecated folder (#7291)
* chore(sdk): move v1 to deprecated

* fix testsg

* fix testsg

* fix setup.py

* fix test

* s'

* fix tests

* fix test

* retore v2 test changes

* fix py3.6 test

* fix py3.6 test

* fix py3.6 import fallback error

* remove deprecated

* fix samples test

* sample test

* fix samples

* add readme

* restroe test

* python require

* remove path

* fix tests

* inteegration tests

* remove tfx tests for dependency with kfp v1

* fix e2e

* fix e2e

* fix integration tests

* fix sampe

* move client down

* change to kfp

* add import alias

* fix

* runid

* fix dsl

* only use kfp for function

* revert train_until_good

* tfx test

* kfp

* try import

* onprem
2022-03-04 00:52:59 +00:00
Yuan (Bob) Gong c7448c71df
test(v2): refactor sample test utils into a py package (#7105)
* test(v2): refactor sample test utils into a py package

* fix error
2021-12-23 11:10:01 +08:00
Chen Sun 5bbb384cc7
test: clean up v2 compatible mode sample tests (#6943) 2021-11-23 02:08:06 +00:00
Yuan (Bob) Gong ee663d9593
chore(v2): standardize MLMD data model. Fixes #5669 (#6054)
* chore(v2): standardize MLMD data model

* change context type to system namespace

* update sdk snapshots

* fix go v2 tests

* update

* update v2 compat snapshots

* fix all samples

* fix must specify pipeline root

* add artifact display name

* add UI rendering of new fields

* fix sample tests

* let ui read artifact and execution names consistently

* fix samples

* fix frontend tests

* fix sample test

* fix last sample

* address feedback
2021-07-19 22:26:15 -07:00
Chen Sun 2991602cfc
feat(sdk): Better error message when compiling for v2. Fixes #5688, Fixes #5727 (#5759)
* Better error message when compiling for v2

* update samples
2021-05-31 13:39:03 -07:00
Diana Atanasova fd72b3c9d5
refactor: update pipeline names in samples/core dir. Fixes #5683 (#5720)
Update pipeline names in sample/core directory to contain only lowercase
letters, hyphen, and numbers.

Signed-off-by: Diana Atanasova <dianaa@vmware.com>
2021-05-24 00:36:32 -07:00
Yuan (Bob) Gong 31429dc296
chore(samples): v2 compatible exit handler sample (#5541)
* chore(samples): v2 compatible exit handler sample

* legacy-exit-handler only run on v1 mode

* fix legacy_exit_handler_test

* Update exit_handler_test.py
2021-05-07 06:48:06 -07:00
Alexey Volkov cc83e1089b
Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
Yuan (Bob) Gong f757a8842f
test: add exit_handler, loop_parallism and output_a_directory pipelines to v2 sample test (#5473)
* test: add exit_handler to v2 test infra

* test: add loop_parallelism_test

* test: add output_a_directory_test

* load common component from url

* add tests to samples/test/config.yaml

* enable output directory test
2021-04-19 19:46:37 -07:00
Jiaxiao Zheng 88ee54fa32
[Samples] Change the data url to point to ml-pipeline instead of ml-pipeline-playground (#3890)
* update location in tfx sample

* update xgboost

* update the rest

* update notebook sample
2020-06-01 22:30:15 -07:00
Clayton Lemons 4286953271
Remove unused parameter (#3306) 2020-03-18 23:27:13 -07:00
Yuan (Bob) Gong 02fabd306e
[Testing] Use google/cloud-sdk:279.0.0 to resolve workload identity flakiness (#3019)
* [Testing] Use gke 1.15.8 to mitigate workload identity flakiness

* Upgrade gcloud version

* Update image builder image too

* Turn on workload identity

* Update deploy-cluster.sh

* secret sample uses python3 instead

* Increase xgboost time limit

* Revert files with bad format

* Update component and pipelines to use gcloud 279.0.0

* Fix secret sample using python3

* Upgrade frontend integration test image

* Rebuild frontend integration test image
2020-02-11 18:34:07 -08:00
Yuan (Bob) Gong 4a8d262abb Migrate standalone deployment to workload identity on GCP (#2619)
* Script to set up workload identity for standalone deployment

* Migrate tests to run on standalone + workload identity

* Fix test script

* Switch to static GSAs for testing, because they have name length limit

* Add workload identity binding for argo

* Fix argo workload identity bindings

* Remove user-gcp-sa from tests

* Remove use_gcp_secret from xgboost sample

* Allow debugging tests locally

* Wait for policies to take effect

* Update deploy-pipeline-lite.sh

* Update deploy-pipeline-lite.sh

* [WIP] test gcloud auth list with test-runner sa

* Add namespace

* test again

* Use new image builder

* test again

* Remove debug code

* Remove usages of use_gcp_secret

* Fix unit test and tensorboard pod template

* Add debug code again to test

* Try waiting until workload identity bindings are ready

* Fix some other samples

* Fix parameterized tfx oss sample

* Add retry to image building

* Try fixing tfx oss sample

* Fix compiled tfx oss sample

* Update all google/cloud-sdk to latest

* Try fixing parameterized tfx oss sample again

* Also verify pipeline-runner ksa is working

* Fix parameterized_tfx_oss sample

* Update gcp-workload-identity-setup.sh

* Revert unneeded change

* Pin to new google/cloud-sdk

* Remove wrongly commited binaries
2019-12-16 22:05:58 -08:00
Jiaxiao Zheng ef381aafcc [Sample test] Add parameterized_tfx_oss to 'normal' sample test corpus (#2695)
* Update to tfx 0.15.0 release

* update file name

* modernize naming

* move to core

* Update sample test launcher/config/file name.

* No longer need precompiled package.

* add dependency

* add pip

* no longer call dsl-compile

* fix launcher

* give additional time for init

* Try skip check in backend docker

* fix

* try fix

* revert the timeout

* fix per Alexey's suggestion

* fix typo

* print log

* fix

* remove log
2019-12-10 16:06:06 -08:00
Alexey Volkov fd6c756dd2 SDK - DSL - Make is_exit_handler unnecessary in ContainerOp (#2411)
Fixed two broken tests. The tests did not have `is_exit_handler=True` which was required before this commit.
2019-10-16 13:26:15 -07:00
Ning e8a6feb229
Restructure samples (#1710)
* restructure samples
* update apiserver and sample test for the new location
2019-08-01 17:31:37 -07:00