1d9690321f
fix(samples): Updated samples/core to V2 ( #9879 )
...
* Updated output_a_directory.py to V2
* Update output_a_directory_test.py to V2
* Update parallel_join.py to V2
* Update multiple_outputs.ipynb to V2
* Update multiple_outputs_test.py to V2
* Updated kfp_env_validation to V2
* Updated loop_parallelism to V2
2024-03-06 22:57:14 +00:00
b40912cc5d
fix(sdk): use kfp.dsl.types to replace kfp.components.types Fixes #10282 ( #10283 )
...
* use kfp.dsl.types to replace kfp.components.types
Fixes : #10282
Signed-off-by: hsinhoyeh <yhh92u@gmail.com>
* fix import artifact_types
---------
Signed-off-by: hsinhoyeh <yhh92u@gmail.com>
2023-12-12 19:38:38 -05:00
cd7e316db6
test: upgrade e2e sample tests to using sdk v2 ( #9885 )
...
* install kfp sdk 2.0 in sample test
* remove reference to GCSHelper from deprecated module
* exclude obsolete samples
* update test script
* update client usage
* revert recent changes to exit_handler.py
2023-08-17 09:33:38 +00:00
90cec167c0
fix(samples): Update volume_ops sample to v2 pipelines ( #9877 )
...
* Update volume_ops sample to v2 pipelines
* Update samples/core/kubernetes_pvc/kubernetes_pvc.py
minor change on copyright range
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
---------
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
2023-08-17 00:21:38 +00:00
3e3a747b58
fix(samples): Update execution_order, loop_output samples to v2 pipelines ( #9867 )
...
* Update execution_order, loop_output samples to v2 pipelines
* Update execution_order, loop_output samples to v2 pipelines
2023-08-16 14:37:03 +00:00
ff2e002157
fix(samples): Update loop_parameter, loop_static samples to v2 pipelines ( #9870 )
2023-08-15 17:59:04 +00:00
1002e0cf8c
fix(samples): update samples to v2 pipelines ( #9851 )
...
* Update samples to v2 pipelines
* lint
2023-08-11 19:53:33 +00:00
f626629f79
chore(testing): Enable samples in e2e tests with v1 kfp SDK ( #9169 )
...
* Add pipeline_id to run and recurring run protos
* Add pipeline_id to run and recurring_run
* Set status when creating a new job. Closes #9125 .
* Enable sample tests
2023-04-19 04:55:53 +00:00
37dac45caf
feat(backend): add support for resource requests. Closes #7047 . Closes #6354 . ( #9088 )
...
* Add support for resource requests
* Update pipelinespec reference
* Stash unit tests
* Fix tests
* Update license
* Update driver image
* Fix tests
* Add resource requests sample test
* Revert sample test and add TODO
2023-04-09 17:51:08 -07:00
5fe67919db
chore(components): clean up deprecated GCP components ( #8685 )
...
* delete deprecated GCP components
* remove build and test scripts, references to code/folder from master
2023-02-14 22:42:33 +00:00
bdff332ac6
feat(sdk): support more than one exit handler per pipeline ( #8088 )
...
* add compiler test pipeline with multiple exit handlers
* remove blocker of multiple exit handlers
* move exit handler builder logic to pipeline_spec_builder
* build all exit handlers per pipeline
* add compiler test with IR inspection
* prevent usage of cross-pipeline after
* test cross-pipeline after is prevented
* update existing task dependency logic and tests
* add v2 sample test
* remove cross-pipeline .after
* prevent cross-dag data dependency for dsl features
* add compiler test pipeline with nested exit handlers
* add support for nested exit handlers
* clean up pipeline with nested exit handlers
* remove sample with multiple exit handlers
* remove compiler test with nested exit handlers
* add compilation guard against nested exit handlers in subdag
* update release notes
2022-08-10 16:56:21 +00:00
d25bbc88b1
fix(test): Make exit handler test install KFP SDK from source ( #7785 )
...
* Update exit_handler_v2.py
* Update exit_handler_v2.py
* Update exit_handler_v2.py
2022-06-01 23:30:51 +00:00
98a58c7a0c
Pipeline level exit handler and v2 test ( #7642 )
...
* support IR YAML format in API
* Check the error message and return false if it is not nil
* update error message
* fixed simple loop but need cleaning up
* Deleted debug logs
* remove logs and fix some format
* fix static_loop_arguments
* change the driver image
change the driver image back to the kfp container registry.
* change variable declaration
* remove logs
* remove log
* changed argo template
* change argocompiler dependency to my own repo
* Update go.mod
* Update VERSION
* v2 pipeline level exit handler; add test case
* Update VERSION
* remove logs and change temp dependency
* remove logs
* Update argo.go
* Update argo_test.go
* Update driver.go
* Update driver.go
* change VERSION
* Update VERSION
* Modifyb PR: remove loop fix and some changes
* update dsl compile command
* revert VERSION
* Update Makefile
* Update Makefile
2022-05-05 02:37:12 +00:00
1180e10632
fix(backend): fix simple loop bug ( #7578 )
...
* support IR YAML format in API
* Check the error message and return false if it is not nil
* update error message
* fixed simple loop but need cleaning up
* Deleted debug logs
* remove logs and fix some format
* fix static_loop_arguments
* change the driver image
change the driver image back to the kfp container registry.
* change variable declaration
* remove logs
* remove log
* move `ok` definition
* change test file for debug purpose
* change test for debug purpose
* update sample test for static loop
* update test file, remove code for debug
2022-05-05 01:31:12 +00:00
67c19fab67
fix: typo in VolumeOps example ( #7479 )
2022-03-31 19:07:31 +00:00
8bb0d0e025
feat(sdk)!: move v2 to main namespace ( #7376 )
...
* feat(sdk)!: move v2 to main namespace
* edit samples
2022-03-04 21:56:51 +00:00
b693909a08
feat(sdk)!: move v1 to deprecated folder ( #7291 )
...
* chore(sdk): move v1 to deprecated
* fix testsg
* fix testsg
* fix setup.py
* fix test
* s'
* fix tests
* fix test
* retore v2 test changes
* fix py3.6 test
* fix py3.6 test
* fix py3.6 import fallback error
* remove deprecated
* fix samples test
* sample test
* fix samples
* add readme
* restroe test
* python require
* remove path
* fix tests
* inteegration tests
* remove tfx tests for dependency with kfp v1
* fix e2e
* fix e2e
* fix integration tests
* fix sampe
* move client down
* change to kfp
* add import alias
* fix
* runid
* fix dsl
* only use kfp for function
* revert train_until_good
* tfx test
* kfp
* try import
* onprem
2022-03-04 00:52:59 +00:00
0c0e4b6105
fix(samples): Replace deprecated tfx kubeflow example ( #7342 )
...
**Problem:**
In this commit cd029714cbhttps://raw.githubusercontent.com/tensorflow/tfx/master/tfx/examples/chicago_taxi_pipeline/taxi_pipeline_kubeflow_gcp.py (broken link).
**Solution:**
Replace references to taxi pipeline tfx kubeflow example with penguin tfx kubeflow example.
2022-02-24 09:19:04 +00:00
e74c7bd006
fix(samples): Update tensorboard_minio.py to load component from a permanent link ( #7211 )
...
* Update tensorboard_minio.py
* Update tensorboard_minio.py
2022-01-25 18:35:20 +00:00
f9db81e2c8
test: fix integration-v2 test ( #7122 )
...
* Update parameterized_tfx_oss.py
* test: fix integration-v2 test
* reduce test log noise
2021-12-29 19:03:48 +00:00
2460aaa653
test: migrate sample integration test to v2 sample test infra ( #7115 )
...
* test: migrate sample integration test to v2 sample test infra
* test: configure v2 integration test timeout
2021-12-26 04:17:44 +00:00
ac14a3b610
test: migrate all sample tests to v2 sample test infra ( #7114 )
...
* test: migrate all sample tests to v2 sample test infra
* fix
2021-12-24 08:15:43 +00:00
44228c242b
test(v2): v2 sample test supports notebooks ( #7109 )
...
* test(v2): v2 sample test supports notebooks
* fix
* fix relative path problem
2021-12-24 02:37:42 +00:00
c7448c71df
test(v2): refactor sample test utils into a py package ( #7105 )
...
* test(v2): refactor sample test utils into a py package
* fix error
2021-12-23 11:10:01 +08:00
048218b091
feat(v2): control flow - conditional ( #7088 )
...
* feat(v2): control flow - conditional
* fix v2 go tests
* fix importer sample
added condition_test
* address comments
* fix
2021-12-22 10:13:21 +08:00
b37e575de1
refactor(v2): driver decides container spec via pod spec patch ( #7060 )
...
* refactor(v2): driver decides container spec via pod spec patch
* fix unstable protojson marshal result
* clean up importer as well
2021-12-19 17:35:54 +08:00
8c9d829909
chore(v2): v2 condition sample and expression ( #7059 )
...
* chore(v2/samples): condition v2 sample
* chore(v2): condition expression
* driver changes for condition control flow
2021-12-14 12:06:10 +00:00
f461f479d8
chore(v2): static memory & cpu resource limits ( #7045 )
...
* chore(v2): memory & cpu resource limits
* fix v2 go test
* add launcher init container resources
* update snapshots
2021-12-14 01:50:09 +00:00
26d33ead40
chore(v2): v2 iterator expr selector support ( #7020 )
...
* chore(v2): parameter expression selector support
* implement input parameter type validation and conversion
* fix some issues
* test: add tests for remaining loop samples
* fix
* fix2
* fix3
* address comments
* fix some tests
2021-12-10 19:01:49 +00:00
76b03cd2d8
chore: Remove AI Platform sample ( #7026 )
2021-12-09 19:31:56 +00:00
39af924cb2
chore: Remove Container Build sample ( #7027 )
2021-12-09 18:44:55 +00:00
713558cfb8
feat(v2): iterator and sub-DAG basic support ( #6985 )
...
* chore(samples): add loop_output_v2 sample
* feat(v2): iterator support
* refactor v2 backend compiler tests to golden files
* fix unit tests
* test(v2): verify loop_output_test MLMD data
* clean up
* fix
* fix2
* fix
2021-12-09 14:23:55 +00:00
dcf564eae4
test(samples): fix xgboost sample OOM flakiness ( #7030 )
2021-12-09 21:30:57 +08:00
3cfff3db1d
fix(samples): Use TFX library version for parameterized_tfx_oss. Fix #6974 ( #6993 )
...
* fix(samples): Use TFX library version for parameterized_tfx_oss
* change version reference
2021-12-02 10:00:04 +00:00
ca9b9444ac
chore(release): bumped version to 1.8.0-alpha.0
2021-12-01 00:49:31 +00:00
f099b93a2b
test(samples): simplify v2 sample test verify funcs + clean up ( #6964 )
...
* test(samples): simplify v2 sample test verify funcs + clean up
* chore(samples): add placeholder_if tests back, fix flakiness in xgboost sample
* test(samples): clean up empty tests in config.yaml
* chore: add placeholder_if back
* chore(samples): clean up all samples
* fix
* Update xgboost_sample.py
2021-11-30 10:55:30 +00:00
288caa67e9
test(samples): fix xgboost sample OOM flakiness ( #6968 )
2021-11-26 13:33:54 +00:00
5bbb384cc7
test: clean up v2 compatible mode sample tests ( #6943 )
2021-11-23 02:08:06 +00:00
73804f8928
test(sample): reenable samples tests run with V2_ENGINE mode ( #6932 )
...
* reenable v2 engine
* fix
* fix
* fix
* log v2 engine test
* don't fill in default value at test util
* fix
* pass to job instead of spec
2021-11-20 18:18:35 +00:00
e094b0aa4b
chore(release): bump version to 1.7.1 on master branch ( #6842 )
2021-11-01 21:39:59 -07:00
6ccf5bcd0b
chore(release): bumped version to 1.7.0
2021-08-25 06:44:50 +00:00
78491d12b8
chore(release): bumped version to 1.7.0-rc.4
2021-08-19 05:04:29 +00:00
91dc80d86a
feat(sample): Use TFX 1.2.0 for Taxi tips prediction sample. Partial #6138 ( #6381 )
...
* fix(sample): Use TFX1.2.0 for Taxi tips prediction sample
* also update python
* Update parameterized_tfx_oss.py
* Update taxi_pipeline_notebook.ipynb
* Update parameterized_tfx_oss.py
Co-authored-by: Yuan (Bob) Gong <4957653+Bobgy@users.noreply.github.com>
2021-08-19 09:56:40 +08:00
053edb518d
chore: use a runtime parameter for pusher destination in the TFX example. ( #6373 )
...
{} Placeholder doesn't work well in component parameters and
it is better to have it as a runtime parameter for flexibility.
https://github.com/kubeflow/pipelines/issues/6311#issuecomment-900245391
2021-08-17 20:12:37 -07:00
afd28c605d
test(v2): add more samples. Part of #6150 ( #6353 )
...
* test(v2): add all samples using parameter + artifact passing features to v2 engine
* fix some samples
* fix output file
* comment failing samples
* improve comments
2021-08-16 21:38:41 -07:00
befd89c6ec
fix(sdk): pipeline name is required in v2 compatible mode. Fixes #6197 ( #6332 )
...
* fix(sdk): pipeline name is required in v2 compatible mode
* Update xgboost_sample.py
2021-08-13 14:00:04 -07:00
7ae0d9f935
chore: Fixes wrong Evaluator config. ( #6326 )
...
* Fixes wrong Evaluator config.
Also changes dataset to the one included in the tfx Python package.
* Rollback {{kfp-default-bucket}}.
This placeholder will be replaced if Kubeflow Pipelines is installed
from Marketplace.
2021-08-13 01:43:54 -07:00
a1f7160cf5
chore: upgrade tfx to 1.0.0 ( #6316 )
...
* test: fix sample test image build
* update to TFX 1.0.0
* update argo CLI
* Uses TFX 1.0 APIs in parameterized_tfx_oss example.
Co-authored-by: Yuan Gong <gongyuan94@gmail.com>
2021-08-12 04:02:03 -07:00
402549d81f
fix(samples): Add `kfp_package_path` to a core sample test. ( #6259 )
2021-08-06 16:49:26 -07:00
3c6d1285a0
chore(release): bumped version to 1.7.0-rc.3
2021-08-06 07:13:16 +00:00
Pratyusha R
hsinhoyeh
Chen Sun
Junggil Lee
Junggil Lee
Ryan Jaemun Jung
gkcalat
Connor McCarthy
Lingqing Gan
Benjamin Tan Wei Hao
Yaqi Ji
vr140
Yuan (Bob) Gong
James Liu
Yuan Gong
Jiyong Jung
Ajay Gopinathan