55f174e5bb
chore(sdk): clean up compiler_test.py ( #10070 )
2023-10-09 20:50:18 +00:00
fcdff294a6
fix(sdk): fix incorrect sub-DAG output type when using `dsl.Collected` ( #10069 )
2023-10-09 17:04:18 +00:00
87005cc015
chore(sdk): wrap conditional branch groups in outer pipeline ( #10034 )
2023-10-05 02:45:15 +00:00
2ef5afe0bb
chore(sdk): release KFP SDK 2.3.0 ( #10024 )
2023-09-22 21:32:57 +00:00
e99f2704fc
feat(sdk): support `PipelineTaskFinalStatus` in tasks that use `.ignore_upstream_failure()` ( #10010 )
...
* support taskfinalstatus in tasks that ignore upstream failure
* address review feedback
2023-09-21 01:46:37 +00:00
20218a9c1d
chore(sdk): test observability, refactorings, and cleanup ( #10005 )
...
* chore(sdk): test observability, refactorings, and cleanup
* address review feedback
* rename classes
2023-09-18 22:51:27 +00:00
d001b8055f
chore(sdk): refactor python component executor ( #9990 )
2023-09-15 22:00:30 +00:00
a13e143038
test: fix sdk execution test ( #9988 )
...
* fix sdk execution test
* fix missing key
* fix state
* fix state
* fix capitalization
2023-09-14 20:02:30 +00:00
659c54bbeb
chore(sdk): release KFP SDK 2.2.0 ( #9984 )
2023-09-14 17:37:30 +00:00
cc2cd58918
fix(sdk): fix --no-deps flag usage ( #9982 )
...
* update --no-deps flag usage
* update runtime test code
2023-09-13 15:31:02 -07:00
c6b236d1a0
feat(sdk): support dsl.If, dsl.Elif, and dsl.Else ( #9894 )
...
* support if/elif/else
* deprecate dsl.Condition
* alter rebase
* update release notes
* address review feedback
* change BinaryOperation to ConditionOperation
2023-09-11 20:19:35 +00:00
2b05ec867f
chore(sdk): remove old file ( #9959 )
2023-09-05 20:57:31 +00:00
959f54efd7
chore(sdk): release kfp sdk 2.1.3 ( #9896 )
...
* release kfp sdk 2.1.3
* update tests
* update OWNERS file
2023-09-05 18:32:31 +00:00
ef0788d986
feat(sdk): add logging at end of executor execution ( #9895 )
...
* feat(sdk): add logging at end of executor execution
* Update executor.py
* Update executor.py
2023-09-05 17:25:31 +00:00
cf0e0cf87c
feat(sdk): enable dependency-free runtime install of kfp ( #9886 )
...
* handle problematic imports
* separate runtime and compile-time symbols
* wrap kfp/__init__.py imports unavailable at runtime
* update component factory + tests
* add runtime tests
* add --no-deps flag to component factory
* update release notes
* clean up
* handle containerized python components
* update golden snapshots
* update component_factory unit tests
* respond to review feedback
* fix runtime test and compilation logic
* update tests
2023-08-24 18:31:16 +00:00
f6d0772175
chore(sdk): remove kfp-dsl source code ( #9881 )
2023-08-16 16:47:04 +00:00
c9e54798cd
chore(sdk): undo creation of kfp-dsl namespace package ( #9874 )
...
* revert to c8204d0#9742
* modify #9791
* reapply #9800
* reapply #9827
* revert parts of #9738
* reapply parts of #9785
* remove duplicated dsl-test code
* reapply parts of #9791
* correct version
2023-08-15 16:24:52 -07:00
11a5ae1dd7
chore(sdk): clean up artifact code ( #9800 )
2023-08-04 17:12:20 +00:00
25f6ee6389
chore(sdk): release KFP SDK 2.1.2 ( #9797 )
2023-07-28 16:47:34 +00:00
540294aedb
fix(sdk): fix GCPC break in KFP SDK ( #9791 )
2023-07-27 23:02:32 +00:00
49db9effe4
chore(sdk): release KFP SDK 2.1.1 ( #9787 )
2023-07-27 00:26:51 +00:00
8ad97167ff
fix(sdk): various kfp-dsl fixes ( #9785 )
...
* specify kfp-dsl version directly
* make presubmit test executable
* add yaml requirement
* assorted test and import fixes
2023-07-26 22:41:51 +00:00
309c5b9f0a
chore(sdk): release KFP SDK 2.1.0 ( #9782 )
2023-07-26 17:09:02 +00:00
525ff90684
chore(sdk): extract DSL into `kfp-dsl` package ( #9738 )
...
* move dsl dir
* make kfp-dsl a package
* make kfp-dsl a package
* additional changes
* address review feedback
2023-07-25 23:34:02 +00:00
1045c95517
chore(sdk): resolve mypy finding ( #9742 )
2023-07-17 19:54:37 +00:00
c8204d0285
chore(sdk): partition KFP SDK source code into runtime and non-runtime code ( #9710 )
2023-07-07 23:18:52 +00:00
a727783abd
chore: remove 'dummy_root' in test samples ( #9640 )
...
* remove 'dummy_root' in test samples
* more cleanup
2023-06-21 23:51:41 +00:00
6dfcd533f4
chore(sdk): release KFP SDK 2.0.1 ( #9669 )
2023-06-21 00:22:40 +00:00
e5fe981c1a
fix(sdk): fix kfp sdk v2 readme ( #9668 )
2023-06-21 00:15:40 +00:00
b3bdeb286d
chore(sdk): release KFP SDK 2.0.0 ( #9624 )
...
* bump kfp-server-api version
* release 2.0.0
* add 2.0.0 release notes
* apply suggested changes
2023-06-20 20:00:40 +00:00
92c02a6172
fix(sdk): fix click dependency bug ( #9634 )
...
* fix(sdk): fix click dependency bug
* update release notes
2023-06-20 18:33:41 +00:00
c33afa74f3
chore(sdk): relax name validation when upload pipeline ( #9646 )
...
* relax name validation when upload pipeline
* fix typo
2023-06-16 22:20:36 +00:00
a9ac0b95bc
fix(sdk): print docker error messages ( #9595 )
...
Without this commit, `kfp component build` doesn't print any error
messages that docker emits, e.g. if docker push fails.
Co-authored-by: Christian Geier <geier@lostpackets.de>
2023-06-13 00:47:02 +00:00
b6c660853e
chore(sdk): release KFP SDK 2.0.0-rc.2 ( #9605 )
...
* release 2.0.0-rc.2
* update comment
2023-06-09 10:59:05 -07:00
48ecc7d879
fix(sdk): fix bug when iterating over output from upstream task in nested parallelfor loops ( #9580 )
...
* refactor for clarity
* fix iter over upstream output in nested parallelfor
* update release notes
* support empty `List` generic annotation
* fix typos
* add comment about refactoring opportunity
* fix annotation
* update comment again
2023-06-09 08:58:28 -07:00
0587a3fd35
chore(sdk): loosen python kubernetes version requirement ( #9545 )
...
* loosen python kubernetes version requirement
* update sdk release note
2023-06-01 19:17:06 +00:00
296120290b
fix(sdk): relax executor type checking ( #9481 )
...
* relax executor type checking
* address review comments
* add release note
2023-05-23 19:05:35 +00:00
209df547ef
chore(sdk): release KFP SDK 2.0.0-rc.1 ( #9429 )
2023-05-16 18:23:00 +00:00
96742a648a
docs(sdk): add dsl placeholder exports ( #9423 )
2023-05-15 19:46:31 +00:00
78902274f1
feat(sdk): support compiling components with `dsl.PipelineTaskFinalStatus` type ( #9082 )
2023-05-12 22:14:28 +00:00
4d2a1cf173
fix(sdk): Optimize wait_for_run_completion sdk function to skip an extra wait ( #9407 )
2023-05-12 21:49:28 +00:00
96947e6fb9
fix(sdk): fix compilation of boolean constant passed to component ( #9390 )
2023-05-12 17:13:39 +00:00
c01288d673
feat(sdk): support writing `TASK_FINAL_STATUS` parameter type ( #9080 )
...
* write new parameter type
* update golden snapshots
* add compiler tests
2023-05-12 17:13:32 +00:00
3ad91a9bbe
chore: release KFP SDK 2.0.0-beta.17 ( #9355 )
...
* Release KFP SDK 2.0.0-beta.17
* update release note
2023-05-08 17:42:55 +00:00
b6b8f94da2
chore: recompile/update v2 samples to avoid possible runtime breakage ( #9333 )
...
* recompile/update v2 samples
* fix test
2023-05-05 17:28:24 +00:00
58ebd202c4
release 2.0.0b16 ( #9325 )
2023-05-04 10:56:24 -07:00
c437004761
fix(sdk): fix appengine import error ( #9323 )
2023-05-04 17:31:45 +00:00
b59b70875e
fix(sdk): allow user to choose platform for component container build. Fixes #8883 ( #9212 )
...
Co-authored-by: Jonny Browning <jonathan.browning@datatonic.com>
2023-04-25 19:41:45 +00:00
04306dc3cb
chore(sdk): release KFP SDK 2.0.0b15 ( #9218 )
2023-04-24 18:32:43 +00:00
946c51bafe
feat(sdk): support extracting input/output descriptions from component/pipeline docstrings ( #9156 )
...
* feat(sdk): support extracting input/output descriptions from component/pipeline docstrings
* remove numpydoc tests
2023-04-24 17:16:44 +00:00
91277b1e91
fix(SDK): fix `client.run_pipeline` with `pipeline_id` and `version_id` gives error ( #9191 )
2023-04-20 21:17:40 +00:00
41c0d78f5f
fix(sdk): fix handling of PipelineTaskFinalStatus in ir_type_ attribute hack ( #9186 )
2023-04-20 13:53:31 -07:00
91abbeaf2f
feat(sdk): support `display_name` and `description` in `@dsl.pipeline` decorator ( #9153 )
...
* feat(sdk): support display_name and description in @dsl.pipeline decorator
* add release note
* test additional case
2023-04-17 14:15:39 -07:00
3164dfb01a
fix(sdk): fixes module not found error for containerized python components. Fixes #8385 ( #9157 )
...
* Add module directory to sys.path
* Add nested module imports unit test
* Add release note to release.md
2023-04-17 11:46:02 -07:00
30466586d0
chore(sdk): add kfp-kubernetes as kfp extras_require ( #9152 )
2023-04-17 10:08:22 -07:00
80bd8f17b1
chore(sdk): bump kfp version in test data to 2.0.0-beta.14 ( #9145 )
2023-04-12 16:27:09 -07:00
a107fb1bcf
chore: release SDK 2.0.0b14 ( #9144 )
...
* release SDK 2.0.0b14
* doc ref
2023-04-12 14:29:29 -07:00
952e0730a8
feat(sdk): Support submitting pipeline with platform config. ( #9140 )
...
* SDK client v2beta1 API integration.
* fix unit tests
* create alias for use of methods and fields referencing `job`.`
* fix cli
* fix pipeline list-versions
* more cli fixes
* client fix
* Support submitting pipeline with platform spec.
* unit test
* move util functions
2023-04-12 13:06:12 -07:00
07d6c1816b
feat(SDK)!: SDK client v2beta1 API integration. Fixes #8706 ( #9112 )
...
* SDK client v2beta1 API integration.
* fix unit tests
* create alias for use of methods and fields referencing `job`.`
* fix cli
* fix pipeline list-versions
* more cli fixes
* client fix
2023-04-12 09:54:45 -07:00
5e98b3b6f0
feat(sdk): Support PIPELINE_ROOT_PLACEHOLDER ( #9134 )
...
* Support PIPELINE_ROOT_PLACEHOLDER
/assign @connor-mccarthy
* Update __init__.py
2023-04-11 08:56:49 -07:00
a6ef77b784
feat(sdk): Support setting cpu/memory requests. ( #9121 )
...
* Support setting cpu/memory requests.
* address review comments
* address review comments
2023-04-10 23:35:16 -07:00
e907b6343e
set _ir_type attribute ( #9105 )
2023-04-10 12:01:36 -07:00
5828632c2f
feat(sdk): support indexing into top-level artifact metadata in Container Components ( #9131 )
2023-04-10 11:46:01 -07:00
5a769e9080
Revert "update compiler type annotations ( #9090 )" ( #9095 )
...
This reverts commit ad6f55e8f3
2023-04-03 16:01:26 -07:00
ad6f55e8f3
update compiler type annotations ( #9090 )
2023-04-03 09:20:17 -07:00
0cbcebcf88
fix(sdk): support accessing `.dsl` attribute on `kfp` module object ( #9048 )
2023-03-27 15:12:17 -07:00
3026e452ff
fix(sdk): fix loading non-canonical type strings from v1 component YAML ( #9041 )
2023-03-27 10:25:23 -07:00
3e1f3c6d7a
feat(sdk): add experiment_id parameter to create run methods ( #9004 )
...
Co-authored-by: alenawang <alena.n.wang@gmail.com>
Co-authored-by: andreafehrman <andrea.k.fehrman@vanderbilt.edu>
Co-authored-by: owmasch <owenmaschal0598@gmail.com>
Co-authored-by: ryanrusson <ryan.russon@gmail.com>
2023-03-24 09:56:48 -07:00
2bbfd5e89f
fix: support setting task dependencies via kfp.kubernetes.mount_pvc ( #8999 )
...
* enable accessing .task on pipeline channel
* set task dependencies in mount_pvc
* update tests
2023-03-16 21:10:54 -07:00
0fd1e69c74
cpu and memory resource can be optional ( #8992 )
...
Co-authored-by: Ze Mao <zemao@google.com>
2023-03-16 09:23:55 -07:00
e96754cb53
feat(sdk): deprecate .add_node_selector_constraint in favor of .set_accelerator_type ( #8980 )
...
* rename method
* update docstring
2023-03-14 09:22:35 -07:00
a38d7479d7
feat(sdk): support compiling platform specific features ( #8940 )
...
* support compiling platform-specific features
* fix map size change during iteration
2023-03-10 11:18:25 -08:00
abdc08b773
chore(sdk): pin kfp-pipeline-spec to 0.2.1 ( #8971 )
2023-03-10 10:19:03 -08:00
6777e8f445
fix(sdk): Support python 3.11 ( #8907 )
...
* new branch
* add to release.md
* edit setup.py
2023-03-03 13:36:53 -08:00
ed66ba327f
feat(sdk): using component's pip_index_urls for Dockerfile generation. Fixes #8816 ( #8871 )
...
* feat(sdk): using component's pip_index_urls for Dockerfile generation
* Update sdk/python/kfp/cli/component.py
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
* feat(sdk): PR changes: removed additional space from index url options string; duplicate removal not necessary
* feat(sdk): feature mentioned in RELEASE.md
* deduplication back - more components in a file can specify same pypi url - we need it only once for docker image
* feature note moved to current development section
---------
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
2023-03-01 09:32:56 -08:00
86554be46a
release kfp sdk 2.0.b13 ( #8904 )
2023-02-27 16:52:53 -08:00
87062dbf22
fix(sdk): fix attribute error for Containerized Python Components ( #8887 )
2023-02-27 16:42:50 -08:00
8d3e0dd805
fix(sdk): Fix environment variable set in component yaml lost during compilation. Fixes #8884 ( #8885 )
...
* Fix env set in component yaml not captured
* release note
* clean up unnecessary code in test
2023-02-23 14:28:26 -08:00
386ff88236
fix(sdk): fix unknown artifact string in dsl.OutputPath/dsl.InputPath ( #8880 )
2023-02-23 08:37:04 -08:00
31298c479b
fix(sdk): fix bug when passing data between tasks using f-strings ( #8879 )
2023-02-23 08:36:53 -08:00
c046879dda
support fanning-in artifacts ( #8808 )
2023-02-22 10:39:00 -08:00
41752608bc
support list of artifact input placeholders ( #8484 )
2023-02-16 09:26:53 -08:00
1e1a02066a
fix(sdk): Add outputDefinitions of root pipeline to compiled components IR YAML file ( #8848 )
...
* base
* add to readme
* adress comments
2023-02-15 09:28:33 -08:00
8552226c41
chore(test): revert api changes to deprecated client ( #8841 )
...
* revert api changes to deprecated client
* list experiments
* fix
* fix sample v2 test
2023-02-14 11:01:58 -08:00
5684bfccff
feat(sdk): Ignore upstream failure ( #8838 )
...
* main
* adress comments
* adress comments
* adress comments 2
* address comments 3
* address comments 4
2023-02-13 11:40:42 -08:00
e7ad073f1d
feat(sdk): deprecate .set_gpu_limit in favor of .set_accelerator_limit ( #8836 )
2023-02-09 15:46:10 -08:00
0fc174ae28
chore(sdk): release KFP SDK 2.0.0b12 ( #8823 )
2023-02-08 09:46:17 -08:00
cc5373281d
chore(sdk): Pin kfp-server-api to an exact version ( #8822 )
2023-02-07 23:04:44 -08:00
b575950a75
feat(sdk): support for fan-in from `dsl.ParallelFor` ( #8631 )
...
* implement parallelfor fan-in
* temporarily block fan-in of artifacts
* clean up
* unify and clarify error messages
* remove comment
* remove unnecessary compiler condition
* make test cases more realistic
* use better ParallelFor variable name
* add and rename test cases
* add ParallelFor fan-in test cases
* fix return collected bug
* clean up compilation tests
* update docstrings; update error message
* remove fan-in artifacts read/write tests
* implement review feedback
* change use of additional_input_name_for_pipeline_channel
* move additional_input_name_for_pipeline_channel
* provide additional pipeline topology validation and tests; refactor existing validation
* add release note for parameter fan-in
2023-02-04 20:50:51 -08:00
e50f40305d
fix(sdk): fix upload_pipeline when no pipeline name is provided ( #8695 )
2023-01-31 13:45:39 -08:00
edfb267790
bump version; update release notes; add docs ( #8745 )
2023-01-20 12:56:56 -08:00
1c7ab4c5f5
pin kfp-pipeline-spec version exactly ( #8737 )
2023-01-20 12:49:36 -08:00
78d31dc8a3
chore(sdk): remove v1 dependencies from SDK v2 ( #8668 )
...
* split out deprecated requirements
* pip install deprecated requirements for sample tests
* remove superfluous comments in requirements.in
* update release notes
* ignore deprecated directory in unit tests
* remove unused argo compiler
* ignore deprecated path
* update run sample yaml
* install deprecated requirements for e2e tests
* add todos
2023-01-19 12:53:27 -08:00
07615eb88a
chore(sdk): Improve CLI utility function ( #8686 )
2023-01-17 14:51:50 -08:00
5adc7f1b50
chore(sdk): clean up client.py ( #8656 )
2023-01-17 11:04:46 -08:00
6040334e84
feat(sdk): Accepts higher PyYAML versions ( #8665 )
...
* Accepts higher PyYAML versions
* Update the release note
2023-01-17 10:46:42 -08:00
fdf3ee7b68
chore(sdk): release KFP SDK 2.0.0b10 ( #8638 )
...
* bump version
* update release notes
* add docs version
2022-12-29 18:05:36 -05:00
84241d6460
fix(sdk): fix deserializing v1 component yaml defaults ( #8639 )
...
* fix deserializing v1 defaults
* update test cases
* update type annotation
* add unit tests
2022-12-29 17:48:58 -05:00
151e8a3c6e
feat(sdk): support optional artifact inputs ( #8623 )
...
* add compiler tests
* support optional artifact inputs
* update executor test
* update existing golden snapshot
* add new compiler test/snapshot
* add release note
* bump kfp-pipeline-spec dependency
2022-12-28 13:40:12 -05:00
5491f6f3db
fix(sdk): support writing/reading isOptional for input parameters ( #8612 )
...
* add tests
* support writing/reading isOptional for parameters
* update golden snapshots
* temporarily maintain workaround for v1 optional artifact inputs
* add release note
2022-12-21 14:46:59 -08:00
Connor McCarthy
Chen Sun
Alex
hsteude
Tommy Li
Jonny Browning
Mithil Poojary
IronPan
Alex
Ze Mao
JOCSTAA
b4sus
Tara Tufano
Owen
Mai Nakagawa