Commit Graph

1352 Commits

Author SHA1 Message Date
connor-mccarthy 7ec10c63d9 improve DockerRunner tests 2024-01-05 14:00:20 -08:00
Connor McCarthy 2897a10f59
feat(sdk): support Concat and IfPresent placeholder in local container component execution #localexecution (#10348)
* feat(sdk): support Concat and IfPresent placeholder in local container component execution #localexecution

* address review feedback

* fix test
2024-01-05 21:53:56 +00:00
Connor McCarthy 64d46dfed0
fix(sdk): permit empty local execution outputs #localexecution (#10338) 2024-01-05 20:52:08 +00:00
Connor McCarthy 8a5a17e910
feat(sdk): remove local execution feature flag #localexecution (#10355) 2024-01-05 20:20:08 +00:00
Connor McCarthy dcab0511a2
test(sdk): improve KFP SDK local runner test safety #localexecution (#10336) 2024-01-03 18:39:15 +00:00
Connor McCarthy 86b7e23985
fix(sdk): remove redundant newline character in local `DockerRunner` logs (#10354) 2024-01-02 23:40:14 +00:00
Connor McCarthy 89d4234a5b
fix(sdk): fix presentation of strings in local execution #localexecution (#10353) 2024-01-02 23:00:14 +00:00
Connor McCarthy 846f88770c
feat(sdk): support local Container Component execution #localexecution (#10333)
* support Container Components

* address review feedback
2023-12-20 18:34:20 +00:00
Connor McCarthy f52ba56784
chore(sdk): write local execution logs to stdout #localexecution (#10330) 2023-12-20 09:37:20 +00:00
Connor McCarthy 654bbdebe6
feat(sdk): add special `dsl.OutputPath` read logic #localexecution (#10334) 2023-12-20 09:09:20 +00:00
Connor McCarthy adc5b3b160
feat(sdk): add DockerRunner #localexecution (#10328) 2023-12-19 23:14:19 +00:00
Connor McCarthy 784927205c
feat(sdk): add local execution logging #localexecution (#10326) 2023-12-18 23:50:37 +00:00
Connor McCarthy d0da0ad940
chore(sdk): fix use of invalid escape sequence in tests (#10310) 2023-12-18 22:53:38 +00:00
Connor McCarthy 76aad8b18a
feat(sdk): add local execution output collection #localexecution (#10325)
* feat(sdk): add local execution output collection #localexecution

* update ExecutorInput processing

* address review feedback

* fix exception handling test
2023-12-16 09:53:18 -05:00
Connor McCarthy 21f8e9c72b
feat(sdk): add subprocess task handler #localexecution (#10302)
* add subprocess task handler

* response to review feedback
2023-12-15 19:51:55 +00:00
Connor McCarthy 33bf920255
test(sdk): add placeholder_utils_test.py (#10301) 2023-12-13 13:59:57 -05:00
hsinhoyeh b40912cc5d
fix(sdk): use kfp.dsl.types to replace kfp.components.types Fixes #10282 (#10283)
* use kfp.dsl.types to replace kfp.components.types

Fixes: #10282
Signed-off-by: hsinhoyeh <yhh92u@gmail.com>

* fix import artifact_types

---------

Signed-off-by: hsinhoyeh <yhh92u@gmail.com>
2023-12-12 19:38:38 -05:00
Connor McCarthy d41efc3e96
update task dispatcher (#10298) 2023-12-12 14:23:49 -05:00
Connor McCarthy 5cd708de37
feat(sdk): add local execution skeleton #localexecution (#10292) 2023-12-09 14:08:21 -05:00
Connor McCarthy 5c60d37616
remove cleanup param in local init (#10293) 2023-12-09 14:07:58 -05:00
Connor McCarthy d3b3e7a5ba
chore(sdk): support Python 3.12 (#10272)
* chore(sdk): support Python 3.12

* fix kfp-kubernetes test import error

* use assertEqual instead of assertEquals

* fix kfp-kubernetes snapshot yaml
2023-11-29 23:18:03 +00:00
Connor McCarthy 11f60d813a
feat(sdk): support `.after()` referencing task in `ParallelFor` group (#10257) 2023-11-29 10:22:14 -05:00
Connor McCarthy d3323c06f3
feat(sdk): add executor output path and executor input message placeholders (#10240)
* add support for '{{$.outputs.output_file}}' placeholder

* dedupe executor input code
2023-11-20 20:30:10 +00:00
Connor McCarthy 0d7913ce4e
feat(sdk): add local execution config #localexecution (#10234) 2023-11-20 18:18:10 +00:00
Connor McCarthy b4d7e1857b
chore(sdk): nest requests_toolbelt import (#10247) 2023-11-20 18:12:10 +00:00
Connor McCarthy 23fb94caa8
chore(sdk): update CONTRIBUTING.md testing instructions (#10218)
Required after #10160, since specifying a test path will override the `pytest.ini` configuration.
2023-11-08 18:28:09 +00:00
Connor McCarthy f250700e4d
chore(sdk): release KFP SDK 2.4.0 (#10167) 2023-10-27 17:04:10 +00:00
Connor McCarthy 03df9df68c
chore(sdk): fix local test failure (#10161) 2023-10-26 01:39:16 +00:00
Mathew Wicks 02e00e8439
fix(sdk): type annotation for client credentials (#10158) 2023-10-26 01:39:10 +00:00
rickyxie0929 7ab05d8a84
chore(sdk): Remove the ` (<type>)`from docstring args. (#10159)
* chore(sdk): Remove the ` (<type>)`from docstring args.

Remote the ` (<type>)` from the docstring args, which are redundant since there are type annotations.

* Trim the unnecessary leading space.
2023-10-26 00:29:09 +00:00
Connor McCarthy e3b1863791
chore(sdk): add Python 3.7 EOL notice (#10139) 2023-10-24 22:06:08 +00:00
Connor McCarthy c0adea9946
chore(sdk): add test for key error bug resolved in #10067 (#10128) 2023-10-19 23:49:03 +00:00
Connor McCarthy 8d00d0eb9a
feat(sdk): support a Pythonic artifact authoring style (#9932)
* chore(sdk): test observability, refactorings, and cleanup

* implement unified i/o syntax

* clarify container component error message

* address review feedback

* add dill to packages_to_install

* update get_uri logic for KFP and tests

* update execution tests & cleanup
2023-10-19 18:49:58 +00:00
Connor McCarthy 73d51c8a23
fix(sdk): fix bug when `dsl.importer` argument is provided by loop variable (#10116) 2023-10-18 20:37:56 +00:00
Connor McCarthy 2d3171cbfe
feat(sdk): support collecting outputs from conditional branches using `dsl.OneOf` (#10067)
* support dsl.OneOf

* address review feedback

* address review feedback
2023-10-18 02:34:55 +00:00
gkcalat ab33e2af4e
chore(sdk): Improve a message printed for users. Closes #10100 (#10101) 2023-10-15 22:58:53 +00:00
gkcalat c9032716ab
fix(sdk): Fix OOB for IPython and refactor. Closes #10075. (#10094)
* Fix OOB for IPython and refactor. Closes #10075.

* Address comments
2023-10-13 22:24:52 +00:00
Connor McCarthy 55f174e5bb
chore(sdk): clean up compiler_test.py (#10070) 2023-10-09 20:50:18 +00:00
Connor McCarthy fcdff294a6
fix(sdk): fix incorrect sub-DAG output type when using `dsl.Collected` (#10069) 2023-10-09 17:04:18 +00:00
Connor McCarthy 87005cc015
chore(sdk): wrap conditional branch groups in outer pipeline (#10034) 2023-10-05 02:45:15 +00:00
Connor McCarthy 2ef5afe0bb
chore(sdk): release KFP SDK 2.3.0 (#10024) 2023-09-22 21:32:57 +00:00
Connor McCarthy e99f2704fc
feat(sdk): support `PipelineTaskFinalStatus` in tasks that use `.ignore_upstream_failure()` (#10010)
* support taskfinalstatus in tasks that ignore upstream failure

* address review feedback
2023-09-21 01:46:37 +00:00
Connor McCarthy 20218a9c1d
chore(sdk): test observability, refactorings, and cleanup (#10005)
* chore(sdk): test observability, refactorings, and cleanup

* address review feedback

* rename classes
2023-09-18 22:51:27 +00:00
Connor McCarthy d001b8055f
chore(sdk): refactor python component executor (#9990) 2023-09-15 22:00:30 +00:00
Chen Sun a13e143038
test: fix sdk execution test (#9988)
* fix sdk execution test

* fix missing key

* fix state

* fix state

* fix capitalization
2023-09-14 20:02:30 +00:00
Connor McCarthy 659c54bbeb
chore(sdk): release KFP SDK 2.2.0 (#9984) 2023-09-14 17:37:30 +00:00
Connor McCarthy cc2cd58918
fix(sdk): fix --no-deps flag usage (#9982)
* update --no-deps flag usage

* update runtime test code
2023-09-13 15:31:02 -07:00
Connor McCarthy c6b236d1a0
feat(sdk): support dsl.If, dsl.Elif, and dsl.Else (#9894)
* support if/elif/else

* deprecate dsl.Condition

* alter rebase

* update release notes

* address review feedback

* change BinaryOperation to ConditionOperation
2023-09-11 20:19:35 +00:00
Connor McCarthy 2b05ec867f
chore(sdk): remove old file (#9959) 2023-09-05 20:57:31 +00:00
Connor McCarthy 959f54efd7
chore(sdk): release kfp sdk 2.1.3 (#9896)
* release kfp sdk 2.1.3

* update tests

* update OWNERS file
2023-09-05 18:32:31 +00:00