11f60d813a
feat(sdk): support `.after()` referencing task in `ParallelFor` group ( #10257 )
2023-11-29 10:22:14 -05:00
7ab05d8a84
chore(sdk): Remove the ` (<type>)`from docstring args. ( #10159 )
...
* chore(sdk): Remove the ` (<type>)`from docstring args.
Remote the ` (<type>)` from the docstring args, which are redundant since there are type annotations.
* Trim the unnecessary leading space.
2023-10-26 00:29:09 +00:00
c0adea9946
chore(sdk): add test for key error bug resolved in #10067 ( #10128 )
2023-10-19 23:49:03 +00:00
8d00d0eb9a
feat(sdk): support a Pythonic artifact authoring style ( #9932 )
...
* chore(sdk): test observability, refactorings, and cleanup
* implement unified i/o syntax
* clarify container component error message
* address review feedback
* add dill to packages_to_install
* update get_uri logic for KFP and tests
* update execution tests & cleanup
2023-10-19 18:49:58 +00:00
73d51c8a23
fix(sdk): fix bug when `dsl.importer` argument is provided by loop variable ( #10116 )
2023-10-18 20:37:56 +00:00
2d3171cbfe
feat(sdk): support collecting outputs from conditional branches using `dsl.OneOf` ( #10067 )
...
* support dsl.OneOf
* address review feedback
* address review feedback
2023-10-18 02:34:55 +00:00
55f174e5bb
chore(sdk): clean up compiler_test.py ( #10070 )
2023-10-09 20:50:18 +00:00
fcdff294a6
fix(sdk): fix incorrect sub-DAG output type when using `dsl.Collected` ( #10069 )
2023-10-09 17:04:18 +00:00
87005cc015
chore(sdk): wrap conditional branch groups in outer pipeline ( #10034 )
2023-10-05 02:45:15 +00:00
e99f2704fc
feat(sdk): support `PipelineTaskFinalStatus` in tasks that use `.ignore_upstream_failure()` ( #10010 )
...
* support taskfinalstatus in tasks that ignore upstream failure
* address review feedback
2023-09-21 01:46:37 +00:00
20218a9c1d
chore(sdk): test observability, refactorings, and cleanup ( #10005 )
...
* chore(sdk): test observability, refactorings, and cleanup
* address review feedback
* rename classes
2023-09-18 22:51:27 +00:00
c6b236d1a0
feat(sdk): support dsl.If, dsl.Elif, and dsl.Else ( #9894 )
...
* support if/elif/else
* deprecate dsl.Condition
* alter rebase
* update release notes
* address review feedback
* change BinaryOperation to ConditionOperation
2023-09-11 20:19:35 +00:00
c9e54798cd
chore(sdk): undo creation of kfp-dsl namespace package ( #9874 )
...
* revert to c8204d0#9742
* modify #9791
* reapply #9800
* reapply #9827
* revert parts of #9738
* reapply parts of #9785
* remove duplicated dsl-test code
* reapply parts of #9791
* correct version
2023-08-15 16:24:52 -07:00
309c5b9f0a
chore(sdk): release KFP SDK 2.1.0 ( #9782 )
2023-07-26 17:09:02 +00:00
525ff90684
chore(sdk): extract DSL into `kfp-dsl` package ( #9738 )
...
* move dsl dir
* make kfp-dsl a package
* make kfp-dsl a package
* additional changes
* address review feedback
2023-07-25 23:34:02 +00:00
1045c95517
chore(sdk): resolve mypy finding ( #9742 )
2023-07-17 19:54:37 +00:00
c8204d0285
chore(sdk): partition KFP SDK source code into runtime and non-runtime code ( #9710 )
2023-07-07 23:18:52 +00:00
a727783abd
chore: remove 'dummy_root' in test samples ( #9640 )
...
* remove 'dummy_root' in test samples
* more cleanup
2023-06-21 23:51:41 +00:00
48ecc7d879
fix(sdk): fix bug when iterating over output from upstream task in nested parallelfor loops ( #9580 )
...
* refactor for clarity
* fix iter over upstream output in nested parallelfor
* update release notes
* support empty `List` generic annotation
* fix typos
* add comment about refactoring opportunity
* fix annotation
* update comment again
2023-06-09 08:58:28 -07:00
78902274f1
feat(sdk): support compiling components with `dsl.PipelineTaskFinalStatus` type ( #9082 )
2023-05-12 22:14:28 +00:00
96947e6fb9
fix(sdk): fix compilation of boolean constant passed to component ( #9390 )
2023-05-12 17:13:39 +00:00
c01288d673
feat(sdk): support writing `TASK_FINAL_STATUS` parameter type ( #9080 )
...
* write new parameter type
* update golden snapshots
* add compiler tests
2023-05-12 17:13:32 +00:00
946c51bafe
feat(sdk): support extracting input/output descriptions from component/pipeline docstrings ( #9156 )
...
* feat(sdk): support extracting input/output descriptions from component/pipeline docstrings
* remove numpydoc tests
2023-04-24 17:16:44 +00:00
91abbeaf2f
feat(sdk): support `display_name` and `description` in `@dsl.pipeline` decorator ( #9153 )
...
* feat(sdk): support display_name and description in @dsl.pipeline decorator
* add release note
* test additional case
2023-04-17 14:15:39 -07:00
a6ef77b784
feat(sdk): Support setting cpu/memory requests. ( #9121 )
...
* Support setting cpu/memory requests.
* address review comments
* address review comments
2023-04-10 23:35:16 -07:00
5a769e9080
Revert "update compiler type annotations ( #9090 )" ( #9095 )
...
This reverts commit ad6f55e8f3
2023-04-03 16:01:26 -07:00
ad6f55e8f3
update compiler type annotations ( #9090 )
2023-04-03 09:20:17 -07:00
0cbcebcf88
fix(sdk): support accessing `.dsl` attribute on `kfp` module object ( #9048 )
2023-03-27 15:12:17 -07:00
0fd1e69c74
cpu and memory resource can be optional ( #8992 )
...
Co-authored-by: Ze Mao <zemao@google.com>
2023-03-16 09:23:55 -07:00
a38d7479d7
feat(sdk): support compiling platform specific features ( #8940 )
...
* support compiling platform-specific features
* fix map size change during iteration
2023-03-10 11:18:25 -08:00
31298c479b
fix(sdk): fix bug when passing data between tasks using f-strings ( #8879 )
2023-02-23 08:36:53 -08:00
c046879dda
support fanning-in artifacts ( #8808 )
2023-02-22 10:39:00 -08:00
41752608bc
support list of artifact input placeholders ( #8484 )
2023-02-16 09:26:53 -08:00
1e1a02066a
fix(sdk): Add outputDefinitions of root pipeline to compiled components IR YAML file ( #8848 )
...
* base
* add to readme
* adress comments
2023-02-15 09:28:33 -08:00
5684bfccff
feat(sdk): Ignore upstream failure ( #8838 )
...
* main
* adress comments
* adress comments
* adress comments 2
* address comments 3
* address comments 4
2023-02-13 11:40:42 -08:00
b575950a75
feat(sdk): support for fan-in from `dsl.ParallelFor` ( #8631 )
...
* implement parallelfor fan-in
* temporarily block fan-in of artifacts
* clean up
* unify and clarify error messages
* remove comment
* remove unnecessary compiler condition
* make test cases more realistic
* use better ParallelFor variable name
* add and rename test cases
* add ParallelFor fan-in test cases
* fix return collected bug
* clean up compilation tests
* update docstrings; update error message
* remove fan-in artifacts read/write tests
* implement review feedback
* change use of additional_input_name_for_pipeline_channel
* move additional_input_name_for_pipeline_channel
* provide additional pipeline topology validation and tests; refactor existing validation
* add release note for parameter fan-in
2023-02-04 20:50:51 -08:00
84241d6460
fix(sdk): fix deserializing v1 component yaml defaults ( #8639 )
...
* fix deserializing v1 defaults
* update test cases
* update type annotation
* add unit tests
2022-12-29 17:48:58 -05:00
151e8a3c6e
feat(sdk): support optional artifact inputs ( #8623 )
...
* add compiler tests
* support optional artifact inputs
* update executor test
* update existing golden snapshot
* add new compiler test/snapshot
* add release note
* bump kfp-pipeline-spec dependency
2022-12-28 13:40:12 -05:00
5491f6f3db
fix(sdk): support writing/reading isOptional for input parameters ( #8612 )
...
* add tests
* support writing/reading isOptional for parameters
* update golden snapshots
* temporarily maintain workaround for v1 optional artifact inputs
* add release note
2022-12-21 14:46:59 -08:00
f4588f3112
chore(sdk): idiomatic Python pure refactorings ( #8537 )
...
* use f-strings
* inline immediately returned variables
* simplify comprehension
* use list literals
* lift return statements into control flow blocks
* other pure refactorings
* remove passthrough function
* use a ternary when it improves readability
* use __getitem__ for regex match groups; introduced in Python 3.6
* fix changed comments
* revert one change
2022-12-06 23:44:36 +00:00
49db63c916
feat(sdk): Add comments to IR YAML file ( #8467 )
...
* base
* add tests
* fix bug
* adress comments
* address comments 2
* sort comments
* sort signatures
* add indempotent test
* add indempotent test2
* support multiline docstring
* review
* docformatter presubmit exclude
* docformatter presubmit exclude
* docformatter
* docformatter
* merge 1
* update readme
* nit .items()
* remove reduntant test
2022-12-05 14:43:14 -08:00
614c231ae6
fix(sdk): unblock valid topology. ( #8416 )
...
* unblock valid topology
* add more tests
* handle git fork
* sample_test_cases
* main
* restore to master
* resolve comments on PR
* resolve conflicts
* resolve conflict 2
* revert conflict fix
* fix changes
* address comments
* review
* docformatter presubmit
* revert docformatter
* update release.md
2022-12-02 00:15:15 +00:00
9e9e1081c6
fix(sdk): fix boolean default value serialization bug ( #8444 )
...
* add test cases
* fix bug
* add v1 component yaml back compat changes
* update golden snapshot
2022-11-11 15:42:33 -08:00
24e17f7352
chore(sdk): quiet noisy SDK unit test output ( #8434 )
2022-11-09 17:47:39 +00:00
4c6abe5afc
refactor(sdk): simplify placeholders and logic ( #8402 )
...
* simplify placeholders
* update placeholders_test.py
* update structures to use placeholders correctly
* update use of placeholders elsewhere in codebase
* clean up
* move placeholder type checking logic to PipelineTask to prevent v1 load breaks
* clean up
* update docstrings
* make to_dict and to_string private methods
* revert unnecessary change to tests
* improve type safety of code
* prefix module-level constant with underscore
2022-11-02 00:31:45 +00:00
6443fc17dc
chore(sdk): hide private methods in placeholders for simpler doc ( #8271 )
...
* hide private methods in ConcatPlaceholders and IfPresentPlaceholder by renaming
* rename transform and validate methods
2022-09-16 16:16:30 -06:00
e371fa3795
feat(sdk): add compilation logic for google artifact types (support custom artifact types pt. 2) ( #8232 )
...
* remove dependency of deprecated on v2
* use new artifact identification logic
* add custom artifact type compiler test
* compile schema_version from non-kfp artifacts
* update tests
* add importer to compiler test
* update golden snapshots
* fix type checking code
* undo executor_test diff
* remove gcpc types compiler test
* add compilation test comment
* update google artifact handling
* update compiler test
* only permit system. and google. artifact types; add tests
* update compiler test
* simplify type checking logic
* use bundled artifact types in PipelineArtifactChannel
* raise error on unknown v1 input or output
* add type utils and move functions
* add type utils tests
* remove schema_version from inputspec and outputspec
* update artifact proto creation and tests
* propogate bundled artifact type string changes throughout sdk
* update golden snapshot
* remove unused v1 compat and tests
* update type checking and tests to validate schema_version using semantic versioning
* update importer sample
* update test data config
* update test_data config
* clean up sample
* clean up sample
2022-09-12 14:00:02 -06:00
16c7d6658e
test(sdk): create SDK execution test suite and restructure read/write tests ( #8245 )
...
* update client for graph components
* use yaml for config
* simplify read write tests
* add test script
* restructure yaml and read/write tests
* add execution test file
* move test_data; update paths; update readme
* update kfp endpoint
* update pipeline with env
* update logging
* update config
* update kfp endpoint and timeout
* add kfp as requirement
* mock kfp_package_path for lightweight component execution
* revert pipeline name changes
* update config with name
* update read write tests to use explicit name
* update execution tests to use explicit name
2022-09-12 18:33:06 +00:00
d88358201d
feat(sdk): Support using pipeline in exit handlers ( #8220 )
...
* Support using pipeline in exit handlers
* release note
* remove dead code
* fix component spec merging flakiness
* address comments in PR#8209
* address review comments
* fix bad merge in release note
2022-09-01 01:45:54 +00:00
410e99c604
feat(sdk): Support loading pipeline from yaml ( #8209 )
...
* support loading pipeline from yaml
* release note
* cleanup
* maintain read/write consistency with component/pipeline compilation.
2022-08-31 12:05:55 +00:00
Connor McCarthy
rickyxie0929
Alex
Chen Sun
Ze Mao
JOCSTAA
Scott_Xu