ef94ccd734
fix(sdk): Resolves issue when using ParallelFor with param and depending tasks ( #11903 )
...
Signed-off-by: Mai Nakagawa <nakagawa.mai@gmail.com>
2025-05-09 15:49:41 +00:00
f9d487cb60
fix(sdk): Add SDK support for setting resource limits on older KFP versions ( #11839 )
...
For context, the commit 70aaf8a9a46ebf4aae037c931ae201
2025-04-21 13:46:27 +00:00
7c931ae201
feat(sdk/backend): Add support for placeholders in resource limits ( #11501 )
...
The API introduced new fields prefixed with Resource (e.g.
ResourceCpuLimit) to replace the old fields without the prefix. The
Driver hadn't been updated to honor those fields but the SDK started
using them which led to unexpected behavior.
The Driver now honors both fields but prioritizes the new fields. The
SDK now only sets the new fields.
The outcome is that resource limits/requests can now use input
parameters.
Note that pipeline_spec_builder.py was doing some validation on the
limits/requests being set, but that's already handled in the user facing
method (e.g. set_cpu_limit).
Resolves:
https://github.com/kubeflow/pipelines/issues/11500
Signed-off-by: mprahl <mprahl@users.noreply.github.com>
2025-01-09 15:35:24 +00:00
6ebf4aae03
fix(sdk): Backport fixes in kubeflow/pipelines#11075 ( #11392 )
...
Introduced back the functions to convert k8s size values to float, but
moved to kfp.dsl.utils
Signed-off-by: Ricardo M. Oliveira <rmartine@redhat.com>
2024-12-03 22:32:59 +00:00
b57f9e8588
feat(components): Support dynamic machine type paramters in CustomTrainingJobOp ( #10883 )
...
* feat(components): Support dynamic machine type paramters in CustomTrainingJobOp
Signed-off-by: KevinGrantLee <kglee@google.com>
* fix formatting
Signed-off-by: KevinGrantLee <kglee@google.com>
* Remove unused imports and try to fix type annotation error.
Signed-off-by: KevinGrantLee <kglee@google.com>
* fix formatting
Signed-off-by: KevinGrantLee <kglee@google.com>
* Fix string formatting
Signed-off-by: KevinGrantLee <kglee@google.com>
* Fix string formatting
Signed-off-by: KevinGrantLee <kglee@google.com>
* Add new test pipeline
Signed-off-by: KevinGrantLee <kglee@google.com>
* update release.md
Signed-off-by: KevinGrantLee <kglee@google.com>
* Rename recursive_replace
Signed-off-by: KevinGrantLee <kglee@google.com>
* Refactor code and add condition test pipeline
Signed-off-by: KevinGrantLee <kglee@google.com>
* fix formatting
Signed-off-by: KevinGrantLee <kglee@google.com>
* minor clean up
Signed-off-by: KevinGrantLee <kglee@google.com>
---------
Signed-off-by: KevinGrantLee <kglee@google.com>
2024-06-13 07:09:08 +00:00
21c5ffebb0
fix(sdk): fix bug where `dsl.OneOf` with multiple consumers cannot be compiled ( #10452 )
2024-02-08 23:51:37 +00:00
b528568718
feat(sdk): Support dsl.ParallelFor over list of Artifacts ( #10441 )
...
* samples message
* update pr number
* Split LoopArgument into LoopParameterArgument and LoopArtifactArgument
* formatting
* address some comments
* resolve release notes conflict
* flatten loops in pipeline_spec_builder
* update artifact type checking logic
* simplify artifact checking logic
* re-add issubtype_of_artifact
* move name_is_loop_argument to for_loop_test.py
* simplify LoopArtifactArgument is_artifact_list=False logic
* update typeerror
* typo
* typo
* small fix
* formatting
* formatting
* remove issubtype_of_artifact()
* small changes
* assert LoopArtifactArgument channel.is_artifact_list is True
* remove whitespace
* remove newline
* Update single artifact check and error message
* formatting
* add unit test for is_artifact_list==False
* formatting
* update valueerror test.
* typo
* regex formatting
* formatting
2024-02-06 23:03:40 +00:00
11f60d813a
feat(sdk): support `.after()` referencing task in `ParallelFor` group ( #10257 )
2023-11-29 10:22:14 -05:00
2d3171cbfe
feat(sdk): support collecting outputs from conditional branches using `dsl.OneOf` ( #10067 )
...
* support dsl.OneOf
* address review feedback
* address review feedback
2023-10-18 02:34:55 +00:00
c6b236d1a0
feat(sdk): support dsl.If, dsl.Elif, and dsl.Else ( #9894 )
...
* support if/elif/else
* deprecate dsl.Condition
* alter rebase
* update release notes
* address review feedback
* change BinaryOperation to ConditionOperation
2023-09-11 20:19:35 +00:00
c8204d0285
chore(sdk): partition KFP SDK source code into runtime and non-runtime code ( #9710 )
2023-07-07 23:18:52 +00:00
48ecc7d879
fix(sdk): fix bug when iterating over output from upstream task in nested parallelfor loops ( #9580 )
...
* refactor for clarity
* fix iter over upstream output in nested parallelfor
* update release notes
* support empty `List` generic annotation
* fix typos
* add comment about refactoring opportunity
* fix annotation
* update comment again
2023-06-09 08:58:28 -07:00
c046879dda
support fanning-in artifacts ( #8808 )
2023-02-22 10:39:00 -08:00
b575950a75
feat(sdk): support for fan-in from `dsl.ParallelFor` ( #8631 )
...
* implement parallelfor fan-in
* temporarily block fan-in of artifacts
* clean up
* unify and clarify error messages
* remove comment
* remove unnecessary compiler condition
* make test cases more realistic
* use better ParallelFor variable name
* add and rename test cases
* add ParallelFor fan-in test cases
* fix return collected bug
* clean up compilation tests
* update docstrings; update error message
* remove fan-in artifacts read/write tests
* implement review feedback
* change use of additional_input_name_for_pipeline_channel
* move additional_input_name_for_pipeline_channel
* provide additional pipeline topology validation and tests; refactor existing validation
* add release note for parameter fan-in
2023-02-04 20:50:51 -08:00
614c231ae6
fix(sdk): unblock valid topology. ( #8416 )
...
* unblock valid topology
* add more tests
* handle git fork
* sample_test_cases
* main
* restore to master
* resolve comments on PR
* resolve conflicts
* resolve conflict 2
* revert conflict fix
* fix changes
* address comments
* review
* docformatter presubmit
* revert docformatter
* update release.md
2022-12-02 00:15:15 +00:00
5ce4954d6b
feat(sdk)!: Implement Graph Component ( #8179 )
...
* Implement Graph Component
* isort imports
* release note
* address review comments
* address review comments
2022-08-26 09:28:49 +00:00
Mai Nakagawa
Matt Prahl
Ricardo Martinelli de Oliveira
KevinGrantLee
Connor McCarthy
JOCSTAA
Chen Sun