11f60d813a
feat(sdk): support `.after()` referencing task in `ParallelFor` group ( #10257 )
2023-11-29 10:22:14 -05:00
8d00d0eb9a
feat(sdk): support a Pythonic artifact authoring style ( #9932 )
...
* chore(sdk): test observability, refactorings, and cleanup
* implement unified i/o syntax
* clarify container component error message
* address review feedback
* add dill to packages_to_install
* update get_uri logic for KFP and tests
* update execution tests & cleanup
2023-10-19 18:49:58 +00:00
2d3171cbfe
feat(sdk): support collecting outputs from conditional branches using `dsl.OneOf` ( #10067 )
...
* support dsl.OneOf
* address review feedback
* address review feedback
2023-10-18 02:34:55 +00:00
fcdff294a6
fix(sdk): fix incorrect sub-DAG output type when using `dsl.Collected` ( #10069 )
2023-10-09 17:04:18 +00:00
87005cc015
chore(sdk): wrap conditional branch groups in outer pipeline ( #10034 )
2023-10-05 02:45:15 +00:00
20218a9c1d
chore(sdk): test observability, refactorings, and cleanup ( #10005 )
...
* chore(sdk): test observability, refactorings, and cleanup
* address review feedback
* rename classes
2023-09-18 22:51:27 +00:00
a13e143038
test: fix sdk execution test ( #9988 )
...
* fix sdk execution test
* fix missing key
* fix state
* fix state
* fix capitalization
2023-09-14 20:02:30 +00:00
cc2cd58918
fix(sdk): fix --no-deps flag usage ( #9982 )
...
* update --no-deps flag usage
* update runtime test code
2023-09-13 15:31:02 -07:00
c6b236d1a0
feat(sdk): support dsl.If, dsl.Elif, and dsl.Else ( #9894 )
...
* support if/elif/else
* deprecate dsl.Condition
* alter rebase
* update release notes
* address review feedback
* change BinaryOperation to ConditionOperation
2023-09-11 20:19:35 +00:00
cf0e0cf87c
feat(sdk): enable dependency-free runtime install of kfp ( #9886 )
...
* handle problematic imports
* separate runtime and compile-time symbols
* wrap kfp/__init__.py imports unavailable at runtime
* update component factory + tests
* add runtime tests
* add --no-deps flag to component factory
* update release notes
* clean up
* handle containerized python components
* update golden snapshots
* update component_factory unit tests
* respond to review feedback
* fix runtime test and compilation logic
* update tests
2023-08-24 18:31:16 +00:00
c9e54798cd
chore(sdk): undo creation of kfp-dsl namespace package ( #9874 )
...
* revert to c8204d0#9742
* modify #9791
* reapply #9800
* reapply #9827
* revert parts of #9738
* reapply parts of #9785
* remove duplicated dsl-test code
* reapply parts of #9791
* correct version
2023-08-15 16:24:52 -07:00
525ff90684
chore(sdk): extract DSL into `kfp-dsl` package ( #9738 )
...
* move dsl dir
* make kfp-dsl a package
* make kfp-dsl a package
* additional changes
* address review feedback
2023-07-25 23:34:02 +00:00
c8204d0285
chore(sdk): partition KFP SDK source code into runtime and non-runtime code ( #9710 )
2023-07-07 23:18:52 +00:00
a727783abd
chore: remove 'dummy_root' in test samples ( #9640 )
...
* remove 'dummy_root' in test samples
* more cleanup
2023-06-21 23:51:41 +00:00
78902274f1
feat(sdk): support compiling components with `dsl.PipelineTaskFinalStatus` type ( #9082 )
2023-05-12 22:14:28 +00:00
c01288d673
feat(sdk): support writing `TASK_FINAL_STATUS` parameter type ( #9080 )
...
* write new parameter type
* update golden snapshots
* add compiler tests
2023-05-12 17:13:32 +00:00
b6b8f94da2
chore: recompile/update v2 samples to avoid possible runtime breakage ( #9333 )
...
* recompile/update v2 samples
* fix test
2023-05-05 17:28:24 +00:00
946c51bafe
feat(sdk): support extracting input/output descriptions from component/pipeline docstrings ( #9156 )
...
* feat(sdk): support extracting input/output descriptions from component/pipeline docstrings
* remove numpydoc tests
2023-04-24 17:16:44 +00:00
91abbeaf2f
feat(sdk): support `display_name` and `description` in `@dsl.pipeline` decorator ( #9153 )
...
* feat(sdk): support display_name and description in @dsl.pipeline decorator
* add release note
* test additional case
2023-04-17 14:15:39 -07:00
80bd8f17b1
chore(sdk): bump kfp version in test data to 2.0.0-beta.14 ( #9145 )
2023-04-12 16:27:09 -07:00
a6ef77b784
feat(sdk): Support setting cpu/memory requests. ( #9121 )
...
* Support setting cpu/memory requests.
* address review comments
* address review comments
2023-04-10 23:35:16 -07:00
e96754cb53
feat(sdk): deprecate .add_node_selector_constraint in favor of .set_accelerator_type ( #8980 )
...
* rename method
* update docstring
2023-03-14 09:22:35 -07:00
8d3e0dd805
fix(sdk): Fix environment variable set in component yaml lost during compilation. Fixes #8884 ( #8885 )
...
* Fix env set in component yaml not captured
* release note
* clean up unnecessary code in test
2023-02-23 14:28:26 -08:00
31298c479b
fix(sdk): fix bug when passing data between tasks using f-strings ( #8879 )
2023-02-23 08:36:53 -08:00
c046879dda
support fanning-in artifacts ( #8808 )
2023-02-22 10:39:00 -08:00
1e1a02066a
fix(sdk): Add outputDefinitions of root pipeline to compiled components IR YAML file ( #8848 )
...
* base
* add to readme
* adress comments
2023-02-15 09:28:33 -08:00
5684bfccff
feat(sdk): Ignore upstream failure ( #8838 )
...
* main
* adress comments
* adress comments
* adress comments 2
* address comments 3
* address comments 4
2023-02-13 11:40:42 -08:00
e7ad073f1d
feat(sdk): deprecate .set_gpu_limit in favor of .set_accelerator_limit ( #8836 )
2023-02-09 15:46:10 -08:00
b575950a75
feat(sdk): support for fan-in from `dsl.ParallelFor` ( #8631 )
...
* implement parallelfor fan-in
* temporarily block fan-in of artifacts
* clean up
* unify and clarify error messages
* remove comment
* remove unnecessary compiler condition
* make test cases more realistic
* use better ParallelFor variable name
* add and rename test cases
* add ParallelFor fan-in test cases
* fix return collected bug
* clean up compilation tests
* update docstrings; update error message
* remove fan-in artifacts read/write tests
* implement review feedback
* change use of additional_input_name_for_pipeline_channel
* move additional_input_name_for_pipeline_channel
* provide additional pipeline topology validation and tests; refactor existing validation
* add release note for parameter fan-in
2023-02-04 20:50:51 -08:00
151e8a3c6e
feat(sdk): support optional artifact inputs ( #8623 )
...
* add compiler tests
* support optional artifact inputs
* update executor test
* update existing golden snapshot
* add new compiler test/snapshot
* add release note
* bump kfp-pipeline-spec dependency
2022-12-28 13:40:12 -05:00
5491f6f3db
fix(sdk): support writing/reading isOptional for input parameters ( #8612 )
...
* add tests
* support writing/reading isOptional for parameters
* update golden snapshots
* temporarily maintain workaround for v1 optional artifact inputs
* add release note
2022-12-21 14:46:59 -08:00
47eb1cad92
chore(sdk): update golden snapshots with pipeline interface comments ( #8575 )
...
* update path in readme
* update golden snapshots
2022-12-15 01:32:21 +00:00
b829e4a5da
feat(sdk): enable use of primitive placeholders in f-string ( #8494 )
...
* support primitive placeholders in fstring
* add read/write test component
2022-11-28 14:45:48 -08:00
9e9e1081c6
fix(sdk): fix boolean default value serialization bug ( #8444 )
...
* add test cases
* fix bug
* add v1 component yaml back compat changes
* update golden snapshot
2022-11-11 15:42:33 -08:00
24e17f7352
chore(sdk): quiet noisy SDK unit test output ( #8434 )
2022-11-09 17:47:39 +00:00
c353245baf
feat(sdk): enable dynamic importer metadata ( #7660 )
...
* support placeholder in metadata dict
* add unit tests
* add compilation test
* fix tests
* update compiler test
* use 'metadata' as input key
* update compiler test and golden snapshot
* add int input to compiler test
* change constant name
* use util function
* support f-strings in dynamic import metadata; add and update tests
* update method name after rebase
* make compile consistent across python versions
2022-09-19 21:58:31 +00:00
166d6bb917
feat(sdk): add runtime logic for custom artifact types (support for custom artifact types pt. 3) ( #8233 )
...
* add runtime artifact instance creation logic
* refactor executor
* add executor tests
* add custom artifact type import handling and tests
* fix artifact class construction
* fix custom artifact type in tests
* add typing extensions dependency for all python versions
* use mock google namespace artifact for tests
* remove print statement
* update google artifact golden snapshot
* resolve some review feedback
* remove handling for OutputPath and InputPath custom artifact types; update function names and tests
* clarify named tuple tests
* update executor tests
* add artifact return and named tuple support; refactor; clean tests
* implement review feedback; clean up artifact names
* move test method
2022-09-15 01:00:40 +00:00
5dff29a071
fix(sdk): keeping placeholder strings in command and args even if default value is set ( #8259 )
...
* return string representation of placeholder in pipeline_task
* return string representation of placeholder in pipeline_task
2022-09-13 23:44:31 +00:00
e371fa3795
feat(sdk): add compilation logic for google artifact types (support custom artifact types pt. 2) ( #8232 )
...
* remove dependency of deprecated on v2
* use new artifact identification logic
* add custom artifact type compiler test
* compile schema_version from non-kfp artifacts
* update tests
* add importer to compiler test
* update golden snapshots
* fix type checking code
* undo executor_test diff
* remove gcpc types compiler test
* add compilation test comment
* update google artifact handling
* update compiler test
* only permit system. and google. artifact types; add tests
* update compiler test
* simplify type checking logic
* use bundled artifact types in PipelineArtifactChannel
* raise error on unknown v1 input or output
* add type utils and move functions
* add type utils tests
* remove schema_version from inputspec and outputspec
* update artifact proto creation and tests
* propogate bundled artifact type string changes throughout sdk
* update golden snapshot
* remove unused v1 compat and tests
* update type checking and tests to validate schema_version using semantic versioning
* update importer sample
* update test data config
* update test_data config
* clean up sample
* clean up sample
2022-09-12 14:00:02 -06:00
16c7d6658e
test(sdk): create SDK execution test suite and restructure read/write tests ( #8245 )
...
* update client for graph components
* use yaml for config
* simplify read write tests
* add test script
* restructure yaml and read/write tests
* add execution test file
* move test_data; update paths; update readme
* update kfp endpoint
* update pipeline with env
* update logging
* update config
* update kfp endpoint and timeout
* add kfp as requirement
* mock kfp_package_path for lightweight component execution
* revert pipeline name changes
* update config with name
* update read write tests to use explicit name
* update execution tests to use explicit name
2022-09-12 18:33:06 +00:00
Connor McCarthy
Chen Sun
JOCSTAA
Scott_Xu