b59b70875e
fix(sdk): allow user to choose platform for component container build. Fixes #8883 ( #9212 )
...
Co-authored-by: Jonny Browning <jonathan.browning@datatonic.com>
2023-04-25 19:41:45 +00:00
04306dc3cb
chore(sdk): release KFP SDK 2.0.0b15 ( #9218 )
2023-04-24 18:32:43 +00:00
946c51bafe
feat(sdk): support extracting input/output descriptions from component/pipeline docstrings ( #9156 )
...
* feat(sdk): support extracting input/output descriptions from component/pipeline docstrings
* remove numpydoc tests
2023-04-24 17:16:44 +00:00
91277b1e91
fix(SDK): fix `client.run_pipeline` with `pipeline_id` and `version_id` gives error ( #9191 )
2023-04-20 21:17:40 +00:00
41c0d78f5f
fix(sdk): fix handling of PipelineTaskFinalStatus in ir_type_ attribute hack ( #9186 )
2023-04-20 13:53:31 -07:00
91abbeaf2f
feat(sdk): support `display_name` and `description` in `@dsl.pipeline` decorator ( #9153 )
...
* feat(sdk): support display_name and description in @dsl.pipeline decorator
* add release note
* test additional case
2023-04-17 14:15:39 -07:00
3164dfb01a
fix(sdk): fixes module not found error for containerized python components. Fixes #8385 ( #9157 )
...
* Add module directory to sys.path
* Add nested module imports unit test
* Add release note to release.md
2023-04-17 11:46:02 -07:00
30466586d0
chore(sdk): add kfp-kubernetes as kfp extras_require ( #9152 )
2023-04-17 10:08:22 -07:00
80bd8f17b1
chore(sdk): bump kfp version in test data to 2.0.0-beta.14 ( #9145 )
2023-04-12 16:27:09 -07:00
a107fb1bcf
chore: release SDK 2.0.0b14 ( #9144 )
...
* release SDK 2.0.0b14
* doc ref
2023-04-12 14:29:29 -07:00
952e0730a8
feat(sdk): Support submitting pipeline with platform config. ( #9140 )
...
* SDK client v2beta1 API integration.
* fix unit tests
* create alias for use of methods and fields referencing `job`.`
* fix cli
* fix pipeline list-versions
* more cli fixes
* client fix
* Support submitting pipeline with platform spec.
* unit test
* move util functions
2023-04-12 13:06:12 -07:00
07d6c1816b
feat(SDK)!: SDK client v2beta1 API integration. Fixes #8706 ( #9112 )
...
* SDK client v2beta1 API integration.
* fix unit tests
* create alias for use of methods and fields referencing `job`.`
* fix cli
* fix pipeline list-versions
* more cli fixes
* client fix
2023-04-12 09:54:45 -07:00
5e98b3b6f0
feat(sdk): Support PIPELINE_ROOT_PLACEHOLDER ( #9134 )
...
* Support PIPELINE_ROOT_PLACEHOLDER
/assign @connor-mccarthy
* Update __init__.py
2023-04-11 08:56:49 -07:00
a6ef77b784
feat(sdk): Support setting cpu/memory requests. ( #9121 )
...
* Support setting cpu/memory requests.
* address review comments
* address review comments
2023-04-10 23:35:16 -07:00
e907b6343e
set _ir_type attribute ( #9105 )
2023-04-10 12:01:36 -07:00
5828632c2f
feat(sdk): support indexing into top-level artifact metadata in Container Components ( #9131 )
2023-04-10 11:46:01 -07:00
5a769e9080
Revert "update compiler type annotations ( #9090 )" ( #9095 )
...
This reverts commit ad6f55e8f3
2023-04-03 16:01:26 -07:00
ad6f55e8f3
update compiler type annotations ( #9090 )
2023-04-03 09:20:17 -07:00
0cbcebcf88
fix(sdk): support accessing `.dsl` attribute on `kfp` module object ( #9048 )
2023-03-27 15:12:17 -07:00
3026e452ff
fix(sdk): fix loading non-canonical type strings from v1 component YAML ( #9041 )
2023-03-27 10:25:23 -07:00
3e1f3c6d7a
feat(sdk): add experiment_id parameter to create run methods ( #9004 )
...
Co-authored-by: alenawang <alena.n.wang@gmail.com>
Co-authored-by: andreafehrman <andrea.k.fehrman@vanderbilt.edu>
Co-authored-by: owmasch <owenmaschal0598@gmail.com>
Co-authored-by: ryanrusson <ryan.russon@gmail.com>
2023-03-24 09:56:48 -07:00
2bbfd5e89f
fix: support setting task dependencies via kfp.kubernetes.mount_pvc ( #8999 )
...
* enable accessing .task on pipeline channel
* set task dependencies in mount_pvc
* update tests
2023-03-16 21:10:54 -07:00
0fd1e69c74
cpu and memory resource can be optional ( #8992 )
...
Co-authored-by: Ze Mao <zemao@google.com>
2023-03-16 09:23:55 -07:00
e96754cb53
feat(sdk): deprecate .add_node_selector_constraint in favor of .set_accelerator_type ( #8980 )
...
* rename method
* update docstring
2023-03-14 09:22:35 -07:00
a38d7479d7
feat(sdk): support compiling platform specific features ( #8940 )
...
* support compiling platform-specific features
* fix map size change during iteration
2023-03-10 11:18:25 -08:00
abdc08b773
chore(sdk): pin kfp-pipeline-spec to 0.2.1 ( #8971 )
2023-03-10 10:19:03 -08:00
6777e8f445
fix(sdk): Support python 3.11 ( #8907 )
...
* new branch
* add to release.md
* edit setup.py
2023-03-03 13:36:53 -08:00
ed66ba327f
feat(sdk): using component's pip_index_urls for Dockerfile generation. Fixes #8816 ( #8871 )
...
* feat(sdk): using component's pip_index_urls for Dockerfile generation
* Update sdk/python/kfp/cli/component.py
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
* feat(sdk): PR changes: removed additional space from index url options string; duplicate removal not necessary
* feat(sdk): feature mentioned in RELEASE.md
* deduplication back - more components in a file can specify same pypi url - we need it only once for docker image
* feature note moved to current development section
---------
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
2023-03-01 09:32:56 -08:00
86554be46a
release kfp sdk 2.0.b13 ( #8904 )
2023-02-27 16:52:53 -08:00
87062dbf22
fix(sdk): fix attribute error for Containerized Python Components ( #8887 )
2023-02-27 16:42:50 -08:00
8d3e0dd805
fix(sdk): Fix environment variable set in component yaml lost during compilation. Fixes #8884 ( #8885 )
...
* Fix env set in component yaml not captured
* release note
* clean up unnecessary code in test
2023-02-23 14:28:26 -08:00
386ff88236
fix(sdk): fix unknown artifact string in dsl.OutputPath/dsl.InputPath ( #8880 )
2023-02-23 08:37:04 -08:00
31298c479b
fix(sdk): fix bug when passing data between tasks using f-strings ( #8879 )
2023-02-23 08:36:53 -08:00
c046879dda
support fanning-in artifacts ( #8808 )
2023-02-22 10:39:00 -08:00
41752608bc
support list of artifact input placeholders ( #8484 )
2023-02-16 09:26:53 -08:00
1e1a02066a
fix(sdk): Add outputDefinitions of root pipeline to compiled components IR YAML file ( #8848 )
...
* base
* add to readme
* adress comments
2023-02-15 09:28:33 -08:00
8552226c41
chore(test): revert api changes to deprecated client ( #8841 )
...
* revert api changes to deprecated client
* list experiments
* fix
* fix sample v2 test
2023-02-14 11:01:58 -08:00
5684bfccff
feat(sdk): Ignore upstream failure ( #8838 )
...
* main
* adress comments
* adress comments
* adress comments 2
* address comments 3
* address comments 4
2023-02-13 11:40:42 -08:00
e7ad073f1d
feat(sdk): deprecate .set_gpu_limit in favor of .set_accelerator_limit ( #8836 )
2023-02-09 15:46:10 -08:00
0fc174ae28
chore(sdk): release KFP SDK 2.0.0b12 ( #8823 )
2023-02-08 09:46:17 -08:00
cc5373281d
chore(sdk): Pin kfp-server-api to an exact version ( #8822 )
2023-02-07 23:04:44 -08:00
b575950a75
feat(sdk): support for fan-in from `dsl.ParallelFor` ( #8631 )
...
* implement parallelfor fan-in
* temporarily block fan-in of artifacts
* clean up
* unify and clarify error messages
* remove comment
* remove unnecessary compiler condition
* make test cases more realistic
* use better ParallelFor variable name
* add and rename test cases
* add ParallelFor fan-in test cases
* fix return collected bug
* clean up compilation tests
* update docstrings; update error message
* remove fan-in artifacts read/write tests
* implement review feedback
* change use of additional_input_name_for_pipeline_channel
* move additional_input_name_for_pipeline_channel
* provide additional pipeline topology validation and tests; refactor existing validation
* add release note for parameter fan-in
2023-02-04 20:50:51 -08:00
e50f40305d
fix(sdk): fix upload_pipeline when no pipeline name is provided ( #8695 )
2023-01-31 13:45:39 -08:00
edfb267790
bump version; update release notes; add docs ( #8745 )
2023-01-20 12:56:56 -08:00
1c7ab4c5f5
pin kfp-pipeline-spec version exactly ( #8737 )
2023-01-20 12:49:36 -08:00
78d31dc8a3
chore(sdk): remove v1 dependencies from SDK v2 ( #8668 )
...
* split out deprecated requirements
* pip install deprecated requirements for sample tests
* remove superfluous comments in requirements.in
* update release notes
* ignore deprecated directory in unit tests
* remove unused argo compiler
* ignore deprecated path
* update run sample yaml
* install deprecated requirements for e2e tests
* add todos
2023-01-19 12:53:27 -08:00
07615eb88a
chore(sdk): Improve CLI utility function ( #8686 )
2023-01-17 14:51:50 -08:00
5adc7f1b50
chore(sdk): clean up client.py ( #8656 )
2023-01-17 11:04:46 -08:00
6040334e84
feat(sdk): Accepts higher PyYAML versions ( #8665 )
...
* Accepts higher PyYAML versions
* Update the release note
2023-01-17 10:46:42 -08:00
fdf3ee7b68
chore(sdk): release KFP SDK 2.0.0b10 ( #8638 )
...
* bump version
* update release notes
* add docs version
2022-12-29 18:05:36 -05:00
84241d6460
fix(sdk): fix deserializing v1 component yaml defaults ( #8639 )
...
* fix deserializing v1 defaults
* update test cases
* update type annotation
* add unit tests
2022-12-29 17:48:58 -05:00
151e8a3c6e
feat(sdk): support optional artifact inputs ( #8623 )
...
* add compiler tests
* support optional artifact inputs
* update executor test
* update existing golden snapshot
* add new compiler test/snapshot
* add release note
* bump kfp-pipeline-spec dependency
2022-12-28 13:40:12 -05:00
5491f6f3db
fix(sdk): support writing/reading isOptional for input parameters ( #8612 )
...
* add tests
* support writing/reading isOptional for parameters
* update golden snapshots
* temporarily maintain workaround for v1 optional artifact inputs
* add release note
2022-12-21 14:46:59 -08:00
ecfdc665c4
chore(sdk): release KFP SDK 2.0.0b9 ( #8609 )
2022-12-21 00:55:12 +00:00
f64348f29e
fix(sdk): make executor output writer more permissive [sdk v2] ( #8607 )
2022-12-21 00:23:13 +00:00
2eef97cfc7
fix(sdk): add special handling for PipelineTaskFinalStatus to get_parameter_type ( #8587 )
2022-12-19 18:48:46 +00:00
08b2408d3d
feat(sdk): add support for lists of artifacts in Python components [support lists of artifacts pt. 3] ( #8465 )
...
* remove unused codepath
* rename to avoid confusing variable names
* implement support in executor
* add comment
* make executor code safer
2022-12-15 21:34:21 +00:00
47eb1cad92
chore(sdk): update golden snapshots with pipeline interface comments ( #8575 )
...
* update path in readme
* update golden snapshots
2022-12-15 01:32:21 +00:00
d4b896d3e3
fix(sdk): new fix for multiple workers writing with gcsfuse bug [KFP SDK v2] ( #8544 )
...
* fix and test
* update release notes
* extract constants
2022-12-13 18:01:29 +00:00
f4588f3112
chore(sdk): idiomatic Python pure refactorings ( #8537 )
...
* use f-strings
* inline immediately returned variables
* simplify comprehension
* use list literals
* lift return statements into control flow blocks
* other pure refactorings
* remove passthrough function
* use a ternary when it improves readability
* use __getitem__ for regex match groups; introduced in Python 3.6
* fix changed comments
* revert one change
2022-12-06 23:44:36 +00:00
49db63c916
feat(sdk): Add comments to IR YAML file ( #8467 )
...
* base
* add tests
* fix bug
* adress comments
* address comments 2
* sort comments
* sort signatures
* add indempotent test
* add indempotent test2
* support multiline docstring
* review
* docformatter presubmit exclude
* docformatter presubmit exclude
* docformatter
* docformatter
* merge 1
* update readme
* nit .items()
* remove reduntant test
2022-12-05 14:43:14 -08:00
1d6a52b9ff
fix(sdk): print max_attempts instead of sleep_duration ( #8523 )
2022-12-02 19:32:16 +00:00
00c6b01040
fix(sdk): fix bug when referencing component.pipeline_spec in a pipeline body ( #8514 )
...
* add tests
* implement fix
* add TODO
2022-12-02 09:35:16 +00:00
614c231ae6
fix(sdk): unblock valid topology. ( #8416 )
...
* unblock valid topology
* add more tests
* handle git fork
* sample_test_cases
* main
* restore to master
* resolve comments on PR
* resolve conflicts
* resolve conflict 2
* revert conflict fix
* fix changes
* address comments
* review
* docformatter presubmit
* revert docformatter
* update release.md
2022-12-02 00:15:15 +00:00
b26dd100e1
test(sdk): implement small tests for Client class ( #8517 )
...
Co-authored-by: droctothorpe <mythicalsunlight@gmail.com>
Co-authored-by: droctothorpe <mythicalsunlight@gmail.com>
2022-12-01 19:39:45 +00:00
e7c82c0593
feat(sdk): support list of artifacts annotations [list of artifacts support pt. 1] ( #8464 )
...
* update get_io_artifact_class function
* update various type annotation tests
* support list of artifacts in component interfaces
* move helper function
* clarify function name
* respond to review feedback
2022-11-30 05:24:15 +00:00
249e7af384
chore(sdk): release KFP SDK 2.0.0b8 ( #8508 )
...
* update release notes
* bump version
* add beta 8 to docs; remove beta 7
2022-11-30 05:04:15 +00:00
834bfbe90a
fix(sdk): revert premature client changes ( #8507 )
...
* revert _v1 client changes
* do not install kfp_server_api from source
2022-11-29 20:39:23 -08:00
1043a88842
chore(sdk): release KFP SDK 2.0.0b7 ( #8503 )
...
* update release notes
* bump version
* add docs version to dropdown
2022-11-28 22:58:42 +00:00
b829e4a5da
feat(sdk): enable use of primitive placeholders in f-string ( #8494 )
...
* support primitive placeholders in fstring
* add read/write test component
2022-11-28 14:45:48 -08:00
4bb57e6723
fix(sdk): fix multiple workers writing with gcsfuse bug [KFP SDK v2] ( #8455 )
...
* fix(sdk): fix multiple workers writing with gcsfuse bug [KFP SDK v1]
* add comment for context
2022-11-23 00:38:48 +00:00
e97fe8e04f
feat(sdk): add pipeline job placholders for scheduled runs ( #8447 )
...
* add pipeline job placeholders
* simplify example code in docs
* add _PLACEHOLDER suffix to variable names
2022-11-23 00:38:42 +00:00
be766dc352
docs(sdk): update pipeline task .set_gpu_limit reference docs ( #8477 )
2022-11-18 17:58:22 +00:00
6b3b4df3d2
refactor(sdk): simplify InputSpec ( #8422 )
...
* simplify usage of inputspec
* remove underscore from optional
* add error message and test
* remove optional
2022-11-14 18:39:26 +00:00
9e9e1081c6
fix(sdk): fix boolean default value serialization bug ( #8444 )
...
* add test cases
* fix bug
* add v1 component yaml back compat changes
* update golden snapshot
2022-11-11 15:42:33 -08:00
3f2e20d14f
chore(backend): regenerate v1beta1 api clients ( #8403 )
...
* add v2 experiment.proto
* regenerate v1beta1 client
* remove api version from combined swagger file
* Revert "regenerate v1beta1 client"
This reverts commit 02769d779bc24fcfbed9
2022-11-10 19:18:00 +00:00
24e17f7352
chore(sdk): quiet noisy SDK unit test output ( #8434 )
2022-11-09 17:47:39 +00:00
d33f359797
fix(sdk): fix nested placeholders and block illegal IfPresent form in Concat ( #8414 )
...
* support single element IfPresentPlaceholders that contain a primitive placeholder
* block list then and else_ inside concat
* support single element IfPresent in v1
* clean up typing
* skip loading batch_predict_job yaml
2022-11-04 18:20:16 +00:00
08a8b1df97
refactor(sdk): remove dead BaseModel code ( #8415 )
...
* remove dead basemodel code
* remove unused deserialization logic
* remove more dead code
* remove basemodel entirely
2022-11-03 21:58:15 +00:00
4c6abe5afc
refactor(sdk): simplify placeholders and logic ( #8402 )
...
* simplify placeholders
* update placeholders_test.py
* update structures to use placeholders correctly
* update use of placeholders elsewhere in codebase
* clean up
* move placeholder type checking logic to PipelineTask to prevent v1 load breaks
* clean up
* update docstrings
* make to_dict and to_string private methods
* revert unnecessary change to tests
* improve type safety of code
* prefix module-level constant with underscore
2022-11-02 00:31:45 +00:00
ca6ad27020
chore(sdk): remove unused linter comments ( #8405 )
2022-11-01 18:31:46 +00:00
5b75d75ad7
feat(sdk): add --build-image option to 'kfp components build' to allow users to skip docker build. Fixes #8382 for 2.0 ( #8387 )
...
* feat: added --build-image/--no-build-image flags to skip docker build
* fix: logic error for warnings/errors
2022-10-26 15:58:31 +00:00
8df22db757
chore(release): release sdk 2.0.0b6 ( #8373 )
...
* release sdk 2.0.0b6
* address review comments
2022-10-18 00:30:27 +00:00
ac02a0e191
fix(sdk): Throw exception for component parameter named Output ( #8367 )
...
* fix(sdk): Throw exception for component parameter named Output
* Add unit test
* lint fix
* nit fixes
* nit
* nit
* format fixes
2022-10-17 23:40:27 +00:00
30669e46dc
chore(sdk): improve KFP SDK reference documentation ( #8337 )
...
* use autodoc default options
* sort symbol order by typical usage patterns
* include summary of submodule symbols
* expose Input[] and Output[] type generics
* include tables of contents on API reference docs pages
* expand global TOC for discoverability
* fix missing reference / circular import problem
2022-10-17 19:58:53 +00:00
54922b32c6
fix(sdk): fix `kfp component build` only produces empty requirements.txt ( #8372 )
...
* fix CUJ2 always writes empty requirements.txt
* sort dependencies
2022-10-17 19:22:53 +00:00
4cc0e80183
fix(sdk): Allow using cli `kfp dsl compile` to compile components ( #8371 )
...
* Allow using cli `kfp dsl compile` to compile components
* improve error message
* address review comments
2022-10-17 16:06:52 +00:00
83e0f16af1
fix(sdk): Fix "No KFP components found in file" error. Fixes #8353 ( #8359 )
...
* Update component.py
* Update component.py
* Update component.py
2022-10-13 21:04:03 +00:00
764140153f
fix(sdk): Fix PyPi typo in cli/component docstring ( #8361 )
2022-10-13 20:28:03 +00:00
26f1fe4dfa
fix(sdk): fix NamedTuple output with Dict/List bug ( #8316 )
2022-09-28 02:25:10 +00:00
19b8709ca3
chore(release): release sdk 2.0.0b5 ( #8309 )
...
* bump reference docs
* update release notes; modify structure
* bump version
* implement review feedback
2022-09-26 21:38:35 +00:00
8ab690f3ed
feat(sdk): add workaround for google artifact (support custom artifact types pt. 4) ( #8279 )
2022-09-19 22:32:38 +00:00
0cb0d99d69
fix(sdk): fix InputPath error in executor ( #8273 )
...
* fix inputpath error
* add tests
* fix nit
2022-09-19 22:32:31 +00:00
c353245baf
feat(sdk): enable dynamic importer metadata ( #7660 )
...
* support placeholder in metadata dict
* add unit tests
* add compilation test
* fix tests
* update compiler test
* use 'metadata' as input key
* update compiler test and golden snapshot
* add int input to compiler test
* change constant name
* use util function
* support f-strings in dynamic import metadata; add and update tests
* update method name after rebase
* make compile consistent across python versions
2022-09-19 21:58:31 +00:00
6443fc17dc
chore(sdk): hide private methods in placeholders for simpler doc ( #8271 )
...
* hide private methods in ConcatPlaceholders and IfPresentPlaceholder by renaming
* rename transform and validate methods
2022-09-16 16:16:30 -06:00
65f2b644df
fix(sdk): Replace OOB authentication with loopback flow ( #8262 )
...
* Replace oob authentication with loopback flow. Fixes #7474 . Closes #7526 .
* Replace oob authentication with loopback flow. Fixes #7474 . Closes #7526 .
* Improve error messages
* Resolve comments in the PR
* Fix refresh/access token misconception
* Fix type hint annotation
2022-09-15 23:27:24 +00:00
f54f0b9341
fix v1 output deserialization ( #8272 )
2022-09-15 15:51:23 -06:00
166d6bb917
feat(sdk): add runtime logic for custom artifact types (support for custom artifact types pt. 3) ( #8233 )
...
* add runtime artifact instance creation logic
* refactor executor
* add executor tests
* add custom artifact type import handling and tests
* fix artifact class construction
* fix custom artifact type in tests
* add typing extensions dependency for all python versions
* use mock google namespace artifact for tests
* remove print statement
* update google artifact golden snapshot
* resolve some review feedback
* remove handling for OutputPath and InputPath custom artifact types; update function names and tests
* clarify named tuple tests
* update executor tests
* add artifact return and named tuple support; refactor; clean tests
* implement review feedback; clean up artifact names
* move test method
2022-09-15 01:00:40 +00:00
231c5079f0
fix(sdk): make sdk installable with poetry ( #8263 )
...
* Make kfp package installable with poetry
The poetry package manager executes `setup.py` differently than pip, which means that `__file__` is initialized to just `setup.py`, which means that `where` would be set to the empty string. The proposed change has been tested with both pip and poetry.
* apply yapf formatting
2022-09-14 21:08:08 +00:00
5dff29a071
fix(sdk): keeping placeholder strings in command and args even if default value is set ( #8259 )
...
* return string representation of placeholder in pipeline_task
* return string representation of placeholder in pipeline_task
2022-09-13 23:44:31 +00:00
e371fa3795
feat(sdk): add compilation logic for google artifact types (support custom artifact types pt. 2) ( #8232 )
...
* remove dependency of deprecated on v2
* use new artifact identification logic
* add custom artifact type compiler test
* compile schema_version from non-kfp artifacts
* update tests
* add importer to compiler test
* update golden snapshots
* fix type checking code
* undo executor_test diff
* remove gcpc types compiler test
* add compilation test comment
* update google artifact handling
* update compiler test
* only permit system. and google. artifact types; add tests
* update compiler test
* simplify type checking logic
* use bundled artifact types in PipelineArtifactChannel
* raise error on unknown v1 input or output
* add type utils and move functions
* add type utils tests
* remove schema_version from inputspec and outputspec
* update artifact proto creation and tests
* propogate bundled artifact type string changes throughout sdk
* update golden snapshot
* remove unused v1 compat and tests
* update type checking and tests to validate schema_version using semantic versioning
* update importer sample
* update test data config
* update test_data config
* clean up sample
* clean up sample
2022-09-12 14:00:02 -06:00
16c7d6658e
test(sdk): create SDK execution test suite and restructure read/write tests ( #8245 )
...
* update client for graph components
* use yaml for config
* simplify read write tests
* add test script
* restructure yaml and read/write tests
* add execution test file
* move test_data; update paths; update readme
* update kfp endpoint
* update pipeline with env
* update logging
* update config
* update kfp endpoint and timeout
* add kfp as requirement
* mock kfp_package_path for lightweight component execution
* revert pipeline name changes
* update config with name
* update read write tests to use explicit name
* update execution tests to use explicit name
2022-09-12 18:33:06 +00:00
92a1e1fe26
feature(sdk): print links outside of ipython. Fixes #8141 ( #8142 )
2022-09-09 00:00:27 +00:00
e14a784327
feat(sdk): migrate v1 auth to v2 ( #7789 )
...
* initial auth migration (@ji-yaqi)
* clean up files
* update imports
* add type annotations
* update docstrings
* add tests
2022-09-06 17:54:29 -06:00
eb453d10a9
fix(sdk): fix dsl control flow docstrings ( #8183 )
...
* fix dsl control flow docstrings
* update docstring
2022-09-01 16:33:07 -06:00
2a566181bf
chore(sdk): bump kfp version to 2.0.0-beta.4 ( #8229 )
2022-09-01 06:41:55 +00:00
25cd45d772
chore(sdk): add script to check for unused imports ( #8221 )
...
* add dev requirement
* add unused import check to isort script
2022-08-31 22:53:51 -07:00
d88358201d
feat(sdk): Support using pipeline in exit handlers ( #8220 )
...
* Support using pipeline in exit handlers
* release note
* remove dead code
* fix component spec merging flakiness
* address comments in PR#8209
* address review comments
* fix bad merge in release note
2022-09-01 01:45:54 +00:00
c34e5e4edd
chore(sdk): Update upper bound for kubernetes dependency ( #8173 )
...
* Update upper bound for kubernetes dependency
* Add in release note for change
2022-08-31 22:47:54 +00:00
889637ade1
fix(sdk): fix constructor signature on dsl.Condition ( #8182 )
2022-08-31 21:35:54 +00:00
410e99c604
feat(sdk): Support loading pipeline from yaml ( #8209 )
...
* support loading pipeline from yaml
* release note
* cleanup
* maintain read/write consistency with component/pipeline compilation.
2022-08-31 12:05:55 +00:00
f5277d3078
Fix the example pipeline in sdk/python/README.md ( #8206 )
2022-08-30 14:55:45 -06:00
48574dc648
feat(sdk): Support pipeline outputs ( #8204 )
...
* Support pipeline outputs
* release note
2022-08-29 20:26:53 +00:00
72c1d10a33
feat(sdk): rename kfp artifact attributes (support custom artifact types pt. 1) ( #8191 )
2022-08-27 13:20:50 +00:00
5ce4954d6b
feat(sdk)!: Implement Graph Component ( #8179 )
...
* Implement Graph Component
* isort imports
* release note
* address review comments
* address review comments
2022-08-26 09:28:49 +00:00
121ddcbc04
fix(sdk): fix v2 sample tests for cuj3 ( #8168 )
...
* fix v2 sample tests for cuj3
* separate sh -c
* fix two step pipeline
* fix piping error
* fix nit
* change path to uri to correct cat command
* correct bash script
* remove display name
* change directory reference in commands
* remove old files
2022-08-25 15:30:22 -07:00
459e5358e6
chore(sdk): Bump KFP Python SDK to support Python v3.10 ( #8186 )
...
* Update upper bound for kubernetes dependency
* Bump KFP Python SDK to support Py v3.10
* Reset requirements.* changes to isolate out Python v3.10 changes
* Add in release note
2022-08-24 21:34:08 +00:00
5ccf53a251
fix(sdk): fix output type of importer ( #8172 )
...
* change output type of importer to artifact_class.TYPE_NAME
* use the same logic as component_factory._annotation_to_type_struct for determining type_name
* format using yapf
* fix type_name
* format using yapf
* add yaml file
* changes with regards to PR comments
* format pipeline_with_importer_and_gcpc_types.py
* remove unused imports
* change name of test component
* add pipeline_with_importer_and_gcpc_types to config
* update yaml file
2022-08-24 20:15:07 +00:00
88a1b314c9
feat(sdk): add support for metadata placeholders ( #8151 )
2022-08-17 23:10:06 +00:00
51bea09833
feat(sdk): Support Parallelism in ParallelFor in KFP SDK V2 ( #8146 )
...
* feat(sdk): add support for ParallelFor parallelism setting
* add parallelism value check
* Add unit tests
* Adding a compiler test
* used None as default parallelism input signature, other minor fixes on format and tests
* fix import statements
* add release.md message
* update ParallelFor docstring
* fixed docstring comments
* removed 'optional' in docstrings
2022-08-17 21:06:01 +00:00
41b69fd90d
chore(sdk): bump kfp version to 2.0.0-beta.3 ( #8149 )
...
* update release notes
* bump version
2022-08-17 14:21:52 +00:00
c7a90a3fa8
feat(sdk): expose IfPresentPlaceholder and ConcatPlaceholder to kfp.dsl ( #8145 )
2022-08-15 23:22:06 +00:00
35fccb1fae
feat(sdk): expose new ContainerSpec API ( #8144 )
...
* rename ContainerSpec to ContainerSpecImplementation for internal usage
* fix nit
2022-08-15 20:33:34 +00:00
6137a236a5
chore(sdk): bump kfp sdk version to 2.0.0b2 ( #8135 )
2022-08-11 00:08:03 +00:00
d3c0537c8b
chore(sdk): Add test for if and concat placeholder and validate artifact channel usage ( #8120 )
...
* test if and concat placeholder and validate artifact channel usage
* fix nit in read write test config
* resolve comments
* resolve comments and move test cases
* resolve comments and add placeholder validation
* remove unused imports
* update release file
2022-08-10 23:35:03 +00:00
bdff332ac6
feat(sdk): support more than one exit handler per pipeline ( #8088 )
...
* add compiler test pipeline with multiple exit handlers
* remove blocker of multiple exit handlers
* move exit handler builder logic to pipeline_spec_builder
* build all exit handlers per pipeline
* add compiler test with IR inspection
* prevent usage of cross-pipeline after
* test cross-pipeline after is prevented
* update existing task dependency logic and tests
* add v2 sample test
* remove cross-pipeline .after
* prevent cross-dag data dependency for dsl features
* add compiler test pipeline with nested exit handlers
* add support for nested exit handlers
* clean up pipeline with nested exit handlers
* remove sample with multiple exit handlers
* remove compiler test with nested exit handlers
* add compilation guard against nested exit handlers in subdag
* update release notes
2022-08-10 16:56:21 +00:00
e728d0871b
chore(sdk): add pre-commit hook to remove unused imports ( #8123 )
...
* add pycln pre-commit hook to remove unused imports
* remove existing unused imports
2022-08-10 08:17:21 +00:00
e47a784035
docs(sdk): clean up kfp v2 cli reference docs ( #8115 )
...
* use expanded "Command Line Interface" for title
* remove alpha feature stage in help text
* hide deprecated argument
* use compile as command name
2022-08-09 18:40:39 +00:00
04c827fc3a
feat(sdk): supporting bring your own container for arbitrary input and outputs ( #8066 )
...
* support container_component decorator for function with no inputs
* resolve review comments
* add sample tests for milestone 1
* modify compiler test data
* resolve reviews
* resolve reviews
* WIP
* implementation of function of no inputs
* fixed sample test
* re-fix sample test
* fix rebase merge conflict
* resolve formatting
* resolve isort error for test data
* resolve comments
* fix nit
* resolve nit
* add implementation for placeholders i/o, sample and compiler tests
* resolve comments and merge logic for constructing container component
* resolve comments
* resolve comments
* fix assertion messages
* add error handling for accessing artifact by itself
* add test for raising error for accessing artifact by itself
2022-08-08 21:00:53 +00:00
45f3bacc0d
fix(sdk): fix compiling components with artifact inputs ( #8106 )
2022-08-05 17:36:15 +00:00
257f8dbe0a
chore(sdk): change "PipelienSpec" to "PipelineSpec" ( #8102 )
2022-08-04 16:55:06 +00:00
f221d855b1
chore(sdk): remove the unreachable validation code ( #8082 )
2022-08-01 21:03:47 +00:00
1dd06cd8e8
feat(sdk): Allow artifact inputs in pipeline definition. ( #8044 )
...
* Allow artifact inputs in pipeline definition.
* release note
* isort fix
* remove unused imports
* Delete flycheck_compiler.py
* Delete flycheck_pipeline_spec_builder.py
* Delete flycheck_structures.py
* address cr comment
2022-07-28 00:17:53 +00:00
c6a5e387dd
docs(sdk): refresh SDK public API docstrings ( #8057 )
...
* add module-level docstrings
* update compiler docstrings
* update registry module docstrings
* add BaseComponent and children to public api, but discourage use
* update artifact docstrings and type annotations
* update dsl docstrings
* update client docstrings
* clean up kfp.__init__
* add dsl placeholder docstrings
* many more docstring updates
* document type aliases in dsl module
2022-07-21 19:51:18 +00:00
d8b9439ef9
add kfp v2 api reference docs ( #8042 )
...
* use __all__ in top-level modules to record public api
* add index and source files
* add kubeflow assets to _static/
* add and pin requirements
* use block quote instead of header for readme notice
* update conf.py
* delete old files
2022-07-19 15:53:54 -06:00
e6c9411ddf
fix(sdk): fixes kfp.components module type annotations, docstrings, and string/bytes processing ( #8040 )
...
* clean up components types, docstrings, and processing
* update tests
* use real request to get component yaml
* move docstring below copyright
2022-07-19 21:02:49 +00:00
13736b85b6
chore(sdk): clean up kfp code style ( #8027 )
...
* remove unused imports
* use google as isort profile
* sort imports
* format with yapf
* clean end of file new line, trailing whitespace, double quoted strings
2022-07-15 00:06:34 +00:00
be5ff7215f
chore(sdk): update contributing guidelines/tests ( #7802 )
...
* add presubmit checks, contributing guidelines, and pre-commit file
* delete old unused formatting script
* modify scripts
2022-07-12 20:35:29 -06:00
b95c5aba85
feat(sdk/client): implements overriding caching options at submission ( #7912 )
...
* implement override caching options for client
* use absolute path in test for overriding caching
* format code and add unit tests from pipeline
* delete typo
* add two-step pipeline test and review changes
* fix formatting
* fix typo on deprecated code
* update release file
* resolve comments on docstring and testing
* update release file
2022-07-12 16:49:30 +00:00
850a750496
feat(sdk): add retry policy support to kfp v2 ( #7867 )
...
* depend on kfp-pipeline-spec>=0.1.16
* implement RetryPolicy structure
* implement RetryPolicy structure tests
* implement .set_retry method on PipelineTask
* implement retry policy compilation logic
* implement compiler tests for retry
* implement pipeline with retry compilation test
* add release note
* clean up
2022-07-11 16:44:52 +00:00
d3592b9bbd
chore(sdk): bump kfp version to 2.0.0b1 ( #7993 )
...
* bump version
* update release notes
2022-07-07 11:30:01 -06:00
7253203239
add registry json to package data ( #7987 )
2022-07-07 10:48:40 -06:00
aab7fda89b
feat(sdk): Add default registry context ( #7948 )
...
* Add default registry context
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>
* Address review comments
* Reinstate `_is_ar_host()` check
* Remove `view=FULL` flag for default list and get
version URLs
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>
* Change outdated test name
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>
2022-06-30 13:51:05 -06:00
42600a796d
chore(sdk): bump kfp version to 2.0.0b0 ( #7952 )
2022-06-27 18:11:59 -06:00
f877a1ffc8
fix(sdk): fix ifpresent and concat placeholder compile ( #7930 )
...
* fix ifpresent and concat placeholder expansion
* update golden snapshots
* uncomment v1 ifpresent component test case
2022-06-27 16:59:34 -06:00
ad458d535c
fix(sdk): fix placeholder read/write logic ( #7928 )
...
* fix ExecutorInputPlaceholder() usage
* fix OutputPath and OutputParameter placeholder task creation logic
* fix v1 component yaml deserialization logic
* fix typo in test
* ignore description when comparing ComponentSpec
* uncomment fixed test cases
2022-06-27 15:06:38 -06:00
ecb55bde74
feat(sdk): throw informative exception on uncalled @pipeline decorator ( #7913 )
...
* add informative error message for uncalled @pipeline
* add test for exception
2022-06-27 12:29:48 -06:00
a020c9b01c
fix(sdk): fix extract docstring in load component ( #7921 )
...
* fix extract docstring in load component
* implement review feedback
2022-06-24 12:12:35 -06:00
281151c051
chore(sdk): improve IR read/write testing ( #7920 )
...
* add more load component test cases
* add read write equality tests
* add copyright
* remove duplicative structures tests
* add if __name__ == '__main__' block
2022-06-24 09:45:15 -06:00
58f7ab8b49
chore(sdk): Registry Client - default to read full spec of versions ( #7936 )
...
* default to read full spec of versions
* update tests
* fix formatting
2022-06-24 09:43:51 -06:00
Jonny Browning
Connor McCarthy
Chen Sun
Mithil Poojary
IronPan
Alex
Ze Mao
JOCSTAA
b4sus
Tara Tufano
Owen
Alex
Mai Nakagawa
Sune Debel
andreafehrman
Lingqing Gan
Jonny Browning (Datatonic)
Scott_Xu
gkcalat
Alex Latchford
Yusuke Nishioka
chongyouquan
Bo
Deepyaman Datta
Christian Kadner