540294aedb
fix(sdk): fix GCPC break in KFP SDK ( #9791 )
2023-07-27 23:02:32 +00:00
ca6ad27020
chore(sdk): remove unused linter comments ( #8405 )
2022-11-01 18:31:46 +00:00
e728d0871b
chore(sdk): add pre-commit hook to remove unused imports ( #8123 )
...
* add pycln pre-commit hook to remove unused imports
* remove existing unused imports
2022-08-10 08:17:21 +00:00
13736b85b6
chore(sdk): clean up kfp code style ( #8027 )
...
* remove unused imports
* use google as isort profile
* sort imports
* format with yapf
* clean end of file new line, trailing whitespace, double quoted strings
2022-07-15 00:06:34 +00:00
5fb126a715
feat(sdk): alias v2 imports and provide DeprecationWarning ( #7398 )
...
* feat(sdk): alias v2 imports and provide DeprecationWarning
* simplify tests
* enable import of v2 namespace
* assert DeprecationWarning is raised
* update copyright
* silence pylint
* move v2 alias tests
* use assertWarnsRegex instead of assertWarns
* fix tests
* remove test for deprecation warning
* update copyright
2022-03-15 03:28:39 +00:00
8bb0d0e025
feat(sdk)!: move v2 to main namespace ( #7376 )
...
* feat(sdk)!: move v2 to main namespace
* edit samples
2022-03-04 21:56:51 +00:00
b693909a08
feat(sdk)!: move v1 to deprecated folder ( #7291 )
...
* chore(sdk): move v1 to deprecated
* fix testsg
* fix testsg
* fix setup.py
* fix test
* s'
* fix tests
* fix test
* retore v2 test changes
* fix py3.6 test
* fix py3.6 test
* fix py3.6 import fallback error
* remove deprecated
* fix samples test
* sample test
* fix samples
* add readme
* restroe test
* python require
* remove path
* fix tests
* inteegration tests
* remove tfx tests for dependency with kfp v1
* fix e2e
* fix e2e
* fix integration tests
* fix sampe
* move client down
* change to kfp
* add import alias
* fix
* runid
* fix dsl
* only use kfp for function
* revert train_until_good
* tfx test
* kfp
* try import
* onprem
2022-03-04 00:52:59 +00:00
913539728b
chore(sdk): Fix a minor bug in task final status test sample ( #7314 )
2022-02-17 00:27:34 +00:00
de2a4a45f2
chore(sdk): bump version to 2.0.0-alpha.0 ( #7313 )
...
* chore(sdk): bump version to 2.0.0-alpha.0
* relax regex to match sdk version in tests
2022-02-15 20:07:41 +00:00
ab805f886c
feat(sdk): Support getting pipeline status in exit handler. ( #7309 )
...
* Support PipelineTaskFinalStatus in exit handler.
* release note
2022-02-15 17:58:41 +00:00
1acb90fc94
fix(sdk.v2): Fix importer not using correct output artifact type ( #7235 )
...
* fix importer output type
* release note
2022-01-31 14:34:08 -08:00
83d54d3cb7
fix(sdk): Fix create_runtime_artifact not covering all types. ( #7168 )
...
* Fix create_runtime_artifact not covering all types.
* release note
2022-01-19 04:16:19 +00:00
5a29d7f95b
feat(sdk): add metadata field for importer ( #7112 )
...
* feat(sdk): add metadata
* metadata
* release notes
* address comments
2022-01-06 00:41:35 +00:00
44228c242b
test(v2): v2 sample test supports notebooks ( #7109 )
...
* test(v2): v2 sample test supports notebooks
* fix
* fix relative path problem
2021-12-24 02:37:42 +00:00
6dfaeebd92
fix(sdk.v2)!: Block task dependency referencing tasks inside a sibling condition or loop group. ( #7050 )
2021-12-13 19:08:37 +00:00
85d74337d6
fix(sdk): fix input/outputspec and positional arguments ( #6980 )
...
* fix(sdk): fix load_test
* concat placeholder
* release notes
* fix(sdk): fix resource spec test
* release notes
* test for ontology
* fix(sdk): fix input/outputspec
* fix reuse
* fix xgboost sample
* release
* fix tests
* address comments
* fix test
* restore tests
* address comments
2021-12-13 18:12:37 +00:00
1a0bef7102
fix(sdk.v2): fix boolean error for inputs ( #7041 )
2021-12-11 17:25:35 +00:00
c1b67154a2
fix(sdk.v2): fix regression on nested loops ( #6990 )
...
* fix regression on nested loops
* release note
2021-12-01 22:34:05 +00:00
485a07719f
fix(sdk): fix resource spec test ( #6979 )
...
* fix(sdk): fix load_test
* concat placeholder
* release notes
* fix(sdk): fix resource spec test
* release notes
* test for ontology
2021-11-30 19:25:31 +00:00
0842934bae
fix(sdk): fix load_test ( #6978 )
...
* fix(sdk): fix load_test
* concat placeholder
* release notes
2021-11-29 23:00:30 +00:00
59dfffebea
fix(sdk.v2): Avoid pydantic union bug ( #6957 )
...
* avoid pydantic union bug
* release note
2021-11-24 21:48:34 +00:00
cd20373310
feat(sdk): add set_env_variable for Pipeline task ( #6919 )
...
* feat(sdk): add set_env_variable
* release notes
* enable test
* fix test
* fix test
* address comments
2021-11-23 21:46:47 +00:00
4484ccd0b3
fix(sdk): change otherwise to else in yaml ( #6952 )
...
* fix(sdk): change otherwise to else in yaml
* release notes
2021-11-23 21:31:49 +00:00
228d37ec5c
chore(sdk.v2): Add test sample for runtime placeholders ( #6946 )
2021-11-23 17:23:06 +00:00
2910d10cc3
fix(sdk.v2): Fix bug on executor input with default value. ( #6937 )
...
* fix bug on executor input
* release note
* fix test on empty message
* fix part of the test
2021-11-22 21:09:06 +00:00
c4d8dfd8c6
feat(sdk): ImporterSpec v2 ( #6917 )
...
* importer
* compiler
* check diff
* fix format
* remove v1 node test
* remove importer v2 compat test
* release note
* address comments
2021-11-18 23:11:32 +00:00
9ee4534aef
fix(sdk.v2): Fix regression on optional inputs ( #6905 )
...
* Fix regression on optional inputs
* add release note
2021-11-15 18:43:43 +00:00
7921946b07
feat(sdk.v2)!: Merge v2 experimental change back to v2 namespace ( #6890 )
...
* merge v2 experimental change back to v2 namespace
* release note
* fix tests
* fix cli
* fix py3.6 test
* skip v2 compatible mode tests
* fix tests
* skip v2 engine tests
2021-11-11 00:25:42 +00:00
b7fd47f70b
chore(sdk): Remove sdk/python/kfp/v2/google directory for v2 ( #6886 )
...
* Delete sdk/python/kfp/v2/google directory
* Update RELEASE.md
* Update MANIFEST.in
* Update requirements-test.txt
* Update requirements.in
* fix setup
* remove test
2021-11-10 01:18:04 -08:00
dea0823fe8
fix(sdk): Make `Artifact` type be compatible with any sub-artifact types bidirectionally ( #6859 )
2021-11-03 16:34:21 -07:00
334307d6c3
chore(sdk): add deprecation warnings for v2 ( #6851 )
2021-11-02 16:26:06 -07:00
b2414f3eb8
fix(sdk): fix cloud scheduler's job name ( #6844 )
2021-11-02 13:46:06 -07:00
0f2cab9b39
fix(sdk.v2): Fix display name support for groups ( #6832 )
...
* fix display name for experimental tasks_group
* release note
2021-10-29 00:30:51 -07:00
f1bb852852
feat(sdk): add load_component_from_* ( #6822 )
...
* feat(sdk): add load_component_from_*
* address comments'
:
* update release notes
2021-10-28 22:30:34 -07:00
8c6843fdb8
fix(sdk.v2): Fix importer ignoring `reimport` setting, and switch to Protobuf.Value for import uri. ( #6827 )
...
* fix importer
* release note
* remove error usage in test sample
* disable importer test
2021-10-28 20:40:51 -07:00
ea2e5be81a
feat(sdk.v2): Implement experimental `@component` component ( #6825 )
...
* Implement experimental `@component` component
* release note
2021-10-28 18:29:51 -07:00
0be57c3880
feat(sdk): Use google.protobuf.Value in v2 for passing parameters. ( #6804 )
...
* Use google.protobuf.Value in v2 for passing parameters.
* retest samples.
* Fix tests.
* Update release, more cleanup.
* Use github.com/kubeflow/pipelines/api from same repo.
* Run go mod tidy
* chore: go mod tidy
* fix v2 compile error and clean up unused code
* pr comments.
* update goldens
* Fix metadata recording.
* Update kfp mlmd client.
* fix test again
* another try.
* chore: migrate v2 DAG driver input parameters to protobuf.Value + small refactorings
* fix v2 launcher + clean up
* fix a compile error
* fix a few more tests
* fix number parsing
* clean up
* disable cache_v2 test.
Co-authored-by: Yuan Gong <gongyuan94@gmail.com>
2021-10-28 14:25:51 -07:00
7768f28de3
fix(sdk.v2): Fix missing experimental modules in setup.py ( #6817 )
...
* fix missing experimental modules
* fix typo
* fix test
2021-10-27 10:58:39 -07:00
02b96f0a63
feat(sdk.v2): Implement v2 experimental compiler. ( #6803 )
...
* Implement v2 experimental compiler.
* add release note
* Support protobuf.Value in Condition operand
2021-10-26 14:12:37 -07:00
61fea6ba1f
fix(sdk.v2): Fix InputPathPlaceholder changed to InputValuePlaceholder ( #6794 )
...
* fix InputPathPlaceholder changed to InputValuePlaceholder
* release note
2021-10-24 17:10:07 -07:00
78dfac78ac
feat(sdk): Pipeline task settings ( #6746 )
...
* task
* caching options + cpu limit
* add container ops
* remove pipelineparam
* update release notes
* address comments
2021-10-19 11:39:05 -07:00
7f1e34f17d
chore(sdk.v2): Implement LoopArgument and LoopArgumentVariable v2 ( #6755 )
...
* for_loop v2
* release note
* address review comments
2021-10-19 03:09:23 -07:00
77de39de75
fix(sdk): Try alternatives for obtaining pip when `ensurepip` does not exist in container. ( #6737 )
...
* Try alternatives for obtaining pip when `ensurepip` does not
exist in container.
This PR also fixes the tests that have golden files. Developers can
update golden files by setting UPDATE_GOLDENS=True before running
tests. E.g., run the following from sdk/python directory:
UPDATE_GOLDENS=True python3 -m unittest discover --verbose --pattern '*test*.py' --top-level-directory .
* Update RELEASE.
2021-10-15 14:33:52 -07:00
fe663772c8
feat(sdk): Enable containerizing v2 Python components ( #6417 )
...
* Refactor and move all v2 related code to under the v2 namespace.
Most of the changes are around imports and restructuring of the
codebase. While it looks like a lot of code was added, most of the code
already existed and was simply moved or copied over to v2. The only
exceptions are:
- under kfp/v2/components/component_factory.py: some helper functions
were copied with simplification from _python_op.py
- we no longer strip the `_path` suffix in v2 components.
Note: there is still some duplication of code (particularly between
component_factory.py and _python_op.py), but it's ok for now since we
intend to replace some of this with v2 ComponentSpec + BaseComponent.
* Update setup.py.
* update tests.
* revert accidental change of gcpc
* Fix component entrypoint.
* Update goldens.
* fix tests.
* fix merge conflict.
* revert gcpc change.
* fix tests.
* fix tests.
* Add type aliases for moved files.
* merge and update goldens.
* Add a CLI command to help containerize and build v2 components.
* Flesh out the CLI for Docker.
* update comments.
* Support multiple component files in the same directory.
* address pr comments.
* Let user specify just the directory containing components.
The CLI will search for components in all Python files by default. It
can also search for a specific filepattern (as supported by
pathlib.Path objects).
Also add a bunch of tests.
* Clean up.
* fix merge error.
* Fix docker import error for tests.
* Update release notes.
* address PR comments.
2021-10-12 09:33:44 -07:00
d3114638f1
chore(sdk.v2): Implement PipelineTask creation ( #6713 )
...
* Implement PipelineTask __init__()
* add release note
2021-10-11 13:17:43 -07:00
073c819ebc
fix(sdk.v2): Fix the issue that using `dsl.` prefix in component I/O type annotation breaks component at runtime. ( #6714 )
...
* Import kfp and kfp.v2.dsl for component runtime
* update golden
* add release note
* update golden
2021-10-11 09:59:45 -07:00
5892bf97d0
fix(sdk): update v2 yaml format ( #6661 )
...
* fix(sdk): change names to input_name and output_name
* ordered dict and schema version numbering
* Remove schema_version, annotatations and labels
* change container/dagspec format
* change to bracket
* fix tests
* fix py3.6 forwardref issue
* fix ordered dict
* address comments
* release notes + test
* address comments
2021-10-07 16:48:11 -07:00
2b2e5fdd69
fix(sdk.v2): fix function-based components not preserving the namespace of GCPC artifact types. ( #6702 )
...
* Support GCPC types with function-based component
* add release note
2021-10-07 15:35:22 -07:00
a2cdf740c5
feat(sdk): add v2 placeholder variables ( #6693 )
...
* feat(sdk): add placeholder variables
* add release notes
* address comments
2021-10-06 14:17:22 -07:00
82ed66ece1
fix(sdk.v2): Fix executor getting None as value when float 0 is passed in. Fixes #6636 ( #6682 )
...
* fix getting None as value when float 0 is passed in
* add release note
2021-10-05 19:03:08 -07:00
Connor McCarthy
Yaqi Ji
Chen Sun
Yuan (Bob) Gong
Trs
Ajay Gopinathan