Commit Graph

32 Commits

Author SHA1 Message Date
Chen Sun 6dfaeebd92
fix(sdk.v2)!: Block task dependency referencing tasks inside a sibling condition or loop group. (#7050) 2021-12-13 19:08:37 +00:00
Yaqi Ji 1a0bef7102
fix(sdk.v2): fix boolean error for inputs (#7041) 2021-12-11 17:25:35 +00:00
Chen Sun 7921946b07
feat(sdk.v2)!: Merge v2 experimental change back to v2 namespace (#6890)
* merge v2 experimental change back to v2 namespace

* release note

* fix tests

* fix cli

* fix py3.6 test

* skip v2 compatible mode tests

* fix tests

* skip v2 engine tests
2021-11-11 00:25:42 +00:00
Chen Sun dea0823fe8
fix(sdk): Make `Artifact` type be compatible with any sub-artifact types bidirectionally (#6859) 2021-11-03 16:34:21 -07:00
Yaqi Ji 5d82fa7dc8
chore(sdk): v2 compiler to throw no task defined error (#6545)
* chore(sdk): v2 compiler to throw  no task defined error

* update and clean up release notes

* Update RELEASE.md

* Update RELEASE.md
2021-09-14 01:13:32 -07:00
Chen Sun 0fba85cfab
fix(sdk.v2): Support dict, list, bool typed input parameters from constant values and pipeline inputs. (#6523)
* fix dict list typed inputs support

* Support passing dict, list, bool typed parameter via client

* update release note

* fix test
2021-09-07 19:53:07 -07:00
Chen Sun 2f19a26ffd
chore(sdk): Format all Python files under SDK folder. (#6501)
* Reformat sdk only using the new yapf config.

* Reformat docstrings using docformatter.

* update golden files to resolve diff caused by whitespaces

* fix some tests

* format .py files under sdk/python/tests using yapf

* additional docformatter

* fix some tests
2021-09-03 11:25:11 -07:00
Ajay Gopinathan f3f383c2ff
chore(sdk): Refactor and move all v2 related code to under the v2 namespace. (#6358)
* Refactor and move all v2 related code to under the v2 namespace.

Most of the changes are around imports and restructuring of the
codebase. While it looks like a lot of code was added, most of the code
already existed and was simply moved or copied over to v2. The only
exceptions are:
- under kfp/v2/components/component_factory.py: some helper functions
  were copied with simplification from _python_op.py
- we no longer strip the `_path` suffix in v2 components.

Note: there is still some duplication of code (particularly between
component_factory.py and _python_op.py), but it's ok for now since we
intend to replace some of this with v2 ComponentSpec + BaseComponent.

* Update setup.py.

* update tests.

* revert accidental change of gcpc

* Fix component entrypoint.

* Update goldens.

* fix tests.

* fix merge conflict.

* revert gcpc change.

* fix tests.

* fix tests.

* Add type aliases for moved files.

* merge and update goldens.
2021-08-17 19:25:37 -07:00
Chen Sun 6b87155a33
feat(sdk.v2): Support Exit handler in v2 compiler. (#5784) 2021-06-03 12:58:36 -07:00
Chen Sun 2991602cfc
feat(sdk): Better error message when compiling for v2. Fixes #5688, Fixes #5727 (#5759)
* Better error message when compiling for v2

* update samples
2021-05-31 13:39:03 -07:00
Chen Sun 559fe40bc9
feat(sdk): Type checking to treat "explicit" generic artifact as any artifact type. (#5661)
* allow using explicit generic artifact as any artifact type.

* address review comment
2021-05-17 20:56:40 -07:00
Alexey Volkov cc83e1089b
Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
Chen Sun d5a1853857
feat(sdk.v2): Raise error on artifacts vs. parameters mismatch in v2. (#5557)
* error out on passing artifact to parameter or vice versa.

* error out on missing type annotation for pipeline arguments.

* address test failures
2021-04-29 03:17:29 -07:00
Chen Sun a2d3f234ea
chore(sdk.v2): Remove warning on missing pipeline root at compile time (#5527) 2021-04-22 16:48:43 -07:00
Chen Sun 5dafda1270
feat(sdk.v2): Support explicit importer (#5502) 2021-04-19 13:14:09 -07:00
Chen Sun 5f97a4502d
feat(sdk.v2): remove pipeline_root from compiler interface (#5492) 2021-04-16 08:49:13 -07:00
Chen Sun 0e921c3083
feat(sdk.v2): use consistent argument name in v2 compiler (#5430) 2021-04-06 09:05:01 -07:00
Chen Sun 4c51b576f5
chore(sdk.v2): fix a couple of issues related to cross-DAG i/o referencing and more. (#5382)
* fix a couple of issues related to cross-DAG i/o

* address review comments
2021-03-29 19:12:17 -07:00
Chen Sun 21a012da4b
feat(sdk.v2): Parse pipeline param in argument values (#5183)
* parse pipeline param in argument values

* avoid duplicating input/output when type missing

* use double dash for compiler injected input name, and check for collision.
2021-03-18 09:48:16 -07:00
Chen Sun d7eb168347
chore(sdk): merge v2 dsl code back to v1. (#5227)
* Merge v2 dsl back to v1 dsl

* fix tests

* fix tests and always attach container_spec to container_op

* address review comments

* fix a bad indentation
2021-03-04 13:48:48 -08:00
Chen Sun 280cae3718
feat(sdk.v2): Support dsl.ParallelFor in v2. (#5162) 2021-02-25 05:12:15 -08:00
Chen Sun 665f3ce8ba
chore(sdk.v2): Error out on input value being a combination of constant value and PipelineParam reference. (#5185)
* parse pipeline param in argument values

* error out on input value using `.format()`
2021-02-24 14:40:14 -08:00
Chen Sun a2e2e6ca6f
feat(sdk.v2): support dsl.Condition in v2. (#5127)
* support condition in sdk.v2

* address review comments
2021-02-12 15:05:57 -08:00
Chen Sun 051a022937
feat(sdk.v2): Allow set pipeline_root via @dsl.pipeline decorator. Make pipeline_root optional. (#5107)
* Allow set pipeline_root via @dsl.pipeline decorator.

* test covering pipeline_root not set
2021-02-07 02:06:32 -08:00
Chen Sun 9e0b9aa1f6
chore(sdk): Bring back kfp.v2.components as an alias to kfp.components (#5083)
* Bring back kfp.v2.components with a DeprecationWarning

* fix test

* no deprecation warning
2021-02-03 02:07:36 -08:00
Chen Sun ecb14f40bb
chore(sdk): Remove v2 components fork, use v1 instead. (#5042)
* Remove v2.components fork

* fix setup.py
2021-01-28 18:20:07 -08:00
Chen Sun a347765630
chore (sdk.v2): Error out on using InputPath placeholder for artifacts require importer. (#4929)
* throw error on importer-inputpath combination

* address review comments
2020-12-22 17:44:27 -08:00
Chen Sun 03904ebacd
chore(sdk.v2): Error out on invalid pipeline name (#4851) 2020-12-01 14:36:51 -08:00
Chen Sun a51380b065
chore(sdk): Add InputUri and, OutputUris placeholders in v2 (#4728)
* Fix bug where we missed injecting importer node

* moved files

* address review comments

* Add InputUriPlaceholder and OutputUriPlaceholder

* support uri placholder in v2

* lint

* test

* Preserve a test case with inputPath and outputPath usage.

* fix ut

* fix import and setup

* address comments
2020-11-09 11:37:58 -08:00
Chen Sun 82b1ed4d17
chore(sdk): v2 compiler compiles to pipeline job spec (#4729)
* fix proto

* compile to pipeline job spec

* make pipeline_root required

* address comments
2020-11-06 15:05:38 -08:00
Chen Sun 55feb218d5
chore(sdk): Raises error on unsupported features in v2 compiler (#4650)
* Raises error on unsupported features

* use v2.dsl in tests

* update test
2020-10-24 15:53:00 -07:00
Chen Sun 5020fd1079
compiler for IR (#4529)
* Compile IR proto in setup.py

* compile to IR

* Fix importer node logic and lint

* cleanup and lint

* merge, undo setup.py change

* cleanup and lint

* remove currently unused code

* format _component_bridge.py

* cleanup and format

* cleanup

* upgrade protobuf in test

* restructure and test

* address review comments

* fix bug

* avoid f-strings formatting

* address review comments

* address review comments

* limit the primitive types to only int, double, and string.

* Fix test for python3.5

* use instance_schema instead of schema_title

* add v2 to setup.py

* address review comments

* move the tests closer to the code

* add more tests

* cleanup and linting

* add more tests

* fix bug on input paramter connection

* linting

* restructure tests

* fix python3.5 test failure

* support outputs.parameters placeholder

* remove pipeline decorator from v2.dsl
2020-10-13 17:13:54 -07:00