Commit Graph

62 Commits

Author SHA1 Message Date
Ning c829115574 Add type check (#938)
* add core types and type checking function

* fix unit test bug

* avoid defining dynamic classes

* typo fix

* add component metadata format

* add a construct for the component decorator

* add default values for the meta classes

* add input/output types to the metadata

* add from_dict in TypeMeta

* small fix

* add unit tests

* use python struct for the openapi schema

* add default in parameter

* add default value

* remove the str restriction for the param default

* bug fix

* add pipelinemeta

* add pipeline metadata

* ignore annotation if it is not str/BaseType/dict

* update param name in the check_type functions
remove schema validators for GCRPath, and adjust for GCRPath, GCSPath
change _check_valid_dict to _check_valid_type_dict to avoid confusion
fix typo in the comments
adjust function order for readability

* remove default values for non-primitive types in the function signature
update the _check_valid_type_dict name

* pass metadata from component decorator and task factory to containerOp

* pass pipeline metadata to Pipeline

* fix unit test

* typo in the comments

* move the metadata classes to a separate module

* fix unit test

* small change

* add __eq__ to meta classes
not export _metadata classes

* nothing

* fix unit test

* unit test python component

* unit test python pipeline

* fix bug: duplicate variable of args

* fix unit tests

* move python_component and _component decorator in _component file

* remove the print

* change parameter default value to None

* add functools wraps around _component decorator

* TypeMeta accept both str and dict

* fix indent, add unit test for type as strings

* do not set default value for the name field in ParameterMeta, ComponentMeta, and PipelineMeta

* add type check in task factory

* output error message

* add type check in component decorator; move the metadata assignment out of the containerop __init__ function

* fix bug; add unit test

* add more unit tests

* more unit tests; fix bugs

* more unit tests; fix bugs

* add unit tests

* more unit tests

* add type check switch; add unit tests

* add compiler option for type check

* resolving pr comments

* add unit test for pipeline param check with component types; fix the bug; also fix the bug when there are not a single return annotations
2019-03-11 11:22:12 -07:00
Alexey Volkov 7ce03f07d4 SDK/DSL/Compiler - Fixed compilation when using ContainerOp.after (#943)
Fixes https://github.com/kubeflow/pipelines/issues/941
2019-03-07 18:47:30 -08:00
Ning 9ebbaa313d support pipeline level imagepullsecret in DSL (#745)
* support pipeline level imagepullsecret in DSL

* use kubernetes native input parameter for imagepullsecrets

* expose a module level function to configure the pipeline settings for the current default pipeline
2019-02-05 13:16:43 -08:00
qimingj 3b3a15e16a Add "set_retry()" on ContainerOp. (#723)
* Add "set_retry()" on ContainerOp.

* Follow up on CR comments.

* Update docstring.

* Increase retry times for test.

* Fix test.
2019-01-23 17:35:34 -08:00
Ning 85c6413a2e Refactor Python SDK (#568)
* add some comments

* remove unused import; add license to dsl_bridge

* move_convert_k8s_obj_to_dic from compiler to k8s_helper

* move unit test
2018-12-20 09:51:09 -08:00
Chris Van Pelt 26fd724ec5 Fix for k8s dict parsing (#411)
* Add support for minio artifacts

* Add new tests for parity

* Fix for sdk env bug

* improved test
2018-11-30 11:44:38 -08:00
qimingj 0b7120c322 Now pipeline function takes direct default values rather than dsp.PipelineParam. (#110)
* Now pipeline function takes direct default values rather than dsp.PipelineParam. It simplifies the sample code a lot.

* Remove extraneous parenthesis.

* Follow up CR comments.

* Change Dockerfile (not done).

* Fix dockerfile.

* Fix Dockerfile again.

* Remove unneeded installation of packages in Dockerfile.
2018-11-26 17:13:55 -08:00
Yang Pan f28a9891d2 return self for container op initialization methods to allow chaining the construction (#313)
* switch set() to add() for better inheritance.

* return self for container op initialization methods to allow chaining the initialization

* comment
2018-11-19 16:46:55 -08:00
Yang Pan dfcf92bea8 switch set() to add() for better inheritance. (#312) 2018-11-19 15:18:56 -08:00
Yang Pan a47eb10558 Add volume, volumemount and env to container op (#300)
* [WIP] change deployment platform to gcp

* debug

* revert test

* add volume

* update test

* to list

* fix

* to list

* to list

* to list

* to list

* stage

* update

* update

* Undid style changes

* address comments

* update comments
2018-11-16 18:36:38 -08:00
Yang Pan 7e34b12e8d Add gcp secret parameter to container op (#261)
* add secret

* add secret to contianer op

* update comments

* address comments

* update logic

* fix
2018-11-15 10:06:14 -08:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00