Commit Graph

12 Commits

Author SHA1 Message Date
Ning 9f97a94f5e
remove support for deserializing a string of a dict for typemeta (#1439) 2019-06-06 13:58:31 -07:00
Ning b6967d88aa add default value type checking (#1407)
* add default value type checking

* add jsonschema dependency

* fix unit test error

* workaround for travis python package installation

* add back jsonschema version

* fix sample test error in type checking sample

* add jsonschema in requirements such that sphinx works fine
2019-06-03 15:33:32 -07:00
Alexey Volkov a2eab56ae8 Fixed the handling of PipelineParam-based default values in extract_metadata (#1380) 2019-05-31 20:48:23 -07:00
Alexey Volkov 871a97883c SDK - Simplified the @component decorator (#1166)
Moved metadata-related code to _metadata.
2019-04-15 19:43:06 -07:00
Alexey Volkov c67aea779e SDK - Simplified the @pipeline decorator (#1120)
* SDK - Simplified the @pipeline decorator
Moved metadata-related code to _metadata.
`Pipeline.get_pipeline_functions` now returns the list of pipeline functions.

* Addressed @gaoning777's PR feedback
2019-04-12 13:14:47 -07:00
Ning 1c4f9eb431
exposing type checking (#1022)
* exposing types under dsl.types
2019-03-26 09:33:16 -07:00
Ning 2accf4180a
Add unit tests pipelineparam (#975)
* add unit test to the pipelineparam with types
* create TypeMeta deserialize function, add comments
* strongly typed pipelineparamtuple
* addressing pr comments
2019-03-18 18:07:36 -07:00
Ning 754db1f724
Fix sample test failure because of the type information in the pipelineparam (#972)
* fix bug: op_to_template resolve the raw arguments by mapping to the argument_inputs but the argument_inputs lost the type information

* fix type pattern matching

* convert orderedDict to dict from the component module
2019-03-15 13:49:21 -07:00
Ning c829115574 Add type check (#938)
* add core types and type checking function

* fix unit test bug

* avoid defining dynamic classes

* typo fix

* add component metadata format

* add a construct for the component decorator

* add default values for the meta classes

* add input/output types to the metadata

* add from_dict in TypeMeta

* small fix

* add unit tests

* use python struct for the openapi schema

* add default in parameter

* add default value

* remove the str restriction for the param default

* bug fix

* add pipelinemeta

* add pipeline metadata

* ignore annotation if it is not str/BaseType/dict

* update param name in the check_type functions
remove schema validators for GCRPath, and adjust for GCRPath, GCSPath
change _check_valid_dict to _check_valid_type_dict to avoid confusion
fix typo in the comments
adjust function order for readability

* remove default values for non-primitive types in the function signature
update the _check_valid_type_dict name

* pass metadata from component decorator and task factory to containerOp

* pass pipeline metadata to Pipeline

* fix unit test

* typo in the comments

* move the metadata classes to a separate module

* fix unit test

* small change

* add __eq__ to meta classes
not export _metadata classes

* nothing

* fix unit test

* unit test python component

* unit test python pipeline

* fix bug: duplicate variable of args

* fix unit tests

* move python_component and _component decorator in _component file

* remove the print

* change parameter default value to None

* add functools wraps around _component decorator

* TypeMeta accept both str and dict

* fix indent, add unit test for type as strings

* do not set default value for the name field in ParameterMeta, ComponentMeta, and PipelineMeta

* add type check in task factory

* output error message

* add type check in component decorator; move the metadata assignment out of the containerop __init__ function

* fix bug; add unit test

* add more unit tests

* more unit tests; fix bugs

* more unit tests; fix bugs

* add unit tests

* more unit tests

* add type check switch; add unit tests

* add compiler option for type check

* resolving pr comments

* add unit test for pipeline param check with component types; fix the bug; also fix the bug when there are not a single return annotations
2019-03-11 11:22:12 -07:00
Ning 974d602b74
Pass meta to containerop and pipeline (#905)
pass metadata from python conf to containerop and the pipeline
2019-03-06 13:42:23 -08:00
Ning 05b1a07740 Add python conf to the metadata (#894)
* add core types and type checking function

* fix unit test bug

* avoid defining dynamic classes

* typo fix

* add component metadata format

* add a construct for the component decorator

* add default values for the meta classes

* add input/output types to the metadata

* add from_dict in TypeMeta

* small fix

* add unit tests

* use python struct for the openapi schema

* add default in parameter

* add default value

* remove the str restriction for the param default

* bug fix

* add pipelinemeta

* add pipeline metadata

* ignore annotation if it is not str/BaseType/dict

* update param name in the check_type functions
remove schema validators for GCRPath, and adjust for GCRPath, GCSPath
change _check_valid_dict to _check_valid_type_dict to avoid confusion
fix typo in the comments
adjust function order for readability

* remove default values for non-primitive types in the function signature
update the _check_valid_type_dict name

* typo in the comments

* move the metadata classes to a separate module

* fix unit test

* add __eq__ to meta classes
not export _metadata classes

* fix unit test

* fix bug: duplicate variable of args

* move python_component and _component decorator in _component file

* remove the print
2019-03-05 22:14:02 -08:00
Ning f6acbadadd
Add component metadata (#891)
* add component metadata
2019-03-05 16:02:05 -08:00