Commit Graph

14 Commits

Author SHA1 Message Date
Alexey Volkov a41bd106a1 SDK - Removing unneeded uses of dsl.Pipeline (#1229)
* SDK - Removing unneeded usages of dsl.Pipeline

* Fixed the naming-related issue
2019-05-14 18:48:18 -07:00
Alexey Volkov f40a22a3f4 SDK - Made ComponentSpec.implementation field optional (#1188)
* SDK - Made ComponentSpec.implementation field optional
Improved the error message when trying to convert tasks to ContainerOp.

* Switched from attribute checking to type checking
2019-04-24 12:54:46 -07:00
Alexey Volkov 929ff52fd2 Passing the annotations and labels to the ContainerOp (#1077)
Currently the annotations and labels are not passed from component to the ContainerOp. This PR fixes that.

Fixes https://github.com/kubeflow/pipelines/issues/1013
2019-04-08 22:03:05 -07:00
Eterna2 825f64d672 Feature: sidecar for ContainerOp (#879)
* Feature: sidecar for ContainerOp

* replace f-string with string format for compatibility with py3.5

* ContainerOp now can be updated with any k8s V1Container attributes as well as sidecars with Sidecar class. ContainerOp accepts PipelineParam in any valid k8 properties.

* WIP: fix conflicts and bugs with recent master. TODO: more complex template with pipeline params

* fix proxy args

* Fixed to work with latest master head

* Added container_kwargs to ContainerOp to pass in k8s container kwargs

* Fix comment bug, updated with example in ContainerOp docstring

* fix copyright year

* expose match_serialized_pipelineparam as public for compiler to process serialized pipeline params

* fixed pydoc example and removed unnecessary ContainerOp.container.parent

* Fix conflicts in compiler tests
2019-03-28 11:11:30 -07:00
Ning c829115574 Add type check (#938)
* add core types and type checking function

* fix unit test bug

* avoid defining dynamic classes

* typo fix

* add component metadata format

* add a construct for the component decorator

* add default values for the meta classes

* add input/output types to the metadata

* add from_dict in TypeMeta

* small fix

* add unit tests

* use python struct for the openapi schema

* add default in parameter

* add default value

* remove the str restriction for the param default

* bug fix

* add pipelinemeta

* add pipeline metadata

* ignore annotation if it is not str/BaseType/dict

* update param name in the check_type functions
remove schema validators for GCRPath, and adjust for GCRPath, GCSPath
change _check_valid_dict to _check_valid_type_dict to avoid confusion
fix typo in the comments
adjust function order for readability

* remove default values for non-primitive types in the function signature
update the _check_valid_type_dict name

* pass metadata from component decorator and task factory to containerOp

* pass pipeline metadata to Pipeline

* fix unit test

* typo in the comments

* move the metadata classes to a separate module

* fix unit test

* small change

* add __eq__ to meta classes
not export _metadata classes

* nothing

* fix unit test

* unit test python component

* unit test python pipeline

* fix bug: duplicate variable of args

* fix unit tests

* move python_component and _component decorator in _component file

* remove the print

* change parameter default value to None

* add functools wraps around _component decorator

* TypeMeta accept both str and dict

* fix indent, add unit test for type as strings

* do not set default value for the name field in ParameterMeta, ComponentMeta, and PipelineMeta

* add type check in task factory

* output error message

* add type check in component decorator; move the metadata assignment out of the containerop __init__ function

* fix bug; add unit test

* add more unit tests

* more unit tests; fix bugs

* more unit tests; fix bugs

* add unit tests

* more unit tests

* add type check switch; add unit tests

* add compiler option for type check

* resolving pr comments

* add unit test for pipeline param check with component types; fix the bug; also fix the bug when there are not a single return annotations
2019-03-11 11:22:12 -07:00
Alexey Volkov fa02e750da SDK/Components - Added naming.generate_unique_name_conversion_table (#716)
generate_unique_name_conversion_table replaces _make_name_unique_by_adding_index and simplifies code in several places.
2019-03-06 15:12:58 -08:00
Ning 974d602b74
Pass meta to containerop and pipeline (#905)
pass metadata from python conf to containerop and the pipeline
2019-03-06 13:42:23 -08:00
Alexey Volkov 85738cbaaf Passing the environment variables to ContainerOp (#877)
When the DSL bridge code was written, ContainerOp did not support env, so we did not pass it. Now we're adding the passing code.
Added test that chacks that the env variables get to the ContainerOp.
2019-02-28 19:29:54 -08:00
Alexey Volkov 4457e7e940 SDK/Components - More meaningful error when trying to convert graph component to ContainerOp (#710) 2019-01-24 18:15:07 -08:00
Alexey Volkov a53cb586fc SDK/Components - Added _naming._convert_to_human_name function (#715)
* SDK/Components - Moved naming-related functions to _naming.py

* SDK/Components - Added _naming._convert_to_human_name function
2019-01-24 16:07:46 -08:00
Alexey Volkov fd282d67cd SDK/Components - Simplified _create_task_factory_from_component_spec function (#662) 2019-01-14 18:02:16 -08:00
qimingj 875efea1f9 Support replacable arguments in command as well (besides arguments) in container op. (#623)
* Support replacable arguments in command as well (besides arguments) in container op.

* Fix components builder.

* Fix tests.

* Follow up CR comments.

* Fix test.
2019-01-07 07:57:36 -08:00
Alexey Volkov 199a962e42 SDK - Relative imports (#156)
Made all SDK import relative so that they files always refer to the sibling files instead of the installed package. This makes debugging and development easier since you can be sure the correct files are used.
2018-11-10 13:56:12 -08:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00