* SDK - Testing - Run some unit-tests in a more correct way
Replaced `@unittest.expectedFailure` with `with self.assertRaises(...):`.
Replaced `assert` with `self.assertEqual(...)`.
Stopped producing the stray "comp.yaml" file.
Enabled the test_load_component_from_url test.
* Removed a stray comment
* Addded two tests for output_component_file
* SDK - Components - Added type to TaskOutputReference
Now the task output references taken from TaskSpec instances can be
type-checked when passed to components.
* Renamed TypeType to TypeSpecType
Problem: When the user loads component using the load_component function, the object they get back is a task factory function. Since it's a normal function object, the user cannot inspect any of the attributes of the component they just loaded (they can only see the name, description and input names). For example, the user cannot see the list of component outputs, the annotations etc.
This change fixes the issue by adding the original component properties to the function object.
Example usage:
```python
train_op = load_component_from_url(...)
print(train_op.outputs)
```
* SDK - Added support for raw artifact values to ContainerOp
* `ContainerOp` now gets artifact artguments from command line instead of the constructor.
* Added back input_artifact_arguments to the ContainerOp constructor.
In some scenarios it's hard to provide the artifact arguments through the `command` list when it already has resolved artifact paths.
* Exporting InputArtifactArgument from kfp.dsl
* Updated the sample
* Properly passing artifact arguments as task arguments
as opposed to default input values.
* Renamed input_artifact_arguments to artifact_arguments to reduce confusion
* Renamed InputArtifactArgument to InputArgumentPath
Also renamed input_artifact_arguments to artifact_argument_paths in the ContainerOp's constructor
* Replaced getattr with isinstance checks.
getattr is too fragile and can be broken by renames.
* Fixed the type annotations
* Unlocked the input artifact support in components
Added the test_input_path_placeholder_with_constant_argument test
* Collecting coiverage when running python tests
* Added coveralls to python unit tests
* Try removing the PATH modification
* Specifying coverage run --source
* Using the installed package
* Try getting the correct coverage paths
* Feature: sidecar for ContainerOp
* replace f-string with string format for compatibility with py3.5
* ContainerOp now can be updated with any k8s V1Container attributes as well as sidecars with Sidecar class. ContainerOp accepts PipelineParam in any valid k8 properties.
* WIP: fix conflicts and bugs with recent master. TODO: more complex template with pipeline params
* fix proxy args
* Fixed to work with latest master head
* Added container_kwargs to ContainerOp to pass in k8s container kwargs
* Fix comment bug, updated with example in ContainerOp docstring
* fix copyright year
* expose match_serialized_pipelineparam as public for compiler to process serialized pipeline params
* fixed pydoc example and removed unnecessary ContainerOp.container.parent
* Fix conflicts in compiler tests
Simplified the type compatibility tests by removing component inputs/outputs and properties that are not required for the tests to work.
Renamed (+split) some of the tests:
`test_type_check_all_with_types` -> `test_type_compatibility_check_for_simple_types` + `test_type_compatibility_check_for_types_with_parameters`
`test_type_check_with_lacking_types` -> `test_type_compatibility_check_when_input_type_is_missing` + `test_type_compatibility_check_when_argument_type_is_missing`
`test_type_check_with_inconsistent_types_property_value` -> `test_fail_type_compatibility_check_when_type_property_value_is_different`
`test_type_check_with_inconsistent_types_type_name` -> `test_fail_type_compatibility_check_when_simple_type_name_is_different + test_fail_type_compatibility_check_when_parametrized_type_name_is_different`
`test_type_check_with_consistent_types_nonnamed_inputs` -> `test_type_compatibility_check_when_using_positional_arguments`
`test_type_check_with_inconsistent_types_disabled` ->` test_type_compatibility_check_not_failing_when_disabled`
`test_type_check_with_openapi_shema` -> `test_type_compatibility_check_for_types_with_schema`
`test_type_check_ignore_type` -> test_fail_type_compatibility_check_for_types_with_different_schemas` + `test_type_compatibility_check_not_failing_when_type_is_ignored`
Added two disabled tests:
`test_type_compatibility_check_when_argument_type_has_extra_type_parameters`
`test_fail_type_compatibility_check_when_argument_type_has_missing_type_parameters`
* remove GCSPath fields to avoid artifact type confusion
change the type json schema field name to openAPIV3Schema
* fix unit tests; add unit test for openapishema property
* add ignore_type in pipelineparam
* change the names in the artifact types to avoid confusion with the parameter types
* based on the google python style guide, change the camel case to lower case with underscores
* add core types and type checking function
* fix unit test bug
* avoid defining dynamic classes
* typo fix
* add component metadata format
* add a construct for the component decorator
* add default values for the meta classes
* add input/output types to the metadata
* add from_dict in TypeMeta
* small fix
* add unit tests
* use python struct for the openapi schema
* add default in parameter
* add default value
* remove the str restriction for the param default
* bug fix
* add pipelinemeta
* add pipeline metadata
* ignore annotation if it is not str/BaseType/dict
* update param name in the check_type functions
remove schema validators for GCRPath, and adjust for GCRPath, GCSPath
change _check_valid_dict to _check_valid_type_dict to avoid confusion
fix typo in the comments
adjust function order for readability
* remove default values for non-primitive types in the function signature
update the _check_valid_type_dict name
* pass metadata from component decorator and task factory to containerOp
* pass pipeline metadata to Pipeline
* fix unit test
* typo in the comments
* move the metadata classes to a separate module
* fix unit test
* small change
* add __eq__ to meta classes
not export _metadata classes
* nothing
* fix unit test
* unit test python component
* unit test python pipeline
* fix bug: duplicate variable of args
* fix unit tests
* move python_component and _component decorator in _component file
* remove the print
* change parameter default value to None
* add functools wraps around _component decorator
* TypeMeta accept both str and dict
* fix indent, add unit test for type as strings
* do not set default value for the name field in ParameterMeta, ComponentMeta, and PipelineMeta
* add type check in task factory
* output error message
* add type check in component decorator; move the metadata assignment out of the containerop __init__ function
* fix bug; add unit test
* add more unit tests
* more unit tests; fix bugs
* more unit tests; fix bugs
* add unit tests
* more unit tests
* add type check switch; add unit tests
* add compiler option for type check
* resolving pr comments
* add unit test for pipeline param check with component types; fix the bug; also fix the bug when there are not a single return annotations
The zip-packed components are supported in all load_component APIs:
`kfp.components.load_component`
`kfp.components.load_component_from_file`
`kfp.components.load_component_from_url`
`kfp.components.ComponentStore.load_component`
When the DSL bridge code was written, ContainerOp did not support env, so we did not pass it. Now we're adding the passing code.
Added test that chacks that the env variables get to the ContainerOp.
Ultimately, command line is an array of strings. Component yaml files should have the arguments as strings instead of Python SDK doing conversion sometimes.
* Reworked the Component structures.
Rewrote parsing, type checking and serialization code.
Improved the graph component structures.
Added most of the needed k8s structures.
Added model validation (input/output existence etc).
Added task cycle detection and topological sorting to GraphSpec.
All container component tests now work.
Added some graph component tests.
* Fixed incompatibilities with python <3.7
* Added __init__.py to make the Travis tests work.
* Adding kubernetes structures to setup.py
* Addressed PR feedback: Renamed _original_names to _serialized_names
* Addressed PR feedback: Reduced indentation.
* Added descriptions for all component structures.
* Fixed a bug in ComponentSpec._post_init()
* Added documentation for ModelBase class and functions.
* Added __eq__/__ne__ and improved __repr__
* Added ModelBase tests
* Fixed string boolean handling in if condition
* Fixed bug in isPresent
* Fixed list expansion when an item expands to a list
* Renamed two tests
* Fixed resolving primitive types (yaml supports and decodes them)
* Added test that checks handling arguments of all yaml types
* Added tests for handling true and false booleand and string literals in conditional expressions
Alexey Volkov
hongye-sun
Alexey Volkov
Eterna2
Ning
Alexey Volkov
Pascal Vicaire
Pascal Vicaire