This is needed for the future storage system based on volume mounts:
If outputs were written to files in the same dir (e.g. /outputs/out1.txt and /outputs/out2.txt), then we cannot separate them and mount to the downstream task containers independently.
* Reworked the Component structures.
Rewrote parsing, type checking and serialization code.
Improved the graph component structures.
Added most of the needed k8s structures.
Added model validation (input/output existence etc).
Added task cycle detection and topological sorting to GraphSpec.
All container component tests now work.
Added some graph component tests.
* Fixed incompatibilities with python <3.7
* Added __init__.py to make the Travis tests work.
* Adding kubernetes structures to setup.py
* Addressed PR feedback: Renamed _original_names to _serialized_names
* Addressed PR feedback: Reduced indentation.
* Added descriptions for all component structures.
* Fixed a bug in ComponentSpec._post_init()
* Added documentation for ModelBase class and functions.
* Added __eq__/__ne__ and improved __repr__
* Added ModelBase tests
* Support replacable arguments in command as well (besides arguments) in container op.
* Fix components builder.
* Fix tests.
* Follow up CR comments.
* Fix test.
* SDK/Components - Made component search locations configurable
* Raise proper error on existing, but malformed components instead of saying the component was not found.
Component name must be non-empty.
Addressed PR feedback.
* SDK/Components/Python - Removed python_op in favor of python_component
* Added test for python_component + func_to_container_op.
* dsl.PythonComponent class is removed, because the metadata is now properly stored in the function object itself.
* build_python_component now optionally accepts base_image and no longer requires the decorator (but can use decorator's metadata).
* Made staging_gcs_path optional since it's only needed when build_image == True
* Added more validation
* Added description and parameter help to python_component
* Fixed the pipeline path string after merge.
* Addressed the PR comments.
Added more detailed explanation of base_image selection to the docstring.
* Removed extra empty lines
* Fixed string boolean handling in if condition
* Fixed bug in isPresent
* Fixed list expansion when an item expands to a list
* Renamed two tests
* Fixed resolving primitive types (yaml supports and decodes them)
* Added test that checks handling arguments of all yaml types
* Added tests for handling true and false booleand and string literals in conditional expressions
Made all SDK import relative so that they files always refer to the sibling files instead of the installed package. This makes debugging and development easier since you can be sure the correct files are used.
* Switching to map-based syntax for the arguments.
The list-based syntax is going to be deprecated.
* Switched to map-style arguments in _func_to_component_spec
* Updated testdata
Alexey Volkov
qimingj
Ning
Alexey Volkov
Pascal Vicaire