* SDK - Components - Calculate component hash digest
The digest is calculated when loading the component from URL, tfile or text.
Slightly refactored component loading - streams are no longer used, only bytes.
TODO: Calculate the digest if missing
TODO: Report possible digest conflicts
* Updated the test graph component
* Using the actual digest in the test
This makes the graph input references consistent with task output references.
This is a breaking change, but the graph components are not exposed in the documentation or samples yet.
* SDK - Tests - Test creating component from the real AutoML pipeline
Creating component from the AutoML retail_product_stockout_prediction pipeline.
* Ignoring flake8 error E821
* SDK/Components - Creating graph components from python pipeline function
`create_graph_component_from_pipeline_func` converts python pipeline function to a graph component object that can be saved, shared, composed or submitted for execution.
Example:
producer_op = load_component(component_with_0_inputs_and_2_outputs)
processor_op = load_component(component_with_2_inputs_and_2_outputs)
def pipeline1(pipeline_param_1: int):
producer_task = producer_op()
processor_task = processor_op(pipeline_param_1, producer_task.outputs['Output 2'])
return OrderedDict([
('Pipeline output 1', producer_task.outputs['Output 1']),
('Pipeline output 2', processor_task.outputs['Output 2']),
])
graph_component = create_graph_component_from_pipeline_func(pipeline1)
* Changed the signatures of exported functions
Non-public create_graph_component_spec_from_pipeline_func creates ComponentSpec
Public create_graph_component_from_pipeline_func creates component and writes it to file.
* Switched to using _extract_component_interface to analyze function signature
Stopped humanizing the input names for now. I think it's benefitial to extract interface from function signature the same way for both container and graph python components.
* Support outputs declared using pipeline function's return annotation
* Cleaned up the test
* Stop including the whole parent tasks in task output references
* By default, do not include task component specs in the graph component
Remove the component spec from component reference unless it will make the reference empty or unless explicitly asked by the user
* Exported the create_graph_component_from_pipeline_func function
* Fixed imports
* Updated the copyright year.
* SDK - Controlling which modules are captured with Lightweight components
All func_to_* functions now accept the modules_to_capture parameter: List of module names that will be captured (instead of just referencing) during the dependency scan. By default the func.__module__ is captured.
* Described the behavior more in depth.
* Added a test to check that only dependencies are captured
The zip-packed components are supported in all load_component APIs:
`kfp.components.load_component`
`kfp.components.load_component_from_file`
`kfp.components.load_component_from_url`
`kfp.components.ComponentStore.load_component`
* Switching to map-based syntax for the arguments.
The list-based syntax is going to be deprecated.
* Switched to map-style arguments in _func_to_component_spec
* Updated testdata
Alexey Volkov
Alexey Volkov
Alexey Volkov
Pascal Vicaire