Commit Graph

16 Commits

Author SHA1 Message Date
Alexey Volkov 9a1d47a185 SDK - Capturing function dependencies when creating lightweight components (#1372)
* Transitively capturing code dependencies
Using cloudpickle.

* Got rid of func_type_declarations_code variable

* Extracted the function code extraction functions

* Improved support for capturing module-level dependencies

* Added test for capturing module-level dependencies

* Removed the _capture_function_code_using_source_copy function
As requested by Ning
2019-05-28 18:18:18 -07:00
Alexey Volkov dcd5ba5ef7 SDK - Failing faster in python_op tests (#1291)
* SDK - Failing faster in python_op tests

* Switched to `subprocess.run(full_command, check=True)`
2019-05-08 13:10:36 -07:00
Eterna2 825f64d672 Feature: sidecar for ContainerOp (#879)
* Feature: sidecar for ContainerOp

* replace f-string with string format for compatibility with py3.5

* ContainerOp now can be updated with any k8s V1Container attributes as well as sidecars with Sidecar class. ContainerOp accepts PipelineParam in any valid k8 properties.

* WIP: fix conflicts and bugs with recent master. TODO: more complex template with pipeline params

* fix proxy args

* Fixed to work with latest master head

* Added container_kwargs to ContainerOp to pass in k8s container kwargs

* Fix comment bug, updated with example in ContainerOp docstring

* fix copyright year

* expose match_serialized_pipelineparam as public for compiler to process serialized pipeline params

* fixed pydoc example and removed unnecessary ContainerOp.container.parent

* Fix conflicts in compiler tests
2019-03-28 11:11:30 -07:00
Alexey Volkov 5ab368ac10 Added support for default values to Lightweight python components (#890) 2019-03-01 14:51:18 -08:00
Alexey Volkov 0c92b83013 SDK - Removed stray debug pring from tests (#701) 2019-01-22 16:28:35 -08:00
Alexey Volkov 4b5c031514 SDK/Tests - Fixed the output ordering instability in tests (#664) 2019-01-10 19:20:43 -08:00
Alexey Volkov 96ec194260 SDK/Components - Removed outputs from task factory function signature (#388)
This realizes the outputs handling roadmap and solves problems with input and output name clashes.
2018-12-03 14:52:32 -08:00
Alexey Volkov dd0bd45aa3 SDK/Components - Renamed DockerContainer spec to to Container (#323) 2018-11-20 12:47:49 -08:00
Alexey Volkov 6a37388a79 SDK/Components/Python - Removed python_op in favor of python_component (#85)
* SDK/Components/Python - Removed python_op in favor of python_component

* Added test for python_component + func_to_container_op.
* dsl.PythonComponent class is removed, because the metadata is now properly stored in the function object itself.
* build_python_component now optionally accepts base_image and no longer requires the decorator (but can use decorator's metadata).
* Made staging_gcs_path optional since it's only needed when build_image == True
* Added more validation
* Added description and parameter help to python_component

* Fixed the pipeline path string after merge.

* Addressed the PR comments.
Added more detailed explanation of base_image selection to the docstring.

* Removed extra empty lines
2018-11-16 19:32:08 -08:00
nealgao b7440570b0 minor fixes (#217)
* minor fixes

* remove the yaml output
2018-11-12 18:01:45 -08:00
Alexey Volkov 74f2a139db SDK/Tests - Removed sys.path manipulations. (#151)
Also fixed compilation path.
2018-11-08 20:15:09 -08:00
Alexey Volkov 13c65151fe Removed mentions of ark7 in tests (#111) 2018-11-06 22:23:55 -08:00
Alexey Volkov 2e3b328468 SDK/Tests/Components - Improve temporary file handling (#37)
* Improve temporary file handling in python op tests

* More small temp path fixes

* Fix tfx name bug in the tfma sample test (#67)

* fix tfx name bug

* update release build for the data publish
2018-11-05 23:44:10 -08:00
Alexey Volkov fa921babac
Using the float comparison 2018-11-05 14:16:25 -08:00
Alexey Volkov 270c0c5f5c SDK/Tests/Components - Corrected test argument types 2018-11-05 01:46:12 -08:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00