Commit Graph

18 Commits

Author SHA1 Message Date
Alexey Volkov 71c7100083 SDK - Containers - Made python package installation more robust (#2316)
Fixes https://github.com/kubeflow/pipelines/issues/2252
On some systems (e.g. in DL VM containers) `pip3` does not point to the same environment as `python3`.
2019-10-07 13:35:11 -07:00
Alexey Volkov 342abae27a SDK - Moved the _container_builder from kfp.compiler to kfp.containers (#2192)
* SDK - Moved the _container_builder from kfp.compiler to kfp.containers
This only moves the files. The imports remain the same for now.

* Simplified the imports.
2019-09-25 18:27:06 -07:00
Alexey Volkov cf681cb0f1 SDK - Switching python container components to Lightweight components code generator (#1889)
* SDK - Switching python container components to Lightweight components code generator

* Fixed the tests

Had to remove the python2 test since python2 code generation is going away (python2 is near its End of Life and Kubeflow Pipelines only support python 3.5+).

* Added description for the internal add_files parameter

* Fixed typo

* Removed the `test_func_to_entrypoint` test
This was proposed by @gaoning777: `_func_to_entrypoint` is now just a reference to `_func_to_component_spec` which is extensively covered by other tests.
2019-09-03 17:10:58 -07:00
Zane Durante f18b7fd121 Adding multiple outputs into sdk with sample (#1667)
* Added support for mulitple outputs

* Added test for multiple output

* Adding sample for multiple outputs

* func_signature now shorter form

* Added parameters tag

* Fixed func_signature mistake
2019-08-01 13:42:14 -07:00
Ning 5387e24d87 Separate codegen from containerbuild 2 (#1680)
* refactor component build code

* remove unnecessary import

* minor changes

* fix unit tests

* separate the container build from the component build; add support for directories in the containerbuilder

* minor fixes

* fix unit test

* fix tarball error

* revert changes

* unit test fix

* minor fix

* addressing comments

* removing the check_gcs_path function

* move namespace to the contructor of containerbuilder

* fix bugs
2019-07-30 14:43:50 -07:00
Ning 0283dddbee Separate codegen from containerbuild (#1679)
* refactor component build code

* remove unnecessary import

* minor changes

* fix unit tests

* fix sample test bug

* revert the change of the dependency orders

* add -u to disable python stdout buffering

* address the comments

* separate lines to look clean

* fix unit tests

* fix
2019-07-26 16:09:58 -07:00
Ning 46aafbd062
move gcshelper out of component_builder (#1658)
* move gcshelper out of component_builder
2019-07-24 14:24:08 -07:00
Alexey Volkov 848d4fb99c SDK - Replaced insecure yaml.load with yaml.safe_load (#1170)
This improves security and gets rid of security warnings.
See https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation
2019-04-23 15:26:00 -07:00
Alexey Volkov b588ba087a SDK/Tests - Properly closing tar files opened for writing (#1169) 2019-04-23 14:34:01 -07:00
Ning a6763b9599 component build support for both python2 and python3 (#730)
* component build support for both python2 and python3

* add sample test

* remove the annotations for python2 component build

* add pathlib for python2 component build

* fix component build unit test

* fix bug in the dockerfile generator

* remove exist_ok in path.mkdir to make python2 compatible

* adjust unit test

* remove pathlib dependency for python2 component build

* remove the pathlib codes in python3 component build, but use python2 code instead; add a todo to create a new sample
2019-02-25 12:56:19 -08:00
Alexey Volkov d9c133f7f7 SDK/Components/PythionContainerOp - Make the local output path configurableThis is part of a bigger effort to make all output locations manageable in preparation for storage system. (#424) 2018-12-04 19:56:56 -08:00
nealgao cc38ca063a remove fire dependency in the component image build (#384) 2018-11-26 16:31:28 -08:00
nealgao 6937e780c3 refactor component build codes (#260)
* refactor codes

* change name handler to handle
2018-11-20 20:03:34 -08:00
nealgao 5908496852 fix bug for the component build with requirements (#320)
* fix bug for the component build with requirements

* fix unit test failure
2018-11-20 15:43:28 -08:00
nealgao 7d811bff0a add support for dependencies in the component image building (#219)
* add requirement to component image build

* add unit test for DependencyVersion

* add unit tests, fix bugs

* revert some unit test change

* include the package name inside the VersionedDependency, enable the unit test for the component builder

* disable the component image build, this is done in another PR

* update the version name to dependency

* delete unuseful tests and add more useful unit tests

* add unit tests

* fix some unit tests

* fix bug in DependencyHelper
2018-11-16 21:49:14 -08:00
nealgao 263812b070 enable component build unit test (#228)
* enable component build unit test

* minor fix
2018-11-15 15:20:51 -08:00
nealgao b7440570b0 minor fixes (#217)
* minor fixes

* remove the yaml output
2018-11-12 18:01:45 -08:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00