Commit Graph

24 Commits

Author SHA1 Message Date
Connor McCarthy 13736b85b6
chore(sdk): clean up kfp code style (#8027)
* remove unused imports

* use google as isort profile

* sort imports

* format with yapf

* clean end of file new line, trailing whitespace, double quoted strings
2022-07-15 00:06:34 +00:00
Yaqi Ji b693909a08
feat(sdk)!: move v1 to deprecated folder (#7291)
* chore(sdk): move v1 to deprecated

* fix testsg

* fix testsg

* fix setup.py

* fix test

* s'

* fix tests

* fix test

* retore v2 test changes

* fix py3.6 test

* fix py3.6 test

* fix py3.6 import fallback error

* remove deprecated

* fix samples test

* sample test

* fix samples

* add readme

* restroe test

* python require

* remove path

* fix tests

* inteegration tests

* remove tfx tests for dependency with kfp v1

* fix e2e

* fix e2e

* fix integration tests

* fix sampe

* move client down

* change to kfp

* add import alias

* fix

* runid

* fix dsl

* only use kfp for function

* revert train_until_good

* tfx test

* kfp

* try import

* onprem
2022-03-04 00:52:59 +00:00
Chen Sun 2f19a26ffd
chore(sdk): Format all Python files under SDK folder. (#6501)
* Reformat sdk only using the new yapf config.

* Reformat docstrings using docformatter.

* update golden files to resolve diff caused by whitespaces

* fix some tests

* format .py files under sdk/python/tests using yapf

* additional docformatter

* fix some tests
2021-09-03 11:25:11 -07:00
Joshua Carp f43ecc223d
chore(sdk): Import mock from stdlib and drop dependency. (#6456)
* Import mock from stdlib and drop dependency.

* Drop mock from requirements.

h/t @chensun
2021-08-31 16:38:44 -07:00
Alexey Volkov cc83e1089b
Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
Jiaxiao Zheng 85a3b51713
feat(sdk): Add v2 component to build_python_component (#5079)
* porting the original PR

* comment

* refactor

* remove python2

* comment on default entrypoint

* update comment

* min versioned KFP

* fix tests
2021-02-04 01:50:36 -08:00
Alexey Volkov 71c7100083 SDK - Containers - Made python package installation more robust (#2316)
Fixes https://github.com/kubeflow/pipelines/issues/2252
On some systems (e.g. in DL VM containers) `pip3` does not point to the same environment as `python3`.
2019-10-07 13:35:11 -07:00
Alexey Volkov 342abae27a SDK - Moved the _container_builder from kfp.compiler to kfp.containers (#2192)
* SDK - Moved the _container_builder from kfp.compiler to kfp.containers
This only moves the files. The imports remain the same for now.

* Simplified the imports.
2019-09-25 18:27:06 -07:00
Alexey Volkov cf681cb0f1 SDK - Switching python container components to Lightweight components code generator (#1889)
* SDK - Switching python container components to Lightweight components code generator

* Fixed the tests

Had to remove the python2 test since python2 code generation is going away (python2 is near its End of Life and Kubeflow Pipelines only support python 3.5+).

* Added description for the internal add_files parameter

* Fixed typo

* Removed the `test_func_to_entrypoint` test
This was proposed by @gaoning777: `_func_to_entrypoint` is now just a reference to `_func_to_component_spec` which is extensively covered by other tests.
2019-09-03 17:10:58 -07:00
Zane Durante f18b7fd121 Adding multiple outputs into sdk with sample (#1667)
* Added support for mulitple outputs

* Added test for multiple output

* Adding sample for multiple outputs

* func_signature now shorter form

* Added parameters tag

* Fixed func_signature mistake
2019-08-01 13:42:14 -07:00
Ning 5387e24d87 Separate codegen from containerbuild 2 (#1680)
* refactor component build code

* remove unnecessary import

* minor changes

* fix unit tests

* separate the container build from the component build; add support for directories in the containerbuilder

* minor fixes

* fix unit test

* fix tarball error

* revert changes

* unit test fix

* minor fix

* addressing comments

* removing the check_gcs_path function

* move namespace to the contructor of containerbuilder

* fix bugs
2019-07-30 14:43:50 -07:00
Ning 0283dddbee Separate codegen from containerbuild (#1679)
* refactor component build code

* remove unnecessary import

* minor changes

* fix unit tests

* fix sample test bug

* revert the change of the dependency orders

* add -u to disable python stdout buffering

* address the comments

* separate lines to look clean

* fix unit tests

* fix
2019-07-26 16:09:58 -07:00
Ning 46aafbd062
move gcshelper out of component_builder (#1658)
* move gcshelper out of component_builder
2019-07-24 14:24:08 -07:00
Alexey Volkov 848d4fb99c SDK - Replaced insecure yaml.load with yaml.safe_load (#1170)
This improves security and gets rid of security warnings.
See https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation
2019-04-23 15:26:00 -07:00
Alexey Volkov b588ba087a SDK/Tests - Properly closing tar files opened for writing (#1169) 2019-04-23 14:34:01 -07:00
Ning a6763b9599 component build support for both python2 and python3 (#730)
* component build support for both python2 and python3

* add sample test

* remove the annotations for python2 component build

* add pathlib for python2 component build

* fix component build unit test

* fix bug in the dockerfile generator

* remove exist_ok in path.mkdir to make python2 compatible

* adjust unit test

* remove pathlib dependency for python2 component build

* remove the pathlib codes in python3 component build, but use python2 code instead; add a todo to create a new sample
2019-02-25 12:56:19 -08:00
Alexey Volkov d9c133f7f7 SDK/Components/PythionContainerOp - Make the local output path configurableThis is part of a bigger effort to make all output locations manageable in preparation for storage system. (#424) 2018-12-04 19:56:56 -08:00
nealgao cc38ca063a remove fire dependency in the component image build (#384) 2018-11-26 16:31:28 -08:00
nealgao 6937e780c3 refactor component build codes (#260)
* refactor codes

* change name handler to handle
2018-11-20 20:03:34 -08:00
nealgao 5908496852 fix bug for the component build with requirements (#320)
* fix bug for the component build with requirements

* fix unit test failure
2018-11-20 15:43:28 -08:00
nealgao 7d811bff0a add support for dependencies in the component image building (#219)
* add requirement to component image build

* add unit test for DependencyVersion

* add unit tests, fix bugs

* revert some unit test change

* include the package name inside the VersionedDependency, enable the unit test for the component builder

* disable the component image build, this is done in another PR

* update the version name to dependency

* delete unuseful tests and add more useful unit tests

* add unit tests

* fix some unit tests

* fix bug in DependencyHelper
2018-11-16 21:49:14 -08:00
nealgao 263812b070 enable component build unit test (#228)
* enable component build unit test

* minor fix
2018-11-15 15:20:51 -08:00
nealgao b7440570b0 minor fixes (#217)
* minor fixes

* remove the yaml output
2018-11-12 18:01:45 -08:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00