* Use pipeline_spec proto from kfp-pipeline-spec package
* update imports
* use pre-released kfp-pipeline-spec (temporarily)
* Revert "use pre-released kfp-pipeline-spec (temporarily)"
This reverts commit 77f2e9a39c.
* test_requires
* version
* make kfp a namespace package
* move to the top
* version rc0
* Fix bug where we missed injecting importer node
* moved files
* address review comments
* Add InputUriPlaceholder and OutputUriPlaceholder
* support uri placholder in v2
* lint
* test
* Preserve a test case with inputPath and outputPath usage.
* fix ut
* fix import and setup
* address comments
* Compile IR proto in setup.py
* compile to IR
* Fix importer node logic and lint
* cleanup and lint
* merge, undo setup.py change
* cleanup and lint
* remove currently unused code
* format _component_bridge.py
* cleanup and format
* cleanup
* upgrade protobuf in test
* restructure and test
* address review comments
* fix bug
* avoid f-strings formatting
* address review comments
* address review comments
* limit the primitive types to only int, double, and string.
* Fix test for python3.5
* use instance_schema instead of schema_title
* add v2 to setup.py
* address review comments
* move the tests closer to the code
* add more tests
* cleanup and linting
* add more tests
* fix bug on input paramter connection
* linting
* restructure tests
* fix python3.5 test failure
* support outputs.parameters placeholder
* remove pipeline decorator from v2.dsl
* SDK - Components - Replaced Kubernetes options with generic launcher options
This reduces the schema size and makes the task launcher options more flexible.
* Removed the launcherOptions for now
* SDK - Added pinned dependency snapshot
* Downgraded zipp
The zipp package has dropped support for python3.5. https://zipp.readthedocs.io/en/latest/history.html#v2-0-0https://github.com/jaraco/zipp/issues/28
* Fixing sample building in the backend Dockerfile
Installing SDK using pip.
Using SDK's requirements.txt.
* Enabled kubernetes v11
* Reverted the backend/Dockerfile for now
* Fixed the version of kfp-server-api
* SDK - Reduce python component limitations - no import errors for custom type annotations
By default, create_component_from_func copies the source code of the function and creates a component using that source code. No global imports are captured. This is problematic for the function definition, since any annotation, that uses a type that needs to be imported, will cause error. There were some special provisions for
NamedTuple, InputPath and OutputPath, but even they were brittle (for example, "typing.NamedTuple" or "components.InputPath" annotations still caused failures at runtime).
This commit fixes the issue by stripping the type annotations from function declarations.
Fixes cases that were failing before:
```python
import typing
import collections
MyFuncOutputs = typing.NamedTuple('Outputs', [('sum', int), ('product', int)])
@create_component_from_func
def my_func(
param1: CustomType, # This caused failure previously
param2: collections.OrderedDict, # This caused failure previously
) -> MyFuncOutputs: # This caused failure previously
pass
```
* Fixed the compiler tests
* Fixed crashes on print function
Code `print(line, end="")` was causing error: "lib2to3.pgen2.parse.ParseError: bad input: type=22, value='=', context=('', (2, 15))"
* Using the strip_hints library to strip the annotations
* Updating test workflow yamls
* Workaround for bug in untokenize
* Switched to the new strip_string_to_string method
* Fixed typo.
Co-Authored-By: Jiaxiao Zheng <jxzheng@google.com>
Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>
* Updated component images to version bae654dc5c
* Updated components to version ff116b6f1a
* 0.1.40
* append old items
* fix line
Co-authored-by: renmingu <40223865+renmingu@users.noreply.github.com>
* added new secret support
* updated the documentation and env settings
* updated after feedback
* added tests
* nameing issue fixed
* renamed test to follow unittest standard
* updated after feedback
* the new test after renaming
* added the test to main
* updates after feedback
* added licensce agreement
* removed space
* updated the volume named to be generated
* secret_name as volume name and updated test
* updated the file structure
* fixed build
* SDK - Containers - Build container image from current environment
* Removed the ability to capture the active python environment (as requested by @hongye-sun)
* Added the type hint and docstring to for the return type.
* Renamed `build_image_from_env` function to `build_image_from_working_dir`
as requested by @hongye-sun
* Explained the function behavior in the documentation.
* Removed extra empty line
* Improved caching by copying python files only after installing python packages
* Made test more portable
* Added support for specifying the base_image
`kfp.containers.default_base_image = ...`
The image can also be a callable returning the image name.
* Renamed `get_python_image` to `get_python_image_for_current_version`
* Switched the default base image to Google Deep Learning container image as requested by @hongye-sun
The size of this image is 4.35GB which really concerns me. The GPU image size is 6.45GB.
* Stopped importing kfp.containers.* into kfp.*
* Fixed test
* Fixed the regex string
* Fixed the type annotation style
* Addressed @hongye-sun feedback
* Removed the container image size warning
* Fixed import failure
* add default value type checking
* add jsonschema dependency
* fix unit test error
* workaround for travis python package installation
* add back jsonschema version
* fix sample test error in type checking sample
* add jsonschema in requirements such that sphinx works fine
* Transitively capturing code dependencies
Using cloudpickle.
* Got rid of func_type_declarations_code variable
* Extracted the function code extraction functions
* Improved support for capturing module-level dependencies
* Added test for capturing module-level dependencies
* Removed the _capture_function_code_using_source_copy function
As requested by Ning
* SDK - Separated the generated api client package
* Splitting the package build scripts
* Pinning the API client package version
* Moved import kfp_server_api to the top of the file
* Added the Mac OS X prerequisite install instructions
* Moved the build_kfp_server_api_python_package.sh script to the backend dir
* Updated the dependency version span
* Reworked the Component structures.
Rewrote parsing, type checking and serialization code.
Improved the graph component structures.
Added most of the needed k8s structures.
Added model validation (input/output existence etc).
Added task cycle detection and topological sorting to GraphSpec.
All container component tests now work.
Added some graph component tests.
* Fixed incompatibilities with python <3.7
* Added __init__.py to make the Travis tests work.
* Adding kubernetes structures to setup.py
* Addressed PR feedback: Renamed _original_names to _serialized_names
* Addressed PR feedback: Reduced indentation.
* Added descriptions for all component structures.
* Fixed a bug in ComponentSpec._post_init()
* Added documentation for ModelBase class and functions.
* Added __eq__/__ne__ and improved __repr__
* Added ModelBase tests
* Expose that the python API is Python3.5+ only
Adding these classifiers (many are the same as e.g. TensorFlow) makes sure pip/pypi index pick these up as being 3.5+ only, so that developers on Python 2 are not suprised when it fails on Python 3 syntax (e.g. typehints are only py 3.5+).
* Update setup.py
* Update setup.py
* Update setup.py
Chen Sun
Jiaxiao Zheng
Yuan (Bob) Gong
Abhishek Vilas Munagekar
Alexey Volkov
hongye-sun
Alexey Volkov
Ajay Gopinathan
Renmin
Niklas Hansson
Stephen Henderson
IronPan
Ning
jingzhang36
Riley Bauer
Kevin Bache
Eterna2
IronPan
Ning
Ajay Gopinathan
Tim Zaman
Bradley Jiang
qimingj
Pascal Vicaire