* Use `Annotated` rather than `Union` hack
* Add dependency to `setup.py`
* Add release note
Co-authored-by: Chen Sun <chensun@users.noreply.github.com>
* pydantic setup
* switch to union types
* Fix base_component tests
* switch to pydantic for python3.6 compat and fix tests
* Fix tests
* use str for type
* fix tests
* Consistent quote sign
* Address comments
* Update type
* feat(sdk): add default schema_version to pipeline
* sync api for go
* Fix tests and address comments
* Bump pipeline_spec version
* Fix v1 tests
* rebase to master
* sync api for go
* Fix tests and address comments
* Bump pipeline_spec version
* Fix v1 tests
* Refactor and move all v2 related code to under the v2 namespace.
Most of the changes are around imports and restructuring of the
codebase. While it looks like a lot of code was added, most of the code
already existed and was simply moved or copied over to v2. The only
exceptions are:
- under kfp/v2/components/component_factory.py: some helper functions
were copied with simplification from _python_op.py
- we no longer strip the `_path` suffix in v2 components.
Note: there is still some duplication of code (particularly between
component_factory.py and _python_op.py), but it's ok for now since we
intend to replace some of this with v2 ComponentSpec + BaseComponent.
* Update setup.py.
* update tests.
* revert accidental change of gcpc
* Fix component entrypoint.
* Update goldens.
* fix tests.
* fix merge conflict.
* revert gcpc change.
* fix tests.
* fix tests.
* Add type aliases for moved files.
* merge and update goldens.
* Introduce ServiceAccountTokenVolumeCredentials
Part of kubeflow/pipelines#5138
This is a subclass of TokenCredentials and implements the logic of
retrieving a service account token provided via a ProjectedVolume.
The 'get_token()' method reads and doesn't store the token as the
kubelet is refreshing it quite often.
Relevant docs:
https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection
* Attemp to use credentials when in-cluster
If the KFP client detects it's running inside a pod and the user hasn't
provided any credentials, it now attempts to set up credentials based on
a projected service account token.
* Move credentials to kfp.auth module
Also change some names and values to avoid "ml-pipeline" references.
* Import kfp.auth lazily during the Client init
* Have credentials handle errors internally
* Let ServiceAccountTokenVolumeCredentials handle OSErrors internally
* Have the default-creds-setter only check if credentials provide a
valid value
* Use Configuration's 'refresh_api_key_hook' instead of having duplicate
code
* Remove dead code
* Add kfp.auth in the setup.py
* Move abstraction to the 'refresh_api_key_hook' method
* Add link and Python 3.8/3.9 classifiers to setup.py
The link should appear on PyPI: https://pypi.org/project/kfp/#description
* use "The Kubeflow Authors" in setup.py
* Include more links in Python package description
Change updates dsl Artifact class to support protobuff based metadata
removing existing implementation that uses properties/custom_properties
to handle metadata.
This change modifies the ontology artifacts with enhanced schemas for
model and datasets.
* Use pipeline_spec proto from kfp-pipeline-spec package
* update imports
* use pre-released kfp-pipeline-spec (temporarily)
* Revert "use pre-released kfp-pipeline-spec (temporarily)"
This reverts commit 77f2e9a39c.
* test_requires
* version
* make kfp a namespace package
* move to the top
* version rc0
* Fix bug where we missed injecting importer node
* moved files
* address review comments
* Add InputUriPlaceholder and OutputUriPlaceholder
* support uri placholder in v2
* lint
* test
* Preserve a test case with inputPath and outputPath usage.
* fix ut
* fix import and setup
* address comments
* Compile IR proto in setup.py
* compile to IR
* Fix importer node logic and lint
* cleanup and lint
* merge, undo setup.py change
* cleanup and lint
* remove currently unused code
* format _component_bridge.py
* cleanup and format
* cleanup
* upgrade protobuf in test
* restructure and test
* address review comments
* fix bug
* avoid f-strings formatting
* address review comments
* address review comments
* limit the primitive types to only int, double, and string.
* Fix test for python3.5
* use instance_schema instead of schema_title
* add v2 to setup.py
* address review comments
* move the tests closer to the code
* add more tests
* cleanup and linting
* add more tests
* fix bug on input paramter connection
* linting
* restructure tests
* fix python3.5 test failure
* support outputs.parameters placeholder
* remove pipeline decorator from v2.dsl
* SDK - Components - Replaced Kubernetes options with generic launcher options
This reduces the schema size and makes the task launcher options more flexible.
* Removed the launcherOptions for now
* SDK - Added pinned dependency snapshot
* Downgraded zipp
The zipp package has dropped support for python3.5. https://zipp.readthedocs.io/en/latest/history.html#v2-0-0https://github.com/jaraco/zipp/issues/28
* Fixing sample building in the backend Dockerfile
Installing SDK using pip.
Using SDK's requirements.txt.
* Enabled kubernetes v11
* Reverted the backend/Dockerfile for now
* Fixed the version of kfp-server-api
* SDK - Reduce python component limitations - no import errors for custom type annotations
By default, create_component_from_func copies the source code of the function and creates a component using that source code. No global imports are captured. This is problematic for the function definition, since any annotation, that uses a type that needs to be imported, will cause error. There were some special provisions for
NamedTuple, InputPath and OutputPath, but even they were brittle (for example, "typing.NamedTuple" or "components.InputPath" annotations still caused failures at runtime).
This commit fixes the issue by stripping the type annotations from function declarations.
Fixes cases that were failing before:
```python
import typing
import collections
MyFuncOutputs = typing.NamedTuple('Outputs', [('sum', int), ('product', int)])
@create_component_from_func
def my_func(
param1: CustomType, # This caused failure previously
param2: collections.OrderedDict, # This caused failure previously
) -> MyFuncOutputs: # This caused failure previously
pass
```
* Fixed the compiler tests
* Fixed crashes on print function
Code `print(line, end="")` was causing error: "lib2to3.pgen2.parse.ParseError: bad input: type=22, value='=', context=('', (2, 15))"
* Using the strip_hints library to strip the annotations
* Updating test workflow yamls
* Workaround for bug in untokenize
* Switched to the new strip_string_to_string method
* Fixed typo.
Co-Authored-By: Jiaxiao Zheng <jxzheng@google.com>
Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>
* Updated component images to version bae654dc5c
* Updated components to version ff116b6f1a
* 0.1.40
* append old items
* fix line
Co-authored-by: renmingu <40223865+renmingu@users.noreply.github.com>
* added new secret support
* updated the documentation and env settings
* updated after feedback
* added tests
* nameing issue fixed
* renamed test to follow unittest standard
* updated after feedback
* the new test after renaming
* added the test to main
* updates after feedback
* added licensce agreement
* removed space
* updated the volume named to be generated
* secret_name as volume name and updated test
* updated the file structure
* fixed build
Judah Rand
Alex Latchford
Chen Sun
Joshua Carp
Yaqi Ji
Ajay Gopinathan
Ilias Katsakioris
john li
Alexey Volkov
dushyanthsc
Yuan (Bob) Gong
Jiaxiao Zheng
Yuan (Bob) Gong
Abhishek Vilas Munagekar
hongye-sun
Alexey Volkov
Renmin
Niklas Hansson
Stephen Henderson