* remove unused imports
* use google as isort profile
* sort imports
* format with yapf
* clean end of file new line, trailing whitespace, double quoted strings
* Support per workflow TTL (ttl_seconds_after_finished) with new format of Argo workflow manifest
* Update test for TTL
* Declare fix in release note of SDK
* update syntaxe
* Update RELEASE.md
Add Braking change due to incompatibility with KFP pre 1.7 due to Argo 2.X
* bump kfp-pipeline-spec to 0.1.11, tests for local runner will fail with kfp-pipeline-spec 0.1.9
* Local client supports additional docker options
* Format _local_client.py using yapf
* Add release note that local runner supports additional docker options
* Fix podSpecPatch bug
* wrong structure for nodeselector and bring back integreation test
* updated python compiler test
* updated to use the correct GPU type
* missed to commit the updated test config
* Updated release notes
* remove test to see if it solved the issue
* Reformat sdk only using the new yapf config.
* Reformat docstrings using docformatter.
* update golden files to resolve diff caused by whitespaces
* fix some tests
* format .py files under sdk/python/tests using yapf
* additional docformatter
* fix some tests
* feat(sdk): add default schema_version to pipeline
* sync api for go
* Fix tests and address comments
* Bump pipeline_spec version
* Fix v1 tests
* rebase to master
* sync api for go
* Fix tests and address comments
* Bump pipeline_spec version
* Fix v1 tests
* Refactor and move all v2 related code to under the v2 namespace.
Most of the changes are around imports and restructuring of the
codebase. While it looks like a lot of code was added, most of the code
already existed and was simply moved or copied over to v2. The only
exceptions are:
- under kfp/v2/components/component_factory.py: some helper functions
were copied with simplification from _python_op.py
- we no longer strip the `_path` suffix in v2 components.
Note: there is still some duplication of code (particularly between
component_factory.py and _python_op.py), but it's ok for now since we
intend to replace some of this with v2 ComponentSpec + BaseComponent.
* Update setup.py.
* update tests.
* revert accidental change of gcpc
* Fix component entrypoint.
* Update goldens.
* fix tests.
* fix merge conflict.
* revert gcpc change.
* fix tests.
* fix tests.
* Add type aliases for moved files.
* merge and update goldens.
* Add runtime resource request for GPUs
* clean up
* Updated docks and add check
* updated with test
* remove from branch
* run tests
* fix gpu vendor format
* Update after feedback
* add unit tet
* remove integration test
* clean up
* Clean up
* Updated to resource_constraints instead of resource
* fix uri placeholder in v2 compatible mode
* fix tests
* fix path generation
* fix tests
* fix test
* cleanup
* clean up
* fix test
* fix test
* fix test
* Updt argoproj/argo URLs to argoproj/argo-workflows
* Update link to workflows.ts
* Update license.txt to reduce # of changed lines
* Revert changes to backend Dockerfile & license.txt
* Update license.txt, keep line endings
* feat(sdk/dsl/compiler): support --mode flag which can turn on v2 compatible mode
* override compiler default mode using KF_PIPELINES_COMPILER_MODE env var
* update V1_LEGACY to V1
* add unit tests
* address feedback
* clean up
* cleanup again
* use absl.testing.parameterized for table driven tests
* update
* added resource request at runtime
* fixed things
* Update to use read only parameter insteadt
* added test case and better example
* Updated again
* add the validation
* add to the test suit
* work in progress
* update after feedback
* fix the test
* clean up
* clean up
* fix the path
* add the test again
* clean up
* fix tests
* feedback fix
* comment out and clean up
* handle type annotation with Optional
* remove typing._GenericAlias, which is available only after Py 3.7
* fix
* fix py3.6 test
* address review comments
* Update syntax for lightweight components v2.
Use Input[T] and Output[T] for representing input/output artifacts
of type T. This enables users to see these annotations as merely the
classes T, enabling autocompletion, reducing verbosity and simplifying
I/O types code. Users also no longer
Other changes include:
- no longer need to use .get() to obtain the artifact class
- no auto-creation of the `data` file for output artifacts. This enables
artifacts to be either directories or files.
* fix for Py 3.6
* Remove stray file.
* Add error message if Input/Output are used with non Artifact types.
* Fix e2e test.
* SDK - Refactored command-line resolving
Moved the execution engine specific code to the component bridge.
* Added placeholder_resolver
This simplifies adding custom placeholder resolving logic.
* init
* Update type annotation.
Arguments can be String, Integer, Float, Bool and PipelineParam passed from pipeline.
* Fix type annotation consistently.
* Add unit test.
* Fix unit test case.
* Fix after review, simplify by converting arguments and command to lists of str.
Adds Metrics, ClassificationMetrics SlicedClassificationMetrics to ontology types.
Common methods used by helper metric classes and Artifact class are moved to
_artifact_utils.py
Change updates dsl Artifact class to support protobuff based metadata
removing existing implementation that uses properties/custom_properties
to handle metadata.
This change modifies the ontology artifacts with enhanced schemas for
model and datasets.
* feat(sdk): Support backoffs in retry strategy
Signed-off-by: Stefano Fioravanzo <stefano@arrikto.com>
* Add Optional type hint
Signed-off-by: Stefano Fioravanzo <stefano@arrikto.com>
* add local runner which will run ops in docker or locally
* use str.format rather than f-string
* add some brief doc string in local client
* comment the unittest about running op in docker, which is not supported in CI env for now
* Add some brief docstring about DAG used in local client
* make graph/reverse_graph of DAG as property to keep them in sync
* make some methods of LocalClient static
* remove circular reference in local client
* Incapsulate artifact storage root in the constuctor of LocalClient
* Add Alpha notice for kfp.run_pipeline_func_locally
* Support list of local images in kfp.run_pipeline_func_locally
* make staticmethod to module level private method
* Trivial modification according to code review, some renaming or docstring
* local runner support components without '--' as argument prefix
* make output file of op in loop unique
* Local runner decides whether run component in docker or in local process base on ExecutionMode
Connor McCarthy
Yaqi Ji
Seb
Chen Sun
juliusvonkohout
Lin Yiming
Judah Rand
Niklas Hansson
Joshua Carp
Ajay Gopinathan
Yuan (Bob) Gong
Joe Liedtke
capri-xiyue
Alexey Volkov
dushyanthsc
Asei Sugiyama
Ilias Katsakioris
Tommy Li
StefanoFioravanzo
Jiaxiao Zheng
Lin Yiming