* Added examples for ArtifactLocation, ResourceOp, VolumeOp, and Sidecar.
* remove resourceop and volumeop examples
* Sample for pipeline level artifact location only
* update new openscale components and pipeline
* cleanup unused library
* fix readme and naming as requested and minor cleanup
* condense Dockerfile with new base image, load PipelineModel from spark.stage
* retrigger github checks from Travis
* add step id and step name
* add step id and step name
* update python sdk version
* fix samples
* fix typo
* add env step id
* pass the name to next step
* change to wf name
* SDK: Create BaseOp class
* BaseOp class is the base class for any Argo Template type
* ContainerOp derives from BaseOp
* Rename dependent_names to deps
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: In preparation for the new feature ResourceOps (#801)
* Add cops attributes to Pipeline. This is a dict having all the
ContainerOps of the pipeline.
* Set some processing in _op_to_template as ContainerOp specific
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Simplify the consumption of Volumes by ContainerOps
Add `pvolumes` argument and attribute to ContainerOp. It is a dict
having mount paths as keys and V1Volumes as values. These are added to
the pipeline and mounted by the container of the ContainerOp.
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Add ResourceOp
* ResourceOp is the SDK's equivalent for Argo's resource template
* Add rops attribute to Pipeline: Dictionary containing ResourceOps
* Extend _op_to_template to produce the template for ResourceOps
* Use processed_op instead of op everywhere in _op_to_template()
* Add samples/resourceop/resourceop_basic.py
* Add tests/dsl/resource_op_tests.py
* Extend tests/compiler/compiler_tests.py
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Simplify the creation of PersistentVolumeClaim instances
* Add VolumeOp: A specified ResourceOp for PVC creation
* Add samples/resourceops/volumeop_basic.py
* Add tests/dsl/volume_op_tests.py
* Extend tests/compiler/compiler_tests.py
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Emit a V1Volume as `.volume` from dsl.VolumeOp
* Extend VolumeOp so it outputs a `.volume` attribute ready to be
consumed by the `pvolumes` argument to ContainerOp's constructor
* Update samples/resourceop/volumeop_basic.py
* Extend tests/dsl/volume_op_tests.py
* Update tests/compiler/compiler_tests.py
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Add PipelineVolume
* PipelineVolume inherits from V1Volume and it comes with its own set of
KFP-specific dependencies. It is aligned with how PipelineParam
instances are used. I.e. consuming a PipelineVolume leads to implicit
dependencies without the user having to call the `.after()` method on
a ContainerOp.
* PipelineVolume comes with its own `.after()` method, which can be used
to append extra dependencies to the instance.
* Extend ContainerOp to handle PipelineVolume deps
* Set `.volume` attribute of VolumeOp to be a PipelineVolume instead
* Add samples/resourceops/volumeop_{parallel,dag,sequential}.py
* Fix tests/dsl/volume_op_tests.py
* Add tests/dsl/pipeline_volume_tests.py
* Extend tests/compiler/compiler_tests.py
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Simplify the creation of VolumeSnapshot instances
* VolumeSnapshotOp: A specified ResourceOp for VolumeSnapshot creation
* Add samples/resourceops/volume_snapshotop_{sequential,rokurl}.py
* Add tests/dsl/volume_snapshotop_tests.py
* Extend tests/compiler/compiler_tests.py
NOTE: VolumeSnapshots is an Alpha feature at the time of this commit.
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* Extend UI for the ResourceOp and Volumes feature of the Compiler
* Add VolumeMounts tab/entry (Run/Pipeline view)
* Add Manifest tab/entry (Run/Pipeline view)
* Add & Extend tests
* Update tests snapshot files
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* Cleaning up the diff (before moving things back)
* Renamed op.deps back to op.dependent_names
* Moved Container, Sidecar and BaseOp classed back to _container_op.py
This way the diff is much smaller and more understandable. We can always split or refactor the file later. Refactorings should not be mixed with genuine changes.
* add arena samples to pipelines
* update directory
* update docs
* update docs
* update func name according to comments
* update samples
* update samples
* update samples
* add installation guide
* add installation guide
* update kfp package
* update author name
* change timeout unit to hour
* update API docs
* reduce image size
* update samples
* update docker images
* update docker images
* update notebook
* update notebook
* re-design API
* update the design of arena op API
* re-design ops API
* update arena launcher implementation
* refactor launcher
* fix api
* fix launcher issue
* update arena version
* update demos
* add mpi op
fix the length of name
fix the length of name
change the default image
* fix typo of metric
* update api version
* fix metric name
* fix metric name
* fix metric name
* fix metric name
* fix metric name
* fix metric name
* make it show in pipeline ui
* make it show in pipeline ui
* update demos
* update demos
* add type checking sample to sample tests
* Dadd the test script exit code to the sample test result; update the check_notebook_result script to not validate the pipeline runs when experiment arg is not provided
* fix typo
* Add a ResNet example from NVIDIA
* Integrate webapp into the pipeline
* Integrate webapp into the pipeline
* Change license from BSD to Apache
* Route webapp UI to Kubeflow UI subpath
* Passing output between ops to establish flow
* Explicit input & output dir path
* Restructure folder
* Use test images from storage bucket
Ning
Matthias Popp
Eterna2
cheyang
IronPan
Tommy Li
Riley Bauer
Jiaxin Shan
hongye-sun
Animesh Singh
Alexey Volkov
ucdmkt
Ilias Katsakioris
rostam-github
Ning
Khoa Ho