Commit Graph

122 Commits

Author SHA1 Message Date
Tommy Li 453e261892
Code clean up and refactor (#108)
* support input artifacts

* fix conflicts

* initial refactor;

* refactor code

* refactor code

* code refactor
2020-04-23 17:10:08 -07:00
Andrew Butler e8d973a57d
Add support for passing task outputs to conditions as parameters (#105)
* Add support for task output parameter passing to conditions
Add support for highly nested conditions

* Add minor commenting

* Minor formatting

* Update test case and sample report

* Refactor and add warning

* Add licenses and unit test

* Remove redundant import
2020-04-22 13:17:44 -07:00
Tommy Li a47ae08d51
Add volumeop and volumesnapshot support (#93)
* add volumeop support workaround

* add volumesnapshot workaround

* update resourceop integration

* add sorting to avoid randomness

* fix bug

* update sample

* minor code cleanup
2020-04-21 13:11:52 -07:00
Tommy Li 3d57d617c3
Support artifact outputs (#92)
* init commit

* update tests

* update readme

* add missing comments

* fix confilcts

* update test cases with integration test

* remove extra spaces

* fix readme command

* address comments

* fix init condition for new test methods
2020-04-21 11:37:52 -07:00
Andrew Butler 6aa7ed3a10
Fix special compilation cases for kfp samples (#91)
* fix test cases

* add eof newline

* fix file restructure

* Generalize for special cases
Generalize for any nested pipeline or pipeline without a decorator as long as there is a config file with the necessary information to compile the pipelines

* Add more clear error throwing
also fix test_kfp_samples_report.txt

* Check for special pipelines from config file

* Add comment
2020-04-19 19:49:39 -07:00
Feng Li ebe05e3a4b
Add support for dsl.ResourceOp (#89) 2020-04-11 16:27:47 -07:00
Kevin Yu cf8e99866d
Add support for pulling the image secret (#88)
* initial code

* address comments

* address comments

* generate service account name based on pipelinerun name

* re-generate the yaml
2020-04-10 11:31:47 -07:00
Tommy Li 768220989a
fix regex to only find argo specific variables (#83) 2020-04-09 13:01:47 -07:00
Andrew Butler 9295e4ffe8
Add support for conditions (#84)
* Add conditional functionality
Adds conditional functionality except for having condition parameters being passed in from task outputs

* Remove extra parameter passing functionality

* Cleanup

* Add test example and update test report
2020-04-08 10:55:43 -07:00
Tommy Li 332881895a
Add argo variable checking when compile to Tekton. (#80)
* add initial parsing

* add regex mapping for the equivalent tekton variables

* revert old changes

* fix typos

* raise error when unsupported Argo vars are used

* fix comment
2020-04-03 15:29:47 -07:00
Feng Li cf3968125d
Extend compiler to support parallelFor with loop static params (#67) 2020-04-02 19:55:46 -07:00
Tommy Li 910ba9b604
Fix no default param bug (#79) 2020-04-02 15:57:45 -07:00
Tommy Li d990f2414f
Copy param output files using emptyDir volumes (#75)
* init code

* add initial workspace workaround

* added workspaces to pipelinerun

* fix typo

* only use workspace if necessary

* add extra steps for copying non configurable output file path

* update comment

* update container image

* update copy step to use busybox

* update comments

* replace workspaces with emptydir volume

* revert workspace changes

* fix volume edge case

* update comments

* update test report
2020-04-02 15:55:46 -07:00
Tommy Li f088d2a465
Add more k8s resource support (#65)
* add more k8s resource support

* add test main functions as requested

* add description on why we need pipelinerun
2020-03-30 16:18:40 -07:00
Feng Li 0fb5ab8440
Extend compiler to support workflow level timeout (#68) 2020-03-27 14:57:53 -07:00
Tommy Li 5b52b79bd8
Add initial pipelinerun support (#62)
* add initial pipelinerun support

* update comment

* rearrange compile function
2020-03-26 14:20:23 -07:00
Feng Li 359e750b17
support retry, address the comments (#55) 2020-03-24 18:13:02 -07:00
Feng Li 10896c8036
Support timeout defining for pipeline tasks (#46) 2020-03-24 12:57:02 -07:00
Tommy Li 8e944cfc1e
Add volumemount support and fix parameter bugs (#40)
* add volumemount support and fix parameter bugs

* remove sdk/samples folder and use real kfp example

* fix formatting

* update unittest and instructions

* update license header

* update license header

* update license
2020-03-20 15:32:44 -07:00
Christian Kadner 9ff9467daf
Task output parameters as input for subsequent tasks (#27)
Closes #19
2020-03-17 17:08:14 -07:00
Christian Kadner 3167232886
Generate Tekton Pipeline with Tasks (#17)
Replace Steps with Tasks to allow parallel task execution.
Use 'runAfter' to support sequential task execution.
2020-03-11 19:46:38 -07:00
Christian Kadner faf700f222
Refactor code to extend KFP SDK compiler (#16)
This code change removes the requirement to clone and
copy the kubeflow/pipelines repository. The project
has been refactored to only contain the minimal set of
Python classes and methods that needed to be modified
to generate Tekton YAML instead of Argo YAML. The
overriden methods are dynamically linked at runtime
by monkey-patching the KFP SDK module.

Additionally adding a simple test case and setup.py
install script that creates the dsl-compile-tekton
CLI executable which works independent of the KFP
compiler executable.
2020-03-03 18:37:48 -08:00