eliminate the use of `script`, replace it with
command and args. this also removes the init container that
prepares the script.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* upgrade tekton to 0.35.1
Upgrade tekton to 0.35.1, including:
- tekton manifests
- go.mod to use new pkg and update all related pkgs
- update pipeline-loops to also use new tekton pkg
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* update tekton to 0.36
update tekton, license files, and cache server
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* fix compile error on test code
for taskrun's step, it doesn't use container data struct
any more
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* update compiler
add `type` for the taskSpec.results and update all
yaml files accordingly
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* add pipelinerun pod template support for security context and automount_service_account_token
* update flag to the correct type
* update flag to the correct type
* update flag to the correct type
* update usage docs
* feat(sdk): Add helper class AddOnGroup
add a helper class for users to write a custom
OpsGroup which maps to a custom task in Tekton.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* move finally under pipelineSpec
`finally` shall be under `pipelineSpec`.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* Enhance params of AddOnGroup
make `params` argument of AddOnGroup class support
static and dynamic values.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* Add post_param() api in AddOnGroup
Add `post_param()` api to update the params of
the task
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* Handle intermediate params
Create a dedicated API to create intermediate params
which are only used by downstream Ops/OpsGroup, but not
list in the spec.params
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* fix incorrect param value
properly handle params that are created
from AddOnGroups in downstream Ops
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* handle nested AddOnGroup case
An AddOnGroup can contains another AddOnGroup
as downstream OpsGroup. The group name of an AddOnGroup
needs to follow the correct name pattern to allow
the compiler to calculate the nested case properly.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* clean up params inside the taskSpec.pipelineSpec
only keep params that are used by downstream tasks in
taskSpec.pipelineSpec.params
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* support annotations and labels
Add properties in AddOnGroup to store annotations
and labels. Then add their key/value pairs to
metadata.annotation and metadata.labels.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* rename the annotation property to annotations
amend the property name to plural
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* map sanitization logic to original kfp
* update test data to reproduce sanitization bug
* fix upstream dsl bugs on undeterministic params on long names
* fix(sdk): Apply node_selector in PipelineConf
Apply the node_selector in PipelineConf to spec.podTemplate
instead of taskPodTemplate of each task.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* update Makefile under pipelineloop
when running cli, it should do update target first
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Handle the node selector info from PipelineConf in
compiler. Node selector info in Op level could overide
PipelinConf.
settings.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* feat(SDK): extension function `Loop.enumerate()`
Add helper function: `enumerate()` in Loop to
return indices and item. It could be used to get
current iteration number and item value.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* add logic to handle nested loop
Add logic to handle nested loop and test cases
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* add tests for non-literal separator
* plug-in the tests
* process 'separator' like 'from'-'step'-'to'
* explicitly allow PipelineParam as a separator
* add test for custom-tasks with literals
* fix orig_params check for custom tasks
* fix: style: min 2 spaces before inline comment
* fix: style: 2 empty line before function decl
* add uid into big data passing midpath and update uid replacement to also replace with Tekton uid variable
* use taskrun name as midpath
* regenerate tests
* fix tests
* fix tests
* fix lint
* fix tests
* update midpath from uid to original pipelinerun
* fix lint
* replace original pipelinerun name in api to make metadata writer to work. Also add big data passing format to pipeline annotations
* update sdk loop logic to pass workspaces to sub-pipelines
* fix tests
* chain nested conditions
* add missing license
* add dependency check to remove unnecessary when expression
* minor improvement to check for the exact condition rather than just the condition task
* add auto apply custom task CRs before pipelineRun creation
* fix lint
* store loop resource in ordered dict
* Update kfp-admin-guide.md
* sorted pipeline params to avoid randomness
* sorted pipeline params to avoid randomness
* regenerate tests
* convert template string from yaml to json
* regenerate tests
* clean up unnecessary helper function
* resolve merge conflicts
* resolve merge conflicts
* remove unnecessary go mod package
* only opt-in to embed the recursion loop annotations
* only opt-in to embed the recursion loop annotations
* enable opt-in only via package variable
* add example for how to inline recursive loops
* resolve conflicts
* regenerate tests
* make auto generate and apply recursive pipelineloop in annotations as default
* move parameter sorting to the end
* add parameter sorting for recursion loops
* sort recursive task parameters
* add more details on how to opt-out auto apply custom resource
* TODO1: PipelineLoop inlined including nested loops and added tests.
* Supported recursive and parallel tasks inlining.
For recursive tasks, we simply skip them.
For Parallel tasks, relevant tests were added.
* Run all tests against inlined taskSpec and normal taskRefs
* Added CLI Flag disable/enable spec inlining.
* add cel custom task poc
* apply suggestions with minor fixes
* add initial version of plugable cel template
* fix lint
* update staging code for converting containerOp to custom task
* update staging code for converting containerOp to custom task
* remove duplicate cel conditions
* update flip-coin example to use custom task
* update flip-coin example to use custom task
* remove dsl dependencies;
* add POC comments
* fix lint
* fix self-defined CEL custom task mapping
* move custom task condition example to a separate folder
* fix tests
* make the new custom task spec and CEL condition feature as opt-in
* update instructions and generalize custom task images
* only let custom task result comparasion to skip the condition container to provide backward compatibility
* only let custom task result comparasion to skip the condition container to provide backward compatibility
* update example
* update tests
* fix comment
Tommy Li
Yihong Wang
Michalina Kotwica
Prashant Sharma
Gang Pu