Commit Graph

14 Commits

Author SHA1 Message Date
Chen Sun 758c91f767
chore: Batch update pipeline spec go generated code (#9083)
* Batch update IR go generated code

* regenerate
2023-03-31 21:53:58 +00:00
Connor McCarthy a38d7479d7
feat(sdk): support compiling platform specific features (#8940)
* support compiling platform-specific features

* fix map size change during iteration
2023-03-10 11:18:25 -08:00
Connor McCarthy 643f421fcd
feat: add platform-specific configuration protos to `kfp-pipeline-spec` package (#8896)
* feat: add platform-specific configuration protos

* update comment
2023-03-03 00:12:59 +00:00
Chen Sun c30648a390
feat(API): Update IR go package (#8614) 2022-12-21 20:18:13 +00:00
Chen Sun 5fe4f50a4f
chore(backend): clean up pipelinespec.Value usage (#7407) 2022-03-14 17:32:57 +00:00
Ajay Gopinathan 0be57c3880
feat(sdk): Use google.protobuf.Value in v2 for passing parameters. (#6804)
* Use google.protobuf.Value in v2 for passing parameters.

* retest samples.

* Fix tests.

* Update release, more cleanup.

* Use github.com/kubeflow/pipelines/api from same repo.

* Run go mod tidy

* chore: go mod tidy

* fix v2 compile error and clean up unused code

* pr comments.

* update goldens

* Fix metadata recording.

* Update kfp mlmd client.

* fix test again

* another try.

* chore: migrate v2 DAG driver input parameters to protobuf.Value + small refactorings

* fix v2 launcher + clean up

* fix a compile error

* fix a few more tests

* fix number parsing

* clean up

* disable cache_v2 test.

Co-authored-by: Yuan Gong <gongyuan94@gmail.com>
2021-10-28 14:25:51 -07:00
Chen Sun a03b41e129
chore(api): Add parameter_values in RuntimeConfig (#6790) 2021-10-21 12:27:02 -07:00
Chen Sun 3dd9e2a9b5
chore(api): release kfp-pipeline spec 0.1.12 and update golang generated code (#6684) 2021-10-05 00:53:18 -07:00
Chen Sun e23f89852d
chore: release kfp-pipeline spec 0.1.11 and update golang generated code (#6570) 2021-09-14 19:24:33 -07:00
Chen Sun c3d982f69f
chore: Release kfp-pipeline-spec 0.1.10 and update golang generated code (#6489) 2021-09-01 14:38:30 -07:00
Yaqi Ji 703ac8723e
chore: sync pipeline_spec to current version. (#6401) 2021-08-20 16:20:55 -07:00
Yuan (Bob) Gong b653ea244c
feat: pipeline spec as a separate go module (#6000)
* feat: pipeline spec as a separate go module

* use api-generator image & update release image

* clean up & update licenses

* go mod tidy
2021-07-12 20:51:16 -07:00
capri-xiyue 724e5b4c7d
chore(v2): Fetch cache result from MLMD and put cache result into argo output artifacts/parameters (#5957)
* added model and storage layer for task

* added create task api

* added api to list tasks

* modified task proto and fixed nits

* renamed variable

* fixed ut

* fixed UT

* added UT for api_converter and resource manager

* added UT for api_converter and resource manager

* fixed BE UT

* added task storage layer UT

* changed UT

* fixed foreign key typo

* added some draft code for replay argo result

* added kfp client in v2

* deleted unused code

* run go mode

* upgraded go version in go v2 test

* run go mod tidy

* added cache client

* fixed nits

* fetched MLMD output parameter

* store output artifacts metadata from cache

* added mlmd pipeline context, exuection and output event

* fixed grpc endpoint of mlpipeline

* fixed task backend bug and fixed launcher connect bug

* fixed cache bug

* added cache info

* fixed unused test

* fixed backend test

* fixed v2 build

* refactored launcher

* removed unused code

* fixed typo
2021-07-08 23:38:23 -07:00
Yuan (Bob) Gong d024935339
WIP: KFP v2 POC (#5092)
* Changes:
* add kfp v2 hello world sample
* generate go proto code
* code to unmarshal via jsonpb
* generate container spec
* add root DAG task and Makefile to run in argo
* env setup to quickly build driver container
* generate mlmd grpc golang client
* mlmd grpc client usage example
* driver-dag: put execution and context
* driver dag: put context only when execution is a DAG
* driver task: resolve parameter from parent execution
* chain dag driver with task driver
* driver: output argo parameters
* driver: build driver docker image
* driver: push image to dev registry
* compiler: root dag driver compiler
* mlmd doc
* driver: use task spec and executor spec as driver input instead
* driver: pod spec patch output for type EXECUTOR
* compiler: passing - parameter placeholder with hello world example
* include generated argo workflow in source control
* driver no longer outputs parameters
* publisher: publish parameters to execution custom properties
* driver, publisher: execution state
* publisher: remove built binary from repo
* sample: add producer consumer sample
* sample: producer consumer sample with only parameters
* e2e output parameter support
* e2e: driver resolves input parameter from tasks in the same DAG
* compiler: convert KFP task dependency to argo task dependency
* feat: refactor publisher so it can run as an entrypoint
* build: entrypoint image
* feat: executor and publisher in one container via entrypoint rewriting

* fixed compile error (#1)

* add licenses

* update readme

Co-authored-by: capri-xiyue <52932582+capri-xiyue@users.noreply.github.com>
2021-02-08 10:18:33 -08:00