cab955f7e3
wip
2024-03-14 14:47:40 -07:00
63ae8509e3
wip
2024-03-13 14:23:49 -07:00
dde2a2ee55
wip
2024-03-13 14:23:15 -07:00
b96b7bcb5e
feat(backend): Upgrade go version to 1.20 ( #10502 )
...
* upgrade go version to 1.21
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
* upgrade integration test to go 1.21
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
* refresh go mod tidy
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
* fix license
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
* update go-sqlite3 to v1.14.19 to support go 1.21 on cache server
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
* downgrade go version to 1.20 and revert dockerfile upgrade
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
---------
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
2024-03-07 08:04:15 +00:00
ad85bad916
chore(components): Consolidate validation components in `preview.llm.rlhf_pipeline`
...
PiperOrigin-RevId: 613366137
2024-03-06 15:48:12 -08:00
1d9690321f
fix(samples): Updated samples/core to V2 ( #9879 )
...
* Updated output_a_directory.py to V2
* Update output_a_directory_test.py to V2
* Update parallel_join.py to V2
* Update multiple_outputs.ipynb to V2
* Update multiple_outputs_test.py to V2
* Updated kfp_env_validation to V2
* Updated loop_parallelism to V2
2024-03-06 22:57:14 +00:00
731cb819cd
feat(components): Implement the train time evaluation in reward model training. With the train time eval dataset available, the pipeline outputs the accuracy and cross entropy metrics to the log
...
PiperOrigin-RevId: 613057150
2024-03-05 20:08:44 -08:00
547a8aecc3
docs(components): fix `create_custom_training_job_from_component` docs rendering
...
PiperOrigin-RevId: 612985431
2024-03-05 15:16:05 -08:00
83cabab50e
feat(Backend + SDK): Update kfp backend and kubernetes sdk to support ImagePullPolicy ( #10417 )
...
* feat(Backend + SDK): Update kfp backend and kubernetes sdk to support ImagePullPolicy.
Signed-off-by: Revital Sur <eres@il.ibm.com>
* Fix format.
Signed-off-by: Revital Sur <eres@il.ibm.com>
* Update apiserver.csv.
Signed-off-by: Revital Sur <eres@il.ibm.com>
* Update licenses.
Signed-off-by: Revital Sur <eres@il.ibm.com>
---------
Signed-off-by: Revital Sur <eres@il.ibm.com>
2024-03-05 22:57:06 +00:00
b7ea6e7831
feat(components): Add CMEK validation to `preview.llm.infer_pipeline`
...
PiperOrigin-RevId: 612956960
2024-03-05 13:49:30 -08:00
19a24e3e99
fix(components): Return None as sliced feature attribution values for the classes which are not predicted in bp outputs
...
PiperOrigin-RevId: 612920651
2024-03-05 11:57:00 -08:00
b734420652
feat(backend + SDK): Add Backend and SDK support for timeout in pod spec ( #10481 )
...
* Add backend and sdk support for pod spec timeout
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
* fix conflicts
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
---------
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
2024-03-05 19:19:07 +00:00
c0cf4ad48f
fix(docs): Updated legal info due to migration from CLA to DCO ( #10501 )
...
* Updated legal info due to migration from CLA to DCO
Signed-off-by: hbelmiro <helber.belmiro@gmail.com>
* Updated DCO link
Signed-off-by: hbelmiro <helber.belmiro@gmail.com>
---------
Signed-off-by: hbelmiro <helber.belmiro@gmail.com>
2024-03-05 17:21:06 +00:00
c3895ba534
chore(components): Change docker image URI used by `preview.llm` pipelines
...
PiperOrigin-RevId: 612662160
2024-03-04 18:47:53 -08:00
dd0c17d991
feat(backend + SDK): Add backend and SDK support to use Kubernetes FieldPath as env ( #10496 )
...
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
2024-03-04 22:31:06 +00:00
624fc04fc9
fix(components): Propagate location to sub-components in AutoSxS
...
PiperOrigin-RevId: 612553652
2024-03-04 12:54:19 -08:00
bab437381d
chore(components): Switch default machine type for LLM Text Generation Eval pipeline and components to e2-standard-4
...
PiperOrigin-RevId: 612531671
2024-03-04 11:55:39 -08:00
0b1553eb05
fix(components): rename custom task calibration_score_rubric -> score_rubric
...
PiperOrigin-RevId: 611161020
2024-02-28 11:06:05 -08:00
c051e55dc3
test: Add ParallelFor compile test over single artifact. ( #10531 )
...
* .
* formatting
2024-02-28 06:14:59 +00:00
755c1f9898
fix(components): Pass tuned model checkpoint to inference pipeline after RLHF tuning
...
PiperOrigin-RevId: 610918020
2024-02-27 16:53:57 -08:00
144761c948
fix(sdk): Prevents dsl.ParallelFor over single parameter from compiling. ( #10494 )
...
* fix(sdk): Prevents dsl.ParallelFor over single paramter from compiling.
* fix(sdk): Prevents dsl.ParallelFor over single paramter from compiling.
* update PR number in release notes
* formatting
* Add compiler_test.py test for single param compile failure
* Update some docstrings and add todo
* formatting
* Update sdk/python/kfp/compiler/compiler_test.py
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
* Update sdk/python/kfp/compiler/compiler_test.py
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
* Update sdk/python/kfp/dsl/for_loop.py
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
* Use print_and_return and other small changes
* typo
* typo
---------
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
2024-02-27 23:41:59 +00:00
1f6ada654a
feat(components): Release Forecasting training pipelines to V1 namespace
...
PiperOrigin-RevId: 610830518
2024-02-27 12:05:52 -08:00
1edd85f1a1
feat(Backend + SDK): Update kfp backend and kubernetes sdk to support ConfigMaps as volumes and as env variables ( #10483 )
...
* Update kfp backend and kubernetes sdk to support ConfigMaps as volumes and as env
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* update go.mod, apiserver.csv and driver.csv
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* add test/snapshot/data files
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* fix tests
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* go mod tidy
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* update backend/third_party_licenses/apiserver.csv
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* update backend/third_party_licenses/apiserver.csv
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* fix comments
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* fix comments
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* update go.mod, apiserver.csv and driver.csv
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
---------
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
2024-02-24 06:36:56 +00:00
3dbf3cfb50
feat(components): Add CMEK support to `preview.llm.rlhf_pipeline`
...
PiperOrigin-RevId: 609832020
2024-02-23 14:06:47 -08:00
43c306b5d0
chore(components): internal change
...
PiperOrigin-RevId: 609757226
2024-02-23 09:50:26 -08:00
544d1fda65
feat(components): Add configurable image prefix to llm utility method
...
PiperOrigin-RevId: 609560776
2024-02-22 18:02:09 -08:00
2983a7d490
feat(Backend + SDK): Update kfp backend and kubernetes sdk to support tolerations ( #10471 )
...
* feat(Backend + SDK): Update kfp backend and kubernetes sdk to support tolerations
Signed-off-by: droctothorpe <mythicalsunlight@gmail.com>
Co-authored-by: edmondop <edmondo.porcu@gmail.com>
Co-authored-by: tarat44 <32471142+tarat44@users.noreply.github.com>
* Address PR review 1
Signed-off-by: droctothorpe <mythicalsunlight@gmail.com>
Co-authored-by: edmondop <edmondo.porcu@gmail.com>
Co-authored-by: tarat44 <32471142+tarat44@users.noreply.github.com>
* Refactor add_toleration to use Python primitives
Signed-off-by: droctothorpe <mythicalsunlight@gmail.com>
Co-authored-by: edmondop <edmondo.porcu@gmail.com>
Co-authored-by: tarat44 <32471142+tarat44@users.noreply.github.com>
* Update go.mod to pull in latest kubernetes_platform package
Signed-off-by: droctothorpe <mythicalsunlight@gmail.com>
Co-authored-by: edmondop <edmondo.porcu@gmail.com>
Co-authored-by: tarat44 <32471142+tarat44@users.noreply.github.com>
---------
Signed-off-by: droctothorpe <mythicalsunlight@gmail.com>
Co-authored-by: edmondop <edmondo.porcu@gmail.com>
Co-authored-by: tarat44 <32471142+tarat44@users.noreply.github.com>
2024-02-22 21:31:31 +00:00
e47a0e1d12
chore(components): release GCPC SDK 2.10.0
...
PiperOrigin-RevId: 609459972
2024-02-22 12:26:41 -08:00
f00df96cf1
feat(components): Added experimental args to batch_prediction_pairwise component
...
PiperOrigin-RevId: 609129336
2024-02-21 14:47:27 -08:00
4392b4a47c
docs(components): internal
...
PiperOrigin-RevId: 609107204
2024-02-21 13:38:48 -08:00
c97fec8707
chore(components): internal
...
PiperOrigin-RevId: 609081795
2024-02-21 12:17:00 -08:00
066f229e27
fix(rlhf): Supporting adapter only output for reward model training
...
PiperOrigin-RevId: 608740017
2024-02-20 14:27:59 -08:00
1ae0a8210d
fix(backend): fixes "cannot save parameter" error message. Fixes #9678 ( #10459 )
...
Signed-off-by: hbelmiro <helber.belmiro@gmail.com>
2024-02-20 21:32:53 +00:00
48243d1250
chore(components): bump highest support KFP SDK version in GCPC to KFP SDK 2.7.0
...
PiperOrigin-RevId: 608646229
2024-02-20 10:05:03 -08:00
e129b05013
docs(components): internal
...
PiperOrigin-RevId: 607802006
2024-02-16 14:29:51 -08:00
6fb997a611
feat(kubernetes_platform): Update kubernetes_platform go package to i… ( #10442 )
...
Signed-off-by: droctothorpe <mythicalsunlight@gmail.com>
Co-authored-by: edmondop <edmondo.porcu@gmail.com>
Co-authored-by: tarat44 <32471142+tarat44@users.noreply.github.com>
2024-02-16 19:42:15 +00:00
fc183f3acb
chore(components): Rename several `_implementation.llm` components
...
PiperOrigin-RevId: 607487816
2024-02-15 16:18:27 -08:00
16c2ec39b8
chore(README): Update Kubeflow Pipelines on Tekton blog ( #10482 )
2024-02-15 23:34:13 +00:00
f83ec2e7c1
chore(components): Sync AutoML components
...
PiperOrigin-RevId: 607435076
2024-02-15 13:24:14 -08:00
a332443d39
chore: Add Tomcli as a backend approver ( #10490 )
...
Signed-off-by: Chen Sun <chensun@users.noreply.github.com>
2024-02-15 20:12:20 +00:00
b9ae0951e9
feat(kubernetes_platform): Add k8s FieldPath as env to the kubernetes_platform ( #10485 )
...
Signed-off-by: Tommy Li <Tommy.chaoping.li@ibm.com>
2024-02-15 08:39:19 +00:00
449c304686
fix(components): Use PipelineJob location in AutoSxS components, add init file
...
PiperOrigin-RevId: 607055407
2024-02-14 11:40:06 -08:00
ae62b1113a
chore(sdk): release KFP SDK 2.7.0 ( #10461 )
2024-02-14 11:37:53 -08:00
664deaf933
test(components): Reduce sagemaker component test flakiness ( #10225 )
...
Signed-off-by: ananth102 <abashyam@amazon.com>
2024-02-14 19:29:10 +00:00
3f0fc06295
chore(components): support jinja 3 in GCPC
...
PiperOrigin-RevId: 606703078
2024-02-13 12:06:15 -08:00
1fcc68121c
feat(kubernetes_platform): Add ActiveDeadlineSeconds(timeout) to the kubernetes platform spec ( #10464 )
2024-02-13 00:19:04 +00:00
0b75afdd8a
chore(components): Update AutoSxS and RLHF image tags
...
PiperOrigin-RevId: 606330905
2024-02-12 12:30:46 -08:00
b3978c1e98
feat(Backend + SDK): Update kfp backend and kubernetes sdk to support pod labels and annotations ( #10393 )
...
* update kfp kubernetes sdk to include pod labels and annotations
* fix unit test output order
* add podmetadata changes
* update argo compiler to support pod metadata
* update tests
* update go mod to use the latest kubernetes_platform package
* update licenses
* address comments
* update kubernetes_platform package to include the latest spec
---------
Co-authored-by: Chen Sun <chensun@users.noreply.github.com>
2024-02-09 21:15:01 +00:00
6cc234b3f1
feat(kubernetes_platform): Update kubernetes_platform go package to include ConfigMaps as volumes and as env variables. ( #10400 )
...
* Update kubernetes_platform go package to include ConfigMaps as volumes and as env variables
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
* add ConfigMapKeyToEnvMap
* change config_map_key string
* add list of map
* rename "CmAsVolume" to "ConfigMapAsVolume" and "CmAsEnv" to "ConfigMapAsEnv"
---------
Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
2024-02-09 20:12:44 +00:00
e6ddb0c012
fix(backend): get pipeline by name is broken due to version typo, Fixes #9940 ( #10268 )
...
function getPipelineByName have a version typo:
V1beta1 -> v1beta1
2024-02-09 20:12:38 +00:00
connor-mccarthy
Tommy Li
Googler
Pratyusha R
Connor McCarthy
Revital Sur
Helber Belmiro
Michael Hu
Jason Dai
KevinGrantLee
Alexey Roytman
Alex
Alex
Chen Sun
ananth102
Cyber Nagle