Commit Graph

95 Commits

Author SHA1 Message Date
Googler a66c5990e4 feat(components): Output errors as a separate table from Arbiter
PiperOrigin-RevId: 592769441
2023-12-21 00:03:12 -08:00
Michael Hu 14de087e74 No public description
PiperOrigin-RevId: 592702825
2023-12-20 17:53:15 -08:00
Googler 44f9992d0c feat(components): change output format to allow possible post eval
PiperOrigin-RevId: 592257929
2023-12-19 09:25:57 -08:00
Googler 8eddc27bc9 chore(components): Fix argument description
PiperOrigin-RevId: 592033936
2023-12-18 15:56:52 -08:00
Googler 075d58f89f fix(components): Resolve unique model display name on each `preview.llm.rlhf_pipeline` run instead of reusing cached result
PiperOrigin-RevId: 591365087
2023-12-15 14:48:17 -08:00
Googler f51a930120 fix(components): Use `large_model_reference` as `model_reference_name` when uploading models from `preview.llm.rlhf_pipeline` instead of hardcoding value as `text-bison@001`
PiperOrigin-RevId: 591346782
2023-12-15 13:36:40 -08:00
Jason Dai 8c59816bf2 feat(components): Release new LLM Eval image version 0.5
PiperOrigin-RevId: 590702775
2023-12-13 13:53:17 -08:00
Googler 3d62d26727 feat(component): Migrate AutoSxS pipeline to preview and move related files to _implementation/llm directory to help Model Eval team use side by side metrics as part of their pipeline
PiperOrigin-RevId: 590677273
2023-12-13 12:20:20 -08:00
Googler efeed83406 feat(components): Enable text generation pipeline to generate row based metrics
docs: update description of ModelEvaluation's metadata field accordingly

PiperOrigin-RevId: 590486026
2023-12-13 00:43:57 -08:00
Googler 227eab1c68 fix(components): Use `llama-2-7b` for the base reward model when tuning `llama-2-13` with the `preview.llm.rlhf_pipeline`
PiperOrigin-RevId: 589490121
2023-12-09 18:23:01 -08:00
Googler 685634d4a3 feat(components): Add `num_microbatches` to `_implementation.llm` training components
PiperOrigin-RevId: 589267101
2023-12-08 16:07:09 -08:00
Googler 9007fb0007 feat(components): Bump image tag used by `preview.llm` pipelines
PiperOrigin-RevId: 589253163
2023-12-08 15:10:38 -08:00
Googler 708b8bd623 fix(components): Append `tune-type` label when uploading models tuned by `preview.llm.rlhf_pipeline`
PiperOrigin-RevId: 588172611
2023-12-05 13:30:22 -08:00
Googler 1f698349f1 feat(components): Enable endpoint_batch_predict component to take in publisher model by either model_address + model_id or just model_id
PiperOrigin-RevId: 586812440
2023-11-30 15:56:05 -08:00
Googler 397b1c97be feat(components): Edit embedding pipeline to use generic batch predict component to support multilingual embeddings model
PiperOrigin-RevId: 586329396
2023-11-29 07:01:49 -08:00
Googler 1e089e6f6d feat(components): [third party model inference] Initialize component
PiperOrigin-RevId: 585766918
2023-11-27 14:56:55 -08:00
Jason Dai a281dd9935 chore(components): release new LLM Eval container image `v0.4`
PiperOrigin-RevId: 584398807
2023-11-21 12:27:54 -08:00
Googler 69446b7817 chore(components): [endpoint_batch_predict] fix pipeline description
PiperOrigin-RevId: 583079495
2023-11-16 09:56:31 -08:00
Googler c23b720f10 feat(components): Group `preview.llm.rlhf_pipeline` components for more readability
PiperOrigin-RevId: 582733817
2023-11-15 10:56:39 -08:00
Jason Dai 6cfad2b348 feat(components): Create new eval dataset preprocessor for formatting eval dataset
PiperOrigin-RevId: 582731483
2023-11-15 10:47:02 -08:00
Googler c038ef0a5b chore(components): [endpoint_batch_predict] update image uri to vertex-evaluation for fishfooding
PiperOrigin-RevId: 582725363
2023-11-15 10:29:05 -08:00
Googler bcd59220f4 feat(components): Group `preview.llm.rlhf_pipeline` components for more readability
PiperOrigin-RevId: 582394326
2023-11-14 11:52:02 -08:00
Googler a927984394 feat(components): Group `preview.llm.rlhf_pipeline` components for more readability
PiperOrigin-RevId: 581354060
2023-11-10 13:32:31 -08:00
Googler 71e5a938ef feat(components):[text2sql] Integration with first party LLM model inference pipeline
PiperOrigin-RevId: 581300725
2023-11-10 10:27:54 -08:00
Googler a54ac75c8c feat(components): [endpoint_batch_predict] add retry function for retryable error
PiperOrigin-RevId: 580980127
2023-11-09 11:52:58 -08:00
Googler 4a5cbbfb8d feat(components): Update image tag used by RLHF components
PiperOrigin-RevId: 580698126
2023-11-08 16:27:14 -08:00
Jason Dai f6aad5e4e0 feat(components): add LLM Eval pipeline parameter for customizing eval dataset reference ground truth field
PiperOrigin-RevId: 580233829
2023-11-07 10:54:21 -08:00
Googler a8dd3117d5 No public description
PiperOrigin-RevId: 579961750
2023-11-06 14:40:08 -08:00
Googler 99b67ea426 chore(components): [endpoint_batch_predict] update input and output interface so that it's consistent with existing batch predict service
PiperOrigin-RevId: 579255083
2023-11-03 12:08:49 -07:00
Googler d9c3b86adc chore(components): [endpoint_batch_predict] add qms_override parameter data type conversion
PiperOrigin-RevId: 578698352
2023-11-01 18:08:16 -07:00
Googler ed9a5abe3a chore(components): [endpoint_batch_predict] update pipeline name
PiperOrigin-RevId: 578361190
2023-10-31 18:26:05 -07:00
Googler 2910d0bb52 feat(components):[text2sql] Generate SQL queries by model batch prediction
PiperOrigin-RevId: 578340803
2023-10-31 16:46:27 -07:00
Googler 9b7a5dc532 chore(components): [endpoint_batch_predict] update parameters' default value
PiperOrigin-RevId: 578266253
2023-10-31 12:28:06 -07:00
Googler f67cbfa81f feat(components): Add ability to tune chat model with `preview.llm.rlhf_pipeline`
PiperOrigin-RevId: 578262705
2023-10-31 12:15:30 -07:00
Googler d8f2c140ce feat(components): Add chat dataset preprocessor to `preview.llm.infer_pipeline`
PiperOrigin-RevId: 577988605
2023-10-30 16:20:21 -07:00
Googler 1bee8be071 feat(components): [text2sql] Generate column names by model batch predict
PiperOrigin-RevId: 576941675
2023-10-26 11:45:14 -07:00
Googler 570e56dd09 fix(components): [text2sql] Turn model_inference_results_path to model_inference_results_directory and remove duplicate comment
PiperOrigin-RevId: 576576299
2023-10-25 10:44:01 -07:00
Googler 0d75611997 feat(components): [endpoint_batch_predict] Initialize component
PiperOrigin-RevId: 576300455
2023-10-24 15:18:22 -07:00
Googler ebb42450d0 feat(components): [text2sql] Generate table names by model batch prediction
PiperOrigin-RevId: 575978329
2023-10-23 17:42:33 -07:00
Googler 21079b5910 feat(components): [text2sql] Implement preprocess component logic
PiperOrigin-RevId: 575976269
2023-10-23 17:32:54 -07:00
Googler 99fd2017a7 feat(components): Add ability to preprocess chat llama datasets to `_implementation.llm.chat_dataset_preprocessor`
PiperOrigin-RevId: 575004978
2023-10-19 14:35:42 -07:00
Googler 0e240db397 No public description
PiperOrigin-RevId: 574969883
2023-10-19 12:38:31 -07:00
Googler ea93979eed feat(components): [text2sql] Initialize evaluation component
PiperOrigin-RevId: 573329341
2023-10-13 15:08:18 -07:00
Googler 633ddeb07e feat(components): [text2sql] Initialize validate and process component
PiperOrigin-RevId: 573317490
2023-10-13 14:18:41 -07:00
Googler 9aa750e62f feat(components): [text2sql] Initialize preprocess component and integrate with text2sql pipeline
PiperOrigin-RevId: 573025616
2023-10-12 15:17:32 -07:00
Googler 4d71fdac3f feat(components): Update image tag used by llm pipelines
PiperOrigin-RevId: 573014609
2023-10-12 14:35:05 -07:00
Googler 412216f832 feat(components): Add question_answer support for AutoSxS default instructions
PiperOrigin-RevId: 572677918
2023-10-11 13:56:06 -07:00
Googler 5c44143742 feat(components): Implement timestamp filtering for chunking
PiperOrigin-RevId: 572444659
2023-10-10 20:31:22 -07:00
Googler 1f37243ec1 feat(components): Update embedding pipeline containers to use llm-pipeline image
PiperOrigin-RevId: 572411103
2023-10-10 17:17:42 -07:00
Jason Dai 9cf92c3161 feat(components): Release new model evaluation image versions
PiperOrigin-RevId: 572375018
2023-10-10 15:02:19 -07:00