Commit Graph

432 Commits

Author SHA1 Message Date
Chen Sun 26de102f82 chore(release): bumped version to 1.4.0-rc.1 2021-02-01 00:18:50 -08:00
Chen Sun 48a34e677d
disable discovery cache (#5058) 2021-01-29 11:08:03 -08:00
Yuku Takahashi c653c30050
fix(components): cast return value of gcp/automl/create_dataset_for_tables. Fixes #5053 (#5054) 2021-01-28 23:33:01 -08:00
Chen Sun d14f64e4fe
fix(components): Update components/local/base/Dockerfile to use Python3 (#5047) 2021-01-28 14:12:01 -08:00
Chen Sun 76246f4758
chore(components): Delete deprecated dataproc components (#5045)
* Stop building deprecated dataproc components, and delete the code

* fix cloudbuild.yaml

* more cleanup
2021-01-28 00:16:01 -08:00
Suraj Kota 2dd8de3f6f
chore(components): SageMaker fix flaky groundtruth test (#5044) 2021-01-27 20:52:01 -08:00
Chen Sun e6aeb9cc39
fix(component): Use Python3.6 to build KFP launcher component (#5039) 2021-01-27 11:06:01 -08:00
shikajiro 3660a63508
feat(components): Add json format for BigQuery output to GCS (#4994)
* Support for json output

* update to_gcs component.yaml

* add output json format test.

* add command-line param.
2021-01-14 21:25:51 -08:00
Andrey Velichkevich 34d23aa924
feat(components): Change Katib component output to JsonObject (#4974) 2021-01-14 00:57:02 -08:00
danishsamad 346bacdd2a
feat(components): Added support for specifying resources for custom model deployment (#4905) 2021-01-07 22:06:53 -08:00
Tommy Li b5af45852b
fix(components): Add default V1ContainerPort protocol value for the kfserving component (#4956) 2021-01-07 16:21:00 -08:00
Alex Lembiyeuski 27fa60b568
refactor(components): parametrize InferenceService resource watching timeout (#4951) 2021-01-07 16:20:53 -08:00
Yang Pan c484cfa46c chore(release): bumped version to 1.3.0 2021-01-07 00:39:26 -08:00
Chen Sun 8463992aaa
fix(components): Update GCP component container to Python 3.7. Fixes #4959 (#4960)
* Update GCP component container to Python 3.7, fix Dataflow client

* fix test

* add back the http patch
2021-01-06 17:09:46 -08:00
Chen Sun 5445ce82c7 chore(release): bumped version to 1.2.0 2020-12-17 23:24:32 -08:00
Andrey Velichkevich 89e42105bd
feat(components): Add v1beta1 Katib launcher and samples (#4798)
* Add Katib v1beta1 launcher and 2 examples

* Generate Notebook output

* Modify README

* Fix links

* Fix comments in mpi-job example

* Change Experiment spec structure to JsonObject

* Remove generated file

* Use component.yaml in MPI example

* Use built-in Katib API client serialization

* Update notebook example

* Assign current status to None

* Fix Copyright
2020-12-14 22:49:48 -08:00
numerology 1449cfe0a5 chore(release): bumped version to 1.1.2 2020-12-14 09:43:07 -08:00
Leonard O' Sullivan 4aa11c3c7f
feat(components) Adds RoboMaker and SageMaker RLEstimator components (#4813)
* Adds RoboMaker and SageMaker RLEstimator components

* Genericise samples

* Genericise samples

* Adds better logging and updates shim component in samples

* Adds fixes for PR comments. Updates tests accordingly

* Adds docker image reference for integration tests. Allows for setting job_name for RLEstimator training jobs

* Separate RM and SM execution roles

* Remove README reference to VPC config items

* Adds more reliable integration test for RoboMaker Simulation Job

* Simplifies integration tests

* Reverted test container entrypoints

* Update black formatting

* Update components for redbackthomson repo

* Prefix RLEstimator job name

* Add RoboMakerFullAccess to generated roles

* Update version to official 1.1.0

* Formatting int test file

* Add PassRole IAM permission to OIDC

* Adds ROBOMAKER_EXECUTION_ROLE_ARN to build vars

Co-authored-by: Nicholas Thomson <nithomso@amazon.com>
2020-12-11 13:27:27 -08:00
Kartik Kalamadi c78781d96c
fix(components): Login to Dockerhub before running tests for Sa… (#4826)
* fix(components): Login to Dockerhub before running integ tests for SageMaker Components

* Add dockerhub login for unit tests
2020-12-09 16:52:53 -08:00
Rui Fang a0a1a5d0cf chore(release): bumped version to 1.1.2-rc.1 2020-12-04 07:09:17 +00:00
hilcj c1aebb5d22 chore(release): bumped version to 1.1.1-beta.1 2020-11-26 17:58:04 +00:00
hilcj 4fe4a30545 Revert "chore(release): bumped version to 1.1.1-beta.1"
This reverts commit 9af3e79c10.
2020-11-26 16:10:10 +00:00
hilcj 9af3e79c10 chore(release): bumped version to 1.1.1-beta.1 2020-11-26 04:32:09 +00:00
hilcj bd86072a8c Revert "chore(release): bumped version to 1.1.1.beta.1"
This reverts commit 5928a2659b.
2020-11-26 04:20:10 +00:00
hilcj 5928a2659b chore(release): bumped version to 1.1.1.beta.1 2020-11-26 03:07:26 +00:00
Alexey Volkov c5e6ca2317
feat(components): Added a sample C# script component (#4815) 2020-11-24 01:53:01 -08:00
Alexey Volkov 3826edfcde
feat(components): Added a sample R script component (#4817) 2020-11-23 10:37:01 -08:00
Tommy Li 7f32dde711
fix(components): Fix kfserving component url parsing (#4780) 2020-11-19 09:30:51 -08:00
yilun-msft ec721fe94d
feat(components): Azure ML component to deploy model (#4711)
* Azure ML component to deploy model

* Adding more instructions for image repo name

* Add readme.md for pipeline deployment component

* Adding detailed instruction on model inference and docker image

* adding output for model deployment component

* merge run command and copy the entire folder

* Setting subscription after az login

* edit component yaml to search for command in src

* Add deployment config for aks

* Parsing the json and provide endpoint as the output
2020-11-18 18:54:50 -08:00
Alexey Volkov 41a377ca75
docs(components): Updated the readme (#4767) 2020-11-12 16:34:27 -08:00
Alexey Volkov 25aec5d857
doc(components): Added component index in the readme (#4762)
* Components - Added component index

* Fixed the formatting
2020-11-12 13:46:25 -08:00
Kartik Kalamadi 008985a576
fix(components): AWS SageMaker - Retry delete EKS Cluster after Integ test failure (#4662)
* fix(components): AWS SageMaker - Retry delete EKS Cluster after Integ test failure

* decrease delete eks tiemout to 15 min
2020-11-06 12:12:29 -08:00
Cesar d6a2c23f56
fix(components): sending pyspark jobs and set generated step_id to /output.txt from the generated EMR step (#4725) 2020-11-05 19:22:50 -08:00
Cesar b9e2259b4b
fix(components): sending pyspark jobs to aws EMR with the correct py … (#4721) 2020-11-05 14:52:50 -08:00
Yuan Gong 7d36f48482 chore(release): bumped version to 1.1.0-alpha.1 2020-11-02 03:01:27 +00:00
Nicholas Thomson d81c8095d0
refactor(components): AWS SageMaker - Full component refactoring (#4336)
* Temporary rebase commit

* Add yaml compiler

* Add compiler CLI

* Update Dockerfile to copy all files

* Add validate input list vs dict

* Add unit test for new train

* Add minor bug fixes

* Override tag when generating specs

* Update pydocs with formatter

* Add contributing doc

* Add formatters to CONTRIBUTING

* Add working generic logic applied to train

* Update component input and output to inherit

* Downgrade to Python 3.7

* Update add outputValue to arg list

* Updated outputValue to outputPath

* Add empty string default to not-required inputs

* Update path to component relative to root

* Update faulty False-y condition

* Update outputs to write to file

* Update doc formatting

* Update docstrings to match structure

* Add unit tests for component and compiler

* Add unit tests for component

* Add spec unit tests

* Add training unit tests

* Update unit test automation

* Add sample formatting checks

* Remove extra flake8 check in integ tests

* Add unit test black check

* Update black formatting for all files

* Update include black formatting

* Add batch component

* Remove old transform components

* Update region input description

* Add all component specs

* Add deploy component

* Add ground truth component

* Add HPO component

* Add create model component

* Add processing component

* Add workteam component

* Add spec unit tests

* Add deploy unit tests

* Add ground truth unit tests

* Add tuning component unit tests

* Add create model component unit test

* Add process component unit tests

* Add workteam component unit tests

* Remove output_path from required_args

* Remove old component implementations

* Update black formatting

* Add assume role feature

* Compiled all components

* Update doc formatting

* Fix process terminate syntax error

* Update compiler to use kfp structures

* Update nits

* Update unified requirements

* Rebase on debugging commit

* Add debugger unit tests

* Update formatting

* Update component YAML

* Fix unit test Dockerfile relative directory

* Update unit test context to root

* Update Batch to auto-generate name

* Update minor docs and formatting changes

* Update deploy name to common autogenerated

* Add f-strings to logs

* Add update support

* Add Amazon license header

* Update autogen and autoformat

* Rename SpecValidator to SpecInputParser

* Split requirements by dev and prod

* Support for checking generated specs

* Update minor changes

* Update deploy component output description

* Update components to beta repository

* Update fix unit test requirements

* Update unit test build spec for new results path

* Update deploy wait for endpoint complete

* Update component configure AWS clients in new method

* Update boto3 retry method

* Update license version

* Update component YAML versions

* Add new version to Changelog

* Update component spec types

* Update deploy config ignore overwrite

* Update component for debugging

* Update images back to 1.0.0

* Remove coverage from components
2020-10-27 14:17:57 -07:00
Alexey Volkov 616c48babe
feat(components): JSON - Added the "Build dict", "Build list" and "Combine lists" components (#4631) 2020-10-26 14:01:59 -07:00
Tommy Li 121bfdfc5f
fix(components): Move output path as arguments for ibm components (#4647) 2020-10-22 15:39:36 -07:00
Alexey Volkov f30aa0975d
feat(components): Google Cloud Optimizer - Added the "Suggest parameter sets from measurements" component (#4633)
* Components - Google Cloud Optimizer - Added the "Suggest parameter sets from measurements" component

* Added the multi-stage optimization sample pipeline
2020-10-19 02:52:13 -07:00
Alexey Volkov 46eb6f2678
feat(components): Added the Google Cloud Optimizer components (#4621)
* Components - Added the Google Cloud Optimizer components

* Added the sample pipeline
2020-10-13 02:34:27 -07:00
Alexey Volkov b656fbf41e
feat(components): Added JSON query components (#4601) 2020-10-12 03:26:55 -07:00
Alexey Volkov 329ed482e5
feat(components): Added the Run notebook using papermill component. Fixes #497 (#4578)
* Components - Added the Run notebook using papermill component

Fixes https://github.com/kubeflow/pipelines/issues/497

* Added a notebook to be used in samples

* Added the sample pipeline
2020-10-12 03:26:48 -07:00
Sushant Divate 210eb8afaa
sample azure components (#4509)
* Azure components

* incorporate pr comments
2020-10-09 18:18:47 -07:00
Suraj Kota 466147a2d8
chore(components): SageMaker integ test changes (#4603)
- fix clean up in groundtruth
2020-10-09 16:06:47 -07:00
Chen Sun d115d467ae
fix(components): fix ml_engine/train interface (#4593)
* Fix missing `job_dir_output_path`

* fix test
2020-10-09 14:45:39 -07:00
Suraj Kota e87d74f036
chore(components): SageMaker integ tests, fix for unbound variable (#4595) 2020-10-08 17:17:06 -07:00
Tommy Li 2a1fe45e18
feat(components): Add yaml support to kfserving component (#4587) 2020-10-07 17:12:16 -07:00
Amy 7e4d9f4742
update kfp installation instructions for a number of gcp components (#4594) 2020-10-07 15:12:16 -07:00
Ahmet Altay a013edc03b
fix(components): use default python version for launching dataflow jobs. Fixes #2601 (#4583) 2020-10-07 01:34:42 -07:00
Meghna Baijal 237795539f
chore(components): AWS SageMaker - Fix leaking Workteam(GroundTruth) resources (#4536) 2020-09-30 13:08:54 -07:00