35720448da
switch the release script from staging to test ( #1328 )
...
/assign @gaoning777
2019-05-14 11:29:50 -07:00
13d506c08a
add patch for metric collection ( #1317 )
2019-05-14 11:12:32 -07:00
9cf1fa7d90
update tf version to 1.12.1 ( #1315 )
2019-05-13 20:01:08 -07:00
8418b316f5
Update new Watson OpenScale components and pipeline ( #1287 )
...
* update new openscale components and pipeline
* cleanup unused library
* fix readme and naming as requested and minor cleanup
* condense Dockerfile with new base image, load PipelineModel from spark.stage
* retrigger github checks from Travis
2019-05-10 01:09:57 -07:00
cb87b94507
Make confusion_matrix and roc generic ( #1285 )
2019-05-08 16:48:47 -07:00
442c80449d
add nuclio components (to build/deploy, delete, invoke functions) ( #1295 )
...
* add support for flexible config (via env var) for the pipline service and UI, fix broken links (pointed to API vs UI service)
* add namespace and change address to endpoint
2019-05-08 01:58:33 -07:00
f5c464a87d
Add AWS EMR and Athena components ( #1286 )
...
* Add EMR and Athena components
* Update components/aws/athena/query/component.yaml
Co-Authored-By: Jeffwan <seedjeffwan@gmail.com>
* Clean up athena components
2019-05-06 17:11:44 -07:00
2a42adab79
Updating component versions in samples ( #1283 )
2019-05-06 15:21:38 -07:00
36d31fe23d
make pending timeout customizable ( #1268 )
...
* make pending timeout customizable
* fix the description of arg
2019-05-04 18:53:36 -07:00
cef50553f9
spark components ( #1272 )
...
* spark components
* changing base image
* removing multiple open file methods
* renaming source files
* renaming source files
* adding component yaml files
* removing extra bracket signs
2019-05-03 16:09:39 -07:00
5374b6b2b4
Add SageMaker components and example pipeline ( #1276 )
...
* Add SageMaker components and example pipeline
* Address review feedbacks
* Expose more training job configs
* Update components/aws/sagemaker/batch_transform/component.yaml
Update components descriptions
Co-Authored-By: Jeffwan <seedjeffwan@gmail.com>
2019-05-03 14:55:37 -07:00
f56a8cb72a
Comp yaml eb830cd73c ( #1282 )
2019-05-03 13:11:31 -07:00
4060415df8
Automated the component image release script ( #1172 )
...
The script now checks out the clean repo, makes the changes and commits them.
Then it optionally pushes the changes to a new release branch and even opens the web-browser with a pull request page ready.
2019-05-03 12:23:27 -07:00
eb830cd73c
Component 727c48c690 ( #1280 )
2019-05-03 11:20:40 -07:00
5bbf213829
Update arena SDK version ( #1251 )
...
* update python sdk version
* update python sdk version
* update python sdk version
2019-04-29 16:39:36 -07:00
c77554a762
Fix Watson pipeline example ( #1246 )
...
* update ibm watson examples
* fix labels in components
* typo fixes
* revert component name change
* minor description changes
* move label tags to annotations
2019-04-29 11:12:20 -07:00
9f6d34ff65
Fixing comp names ( #1239 )
...
* changing comp names
* changing comp names
2019-04-26 11:52:29 -07:00
7228ba1f0b
Release b0147bdbed ( #1238 )
...
* Updated component images to version b0147bdbede8524eefb1
2019-04-26 10:24:26 -07:00
8c8e5052df
Expose step id and step name ( #1191 )
...
* add step id and step name
* add step id and step name
* update python sdk version
* fix samples
* fix typo
* add env step id
* pass the name to next step
* change to wf name
2019-04-25 14:56:28 -07:00
b70a8bef7a
shortening names ( #1202 )
2019-04-24 17:10:45 -07:00
173ecbda4c
Marked all scripts as executable ( #1177 )
2019-04-23 16:12:00 -07:00
848d4fb99c
SDK - Replaced insecure yaml.load with yaml.safe_load ( #1170 )
...
This improves security and gets rid of security warnings.
See https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation
2019-04-23 15:26:00 -07:00
fe042540e8
update types to defined core type names ( #1206 )
2019-04-23 12:44:05 -07:00
2eddf0e395
Update arena component with git support ( #1179 )
...
* update sample
* fix git sync
* make downloading docker image automatically
* make downloading docker image automatically
* make downloading docker image automatically
* fix typo
* use extend to replace append
2019-04-18 23:39:54 -07:00
1115fa5823
Apply latest doc review changes to github docs ( #1128 )
...
* Apply latest doc review changes to github docs
* merge changes from tech writer
* adding missing dataproc components
2019-04-18 11:22:00 -07:00
785d474699
release new components and remove the resnet image release in the release script ( #1171 )
2019-04-16 14:19:09 -07:00
6806f83feb
Build Pipeline leveraging Arena ( #1058 )
...
* add arena samples to pipelines
* update directory
* update docs
* update docs
* update func name according to comments
* update samples
* update samples
* update samples
* add installation guide
* add installation guide
* update kfp package
* update author name
* change timeout unit to hour
* update API docs
* reduce image size
* update samples
* update docker images
* update docker images
* update notebook
* update notebook
* re-design API
* update the design of arena op API
* re-design ops API
* update arena launcher implementation
* refactor launcher
* fix api
* fix launcher issue
* update arena version
* update demos
* add mpi op
fix the length of name
fix the length of name
change the default image
* fix typo of metric
* update api version
* fix metric name
* fix metric name
* fix metric name
* fix metric name
* fix metric name
* fix metric name
* make it show in pipeline ui
* make it show in pipeline ui
* update demos
* update demos
2019-04-15 18:55:04 -07:00
d24dba5aae
Adding myself as a reviewer for components ( #1161 )
...
Have been reviewing many components and sample related PRs, and feel pretty confident addressing them
2019-04-15 13:23:21 -07:00
c0da565e08
component readmes ( #1147 )
2019-04-14 20:48:00 -07:00
55f7e328dc
Expose service-type as parameter ( #1136 )
...
* Expose service-type as parameter
* Expose service-type as parameter
2019-04-12 20:16:03 -07:00
39378a6a06
update katib laucher ( #1118 )
...
* update katib laucher
1. sync up since katib directory org changed
2. add option deleteafterdone
* fix comment
* fix comment
2019-04-09 19:00:13 -07:00
b1e4a569f1
uprade tf-serving in deployer component ( #1103 )
2019-04-08 14:36:27 -07:00
fee1aea915
Rewrite ResNet sample by GCP components ( #1018 )
...
* Rewrite resnet cmle sample with new gcp components.
* Remove legacy ResNet sample code.
* Update license header to 2019
* Remove sample_size.
* Pin the component yaml version.
2019-04-08 10:40:22 -07:00
b36c7bc233
Fixed bug in ROC component definition ( #1093 )
...
* Fixed bug in ROC component definition
* Fixed the XGBoost sample
* Fixed the ROC op usage in XGBoost sample
2019-04-05 22:06:27 -07:00
e7fe738e61
Rolling back TFMA image ( #1095 )
...
From a277f87ea12c2445df83
2019-04-05 20:12:59 -07:00
4a195b130c
Fixed bug in TFMA component code ( #1094 )
2019-04-05 18:02:29 -07:00
6800c3bae0
Added "Target lambda" parameter to "Confusion matrix" component ( #1091 )
2019-04-05 16:16:29 -07:00
f344301638
Release component image version a277f87ea1 ( #1082 )
...
* Re-run script after merge
* Run script again.
2019-04-04 16:11:20 -07:00
4afa1fc88c
Improved the IBM - Create kubernetes secret component ( #1027 )
...
Fixed the following issues:
* Checking that every command finishes successfully (non-zero return code)
* Fixed the path for the secret_name output (the program was producing variable paths different from the path in component.yaml)
2019-04-04 15:21:19 -07:00
8e2ea3902c
Improve doc for gcp components. ( #1049 )
...
* Improve doc for bigquery component.
* Improve doc for mlengine train component.
* Improving deploy component documentataion.
* Refactor notebooks
* Update notebooks with more descriptions.
* Generate README changes.
* Update dataproc docs
* Remove model parameter
* Add release script and fix doc bugs.
* Fix PR comments
* FIx PR comment
2019-04-04 13:17:21 -07:00
6b402f879a
Added component definition files for our components ( #539 )
...
* Added component definitions to our components
Added the Kubeflow training sample pipeline that uses components
* Added the definition for "TFX - Data Validation"
* Added the definition for the "TFX - Analyze model" component
* Fixed bug in "ROC curve"
* Updated "Predict using TF on Dataflow"
* Updated "TFX - Data Validation"
* Updated the component definitions.
* Updated the pipeline to make the lines shorter and explicitly name the function parameters
* Changed the GCSPath type casing
* Added the definition for the "Kubeflow - Serve TF model" component
* Added the definition for the "Kubeflow - Launch StudyJob" component
* Removed all properties from GCPPath
This will confuse our users and make type checking worse, but Hongye and Ajay requested that.
`s/type: (\{GCSPath:.*?}})(.*)/type: GCPPath$2 # type: $1/g`
* Removed the usage of the ComponentStore
Now the samples are invalid until they're merged to master, but Hongye asked for that.
2019-04-04 12:27:19 -07:00
a277f87ea1
metadata tag for watson components ( #1072 )
2019-04-02 19:49:41 -07:00
7c0558e5d1
metadata for open source components ( #1071 )
2019-04-02 19:05:41 -07:00
443feb7ed9
Fixed some typos ( #1038 )
...
* Fixed some typos
Fixed typos / spelling errors
Added a link to Cloud ML Engine runtime versions
Made some formatting (of required / optional) more consistent
* Mirrors edits into component.yaml
2019-04-02 18:17:44 -07:00
68beebf63d
Test loading all component.yaml definitions ( #1045 )
2019-04-02 12:25:18 -07:00
add2d66d8b
Release 03/28/2019 ( #1063 )
2019-03-28 23:29:09 -07:00
211f53e86d
Allow passing comma-delimited column names ( #1044 )
2019-03-28 16:48:12 -07:00
4b56e7b425
add licenses for katib-launcher ( #1056 )
2019-03-27 21:52:42 -07:00
40295135c7
upgrade ks in deployer to fix one block issue ( #1005 )
2019-03-26 22:14:47 -07:00
a26aee35a6
Keras - Train classifier: Improved the README.md ( #934 )
2019-03-25 19:31:16 -07:00
e6536a29aa
Pin the component.yaml version in all the samples. ( #1037 )
...
* Pin the component.yaml version in all the samples.
* Remove output cells from the readme and notebook
2019-03-25 14:14:19 -07:00
d2f5cc92a4
Release components to include fix #1028 . ( #1032 )
2019-03-25 11:08:18 -07:00
034e49f4f2
update ( #1021 )
2019-03-22 10:18:52 -07:00
dda1eb790e
Updating the description for model_uid param ( #1002 )
2019-03-20 14:24:59 -07:00
0ab9e6bffb
Fix dataflow and mlengine samples ( #1006 )
2019-03-20 13:26:58 -07:00
0fa221aba6
Add component.yaml to every ibm-components ( #984 )
...
* add component.yaml to every ibm-components
* replace colon conflicts with dashes
* update ffdl serve component.yaml description
2019-03-19 20:18:24 -07:00
7e45dda40b
Update component.yaml for dataflow and cmle with recent changes. ( #987 )
...
* Update component.yaml for dataflow and cmle with recent changes.
* Add type information in the GCP component yaml files
* Fix typo in component yaml
2019-03-19 18:56:26 -07:00
cc3f2149e1
Watson Openscale Components to manage models ( #950 )
...
* openscale componenets to manage models
* renamed component filename, updated with component.yaml (#1 )
2019-03-14 18:36:51 -07:00
70ee7482b1
Add documentation and sample notebook for dataproc components. ( #971 )
2019-03-14 17:44:58 -07:00
f7faf3eea9
Add missing main_class arg to spark job and set default values. ( #966 )
...
* Add missing main_class arg to spark job and set default values.
* Adding missing main_class parameter passing.
2019-03-13 18:36:24 -07:00
586815826a
Add dataproc component yaml files ( #956 )
...
* Add dataproc component yaml files
* Update license to 2019
* Remove unused parameter
2019-03-12 16:18:38 -07:00
cfd50fb1ef
Fixed typo: Replaced cencus with census ( #937 )
...
Fixes https://github.com/kubeflow/pipelines/issues/936
2019-03-07 14:43:43 -08:00
7243ded0bf
add tommy as the approver for the ibm pipeline ( #915 )
2019-03-07 14:28:11 -08:00
e54a8e3570
Keras/Train classifier: Changed the types to conform to the chosen type system ( #933 )
2019-03-07 13:42:12 -08:00
b94c731603
Fix bugs in bigquery component ( #921 )
...
* Update bigquery component yaml with latest interface.
* fix a typo in bigquery component.
* Add logging to bigquery component
2019-03-05 19:56:03 -08:00
ebf9b3b7c9
fix license links ( #917 )
2019-03-05 17:10:00 -08:00
b4ad648fb0
Add sample notebook and readme markdown for GCP components. ( #899 )
...
* Add sample notebook and readme markdown for GCP components.
* Add cloud ml train component notebook
* Fix cmle deploy component spec.
* Add CMLE deploy notebook.
* Add notebook for CMLE batch prediction component.
* Add notebook for dataflow launch template component.
* Apply AIHub doc template and fix review comments
* Updated the image gcr to public repo in component specs.
2019-03-05 15:06:20 -08:00
86c4d3bda7
add katib studyjob launcher ( #754 )
...
* add katib studyjob launcher
* delete tmp file
* fix link error to tf-laucher
* import studyjob client from katib project
* specify output file with a parameter
undo tf-launcher
2019-03-04 13:54:13 -08:00
788d8c6ed4
Added license for google-cloud-bigtable ( #896 )
2019-03-01 16:10:39 -08:00
77df6c2438
GCPcomponents yaml spec ( #887 )
...
* add component yaml for GCP components
* Add bigquery component yaml
* Fix typo and set default instead of optional setting.
2019-03-01 10:35:47 -08:00
926c321199
Witch to python 2.7 and install beam for the gcp image. ( #876 )
2019-02-28 11:29:18 -08:00
9511d964d5
release new components: deployer ( #849 )
...
release new component images
2019-02-26 16:40:33 -08:00
7775692adf
fix bug in the gcp oob ( #861 )
2019-02-26 12:56:59 -08:00
f175aa44be
add attrs and pyrsistent ( #857 )
2019-02-25 15:18:17 -08:00
26622b74b7
Releasing component SDK ( #842 )
...
* add container image uris support in train API
* Build GCP container
* Support releasing component SDK with docker and python package.
* Add licenses to image
2019-02-25 14:02:55 -08:00
e844f40e69
Removes unnecessary package from TFDV Dockerfile ( #856 )
2019-02-25 12:07:34 -08:00
5c9b2d651e
Added the "Keras - Train classifier" component ( #809 )
...
* Added sample component
* Replaced human-readable names with pythonic names
Some people are confused.
* Removed the tag from image name.
2019-02-25 01:31:48 -08:00
89e21dfb64
Training and Serving Pipeline leveraging WML ( #800 )
...
* initial watson pipeline
* updates
* updates
* Update creds
* updated dockerhub base files
* update watson pipeline (#2 )
Removing references to wml-base image, and recreating it with python-slim as base image
* address review comment (#3 )
* change s3 to cos (#4 )
2019-02-23 03:52:14 -08:00
d2f9cc4fa1
free tfdv dependency freeze ( #852 )
2019-02-22 18:14:01 -08:00
1045e023f3
Enhance hard code for export dir in deploy component ( #823 )
2019-02-21 10:27:02 -08:00
7a9d49a435
fix apache-beam and pandas versions in tfdv image ( #828 )
2019-02-15 09:30:32 -08:00
fdfd25facf
common build image script ( #815 )
2019-02-13 10:37:19 -08:00
7c633a97d7
Sample updates for release v0.1.10 ( #816 )
2019-02-12 15:24:15 -08:00
c565b42e29
add pyarrow in license csv. ( #810 )
2019-02-11 16:39:52 -08:00
0062f2ca54
add trainer license to unblock release ( #806 )
2019-02-11 13:49:18 -08:00
b5e529fb33
placeholder for ibm sample pipelines and corresponding components ( #731 )
...
* ibm sample components and pipeline
* License headers and remocing zip files
* License headers and removing zip files
* removing ffdl binary
2019-02-05 23:46:12 -08:00
be58c086f2
Modificaitons done by the execution of the release script. ( #777 )
2019-02-05 19:06:43 -08:00
a8164a07ed
Enhance deploy.sh to support local storage ( #755 )
2019-02-04 12:01:25 -08:00
028685edc1
Releasing components fixing xgboost ( #747 )
...
* try remove the deployer cloud build
* add the deployer back
* releasing the components containing the xgboost fix
2019-01-28 17:02:24 -08:00
97645e7e43
fix dataproc cluster version ( #735 )
2019-01-25 16:07:30 -08:00
4267a6307b
create nonexistent directory for roc and confusion_matrix component ( #720 )
2019-01-21 23:08:06 -08:00
a0d4be3851
component release script ( #694 )
2019-01-16 17:30:49 -08:00
2ed60100d1
add credential for tf-job ( #689 )
2019-01-15 18:59:34 -08:00
d9a1313b88
foo ( #655 )
2019-01-09 13:48:03 -08:00
a8b6e3722b
Add resnet-cmle sample back. Update all component images. ( #632 )
...
* Add resnet-cmle sample back. Update all component images.
* Remove a binary file.
2019-01-07 12:11:11 -08:00
1c689ed70d
First step to bring back CMLE sample. ( #599 )
...
* First step to bring back CMLE sample.
Set GCP credential for gcloud. Do not export TPU Model because a) Serving does not need TPU. b) Workaround a CMLE 1.9 issue.
* Follow up on CR comments.
* Add comments.
2019-01-02 17:58:44 -08:00
99227e858a
Initial version of BigQuery query execution component. ( #601 )
...
* Initial version of BigQuery query execution component.
* Remove unnecesary setup.py and build-essential installation from Dockerfile.
* Make table_id and dataset_id required arguments.
2018-12-28 19:54:14 -08:00
ae3329df95
Add a gpu sample ( #575 )
...
* Add gpu sample
* Add gpu image to release script and update image version to commit hash
2018-12-20 23:03:23 -08:00
379d4d7fcf
Add script for component image release ( #571 )
...
* add script
* update script
* fix
* Update .release.cloudbuild.yaml
* comments
* update
2018-12-20 15:28:50 -08:00
f3de359883
Support GPU image for dnntrainer component ( #530 )
...
* add gpu support for dnntrainer
* Add args for build gpu image for dnntrainer and update build scripts
* add unsaved changes
* make base image tag and local image name configurable
* fix license link
2018-12-18 17:57:02 -08:00
IronPan
Ning
Tommy Li
Jiaxin Shan
Yaron Haviv
Alexey Volkov
cheyang
Animesh Singh
hongye-sun
Alexey Volkov
Ning
Peng Li
Hougang Liu
Jin Chi He
Ajay Gopinathan
Crystal Gomes
Pascal Vicaire
Riley Bauer
Yasser Elsayed
qimingj
Cristiano Breuel