Commit Graph

277 Commits

Author SHA1 Message Date
Konstantinos Samaras-Tsakiris 5c38c96fae Fix #272 (#273)
* Fix #272

Fix #272 where the `create-pet-record-job` pod produces this error: `models/research/object_detection/data/pet_label_map.pbtxt; No such file or directory`

* Update create-pet-record-job.jsonnet
2018-10-22 14:57:24 -07:00
Konstantinos Samaras-Tsakiris 6edf7915f5 Fix #275 (#276)
Fix #275 by changing the default mount path for the training data.
2018-10-22 12:14:13 -07:00
Konstantinos Samaras-Tsakiris b0f9b4cfd0 Fix bash (#271)
Remove spaces around a bash variable declaration.
2018-10-22 12:02:04 -07:00
Svendegroote91 bc0380dda6 minor fixes for instructions (#267) 2018-10-15 10:02:17 -07:00
Jeremy Lewi 90044d24c4 Remove v1alpah1 TFJobs from the GH issue summarization example. (#264)
* We should be using v1alpha2 exclusively now.
2018-10-15 09:52:01 -07:00
Jeremy Lewi 4ea761630d Fix gh-demo.kubeflow.org and make it easy to setup. (#261)
* Fix gh-demo.kubeflow.org and make it easy to setup.

* Our public demo of the GitHub issue summarization example
  (gh-demo.kubeflow.org) is down. It was running in one of our dev
   clusters and with the the churn in dev clusters it ended up getting deleted.

* To make it more stable lets move it to project kubecon-gh-demo-1
  and create a separate cluster for running it.
  This cluster can also serve as a readily available Kubeflow cluster
  setup for giving demos.

* Create the directory demo within the github_issue_summarization example
  to contain all the required files.

* Add a makefile to make building the image work.

* The ksonnet app for the public demo was previously stored here
  https://github.com/kubeflow/testing/tree/master/deployment/ks-app

* Fix the uiservice account.

* Address comments.
2018-10-15 08:36:11 -07:00
Svendegroote91 d3e1731d7f add financial time series example (#252)
* add financial time series example

* fix ReadMe comments

* fix PyLint remarks

* clean up based on PR remarks

* Completing docstrings and fixing PR remarks
2018-10-12 08:04:07 -07:00
Jeremy Lewi adf614fc5f Add tensorboard and check in vendor for the code search example. (#255)
* Add tensorboard and check in vendor for the code search example.

* * Remove the default env; when I ran ks show I got errors but
  removing it and adding a fresh env worked. It also won't point to
  the correct cluster for users.
2018-10-04 10:18:58 -07:00
Ankush Agarwal 2064b43def Ankush Signing Out (#253) 2018-09-28 16:17:20 -07:00
Michelle Casbon 5c2d8aefc2 Remove reviewers who are already approvers (#247)
* Remove reviewers who are already approvers

Remove ScorpioCPH and zjj2wry due to inactivity (no PRs or comments on PRs).

* Add zjj2wry back on request
2018-09-24 17:25:32 -07:00
Akado2009 5329bfa59b docs updated (#240) 2018-09-24 15:07:27 -07:00
Michelle Casbon 42592fed4a Update demo script & add notebook (#248)
* Update demo script

Update demo script to include deploy script and notebook created by @drscott173
Simplify by removing unnecessary commands
Use default namespace instead of kubeflow

* Add yelp notebook readme

* Add cluster creation commands

Add instructions for highlighting changes resulting from each command
2018-09-11 11:17:02 -07:00
Inki Hwang 8e30631c54 example mnist upgrade to v1alpha2 (#246)
* example mnist upgrade to v1alpha2

* Remove cleanPodPolicy

* Fix kubeflow branch to v0.2.4
2018-09-09 13:01:21 -07:00
Michelle Casbon d878462bc5 Upgrade demo to use latest versions of kubeflow, tfjob, ksonnet, & gke (#242)
* Upgrade ks dir to 0.12.0

* Upgrade kubeflow to v0.2.0-rc.1

Use https://github.com/kubeflow/kubeflow/blob/master/scripts/upgrade_ks_app.py
to upgrade ks registry
Add t2tcpu-v1alpha2 component

* Rename t2tcpu-v1alpha2 -> t2tcpu

Rename t2tcpu -> t2tcpu-v1alpha1 and t2tcpu-v1alpha2 -> t2tcpu
Update demo_setup/README.md to reflect ks v0.12.0
Update REPO_PATH in demo_setup/kubeflow-demo-base.env
Update initialClusterVersion in k8s cluster creation script to 1.10.6-gke.2
Remove quotation marks from serving.deployHttpProxy so that it is parsed as a boolean instead of string

* Rename t2tgpu & t2ttpu

Rename t2tgpu -> t2tgpu-v1alpha1 and add t2tgpu-v1alpha2 as t2tgpu
Rename t2ttpu -> t2ttpu-v1alpha1 and add t2ttpu-v1alpha2 as t2ttpu
Resolve jsonnet parsing issues

* Upgrade kubeflow to v0.2.4

Add gke environment

* Add instructions for creating TPU clusters

* Replace hard-coded value with env var

* Update kf version to v0.2.4 in env var file

* Add non-gke requirements to t2tcpu component

Sync t2tgpu with t2tcpu
Remove non-gke statements from t2ttpu component
Add k8s v1.10.6 to minikube start command

* Fix bug with non-gke environment setup in t2t

Add service account setup and k8s secret creation instructions for serving & UI

* Single cluster with GPU & TPU

Add creation script for single cluster with access to CPU, GPU, & TPU
Update GPU driver installation to k8s-1.10

* Remove v1alpha1 components

* Update parameter values for t2t components

Increase disk size for minikube cluster creation since 0.2.4 is larger
Update gke cluster creation command

* Update TPU annotation to TF 1.9

* Update kf version to v0.2.5

Update tfJobImage version to v20180809-d2509aa
2018-09-05 05:46:33 -07:00
Katsunori Kanda 1b7df0c141 Fixed broken link in github issue summarization example (#235) 2018-08-26 18:01:31 -07:00
Michał Jastrzębski 35786ed9cb Add estimator example for github issues (#203)
* Add estimator example for github issues

This is code input for doc about writing Keras for tfjob.

There are few todos:

1. bug in dataset injection, can't raise number of steps
2. intead of adding hostpath for data, we should have quick job + pvc
for this

* pyling

* wip

* confirmed working on minikube

* pylint

* remove t2t, add documentation

* add note about storageclass

* fix link

* remove code redundancy

* adress review

* small language fix
2018-08-24 18:10:27 -07:00
Puneith Kaul 1d5ddf560b
Merge pull request #236 from kubeflow/xgboost_readme
Update README.md
2018-08-24 15:35:07 -07:00
Puneith Kaul ab61a75373
Update README.md 2018-08-24 15:34:48 -07:00
Puneith Kaul 7b7d671b87
Update README.md 2018-08-24 07:49:18 -07:00
Puneith Kaul e7996c33a2
Update README.md 2018-08-24 07:48:18 -07:00
Puneith Kaul bd07a2f84e new PR for XGBoost due to problems with history rewrite (#232)
* new PR for XGBoost due to problems with history rewrite

* Update housing.py

* Update HousingServe.py

* Update housing.py

* added bitly

* removed test function

* reorder imports

* fix spaces

* fix spaces

* fixed lint errors

* renamed to xgboost_ames_housing
2018-08-22 06:01:36 -07:00
Daniel Castellanos e6b6730650 Updated object detection training example (#228)
* Updated Dockerfile.traning to use latest tensorflow
  and tensorflow object detetion api.
* Updated tf-training-job component and added a chief
  replica spec
* Corrected some typos and updated some instructions
2018-08-20 19:32:12 -07:00
Sanyam Kapoor f9873e6ac4 Upgrade notebook commands and other relevant changes (#229)
* Replace double quotes for field values (ks convention)

* Recreate the ksonnet application from scratch

* Fix pip commands to find requirements and redo installation, fix ks param set

* Use sed replace instead of ks param set.

* Add cells to first show JobSpec and then apply

* Upgrade T2T, fix conflicting problem types

* Update docker images

* Reduce to 200k samples for vocab

* Use Jupyter notebook service account

* Add illustrative gsutil commands to show output files, specify index files glob explicitly

* List files after index creation step

* Use the model in current repository and not upstream t2t

* Update Docker images

* Expose TF Serving Rest API at 9001

* Spawn terminal from the notebooks ui, no need to go to lab
2018-08-20 16:35:07 -07:00
Michelle Casbon 0843cdad66 Add Yelp restaurant review demo files (#220)
* Add Yelp restaurant review demo files

* Add video links

* Resolve lint issues
2018-08-15 22:49:00 -07:00
Sanyam Kapoor 4e015e76a3 Cherry pick changes to PredictionDoFn (#226)
* Cherry pick changes to PredictionDoFn

* Disable lint checks for cherry picked file

* Update TODO and notebook install instructions

* Restore CUSTOM_COMMANDS todo
2018-08-15 06:21:00 -07:00
Sanyam Kapoor 18829159b0 Add a new github function docstring extended problem (#225)
* Add a new github function docstring extended problem

* Fix lint errors

* Update images
2018-08-14 15:41:47 -07:00
Sanyam Kapoor 8fce4a7799 Allow ks param set for Code Search Ksonnet Application (#224)
* Allow ks param set for t2t-code-search

* Update notebook with working directory param set

* Abstract out common variables for easy ks param set
2018-08-14 15:29:04 -07:00
Lun-Kai Hsu f3806d0bac Small fix to TF serving gpu (#221)
* Small fix to TF serving gpu

* fix

* fix

* fix
2018-08-14 14:27:35 -07:00
Sanyam Kapoor a687c51036 Add a Jupyter notebook to be used for Kubeflow codelabs (#217)
* Add a Jupyter notebook to be used for Kubeflow codelabs

* Add help command for create_function_embeddings module

* Update README to point to Jupyter Notebook

* Add prerequisites to readme

* Update README and getting started with notebook guide

* [wip]

* Update noebook with BigQuery previews

* Update notebook to automatically select the latest MODEL_VERSION
2018-08-13 21:43:26 -07:00
Ankush Agarwal a80c15b50e
Merge pull request #213 from activatedgeek/search-server-kubeflow
Update Search Index server spec
2018-08-09 14:57:49 -07:00
Sanyam Kapoor 6e9150bad6 Parametrize volumes and ports for nmslib containers 2018-08-09 10:53:23 -07:00
Sanyam Kapoor 133e054033 Refactor job and deployment specs into different functions 2018-08-09 10:53:23 -07:00
Sanyam Kapoor e34f9aca75 Build just one image with the correct tag instead of double the number 2018-08-09 10:53:23 -07:00
Sanyam Kapoor c86f306d79 Use kind Job instead of Pod 2018-08-09 10:53:23 -07:00
Sanyam Kapoor 6527aba7c1 Upgrade JS app to be served at any path prefix 2018-08-09 10:53:23 -07:00
Sanyam Kapoor 9ce23d9fc6 Working search index server 2018-08-09 10:53:23 -07:00
Sanyam Kapoor 02db0065c1 Make search index creation a one-off job 2018-08-09 10:53:23 -07:00
Sanyam Kapoor d4669467d8 Update Search Index server spec with new commands 2018-08-09 10:53:23 -07:00
Sanyam Kapoor cfdcb1292c Update Ksonnet version, Add Python2 pip (#216)
* Update Ksonnet version, Add Python2 pip

* Update ks version in README
2018-08-07 22:58:20 -07:00
Richard Liu 082561a75b Create Jupyter notebook image for codelabs (#214)
* Create Jupyter notebook image for codelabs

* Add makefile
2018-08-06 16:16:02 -07:00
Daniel Castellanos 9bda30b7d9 Fixed broken links in object detection example (#211) 2018-08-03 16:05:27 -07:00
Sanyam Kapoor f2151f66fc Merge UI and Search Server (#209)
* Use the nicer tf.gfile interface for search index creation

* Update documentation and more maintainable interface to search server

* Add ability to control number of outputs

* Serve React UI from the Flask server

* Update Dockerfile for the unified server and ui
2018-08-03 15:56:09 -07:00
Sam Shi b6a4d06f00 Batch predict example for object detection using GPU (#199)
* adding batch-predict on GPU example

* Sync with TF-serving GPU example.

* adding visualization instructions

* change the title of readme.md

* changes according to the review comments from jlewi

* Replace the links to personal project with the one in kubeflow-example project in the yaml file

* change the procedure to build images

* polish the md file

* some minor md change

* fix a broken gs link

* fix more merge errors
2018-08-03 11:57:53 -07:00
Sanyam Kapoor e9e844022e Disable Distributed Training (#207)
* Upgrade TFJob and Ksonnet app

* Container name should be tensorflow. See #563.

* Working single node training and serving on Kubeflow

* Add issue link for fixme

* Remove redundant create secrets and use Kubeflow provided secrets
2018-08-02 23:02:05 -07:00
Daniel Castellanos 091eacb4f6 Parametrize Object detection example (#192)
* Added Ksonnet prototypes to parametrize old yaml files

* Modified instructions

* Added tf-training-job component

* Removed yaml manifest files

Modified serving instructions

* Consolidate get-data and decompression jobs

* Deleted registry and prototypes

* Added components to ks-app dir
* Modified instructions

* Fixed references to user guide page

Improved instructions

* General improvements to components and instructions

* Removed obj-detection.libsonnet file
* used specific params in export-graph and create-tf-record
  instead of list params like 'args' and 'command'
* Improved instructions and removed references to yaml files
2018-08-02 18:44:26 -07:00
Sanyam Kapoor fd2e750990 Fix T2T memory problem (#205)
* Update T2T problems to workaround memory limitations

* Add max_samples_for_vocab to prevent memory overflow

* Fix a base URL to download data from, sweet spot for max samples

* Convert class variables to class properties

* Fix lint errors

* Use Python2/3 compatible code for StringIO

* Fix lint errors

* Fix source data files format

* Move to Text2TextProblem instead of TranslateProblem

* Update details for num_shards and T2T problem dataset
2018-08-01 13:37:41 -07:00
Sanyam Kapoor 767c90ff20 Refactor dataflow pipelines (#197)
* Update to a new dataflow package

* [WIP] updating docstrings, fixing redundancies

* Limit the scope of Github Transform pipeline, make everything unicode

* Add ability to start github pipelines from transformed bigquery dataset

* Upgrade batch prediction pipeline to be modular

* Fix lint errors

* Add write disposition to BigQuery transform

* Update documentation format

* Nicer names for modules

* Add unicode encoding to parsed function docstring tuples

* Use Apache Beam options parser to expose all CLI arguments
2018-07-27 06:26:56 -07:00
Lun-Kai Hsu 1746820f8f Example of TF Serving with GPU (#154)
* initial

* wip

* working now

* fix

* fix lint

* fix lint

* fix lint

* review

* move

* fix

* addressing comment

* lint

* fix
2018-07-24 21:44:55 -07:00
Lun-Kai Hsu f340a4c2c7 fix typo in OWNER (#193) 2018-07-24 08:04:55 -07:00
Sanyam Kapoor 994fdf82c0 Integrate nmslib (#194)
* Integrate NMSLib server with new data file

* Integrate UI with query URL of search server
2018-07-23 17:17:24 -07:00