Commit Graph

15 Commits

Author SHA1 Message Date
hongye-sun d626d06ec2 Do not download results to local (#1062) 2019-03-29 23:51:08 -07:00
hongye-sun 5ccbbb9a0c Fix hard-coded model export dir name (#1015)
* FIx hard-coded model export dir name

* Prefix is not the full gcs path.
2019-03-21 14:11:38 -07:00
hongye-sun e09a8ffb0e
Improve dataflow job to use GCS to keep temp job states. (#985)
* Improve dataflow job to use GCS to keep temp job states.

* make default location code reusable.
2019-03-19 18:08:48 -07:00
hongye-sun 82c7c17ad8 Ignore not found error in delete cluster (#991) 2019-03-19 16:44:22 -07:00
hongye-sun 7aa581eba9 Fix cancellation bug for dataproc job. (#967) 2019-03-14 15:23:31 -07:00
hongye-sun 6e24762ed2 DataProc commands implementation (#862)
* dataproc implementation

* Add documentation to dataproc commands.

* Add documentation for dataproc commands.
2019-03-08 16:06:54 -08:00
hongye-sun 2b07bb1832
Add bigquery docstring and dump output path. (#885)
* Add bigquery docstring and dump output path.

* Auto create dataset if it's not exist and dump results in local files

* make dataset location configurable

* Add todo to make kfp output path configurable.

* Fix comment
2019-03-05 16:00:20 -08:00
hongye-sun 639dec2402 Support customized export TF model folder names. (#903)
* Support customized export TF model folder names.

* Fix pr comments.
2019-03-05 14:18:01 -08:00
hongye-sun b00081d01f Dump job id and change output to /tmp/kfp/output (#878)
* Dump job id  and change output to /tmp/kfp/output

* fix test error

* FIx bug in validate_only mode
2019-03-01 18:40:30 -08:00
hongye-sun 17bb778ef9 Refactor MLEngine code and add deploy and set_default commands (#864)
* Refactor MLEngine code and add deploy and set_default commands

* Rename to short name to avoid confusion.

* Add comment to version names

* Rename short name to id
2019-02-28 16:46:06 -08:00
hongye-sun a791197baa Fix bugs in dataflow pipeline (#870) 2019-02-27 21:32:49 -08:00
hongye-sun ea26a574bf Dataflow SDK to support launch beam python code or template (#833)
* initial files for dataflow commands

* dataflow commands

* add launch python command

* Support launch_python command

* Use display API and default outputs to /tmp/output folder
2019-02-26 16:18:43 -08:00
hongye-sun c4baa8a0c6 bigquery command (#836) 2019-02-22 15:43:48 -08:00
hongye-sun e282bd38b2
add container image uris support in train API (#837) 2019-02-20 17:32:11 -08:00
hongye-sun bd1e770a5b MLEngine Commands Implementation (#773)
* MLEngine API commands

* Add job_id_prefix to create_job command

* add create training job command

* add display API

* Refactor CMLE APIs

* Fix PR comments.

* Add warnings before overwrite an existing file.
2019-02-16 12:47:44 -08:00