Commit Graph

7 Commits

Author SHA1 Message Date
IronPan 4a7e2c868c fix bq table dupliation (#418)
* fix bq table dupliation

* fix bq table dupliation

* update

* update image

* use index for placeholder
2018-12-10 18:50:28 -08:00
IronPan 0d2f5b6342 Clean up code search pipeline (#406)
* update pipeline to use out of box gcp credential support

* Update index_update_pipeline.py
2018-12-07 10:13:12 -08:00
IronPan 206ad8fda4 Add preprocess github data step to code search pipeline (#396)
* refactor ks

* remove unecessary params

* update ks

* address comments

* add preprocess step

* update images

* update preprocess code

* reformat

* minor fix

* reuse function embedding pipeline to preprocess

* add preprocess

* update pipeline

* propagate failed token table

* format code

* copy vocabulary

* address comments

* address comments

* update

* fix

* fix format

* Update arguments.py
2018-12-05 18:06:06 -08:00
IronPan cea0ffde0d Update the ks parameter (#394)
* refactor ks

* remove unecessary params

* update ks

* address comments
2018-12-02 22:14:11 -08:00
IronPan e8cf9c58ce add pipeline step to push to git (#387)
* add push to git

* small fixes

* work around .after()

* format
2018-12-02 09:37:21 -08:00
IronPan 7ffc50e0ee Add dataflow launcher script (#364)
* add search index creator container

* add pipeline

* update op name

* update readme

* update scripts

* typo fix

* Update Makefile

* Update Makefile

* address comments

* fix ks

* update pipeline

* restructure the images

* remove echo

* update image

* add code embedding launcher

* small fixes

* format

* format

* address comments

* add flag

* Update arguments.py

* update parameter

* revert to use --wait_until_finished. --wait_until_finish never works

* update image
2018-11-27 19:23:54 -08:00
IronPan 31390d39a0 Add update search index pipeline (#361)
* add search index creator container

* add pipeline

* update op name

* update readme

* update scripts

* typo fix

* Update Makefile

* Update Makefile

* address comments

* fix ks

* update pipeline

* restructure the images

* remove echo

* update image

* format

* format

* address comments
2018-11-27 04:43:55 -08:00