Commit Graph

3 Commits

Author SHA1 Message Date
Jeremy Lewi 8bc13cc6fe
Fix xgboost_synthetic notebook for Kubeflow 1.1. (#811)
* Fairing needs to be upgraded to fairing 1.0.1

Related to kubeflow/gcp-blueprints#78
2020-07-11 10:59:21 -07:00
Jeremy Lewi 712c29a18e Fix issues with the xgboost_synthetic example and deploying the model. (#682)
* Fix issues with the xgboost_synthetic example and deploying the model.

* install newer version of fairing
* modify preprocessor to use custom dockerfile
* use newer 0.7 base image.
* Fix endpoint.

Related to:

kubeflow/examples#673 model doesn't deploy its crash looping
Related to kubeflow/examples#655 update example to work with 0.7

* Add some comments to the notebook.
2019-11-25 14:55:10 -08:00
Jeremy Lewi 7e28cd6b23 Update xgboost_synthetic test infra; preliminary updates to work with 0.7.0 (#666)
* Update xgboost_synthetic test infra to use pytest and pyfunc.

* Related to #655 update xgboost_synthetic to use workload identity

* Related to to #665 no signal about xgboost_synthetic

* We need to update the xgboost_synthetic example to work with 0.7.0;
  e.g. workload identity

* This PR focuses on updating the test infra and some preliminary
  updates the notebook

* More fixes to the test and the notebook are probably needed in order
  to get it to actually pass

* Update job spec for 0.7; remove the secret and set the default service
  account.

  * This is to make it work with workload identity

* Instead of using kustomize to define the job to run the notebook we can just modify the YAML spec using python.
* Use the python API for K8s to create the job rather than shelling out.

* Notebook should do a 0.7 compatible check for credentials

  * We don't want to assume GOOGLE_APPLICATION_CREDENTIALS is set
    because we will be using workload identity.

* Take in repos as an argument akin to what checkout_repos.sh requires

* Convert xgboost_test.py to a pytest.

  * This allows us to mark it as expected to fail so we can start to get
    signal without blocking

  * We also need to emit junit files to show up in test grid.

* Convert the jsonnet workflow for the E2E test to a python function to
  define the workflow.

  * Remove the old jsonnet workflow.

* Address comments.

* Fix issues with the notebook
* Install pip packages in user space
  * 0.7.0 images are based on TF images and they have different permissions
* Install a newer version of fairing sdk that works with workload identity

* Split pip installing dependencies out of util.py and into notebook_setup.py

  * That's because util.py could depend on the packages being installed by
    notebook_setup.py

* After pip installing the modules into user space; we need to add the local
  path for pip packages to the python otherwise we get import not found
  errors.
2019-10-24 19:53:38 -07:00