Commit Graph

17 Commits

Author SHA1 Message Date
IronPan bbfe5e09cc move pipeline runner service account to backend (#1988)
* move pipeline runner service account to backend

* revert swf change

* revert swf change

* update tests

* update tests

* Update compiler.py

revert python change for backward compatibility

* Update compiler.py

revert python change for backward compatibility
2019-08-29 16:03:14 -07:00
dushyanthsc 23993486c5 apiserver: Remove TFX output artifact recording to metadatastore (#1904) 2019-08-21 13:44:31 -07:00
Andy Wei 60fd70c093 Let backend apiserver mysql dbname configurable (#1714)
* let mysql dbname configurable

* solve conflict

* move declaration out to fix scope
2019-08-20 00:33:33 -07:00
dushyanthsc 4c5d34fe71 test/project-cleanup: Support to cleanup gke-clusters in test project (#1857)
Change to add base framework for cleaning up resources in a GCP project.
The resource specification is specified declaratively using a YAML file.

As per current requirements this change only adds cleaning up of GKE
clusters.
2019-08-19 18:57:32 -07:00
Alexey Volkov 6aa77b774a Revert "Backend - Updated the version of the ml metadata package (#1725)" (#1747)
This reverts commit 0dd1b1ce5f.
2019-08-07 03:29:22 -07:00
Alexey Volkov 0dd1b1ce5f Backend - Updated the version of the ml metadata package (#1725)
Fixes https://github.com/kubeflow/pipelines/issues/1722
2019-08-02 17:03:52 -07:00
Alexey Volkov 9bf039104e Backend - Updated Argo package from v2.3.0 RC to final version (#1428) 2019-06-04 12:55:54 -07:00
Alexey Volkov b2d9a98f1c Backend - Marking auto-added artifacts as optional (#1289)
* Backend - Marking auto-added artifacts as optional

* Updated Argo version in the WORKSPACE file

* Updated WORKSPACE using gazelle

* Added the package that gazelle has missed

* Fixed syntax error

* Updated Argo package to v2.3.0-rc3

* Reworded the comment
2019-05-09 14:31:59 -07:00
Ajay Gopinathan ba64bd3866 Record TFX output artifacts in Metadata store (#884)
* WIP: ML Metadata in KFP

* Move metadata tracking to its own package.

* Clean up

* Address review comments, update travis.yml

* Add dependencies for building in Dockerfile

* Log errors but continue to update run when metadata storing fails.

* Update workspace to get latest ml-metadata version.

* Update errors
2019-03-06 09:16:51 -08:00
Ajay Gopinathan 578e8231d0 Update all Pipelines CRD versions to v1beta1. (#681) 2019-01-17 19:35:51 -08:00
Yasser Elsayed df635af409 Support filtering on storage state (#629)
* filter on storage state

* fix case

* Update run.go

* storageState -> storage_state

* ignore generated ts client libs

* remove url import to make tslint happy
2019-01-11 11:01:01 -08:00
Ajay Gopinathan 9d69e8a113 Sanity check filtering/sorting options in list requests. (#625)
If sort by or filtering criteria is specified in conjunction with a next
page token, ensure they match up, otherwise return an error.

Also, change the errors to be InvalidInputErrors instead of standard Go
string errors to be consistent with the rest of apiserver.
2019-01-09 11:05:01 -08:00
Ajay Gopinathan 5a9e3ff14b Add IS_SUBSTRING operator for use in API resource filtering. (#645)
* Add IS_SUBSTRING operator for use in API resource filtering.

This should allow substring matches on fields like names and labels and
so on.

Also bump the version of Mastermind/squirrel so we get the new 'like'
operator for use when building the SQL query.

Additionally, I also had to fix the generate_api.sh script which had a
bug (it modified the wrong file permissions before), and add a dummy
service to generate Swagger definitions for the Filter itself (this was
a hack in the previous Makefile that I lost when we moved to Bazel).

* Add comments for DummyFilterService

* Add more comments

* change errors returned

* fix import
2019-01-08 18:16:12 -08:00
Ajay Gopinathan 5bde521148 Update WORKSPACE and BUILD files incoporating the recent viewer CRD (#639)
change.

That change was submitted in parallel with the PR that moved everything
to Bazel and so wasn't included.

Along the way, fix conflicts in imports that was using
controller-runtime library. We can import it either through
github.com/kubernetes-sigs or sigs.k8s.io/controller-runtime, but
shouldn't be using both imports, which was causing conflicts at build
time.
2019-01-07 13:56:32 -08:00
Ajay Gopinathan eea6999b5b Add the Viewer CRD controller for managing web views such as Tensorboard instances from within the Pipelines UI. (#449)
* Add initial CRD types for Viewer resource, and generate corresponding
code.

* Use controller-runtime to scaffold out a controller main

* Start adding a deployment

* Clean up and separate reconciler logic into its own package for future testing.

* Clean up with comments

* Run dep ensure

* Update auto-generate script. Only need deepcopy funcs for viewer crd types

* Cleanup previously generated but unused viewer client code

* [WIP] Adding tests

* More tests

* Completed unit tests for reconciler with logic for max viewers

* Add CRD definition, sample instance and update README.

* Fix merge conflict

* Fix readme typo for kube and add direct port-forwarding instructions.

* Add tests for when persistent volume is used with Tensorboard viewer.

Also add a sample YAML to show how to mount and use a GCE persistent
disk in the viewer CRD.

* Remove vendor directory
2019-01-04 18:22:48 -08:00
Ajay Gopinathan 163545b370 Use Bazel to build the entire backend and perform API code generation (#609)
* Use Bazel to build the entire backend.

This also uses Bazel to generate code from the API definition in the
proto files.

The Makefile is replaced with a script that uses Bazel to first generate
the code, and then copy them back into the source tree.

Most of the BUILD files were generated automatically using Gazelle.

* Fix indentation in generate_api.sh

* Clean up WORKSPACE

* Add README for building/testing backend.

Also fix the missing licenses in the generated proto files.

* Add license to files under go_http_client
2019-01-04 17:17:20 -08:00
IronPan e7af13263a switch from go dep to go module (#581)
* add vendor to gitignore

* switch to go module

* switch to go module

* switch to go module

* prune go mod

* prune go mod

* turn on go mod for test

* enable go module in docker image

* enable go module in docker image

* fix images

* debug

* debug

* debug

* update image
2018-12-21 16:28:21 -08:00