Commit Graph

43 Commits

Author SHA1 Message Date
Ning 727c48c690 update the image in the samples to use the new component images (#1267)
* update the image in the samples to use the new component images

* replace the image tag in the yaml
2019-05-02 19:46:17 -07:00
Alexey Volkov e2492896aa Testing/Sample - Made checking confusion matrix data more robust (#1196)
* Testing/Sample - Made checking confusion matrix data more robust
The sample tests no longer depend on particular  file names inside the archive. Now they only depend on the artifact name.

* Fixed json loading on python 3.5

`json.load` only supports reading from binary files in python 3.6+. https://docs.python.org/3/library/json.html#json.load
2019-04-25 15:46:27 -07:00
Ning 71325c3316 new kubernetes packages contain breaking change, thus fixing the version in the sample test image (#1159)
* new kubernetes packages contain breaking change, thus fixing the version

* also fixing the kubernetes version in the python sdk dependency

* fix bug
2019-04-14 21:36:00 -07:00
Ning 06e544ba8c add type checking sample to sample tests (#1129)
* add type checking sample to sample tests

* Dadd the test script exit code to the sample test result; update the check_notebook_result script to not validate the pipeline runs when experiment arg is not provided

* fix typo
2019-04-11 21:40:45 -07:00
Alexey Volkov c9382474d6 Fixed Kubeflow sample test (#1096)
* Fixed Kubeflow sample test

* Fixed the artifact-finding logic in `get_artifact_in_minio`.
It was just taking the first artifact before.
Now it properly searches the artifact by name.
2019-04-06 01:00:27 -07:00
Ning 1d617b50bf Add a recursion sample (#1016)
* add a While in the ops group

* deepcopy the while conditions when entering and exiting

* add while condition resolution in the compiler

* define graph component decorator

* remove while loop related codes

* fixes

* remove while loop related code

* fix bugs

* generate a unique ops group name and being able to retrieve by name

* resolve the opsgroups inputs and dependencies based on the pipelineparam in the condition

* add a recursive ops_groups

* fix bugs of the recursive opsgroup template name

* resolve the recursive template name and arguments

* add validity checks

* add more comments

* add usage comment in graph_component

* add a sample

* add unit test for the graph opsgraph

* refactor the opsgroup

* add unit test for the graph_component decorator

* exposing graph_component decorator

* add recursive compiler unit tests

* add the sample test

* fix the bug of opsgroup name
adjust the graph_component usage example
fix index bugs
use with statement in the graph_component instead of directly calling
the enter/exit functions

* add a todo to combine the graph_component and component decorators

* fix some merging bug

* fix typo

* add more comments in the sample

* update comments
2019-03-27 20:44:43 -07:00
Ning 554731e478 dsl generate zip file (#855)
* dsl generate zip file

* minor fix

* fix zip read in the unit test

* update sample tests

* dsl compiler generates pipeline based on the input name suffix

* add unit tests for different output format

* update the sdk client to support tar zip and yaml

* fix typo

* fix file write
2019-03-26 15:14:50 -07:00
Ning b3dee0543a sample test image build failure (#871)
* sample test image build failure

* fix the base image tag to avoid future breaks
2019-02-27 11:16:51 -08:00
Ning a6763b9599 component build support for both python2 and python3 (#730)
* component build support for both python2 and python3

* add sample test

* remove the annotations for python2 component build

* add pathlib for python2 component build

* fix component build unit test

* fix bug in the dockerfile generator

* remove exist_ok in path.mkdir to make python2 compatible

* adjust unit test

* remove pathlib dependency for python2 component build

* remove the pathlib codes in python3 component build, but use python2 code instead; add a todo to create a new sample
2019-02-25 12:56:19 -08:00
hongye-sun 749d0aab9f Update swagger codegen version (#839) 2019-02-21 12:21:38 -08:00
hongye-sun 969bb4ed2c Revert "Add gpu pool to test deployment and enable gpu in sample test (#696)" (#778)
This reverts commit 72a7de9d47.
2019-02-04 21:59:19 -08:00
hongye-sun 72a7de9d47 Add gpu pool to test deployment and enable gpu in sample test (#696)
* add gpu pool to test deployment and enable gpu in sample test

* enable clean up
2019-01-25 09:44:08 -08:00
qimingj 4a043c1823 Add CMLE sample test script. (#724)
The test is not added to the list to run automatically yet since it takes about 25 min.
2019-01-22 20:44:34 -08:00
Ning f86fb2a677 output argo log in case of exception throw (#635) 2019-01-07 10:43:30 -08:00
Ning ea72316ac4 fix deploy model name conflict in case of concurrent notebook sample test (#636)
* fix deploy model name conflict in case of concurrent notebook sample test

* minor fix
2019-01-05 09:15:18 -08:00
qimingj 410f9b979f Update sample notebook to clean up deployed models. (#622)
* Update sample notebook to clean up deployed models.

Update SDK client to return correct links in local Jupyter with user's own proxy connection.

* Fix sample tests.
2019-01-04 13:07:30 -08:00
Ning 5abc1a4f59 Add sample test without image build (#578)
* add another sample test to test the current sample codes instead of using newly built component images

* rename sample test yamls

* use the v2 name

* bash bug

* tf-training bug fix

* output argo log in case of exceptions for tf-training sample

* disable gpu
2019-01-03 15:17:51 -08:00
Ning 2118baf752 Fix sample test failure (#548)
* enable debug mode

* add gcp secret to tfdv; add tensorflow installation in lightweight sample test

* disable debug mode

* update the args in check_notebook_results

* change tfdv output schema

* typo fix

* update roc image in tfx sample test

* update confusionmatrix image in the tfx sample test
2018-12-19 08:55:29 -08:00
qimingj 6b315d4693 Change "namespace" param to "host" in kfp client so we can use Jupyter outside cluster. (#529)
* Change "namespace" param to "host" in kfp client so we can use Jupyter outside cluster.

* Fix tests.
2018-12-12 23:36:18 -08:00
Ning 886e113f78 Improve get_experiment and list_runs in the python sdk (#508)
* add get_experiment_id and list_runs_by_experiment

* offer only one get_experiment function

* return experiment body instead of id

* simply codes

* simply code 2

* remove experiment_id check in the while loop

* minor bug
2018-12-11 22:02:24 -08:00
nealgao ad1950b8e6 Add notebook sample test: tfx sample (#470)
* add notebook sample tests for tfx

* parameterize component image tag

* parameterize base and target image tags

* install tensorflow package for the notebook tfx sample test

* bug fixes

* start debug mode

* fix bugs

* add namespace arg to check_notebook_results, copy test results to gcs, fix minor bugs
add CMLE model deletion

* install the correct KFP version in the notebook; parameterize deployer model name and version

* fix CMLE model name bug

* add notebook sample test in v2

* add gcp sa in notebook tfx sample and shutdown debug mode

* import kfp.gcp
2018-12-07 10:49:30 -08:00
nealgao 548a2d6fe5 Failed the sample tests when it should (#417)
* failed the sample tests when it should

* print log before failure
2018-11-29 23:53:33 -08:00
IronPan 9b77d4a8a6 Switching test to kubeflow deployment (#351)
* test

* fix

* fix

* fix

* fix

* fix

* update

* cleanup

* fix

* coopy test

* chmod

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* update

* fix

* fix

* fix

* fix

* fix

* fix

* fix sample test

* fix

* fix

* merge

* update image builder image

* update script

* mount permission
2018-11-28 21:36:12 -08:00
Alexey Volkov bcbb2c51b9 Tests - Fixed many test issues (external PRs, code de-sync) by getting rid of git clone (#315)
* Tests - Getting rid of git clone in */run_test.sh
run_test.sh scripts no longer pull the repo code, because the code is now correctly baked in during the image build. This saves ~11 pulls per commit
Backend unit test image is now build as part of the test suite

* Added target-image-prefix parameter to simplify test configuration

* Build all images from source code prepared by Prow. Got rid of git pulls
All images are now built from archived version of code the source code prepared by Prow.
This saves 25 more pulls and improves test reliability.
The archived source code location is passed through image-build-context-gcs-uri parameter.

* Addressed the PR feedback.
2018-11-26 13:56:49 -08:00
nealgao 2c6a123595 reenable basic sample tests, serially after frontend integ tests(triggering 5 times, successful always) (#298)
* reenable basic sample tests, serially after frontend integ tests

* fix namespace bug
2018-11-19 13:18:29 -08:00
nealgao b39435765d configurable namespaces in sample test and python SDK (#306)
* configurable namespaces in sample test and python SDK

* update run_test.sh

* fix a typo
2018-11-17 00:31:05 -08:00
Alexey Volkov 1eb0adf616 Tests - Simplified test/sample-test/run_tests.sh a bit (#230) 2018-11-14 11:39:50 -08:00
Alexey Volkov b190e729d2 Tests - Testing master+branch instead of just branch (#165)
* Tests - Testing master+branch instead of just branch

* Updated the image-builder image

* Fixed "fatal: unable to auto-detect email address"

* Updated the image builder images

* Updated the backend-unit-test image
2018-11-09 14:22:41 -08:00
nealgao e61735c3af sample tests using the dataset in the corresponding test project (#152) 2018-11-08 19:28:44 -08:00
Yang Pan a855ffb68f activate public prow service account (#153)
* activate service account

* Update presubmit-tests-gce-minikube.sh

* Update presubmit-tests.gke.sh

* Update presubmit-tests.sh

* update git clone

* update image

* update image

* Removed the SSH key copying

* update git clone

* update image

* update
2018-11-08 16:00:30 -08:00
nealgao 81341d3aa6 sample test fix: software version error (#146)
* typo

* start debug mode

* package version failure fix

* disable debug mode
2018-11-07 18:10:42 -08:00
nealgao a540cf77c6 Fix flaky basic sample test (#134)
* fix typo

* typo fix

* start debug mode

* fix typos

* make the timeout configurable

* disable debug mode

* update variable name

* update timeout to timeout_seconds
2018-11-07 14:50:14 -08:00
nealgao 8427e30c7f Add basic sample tests (#79)
* add sequential sample test

* add condition basic sample

* reuse script

* add all the other basic tests

* update sample test dockerfile to add run_basic_test file

* write test output

* typo bug
2018-11-06 22:26:28 -08:00
nealgao 3ce66aa226 add xgboost: migrate from the old repo (#46)
* migrate from the old repo

* fix bug: accidentally override tfma test

* add tfma test back

* add tfma back

* typo fix

* fix small typo

* if job fails, exit after logs are output
2018-11-06 17:15:49 -08:00
Alexey Volkov c7bb1ff1d2 Add Ning and Alexey to OWNERS for components, samples and sample-test (#102) 2018-11-06 14:59:05 -08:00
nealgao 077a060024 debug tfma failure (#91)
* debug tfma failure

* tft version bug

* minor fix

* comment the test validation
2018-11-06 12:52:56 -08:00
qimingj 414a5afa13
Merge pull request #27 from kubeflow/ngao/fix-tfma-deploy-bug
TFMA deployer bug fix
2018-11-05 14:28:54 -08:00
Pascal Vicaire 3df28a9700 Updating OWNERS files. Adding per-subdirectory OWNER files. 2018-11-05 14:03:33 -08:00
Ning Gao 87f8840294 fix tfx name change 2018-11-05 13:23:12 -08:00
Ning Gao 9886ab10c0 fix tfx name change 2018-11-05 13:20:04 -08:00
Ning Gao 9dcedf094f fix name bug 2018-11-05 13:11:24 -08:00
Pascal Vicaire d18f37785d Updating references to the project repository to kubeflow/pipelines. 2018-11-02 15:06:54 -07:00
Pascal Vicaire 633e2ddcc8 Initial commit of the kubeflow/pipeline project. 2018-11-02 14:02:31 -07:00