kubeflow
/
pipelines
mirror of https://github.com/kubeflow/pipelines.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
858 Commits 314 Branches 182 Tags 516 MiB
Commit Graph

858 Commits

Author SHA1 Message Date
Eterna2 91d941d6e5 [Feature] Supports parameterized S3Artifactory for Pipeline and ContainerOp in kfp package (#1064) 
* kfp can declare custom artifact location in pipeline and containerop.

* Removed default artifact location

* Minor fixes
 2019-05-14 19:48:20 -07:00
Alexey Volkov a41bd106a1 SDK - Removing unneeded uses of dsl.Pipeline (#1229) 
* SDK - Removing unneeded usages of dsl.Pipeline

* Fixed the naming-related issue
 2019-05-14 18:48:18 -07:00
Ning 1ab558a7cd
add argo install in postsubmit (#1333) 2019-05-14 17:43:02 -07:00
Eterna2 4eeeb6e224 [Frontend] minio client in API server can be configured with environment variables (#1324) 
* kfp frontend API service can configure minio client params thru env vars

* minio endpoint is composed from host and namespace to support k8s yaml

* Added kustomize patch for pipeline-ui deploy
 2019-05-14 17:38:18 -07:00
IronPan cab1ce348e
Release c3235d725e (#1331) 
* Updated component images to version c3235d725e

* Updated components to version 3b938d664d
 2019-05-14 16:34:36 -07:00
IronPan 2b8591eae4 bump kfp version in component sdk (#1329) 
/assign @gaoning777
 2019-05-14 16:16:22 -07:00
IronPan 47efd8235f Add proxy agent as optional kustomize component (#1325) 
* clean up

* add proxy

* revert ns
 2019-05-14 15:24:21 -07:00
IronPan 0a94fc7953
clean up (#1323) 2019-05-14 14:21:14 -07:00
IronPan a2fd16e94e bump kfp version in sdk (#1330) 
/assign @gaoning777
 2019-05-14 13:49:18 -07:00
Ning f8253235c0 ml-pipeline-test has a cb job (#1322) 2019-05-14 12:59:13 -07:00
IronPan 24987010ef Merge remote-tracking branch 'upstream/master' 2019-05-14 11:30:11 -07:00
IronPan 35720448da
switch the release script from staging to test (#1328) 
/assign @gaoning777
 2019-05-14 11:29:50 -07:00
Ning 13d506c08a
add patch for metric collection (#1317) 2019-05-14 11:12:32 -07:00
IronPan 324a0a0864 Merge remote-tracking branch 'upstream/master' 2019-05-14 00:00:16 -07:00
IronPan c3235d725e
kustomize kf pipeline (#1305) 
* stage minimum

* update

* working example

* move code

* remove code

* update readme

* address comments
 2019-05-13 23:56:58 -07:00
IronPan 7a7a65d071 clean up 2019-05-13 23:34:47 -07:00
Ning 9cf1fa7d90 update tf version to 1.12.1 (#1315) 2019-05-13 20:01:08 -07:00
Ning 6fcab51a44 Add dsl support for preemptible vm/gpus (#1306) 
* add dsl support for preemptible vm/gpus

* change name to nodepool to avoid confusion
 2019-05-13 19:13:09 -07:00
Ilias Katsakioris 611f6e15ed Fix typo in ContainerOp contructor's help string (#1314) 
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
 2019-05-13 18:17:10 -07:00
Alexey Volkov be4e56a247 SDK - Stopped hard-coding artifact storage configuration in the pipeline packages (#1297) 
We should follow Argo's prefferred way to configure the artifact storage: https://github.com/argoproj/argo/blob/master/ARTIFACT_REPO.md#configure-the-default-artifact-repository and use a cluster-local configMap. This way the pipelines remain clean and portable: https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml
Kubeflow deployer has already been pre-installing the configMap for several months: https://github.com/kubeflow/kubeflow/pull/2238
 2019-05-13 17:29:08 -07:00
Alexey Volkov 32875e4b2f Adding myself to test owners (#1312) 2019-05-13 15:53:09 -07:00
Alexey Volkov e9aa69e353 Testing - Clean up the Argo controller that was used to build images (#1311) 
* Testing - Clean up the Argo controller that was used to build images

* Try to not install Argo in the first place

* Added the test-runner service account

* Cleanup

* Changing the install-argo.sh script instead
 2019-05-13 14:51:09 -07:00
Riley Bauer 1ab4315d0d Shows link to Stackdriver logs if logs retrieval fails and cluster is running in GKE (#1310) 
* Shows link to Stackdriver logs if logs retrieval fails and cluster is running in GKE

* Add additional link for new k8s stackdriver resource names

* Fix spacing
 2019-05-13 12:35:10 -07:00
IronPan 22be84b13c expose namespace config for scheduled workflow (#1309) 
this config allow specifying what namespace the persistent agent listen to
 2019-05-10 14:08:12 -07:00
IronPan 27bdbda71a expose namespace config for persistent agent (#1308) 
this config allow specifying what namespace the persistent agent listen to
 2019-05-10 13:20:16 -07:00
Tommy Li 8418b316f5 Update new Watson OpenScale components and pipeline (#1287) 
* update new openscale components and pipeline

* cleanup unused library

* fix readme and naming as requested and minor cleanup

* condense Dockerfile with new base image, load PipelineModel from spark.stage

* retrigger github checks from Travis
 2019-05-10 01:09:57 -07:00
Ilias Katsakioris b675e0272b Remove cops and rops pipeline attributes (#1298) 
* Remove the separated dictionaries for ContainerOps and ResourceOps
* Fix the sanitization performed by the compiler to iterate through ops
  dict and do type-check for the special fields file_outputs and
  attribute_outputs

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
 2019-05-09 17:25:57 -07:00
Hamed ce6066136d support tolerations for ContainerOps (#1269) 
* add tolerations to ContainerOps

* add test

* add type for tolerations

* remove fix

* remove print
 2019-05-09 16:37:59 -07:00
Ilias Katsakioris c4c2d166fe Fix PipelineParam pattern bug (#1300) 
* Generate a pattern in the constructor if one is not provided
* Add compiler tests

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
 2019-05-09 15:43:58 -07:00
Ning 710fdd8887
Update quickstart sample (#1302) 
* rename quickstart sample name
* add introductions
 2019-05-09 14:43:16 -07:00
Alexey Volkov b2d9a98f1c Backend - Marking auto-added artifacts as optional (#1289) 
* Backend - Marking auto-added artifacts as optional

* Updated Argo version in the WORKSPACE file

* Updated WORKSPACE using gazelle

* Added the package that gazelle has missed

* Fixed syntax error

* Updated Argo package to v2.3.0-rc3

* Reworded the comment
 2019-05-09 14:31:59 -07:00
Alexey Volkov 0cd052b359 Release - Simplified python package building (#1277) 2019-05-08 19:06:48 -07:00
Alexey Volkov 2696d0d732 Tests/Travis - Simplified the Python SDK package installation (#1275) 2019-05-08 18:08:48 -07:00
Jiaxin Shan cb87b94507 Make confusion_matrix and roc generic (#1285) 2019-05-08 16:48:47 -07:00
Riley Bauer 22620b5b1e Add -p to mkdir in quickstart (#1299) 2019-05-08 14:29:51 -07:00
Alexey Volkov dcd5ba5ef7 SDK - Failing faster in python_op tests (#1291) 
* SDK - Failing faster in python_op tests

* Switched to `subprocess.run(full_command, check=True)`
 2019-05-08 13:10:36 -07:00
hongye-sun 51d7e980b5 changelog for v0.1.19 (#1296) 2019-05-08 09:40:39 -07:00
Yaron Haviv 442c80449d add nuclio components (to build/deploy, delete, invoke functions) (#1295) 
* add support for flexible config (via env var) for the pipline service and UI, fix broken links (pointed to API vs UI service)

* add namespace and change address to endpoint
 2019-05-08 01:58:33 -07:00
Alexey Volkov b61bef04a3 SDK - Renamed ModelBase.from_struct/to_struct to from_dict/to_dict (#1290) 2019-05-07 14:06:35 -07:00
Jiaxin Shan f5c464a87d Add AWS EMR and Athena components (#1286) 
* Add EMR and Athena components

* Update components/aws/athena/query/component.yaml

Co-Authored-By: Jeffwan <seedjeffwan@gmail.com>

* Clean up athena components
 2019-05-06 17:11:44 -07:00
Alexey Volkov 892e3082b7 SDK/Client - Supporting pipeline packages with multiple files (#1207) 
* SDK/Client - Supporting pipeline packages with multiple files

* Extracted the pipeline yaml file selection function
 2019-05-06 16:11:39 -07:00
Alexey Volkov 2a42adab79 Updating component versions in samples (#1283) 2019-05-06 15:21:38 -07:00
Jiaxin Shan 5c850b92b1 Frontend - Add support for artifacts stored in S3 (#1278) 
* Add s3 and local support for artifact viewer

* Remove local file support and use ViewerCRD instead

* Fix condition failed tests

* Keep using minio for metadata manifests
 2019-05-06 13:47:42 -07:00
Wei Yan 2b5eb35140 Allow to specify informers namespace in persistence agent (#901) 
* Allow to specify informers namespace in persistence agent

* Update variable to namespace to keep consistent with other components
 2019-05-06 12:37:38 -07:00
cheyang 36d31fe23d make pending timeout customizable (#1268) 
* make pending timeout customizable

* fix the description of arg
 2019-05-04 18:53:36 -07:00
Riley Bauer 25cb766dae Adds a toggle between one-off and recurring runs to NewRun page (#1274) 
* Allows toggling between one-off and recurring runs in the new run page

* Clean up and adds tests

* Fix integration test - account for extra field in form

* Cleanup and PR comments
 2019-05-04 11:29:37 -07:00
Ning d0429b63f9 keep the api image name consistent between the presubmit test and staging (#1279) 2019-05-03 17:49:38 -07:00
Alexey Volkov 819d91d2f1 Retaining the component url, digest or tag when loading (#1090) 2019-05-03 16:55:38 -07:00
Animesh Singh cef50553f9 spark components (#1272) 
* spark components

* changing base image

* removing multiple open file methods

* renaming source files

* renaming source files

* adding component yaml files

* removing extra bracket signs
 2019-05-03 16:09:39 -07:00
Jiaxin Shan 5374b6b2b4 Add SageMaker components and example pipeline (#1276) 
* Add SageMaker components and example pipeline

* Address review feedbacks

* Expose more training job configs

* Update components/aws/sagemaker/batch_transform/component.yaml

Update components descriptions

Co-Authored-By: Jeffwan <seedjeffwan@gmail.com>
 2019-05-03 14:55:37 -07:00