Commit Graph

414 Commits

Author SHA1 Message Date
Dharmit Dalvi 5399585b6a
feat(backend): Enable logging for KFP components (#10288) 2024-01-05 01:47:08 +00:00
Chen Sun c5658f09ec chore(release): bumped version to 2.0.5 2023-12-08 18:00:49 +00:00
Chen Sun a226de2197 chore(release): bumped version to 2.0.4 2023-12-01 00:04:16 +00:00
Chen Sun 05a4583af4
chore(manifest): Update application-crd-controller image to fix CVEs (#10199) 2023-11-16 06:06:05 +00:00
Chen Sun 58ce09e07d chore(release): bumped version to 2.0.3 2023-10-27 04:08:53 +00:00
Chen Sun efe630cf80 chore(release): bumped version to 2.0.2 2023-10-11 20:27:27 +00:00
Diana Atanasova 760c1589ed
feat(backend): enforce SA Token based auth b/w Persistence Agent and Pipeline API Server (#9957)
* Enforece SA-Toben auth b/n Persistence agent & Pipeline server for all reqs

Signed-off-by: Diana Atanasova <dianaa@vmware.com>

* Fix persistence agent license file

Signed-off-by: Diana Atanasova <dianaa@vmware.com>

---------

Signed-off-by: Diana Atanasova <dianaa@vmware.com>
2023-09-07 23:49:33 +00:00
James Liu b6be4ea79b
feat(mlmd): Introduce PostgreSQL kustomization for MLMD. (#9927) 2023-08-24 17:16:16 +00:00
James Liu eaec515812
chore(mlmd): Upgrade ML Metadata to 1.14.0. (#9856)
* chore(mlmd): Upgrade ML Metadata to 1.14.0.

* fix requirements, skip visualization.

* go mod tidy

* mlmd new enum in frontend

* mlmd license

* mlmd license again, using the failure test diff

* again
2023-08-22 17:18:09 +00:00
James Liu 0fed207e40
fix(manifests): Update persistence agent manifests in marketplace helm chart (#9908)
* fix(manifests): Update persistence agent manifests in marketplace helm chart.

* snapshot

* indent

* schema.yaml
2023-08-21 20:08:41 +00:00
Lingqing Gan b086020a24
feat(deployment): add option to deploy mysql in KFP standalone (#9855)
* add option to deploy mysql

* fix deployment errors

* remove temp code in development

* keep mysql deployment path same as before

* change the generic folder

* manifest error

* revert default cache path

* address comments
2023-08-21 20:08:35 +00:00
Chen Sun 40c759f431
chore(release): bump version to 2.0.1 on master branch (#9899) 2023-08-17 14:49:40 -07:00
Diana Atanasova cb18d00bbb
feat(backend) Enable auth between pesistence agent and pipelineAPI (ReportServer) (#9699)
* Enable auth between pesistence agent and pipelineAPI (ReportServer)

1. Add authentication and authorization logic to PipelineAPI's PeportServer &
2. Make Persistence Agent authenticate itself through Service Account Token
Volume Projection.

Signed-off-by: diana <difince@gmail.com>

* Do not use MULTIUSER on report weorkflows

Only Persistent agent can KFPipeline API "Report (Scheduled)Workflows" no matter
the namespace workflows belongs to.

Signed-off-by: diana <difince@gmail.com>

* Add unit tests

- unit tests added
- do not stop the ticker on stopCh

Signed-off-by: diana <difince@gmail.com>

---------

Signed-off-by: diana <difince@gmail.com>
2023-08-17 12:34:38 +00:00
Lingqing Gan e1f0c010f8
feat(backend): add postgres initialization (#9798)
* add postgres initialization

* remove load balancer

* go mod tidy

* update license

* license update for viewer

* (test) disable controller license check

* (test) disable persistence agence licence check

* (test) disable scheduled workflow license check

* (test) disable cacheserver license check

* fix db config location

* fix mysql support

* test

* test

* no long set host address

* address comments

* address comments and enable license check

* format

* remove extra blank line

* update licenses

* cache server license

* address comments

* centralize error message

* remove pv in postgres deployment
2023-08-10 21:51:05 +00:00
James Liu 49bfda90ca
feat(manifests): Add a postgresql deployment manifest in third-party folder (#9581) 2023-07-12 23:49:24 +00:00
Chen Sun e03e312193 chore(release): bumped version to 2.0.0 2023-06-20 16:54:58 +00:00
Linchin 923b67e38f chore(release): bumped version to 2.0.0-rc.2 2023-06-13 18:43:09 +00:00
Joe Li cb93fcc5ca chore(release): bumped version to 2.0.0-rc.1 2023-05-12 21:36:20 +00:00
Chen Sun 270257512a chore(release): bumped version to 2.0.0-beta.2 2023-05-08 09:21:48 +00:00
Chen Sun 931a68096e
chore: upgrade to Argo v3.3.10 (#9336)
* pull argo v3.3.10 upstream

* upgrade to Argo v3.3.10

* fix validateworkflow

* upgrade golang

* update golang

* fix go dep

* update go licenses file

* fix build

* fix go dep

* downgrade k8s.io/client-go to v0.24.3

* update licenses

* fix go dep

* update licenses
2023-05-08 08:38:55 +00:00
Linchin e0529fe26b chore(release): bumped version to 2.0.0-beta.1 2023-04-11 19:14:29 +00:00
gkcalat 59688a3ff7
fix(backend): Remove resource from workflow-controller-configmap. Closes #9087 (#9089) 2023-04-05 00:48:01 +00:00
gkcalat e2f8838280
chore(manifests): Add gkcalat to OWNERS in manifest/kustomize (#8946)
* Add gkcalat to OWNERS in manifest/kustomize

* Update OWNERS
2023-03-08 19:37:12 +00:00
Chen Sun cdc9f2e4fc
fix(manifests): disable mysql binlog (#8941)
* disable mysql binlog

* update snapshot
2023-03-08 18:10:13 +00:00
Mark Gee 3c3ba25b47
fix(deployment): fix custom namespace for viewer crd deployment. Fixes #7714 (#7715)
Updated the env variable from MINIO_NAMESPACE to NAMESPACE as in `backend/Dockerfile.viewercontroller`. This fixes an error that happens when you use a custom namespace instead of the default kubeflow, and the ml-pipeline-viewer-crd deployment goes into CrashLoopBackOff.
2023-02-13 21:07:24 +00:00
Ablai Akhazhanov 438b16597d chore(release): bumped version to 2.0.0-beta.0 2023-02-07 23:41:29 +00:00
Alex 280edbe159
docs[manifests]: fix and standardize pns comments (#8438) 2023-01-17 21:27:20 +00:00
Steve Larkin 3824c5b537
feat(backend): Make cache-server webhook port number configurable (#8401)
* Make cache-server webhook port number configurable

Previously the webhook listened on a fixed port, 8443, which can clash
with other services when the webhook is run on the host network in
Kubernetes, which is required when using some CNI implementations,
notably Calico on EKS [1].

Enable configuration of the webhook listen port via the program flags.

[1] https://projectcalico.docs.tigera.io/getting-started/kubernetes/managed-public-cloud/eks#install-eks-with-calico-networking

Signed-off-by: Steve Larkin <steve.larkin@gmail.com>

* fixup! Make cache-server webhook port number configurable

Signed-off-by: Steve Larkin <steve.larkin@gmail.com>

* fixup! fixup! Make cache-server webhook port number configurable

Signed-off-by: Steve Larkin <steve.larkin@gmail.com>

Signed-off-by: Steve Larkin <steve.larkin@gmail.com>
2023-01-12 12:15:32 -08:00
gkcalat 67f15b7034
chore: Add gkcalat to OWNERS of gcp_marketplace manifests (#8644)
* Add gkcalat to OWNERS of gcp_marketplace manifests

* Update OWNERS
2023-01-04 01:11:24 +00:00
Chen Sun cf3175ef4e
chore: address kubectl dry-run deprecation (#8643) 2023-01-03 23:08:24 +00:00
Chen Sun 352f1c6e52
fix(manifests): Change mysql default binlog expire to 10 days. Fixes #8552 (#8553)
* Change mysql default binlog expire to 10 days

* add comments
2022-12-09 03:51:10 +00:00
gkcalat 3fca792caa
feat(backend): fix partner_id in GCP Marketplace application (#8426)
* Fix partner_id in GCP Marketplace application

Replacing `partner_id` with the appropriate value. 
This change should be included in the next v1 and v2 releases.

* Allow multiple yamls in a document for pre-commit hook

* Rebuild snapshots

* Rebuild snapshots

* Tidy snapshots

* Tidy snapshots
2022-11-07 20:52:37 +00:00
Ablai Akhazhanov 850a5a5e27 chore(release): bumped version to 2.0.0-alpha.6 2022-10-14 23:22:02 +00:00
gkcalat 65ee01a436
fix(backend): Upgrade mysql to 8.0.26 (#8351)
* Upgrade mysql to 8.0-debian

* Downgrade to mysql-8.0.26 to fix mlmd authentication issue

* Fix cloudbuilds

* Fix gcp marketplace manifests

* Cleaning up mysql password from yaml

* Improve comments
2022-10-14 07:34:04 +00:00
juliusvonkohout 9f9f035fb6
feat(backend) add maximum_cache_staleness and default_cache_staleness (#8270)
* Update pipeline-install-config.yaml

* Update cache-deployment.yaml

* Update execution_cache_store_test.go

* Update execution_cache_store.go

* Update watcher.go

* Update mutation.go

* fix identation

* Update execution_cache_store_test.go

* Update execution_cache_store.go

* Update pipeline-install-config.yaml

Set default to infinity. stalenessToSeconds() returns -1 on empty strings.

* remove logging as requested by chensun
2022-10-04 19:19:23 +00:00
Chen Sun 73caa1a66c chore(release): bumped version to 2.0.0-alpha.5 2022-09-26 11:08:27 -07:00
Ablai Akhazhanov 58052eafc1 chore(release): bumped version to 2.0.0-alpha.4 2022-08-31 21:55:27 +00:00
gkcalat 165c98de23
chore: Update GCP Marketplace manifests (#8218)
* fix: cherry-pick changes to fix kubeflow-pipeline-mkp-test (#8210)

* fix(components): remove rsync usage in gcp container build (#7422)

* fix(components): fix legacy gcp dataproc component (#7424)

* fix(components): fix gcp component build script (#7423)

* Update build_image.sh

* Update Dockerfile

* Update build_image.sh

* Update Dockerfile

* Update build_image.sh

* fix(components): Pin pip version to fix apache-beam[gcp] installation issue (#8016)

* chore: skip building KFP sdk and python components in cloudbuild (#7276)

* skip building kfp sdk and python components

* Update .release.cloudbuild.yaml

* Upgrade argo for GCP Markerplace (#8208)

* Upgrade argo for GCP Markerplace

* Add argo snapshots

* fix: cherry-pick changes to fix kubeflow-pipeline-mkp-test (#8210)

* fix(components): remove rsync usage in gcp container build (#7422)

* fix(components): fix legacy gcp dataproc component (#7424)

* fix(components): fix gcp component build script (#7423)

* Update build_image.sh

* Update Dockerfile

* Update build_image.sh

* Update Dockerfile

* Update build_image.sh

* fix(components): Pin pip version to fix apache-beam[gcp] installation issue (#8016)

* chore: skip building KFP sdk and python components in cloudbuild (#7276)

* skip building kfp sdk and python components

* Update .release.cloudbuild.yaml

* Fix cloud build script

* Fix deployer-sa rules

Co-authored-by: Chen Sun <chensun@users.noreply.github.com>

* Fix cloud build script

* Make emissary executor default (#8214)

* Update snapshots

Co-authored-by: Chen Sun <chensun@users.noreply.github.com>
2022-08-31 01:32:54 +00:00
Diana Atanasova 1caa8e0d68
feat(backend): authorize readArtifacts and ReportMetrics endpoints (#7819)
* Authorize readArtifacts and ReportMetrics endpoints

New Verbs (reportMetrics and readArtifact) are added to ClusterRole with name: aggregate-to-kubeflow-pipelines-edit

Signed-off-by: Diana Atanasova <dianaa@vmware.com>

* Add authorization when Persistent Agent communicate with the api-server

Persistent Agent authorize itself based ot the namespace and the current user

Fixes: #7818

* Update persistence_agent.csv license file

Signed-off-by: Diana Atanasova <dianaa@vmware.com>

* Fix lexical error in persistent agent cluster role

Signed-off-by: Diana Atanasova <dianaa@vmware.com>

* Fix integration tests/Fix MULTIUSER= false usecase

Cover MULTIUSER=false usecase/Standalone pipeline installation.
In this case the namespace doesn't have `user` annotation and
there is no need to provide `kubeflow-userid` Header when making
a request against kfp-api-server

Signed-off-by: Diana Atanasova <dianaa@vmware.com>

* rebase: fix conflixt in license file

Signed-off-by: Diana Atanasova <dianaa@vmware.com>

* rebase add new line in the end of licensing file

Signed-off-by: Diana Atanasova <dianaa@vmware.com>
2022-08-04 10:24:06 +00:00
Ablai Akhazhanov f604406238 chore(release): bumped version to 2.0.0-alpha.3 2022-07-13 18:17:33 +00:00
Chen Sun 1d871fe9d2
fix: Update GCP marketplace deployer base image (#8018) 2022-07-12 22:56:02 +00:00
Chen Sun 3bbc1d9afb
fix: Update mysql image to fix CVE-2022-1292 (#8017) 2022-07-12 21:18:02 +00:00
Chen Sun 8bee2922c2
feat: Upgrade argo-workflow to v3.3.8 (#8009)
* Upgrade argo-workflow to v3.3.8

* update go liencenses files

* use golang 1.17 in build and test

* temporarily disable v2 image cloudbuild
2022-07-12 19:22:31 +00:00
Suraj Kota 784f9fac24
feat(backend): use cert-manager for cache server cert (#7843) 2022-06-08 19:54:16 +00:00
James Liu d099864bd9
fix(manifests): Update snapshot to 2.0.0-alpha.2 (#7852) 2022-06-08 16:28:16 +00:00
Alex dd98b1eaf8
feat[deployment]: add startup probe to pipeline API deployment (#7741)
Co-authored-by: moorthy156 <narayanamoorthy.mari@gmail.com>

Co-authored-by: moorthy156 <narayanamoorthy.mari@gmail.com>
2022-05-23 17:23:01 +00:00
Alex afa829a7db
docs: fix rds documentation link (#7635) 2022-05-18 08:07:26 +00:00
Lingqing Gan 38bd93d6fc chore(release): bumped version to 2.0.0-alpha.2 2022-05-05 16:10:59 -07:00
Lingqing Gan 471c7d3425 chore(release): bumped version to 2.0.0-alpha.1 2022-04-04 16:38:48 -07:00
Chen Sun 3fb1a9f8c8 chore(release): bumped version to 2.0.0-alpha.0 2022-03-16 10:08:02 -07:00