Manifests: Rename metadata gRPC server's resources to metadata-grpc-* (#3108)

* Manifests: Rename metadata gRPC server's resources to metadata-grpc-*

The metadata service deployed is a gRPC server.

Proper KF installation deploys both an HTTP server, naming the required
resources as 'metadata-deployment' and 'metadata-service', as well as a
gRPC server, naming the corresponding resources
'metadata-grpc-deployment' and 'metadata-grpc-service'.

KFP standalone installation manifests deploy solely the gRPC server, but
use naming identical to the KF's HTTP server one.
Applying them on top of an existing KF cluster breaks Metadata service.

In this PR we change the naming making it not diverge from a proper KF
installation. We also make MetadataWriter aware of that change.

Closes #2889.

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>

* Fix ConfigMaps' label

* metadata-configmap
* metadata-mysql-configmap

* README: Link to KF installation & reference KFP version

backend/metadata_writer/src/metadata_helpers.py

@@ -21,8 +21,10 @@ from ml_metadata.metadata_store import metadata_store
 
 
 def connect_to_mlmd() -> metadata_store.MetadataStore:
-    metadata_service_host = os.environ.get('METADATA_SERVICE_SERVICE_HOST', 'metadata-service')
-    metadata_service_port = int(os.environ.get('METADATA_SERVICE_SERVICE_PORT', 8080))
+    metadata_service_host = os.environ.get(
+        'METADATA_GRPC_SERVICE_SERVICE_HOST', 'metadata-grpc-service')
+    metadata_service_port = int(os.environ.get(
+        'METADATA_GRPC_SERVICE_SERVICE_PORT', 8080))
 
     mlmd_connection_config = metadata_store_pb2.MetadataStoreClientConfig(
         host=metadata_service_host,

manifests/gcp_marketplace/chart/kubeflow-pipelines/templates/metadata.yaml

@@ -4,33 +4,33 @@ metadata:
   labels:
     app: metadata
     app.kubernetes.io/name: {{ .Release.Name }}
-  name: metadata-service
+  name: metadata-grpc-service
 spec:
   ports:
     - name: md-backendapi
       port: 8080
       protocol: TCP
   selector:
-    component: metadata-server
+    component: metadata-grpc-server
     app.kubernetes.io/name: {{ .Release.Name }}
 ---
 apiVersion: apps/v1beta2
 kind: Deployment
 metadata:
   labels:
-    component: metadata-server
+    component: metadata-grpc-server
     app.kubernetes.io/name: {{ .Release.Name }}
-  name: metadata-deployment
+  name: metadata-grpc-deployment
 spec:
   replicas: 1
   selector:
     matchLabels:
-      component: metadata-server
+      component: metadata-grpc-server
       app.kubernetes.io/name: {{ .Release.Name }}
   template:
     metadata:
       labels:
-        component: metadata-server
+        component: metadata-grpc-server
         app.kubernetes.io/name: {{ .Release.Name }}
     spec:
       containers:
@@ -163,9 +163,9 @@ kind: ConfigMap
 metadata:
   name: metadata-grpc-configmap
   labels:
-    component: metadata-server
+    component: metadata-grpc-server
 data:
-  METADATA_GRPC_SERVICE_HOST: "metadata-service"
+  METADATA_GRPC_SERVICE_HOST: "metadata-grpc-service"
   METADATA_GRPC_SERVICE_PORT: "8080"
 ---
 apiVersion: apps/v1

manifests/kustomize/README.md

@@ -100,6 +100,24 @@ kubectl kustomize env/gcp | kubectl delete -f -
 kubectl kustomize base/crds | kubectl delete -f -
 ```
 
+## Upgrade
+Note - Do **NOT** follow these instructions if you are upgrading KFP in a
+[proper Kubeflow installation](https://www.kubeflow.org/docs/started/getting-started/).
+
+If you have already deployed a standalone KFP installation of version prior to
+0.2.5 and you want to upgrade it, make sure the following resources do not
+exist: `metadata-deployment`, `metadata-service`.
+```
+kubectl -n <KFP_NAMESPACE> get deployments | grep metadata-deployment
+kubectl -n <KFP_NAMESPACE> get service | grep metadata-service
+```
+
+If they exist, you can delete them by running the following commands:
+```
+kubectl -n <KFP_NAMESPACE> delete deployment metadata-deployment
+kubectl -n <KFP_NAMESPACE> delete service metadata-service
+```
+
 ## Troubleshooting
 
 ### Permission error installing Kubeflow Pipelines to a cluster

manifests/kustomize/base/metadata/kustomization.yaml

@@ -4,8 +4,8 @@ kind: Kustomization
 resources:
   - metadata-configmap.yaml
   - metadata-mysql-secret.yaml
-  - metadata-deployment.yaml
-  - metadata-service.yaml
+  - metadata-grpc-deployment.yaml
+  - metadata-grpc-service.yaml
   - metadata-envoy-deployment.yaml
   - metadata-envoy-service.yaml
   - metadata-writer-deployment.yaml

manifests/kustomize/base/metadata/metadata-configmap.yaml

@@ -26,7 +26,7 @@ kind: ConfigMap
 metadata:
   name: metadata-grpc-configmap
   labels:
-    component: metadata-server
+    component: metadata-grpc-server
 data:
-  METADATA_GRPC_SERVICE_HOST: "metadata-service"
+  METADATA_GRPC_SERVICE_HOST: "metadata-grpc-service"
   METADATA_GRPC_SERVICE_PORT: "8080"

manifests/kustomize/base/metadata/metadata-envoy-deployment.yaml

@@ -1,7 +1,7 @@
 apiVersion: apps/v1beta2
 kind: Deployment
 metadata:
-  name: metadata-envoy
+  name: metadata-envoy-deployment
   labels:
     component: metadata-envoy
 spec:

manifests/kustomize/base/metadata/metadata-envoy-service.yaml

@@ -2,7 +2,7 @@ kind: Service
 apiVersion: v1
 metadata:
   labels:
-    app: metadata
+    app: metadata-envoy
   name: metadata-envoy-service
 spec:
   selector:

manifests/kustomize/base/metadata/metadata-grpc-deployment.yaml

@@ -1,18 +1,18 @@
 apiVersion: apps/v1beta2
 kind: Deployment
 metadata:
-  name: metadata-deployment
+  name: metadata-grpc-deployment
   labels:
-    component: metadata-server
+    component: metadata-grpc-server
 spec:
   replicas: 1
   selector:
     matchLabels:
-      component: metadata-server
+      component: metadata-grpc-server
   template:
     metadata:
       labels:
-        component: metadata-server
+        component: metadata-grpc-server
     spec:
       containers:
       - name: container

manifests/kustomize/base/metadata/metadata-grpc-service.yaml

@@ -3,10 +3,10 @@ apiVersion: v1
 metadata:
   labels:
     app: metadata
-  name: metadata-service
+  name: metadata-grpc-service
 spec:
   selector:
-    component: metadata-server
+    component: metadata-grpc-server
   type: ClusterIP
   ports:
   - port: 8080