linkerd
/
linkerd2
mirror of https://github.com/linkerd/linkerd2.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix multicluster StatefulSet test (#6577) 

The multicluster StatefulSet test seemed to be a bit flakey. In the actual PR, the check passed, but as soon as it was merged it failed. I had a quick look and noticed that the flakiness in the test came from the logic that mirrors a namespace. Specifically, when we have a headless service, we create it and mirror its namespace here: https://github.com/linkerd/linkerd2/blob/main/multicluster/service-mirror/cluster_watcher.go#L1120

But, every time we process a new service, we do the same thing:
https://github.com/linkerd/linkerd2/blob/main/multicluster/service-mirror/cluster_watcher.go#L463

The tests were failing when a headless service was being created with the following error:
```
time="2021-07-29T20:58:03Z" level=info msg="Err Inner errors:\n\tnamespaces \"ns2\" already exists: " cluster=remote
```

It seems that the namespace was being created twice in a very short span leading to a race condition (hence why it would pass randomly, I assume). An easy way to fix this is to bail out early in `handleRemoteServiceCreated()` if we have a headless service, before we create the namespace.

Signed-off-by: Matei David <matei@buoyant.io>
This commit is contained in:
Matei David 2021-07-29 23:12:25 +01:00 committed by GitHub
parent 8988d400c3
commit adf1d33476
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
multicluster/service-mirror/cluster_watcher.go
View File

@ -457,6 +457,10 @@ func remapRemoteServicePorts(ports []corev1.ServicePort) []corev1.ServicePort {
func (rcsw *RemoteClusterServiceWatcher) handleRemoteServiceCreated(ctx context.Context, ev *RemoteServiceCreated) error {
remoteService := ev.service.DeepCopy()
if rcsw.headlessServicesEnabled && remoteService.Spec.ClusterIP == corev1.ClusterIPNone {
return nil
}
serviceInfo := fmt.Sprintf("%s/%s", remoteService.Namespace, remoteService.Name)
localServiceName := rcsw.mirroredResourceName(remoteService.Name)
@ -476,10 +480,6 @@ func (rcsw *RemoteClusterServiceWatcher) handleRemoteServiceCreated(ctx context.
},
}
if rcsw.headlessServicesEnabled && remoteService.Spec.ClusterIP == corev1.ClusterIPNone {
return nil
}
rcsw.log.Infof("Creating a new service mirror for %s", serviceInfo)
if _, err := rcsw.localAPIClient.Client.CoreV1().Services(remoteService.Namespace).Create(ctx, serviceToCreate, metav1.CreateOptions{}); err != nil {
if !kerrors.IsAlreadyExists(err) {
@ -1080,6 +1080,7 @@ func (rcsw *RemoteClusterServiceWatcher) createOrUpdateHeadlessEndpoints(ctx con
}
}
}
if len(errors) > 0 {
return RetryableError{errors}
}