Upgraded to hazelcast-bootstraper 0.2 and pires/hazelcast-k8s:0.2.
We now use v1beta3 API. Fixes #7731
This commit is contained in:
Paulo Pires
parent
444a80efe6
commit
0043eb90f1
|
|
@ -7,7 +7,7 @@ Any topology changes are communicated and handled by Hazelcast nodes themselves.
|
|||
This document also attempts to describe the core components of Kubernetes, _Pods_, _Services_ and _Replication Controllers_.
|
||||
|
||||
### Prerequisites
|
||||
This example assumes that you have a Kubernetes cluster installed and running, and that you have installed the ```kubectl``` command line tool somewhere in your path. Please see the [getting started](https://github.com/GoogleCloudPlatform/kubernetes/tree/master/docs/getting-started-guides) for installation instructions for your platform.
|
||||
This example assumes that you have a Kubernetes cluster installed and running, and that you have installed the `kubectl` command line tool somewhere in your path. Please see the [getting started](https://github.com/GoogleCloudPlatform/kubernetes/tree/master/docs/getting-started-guides) for installation instructions for your platform.
|
||||
|
||||
### A note for the impatient
|
||||
This is a somewhat long tutorial. If you want to jump straight to the "do it now" commands, please see the [tl; dr](#tl-dr) at the end.
|
||||
|
|
@ -15,52 +15,14 @@ This is a somewhat long tutorial. If you want to jump straight to the "do it no
|
|||
### Sources
|
||||
|
||||
Source is freely available at:
|
||||
* Docker image - https://github.com/pires/hazelcast-kubernetes
|
||||
* Hazelcast Discovery - https://github.com/pires/hazelcast-kubernetes-bootstrapper
|
||||
* Dockerfile - https://github.com/pires/hazelcast-kubernetes
|
||||
* Docker Trusted Build - https://registry.hub.docker.com/u/pires/hazelcast-k8s
|
||||
|
||||
### Simple Single Pod Hazelcast Node
|
||||
In Kubernetes, the atomic unit of an application is a [_Pod_](http://docs.k8s.io/pods.md). A Pod is one or more containers that _must_ be scheduled onto the same host. All containers in a pod share a network namespace, and may optionally share mounted volumes. In this simple case, we define a single container running Hazelcast for our pod:
|
||||
In Kubernetes, the atomic unit of an application is a [_Pod_](http://docs.k8s.io/pods.md). A Pod is one or more containers that _must_ be scheduled onto the same host. All containers in a pod share a network namespace, and may optionally share mounted volumes.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1beta3
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
name: hazelcast
|
||||
name: hazelcast
|
||||
spec:
|
||||
containers:
|
||||
- image: pires/hazelcast-k8s
|
||||
name: hazelcast
|
||||
ports:
|
||||
- containerPort: 5701
|
||||
name: hazelcast
|
||||
protocol: TCP
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
```
|
||||
|
||||
There are a few things to note in this description. First is that we are running the ```pires/hazelcast-k8s``` image. This is a standard Ubuntu 14.04 installation with Java 8. However it also adds a custom [```application ```](https://github.com/pires/hazelcast-kubernetes-bootstrapper) that finds any Hazelcast nodes in the cluster and bootstraps an Hazelcast instance. The ```HazelcastDiscoveryController``` discovers the Kubernetes API Server using the built in Kubernetes discovery service, and then uses the Kubernetes API to find new nodes (more on this later).
|
||||
|
||||
You may also note that we tell Kubernetes that the container exposes the ```hazelcast``` port. Finally, we tell the cluster manager that we need 1 cpu core.
|
||||
|
||||
Given this configuration, we can create the pod as follows:
|
||||
|
||||
```sh
|
||||
$ kubectl create -f hazelcast.yaml
|
||||
```
|
||||
|
||||
After a few moments, you should be able to see the pod running:
|
||||
|
||||
```sh
|
||||
$ kubectl get pods hazelcast
|
||||
|
||||
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE
|
||||
hazelcast 10.245.2.68 e2e-test-minion-vj7k/104.197.8.214 name=hazelcast Running 11 seconds
|
||||
hazelcast pires/hazelcast-k8s Running 1 seconds
|
||||
```
|
||||
In this case, we shall not run a single Hazelcast pod, because the discovery mechanism now relies on a service definition.
|
||||
|
||||
|
||||
### Adding a Hazelcast Service
|
||||
|
|
@ -82,14 +44,60 @@ spec:
|
|||
name: hazelcast
|
||||
```
|
||||
|
||||
The important thing to note here is the ```selector```. It is a query over labels, that identifies the set of _Pods_ contained by the _Service_. In this case the selector is ```name: hazelcast```. If you look back at the Pod specification above, you'll see that the pod has the corresponding label, so it will be selected for membership in this Service.
|
||||
The important thing to note here is the `selector`. It is a query over labels, that identifies the set of _Pods_ contained by the _Service_. In this case the selector is `name: hazelcast`. If you look at the Replication Controller specification below, you'll see that the pod has the corresponding label, so it will be selected for membership in this Service.
|
||||
|
||||
Create this service as follows:
|
||||
```sh
|
||||
$ kubectl create -f hazelcast-service.yaml
|
||||
```
|
||||
|
||||
Once the service is created, you can query it's endpoints:
|
||||
### Adding replicated nodes
|
||||
The real power of Kubernetes and Hazelcast lies in easily building a replicated, resizable Hazelcast cluster.
|
||||
|
||||
In Kubernetes a _Replication Controller_ is responsible for replicating sets of identical pods. Like a _Service_ it has a selector query which identifies the members of it's set. Unlike a _Service_ it also has a desired number of replicas, and it will create or delete _Pods_ to ensure that the number of _Pods_ matches up with it's desired state.
|
||||
|
||||
Replication Controllers will "adopt" existing pods that match their selector query, so let's create a Replication Controller with a single replica to adopt our existing Hazelcast Pod.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1beta3
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
labels:
|
||||
name: hazelcast
|
||||
name: hazelcast
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
name: hazelcast
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
name: hazelcast
|
||||
spec:
|
||||
containers:
|
||||
- resources:
|
||||
limits:
|
||||
cpu: 1
|
||||
image: pires/hazelcast-k8s:0.2
|
||||
name: hazelcast
|
||||
ports:
|
||||
- containerPort: 5701
|
||||
name: hazelcast
|
||||
```
|
||||
|
||||
There are a few things to note in this description. First is that we are running the `pires/hazelcast-k8s` image, tag `0.2`. This is a `busybox` installation with JRE 8. However it also adds a custom [`application`](https://github.com/pires/hazelcast-kubernetes-bootstrapper) that finds any Hazelcast nodes in the cluster and bootstraps an Hazelcast instance accordingle. The `HazelcastDiscoveryController` discovers the Kubernetes API Server using the built in Kubernetes discovery service, and then uses the Kubernetes API to find new nodes (more on this later).
|
||||
|
||||
You may also note that we tell Kubernetes that the container exposes the `hazelcast` port. Finally, we tell the cluster manager that we need 1 cpu core.
|
||||
|
||||
The bulk of the replication controller config is actually identical to the Hazelcast pod declaration above, it simply gives the controller a recipe to use when creating new pods. The other parts are the ```selector``` which contains the controller's selector query, and the ```replicas``` parameter which specifies the desired number of replicas, in this case 1.
|
||||
|
||||
Create this controller:
|
||||
|
||||
```sh
|
||||
$ kubectl create -f hazelcast-controller.yaml
|
||||
```
|
||||
|
||||
After the controller provisions successfully the pod, you can query the service endpoints:
|
||||
```sh
|
||||
$ kubectl get endpoints hazelcast -o yaml
|
||||
apiVersion: v1beta3
|
||||
|
|
@ -117,53 +125,11 @@ subsets:
|
|||
protocol: TCP
|
||||
```
|
||||
|
||||
You can see that the _Service_ has found the pod we created in step one.
|
||||
You can see that the _Service_ has found the pod created by the replication controller.
|
||||
|
||||
### Adding replicated nodes
|
||||
Of course, a single node cluster isn't particularly interesting. The real power of Kubernetes and Hazelcast lies in easily building a replicated, resizable Hazelcast cluster.
|
||||
Now it gets even more interesting.
|
||||
|
||||
In Kubernetes a _Replication Controller_ is responsible for replicating sets of identical pods. Like a _Service_ it has a selector query which identifies the members of it's set. Unlike a _Service_ it also has a desired number of replicas, and it will create or delete _Pods_ to ensure that the number of _Pods_ matches up with it's desired state.
|
||||
|
||||
Replication Controllers will "adopt" existing pods that match their selector query, so let's create a Replication Controller with a single replica to adopt our existing Hazelcast Pod.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1beta3
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
labels:
|
||||
name: hazelcast
|
||||
name: hazelcast
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
name: hazelcast
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
name: hazelcast
|
||||
spec:
|
||||
containers:
|
||||
- resources:
|
||||
limits:
|
||||
cpu: 1
|
||||
image: pires/hazelcast-k8s
|
||||
name: hazelcast
|
||||
ports:
|
||||
- containerPort: 5701
|
||||
name: hazelcast
|
||||
```
|
||||
|
||||
The bulk of the replication controller config is actually identical to the Hazelcast pod declaration above, it simply gives the controller a recipe to use when creating new pods. The other parts are the ```selector``` which contains the controller's selector query, and the ```replicas``` parameter which specifies the desired number of replicas, in this case 1.
|
||||
|
||||
Create this controller:
|
||||
|
||||
```sh
|
||||
$ kubectl create -f hazelcast-controller.yaml
|
||||
```
|
||||
|
||||
Now this is actually not that interesting, since we haven't actually done anything new. Now it will get interesting.
|
||||
|
||||
Let's resize our cluster to 2:
|
||||
Let's resize our cluster to 2 pods:
|
||||
```sh
|
||||
$ kubectl resize rc hazelcast --replicas=2
|
||||
```
|
||||
|
|
@ -173,34 +139,40 @@ Now if you list the pods in your cluster, you should see two hazelcast pods:
|
|||
```sh
|
||||
$ kubectl get pods
|
||||
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE
|
||||
hazelcast 10.245.2.68 e2e-test-minion-vj7k/104.197.8.214 name=hazelcast Running 14 seconds
|
||||
hazelcast pires/hazelcast-k8s Running 2 seconds
|
||||
hazelcast-ulkws 10.245.1.80 e2e-test-minion-2x1f/146.148.62.37 name=hazelcast Running 7 seconds
|
||||
hazelcast pires/hazelcast-k8s Running 6 seconds
|
||||
hazelcast-pkyzd 10.244.90.3 e2e-test-minion-vj7k/104.197.8.214 name=hazelcast Running 14 seconds
|
||||
hazelcast pires/hazelcast-k8s:0.2 Running 2 seconds
|
||||
hazelcast-ulkws 10.244.66.2 e2e-test-minion-2x1f/146.148.62.37 name=hazelcast Running 7 seconds
|
||||
hazelcast pires/hazelcast-k8s:0.2 Running 6 seconds
|
||||
```
|
||||
|
||||
Notice that one of the pods has the human readable name ```hazelcast``` that you specified in your config before, and one has a random string, since it was named by the replication controller.
|
||||
|
||||
To prove that this all works, you can use the ```log``` command to examine the logs of one pod, for example:
|
||||
To prove that this all works, you can use the `log` command to examine the logs of one pod, for example:
|
||||
|
||||
```sh
|
||||
$ kubectl log 16b2beab-94a1-11e4-8a8b-42010af0e23e hazelcast
|
||||
2014-12-24T01:21:09.731468790Z 2014-12-24 01:21:09.701 INFO 10 --- [ main] c.g.p.h.HazelcastDiscoveryController : Asking k8s registry at http://10.160.211.80:80..
|
||||
2014-12-24T01:21:13.686978543Z 2014-12-24 01:21:13.686 INFO 10 --- [ main] c.g.p.h.HazelcastDiscoveryController : Found 3 pods running Hazelcast.
|
||||
2014-12-24T01:21:13.772599736Z 2014-12-24 01:21:13.772 INFO 10 --- [ main] c.g.p.h.HazelcastDiscoveryController : Added member 10.160.2.3
|
||||
2014-12-24T01:21:13.783689690Z 2014-12-24 01:21:13.783 INFO 10 --- [ main] c.g.p.h.HazelcastDiscoveryController : Added member 10.160.2.4
|
||||
$ kubectl log hazelcast-ulkws hazelcast
|
||||
2015-05-09 22:06:20.016 INFO 5 --- [ main] com.github.pires.hazelcast.Application : Starting Application v0.2-SNAPSHOT on hazelcast-enyli with PID 5 (/bootstrapper.jar started by root in /)
|
||||
2015-05-09 22:06:20.071 INFO 5 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@5424f110: startup date [Sat May 09 22:06:20 GMT 2015]; root of context hierarchy
|
||||
2015-05-09 22:06:21.511 INFO 5 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
|
||||
2015-05-09 22:06:21.549 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Asking k8s registry at http://10.100.0.1:80..
|
||||
2015-05-09 22:06:22.031 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Found 2 pods running Hazelcast.
|
||||
2015-05-09 22:06:22.176 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.4.2] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [10.244.90.3, 10.244.66.2]
|
||||
2015-05-09 22:06:22.177 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.4.2] Prefer IPv4 stack is true.
|
||||
2015-05-09 22:06:22.189 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.4.2] Picked Address[10.244.66.2]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
|
||||
2015-05-09 22:06:22.642 INFO 5 --- [ main] com.hazelcast.spi.OperationService : [10.244.66.2]:5701 [someGroup] [3.4.2] Backpressure is disabled
|
||||
2015-05-09 22:06:22.647 INFO 5 --- [ main] c.h.spi.impl.BasicOperationScheduler : [10.244.66.2]:5701 [someGroup] [3.4.2] Starting with 2 generic operation threads and 2 partition operation threads.
|
||||
2015-05-09 22:06:22.796 INFO 5 --- [ main] com.hazelcast.system : [10.244.66.2]:5701 [someGroup] [3.4.2] Hazelcast 3.4.2 (20150326 - f6349a4) starting at Address[10.244.66.2]:5701
|
||||
2015-05-09 22:06:22.798 INFO 5 --- [ main] com.hazelcast.system : [10.244.66.2]:5701 [someGroup] [3.4.2] Copyright (C) 2008-2014 Hazelcast.com
|
||||
2015-05-09 22:06:22.800 INFO 5 --- [ main] com.hazelcast.instance.Node : [10.244.66.2]:5701 [someGroup] [3.4.2] Creating TcpIpJoiner
|
||||
2015-05-09 22:06:22.801 INFO 5 --- [ main] com.hazelcast.core.LifecycleService : [10.244.66.2]:5701 [someGroup] [3.4.2] Address[10.244.66.2]:5701 is STARTING
|
||||
2015-05-09 22:06:23.108 INFO 5 --- [cached.thread-2] com.hazelcast.nio.tcp.SocketConnector : [10.244.66.2]:5701 [someGroup] [3.4.2] Connecting to /10.244.90.3:5701, timeout: 0, bind-any: true
|
||||
2015-05-09 22:06:23.182 INFO 5 --- [cached.thread-2] c.h.nio.tcp.TcpIpConnectionManager : [10.244.66.2]:5701 [someGroup] [3.4.2] Established socket connection between /10.244.66.2:48051 and 10.244.90.3/10.244.90.3:5701
|
||||
2015-05-09 22:06:29.158 INFO 5 --- [ration.thread-1] com.hazelcast.cluster.ClusterService : [10.244.66.2]:5701 [someGroup] [3.4.2]
|
||||
|
||||
(...)
|
||||
Members [2] {
|
||||
Member [10.244.90.3]:5701
|
||||
Member [10.244.66.2]:5701 this
|
||||
}
|
||||
|
||||
2014-12-24T01:21:16.007729519Z 2014-12-24 01:21:16.000 INFO 10 --- [cached.thread-3] c.h.nio.tcp.TcpIpConnectionManager : [10.160.2.4]:5701 [someGroup] [3.3.3] Established socket connection between /10.160.2.4:54931 and /10.160.2.3:5701
|
||||
2014-12-24T01:21:16.427289059Z 2014-12-24 01:21:16.427 INFO 10 --- [thread-Acceptor] com.hazelcast.nio.tcp.SocketAcceptor : [10.160.2.4]:5701 [someGroup] [3.3.3] Accepting socket connection from /10.160.2.3:50660
|
||||
2014-12-24T01:21:16.433763738Z 2014-12-24 01:21:16.433 INFO 10 --- [cached.thread-3] c.h.nio.tcp.TcpIpConnectionManager : [10.160.2.4]:5701 [someGroup] [3.3.3] Established socket connection between /10.160.2.4:5701 and /10.160.2.3:50660
|
||||
2014-12-24T01:21:23.036227250Z 2014-12-24 01:21:23.035 INFO 10 --- [ration.thread-1] com.hazelcast.cluster.ClusterService : [10.160.2.4]:5701 [someGroup] [3.3.3]
|
||||
2014-12-24T01:21:23.036227250Z
|
||||
2014-12-24T01:21:23.036227250Z Members [3] {
|
||||
2014-12-24T01:21:23.036227250Z Member [10.160.2.4]:5701 this
|
||||
2014-12-24T01:21:23.036227250Z Member [10.160.2.3]:5701
|
||||
2014-12-24T01:21:23.036227250Z }
|
||||
2015-05-09 22:06:31.177 INFO 5 --- [ main] com.hazelcast.core.LifecycleService : [10.244.66.2]:5701 [someGroup] [3.4.2] Address[10.244.66.2]:5701 is STARTED
|
||||
```
|
||||
|
||||
Now let's resize our cluster to 4 nodes:
|
||||
|
|
@ -214,9 +186,6 @@ Examine the status again by checking a node’s log and you should see the 4 mem
|
|||
For those of you who are impatient, here is the summary of the commands we ran in this tutorial.
|
||||
|
||||
```sh
|
||||
# create a single hazelcast node
|
||||
kubectl create -f hazelcast.yaml
|
||||
|
||||
# create a service to track all hazelcast nodes
|
||||
kubectl create -f hazelcast-service.yaml
|
||||
|
||||
|
|
@ -226,9 +195,6 @@ kubectl create -f hazelcast-controller.yaml
|
|||
# scale up to 2 nodes
|
||||
kubectl resize rc hazelcast --replicas=2
|
||||
|
||||
# validate the cluster
|
||||
docker exec <container-id> nodetool status
|
||||
|
||||
# scale up to 4 nodes
|
||||
kubectl resize rc hazelcast --replicas=4
|
||||
```
|
||||
|
|
@ -236,6 +202,10 @@ kubectl resize rc hazelcast --replicas=4
|
|||
### Hazelcast Discovery Source
|
||||
|
||||
```java
|
||||
package com.github.pires.hazelcast;
|
||||
|
||||
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.hazelcast.config.Config;
|
||||
import com.hazelcast.config.GroupConfig;
|
||||
import com.hazelcast.config.JoinConfig;
|
||||
|
|
@ -244,9 +214,8 @@ import com.hazelcast.config.NetworkConfig;
|
|||
import com.hazelcast.config.SSLConfig;
|
||||
import com.hazelcast.config.TcpIpConfig;
|
||||
import com.hazelcast.core.Hazelcast;
|
||||
import io.fabric8.kubernetes.api.KubernetesClient;
|
||||
import io.fabric8.kubernetes.api.KubernetesFactory;
|
||||
import io.fabric8.kubernetes.api.model.Pod;
|
||||
import java.io.IOException;
|
||||
import java.net.URL;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
import java.util.concurrent.CopyOnWriteArrayList;
|
||||
|
|
@ -256,7 +225,7 @@ import org.springframework.boot.CommandLineRunner;
|
|||
import org.springframework.stereotype.Controller;
|
||||
|
||||
/**
|
||||
* Read from Kubernetes API all labeled Hazelcast pods, get their IP and connect to them.
|
||||
* Read from Kubernetes API all Hazelcast service bound pods, get their IP and connect to them.
|
||||
*/
|
||||
@Controller
|
||||
public class HazelcastDiscoveryController implements CommandLineRunner {
|
||||
|
|
@ -264,8 +233,23 @@ public class HazelcastDiscoveryController implements CommandLineRunner {
|
|||
private static final Logger log = LoggerFactory.getLogger(
|
||||
HazelcastDiscoveryController.class);
|
||||
|
||||
private static final String HAZELCAST_LABEL_NAME = "name";
|
||||
private static final String HAZELCAST_LABEL_VALUE = "hazelcast";
|
||||
@JsonIgnoreProperties(ignoreUnknown = true)
|
||||
static class Address {
|
||||
|
||||
public String IP;
|
||||
}
|
||||
|
||||
@JsonIgnoreProperties(ignoreUnknown = true)
|
||||
static class Subset {
|
||||
|
||||
public List<Address> addresses;
|
||||
}
|
||||
|
||||
@JsonIgnoreProperties(ignoreUnknown = true)
|
||||
static class Endpoints {
|
||||
|
||||
public List<Subset> subsets;
|
||||
}
|
||||
|
||||
private static String getEnvOrDefault(String var, String def) {
|
||||
final String val = System.getenv(var);
|
||||
|
|
@ -276,25 +260,39 @@ public class HazelcastDiscoveryController implements CommandLineRunner {
|
|||
|
||||
@Override
|
||||
public void run(String... args) {
|
||||
final String kubeApiHost = getEnvOrDefault("KUBERNETES_RO_SERVICE_HOST",
|
||||
final String hostName = getEnvOrDefault("KUBERNETES_RO_SERVICE_HOST",
|
||||
"localhost");
|
||||
final String kubeApiPort = getEnvOrDefault("KUBERNETES_RO_SERVICE_PORT",
|
||||
final String hostPort = getEnvOrDefault("KUBERNETES_RO_SERVICE_PORT",
|
||||
"8080");
|
||||
final String kubeUrl = "http://" + kubeApiHost + ":" + kubeApiPort;
|
||||
log.info("Asking k8s registry at {}..", kubeUrl);
|
||||
final KubernetesClient kube = new KubernetesClient(new KubernetesFactory(
|
||||
kubeUrl));
|
||||
final List<Pod> hazelcastPods = new CopyOnWriteArrayList<>();
|
||||
kube.getPods().getItems().parallelStream().filter(pod
|
||||
-> pod.getLabels().get(HAZELCAST_LABEL_NAME).equals(
|
||||
HAZELCAST_LABEL_VALUE)).forEach(hazelcastPods::add);
|
||||
log.info("Found {} pods running Hazelcast.", hazelcastPods.size());
|
||||
if (!hazelcastPods.isEmpty()) {
|
||||
runHazelcast(hazelcastPods);
|
||||
String serviceName = getEnvOrDefault("HAZELCAST_SERVICE", "hazelcast");
|
||||
String path = "/api/v1beta3/namespaces/default/endpoints/";
|
||||
final String host = "http://" + hostName + ":" + hostPort;
|
||||
log.info("Asking k8s registry at {}..", host);
|
||||
|
||||
final List<String> hazelcastEndpoints = new CopyOnWriteArrayList<>();
|
||||
|
||||
try {
|
||||
URL url = new URL(host + path + serviceName);
|
||||
ObjectMapper mapper = new ObjectMapper();
|
||||
Endpoints endpoints = mapper.readValue(url, Endpoints.class);
|
||||
if (endpoints != null) {
|
||||
if (endpoints.subsets != null && !endpoints.subsets.isEmpty()) {
|
||||
endpoints.subsets.parallelStream().forEach(subset -> {
|
||||
subset.addresses.parallelStream().forEach(
|
||||
addr -> hazelcastEndpoints.add(addr.IP));
|
||||
});
|
||||
}
|
||||
}
|
||||
} catch (IOException ex) {
|
||||
log.warn("Request to Kubernetes API failed", ex);
|
||||
}
|
||||
|
||||
log.info("Found {} pods running Hazelcast.", hazelcastEndpoints.size());
|
||||
|
||||
runHazelcast(hazelcastEndpoints);
|
||||
}
|
||||
|
||||
private void runHazelcast(final List<Pod> hazelcastPods) {
|
||||
private void runHazelcast(final List<String> nodes) {
|
||||
// configure Hazelcast instance
|
||||
final Config cfg = new Config();
|
||||
cfg.setInstanceName(UUID.randomUUID().toString());
|
||||
|
|
@ -313,9 +311,7 @@ public class HazelcastDiscoveryController implements CommandLineRunner {
|
|||
mcCfg.setEnabled(false);
|
||||
// tcp
|
||||
final TcpIpConfig tcpCfg = new TcpIpConfig();
|
||||
hazelcastPods.parallelStream().forEach(pod -> {
|
||||
tcpCfg.addMember(pod.getCurrentState().getPodIP());
|
||||
});
|
||||
nodes.parallelStream().forEach(tcpCfg::addMember);
|
||||
tcpCfg.setEnabled(true);
|
||||
// network join configuration
|
||||
final JoinConfig joinCfg = new JoinConfig();
|
||||
|
|
@ -331,4 +327,5 @@ public class HazelcastDiscoveryController implements CommandLineRunner {
|
|||
}
|
||||
|
||||
}
|
||||
|
||||
```
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ spec:
|
|||
- resources:
|
||||
limits:
|
||||
cpu: 1
|
||||
image: pires/hazelcast-k8s
|
||||
image: pires/hazelcast-k8s:0.2
|
||||
name: hazelcast
|
||||
ports:
|
||||
- containerPort: 5701
|
||||
|
|
|
|||
|
|
@ -1,17 +0,0 @@
|
|||
apiVersion: v1beta3
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
name: hazelcast
|
||||
name: hazelcast
|
||||
spec:
|
||||
containers:
|
||||
- image: pires/hazelcast-k8s
|
||||
name: hazelcast
|
||||
ports:
|
||||
- containerPort: 5701
|
||||
name: hazelcast
|
||||
protocol: TCP
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
Loading…
Reference in New Issue