mirror of https://github.com/knative/docs.git
Adding better Kafka install doc (#2156)
* 💄 Adding better Kafka install hints, including a script, that installs _latest_ of Strimzi, as very small cluster
Signed-off-by: Matthias Wessendorf <mwessend@redhat.com>
* Adding some steps for pure/plain yaml, but still giving folks the script, if they just one step, instead of some yaml steps ...
Co-Authored-By: Ali Ok <aliok@redhat.com>
* go with ephemeral instead of PVC
Co-authored-by: Ali Ok <aliok@redhat.com>
This commit is contained in:
Matthias Wessendorf
GitHub
parent
048936968e
commit
d886333606
|
|
@ -9,9 +9,78 @@ All examples require:
|
|||
- Knative Serving v0.9+
|
||||
- An Apache Kafka cluster
|
||||
|
||||
If you want to run the Apache Kafka cluster on Kubernetes, the simplest option is to install it by using Strimzi. Check out the [Quickstart](https://strimzi.io/quickstarts/) guides for both Minikube and Openshift. You can also install Kafka on the host.
|
||||
### Setting up Apache Kafka
|
||||
|
||||
## Examples
|
||||
If you want to run the Apache Kafka cluster on Kubernetes, the simplest option is to install it by using [Strimzi](https://strimzi.io).
|
||||
|
||||
1. Create a namespace for your Apache Kafka installation, like `kafka`:
|
||||
```shell
|
||||
kubectl create namespace kafka
|
||||
```
|
||||
1. Install the Strimzi operator, like:
|
||||
```shell
|
||||
curl -L "https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.16.2/strimzi-cluster-operator-0.16.2.yaml" \
|
||||
| sed 's/namespace: .*/namespace: kafka/' \
|
||||
| kubectl -n kafka apply -f -
|
||||
```
|
||||
1. Describe the size of your Apache Kafka installation, like:
|
||||
```yaml
|
||||
apiVersion: kafka.strimzi.io/v1beta1
|
||||
kind: Kafka
|
||||
metadata:
|
||||
name: my-cluster
|
||||
spec:
|
||||
kafka:
|
||||
version: 2.4.0
|
||||
replicas: 1
|
||||
listeners:
|
||||
plain: {}
|
||||
tls: {}
|
||||
config:
|
||||
offsets.topic.replication.factor: 1
|
||||
transaction.state.log.replication.factor: 1
|
||||
transaction.state.log.min.isr: 1
|
||||
log.message.format.version: "2.4"
|
||||
storage:
|
||||
type: ephemeral
|
||||
zookeeper:
|
||||
replicas: 3
|
||||
storage:
|
||||
type: ephemeral
|
||||
entityOperator:
|
||||
topicOperator: {}
|
||||
userOperator: {}
|
||||
```
|
||||
1. Deploy the Apache Kafka cluster
|
||||
```
|
||||
$ kubectl apply -n kafka -f kafka.yaml
|
||||
```
|
||||
|
||||
This will install a small, non-production, cluster of Apache Kafka. To verify your installation,
|
||||
check if the pods for Strimzi are all up, in the `kafka` namespace:
|
||||
|
||||
```shell
|
||||
$ kubectl get pods -n kafka
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
my-cluster-entity-operator-65995cf856-ld2zp 3/3 Running 0 102s
|
||||
my-cluster-kafka-0 2/2 Running 0 2m8s
|
||||
my-cluster-zookeeper-0 2/2 Running 0 2m39s
|
||||
my-cluster-zookeeper-1 2/2 Running 0 2m49s
|
||||
my-cluster-zookeeper-2 2/2 Running 0 2m59s
|
||||
strimzi-cluster-operator-77555d4b69-sbrt4 1/1 Running 0 3m14s
|
||||
```
|
||||
|
||||
> NOTE: For production ready installs check [Strimzi](https://strimzi.io).
|
||||
|
||||
### Installation script
|
||||
|
||||
If you want to install the latest version of Strimzi, in just one step, we have a [script](./kafka_setup.sh) for your convenience, which does exactly the same steps that are listed above:
|
||||
|
||||
```shell
|
||||
$ ./kafka_setup.sh
|
||||
```
|
||||
|
||||
## Examples of Apache Kafka and Knative
|
||||
|
||||
A number of different examples, showing the `KafkaSource` and the `KafkaChannel` can be found here:
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,25 @@
|
|||
apiVersion: kafka.strimzi.io/v1beta1
|
||||
kind: Kafka
|
||||
metadata:
|
||||
name: my-cluster
|
||||
spec:
|
||||
kafka:
|
||||
version: 2.4.0
|
||||
replicas: 1
|
||||
listeners:
|
||||
plain: {}
|
||||
tls: {}
|
||||
config:
|
||||
offsets.topic.replication.factor: 1
|
||||
transaction.state.log.replication.factor: 1
|
||||
transaction.state.log.min.isr: 1
|
||||
log.message.format.version: "2.4"
|
||||
storage:
|
||||
type: ephemeral
|
||||
zookeeper:
|
||||
replicas: 3
|
||||
storage:
|
||||
type: ephemeral
|
||||
entityOperator:
|
||||
topicOperator: {}
|
||||
userOperator: {}
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
#!/usr/bin/env bash
|
||||
set -e
|
||||
# Turn colors in this script off by setting the NO_COLOR variable in your
|
||||
# environment to any value:
|
||||
#
|
||||
# $ NO_COLOR=1 test.sh
|
||||
NO_COLOR=${NO_COLOR:-""}
|
||||
if [ -z "$NO_COLOR" ]; then
|
||||
header=$'\e[1;33m'
|
||||
reset=$'\e[0m'
|
||||
else
|
||||
header=''
|
||||
reset=''
|
||||
fi
|
||||
strimzi_version=`curl https://github.com/strimzi/strimzi-kafka-operator/releases/latest | awk -F 'tag/' '{print $2}' | awk -F '"' '{print $1}' 2>/dev/null`
|
||||
function header_text {
|
||||
echo "$header$*$reset"
|
||||
}
|
||||
header_text "Using Strimzi Version: ${strimzi_version}"
|
||||
header_text "Strimzi install"
|
||||
kubectl create namespace kafka
|
||||
curl -L "https://github.com/strimzi/strimzi-kafka-operator/releases/download/${strimzi_version}/strimzi-cluster-operator-${strimzi_version}.yaml" \
|
||||
| sed 's/namespace: .*/namespace: kafka/' \
|
||||
| kubectl -n kafka apply -f -
|
||||
header_text "Applying Strimzi Cluster file"
|
||||
kubectl -n kafka apply -f "https://raw.githubusercontent.com/strimzi/strimzi-kafka-operator/${strimzi_version}/examples/kafka/kafka-ephemeral-single.yaml"
|
||||
|
||||
Loading…
Reference in New Issue