Merge branch 'v1.3' into fix-responsive-videos
|
@ -12,16 +12,15 @@ description: "How to install Fluentd, Elastic Search, and Kibana to search logs
|
|||
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
|
||||
- [Helm 3](https://helm.sh/)
|
||||
|
||||
|
||||
## Install Elastic search and Kibana
|
||||
|
||||
1. Create namespace for monitoring tool and add Helm repo for Elastic Search
|
||||
1. Create a Kubernetes namespace for monitoring tools
|
||||
|
||||
```bash
|
||||
kubectl create namespace dapr-monitoring
|
||||
```
|
||||
|
||||
2. Add Elastic helm repo
|
||||
2. Add the helm repo for Elastic Search
|
||||
|
||||
```bash
|
||||
helm repo add elastic https://helm.elastic.co
|
||||
|
@ -30,23 +29,23 @@ description: "How to install Fluentd, Elastic Search, and Kibana to search logs
|
|||
|
||||
3. Install Elastic Search using Helm
|
||||
|
||||
By default the chart creates 3 replicas which must be on different nodes. If your cluster has less than 3 nodes, specify a lower number of replicas. For example, this sets it to 1:
|
||||
By default, the chart creates 3 replicas which must be on different nodes. If your cluster has fewer than 3 nodes, specify a smaller number of replicas. For example, this sets the number of replicas to 1:
|
||||
|
||||
```bash
|
||||
helm install elasticsearch elastic/elasticsearch -n dapr-monitoring --set replicas=1
|
||||
```
|
||||
```bash
|
||||
helm install elasticsearch elastic/elasticsearch -n dapr-monitoring --set replicas=1
|
||||
```
|
||||
|
||||
Otherwise:
|
||||
Otherwise:
|
||||
|
||||
```bash
|
||||
helm install elasticsearch elastic/elasticsearch -n dapr-monitoring
|
||||
```
|
||||
```bash
|
||||
helm install elasticsearch elastic/elasticsearch -n dapr-monitoring
|
||||
```
|
||||
|
||||
If you are using minikube or want to disable persistent volumes for development purposes, you can disable it by using the following command:
|
||||
If you are using minikube or simply want to disable persistent volumes for development purposes, you can do so by using the following command:
|
||||
|
||||
```bash
|
||||
helm install elasticsearch elastic/elasticsearch -n dapr-monitoring --set persistence.enabled=false,replicas=1
|
||||
```
|
||||
```bash
|
||||
helm install elasticsearch elastic/elasticsearch -n dapr-monitoring --set persistence.enabled=false,replicas=1
|
||||
```
|
||||
|
||||
4. Install Kibana
|
||||
|
||||
|
@ -54,12 +53,10 @@ helm install elasticsearch elastic/elasticsearch -n dapr-monitoring --set persis
|
|||
helm install kibana elastic/kibana -n dapr-monitoring
|
||||
```
|
||||
|
||||
5. Validation
|
||||
|
||||
Ensure Elastic Search and Kibana are running in your Kubernetes cluster.
|
||||
5. Ensure that Elastic Search and Kibana are running in your Kubernetes cluster
|
||||
|
||||
```bash
|
||||
kubectl get pods -n dapr-monitoring
|
||||
$ kubectl get pods -n dapr-monitoring
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
elasticsearch-master-0 1/1 Running 0 6m58s
|
||||
kibana-kibana-95bc54b89-zqdrk 1/1 Running 0 4m21s
|
||||
|
@ -69,30 +66,29 @@ helm install elasticsearch elastic/elasticsearch -n dapr-monitoring --set persis
|
|||
|
||||
1. Install config map and Fluentd as a daemonset
|
||||
|
||||
Download these config files:
|
||||
- [fluentd-config-map.yaml](/docs/fluentd-config-map.yaml)
|
||||
- [fluentd-dapr-with-rbac.yaml](/docs/fluentd-dapr-with-rbac.yaml)
|
||||
Download these config files:
|
||||
- [fluentd-config-map.yaml](/docs/fluentd-config-map.yaml)
|
||||
- [fluentd-dapr-with-rbac.yaml](/docs/fluentd-dapr-with-rbac.yaml)
|
||||
|
||||
> Note: If you already have Fluentd running in your cluster, please enable the nested json parser to parse JSON formatted log from Dapr.
|
||||
> Note: If you already have Fluentd running in your cluster, please enable the nested json parser so that it can parse JSON-formatted logs from Dapr.
|
||||
|
||||
Apply the configurations to your cluster:
|
||||
Apply the configurations to your cluster:
|
||||
|
||||
```bash
|
||||
kubectl apply -f ./fluentd-config-map.yaml
|
||||
kubectl apply -f ./fluentd-dapr-with-rbac.yaml
|
||||
```
|
||||
```bash
|
||||
kubectl apply -f ./fluentd-config-map.yaml
|
||||
kubectl apply -f ./fluentd-dapr-with-rbac.yaml
|
||||
```
|
||||
|
||||
2. Ensure that Fluentd is running as a daemonset; the number of instances should be the same as the number of cluster nodes. In the example below we only have 1 node.
|
||||
|
||||
```bash
|
||||
kubectl get pods -n kube-system -w
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
coredns-6955765f44-cxjxk 1/1 Running 0 4m41s
|
||||
coredns-6955765f44-jlskv 1/1 Running 0 4m41s
|
||||
etcd-m01 1/1 Running 0 4m48s
|
||||
fluentd-sdrld 1/1 Running 0 14s
|
||||
```
|
||||
2. Ensure that Fluentd is running as a daemonset. The number of FluentD instances should be the same as the number of cluster nodes. In the example below, there is only one node in the cluster:
|
||||
|
||||
```bash
|
||||
$ kubectl get pods -n kube-system -w
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
coredns-6955765f44-cxjxk 1/1 Running 0 4m41s
|
||||
coredns-6955765f44-jlskv 1/1 Running 0 4m41s
|
||||
etcd-m01 1/1 Running 0 4m48s
|
||||
fluentd-sdrld 1/1 Running 0 14s
|
||||
```
|
||||
|
||||
## Install Dapr with JSON formatted logs
|
||||
|
||||
|
@ -106,80 +102,83 @@ fluentd-sdrld 1/1 Running 0 14s
|
|||
|
||||
2. Enable JSON formatted log in Dapr sidecar
|
||||
|
||||
Add `dapr.io/log-as-json: "true"` annotation to your deployment yaml.
|
||||
Add the `dapr.io/log-as-json: "true"` annotation to your deployment yaml. For example:
|
||||
|
||||
Example:
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: pythonapp
|
||||
namespace: default
|
||||
labels:
|
||||
app: python
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: python
|
||||
template:
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: pythonapp
|
||||
namespace: default
|
||||
labels:
|
||||
app: python
|
||||
annotations:
|
||||
dapr.io/enabled: "true"
|
||||
dapr.io/app-id: "pythonapp"
|
||||
dapr.io/log-as-json: "true"
|
||||
...
|
||||
```
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: python
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: python
|
||||
annotations:
|
||||
dapr.io/enabled: "true"
|
||||
dapr.io/app-id: "pythonapp"
|
||||
dapr.io/log-as-json: "true"
|
||||
...
|
||||
```
|
||||
|
||||
## Search logs
|
||||
|
||||
> Note: Elastic Search takes a time to index the logs that Fluentd sends.
|
||||
|
||||
1. Port-forward to svc/kibana-kibana
|
||||
1. Port-forward from localhost to `svc/kibana-kibana`
|
||||
|
||||
```
|
||||
$ kubectl port-forward svc/kibana-kibana 5601 -n dapr-monitoring
|
||||
Forwarding from 127.0.0.1:5601 -> 5601
|
||||
Forwarding from [::1]:5601 -> 5601
|
||||
Handling connection for 5601
|
||||
Handling connection for 5601
|
||||
```
|
||||
```bash
|
||||
$ kubectl port-forward svc/kibana-kibana 5601 -n dapr-monitoring
|
||||
Forwarding from 127.0.0.1:5601 -> 5601
|
||||
Forwarding from [::1]:5601 -> 5601
|
||||
Handling connection for 5601
|
||||
Handling connection for 5601
|
||||
```
|
||||
|
||||
2. Browse `http://localhost:5601`
|
||||
2. Browse to `http://localhost:5601`
|
||||
|
||||
3. Click Management -> Index Management
|
||||
3. Expand the drop-down menu and click **Management → Stack Management**
|
||||
|
||||

|
||||

|
||||
|
||||
4. Wait until dapr-* is indexed.
|
||||
4. On the Stack Management page, select **Data → Index Management** and wait until `dapr-*` is indexed.
|
||||
|
||||

|
||||

|
||||
|
||||
5. Once dapr-* indexed, click Kibana->Index Patterns and Create Index Pattern
|
||||
5. Once `dapr-*` is indexed, click on **Kibana → Index Patterns** and then the **Create index pattern** button.
|
||||
|
||||

|
||||

|
||||
|
||||
6. Define index pattern - type `dapr*` in index pattern
|
||||
6. Define a new index pattern by typing `dapr*` into the **Index Pattern name** field, then click the **Next step** button to continue.
|
||||
|
||||

|
||||

|
||||
|
||||
7. Select time stamp filed: `@timestamp`
|
||||
7. Configure the primary time field to use with the new index pattern by selecting the `@timestamp` option from the **Time field** drop-down. Click the **Create index pattern** button to complete creation of the index pattern.
|
||||
|
||||

|
||||

|
||||
|
||||
8. Confirm that `scope`, `type`, `app_id`, `level`, etc are being indexed.
|
||||
8. The newly created index pattern should be shown. Confirm that the fields of interest such as `scope`, `type`, `app_id`, `level`, etc. are being indexed by using the search box in the **Fields** tab.
|
||||
|
||||
> Note: if you cannot find the indexed field, please wait. it depends on the volume of data and resource size where elastic search is running.
|
||||
> Note: If you cannot find the indexed field, please wait. The time it takes to search across all indexed fields depends on the volume of data and size of the resource that the elastic search is running on.
|
||||
|
||||

|
||||

|
||||
|
||||
9. Click `discover` icon and search `scope:*`
|
||||
9. To explore the indexed data, expand the drop-down menu and click **Analytics → Discover**.
|
||||
|
||||
> Note: it would take some time to make log searchable based on the data volume and resource.
|
||||

|
||||
|
||||

|
||||
10. In the search box, type in a query string such as `scope:*` and click the **Refresh** button to view the results.
|
||||
|
||||
> Note: This can take a long time. The time it takes to return all results depends on the volume of data and size of the resource that the elastic search is running on.
|
||||
|
||||

|
||||
|
||||
## References
|
||||
|
||||
|
|
|
@ -6,7 +6,7 @@ metadata:
|
|||
namespace: kube-system
|
||||
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: fluentd
|
||||
|
@ -24,7 +24,7 @@ rules:
|
|||
|
||||
---
|
||||
kind: ClusterRoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
metadata:
|
||||
name: fluentd
|
||||
namespace: default
|
||||
|
|
Before Width: | Height: | Size: 338 KiB After Width: | Height: | Size: 179 KiB |
Before Width: | Height: | Size: 360 KiB After Width: | Height: | Size: 112 KiB |
Before Width: | Height: | Size: 238 KiB After Width: | Height: | Size: 120 KiB |
Before Width: | Height: | Size: 248 KiB After Width: | Height: | Size: 114 KiB |
Before Width: | Height: | Size: 501 KiB After Width: | Height: | Size: 113 KiB |
Before Width: | Height: | Size: 373 KiB After Width: | Height: | Size: 124 KiB |
Before Width: | Height: | Size: 440 KiB After Width: | Height: | Size: 137 KiB |
After Width: | Height: | Size: 228 KiB |