mirror of https://github.com/knative/docs.git
Move the document and fluent image sample from serving repo to docs repo (#259)
* add doccument * add reasons for plugins * Formatted file * add more details * fix links * Remove typo and duplicate steps + + made a few edits for clarity and consistency
This commit is contained in:
Yanwei Guo
Knative Prow Robot
parent
d630a8fa81
commit
12e3364b2d
|
|
@ -0,0 +1,47 @@
|
|||
# Fluentd Docker Image on Knative Serving
|
||||
|
||||
Knative Serving uses a [Fluentd](https://www.fluentd.org/) docker image to
|
||||
collect logs. Operators can customize their own Fluentd docker image and
|
||||
configuration to define logging output.
|
||||
|
||||
## Requirements
|
||||
|
||||
Knative requires the customized Fluentd docker image with the following plugins
|
||||
installed:
|
||||
|
||||
* [fluentd](https://github.com/fluent/fluentd) >= v0.14.0
|
||||
* [fluent-plugin-kubernetes_metadata_filter](https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter) >=
|
||||
1.0.0 AND < 2.1.0: To enrich log entries with Kubernetes metadata.
|
||||
* [fluent-plugin-detect-exceptions](https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions) >=
|
||||
0.0.9: To combine multi-line exception stack traces logs into one log entry.
|
||||
* [fluent-plugin-multi-format-parser](https://github.com/repeatedly/fluent-plugin-multi-format-parser) >=
|
||||
1.0.0: To detect log format as Json or plain text.
|
||||
|
||||
## Sample images
|
||||
|
||||
Operators can use any Docker image which meets the requirements above and
|
||||
includes the desired output plugin. Two examples below:
|
||||
|
||||
### Send logs to Elasticsearch
|
||||
|
||||
Operators can use
|
||||
[k8s.gcr.io/fluentd-elasticsearch:v2.0.4](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch/fluentd-es-image)
|
||||
which includes
|
||||
[fluent-plugin-elasticsearch](https://github.com/uken/fluent-plugin-elasticsearch)
|
||||
that allows sending logs to a Elasticsearch service.
|
||||
|
||||
### Send logs to Stackdriver
|
||||
|
||||
This sample [Dockerfile](stackdriver/Dockerfile) is based on
|
||||
[k8s.gcr.io/fluentd-elasticsearch:v2.0.4](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch).
|
||||
It additionally adds one more plugin -
|
||||
[fluent-plugin-google-cloud](https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud)
|
||||
which allows sending logs to Stackdriver.
|
||||
|
||||
Operators can build this image and push it to a container registry which their
|
||||
Kubernetes cluster has access to. See [Setting Up A Logging Plugin](/serving/setting-up-a-logging-plugin.md)
|
||||
for details. **NOTE**: Operators need to add credentials
|
||||
file the stackdriver agent needs to the docker image if their Knative Serving is
|
||||
not built on a GCP based cluster or they want to send logs to another GCP
|
||||
project. See [here](https://cloud.google.com/logging/docs/agent/authorization)
|
||||
for more information.
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
FROM k8s.gcr.io/fluentd-elasticsearch:v2.0.4
|
||||
|
||||
RUN BUILD_DEPS="make gcc g++ libc6-dev ruby-dev libffi-dev" \
|
||||
&& clean-install $BUILD_DEPS \
|
||||
ca-certificates \
|
||||
libjemalloc1 \
|
||||
liblz4-1 \
|
||||
ruby \
|
||||
&& echo 'gem: --no-document' >> /etc/gemrc \
|
||||
&& gem install fluent-plugin-google-cloud -v 0.6.19 \
|
||||
&& rm -rf /tmp/* \
|
||||
/var/lib/apt/lists/* \
|
||||
/usr/lib/ruby/gems/*/cache/*.gem \
|
||||
/var/log/* \
|
||||
/var/tmp/*
|
||||
|
|
@ -7,14 +7,36 @@ or
|
|||
You can install only one of these two setups and side-by-side installation of
|
||||
these two are not supported.
|
||||
|
||||
## Before you begin
|
||||
|
||||
The following instructions assume that you cloned the Knative Serving repository.
|
||||
To clone the repository, run the following commands:
|
||||
|
||||
```shell
|
||||
git clone https://github.com/knative/serving knative-serving
|
||||
cd knative-serving
|
||||
git checkout v0.1.1
|
||||
```
|
||||
|
||||
## Elasticsearch, Kibana, Prometheus & Grafana Setup
|
||||
|
||||
If you installed the
|
||||
[full Knative release](../install/README.md#installing-knative),
|
||||
skip this step and continue to
|
||||
[Create Elasticsearch Indices](#create-elasticsearch-indices)
|
||||
the monitoring component is already installed and you can skip down to the
|
||||
[Create Elasticsearch Indices](#create-elasticsearch-indices) section.
|
||||
|
||||
- Install Knative monitoring components from the root of the [Serving repository](https://github.com/knative/serving):
|
||||
To configure and setup monitoring:
|
||||
|
||||
1. Choose a container image that meets the
|
||||
[Fluentd image requirements](fluentd/README.md#requirements). For example, you can use the
|
||||
public image [k8s.gcr.io/fluentd-elasticsearch:v2.0.4](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch/fluentd-es-image).
|
||||
Or you can create a custom one and upload the image to a container registry
|
||||
which your cluster has read access to.
|
||||
1. Follow the instructions in
|
||||
["Setting up a logging plugin"](setting-up-a-logging-plugin.md#Configuring)
|
||||
to configure the Elasticsearch components settings.
|
||||
1. Install Knative monitoring components by running the following command from the root directory of
|
||||
[knative/serving](https://github.com/knative/serving) repository:
|
||||
|
||||
```shell
|
||||
kubectl apply --recursive --filename config/monitoring/100-common \
|
||||
|
|
@ -25,7 +47,7 @@ skip this step and continue to
|
|||
--filename config/monitoring/200-common/100-istio.yaml
|
||||
```
|
||||
|
||||
- The installation is complete when logging & monitoring components are all
|
||||
The installation is complete when logging & monitoring components are all
|
||||
reported `Running` or `Completed`:
|
||||
|
||||
```shell
|
||||
|
|
@ -53,12 +75,12 @@ skip this step and continue to
|
|||
|
||||
### Create Elasticsearch Indices
|
||||
|
||||
To visualize logs with Kibana, you need to set which Elasticsearch indices to explore. We will create two indices in Elasticsearch using `Logstash` for application logs and `Zipkin`
|
||||
To visualize logs with Kibana, you need to set which Elasticsearch indices to explore. We will
|
||||
create two indices in Elasticsearch using `Logstash` for application logs and `Zipkin`
|
||||
for request traces.
|
||||
|
||||
- To open the Kibana UI (the visualization tool for
|
||||
[Elasticsearch](https://info.elastic.co)), start a local proxy with the
|
||||
following command:
|
||||
- To open the Kibana UI (the visualization tool for [Elasticsearch](https://info.elastic.co)),
|
||||
you must start a local proxy by running the following command:
|
||||
|
||||
```shell
|
||||
kubectl proxy
|
||||
|
|
@ -84,24 +106,22 @@ for request traces.
|
|||
|
||||
## Stackdriver, Prometheus & Grafana Setup
|
||||
|
||||
If your Knative Serving is not built on a Google Cloud Platform (GCP) based
|
||||
cluster or you want to send logs to another GCP project, you need to build your
|
||||
own Fluentd image and modify the configuration first. See
|
||||
You must configure and build your own Fluentd image if either of the following are true:
|
||||
|
||||
1. Install
|
||||
[Fluentd image on Knative Serving](https://github.com/knative/serving/blob/master/image/fluentd/README.md).
|
||||
2. [Set up a logging plugin](setting-up-a-logging-plugin.md).
|
||||
3. Install Knative monitoring components:
|
||||
* Your Knative Serving component is not hosted on a Google Cloud Platform (GCP) based cluster.
|
||||
* You want to send logs to another GCP project.
|
||||
|
||||
a. Clone the Knative Serving repository:
|
||||
To configure and setup monitoring:
|
||||
|
||||
```shell
|
||||
git clone https://github.com/knative/serving knative-serving
|
||||
cd knative-serving
|
||||
git checkout v0.1.1
|
||||
```
|
||||
|
||||
b. Apply the monitoring manifests:
|
||||
1. Choose a container image that meets the
|
||||
[Fluentd image requirements](fluentd/README.md#requirements). For example, you can use a
|
||||
public image. Or you can create a custom one and upload the image to a
|
||||
container registry which your cluster has read access to.
|
||||
2. Follow the instructions in
|
||||
["Setting up a logging plugin"](setting-up-a-logging-plugin.md#Configuring)
|
||||
to configure the stackdriver components settings.
|
||||
3. Install Knative monitoring components by running the following command from the root directory of
|
||||
[knative/serving](https://github.com/knative/serving) repository:
|
||||
|
||||
```shell
|
||||
kubectl apply --recursive --filename config/monitoring/100-common \
|
||||
|
|
@ -111,6 +131,31 @@ own Fluentd image and modify the configuration first. See
|
|||
--filename config/monitoring/200-common/100-istio.yaml
|
||||
```
|
||||
|
||||
The installation is complete when logging & monitoring components are all
|
||||
reported `Running` or `Completed`:
|
||||
|
||||
```shell
|
||||
kubectl get pods --namespace monitoring --watch
|
||||
```
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
elasticsearch-logging-0 1/1 Running 0 2d
|
||||
elasticsearch-logging-1 1/1 Running 0 2d
|
||||
fluentd-ds-5kc85 1/1 Running 0 2d
|
||||
fluentd-ds-vhrcq 1/1 Running 0 2d
|
||||
fluentd-ds-xghk9 1/1 Running 0 2d
|
||||
grafana-798cf569ff-v4q74 1/1 Running 0 2d
|
||||
kibana-logging-7d474fbb45-6qb8x 1/1 Running 0 2d
|
||||
kube-state-metrics-75bd4f5b8b-8t2h2 4/4 Running 0 2d
|
||||
node-exporter-cr6bh 2/2 Running 0 2d
|
||||
node-exporter-mf6k7 2/2 Running 0 2d
|
||||
node-exporter-rhzr7 2/2 Running 0 2d
|
||||
prometheus-system-0 1/1 Running 0 2d
|
||||
prometheus-system-1 1/1 Running 0 2d
|
||||
```
|
||||
|
||||
CTRL+C to exit watch.
|
||||
## Learn More
|
||||
|
||||
- Learn more about accessing logs, metrics, and traces:
|
||||
|
|
|
|||
|
|
@ -11,6 +11,10 @@ a new process will be required. In order to minimize the effort for a future
|
|||
migration, we recommend only changing the output configuration of Fluentd and
|
||||
leaving the rest intact.
|
||||
|
||||
**NOTE**: All the files mentioned below are in
|
||||
[knative/serving](https://github.com/knative/serving) repository. You run the
|
||||
commands mentioned below from the root directory of `knative/serving`.
|
||||
|
||||
## Configuring
|
||||
|
||||
### Configure the DaemonSet for stdout/stderr logs
|
||||
|
|
@ -19,15 +23,14 @@ Operators can do the following steps to configure the Fluentd DaemonSet for
|
|||
collecting `stdout/stderr` logs from the containers:
|
||||
|
||||
1. Replace `900.output.conf` part in
|
||||
[fluentd-configmap.yaml](/config/monitoring/fluentd-configmap.yaml) with the
|
||||
desired output configuration. Knative provides samples for sending logs to
|
||||
Elasticsearch or Stackdriver. Developers can simply choose one of `150-*`
|
||||
from [/config/monitoring](/config/monitoring) or override any with other
|
||||
configuration.
|
||||
1. Replace the `image` field of `fluentd-ds` container
|
||||
in [fluentd-ds.yaml](/third_party/config/monitoring/common/fluentd/fluentd-ds.yaml)
|
||||
[100-fluentd-configmap.yaml](https://https://github.com/knative/serving/blob/master/config/monitoring/150-elasticsearch/100-fluentd-configmap.yaml) with the
|
||||
desired output configuration. Knative provides a sample for sending logs to
|
||||
Elasticsearch or Stackdriver. Developers can simply use `100-fluentd-configmap.yaml`
|
||||
or override any with other configuration.
|
||||
2. Replace the `image` field of `fluentd-ds` container
|
||||
in [fluentd-ds.yaml](https://github.com/knative/serving/blob/master/third_party/config/monitoring/common/kubernetes/fluentd/fluentd-ds.yaml)
|
||||
with the Fluentd image including the desired Fluentd output plugin.
|
||||
See [here](/image/fluentd/README.md) for the requirements of Flunetd image
|
||||
See [here](image/fluentd/README.md) for the requirements of Flunetd image
|
||||
on Knative.
|
||||
|
||||
### Configure the Sidecar for log files under /var/log
|
||||
|
|
@ -38,15 +41,15 @@ collecting log files under `/var/log`. An
|
|||
is in process to get rid of the sidecar. The steps to configure are:
|
||||
|
||||
1. Replace `logging.fluentd-sidecar-output-config` flag in
|
||||
[config-observability](/config/config-observability.yaml) with the
|
||||
[config-observability](https://github.com/knative/serving/blob/master/config/config-observability.yaml) with the
|
||||
desired output configuration. **NOTE**: The Fluentd DaemonSet is in
|
||||
`monitoring` namespace while the Fluentd sidecar is in the namespace same with
|
||||
the app. There may be small differences between the configuration for DaemonSet
|
||||
and sidecar even though the desired backends are the same.
|
||||
1. Replace `logging.fluentd-sidecar-image` flag in
|
||||
[config-observability](/config/config-observability.yaml) with the Fluentd image including the
|
||||
desired Fluentd output plugin. In theory, this is the same
|
||||
with the one for Fluentd DaemonSet.
|
||||
[config-observability](https://github.com/knative/serving/blob/master/config/config-observability.yaml)
|
||||
with the Fluentd image including the desired Fluentd output plugin. In theory,
|
||||
this is the same with the one for Fluentd DaemonSet.
|
||||
|
||||
## Deploying
|
||||
|
||||
|
|
@ -78,9 +81,20 @@ the Elasticsearch and Kibana services. Knative provides this sample:
|
|||
kubectl apply --recursive --filename third_party/config/monitoring/elasticsearch
|
||||
```
|
||||
|
||||
See [here](/config/monitoring/README.md) for deploying the whole Knative
|
||||
See [here](/serving/installing-logging-metrics-traces.md) for deploying the whole Knative
|
||||
monitoring components.
|
||||
|
||||
## Uninstalling
|
||||
|
||||
To uninstall a logging plugin, run:
|
||||
|
||||
```shell
|
||||
kubectl delete -f <the-fluentd-config-for-daemonset> \
|
||||
-f third_party/config/monitoring/common/kubernetes/fluentd/fluentd-ds.yaml \
|
||||
-f config/monitoring/200-common/100-fluentd.yaml
|
||||
-f config/monitoring/200-common/100-istio.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Except as otherwise noted, the content of this page is licensed under the
|
||||
|
|
|
|||
Loading…
Reference in New Issue