Add OpenTelemetry Collector doc (#870)

concepts/observability/README.md

@@ -36,4 +36,5 @@ The observability tools listed below are ones that have been tested to work with
 ### Distributed Tracing
 
 * [How-To: Set up Zipkin](../../howto/diagnose-with-tracing/zipkin.md)
-* [How-To: Set up Application Insights](../../howto/diagnose-with-tracing/azure-monitor.md)
+* [How-To: Set up Application Insights with Local Forwarder](../../howto/diagnose-with-tracing/local-forwarder.md)
+* [How-To: Set up Application Insights with Open Telemetry Collector](../../howto/diagnose-with-tracing/open-telemetry-collector.md)

concepts/observability/W3C-traces.md

@@ -102,7 +102,8 @@ The tracestate fields are detailed [here](https://www.w3.org/TR/trace-context/#t
 In the gRPC API calls, trace context is passed through `grpc-trace-bin` header.
 
 ## Related Links
-* [How To set up Application Insights for distributed tracing](../../howto/diagnose-with-tracing/azure-monitor.md)
+* [How To set up Application Insights using Local Forwarder](../../howto/diagnose-with-tracing/local-forwarder.md)
+* [How To set up Application Insights using OpenTelemetry Collector](../../howto/diagnose-with-tracing/open-telemetry-collector.md)
 * [How To set up Zipkin for distributed tracing](../../howto/diagnose-with-tracing/zipkin.md)
 * [How to use Trace Context](../../howto/use-w3c-tracecontext)
 * [W3C trace context specification](https://www.w3.org/TR/trace-context/)

concepts/observability/traces.md

@@ -68,7 +68,8 @@ spec:
 
 ## References
 
-* [How-To: Set up Application Insights for distributed tracing](../../howto/diagnose-with-tracing/azure-monitor.md)
+* [How-To: Set up Application Insights distributed tracing with Local Forwarder](../../howto/diagnose-with-tracing/local-forwarder.md)
+* [How-To: Set up Application Insights distributed tracing with OpenTelemetry Collector](../../howto/diagnose-with-tracing/open-telemetry-collector.md)
 * [How-To: Set up Zipkin for distributed tracing](../../howto/diagnose-with-tracing/zipkin.md)
 * [How-To: Use W3C Trace Context for distributed tracing](../../howto/use-w3c-tracecontext/README.md)
 

howto/diagnose-with-tracing/open-telemetry-collector.md

@@ -0,0 +1,124 @@
+# Using OpenTelemetry Collector to collect traces
+
+Dapr can integrate with [OpenTelemetry
+Collector](https://github.com/open-telemetry/opentelemetry-collector)
+using the OpenCensus API. This guide walks through an example to use Dapr to push trace events to Azure Application Insights, through the OpenTelemetry Collector.
+
+## Requirements
+A installation of Dapr on Kubernetes.
+
+## How to configure distributed tracing with Application Insights
+
+### Setup Application Insights
+
+1. First, you'll need an Azure account. See instructions
+   [here](https://azure.microsoft.com/free/) to apply for a **free**
+   Azure account.
+2. Follow instructions
+   [here](https://docs.microsoft.com/en-us/azure/azure-monitor/app/create-new-resource)
+   to create a new Application Insights resource.
+3. Get the Application Insights Intrumentation key from your Application Insights page.
+
+### Run OpenTelemetry Collector to push to your Application Insights instance
+
+First, save your Application Insights Instrumentation Key in an environment variable
+```
+export APP_INSIGHTS_KEY=<your-app-insight-key>
+```
+
+Next, install the OpenTelemetry Collector to your Kubernetes cluster to push events to your Application Insights instance
+
+1. Check out the file
+   [open-telemetry-collector.yaml](open-telemetry-collector/open-telemetry-collector.yaml),
+   and replace the `<INSTRUMENTATION-KEY>` placeholder with your
+   APP_INSIGHTS_KEY. Or you can run the following
+
+```
+# Download the file
+wget https://raw.githubusercontent.com/dapr/docs/master/howto/diagnose-with-tracing/open-telemetry-collector/open-telemetry-collector.yaml
+
+# Update the instrumentation key.
+sed  -i '' "s/<INSTRUMENTATION-KEY>/$APP_INSIGHTS_KEY/g" open-telemetry-collector.yaml
+```
+
+2. Apply the configuration with `kubectl apply -f open-telemetry-collector.yaml`.
+
+Next, set up both a Dapr configuration file to turn on tracing and deploy a tracing exporter component that uses the OpenTelemetry Collector.
+
+1. Create a collector-component.yaml file with this
+   [content](open-telemetry-collector/collector-component.yaml)
+
+```
+wget https://raw.githubusercontent.com/dapr/docs/master/howto/diagnose-with-tracing/open-telemetry-collector/collector-component.yaml
+```
+
+2. Apply the configuration with `kubectl apply -f collector-component.yaml`.
+
+### Deploy your app with tracing
+
+When running in Kubernetes mode, apply the `appconfig` configuration by adding a
+`dapr.io/config` annotation to the container that you want to participate in the distributed tracing, as shown in the following
+example:
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  ...
+spec:
+  ...
+  template:
+    metadata:
+      ...
+      annotations:
+        dapr.io/enabled: "true"
+        dapr.io/app-id: "MyApp"
+        dapr.io/app-port: "8080"
+        dapr.io/config: "appconfig"
+```
+
+Some of the quickstarts such as  [distributed
+calculator](https://github.com/dapr/quickstarts/tree/master/distributed-calculator)
+already configure these settings, so if you are using those no additional settings are needed.
+
+That's it! There's no need include any SDKs or instrument your
+application code. Dapr automatically handles the distributed tracing
+for you.
+
+> **NOTE**: You can register multiple tracing exporters at the same time, and
+> the tracing logs are forwarded to all registered exporters.
+
+Deploy and run some applications. After a few minutes, you should see
+tracing logs appearing in your Application Insights resource. You can
+also use **Application Map** to examine the topology of your services,
+as shown below:
+
+![Application map](../../images/open-telemetry-app-insights.png)
+
+> **NOTE**: Only operations going through Dapr API exposed by Dapr
+> sidecar (e.g. service invocation or event publishing) are
+> displayed in Application Map topology.
+
+## Tracing configuration
+
+The `tracing` section under the `Configuration` spec contains the
+following properties:
+
+```yml
+tracing:
+    samplingRate: "1"
+```
+
+The following table lists the different properties.
+
+Property      | Type   | Description
+------------- | ------ | -----------
+samplingRate  | string | Set sampling rate for tracing to be enabled or disabled.
+
+
+`samplingRate` is used to enable or disable the tracing. To disable
+the sampling rate , set `samplingRate : "0"` in the configuration. The
+valid range of samplingRate is between 0 and 1 inclusive. The sampling
+rate determines whether a trace span should be sampled or not based on
+value. `samplingRate : "1"` will always sample the traces.By default,
+the sampling rate is 1 in 10,000

howto/diagnose-with-tracing/open-telemetry-collector/collector-component.yaml

@@ -0,0 +1,21 @@
+apiVersion: dapr.io/v1alpha1
+kind: Configuration
+metadata:
+  name: appconfig
+  namespace: default
+spec:
+  tracing:
+    samplingRate: "1"
+---
+apiVersion: dapr.io/v1alpha1
+kind: Component
+metadata:
+  name: native
+  namespace: default
+spec:
+  type: exporters.native
+  metadata:
+  - name: enabled
+    value: "true"
+  - name: agentEndpoint
+    value: "otel-collector.default.svc.cluster.local:55678"

howto/diagnose-with-tracing/open-telemetry-collector/open-telemetry-collector.yaml

@@ -0,0 +1,107 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: otel-collector-conf
+  labels:
+    app: opentelemetry
+    component: otel-collector-conf
+data:
+  otel-collector-config: |
+    receivers:
+      opencensus:
+        endpoint: 0.0.0.0:55678
+    processors:
+      queued_retry:
+      batch:
+    extensions:
+      health_check:
+      pprof:
+        endpoint: :1888
+      zpages:
+        endpoint: :55679
+    exporters:
+      azuremonitor:
+      azuremonitor/2:
+        endpoint: "https://dc.services.visualstudio.com/v2/track"
+        instrumentation_key: "<INSTRUMENTATION-KEY>"
+        # maxbatchsize is the maximum number of items that can be
+        # queued before calling to the configured endpoint
+        maxbatchsize: 100
+        # maxbatchinterval is the maximum time to wait before calling
+        # the configured endpoint.
+        maxbatchinterval: 10s
+    service:
+      extensions: [pprof, zpages, health_check]
+      pipelines:
+        traces:
+          receivers: [opencensus]
+          exporters: [azuremonitor/2]
+          processors: [batch, queued_retry]
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: otel-collector
+  labels:
+    app: opencesus
+    component: otel-collector
+spec:
+  ports:
+  - name: opencensus # Default endpoint for Opencensus receiver.
+    port: 55678
+    protocol: TCP
+    targetPort: 55678
+  selector:
+    component: otel-collector
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: otel-collector
+  labels:
+    app: opentelemetry
+    component: otel-collector
+spec:
+  replicas: 1  # scale out based on your usage
+  selector:
+    matchLabels:
+      app: opentelemetry
+  template:
+    metadata:
+      labels:
+        app: opentelemetry
+        component: otel-collector
+    spec:
+      containers:
+      - name: otel-collector
+        image: otel/opentelemetry-collector-contrib-dev:latest
+        command:
+          - "/otelcontribcol"
+          - "--config=/conf/otel-collector-config.yaml"
+        resources:
+          limits:
+            cpu: 1
+            memory: 2Gi
+          requests:
+            cpu: 200m
+            memory: 400Mi
+        ports:
+          - containerPort: 55678 # Default endpoint for Opencensus receiver.
+        volumeMounts:
+          - name: otel-collector-config-vol
+            mountPath: /conf
+        livenessProbe:
+          httpGet:
+            path: /
+            port: 13133
+        readinessProbe:
+          httpGet:
+            path: /
+            port: 13133
+      volumes:
+        - configMap:
+            name: otel-collector-conf
+            items:
+              - key: otel-collector-config
+                path: otel-collector-config.yaml
+          name: otel-collector-config-vol

howto/use-w3c-tracecontext/README.md

@@ -295,7 +295,8 @@ You can now correlate the calls in your app and across services with Dapr using
 
 * [Observability concepts](../../concepts/observability/traces.md)
 * [W3C Trace Context for distributed tracing](../../concepts/observability/W3C-traces.md)
-* [How to set up Application Insights for distributed tracing](../../howto/diagnose-with-tracing/azure-monitor.md)
+* [How to set up Application Insights distributed tracing with Local Forwarder](../../howto/diagnose-with-tracing/local-forwarder.md)
+* [How to set up Application Insights distributed tracing with OpenTelemetry Collector](../../howto/diagnose-with-tracing/open-telemetry-collector.md)
 * [How to set up Zipkin for distributed tracing](../../howto/diagnose-with-tracing/zipkin.md)
 * [W3C trace context specification](https://www.w3.org/TR/trace-context/)
 * [Observability quickstart](https://github.com/dapr/quickstarts/tree/master/observability)