# Extending the OpenTelemetry .NET SDK

* [Building your own exporter](#exporter)
* [Building your own instrumentation library](#instrumentation-library)
* [Building your own processor](#processor)
* [Building your own sampler](#sampler)
* [Building your own resource detector](#resource-detector)
* [References](#references)

## Exporter

OpenTelemetry .NET SDK has provided the following built-in trace exporters:

* [Console](../../../src/OpenTelemetry.Exporter.Console/README.md)
* [InMemory](../../../src/OpenTelemetry.Exporter.InMemory/README.md)
* [Jaeger](../../../src/OpenTelemetry.Exporter.Jaeger/README.md)
* [OpenTelemetryProtocol](../../../src/OpenTelemetry.Exporter.OpenTelemetryProtocol/README.md)
* [Zipkin](../../../src/OpenTelemetry.Exporter.Zipkin/README.md)

Custom exporters can be implemented to send telemetry data to places which are
not covered by the built-in exporters:

* Exporters should derive from `OpenTelemetry.BaseExporter<Activity>` (which
  belongs to the [OpenTelemetry](../../../src/OpenTelemetry/README.md) package)
  and implement the `Export` method.
* Exporters can optionally implement the `OnForceFlush` and `OnShutdown` method.
* Depending on user's choice and load on the application, `Export` may get
  called with one or more activities.
* Exporters will only receive sampled-in and ended activities.
* Exporters should not throw exceptions from `Export`, `OnForceFlush` and
  `OnShutdown`.
* Exporters should not modify activities they receive (the same activity may be
  exported again by different exporter).
* Exporters are responsible for any retry logic needed by the scenario. The SDK
  does not implement any retry logic.
* Exporters should avoid generating telemetry and causing live-loop, this can be
  done via `OpenTelemetry.SuppressInstrumentationScope`.
* Exporters should use `Activity.TagObjects` collection instead of
  `Activity.Tags` to obtain the full set of attributes (tags).
* Exporters should use `ParentProvider.GetResource()` to get the `Resource`
  associated with the provider.

```csharp
class MyExporter : BaseExporter<Activity>
{
    public override ExportResult Export(in Batch<Activity> batch)
    {
        using var scope = SuppressInstrumentationScope.Begin();

        foreach (var activity in batch)
        {
            Console.WriteLine($"Export: {activity.DisplayName}");
        }

        return ExportResult.Success;
    }
}
```

A demo exporter which simply writes activity name to the console is shown
[here](./MyExporter.cs).

Apart from the exporter itself, you should also provide extension methods as
shown [here](./MyExporterExtensions.cs). This allows users to add the Exporter
to the `TracerProvider` as shown in the example [here](./Program.cs).

### Exporting Activity Status

[DiagnosticSource](https://www.nuget.org/packages/system.diagnostics.diagnosticsource)
package did not originally have a dedicated field for storing
[Status](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md#set-status),
and hence, users were encouraged to follow the convention of storing status
using tags "otel.status_code" and "otel.status_description".
[DiagnosticSource](https://www.nuget.org/packages/system.diagnostics.diagnosticsource)
version 6.0.0 added `Status` and `StatusDescription` to `Activity` class.
Exporters which support reading status from `Activity` directly should fall back
to retrieving status from the tags described above, to maintain backward
compatibility.
[ConsoleActivityExporter](../../../src/OpenTelemetry.Exporter.Console/ConsoleActivityExporter.cs)
may be used as a reference.

## Instrumentation Library

The [inspiration of the OpenTelemetry
project](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/overview.md#instrumentation-libraries)
is to make every library observable out of the box by having them call
OpenTelemetry API directly. However, many libraries will not have such
integration, and as such there is a need for a separate library which would
inject such calls, using mechanisms such as wrapping interfaces, subscribing to
library-specific callbacks, or translating existing telemetry into OpenTelemetry
model.

A library which enables instrumentation for another library is called
[Instrumentation
Library](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#instrumentation-library)
and the library it instruments is called the [Instrumented
Library](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#instrumented-library).
If a given library has built-in instrumentation with OpenTelemetry, then
instrumented library and instrumentation library will be the same.

The [OpenTelemetry .NET Github repo](../../../README.md#getting-started) ships
the following instrumentation libraries. The individual docs for them describes
the library they instrument, and steps for enabling them.

* [ASP.NET
  Core](../../../src/OpenTelemetry.Instrumentation.AspNetCore/README.md)
* [gRPC
  client](../../../src/OpenTelemetry.Instrumentation.GrpcNetClient/README.md)
* [HTTP clients](../../../src/OpenTelemetry.Instrumentation.Http/README.md)
* [SQL client](../../../src/OpenTelemetry.Instrumentation.SqlClient/README.md)

More community contributed instrumentations are available in [OpenTelemetry .NET
Contrib](https://github.com/open-telemetry/opentelemetry-dotnet-contrib/tree/main/src).
If you are writing an instrumentation library yourself, use the following
guidelines.

### Writing own instrumentation library

This section describes the steps required to write your own instrumentation
library.

*If you are writing a new library or modifying an existing library, the
recommendation is to use [ActivitySource API/OpenTelemetry
API](../../../src/OpenTelemetry.Api/README.md#introduction-to-opentelemetry-net-tracing-api)
to instrument it and emit activity/span. If a library is instrumented using
ActivitySource API, then there is no need of writing a separate instrumentation
library, as instrumented and instrumentation library become same in this case.
For applications to collect traces from this library, all that is needed is to
enable the ActivitySource for the library using `AddSource` method of the
`TracerProviderBuilder`. The following section is applicable only if you are
writing an instrumentation library for an instrumented library which you cannot
modify to emit activities directly.*

Writing an instrumentation library typically involves 3 steps.

1. First step involves "hijacking" into the target library. The exact mechanism
   of this depends on the target library itself. For example,
   System.Data.SqlClient for .NET Framework, which publishes events using
   `EventSource`. The [SqlClient instrumentation
   library](../../../src/OpenTelemetry.Instrumentation.SqlClient/Implementation/SqlEventSourceListener.netfx.cs),
   in this case subscribes to the `EventSource` callbacks.

2. Second step is to emit activities using the [ActivitySource
   API](../../../src/OpenTelemetry.Api/README.md#introduction-to-opentelemetry-net-tracing-api).
   In this step, the instrumentation library emits activities *on behalf of* the
   target instrumented library. Irrespective of the actual mechanism used in
   first step, this should be uniform across all instrumentation libraries. The
   `ActivitySource` must be created using the name and version of the
   instrumentation library (eg: "OpenTelemetry.Instrumentation.Http") and *not*
   the instrumented library (eg: "System.Net.Http")
      1. [Context
      Propagation](../../../src/OpenTelemetry.Api/README.md#context-propagation):
      If your library initiates out of process requests or accepts them, the
      library needs to [inject the
      `PropagationContext`](../../../examples/MicroserviceExample/Utils/Messaging/MessageSender.cs)
      to outgoing requests and [extract the
      context](../../../examples/MicroserviceExample/Utils/Messaging/MessageReceiver.cs)
      and hydrate the Activity/Baggage upon receiving incoming requests. This is
      only required if you're using your own protocol to communicate over the
      wire. (i.e. If you're using an already instrumented HttpClient or
      GrpcClient, this is already provided to you and **do not require**
      injecting/extracting `PropagationContext` explicitly again.)

3. Third step is an optional step, and involves providing extension methods on
   `TracerProviderBuilder`, to enable the instrumentation. This is optional, and
   the below guidance must be followed:

    1. If the instrumentation library requires state management tied to that of
       `TracerProvider`, then it must register itself with the provider with the
       `AddInstrumentation` method on the `TracerProviderBuilder`. This causes
       the instrumentation to be created and disposed along with
       `TracerProvider`. If the above is required, then it must provide an
       extension method on `TracerProviderBuilder`. Inside this extension
       method, it should call the `AddInstrumentation` method, and `AddSource`
       method to enable its ActivitySource for the provider. An example
       instrumentation using this approach is [SqlClient
       instrumentation](../../../src/OpenTelemetry.Instrumentation.SqlClient/TracerProviderBuilderExtensions.cs).
       **CAUTION**: The instrumentation libraries requiring state management
       are usually hard to auto-instrument. Therefore, they take the risk of not
       being supported by [OpenTelemetry .NET Automatic Instrumentation](https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation).

    2. If the instrumentation library does not requires any state management
       tied to that of `TracerProvider`, then providing `TracerProviderBuilder`
       extension method is optional. If provided, then it must call `AddSource`
       to enable its ActivitySource for the provider.

    3. If instrumentation library does not require state management, and is not
       providing extension method, then the name of the `ActivitySource` used by
       the instrumented library must be documented so that end users can enable
       it using `AddSource` method on `TracerProviderBuilder`.

### Special case : Instrumentation for libraries producing legacy Activity

There is a special case for libraries which are already instrumented to produce
[Activity](https://github.com/dotnet/runtime/blob/master/src/libraries/System.Diagnostics.DiagnosticSource/src/ActivityUserGuide.md),
but using the
[DiagnosticSource](https://github.com/dotnet/runtime/blob/master/src/libraries/System.Diagnostics.DiagnosticSource/src/DiagnosticSourceUsersGuide.md)
method. These are referred to as "legacy Activity" in this repo. These libraries
already create activities but they do so by using the `Activity` constructor
directly, rather than using `ActivitySource.StartActivity` method. These
activities does not by default runs through the sampler, and will have their
`Kind` set to internal and they'll have empty ActivitySource name associated
with it.

Some common examples of such libraries include [ASP.NET
Core](../../../src/OpenTelemetry.Instrumentation.AspNetCore/README.md), [HTTP
client .NET Core](../../../src/OpenTelemetry.Instrumentation.Http/README.md) .
Instrumentation libraries for these are already provided in this repo. The
[OpenTelemetry .NET
Contrib](https://github.com/open-telemetry/opentelemetry-dotnet-contrib)
repository also has instrumentations for libraries like
[ElasticSearchClient](https://github.com/open-telemetry/opentelemetry-dotnet-contrib/tree/main/src/OpenTelemetry.Contrib.Instrumentation.ElasticsearchClient)
etc. which fall in this category.

If you are writing instrumentation for such library, it is recommended to refer
to one of the above as a reference.

## Processor

OpenTelemetry .NET SDK has provided the following built-in processors:

* [BatchExportProcessor&lt;T&gt;](../../../src/OpenTelemetry/BatchExportProcessor.cs)
* [CompositeProcessor&lt;T&gt;](../../../src/OpenTelemetry/CompositeProcessor.cs)
* [SimpleExportProcessor&lt;T&gt;](../../../src/OpenTelemetry/SimpleExportProcessor.cs)

Custom processors can be implemented to cover more scenarios:

* Processors should inherit from `OpenTelemetry.BaseProcessor<Activity>` (which
  belongs to the [OpenTelemetry](../../../src/OpenTelemetry/README.md) package),
  and implement the `OnStart` and `OnEnd` methods.
* Processors can optionally implement the `OnForceFlush` and `OnShutdown`
  methods. `OnForceFlush` should be thread safe.
* Processors should not throw exceptions from `OnStart`, `OnEnd`, `OnForceFlush`
  and `OnShutdown`.
* `OnStart` and `OnEnd` should be thread safe, and should not block or take long
  time, since they will be called on critical code path.

```csharp
class MyProcessor : BaseProcessor<Activity>
{
    public override void OnStart(Activity activity)
    {
        Console.WriteLine($"OnStart: {activity.DisplayName}");
    }

    public override void OnEnd(Activity activity)
    {
        Console.WriteLine($"OnEnd: {activity.DisplayName}");
    }
}
```

A demo processor is shown [here](./MyProcessor.cs).

### Enriching Processor

A common use case of writing custom processor is to enrich activities with
additional tags. An example of such an "EnrichingProcessor" is shown
[here](./MyEnrichingProcessor.cs). Such processors must be added *before* the
exporters.

This processor also shows how to enrich `Activity` with additional tags from the
`Baggage`.

Many [instrumentation libraries](#instrumentation-library) shipped from this
repo provides a built-in `Enrich` option, which may also be used to enrich
activities. Instrumentation library provided approach may offer additional
capabilities such as offering easy access to more context (library specific).

### Filtering Processor

Another common use case of writing custom processor is to filter Activities from
being exported. Such a "FilteringProcessor" can be written as a wrapper around
an underlying processor. An example "FilteringProcessor" is shown
[here](./MyFilteringProcessor.cs).

When using such a filtering processor, instead of using extension method to
register the exporter, they must be registered manually as shown below:

```csharp
using var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .SetSampler(new MySampler())
    .AddSource("OTel.Demo")
    .AddProcessor(new MyFilteringProcessor(
        new SimpleActivityExportProcessor(new MyExporter("ExporterX")),
        (act) => true))
    .Build();
```

Most [instrumentation libraries](#instrumentation-library) shipped from this
repo provides a built-in `Filter` option to achieve the same effect. In such
cases, it is recommended to use that option as it offers higher performance.

## Sampler

OpenTelemetry .NET SDK has provided the following built-in samplers:

* [AlwaysOffSampler](../../../src/OpenTelemetry/Trace/AlwaysOffSampler.cs)
* [AlwaysOnSampler](../../../src/OpenTelemetry/Trace/AlwaysOnSampler.cs)
* [ParentBasedSampler](../../../src/OpenTelemetry/Trace/ParentBasedSampler.cs)
* [TraceIdRatioBasedSampler](../../../src/OpenTelemetry/Trace/TraceIdRatioBasedSampler.cs)

Custom samplers can be implemented to cover more scenarios:

* Samplers should inherit from `OpenTelemetry.Trace.Sampler` (which belongs to
  the [OpenTelemetry](../../../src/OpenTelemetry/README.md) package), and
  implement the `ShouldSample` method.
* `ShouldSample` should be thread safe, and should not block or take long time,
  since it will be called on critical code path.

```csharp
class MySampler : Sampler
{
    public override SamplingResult ShouldSample(in SamplingParameters samplingParameters)
    {
        return new SamplingResult(SamplingDecision.RecordAndSampled);
    }
}
```

A demo sampler is shown [here](./MySampler.cs).

## Resource Detector

OpenTelemetry .NET SDK provides a resource detector for detecting resource
information from the `OTEL_RESOURCE_ATTRIBUTES` and `OTEL_SERVICE_NAME`
environment variables.

Custom resource detectors can be implemented:

* ResourceDetectors should inherit from
  `OpenTelemetry.Resources.IResourceDetector`, (which belongs to the
  [OpenTelemetry](../../../src/OpenTelemetry/README.md) package), and implement
  the `Detect` method.

A demo ResourceDetector is shown [here](./MyResourceDetector.cs).

## References

* [Exporter
  specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#span-exporter)
* [Processor
  specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#span-processor)
* [Resource
  specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/resource/sdk.md)
* [Sampler
  specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#sampler)
