Mark `service` and `telemetry.sdk` resource attributes as stable. (#3202)

## Changes

This PR marks the set of resource semantic conventions, which a relied
upon in stable API / SDK specification, as stable themselves.

- Add a caveat that changing `service.*` and `telemetry.*` names need
MORE than just schema file changes.
- Mark core Resource attribute semconv document as MIXED stability
- Update key sections for Resource attribute semconv as stable
- Mark `telemetry.sdk` and `service` attribute groups as stable.
- Move `telemetry.auto` and `service.*` "contentious" attributes into
`{section}_experimental.yaml` file. This retains the reserved name, but
does not mark the convention as stable, allowing us to ensure stable
portions of our specification refer to stable semconv.
- Update examples for `service.instance.id` to include an example of the
preferred style instance id before the fallback.

Related issues #3177

## Usage in the Specification

One reason we should mark these attributes as stable is because they are
referenced from numerous stable portions of our specification, and they
are also leveraged from newly stabilizing portions of our specification.

- [Resource
SDK](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/resource/sdk.md#sdk-provided-resource-attributes)
- stable
- [Environment
Variables](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md#general-sdk-configuration)
- stable
- [Jaeger
Exporter](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk_exporters/jaeger.md#resource)
- stable
- [Zipkin
Exporter](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk_exporters/zipkin.md#service-name)
- Stable
- [Log
DataModel](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/data-model.md#rfc5424-syslog)
- Stable
- [Prometheus
compatibility](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/compatibility/prometheus_and_openmetrics.md#resource-attributes)
- still experimental
- [Performance Benchmark
Guidelines](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/performance-benchmark.md)

## A note on ECS compatibility

Here's [ECS service
definition](https://www.elastic.co/guide/en/ecs/current/ecs-service.html).

They include more attributes than we do. The only possible conflict
between us is `service.instance.id` vs. `service.node.name`. In this
case, our `service.instance.id` has a fallback of generating UUID which
is inconsistent w/ `service.node.name` which requires a user-generated
name if another name cannot be provided.

There's an opportunity for us to unify here. However, I don't think
there's appetite in OTEL to require user-specified names in lieu of some
algorithm or generative specification.

semantic_conventions/resource/service.yaml

@@ -16,38 +16,3 @@ groups:
           with [`process.executable.name`](process.md#process), e.g. `unknown_service:bash`.
           If `process.executable.name` is not available, the value MUST be set to `unknown_service`.
         examples: ["shoppingcart"]
-      - id: namespace
-        type: string
-        brief: >
-          A namespace for `service.name`.
-        note: >
-          A string value having a meaning that helps to distinguish a group of services,
-          for example the team name that owns a group of services.
-          `service.name` is expected to be unique within the same namespace.
-          If `service.namespace` is not specified in the Resource then `service.name`
-          is expected to be unique for all services that have no explicit namespace defined
-          (so the empty/unspecified namespace is simply one more valid namespace).
-          Zero-length namespace string is assumed equal to unspecified namespace.
-        examples: ["Shop"]
-      - id: instance.id
-        type: string
-        brief: >
-          The string ID of the service instance.
-        note: >
-          MUST be unique for each instance of the same `service.namespace,service.name` pair
-          (in other words `service.namespace,service.name,service.instance.id` triplet MUST be globally unique).
-          The ID helps to distinguish instances of the same service that exist at the same time
-          (e.g. instances of a horizontally scaled service). It is preferable for the ID to be persistent
-          and stay the same for the lifetime of the service instance, however it is acceptable that
-          the ID is ephemeral and changes during important lifetime events for the service
-          (e.g. service restarts).
-          If the service has no inherent unique ID that can be used as the value of this attribute
-          it is recommended to generate a random Version 1 or Version 4 RFC 4122 UUID
-          (services aiming for reproducible UUIDs may also use Version 5, see RFC 4122
-          for more recommendations).
-        examples: ["627cc493-f310-47de-96bd-71410b7dec09"]
-      - id: version
-        type: string
-        brief: >
-          The version string of the service API or implementation.
-        examples: ["2.0.0"]

semantic_conventions/resource/service_experimental.yaml

@@ -0,0 +1,42 @@
+groups:
+  - id: service_experimental
+    prefix: service
+    type: resource
+    brief: >
+      A service instance.
+    attributes:
+      - id: namespace
+        type: string
+        brief: >
+          A namespace for `service.name`.
+        note: >
+          A string value having a meaning that helps to distinguish a group of services,
+          for example the team name that owns a group of services.
+          `service.name` is expected to be unique within the same namespace.
+          If `service.namespace` is not specified in the Resource then `service.name`
+          is expected to be unique for all services that have no explicit namespace defined
+          (so the empty/unspecified namespace is simply one more valid namespace).
+          Zero-length namespace string is assumed equal to unspecified namespace.
+        examples: ["Shop"]
+      - id: instance.id
+        type: string
+        brief: >
+          The string ID of the service instance.
+        note: >
+          MUST be unique for each instance of the same `service.namespace,service.name` pair
+          (in other words `service.namespace,service.name,service.instance.id` triplet MUST be globally unique).
+          The ID helps to distinguish instances of the same service that exist at the same time
+          (e.g. instances of a horizontally scaled service). It is preferable for the ID to be persistent
+          and stay the same for the lifetime of the service instance, however it is acceptable that
+          the ID is ephemeral and changes during important lifetime events for the service
+          (e.g. service restarts).
+          If the service has no inherent unique ID that can be used as the value of this attribute
+          it is recommended to generate a random Version 1 or Version 4 RFC 4122 UUID
+          (services aiming for reproducible UUIDs may also use Version 5, see RFC 4122
+          for more recommendations).
+        examples: ["my-k8s-pod-deployment-1", "627cc493-f310-47de-96bd-71410b7dec09"]
+      - id: version
+        type: string
+        brief: >
+          The version string of the service API or implementation.
+        examples: ["2.0.0"]

semantic_conventions/resource/telemetry.yaml

@@ -7,6 +7,7 @@ groups:
     attributes:
       - id: sdk.name
         type: string
+        requirement_level: required
         brief: >
           The name of the telemetry SDK as defined above.
         examples: ["opentelemetry"]
@@ -36,15 +37,12 @@ groups:
               value: "webjs"
             - id: swift
               value: "swift"
+        requirement_level: required
         brief: >
           The language of the telemetry SDK.
       - id: sdk.version
         type: string
+        requirement_level: required
         brief: >
           The version string of the telemetry SDK.
         examples: ["1.2.3"]
-      - id: auto.version
-        type: string
-        brief: >
-          The version string of the auto instrumentation agent, if used.
-        examples: ["1.2.3"]

semantic_conventions/resource/telemetry_experimental.yaml

@@ -0,0 +1,12 @@
+groups:
+  - id: telemetry_experimental
+    prefix: telemetry
+    type: resource
+    brief: >
+      The telemetry SDK used to capture data recorded by the instrumentation libraries.
+    attributes:
+      - id: auto.version
+        type: string
+        brief: >
+          The version string of the auto instrumentation agent, if used.
+        examples: ["1.2.3"]

specification/resource/semantic_conventions/README.md

@@ -1,6 +1,6 @@
 # Resource Semantic Conventions
 
-**Status**: [Experimental](../../document-status.md)
+**Status**: [Mixed](../../document-status.md)
 
 This document defines standard attributes for resources. These attributes are typically used in the [Resource](../sdk.md) and are also recommended to be used anywhere else where there is a need to describe a resource in a consistent manner. The majority of these attributes are inherited from
 [OpenCensus Resource standard](https://github.com/census-instrumentation/opencensus-specs/blob/master/resource/StandardResources.md).
@@ -15,7 +15,9 @@ This document defines standard attributes for resources. These attributes are ty
   * [Semantic Attributes with Dedicated Environment Variable](#semantic-attributes-with-dedicated-environment-variable)
 - [Semantic Attributes with SDK-provided Default Value](#semantic-attributes-with-sdk-provided-default-value)
 - [Service](#service)
+- [Service (Experimental)](#service-experimental)
 - [Telemetry SDK](#telemetry-sdk)
+- [Telemetry SDK (Experimental)](#telemetry-sdk-experimental)
 - [Compute Unit](#compute-unit)
 - [Compute Instance](#compute-instance)
 - [Environment](#environment)
@@ -34,12 +36,17 @@ This document defines standard attributes for resources. These attributes are ty
 
 ## Document Conventions
 
+**Status**: [Stable](../../document-status.md)
+
 Attributes are grouped logically by the type of the concept that they described. Attributes in the same group have a common prefix that ends with a dot. For example all attributes that describe Kubernetes properties start with "k8s."
 
-Certain attribute groups in this document have a **Required** column. For these groups if any attribute from the particular group is present in the Resource then all attributes that are marked as Required MUST be also present in the Resource. However it is also valid if the entire attribute group is omitted (i.e. none of the attributes from the particular group are present even though some of them are marked as Required in this document).
+See [Attribute Requirement Levels](../../common/attribute-requirement-level.md) for details on when attributes
+should be included.
 
 ## Attributes with Special Handling
 
+**Status**: [Stable](../../document-status.md)
+
 Given their significance some resource attributes are treated specifically as described below.
 
 ### Semantic Attributes with Dedicated Environment Variable
@@ -55,9 +62,12 @@ These are the attributes which MUST be provided by the SDK
 as specified in the [Resource SDK specification](../sdk.md#sdk-provided-resource-attributes):
 
 - [`service.name`](#service)
+- [`telemetry.sdk` group](#telemetry-sdk)
 
 ## Service
 
+**Status**: [Stable](../../document-status.md)
+
 **type:** `service`
 
 **Description:** A service instance.
@@ -66,15 +76,28 @@ as specified in the [Resource SDK specification](../sdk.md#sdk-provided-resource
 | Attribute  | Type | Description  | Examples  | Requirement Level |
 |---|---|---|---|---|
 | `service.name` | string | Logical name of the service. [1] | `shoppingcart` | Required |
-| `service.namespace` | string | A namespace for `service.name`. [2] | `Shop` | Recommended |
-| `service.instance.id` | string | The string ID of the service instance. [3] | `627cc493-f310-47de-96bd-71410b7dec09` | Recommended |
-| `service.version` | string | The version string of the service API or implementation. | `2.0.0` | Recommended |
 
 **[1]:** MUST be the same for all instances of horizontally scaled services. If the value was not specified, SDKs MUST fallback to `unknown_service:` concatenated with [`process.executable.name`](process.md#process), e.g. `unknown_service:bash`. If `process.executable.name` is not available, the value MUST be set to `unknown_service`.
+<!-- endsemconv -->
 
-**[2]:** A string value having a meaning that helps to distinguish a group of services, for example the team name that owns a group of services. `service.name` is expected to be unique within the same namespace. If `service.namespace` is not specified in the Resource then `service.name` is expected to be unique for all services that have no explicit namespace defined (so the empty/unspecified namespace is simply one more valid namespace). Zero-length namespace string is assumed equal to unspecified namespace.
+## Service (Experimental)
 
-**[3]:** MUST be unique for each instance of the same `service.namespace,service.name` pair (in other words `service.namespace,service.name,service.instance.id` triplet MUST be globally unique). The ID helps to distinguish instances of the same service that exist at the same time (e.g. instances of a horizontally scaled service). It is preferable for the ID to be persistent and stay the same for the lifetime of the service instance, however it is acceptable that the ID is ephemeral and changes during important lifetime events for the service (e.g. service restarts). If the service has no inherent unique ID that can be used as the value of this attribute it is recommended to generate a random Version 1 or Version 4 RFC 4122 UUID (services aiming for reproducible UUIDs may also use Version 5, see RFC 4122 for more recommendations).
+**Status**: [Experimental](../../document-status.md)
+
+**type:** `service`
+
+**Description:** Additions to service instance.
+
+<!-- semconv service_experimental -->
+| Attribute  | Type | Description  | Examples  | Requirement Level |
+|---|---|---|---|---|
+| `service.namespace` | string | A namespace for `service.name`. [1] | `Shop` | Recommended |
+| `service.instance.id` | string | The string ID of the service instance. [2] | `my-k8s-pod-deployment-1`; `627cc493-f310-47de-96bd-71410b7dec09` | Recommended |
+| `service.version` | string | The version string of the service API or implementation. | `2.0.0` | Recommended |
+
+**[1]:** A string value having a meaning that helps to distinguish a group of services, for example the team name that owns a group of services. `service.name` is expected to be unique within the same namespace. If `service.namespace` is not specified in the Resource then `service.name` is expected to be unique for all services that have no explicit namespace defined (so the empty/unspecified namespace is simply one more valid namespace). Zero-length namespace string is assumed equal to unspecified namespace.
+
+**[2]:** MUST be unique for each instance of the same `service.namespace,service.name` pair (in other words `service.namespace,service.name,service.instance.id` triplet MUST be globally unique). The ID helps to distinguish instances of the same service that exist at the same time (e.g. instances of a horizontally scaled service). It is preferable for the ID to be persistent and stay the same for the lifetime of the service instance, however it is acceptable that the ID is ephemeral and changes during important lifetime events for the service (e.g. service restarts). If the service has no inherent unique ID that can be used as the value of this attribute it is recommended to generate a random Version 1 or Version 4 RFC 4122 UUID (services aiming for reproducible UUIDs may also use Version 5, see RFC 4122 for more recommendations).
 <!-- endsemconv -->
 
 Note: `service.namespace` and `service.name` are not intended to be concatenated for the purpose of forming a single globally unique name for the service. For example the following 2 sets of attributes actually describe 2 different services (despite the fact that the concatenation would result in the same string):
@@ -93,6 +116,8 @@ service.name = Shop.shoppingcart
 
 ## Telemetry SDK
 
+**Status**: [Stable](../../document-status.md)
+
 **type:** `telemetry.sdk`
 
 **Description:** The telemetry SDK used to capture data recorded by the instrumentation libraries.
@@ -104,15 +129,14 @@ If another SDK, like a fork or a vendor-provided implementation, is used, this S
 `telemetry.sdk.name` to the fully-qualified class or module name of this SDK's main entry point
 or another suitable identifier depending on the language.
 The identifier `opentelemetry` is reserved and MUST NOT be used in this case.
-The identifier SHOULD be stable across different versions of an implementation.
+All custom identifiers SHOULD be stable across different versions of an implementation.
 
 <!-- semconv telemetry -->
 | Attribute  | Type | Description  | Examples  | Requirement Level |
 |---|---|---|---|---|
-| `telemetry.sdk.name` | string | The name of the telemetry SDK as defined above. | `opentelemetry` | Recommended |
+| `telemetry.sdk.name` | string | The name of the telemetry SDK as defined above. | `opentelemetry` | Required |
-| `telemetry.sdk.language` | string | The language of the telemetry SDK. | `cpp` | Recommended |
+| `telemetry.sdk.language` | string | The language of the telemetry SDK. | `cpp` | Required |
-| `telemetry.sdk.version` | string | The version string of the telemetry SDK. | `1.2.3` | Recommended |
+| `telemetry.sdk.version` | string | The version string of the telemetry SDK. | `1.2.3` | Required |
-| `telemetry.auto.version` | string | The version string of the auto instrumentation agent, if used. | `1.2.3` | Recommended |
 
 `telemetry.sdk.language` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
 
@@ -131,8 +155,24 @@ The identifier SHOULD be stable across different versions of an implementation.
 | `swift` | swift |
 <!-- endsemconv -->
 
+## Telemetry SDK (Experimental)
+
+**Status**: [Experimental](../../document-status.md)
+
+**type:** `telemetry.sdk`
+
+**Description:** Additions to the telemetry SDK.
+
+<!-- semconv telemetry_experimental -->
+| Attribute  | Type | Description  | Examples  | Requirement Level |
+|---|---|---|---|---|
+| `telemetry.auto.version` | string | The version string of the auto instrumentation agent, if used. | `1.2.3` | Recommended |
+<!-- endsemconv -->
+
 ## Compute Unit
 
+**Status**: [Experimental](../../document-status.md)
+
 Attributes defining a compute unit (e.g. Container, Process, Function as a Service):
 
 - [Container](./container.md)
@@ -142,12 +182,16 @@ Attributes defining a compute unit (e.g. Container, Process, Function as a Servi
 
 ## Compute Instance
 
+**Status**: [Experimental](../../document-status.md)
+
 Attributes defining a computing instance (e.g. host):
 
 - [Host](./host.md)
 
 ## Environment
 
+**Status**: [Experimental](../../document-status.md)
+
 Attributes defining a running environment (e.g. Operating System, Cloud, Data Center, Deployment Service):
 
 - [Operating System](./os.md)
@@ -160,12 +204,16 @@ Attributes defining a running environment (e.g. Operating System, Cloud, Data Ce
 
 ## Version attributes
 
+**Status**: [Stable](../../document-status.md)
+
 Version attributes, such as `service.version`, are values of type `string`. They are
 the exact version used to identify an artifact. This may be a semantic version, e.g., `1.2.3`, git hash, e.g.,
 `8ae73a`, or an arbitrary version string, e.g., `0.1.2.20210101`, whatever was used when building the artifact.
 
 ## Cloud-Provider-Specific Attributes
 
+**Status**: [Experimental](../../document-status.md)
+
 Attributes that are only applicable to resources from a specific cloud provider. Currently, these
 resources can only be defined for providers listed as a valid `cloud.provider` in
 [Cloud](./cloud.md) and below. Provider-specific attributes all reside in the `cloud_provider` directory.