open-telemetry
/
semantic-conventions
mirror of https://github.com/open-telemetry/semantic-conventions.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
semantic-conventions/docs/cicd/cicd-metrics.md

49 KiB
Raw Blame History

Semantic Conventions for CICD Metrics

Status: Experimental

CICD Metrics

The conventions described in this section are specific to Continuous Integration / Continuous Deployment (CICD) systems.

Disclaimer: These are initial CICD metrics and attributes but more may be added in the future.

Metric: cicd.pipeline.run.duration

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
cicd.pipeline.run.duration Histogram s Duration of a pipeline run grouped by pipeline, state and result. Experimental
Attribute Type Description Examples Requirement Level Stability
cicd.pipeline.name string The human readable name of the pipeline within a CI/CD system. Build and Test; Lint; Deploy Go Project; deploy_to_environment Required Experimental
cicd.pipeline.run.state string The pipeline run goes through these states during its lifecycle. pending; executing; finalizing Required Experimental
cicd.pipeline.result string The result of a pipeline run. success; failure; timeout; skipped Conditionally Required [1] Experimental
error.type string Describes a class of error the operation ended with. [2] timeout; java.net.UnknownHostException; server_certificate_invalid; 500 Conditionally Required If and only if the pipeline run failed. Stable

[1] cicd.pipeline.result: If and only if the pipeline run result has been set during that state.

[2] error.type: The error.type SHOULD be predictable, and SHOULD have low cardinality.

When error.type is set to a type (e.g., an exception type), its canonical class name identifying the type within the artifact SHOULD be used.

Instrumentations SHOULD document the list of errors they report.

The cardinality of error.type within one instrumentation library SHOULD be low. Telemetry consumers that aggregate data from multiple instrumentation libraries and applications should be prepared for error.type to have high cardinality at query time when no additional filters are applied.

If the operation has completed successfully, instrumentations SHOULD NOT set error.type.

If a specific domain defines its own set of error identifiers (such as HTTP or gRPC status codes), it's RECOMMENDED to:

  • Use a domain-specific attribute
  • Set error.type to capture all errors, regardless of whether they are defined within the domain-specific set or not.

cicd.pipeline.result has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
cancellation The pipeline run was cancelled, eg. by a user manually cancelling the pipeline run. Experimental
error The pipeline run failed due to an error in the CICD system, eg. due to the worker being killed. Experimental
failure The pipeline run did not finish successfully, eg. due to a compile error or a failing test. Such failures are usually detected by non-zero exit codes of the tools executed in the pipeline run. Experimental
skip The pipeline run was skipped, eg. due to a precondition not being met. Experimental
success The pipeline run finished successfully. Experimental
timeout A timeout caused the pipeline run to be interrupted. Experimental

cicd.pipeline.run.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
executing The executing state spans the execution of any run tasks (eg. build, test). Experimental
finalizing The finalizing state spans from when the run has finished executing (eg. cleanup of run resources). Experimental
pending The run pending state spans from the event triggering the pipeline run until the execution of the run starts (eg. time spent in a queue, provisioning agents, creating run resources). Experimental

error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
_OTHER A fallback error value to be used when the instrumentation doesn't define a custom value. Stable

Metric: cicd.pipeline.run.active

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
cicd.pipeline.run.active UpDownCounter {run} The number of pipeline runs currently active in the system by state. Experimental
Attribute Type Description Examples Requirement Level Stability
cicd.pipeline.name string The human readable name of the pipeline within a CI/CD system. Build and Test; Lint; Deploy Go Project; deploy_to_environment Required Experimental
cicd.pipeline.run.state string The pipeline run goes through these states during its lifecycle. pending; executing; finalizing Required Experimental

cicd.pipeline.run.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
executing The executing state spans the execution of any run tasks (eg. build, test). Experimental
finalizing The finalizing state spans from when the run has finished executing (eg. cleanup of run resources). Experimental
pending The run pending state spans from the event triggering the pipeline run until the execution of the run starts (eg. time spent in a queue, provisioning agents, creating run resources). Experimental

Metric: cicd.worker.count

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
cicd.worker.count UpDownCounter {count} The number of workers on the CICD system by state. Experimental
Attribute Type Description Examples Requirement Level Stability
cicd.worker.state string The state of a CICD worker / agent. idle; busy; down Required Experimental

cicd.worker.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
available The worker is not performing work for the CICD system. It is available to the CICD system to perform work on (online / idle). [1] Experimental
busy The worker is performing work for the CICD system. Experimental
offline The worker is not available to the CICD system (disconnected / down). Experimental

[1]: Pipelines might have conditions on which workers they are able to run so not every worker might be available to every pipeline.

Metric: cicd.pipeline.run.errors

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
cicd.pipeline.run.errors Counter {error} The number of errors encountered in pipeline runs (eg. compile, test failures). [1] Experimental

[1]: There might be errors in a pipeline run that are non fatal (eg. they are suppressed) or in a parallel stage multiple stages could have a fatal error. This means that this error count might not be the same as the count of metric cicd.pipeline.run.duration with run result failure.

Attribute Type Description Examples Requirement Level Stability
cicd.pipeline.name string The human readable name of the pipeline within a CI/CD system. Build and Test; Lint; Deploy Go Project; deploy_to_environment Required Experimental
error.type string Describes a class of error the operation ended with. [1] timeout; java.net.UnknownHostException; server_certificate_invalid; 500 Required Stable

[1] error.type: The error.type SHOULD be predictable, and SHOULD have low cardinality.

When error.type is set to a type (e.g., an exception type), its canonical class name identifying the type within the artifact SHOULD be used.

Instrumentations SHOULD document the list of errors they report.

The cardinality of error.type within one instrumentation library SHOULD be low. Telemetry consumers that aggregate data from multiple instrumentation libraries and applications should be prepared for error.type to have high cardinality at query time when no additional filters are applied.

If the operation has completed successfully, instrumentations SHOULD NOT set error.type.

If a specific domain defines its own set of error identifiers (such as HTTP or gRPC status codes), it's RECOMMENDED to:

  • Use a domain-specific attribute
  • Set error.type to capture all errors, regardless of whether they are defined within the domain-specific set or not.

error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
_OTHER A fallback error value to be used when the instrumentation doesn't define a custom value. Stable

Metric: cicd.system.errors

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
cicd.system.errors Counter {error} The number of errors in a component of the CICD system (eg. controller, scheduler, agent). [1] Experimental

[1]: Errors in pipeline run execution are explicitly excluded. Ie a test failure is not counted in this metric.

Attribute Type Description Examples Requirement Level Stability
cicd.system.component string The name of a component of the CICD system. controller; scheduler; agent Required Experimental
error.type string Describes a class of error the operation ended with. [1] timeout; java.net.UnknownHostException; server_certificate_invalid; 500 Required Stable

[1] error.type: The error.type SHOULD be predictable, and SHOULD have low cardinality.

When error.type is set to a type (e.g., an exception type), its canonical class name identifying the type within the artifact SHOULD be used.

Instrumentations SHOULD document the list of errors they report.

The cardinality of error.type within one instrumentation library SHOULD be low. Telemetry consumers that aggregate data from multiple instrumentation libraries and applications should be prepared for error.type to have high cardinality at query time when no additional filters are applied.

If the operation has completed successfully, instrumentations SHOULD NOT set error.type.

If a specific domain defines its own set of error identifiers (such as HTTP or gRPC status codes), it's RECOMMENDED to:

  • Use a domain-specific attribute
  • Set error.type to capture all errors, regardless of whether they are defined within the domain-specific set or not.

error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
_OTHER A fallback error value to be used when the instrumentation doesn't define a custom value. Stable

VCS Metrics

The conventions described in this section are specific to Version Control Systems.

Disclaimer: These are initial VCS metrics and attributes but more may be added in the future.

Metric: vcs.change.count

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
vcs.change.count UpDownCounter {change} The number of changes (pull requests/merge requests/changelists) in a repository, categorized by their state (e.g. open or merged) Experimental
Attribute Type Description Examples Requirement Level Stability
vcs.change.state string The state of the change (pull request/merge request/changelist). open; closed; merged Required Experimental
vcs.repository.url.full string The canonical URL of the repository providing the complete HTTP(S) address in order to locate and identify the repository through a browser. [1] https://github.com/opentelemetry/open-telemetry-collector-contrib; https://gitlab.com/my-org/my-project/my-projects-project/repo Required Experimental
vcs.repository.name string The human readable name of the repository. It SHOULD NOT include any additional identifier like Group/SubGroup in GitLab or organization in GitHub. [2] semantic-conventions; my-cool-repo Recommended Experimental

[1] vcs.repository.url.full: In Git Version Control Systems, the canonical URL SHOULD NOT include the .git extension.

[2] vcs.repository.name: Due to it only being the name, it can clash with forks of the same repository if collecting telemetry across multiple orgs or groups in the same backends.

vcs.change.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
closed Closed means the merge request has been closed without merging. This can happen for various reasons, such as the changes being deemed unnecessary, the issue being resolved in another way, or the author deciding to withdraw the request. Experimental
merged Merged indicates that the change has been successfully integrated into the target codebase. Experimental
open Open means the change is currently active and under review. It hasn't been merged into the target branch yet, and it's still possible to make changes or add comments. Experimental
wip WIP (work-in-progress, draft) means the change is still in progress and not yet ready for a full review. It might still undergo significant changes. Experimental

Metric: vcs.change.duration

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
vcs.change.duration Gauge s The time duration a change (pull request/merge request/changelist) has been in a given state. Experimental
Attribute Type Description Examples Requirement Level Stability
vcs.change.state string The state of the change (pull request/merge request/changelist). open; closed; merged Required Experimental
vcs.ref.head.name string The name of the reference such as branch or tag in the repository. my-feature-branch; tag-1-test Required Experimental
vcs.repository.url.full string The canonical URL of the repository providing the complete HTTP(S) address in order to locate and identify the repository through a browser. [1] https://github.com/opentelemetry/open-telemetry-collector-contrib; https://gitlab.com/my-org/my-project/my-projects-project/repo Required Experimental
vcs.repository.name string The human readable name of the repository. It SHOULD NOT include any additional identifier like Group/SubGroup in GitLab or organization in GitHub. [2] semantic-conventions; my-cool-repo Recommended Experimental

[1] vcs.repository.url.full: In Git Version Control Systems, the canonical URL SHOULD NOT include the .git extension.

[2] vcs.repository.name: Due to it only being the name, it can clash with forks of the same repository if collecting telemetry across multiple orgs or groups in the same backends.

vcs.change.state has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
closed Closed means the merge request has been closed without merging. This can happen for various reasons, such as the changes being deemed unnecessary, the issue being resolved in another way, or the author deciding to withdraw the request. Experimental
merged Merged indicates that the change has been successfully integrated into the target codebase. Experimental
open Open means the change is currently active and under review. It hasn't been merged into the target branch yet, and it's still possible to make changes or add comments. Experimental
wip WIP (work-in-progress, draft) means the change is still in progress and not yet ready for a full review. It might still undergo significant changes. Experimental

Metric: vcs.change.time_to_approval

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
vcs.change.time_to_approval Gauge s The amount of time since its creation it took a change (pull request/merge request/changelist) to get the first approval Experimental
Attribute Type Description Examples Requirement Level Stability
vcs.ref.head.name string The name of the reference such as branch or tag in the repository. my-feature-branch; tag-1-test Required Experimental
vcs.repository.url.full string The canonical URL of the repository providing the complete HTTP(S) address in order to locate and identify the repository through a browser. [1] https://github.com/opentelemetry/open-telemetry-collector-contrib; https://gitlab.com/my-org/my-project/my-projects-project/repo Required Experimental
vcs.repository.name string The human readable name of the repository. It SHOULD NOT include any additional identifier like Group/SubGroup in GitLab or organization in GitHub. [2] semantic-conventions; my-cool-repo Recommended Experimental

[1] vcs.repository.url.full: In Git Version Control Systems, the canonical URL SHOULD NOT include the .git extension.

[2] vcs.repository.name: Due to it only being the name, it can clash with forks of the same repository if collecting telemetry across multiple orgs or groups in the same backends.

Metric: vcs.repository.count

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
vcs.repository.count UpDownCounter {repository} The number of repositories in an organization Experimental

Metric: vcs.ref.count

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
vcs.ref.count UpDownCounter {ref} The number of refs of type branch or tag in a repository Experimental
Attribute Type Description Examples Requirement Level Stability
vcs.ref.type string The type of the reference in the repository. branch; tag Required Experimental
vcs.repository.url.full string The canonical URL of the repository providing the complete HTTP(S) address in order to locate and identify the repository through a browser. [1] https://github.com/opentelemetry/open-telemetry-collector-contrib; https://gitlab.com/my-org/my-project/my-projects-project/repo Required Experimental
vcs.repository.name string The human readable name of the repository. It SHOULD NOT include any additional identifier like Group/SubGroup in GitLab or organization in GitHub. [2] semantic-conventions; my-cool-repo Recommended Experimental

[1] vcs.repository.url.full: In Git Version Control Systems, the canonical URL SHOULD NOT include the .git extension.

[2] vcs.repository.name: Due to it only being the name, it can clash with forks of the same repository if collecting telemetry across multiple orgs or groups in the same backends.

vcs.ref.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
branch branch Experimental
tag tag Experimental

Metric: vcs.ref.lines_delta

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
vcs.ref.lines_delta Gauge {line} The number of lines added/removed in a ref (branch) relative to the ref from the vcs.ref.base.name attribute [1] Experimental

[1]: This metric should be reported for each vcs.line_change.type value. For example if a ref added 3 lines and removed 2 lines, instrumentation SHOULD report two measurements: 3 and 2 (both positive numbers). If number of lines added/removed should be calculated from the start of time, then vcs.ref.base.name SHOULD be set to an empty string.

Attribute Type Description Examples Requirement Level Stability
vcs.line_change.type string The type of line change being measured on a branch or change. added; removed Required Experimental
vcs.ref.base.name string The name of the reference such as branch or tag in the repository. my-feature-branch; tag-1-test Required Experimental
vcs.ref.base.type string The type of the reference in the repository. branch; tag Required Experimental
vcs.ref.head.name string The name of the reference such as branch or tag in the repository. my-feature-branch; tag-1-test Required Experimental
vcs.ref.head.type string The type of the reference in the repository. branch; tag Required Experimental
vcs.repository.url.full string The canonical URL of the repository providing the complete HTTP(S) address in order to locate and identify the repository through a browser. [1] https://github.com/opentelemetry/open-telemetry-collector-contrib; https://gitlab.com/my-org/my-project/my-projects-project/repo Required Experimental
vcs.change.id string The ID of the change (pull request/merge request/changelist) if applicable. This is usually a unique (within repository) identifier generated by the VCS system. 123 Conditionally Required if a change is associate with the ref. Experimental
vcs.repository.name string The human readable name of the repository. It SHOULD NOT include any additional identifier like Group/SubGroup in GitLab or organization in GitHub. [2] semantic-conventions; my-cool-repo Recommended Experimental

[1] vcs.repository.url.full: In Git Version Control Systems, the canonical URL SHOULD NOT include the .git extension.

[2] vcs.repository.name: Due to it only being the name, it can clash with forks of the same repository if collecting telemetry across multiple orgs or groups in the same backends.

vcs.line_change.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
added How many lines were added. Experimental
removed How many lines were removed. Experimental

vcs.ref.base.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
branch branch Experimental
tag tag Experimental

vcs.ref.head.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
branch branch Experimental
tag tag Experimental

Metric: vcs.ref.revisions_delta

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
vcs.ref.revisions_delta Gauge {revision} The number of revisions (commits) a ref (branch) is ahead/behind the branch from the vcs.ref.base.name attribute [1] Experimental

[1]: This metric should be reported for each vcs.revision_delta.direction value. For example if branch a is 3 commits behind and 2 commits ahead of trunk, instrumentation SHOULD report two measurements: 3 and 2 (both positive numbers) and vcs.ref.base.name is set to trunk.

Attribute Type Description Examples Requirement Level Stability
vcs.ref.base.name string The name of the reference such as branch or tag in the repository. my-feature-branch; tag-1-test Required Experimental
vcs.ref.base.type string The type of the reference in the repository. branch; tag Required Experimental
vcs.ref.head.name string The name of the reference such as branch or tag in the repository. my-feature-branch; tag-1-test Required Experimental
vcs.ref.head.type string The type of the reference in the repository. branch; tag Required Experimental
vcs.repository.url.full string The canonical URL of the repository providing the complete HTTP(S) address in order to locate and identify the repository through a browser. [1] https://github.com/opentelemetry/open-telemetry-collector-contrib; https://gitlab.com/my-org/my-project/my-projects-project/repo Required Experimental
vcs.revision_delta.direction string The type of revision comparison. ahead; behind Required Experimental
vcs.change.id string The ID of the change (pull request/merge request/changelist) if applicable. This is usually a unique (within repository) identifier generated by the VCS system. 123 Conditionally Required if a change is associate with the ref. Experimental
vcs.repository.name string The human readable name of the repository. It SHOULD NOT include any additional identifier like Group/SubGroup in GitLab or organization in GitHub. [2] semantic-conventions; my-cool-repo Recommended Experimental

[1] vcs.repository.url.full: In Git Version Control Systems, the canonical URL SHOULD NOT include the .git extension.

[2] vcs.repository.name: Due to it only being the name, it can clash with forks of the same repository if collecting telemetry across multiple orgs or groups in the same backends.

vcs.ref.base.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
branch branch Experimental
tag tag Experimental

vcs.ref.head.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
branch branch Experimental
tag tag Experimental

vcs.revision_delta.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
ahead How many revisions the change is ahead of the target ref. Experimental
behind How many revisions the change is behind the target ref. Experimental

Metric: vcs.ref.time

This metric is recommended.

Name Instrument Type Unit (UCUM) Description Stability
vcs.ref.time Gauge s Time a ref (branch) created from the default branch (trunk) has existed. The ref.type attribute will always be branch Experimental
Attribute Type Description Examples Requirement Level Stability
vcs.ref.head.name string The name of the reference such as branch or tag in the repository. my-feature-branch; tag-1-test Required Experimental
vcs.ref.head.type string The type of the reference in the repository. branch; tag Required Experimental
vcs.repository.url.full string The canonical URL of the repository providing the complete HTTP(S) address in order to locate and identify the repository through a browser. [1] https://github.com/opentelemetry/open-telemetry-collector-contrib; https://gitlab.com/my-org/my-project/my-projects-project/repo Required Experimental
vcs.repository.name string The human readable name of the repository. It SHOULD NOT include any additional identifier like Group/SubGroup in GitLab or organization in GitHub. [2] semantic-conventions; my-cool-repo Recommended Experimental

[1] vcs.repository.url.full: In Git Version Control Systems, the canonical URL SHOULD NOT include the .git extension.

[2] vcs.repository.name: Due to it only being the name, it can clash with forks of the same repository if collecting telemetry across multiple orgs or groups in the same backends.

vcs.ref.head.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value Description Stability
branch branch Experimental
tag tag Experimental

Metric: vcs.contributor.count

This metric is opt-in.

Name Instrument Type Unit (UCUM) Description Stability
vcs.contributor.count Gauge {contributor} The number of unique contributors to a repository Experimental
Attribute Type Description Examples Requirement Level Stability
vcs.repository.url.full string The canonical URL of the repository providing the complete HTTP(S) address in order to locate and identify the repository through a browser. [1] https://github.com/opentelemetry/open-telemetry-collector-contrib; https://gitlab.com/my-org/my-project/my-projects-project/repo Required Experimental
vcs.repository.name string The human readable name of the repository. It SHOULD NOT include any additional identifier like Group/SubGroup in GitLab or organization in GitHub. [2] semantic-conventions; my-cool-repo Recommended Experimental

[1] vcs.repository.url.full: In Git Version Control Systems, the canonical URL SHOULD NOT include the .git extension.

[2] vcs.repository.name: Due to it only being the name, it can clash with forks of the same repository if collecting telemetry across multiple orgs or groups in the same backends.