# Maintaining dependencies In general, several places in the code base need to be adjusted when upgrading a dependency to a new version. This documentation contains notes about which place to fix, to make maintenance easier and less error prone. This doc is only useful when up to date, so make sure to add details about missing parts if any. Also, another good place to start when upgrading something to N+1 is to find the commit that upgraded to version N (use `git blame`), and inspect the commit for the last upgrade. ## Overview | Dependency | Type | | -------------------- | ------------- | | opentelemetry-proto | git submodule | | prometheus-cpp | git submodule | ## opentelemetry-proto ### Comments (opentelemetry-proto) Unlike other opentelemetry SIGs, opentelemetry-cpp generates code from opentelemetry-proto as part of the opentelemetry-cpp build. Only the source code of opentelemetry-proto is required, which is why this repository is used as a git submodule under third_party. Code is generated by the protobuf compiler during the build, so that generated code is never checked in. This allows more flexibility, should the compiler (protobuf) be upgraded even when the source code (opentelemetry-proto) is unchanged. ### Origin (opentelemetry-proto) The repository for opentelemetry-proto is: * [repository](https://github.com/open-telemetry/opentelemetry-proto) Check release notes at: * [release-notes](https://github.com/open-telemetry/opentelemetry-proto/releases) ### Upgrade (opentelemetry-proto) When upgrading opentelemetry-proto to a newer release, a few places in the code need adjustment. In this example, we upgrade from 1.3.1 to 1.3.2 #### directory third_party/opentelemetry-proto You need to update the git submodule third_party/opentelemetry-proto, check [Upgrade a git submodule](#upgrade-a-git-submodule) for more details. #### file third_party_release Update the line pointing to the opentelemetry-proto tag. ```text opentelemetry-proto=v1.3.2 ``` Typical change: ```shell [malff@malff-desktop opentelemetry-cpp]$ git diff third_party_release diff --git a/third_party_release b/third_party_release index 0bbf67f3..7362473f 100644 --- a/third_party_release +++ b/third_party_release @@ -19,7 +19,7 @@ benchmark=v1.8.3 googletest=1.14.0 ms-gsl=v3.1.0-67-g6f45293 nlohmann-json=v3.11.3 -opentelemetry-proto=v1.3.1 +opentelemetry-proto=v1.3.2 opentracing-cpp=v1.6.0 prometheus-cpp=v1.2.4 vcpkg=2024.02.14 ``` #### file bazel/repository.bzl Please follow the guide [Upgrade a bazel dependency](#upgrade-a-bazel-dependency) for more details. #### file cmake/opentelemetry-proto.cmake Update the tag in the CMake logic: ```cmake set(opentelemetry-proto "v1.3.2") ``` Typical change: ```shell [malff@malff-desktop opentelemetry-cpp]$ git diff cmake/opentelemetry-proto.cmake diff --git a/cmake/opentelemetry-proto.cmake b/cmake/opentelemetry-proto.cmake index 19516c3b..dd6213c1 100644 --- a/cmake/opentelemetry-proto.cmake +++ b/cmake/opentelemetry-proto.cmake @@ -49,7 +49,7 @@ else() "opentelemetry-proto=[ \\t]*([A-Za-z0-9_\\.\\-]+)") set(opentelemetry-proto "${CMAKE_MATCH_1}") else() - set(opentelemetry-proto "v1.3.1") + set(opentelemetry-proto "v1.3.2") endif() unset(OTELCPP_THIRD_PARTY_RELEASE_CONTENT) endif() ``` If opentelemetry-proto contains new files, the makefile needs to be adjusted to build the new code. Depending on the actual changes in the opentelemetry-proto files, the C++ code in opentelemetry-cpp may need adjustments. Typically, when opentelemetry-proto: * defines a new message (for example, for profiling) * adds new optional fields to an existing message (for example, trace flags) the existing C++ code should work as is with the new opentelemetry-proto format. In this case, it is better to: * upgrade the opentelemetry-proto version independently with one PR. * upgrade the C++ code later (to use the new message, of provide new fields) in a different PR. When the C++ code requires a newer minimum version of opentelemetry-proto, make sure to document this, including in the release notes. ### Known issues (opentelemetry-proto) For bazel, two different methods to build exists. First, the code can build using file `bazel/repository.bzl`. This option does not depend on bazel central. Secondly, there is also a build using modules, with file `MODULE.bazel`. This option does depend on bazel central, and CI depends on it. ## semantic-conventions and build-tools ### Comments (semantic-conventions) Some code in opentelemetry-cpp is generated automatically, namely files: * api/include/opentelemetry/trace/semantic_conventions.h * sdk/include/opentelemetry/sdk/resource/semantic_conventions.h The semantic conventions C++ declarations are generated using: * data represented in yaml ("semantic-conventions") * a code generator ("build-tools") This generation is not done as part of the build, it is done once by maintainers, and the generated code is added (checked in) in the opentelemetry-cpp repository. ### Origin (semantic-conventions) The repository for semantic-conventions is: * [repository](https://github.com/open-telemetry/semantic-conventions) Check release notes at: * [release-notes](https://github.com/open-telemetry/semantic-conventions/releases) The repository for build-tools is: * [repository](https://github.com/open-telemetry/build-tools) Check release notes at: * [release-notes](https://github.com/open-telemetry/build-tools/releases) Semantic conventions and build-tools works together, make sure to use the proper version of build-tools that is required to use a given version of semantic-conventions. ### Upgrade (semantic-conventions) When upgrading semantic-conventions to a newer release, a few places in the code need adjustment. In this example, we upgrade from semantic-conventions 1.26.0 to 1.27.0 In this case, semantic-conventions 1.27.0 also require a new version of build-tools, because the yaml format for the data changed. In this example, we upgrade from build-tools 0.24.0 to 0.25.0 #### file buildscripts/semantic-convention/generate.sh Update the line pointing to the semantic-conventions tag. ```text SEMCONV_VERSION=1.27.0 ``` Update the line pointing to the build-tools tag. ```text GENERATOR_VERSION=0.25.0 ``` Typical change: ```shell [malff@malff-desktop opentelemetry-cpp]$ git diff buildscripts/semantic-convention/generate.sh diff --git a/buildscripts/semantic-convention/generate.sh b/buildscripts/semantic-convention/generate.sh index 2bcd07e2..8ad3292e 100755 --- a/buildscripts/semantic-convention/generate.sh +++ b/buildscripts/semantic-convention/generate.sh @@ -19,10 +19,10 @@ ROOT_DIR="${SCRIPT_DIR}/../../" # https://github.com/open-telemetry/opentelemetry-specification # Repository from 1.21.0: # https://github.com/open-telemetry/semantic-conventions -SEMCONV_VERSION=1.26.0 +SEMCONV_VERSION=1.27.0 # repository: https://github.com/open-telemetry/build-tools -GENERATOR_VERSION=0.24.0 +GENERATOR_VERSION=0.25.0 SPEC_VERSION=v$SEMCONV_VERSION SCHEMA_URL=https://opentelemetry.io/schemas/$SEMCONV_VERSION ``` This change alone does nothing, the next step is to execute the generate.sh script. If generation is successful, the generated code contains the new schema URL: ```shell [malff@malff-desktop opentelemetry-cpp]$ find . -name semantic_conventions.h ./api/include/opentelemetry/trace/semantic_conventions.h ./sdk/include/opentelemetry/sdk/resource/semantic_conventions.h [malff@malff-desktop opentelemetry-cpp]$ grep kSchemaUrl ./api/include/opentelemetry/trace/semantic_conventions.h static constexpr const char *kSchemaUrl = "https://opentelemetry.io/schemas/1.27.0"; [malff@malff-desktop opentelemetry-cpp]$ grep kSchemaUrl ./sdk/include/opentelemetry/sdk/resource/semantic_conventions.h static constexpr const char *kSchemaUrl = "https://opentelemetry.io/schemas/1.27.0"; ``` Apply clang-format on the generated code, and check-in changes. ### Known issues (semantic-conventions) Depending on tooling changes, the generate.sh script may need adjustments. Depending on changes in code generation, the template used to generate code may need adjustments. This template is implemented in file `buildscripts/semantic-convention/templates/SemanticAttributes.h.j2`. Last, in some special case like name collisions for a given symbol, the template itself may need to be adjusted for special logic. See for example how `messaging.client_id` is treated. ## prometheus-cpp ### Comments (prometheus-cpp) The `prometheus-cpp` library provides a C++ client for Prometheus, facilitating the creation and registration of metrics that Prometheus scrapes. `prometheus-cpp` is used as a git submodule under the `third_party` directory for ease of inclusion in build system. ### Origin (prometheus-cpp) The repository for `prometheus-cpp` can be found here: * [repository](https://github.com/jupp0r/prometheus-cpp) Check release notes at: * [release-notes](https://github.com/jupp0r/prometheus-cpp/releases) ### Upgrade (prometheus-cpp) When upgrading `prometheus-cpp` to a newer release, you’ll need to update a few key files in the codebase to reflect the new version. In this example, we upgrade from `v1.2.3` to `v1.2.4`. #### Directory `third_party/prometheus-cpp` `prometheus-cpp` is a `git submodule`, so it needs to be pointed to the new release tag. ```shell cd third_party/prometheus-cpp git log -1 ``` The current submodule should show something like: ```shell commit abcdef1234567890abcdef1234567890abcdef12 (HEAD, tag: v1.2.3) Author: John Doe Date: Fri Apr 25 17:55:35 2024 +0200 Minor fixes for performance and compatibility ``` Pull new tags: ```shell git pull --tag origin ``` Upgrade to the new tag: ```shell git pull origin v1.2.4 ``` Verify the new commit: ```shell git log -1 commit 1234567890abcdef1234567890abcdef12345678 (HEAD, tag: v1.2.4) Author: Jane Doe Date: Thu Jun 28 08:19:11 2024 -0500 Improved metrics handling for high concurrency ``` Return to the root directory: ```shell cd ../.. git status ``` The status should display: ```shell On branch upgrade_prometheus_1.2.4 Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git restore ..." to discard changes in working directory) modified: third_party/prometheus-cpp (new commits) ``` Add the upgraded submodule: ```shell git add third_party/prometheus-cpp ``` File third_party_release Update the line referencing the prometheus-cpp version. ```shell prometheus-cpp=v1.2.4 ``` Example change: ```shell $ git diff third_party_release diff --git a/third_party_release b/third_party_release index abc1234..def5678 100644 --- a/third_party_release +++ b/third_party_release @@ -19,7 +19,7 @@ some-dependency=v0.8.3 another-dependency=1.14.0 prometheus-cpp=v1.2.3 +prometheus-cpp=v1.2.4 ``` In file bazel/repository.bzl locate the entry for prometheus-cpp: ```shell # C++ Prometheus Client library. maybe( http_archive, name = "com_github_jupp0r_prometheus_cpp", sha256 = "ac6e958405a29fbbea9db70b00fa3c420e16ad32e1baf941ab233ba031dd72ee", strip_prefix = "prometheus-cpp-1.2.3", urls = [ "https://github.com/jupp0r/prometheus-cpp/archive/refs/tags/v1.2.3.tar.gz", ], ) ``` Update the URL to the new tag: ```shell urls = [ "https://github.com/jupp0r/prometheus-cpp/archive/v1.2.4.tar.gz", ], ``` Update strip_prefix to match the new version: ```shell strip_prefix = "prometheus-cpp-1.2.4", ``` Download the new URL: ```shell wget https://github.com/jupp0r/prometheus-cpp/archive/v1.2.4.tar.gz ``` Calculate the checksum: ```shell sha256sum v1.2.4.tar.gz abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234 v1.2.4.tar.gz ``` Update the `sha256`. ## Upgrade a git submodule All the git submodule are under the folder `third_party`. We will use `opentelemetry-propto` as example in this case. This is a `git submodule`, it needs to point to the new tag. ### Get current tag ```shell cd third_party/opentelemetry-proto git log -1 ``` The current submodule show something like: ```shell commit b3060d2104df364136d75a35779e6bd48bac449a (HEAD, tag: v1.3.1) Author: Damien Mathieu <42@dmathieu.com> Date: Thu Apr 25 17:55:35 2024 +0200 generate profiles proto for CI (#552) ``` In this case we can see the current tag is `v.1.3.1`. ### Upgrade to new tag Pull new tags: ```shell git pull --tag origin ``` Upgrade to a new tag: ```shell git pull origin v1.3.2 ``` Check the new state: ```shell git log -1 ``` ```shell commit 40b3c1b746767cbc13c2e39da3eaf1a23e54ffdd (HEAD, tag: v1.3.2) Author: jack-berg <34418638+jack-berg@users.noreply.github.com> Date: Fri Jun 28 08:19:11 2024 -0500 Prepare changelog for 1.3.2 release (#563) Co-authored-by: Armin Ruech <7052238+arminru@users.noreply.github.com> ``` ### Add changes Go back to the root of opentelemetry-cpp ```shell cd ../.. git status ``` ```shell On branch upgrade_proto_1.3.2 Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git restore ..." to discard changes in working directory) modified: third_party/opentelemetry-proto (new commits) ``` Add the upgraded submodule: ```shell git add third_party/opentelemetry-proto ``` ## Upgrade a bazel dependency Same as git submodule, we will continue use `opentelemetry-proto` as example. All the bazel dependencies are defined in [repository.bzl](https://github.com/open-telemetry/opentelemetry-cpp/blob/main/bazel/repository.bzl) and [MODULE.bazel](https://github.com/open-telemetry/opentelemetry-cpp/blob/main/MODULE.bazel). ### Update the dependency in repository.bzl Locate the entry for opentelemetry-proto: ```text # OTLP Protocol definition maybe( http_archive, name = "com_github_opentelemetry_proto", build_file = "@io_opentelemetry_cpp//bazel:opentelemetry_proto.BUILD", sha256 = "bed250ceec8e4a83aa5604d7d5595a61945059dc662edd058a9da082283f7a00", strip_prefix = "opentelemetry-proto-1.3.1", urls = [ "https://github.com/open-telemetry/opentelemetry-proto/archive/v1.3.1.tar.gz", ], ) ``` Update the URL to the new tag: ```text urls = [ "https://github.com/open-telemetry/opentelemetry-proto/archive/v1.3.2.tar.gz", ], ``` Update strip_prefix to the new tag: ```text strip_prefix = "opentelemetry-proto-1.3.2", ``` Download the new URL: ```shell wget https://github.com/open-telemetry/opentelemetry-proto/archive/v1.3.2.tar.gz ``` Run a checksum on the new file: ```shell sha256sum v1.3.2.tar.gz ``` ```shell c069c0d96137cf005d34411fa67dd3b6f1f8c64af1e7fb2fe0089a41c425acd7 v1.3.2.tar.gz ``` Update the checksum in file bazel/repository.bzl: ```text sha256 = "c069c0d96137cf005d34411fa67dd3b6f1f8c64af1e7fb2fe0089a41c425acd7", ``` Typical change: ```shell [malff@malff-desktop opentelemetry-cpp]$ git diff bazel/repository.bzl diff --git a/bazel/repository.bzl b/bazel/repository.bzl index bac1e45b..508b95a3 100644 --- a/bazel/repository.bzl +++ b/bazel/repository.bzl @@ -88,10 +88,10 @@ def opentelemetry_cpp_deps(): http_archive, name = "com_github_opentelemetry_proto", build_file = "@io_opentelemetry_cpp//bazel:opentelemetry_proto.BUILD", - sha256 = "bed250ceec8e4a83aa5604d7d5595a61945059dc662edd058a9da082283f7a00", - strip_prefix = "opentelemetry-proto-1.3.1", + sha256 = "c069c0d96137cf005d34411fa67dd3b6f1f8c64af1e7fb2fe0089a41c425acd7", + strip_prefix = "opentelemetry-proto-1.3.2", urls = [ - "https://github.com/open-telemetry/opentelemetry-proto/archive/v1.3.1.tar.gz", + "https://github.com/open-telemetry/opentelemetry-proto/archive/v1.3.2.tar.gz", ], ) ``` #### Update MODULE.bazel > Remember, the link is different in your case. Replace `opentelemetry-proto` to correct target. Make sure the new tag is available in bazel central: * [opentelemetry-proto bazel-central](https://registry.bazel.build/modules/opentelemetry-proto) If missing, file a PR to add it, or contact the maintainer: * [opentelemetry-proto repository](https://github.com/bazelbuild/bazel-central-registry/tree/main/modules/opentelemetry-proto) Update the `opentelemetry-proto` version to the new tag: ```text bazel_dep(name = "opentelemetry-proto", version = "1.3.2", repo_name = "com_github_opentelemetry_proto") ``` File `MODULE.bazel` is used in the github CI for repository opentelemetry-cpp, so using a tag that does not exist (yet) in bazel central will break the CI build. See the known issues section. Typical change: ```shell [malff@malff-desktop opentelemetry-cpp]$ git diff MODULE.bazel diff --git a/MODULE.bazel b/MODULE.bazel index 7b84c2b7..3161ffb1 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -13,7 +13,7 @@ bazel_dep(name = "bazel_skylib", version = "1.5.0") bazel_dep(name = "curl", version = "8.4.0") bazel_dep(name = "grpc", version = "1.62.1", repo_name = "com_github_grpc_grpc") bazel_dep(name = "nlohmann_json", version = "3.11.3", repo_name = "github_nlohmann_json") -bazel_dep(name = "opentelemetry-proto", version = "1.3.1", repo_name = "com_github_opentelemetry_proto") +bazel_dep(name = "opentelemetry-proto", version = "1.3.2", repo_name = "com_github_opentelemetry_proto") bazel_dep(name = "opentracing-cpp", version = "1.6.0", repo_name = "com_github_opentracing") bazel_dep(name = "platforms", version = "0.0.8") bazel_dep(name = "prometheus-cpp", version = "1.2.4", repo_name = "com_github_jupp0r_prometheus_cpp") ```