This pr gives classes defined in agent and extension class loaders all
permissions. Injected helper classes are also defined with all
permissions. Agent startup is altered so that we won't call methods that
require permission before we are able to get those permissions.
This pr does not attempt to address issues where agent code could allow
user code to circumvent security manager e.g.
https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/javaagent-bootstrap/src/main/java/io/opentelemetry/javaagent/bootstrap/InstrumentationHolder.java
gives access to `Instrumentation` that could be used to redefine classes
and remove security checks. Also this pr does not address failed
permission checks that could arise from user code calling agent code.
When user code, that does not have privileges, calls agent code, that
has the privileges, and agent code performs a sensitive operation then
permission check would fail because it is performed for all calling
classes, including the user classes. To fix this agent code should uses
`AccessController.doPrivileged` which basically means that, hey I have
done all the checks, run this call with my privileges and ignore the
privileges of my callers.
I think that the only way zipslip could happen is when name contains
`..` but codeql isn't able to cope with that. Removing the `..` check
gets rid of the code scanning alert.
Currently we run spotless and other checks for each of the parallel test
steps which seems wasteful. Here is an attempt to run only the tests in
given partition without any extra checks in the `test` step and run all
the checks in the `build` step.
Related discussion #7257Resolves#3413Resolves#5059Resolves#6258Resolves#7179
Adds a logging implementation that'll collect agent logs in memory until
slf4j is detected in the instrumented application; and when that happens
will dump all the logs into the application slf4j and log directly to
the application logger from that time on.
It's still in a POC state, unfortunately: while it works fine with an
app that uses & initializes slf4j directly, Spring Boot applications
actually reconfigure the logging implementation (e.g. logback) a while
after slf4j is loaded; which causes all the startup agent logs (debug
included) to be dumped with the default logback pattern.
Future work:
* ~~Make sure all logs produces by the agent are sent to loggers named
`io.opentelemetry...`
(https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/7446)~~
DONE
* Make this work on Spring Boot
* Documentation
* Smoke test?
This allows custom distributions of the agent to register
`HttpServerResponseCustomizer` implementations. When a supported HTTP
server instrumentation starts processing a response, the `onStart`
method of all registered implementations will be invoked with the
`Context` of the SERVER span, an instrumentation-specific response
object and `HttpServerResponseMutator` instance that allows appending
headers to that response.
The intent of this is to allow custom distributions to set a header
containing span context information, such as the trace and span IDs. As
such, the initial implementation only allows appending response headers
and nothing else.
The `HttpServerResponseCustomizer` and related classes are currently in
a subpackage of the `io.opentelemetry.javaagent.bootstrap` package in
`javaagent-extension-api`. This makes them get loaded in the bootstrap
classloader, thus directly accessible from instrumentations. I am not
aware if there is an elegant way to put it in the agent classloader
instead, yet have the same instance accessible from both
`AgentInstaller` and instrumentations.
This also includes Tomcat-specific implementation in order to be able to
demonstrate that it works, and add automated testing of this to
HttpServerTest including one implementation.
Groovy apps that parse a lot of scripts can generate a lot of
classloaders that will ultimately end up causing the agent to cache a
LOT of memory. For example, some java code that uses the Gremlin groovy
script engine to dynamically create and execute scripts can reproduce
this:
```
GremlinGroovyScriptEngine engine = new GremlinGroovyScriptEngine();
Bindings bindings = new SimpleBindings();
TinkerGraph graph = TinkerGraph.open();
GraphTraversalSource g = graph.traversal();
bindings.put("g", g);
for (int i = 0; i < 100000; i++) {
engine.eval("g.V(" + i + ")", bindings);
if(i % 250 == 0) System.out.println("Iteration " + i);
}
```
I have manually confirmed that ignoring the groovy classloaders (in this
PR) prevent the agent from exploding the cache and holding onto memory.
I could use another brain in deciding if there could be other unintended
consequences.
Fixes#7670
These obscure classloaders should be ignored completely, they process
in-line class creation and should not branch out to interactions that
are useful in distributed tracing. These two are responsible for outages
recently in many of our applications due to driving up memory usage from
WeakKey caching. In some cases, janino processing has ran wild and
stimulated over 5.7mil WeakKey objects accounting for close to 200MB of
heap.
Let's keep close to the SDK repo config.
I reverted some of the changes, only left those that I think make sense
anyway (e.g. comparing enums with `==`)
Related to #7220
Unfortunately it doesn't fix the aforementioned issue; while the CL used
is no longer the agent classloader, gauge collection still throws that
error.
Still, I think this is a good change that removes one source of agent's
CL leaking into application runtime.
Currently our `WeakConcurrentMap` is only cleaned of stale entries when
it is accessed. There is an option to clean from a background thread,
but this creates a separate thread for every map. This pr introduces a
single background thread that cleans all maps.
I removed the option to create a thread per map as we don't use it, if
there is interest I could attempt to find a way to add it back.
Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>
Bumps `byteBuddyVersion` from 1.12.18 to 1.12.19.
Updates `byte-buddy` from 1.12.18 to 1.12.19
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/raphw/byte-buddy/releases">byte-buddy's
releases</a>.</em></p>
<blockquote>
<h2>Byte Buddy 1.12.19</h2>
<ul>
<li>Avoid possible lock through circular class loading of
<code>TypeDescription</code> subtypes.</li>
<li>Avoid access error when using unsafe API on Java 17 with an active
security manager.</li>
<li>Close URL class loader used in Gradle plugin.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/raphw/byte-buddy/blob/master/release-notes.md">byte-buddy's
changelog</a>.</em></p>
<blockquote>
<h2>Byte Buddy release notes</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c93425a1e9"><code>c93425a</code></a>
[maven-release-plugin] prepare release byte-buddy-1.12.19</li>
<li><a
href="b1f4e9b4ee"><code>b1f4e9b</code></a>
[release] New release</li>
<li><a
href="8d17e3a2a3"><code>8d17e3a</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/raphw/byte-buddy/issues/1359">#1359</a>
from eyalkoren/protection-domain</li>
<li><a
href="c57139e69c"><code>c57139e</code></a>
Using explicit ProtectionDomain in dynamically loaded classes</li>
<li><a
href="ff8be9a91b"><code>ff8be9a</code></a>
Attempt cloning protection domain from accessible object to avoid
security ma...</li>
<li><a
href="6fe45f76ef"><code>6fe45f7</code></a>
Make s in message optional.</li>
<li><a
href="9023501243"><code>9023501</code></a>
Fix scope of summary variable.</li>
<li><a
href="02091f13f4"><code>02091f1</code></a>
Update codeql-analysis.yml</li>
<li><a
href="628b6a90c5"><code>628b6a9</code></a>
Close class loader in Gradle plugin, if possible.</li>
<li><a
href="9a81856525"><code>9a81856</code></a>
Remove unused import.</li>
<li>Additional commits viewable in <a
href="https://github.com/raphw/byte-buddy/compare/byte-buddy-1.12.18...byte-buddy-1.12.19">compare
view</a></li>
</ul>
</details>
<br />
Updates `byte-buddy-dep` from 1.12.18 to 1.12.19
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/raphw/byte-buddy/releases">byte-buddy-dep's
releases</a>.</em></p>
<blockquote>
<h2>Byte Buddy 1.12.19</h2>
<ul>
<li>Avoid possible lock through circular class loading of
<code>TypeDescription</code> subtypes.</li>
<li>Avoid access error when using unsafe API on Java 17 with an active
security manager.</li>
<li>Close URL class loader used in Gradle plugin.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/raphw/byte-buddy/blob/master/release-notes.md">byte-buddy-dep's
changelog</a>.</em></p>
<blockquote>
<h2>Byte Buddy release notes</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c93425a1e9"><code>c93425a</code></a>
[maven-release-plugin] prepare release byte-buddy-1.12.19</li>
<li><a
href="b1f4e9b4ee"><code>b1f4e9b</code></a>
[release] New release</li>
<li><a
href="8d17e3a2a3"><code>8d17e3a</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/raphw/byte-buddy/issues/1359">#1359</a>
from eyalkoren/protection-domain</li>
<li><a
href="c57139e69c"><code>c57139e</code></a>
Using explicit ProtectionDomain in dynamically loaded classes</li>
<li><a
href="ff8be9a91b"><code>ff8be9a</code></a>
Attempt cloning protection domain from accessible object to avoid
security ma...</li>
<li><a
href="6fe45f76ef"><code>6fe45f7</code></a>
Make s in message optional.</li>
<li><a
href="9023501243"><code>9023501</code></a>
Fix scope of summary variable.</li>
<li><a
href="02091f13f4"><code>02091f1</code></a>
Update codeql-analysis.yml</li>
<li><a
href="628b6a90c5"><code>628b6a9</code></a>
Close class loader in Gradle plugin, if possible.</li>
<li><a
href="9a81856525"><code>9a81856</code></a>
Remove unused import.</li>
<li>Additional commits viewable in <a
href="https://github.com/raphw/byte-buddy/compare/byte-buddy-1.12.18...byte-buddy-1.12.19">compare
view</a></li>
</ul>
</details>
<br />
Updates `byte-buddy-agent` from 1.12.18 to 1.12.19
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/raphw/byte-buddy/releases">byte-buddy-agent's
releases</a>.</em></p>
<blockquote>
<h2>Byte Buddy 1.12.19</h2>
<ul>
<li>Avoid possible lock through circular class loading of
<code>TypeDescription</code> subtypes.</li>
<li>Avoid access error when using unsafe API on Java 17 with an active
security manager.</li>
<li>Close URL class loader used in Gradle plugin.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/raphw/byte-buddy/blob/master/release-notes.md">byte-buddy-agent's
changelog</a>.</em></p>
<blockquote>
<h2>Byte Buddy release notes</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c93425a1e9"><code>c93425a</code></a>
[maven-release-plugin] prepare release byte-buddy-1.12.19</li>
<li><a
href="b1f4e9b4ee"><code>b1f4e9b</code></a>
[release] New release</li>
<li><a
href="8d17e3a2a3"><code>8d17e3a</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/raphw/byte-buddy/issues/1359">#1359</a>
from eyalkoren/protection-domain</li>
<li><a
href="c57139e69c"><code>c57139e</code></a>
Using explicit ProtectionDomain in dynamically loaded classes</li>
<li><a
href="ff8be9a91b"><code>ff8be9a</code></a>
Attempt cloning protection domain from accessible object to avoid
security ma...</li>
<li><a
href="6fe45f76ef"><code>6fe45f7</code></a>
Make s in message optional.</li>
<li><a
href="9023501243"><code>9023501</code></a>
Fix scope of summary variable.</li>
<li><a
href="02091f13f4"><code>02091f1</code></a>
Update codeql-analysis.yml</li>
<li><a
href="628b6a90c5"><code>628b6a9</code></a>
Close class loader in Gradle plugin, if possible.</li>
<li><a
href="9a81856525"><code>9a81856</code></a>
Remove unused import.</li>
<li>Additional commits viewable in <a
href="https://github.com/raphw/byte-buddy/compare/byte-buddy-1.12.18...byte-buddy-1.12.19">compare
view</a></li>
</ul>
</details>
<br />
Updates `byte-buddy-gradle-plugin` from 1.12.18 to 1.12.19
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/raphw/byte-buddy/releases">byte-buddy-gradle-plugin's
releases</a>.</em></p>
<blockquote>
<h2>Byte Buddy 1.12.19</h2>
<ul>
<li>Avoid possible lock through circular class loading of
<code>TypeDescription</code> subtypes.</li>
<li>Avoid access error when using unsafe API on Java 17 with an active
security manager.</li>
<li>Close URL class loader used in Gradle plugin.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/raphw/byte-buddy/blob/master/release-notes.md">byte-buddy-gradle-plugin's
changelog</a>.</em></p>
<blockquote>
<h2>Byte Buddy release notes</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c93425a1e9"><code>c93425a</code></a>
[maven-release-plugin] prepare release byte-buddy-1.12.19</li>
<li><a
href="b1f4e9b4ee"><code>b1f4e9b</code></a>
[release] New release</li>
<li><a
href="8d17e3a2a3"><code>8d17e3a</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/raphw/byte-buddy/issues/1359">#1359</a>
from eyalkoren/protection-domain</li>
<li><a
href="c57139e69c"><code>c57139e</code></a>
Using explicit ProtectionDomain in dynamically loaded classes</li>
<li><a
href="ff8be9a91b"><code>ff8be9a</code></a>
Attempt cloning protection domain from accessible object to avoid
security ma...</li>
<li><a
href="6fe45f76ef"><code>6fe45f7</code></a>
Make s in message optional.</li>
<li><a
href="9023501243"><code>9023501</code></a>
Fix scope of summary variable.</li>
<li><a
href="02091f13f4"><code>02091f1</code></a>
Update codeql-analysis.yml</li>
<li><a
href="628b6a90c5"><code>628b6a9</code></a>
Close class loader in Gradle plugin, if possible.</li>
<li><a
href="9a81856525"><code>9a81856</code></a>
Remove unused import.</li>
<li>Additional commits viewable in <a
href="https://github.com/raphw/byte-buddy/compare/byte-buddy-1.12.18...byte-buddy-1.12.19">compare
view</a></li>
</ul>
</details>
<br />
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mateusz Rzeszutek <mrzeszutek@splunk.com>
Bumps [jackson-bom](https://github.com/FasterXML/jackson-bom) from
2.13.4.20221013 to 2.14.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="cd99403b64"><code>cd99403</code></a>
[maven-release-plugin] prepare release jackson-bom-2.14.0</li>
<li><a
href="50dedf02e5"><code>50dedf0</code></a>
Prepare for 2.14.0</li>
<li><a
href="69023b4ec9"><code>69023b4</code></a>
Back to snapshots</li>
<li><a
href="ffb3dd2940"><code>ffb3dd2</code></a>
[maven-release-plugin] prepare for next development iteration</li>
<li><a
href="229c0be857"><code>229c0be</code></a>
[maven-release-plugin] prepare release jackson-bom-2.14.0-rc3</li>
<li><a
href="b3e3950eb7"><code>b3e3950</code></a>
Prepare for 2.14.0-rc3 release</li>
<li><a
href="babf49afc8"><code>babf49a</code></a>
Merge branch '2.13' into 2.14</li>
<li><a
href="f52258f22a"><code>f52258f</code></a>
[maven-release-plugin] prepare for next development iteration</li>
<li><a
href="20416ee364"><code>20416ee</code></a>
Merge branch '2.13' into 2.14</li>
<li><a
href="fd343f3524"><code>fd343f3</code></a>
Merge branch '2.13' into 2.14</li>
<li>Additional commits viewable in <a
href="https://github.com/FasterXML/jackson-bom/compare/jackson-bom-2.13.4.20221013...jackson-bom-2.14.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>
This holds onto span context if not ignored. For vanilla `okhttp3` we
have an explicit ignore in the `OkHttp3IgnoredTypesConfigurer` class,
but `presto-jdbc` repackages okhttp3 when it shades it.
Working PR to capture all the changes required to update to otel java
1.19.0. The new log API force allows
`:instrumentation-appender-api-internal` and
`:instrumentation-appender-sdk-internal`, but necessitates a decent
amount of refactoring as a result.
The PR points at the `1.19.0-SNAPSHOT`, which I'll update upon
publication.
Co-authored-by: Mateusz Rzeszutek <mrzeszutek@splunk.com>
Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>
Co-authored-by: Lauri Tulmin <ltulmin@splunk.com>
* Update dependencies
* Strictly pin slf4j and logback versions
* logback 1.3
* Only use slf4j 2.0.0 internally in the javaagent
* Pre-initialize slf4j provider
* Bump jackson version
* licenses
* Update gradle to 7.5
* Bump to 7.5.1
* gradle 7.5.1 with jdk17
* spotless
* one more --add-opens
Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>
Co-authored-by: Lauri Tulmin <ltulmin@splunk.com>
* Merge javaagent-instrumentation-api into javaagent-extension-api
* remove some leftover references to javaagent-instrumentation-api
* add missing instrumentation-api to distro example
* Move concurrent instrumentation utils out from javaagent-instrumentation-api
* Move AgentLogEmitterProvider, InstrumentedTaskClasses and OpenTelemetrySdkAccess out of javaagent-instrumentation-api
* Faster type matching
* make findLoadedClass accessible on java17
* enable jaxrs instrumentation for quarkus test
* fix websphere
* fix muzzle
* javadoc formating
* ignore classes that are know to fail to load for virtual field transforms
* add back jaxrs and jaxws annotation instrumentations
* Apply suggestions from code review
Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>
* fix compile error
* comments
* replace deprecated method usage
* add comment
* add an spi to get access to bootstrap proxy from muzzle module
Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>
* Library instrumentation should read its version from a file
* errorprone
* animalsniffer
* code review comments
* add name as task input too
* code review comments
* Make it possible to register multiple helper resources under the same name
* go back to using the old property in tests after all
* code review comments
* Spring RMI instrumentation
* Change the order of import statements
* remove extra separation in import statements
* stylistic changes
* Fix groovy rule violations
* Formatting changes in groovy file
* Spotless fixes and muzzle check version change
* Fixed minimum version in filenames and fixed muzzle check
* single InstrumentationModule and added context propagation test
* Merged singletons, use random port in test and add stricter matchers.
* Remove unused import
* during install, hook up the log emitter provider for instrumentation to use.
* spotless
* Fix tests
* Default instrumentation name to ROOT when logger name null/empty
Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>
* rename artifacts and packages
* Library users shouldn't need to use internal
* Update docs
* Rename in order to simplify HelperClassPredicate
* Spotless
* Move AgentLogEmitterProvider to javaagent-instrumentation-api
* Fix redefinition failure on openj9
* isntead of remembering the list of interfaces the class has remember whether it has any of the virutal field marker interfaces
* address review comment
* ensure virtual field detection works when internal-reflection instrumentation is disabled
* Expose AutoConfiguredOpenTelemetrySdk to AgentListener
* Only call AgentListener if noop is disabled, deprecate AgentListener methods
* Call AgentListener in DelayedAfterAgentCallback
* Work around jvm crash on early 1.8
* skip retransform if class was already transformed during load
* fix imports after rebase
* add test
* disable test on windows
* Muzzle match only once in each class loader
* Move muzzle matcher caching from ReferenceMatcher to InstrumentationModuleInstaller
* Update comment as caching was moved to a different method
* Fix comment
* Make it possible to use InstrumentationContext (now VirtualField) from library instrumentation
* fix tests
* fix javadocs
* fix some more tests
* code review comments
* setIfNull, computeIfNull
Lauri Tulmin
dependabot[bot]
Mateusz Rzeszutek
Ago Allikmaa
jason plumb
Steve Dodge
wallezhang
jack-berg
David Boreham
Trask Stalnaker
Lauri Tulmin
Anuraag Agrawal
Samudraneel Dasgupta
Nikita Salnikov-Tarnovski
Martin