dd-jmx-collector
Stack Trace is:
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@10.0.1/Native Method)
- parking to wait for <0x00000007a9ce5448> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(java.base@10.0.1/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@10.0.1/AbstractQueuedSynchronizer.java:883)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@10.0.1/AbstractQueuedSynchronizer.java:915)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@10.0.1/AbstractQueuedSynchronizer.java:1238)
at java.util.concurrent.locks.ReentrantLock.lock(java.base@10.0.1/ReentrantLock.java:267)
at org.gradle.internal.remote.internal.hub.MessageHub$ChannelDispatch.dispatch(MessageHub.java:361)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy11.sendOutputEvent(Unknown Source)
at org.gradle.process.internal.worker.child.WorkerLogEventListener.onOutput(WorkerLogEventListener.java:36)
at jdk.internal.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@10.0.1/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@10.0.1/Method.java:564)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:230)
at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:149)
at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:140)
at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy10.onOutput(Unknown Source)
at org.gradle.internal.logging.sink.OutputEventTransformer.onOutput(OutputEventTransformer.java:104)
at org.gradle.internal.logging.sink.OutputEventRenderer.onOutput(OutputEventRenderer.java:420)
- locked <0x00000007aa223fc0> (a java.lang.Object)
at org.gradle.internal.logging.sink.OutputEventListenerManager$1.onOutput(OutputEventListenerManager.java:36)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onTextEvent(TextStreamOutputEventListener.java:57)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onOutput(TextStreamOutputEventListener.java:41)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$OutputEventDestination.onOutput(PrintStreamLoggingSystem.java:167)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$1.text(PrintStreamLoggingSystem.java:44)
at org.gradle.internal.io.LineBufferingOutputStream.flush(LineBufferingOutputStream.java:95)
at org.gradle.internal.io.LineBufferingOutputStream.write(LineBufferingOutputStream.java:80)
at java.io.OutputStream.write(java.base@10.0.1/OutputStream.java:113)
at java.io.PrintStream.write(java.base@10.0.1/PrintStream.java:559)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(java.base@10.0.1/StreamEncoder.java:233)
at sun.nio.cs.StreamEncoder.implFlushBuffer(java.base@10.0.1/StreamEncoder.java:312)
at sun.nio.cs.StreamEncoder.flushBuffer(java.base@10.0.1/StreamEncoder.java:104)
- locked <0x00000007afd03ac0> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flushBuffer(java.base@10.0.1/OutputStreamWriter.java:184)
at java.io.PrintStream.newLine(java.base@10.0.1/PrintStream.java:625)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at java.io.PrintStream.println(java.base@10.0.1/PrintStream.java:883)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at org.gradle.internal.io.LinePerThreadBufferingOutputStream.println(LinePerThreadBufferingOutputStream.java:203)
at datadog.slf4j.impl.SimpleLogger.write(SimpleLogger.java:318)
at datadog.slf4j.impl.SimpleLogger.log(SimpleLogger.java:295)
at datadog.slf4j.impl.SimpleLogger.info(SimpleLogger.java:480)
at org.datadog.jmxfetch.reporter.ConsoleReporter.doSendServiceCheck(ConsoleReporter.java:52)
at org.datadog.jmxfetch.reporter.Reporter.sendServiceCheck(Reporter.java:166)
at org.datadog.jmxfetch.App.sendServiceCheck(App.java:767)
at org.datadog.jmxfetch.App.processCollectionStatus(App.java:1161)
at org.datadog.jmxfetch.App.doIteration(App.java:491)
at org.datadog.jmxfetch.App.start(App.java:424)
at org.datadog.jmxfetch.App.run(App.java:219)
at datadog.trace.agent.jmxfetch.JMXFetch$1.run(JMXFetch.java:99)
at java.lang.Thread.run(java.base@10.0.1/Thread.java:844)
main
Stack Trace is:
java.lang.Thread.State: BLOCKED (on object monitor)
at org.gradle.internal.logging.sink.OutputEventRenderer.onOutput(OutputEventRenderer.java:420)
- waiting to lock <0x00000007aa223fc0> (a java.lang.Object)
at org.gradle.internal.logging.sink.OutputEventListenerManager$1.onOutput(OutputEventListenerManager.java:36)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onTextEvent(TextStreamOutputEventListener.java:57)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onOutput(TextStreamOutputEventListener.java:41)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$OutputEventDestination.onOutput(PrintStreamLoggingSystem.java:167)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$1.text(PrintStreamLoggingSystem.java:44)
at org.gradle.internal.io.LineBufferingOutputStream.flush(LineBufferingOutputStream.java:95)
at org.gradle.internal.io.LineBufferingOutputStream.write(LineBufferingOutputStream.java:80)
at java.io.OutputStream.write(java.base@10.0.1/OutputStream.java:113)
at java.io.PrintStream.write(java.base@10.0.1/PrintStream.java:559)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(java.base@10.0.1/StreamEncoder.java:233)
at sun.nio.cs.StreamEncoder.implFlushBuffer(java.base@10.0.1/StreamEncoder.java:312)
at sun.nio.cs.StreamEncoder.flushBuffer(java.base@10.0.1/StreamEncoder.java:104)
- locked <0x00000007a9c6b990> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flushBuffer(java.base@10.0.1/OutputStreamWriter.java:184)
at java.io.PrintStream.newLine(java.base@10.0.1/PrintStream.java:625)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at java.io.PrintStream.println(java.base@10.0.1/PrintStream.java:883)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at org.gradle.internal.io.LinePerThreadBufferingOutputStream.println(LinePerThreadBufferingOutputStream.java:203)
at datadog.slf4j.impl.SimpleLogger.write(SimpleLogger.java:318)
at datadog.slf4j.impl.SimpleLogger.log(SimpleLogger.java:295)
at datadog.slf4j.impl.SimpleLogger.formatAndLog(SimpleLogger.java:355)
at datadog.slf4j.impl.SimpleLogger.debug(SimpleLogger.java:446)
at datadog.trace.agent.tooling.context.FieldBackedProvider$1$1$1.visitMethodInsn(FieldBackedProvider.java:202)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.onVisitMethodInsn(ExceptionTableSensitiveMethodVisitor.java:180)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.visitMethodInsn(ExceptionTableSensitiveMethodVisitor.java:167)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.utility.visitor.StackAwareMethodVisitor.visitMethodInsn(StackAwareMethodVisitor.java:302)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.jar.asm.ClassReader.readCode(ClassReader.java:2214)
at net.bytebuddy.jar.asm.ClassReader.readMethod(ClassReader.java:1283)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:688)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:400)
at net.bytebuddy.asm.Advice$Dispatcher$Inlining$Resolved$AdviceMethodInliner.apply(Advice.java:7379)
at net.bytebuddy.asm.Advice$AdviceVisitor.onAfterExceptionTable(Advice.java:9427)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.considerEndOfExceptionTable(ExceptionTableSensitiveMethodVisitor.java:49)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.visitLabel(ExceptionTableSensitiveMethodVisitor.java:62)
at net.bytebuddy.jar.asm.Label.accept(Label.java:357)
at net.bytebuddy.jar.asm.ClassReader.readCode(ClassReader.java:1823)
at net.bytebuddy.jar.asm.ClassReader.readMethod(ClassReader.java:1283)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:688)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:400)
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:3393)
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:1930)
at net.bytebuddy.dynamic.scaffold.inline.RedefinitionDynamicTypeBuilder.make(RedefinitionDynamicTypeBuilder.java:217)
at net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:10132)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10551)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10514)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1500(AgentBuilder.java:10280)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10964)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10902)
at java.security.AccessController.doPrivileged(java.base@10.0.1/Native Method)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10470)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
at sun.instrument.TransformerManager.transform(java.instrument@10.0.1/TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(java.instrument@10.0.1/InstrumentationImpl.java:560)
at java.lang.ClassLoader.defineClass1(java.base@10.0.1/Native Method)
at java.lang.ClassLoader.defineClass(java.base@10.0.1/ClassLoader.java:1009)
at java.security.SecureClassLoader.defineClass(java.base@10.0.1/SecureClassLoader.java:174)
at java.net.URLClassLoader.defineClass(java.base@10.0.1/URLClassLoader.java:545)
at java.net.URLClassLoader.access$100(java.base@10.0.1/URLClassLoader.java:83)
at java.net.URLClassLoader$1.run(java.base@10.0.1/URLClassLoader.java:453)
at java.net.URLClassLoader$1.run(java.base@10.0.1/URLClassLoader.java:447)
at java.security.AccessController.doPrivileged(java.base@10.0.1/Native Method)
at java.net.URLClassLoader.findClass(java.base@10.0.1/URLClassLoader.java:446)
at java.lang.ClassLoader.loadClass(java.base@10.0.1/ClassLoader.java:566)
- locked <0x00000007afb4bc50> (a java.lang.Object)
at java.lang.ClassLoader.loadClass(java.base@10.0.1/ClassLoader.java:499)
at org.gradle.internal.remote.internal.hub.MessageHub.addHandler(MessageHub.java:131)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection.addIncoming(MessageHubBackedObjectConnection.java:98)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.startReceivingTests(TestWorker.java:104)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:68)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:46)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:93)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:36)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:125)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:68)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:62)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:67)
Expose new options for configuring JMXFetch with standard datadog-agent config files with `jvm_direct: true` set as an instance attribute (this will be ignored by the datadog-agent).
For Example:
* `dd.agent.conf.d=/opt/datadog-agent/etc/conf.d`
* `dd.jmxfetch.configs=activemq.d/conf.yaml,jmx.d/conf.yaml`
will load jmx configs in those two files that have `jvm_direct: true` in their `instance` setup.
Environment variables can also be used: `DD_AGENT_CONF_D` and `DD_JMXFETCH_CONFIGS`
Depends on https://github.com/DataDog/jmxfetch/releases/tag/0.29.0 being released.
Or all executors, bypassing the allow list.
`dd.trace.executor=com.MyCustomExecutor,com.OtherExecutor`
`dd.trace.executors.all=true`
Turns out in many cases, executors that we say we’re skipping, are still being traced because they extend from an already instrumented executor.
Glassfish’s WebappClassLoader caches when a resource or class load fails, so we can’t load as a resource first to see if it is available.
Also add additional logging.
Ensures that span is closed when Observable is unsubscribed from.
Also added retransform error logger since retransforms might be missed if an exception is thrown, leading to odd behavior.
This change is slightly breaking for existing hystrixCommand code since the resource name changes from run->execute and getFallback->fallback. The fallback span is also now a child of the execute span.
Add the following system property or corresponding environment variable:
```
dd.trace.classes.exclude=some.packagename.*,some.classname.MyClass$NestedClass
```
* 'twilio' of github.com:darylrobbins/dd-trace-java:
Updates to handle async calls, which have broken all tests
Missed Gradle file
WIP Twilio SDK Instrumentation
# Conflicts:
# dd-java-agent/instrumentation/twilio/src/main/java/datadog/trace/instrumentation/twilio/TwilioClientDecorator.java
# dd-java-agent/instrumentation/twilio/src/main/java/datadog/trace/instrumentation/twilio/TwilioInstrumentation.java
# dd-java-agent/instrumentation/twilio/src/test/groovy/test/TwilioClientTest.groovy
Merge branch 'twilio' of github.com:darylrobbins/dd-trace-java into twilio
Improved unit testing
* 'twilio' of github.com:darylrobbins/dd-trace-java:
Updates to handle async calls, which have broken all tests
Missed Gradle file
WIP Twilio SDK Instrumentation
# Conflicts:
# dd-java-agent/instrumentation/twilio/src/main/java/datadog/trace/instrumentation/twilio/TwilioClientDecorator.java
# dd-java-agent/instrumentation/twilio/src/main/java/datadog/trace/instrumentation/twilio/TwilioInstrumentation.java
# dd-java-agent/instrumentation/twilio/src/test/groovy/test/TwilioClientTest.groovy
Fix sleep times and choose Java7-friendly test dependencies
Corrected test assertion
Class `FieldBackedProvider` uses ByteBuddy to add a field and its respective getters and setters to store the context object.
Assuming that we have a class `A` that implements runnable and that we wrap with a `FieldBackedProvider`
`Runnable` interfaces, if later on we use a cglib's `Enancher` class on `A` then the our mechanism will kick in again and try
to add the methods again. This causes a `java.lang.ClassFormatError: Duplicate method name "get__datadogContext$java$lang$Runnable" with signature "()Ljava.lang.Object;"`
to be thrown because CGLIB already copied over those methods from the original class `A` to the newly created class.
With this commit we now check that method were not previously defined before adding them, and if they were then we avoid adding them
again.
The reason why it wasn't faiing before is that we only checked on context field existence, not methods and cglib do not copy over fields, it copies only methods. E.g.
```
public class Main {
public static class A {
private String name = "hey";
public String getName() {
return this.name;
}
}
public static void main(String[] args) {
System.out.println("----- 'A' declared fields -----");
A s = new A();
for (Field f : s.getClass().getDeclaredFields()) {
System.out.println("field: " + f.getName());
}
for (Method m : s.getClass().getDeclaredMethods()) {
System.out.println("method: " + m.getName());
}
System.out.println("----- Proxy declared fields -----");
Enhancer enhancer = new Enhancer();
enhancer.setSuperclass(A.class);
enhancer.setCallback((FixedValue) () -> null);
A proxy = (A) enhancer.create();
for (Field f : proxy.getClass().getDeclaredFields()) {
System.out.println("field: " + f.getName());
}
for (Method m : proxy.getClass().getDeclaredMethods()) {
System.out.println("method: " + m.getName());
}
}
}
```
Results in:
```
----- 'A' declared fields -----
field: name
method: getName
----- Proxy declared fields -----
field: CGLIB$BOUND
field: CGLIB$FACTORY_DATA
field: CGLIB$THREAD_CALLBACKS
field: CGLIB$STATIC_CALLBACKS
field: CGLIB$CALLBACK_0
field: CGLIB$CALLBACK_FILTER
method: equals
method: toString
method: hashCode
method: clone
method: getName
method: newInstance
method: newInstance
method: newInstance
method: setCallbacks
method: CGLIB$SET_STATIC_CALLBACKS
method: CGLIB$SET_THREAD_CALLBACKS
method: getCallback
method: getCallbacks
method: CGLIB$STATICHOOK1
method: CGLIB$BIND_CALLBACKS
method: setCallback
```
Idea complains about duplicates root because we run latest dep tests
and 'later version' tests from the same dir. This is ultimately
TestSetd plugin bug by the looks of it, but we can work around it by
swapping old and new version tests.
Also run both old and new version tests under 'test' task.
Currently helpers will not be injected if instrumentation doesn't
apply. Unfortunately it is possible for some classes to have context
fields added even if instrumentation is not allied (i.g. instrumented
classes are not used). Fix this by always injecting all helpers if we
inject context fields.
This can cause problems if muzzle prevents an integration from loading because the class for the new field being added may not be injected. This results in an error:
```
Caused by: java.lang.NoClassDefFoundError: datadog/trace/bootstrap/instrumentation/context/FieldBackedProvider$ContextAccessor$org$hibernate$Session$datadog$trace$bootstrap$instrumentation$hibernate$SessionState
```
Also add each hibernate instrumentation project as a test dependency to the others. This demonstrates a problem with muzzle and instrumentation context which currently causes the tests to fail.
Looks like AWS SDK doesn't call interceptor when non-sdk exception
occurs. This leads to leaking open spans. Fix that by instrumenting
http client.
Note: currently has no tests.
Serialize payloads as they arrive, send to agent when size exceeds 5MB.
If a flood of traces fills up the RingBuffer, we still count the traces that are discarded for stats reporting.
Do we want to allow configuration of the RingBuffer size?
I notice occasionally some of the instrumentation tests fail due to port
conflicts in TestHttpServer, esp when using the `--parallel` option in
gradle. I think the previous method is subject to a race where two
concurrent workers will both get the same port from `randomOpenPort()`.
Previously we were inheriting from the parent and also often setting as a tag.
Apply default span assertion to verify the spanType is being checked properly. (Include error state too.)
Should eventually revisit this to try and remove them.
Latest failure was on:
```
at ExecutorInstrumentationTest.#poolImpl '#name' reports after canceled jobs(ExecutorInstrumentationTest.groovy:202)
```
Luca Abbati
Tyler Benson
Nikolay Martynov
Luca Abbati
Rafael Winterhalter
Tyler Benson
Daryl Robbins
Daryl Robbins
Will Gittoes
Will Gittoes
Ben Keith
Arthur Bellal