The problem was that on zulu8 loading OkHttp touches JFR which in turn
touches log manager - which would break things like JBOSS.
The fix is to delay installing agent (and writer) until log manager
things have settled down - in way similar to jmxfetch.
Unfortunately for 'main' agent this turns out to be more involved
because of classloader shenanigans.
dd-jmx-collector
Stack Trace is:
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@10.0.1/Native Method)
- parking to wait for <0x00000007a9ce5448> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(java.base@10.0.1/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@10.0.1/AbstractQueuedSynchronizer.java:883)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@10.0.1/AbstractQueuedSynchronizer.java:915)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@10.0.1/AbstractQueuedSynchronizer.java:1238)
at java.util.concurrent.locks.ReentrantLock.lock(java.base@10.0.1/ReentrantLock.java:267)
at org.gradle.internal.remote.internal.hub.MessageHub$ChannelDispatch.dispatch(MessageHub.java:361)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy11.sendOutputEvent(Unknown Source)
at org.gradle.process.internal.worker.child.WorkerLogEventListener.onOutput(WorkerLogEventListener.java:36)
at jdk.internal.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@10.0.1/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@10.0.1/Method.java:564)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:230)
at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:149)
at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:140)
at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy10.onOutput(Unknown Source)
at org.gradle.internal.logging.sink.OutputEventTransformer.onOutput(OutputEventTransformer.java:104)
at org.gradle.internal.logging.sink.OutputEventRenderer.onOutput(OutputEventRenderer.java:420)
- locked <0x00000007aa223fc0> (a java.lang.Object)
at org.gradle.internal.logging.sink.OutputEventListenerManager$1.onOutput(OutputEventListenerManager.java:36)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onTextEvent(TextStreamOutputEventListener.java:57)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onOutput(TextStreamOutputEventListener.java:41)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$OutputEventDestination.onOutput(PrintStreamLoggingSystem.java:167)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$1.text(PrintStreamLoggingSystem.java:44)
at org.gradle.internal.io.LineBufferingOutputStream.flush(LineBufferingOutputStream.java:95)
at org.gradle.internal.io.LineBufferingOutputStream.write(LineBufferingOutputStream.java:80)
at java.io.OutputStream.write(java.base@10.0.1/OutputStream.java:113)
at java.io.PrintStream.write(java.base@10.0.1/PrintStream.java:559)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(java.base@10.0.1/StreamEncoder.java:233)
at sun.nio.cs.StreamEncoder.implFlushBuffer(java.base@10.0.1/StreamEncoder.java:312)
at sun.nio.cs.StreamEncoder.flushBuffer(java.base@10.0.1/StreamEncoder.java:104)
- locked <0x00000007afd03ac0> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flushBuffer(java.base@10.0.1/OutputStreamWriter.java:184)
at java.io.PrintStream.newLine(java.base@10.0.1/PrintStream.java:625)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at java.io.PrintStream.println(java.base@10.0.1/PrintStream.java:883)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at org.gradle.internal.io.LinePerThreadBufferingOutputStream.println(LinePerThreadBufferingOutputStream.java:203)
at datadog.slf4j.impl.SimpleLogger.write(SimpleLogger.java:318)
at datadog.slf4j.impl.SimpleLogger.log(SimpleLogger.java:295)
at datadog.slf4j.impl.SimpleLogger.info(SimpleLogger.java:480)
at org.datadog.jmxfetch.reporter.ConsoleReporter.doSendServiceCheck(ConsoleReporter.java:52)
at org.datadog.jmxfetch.reporter.Reporter.sendServiceCheck(Reporter.java:166)
at org.datadog.jmxfetch.App.sendServiceCheck(App.java:767)
at org.datadog.jmxfetch.App.processCollectionStatus(App.java:1161)
at org.datadog.jmxfetch.App.doIteration(App.java:491)
at org.datadog.jmxfetch.App.start(App.java:424)
at org.datadog.jmxfetch.App.run(App.java:219)
at datadog.trace.agent.jmxfetch.JMXFetch$1.run(JMXFetch.java:99)
at java.lang.Thread.run(java.base@10.0.1/Thread.java:844)
main
Stack Trace is:
java.lang.Thread.State: BLOCKED (on object monitor)
at org.gradle.internal.logging.sink.OutputEventRenderer.onOutput(OutputEventRenderer.java:420)
- waiting to lock <0x00000007aa223fc0> (a java.lang.Object)
at org.gradle.internal.logging.sink.OutputEventListenerManager$1.onOutput(OutputEventListenerManager.java:36)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onTextEvent(TextStreamOutputEventListener.java:57)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onOutput(TextStreamOutputEventListener.java:41)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$OutputEventDestination.onOutput(PrintStreamLoggingSystem.java:167)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$1.text(PrintStreamLoggingSystem.java:44)
at org.gradle.internal.io.LineBufferingOutputStream.flush(LineBufferingOutputStream.java:95)
at org.gradle.internal.io.LineBufferingOutputStream.write(LineBufferingOutputStream.java:80)
at java.io.OutputStream.write(java.base@10.0.1/OutputStream.java:113)
at java.io.PrintStream.write(java.base@10.0.1/PrintStream.java:559)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(java.base@10.0.1/StreamEncoder.java:233)
at sun.nio.cs.StreamEncoder.implFlushBuffer(java.base@10.0.1/StreamEncoder.java:312)
at sun.nio.cs.StreamEncoder.flushBuffer(java.base@10.0.1/StreamEncoder.java:104)
- locked <0x00000007a9c6b990> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flushBuffer(java.base@10.0.1/OutputStreamWriter.java:184)
at java.io.PrintStream.newLine(java.base@10.0.1/PrintStream.java:625)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at java.io.PrintStream.println(java.base@10.0.1/PrintStream.java:883)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at org.gradle.internal.io.LinePerThreadBufferingOutputStream.println(LinePerThreadBufferingOutputStream.java:203)
at datadog.slf4j.impl.SimpleLogger.write(SimpleLogger.java:318)
at datadog.slf4j.impl.SimpleLogger.log(SimpleLogger.java:295)
at datadog.slf4j.impl.SimpleLogger.formatAndLog(SimpleLogger.java:355)
at datadog.slf4j.impl.SimpleLogger.debug(SimpleLogger.java:446)
at datadog.trace.agent.tooling.context.FieldBackedProvider$1$1$1.visitMethodInsn(FieldBackedProvider.java:202)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.onVisitMethodInsn(ExceptionTableSensitiveMethodVisitor.java:180)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.visitMethodInsn(ExceptionTableSensitiveMethodVisitor.java:167)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.utility.visitor.StackAwareMethodVisitor.visitMethodInsn(StackAwareMethodVisitor.java:302)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.jar.asm.ClassReader.readCode(ClassReader.java:2214)
at net.bytebuddy.jar.asm.ClassReader.readMethod(ClassReader.java:1283)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:688)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:400)
at net.bytebuddy.asm.Advice$Dispatcher$Inlining$Resolved$AdviceMethodInliner.apply(Advice.java:7379)
at net.bytebuddy.asm.Advice$AdviceVisitor.onAfterExceptionTable(Advice.java:9427)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.considerEndOfExceptionTable(ExceptionTableSensitiveMethodVisitor.java:49)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.visitLabel(ExceptionTableSensitiveMethodVisitor.java:62)
at net.bytebuddy.jar.asm.Label.accept(Label.java:357)
at net.bytebuddy.jar.asm.ClassReader.readCode(ClassReader.java:1823)
at net.bytebuddy.jar.asm.ClassReader.readMethod(ClassReader.java:1283)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:688)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:400)
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:3393)
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:1930)
at net.bytebuddy.dynamic.scaffold.inline.RedefinitionDynamicTypeBuilder.make(RedefinitionDynamicTypeBuilder.java:217)
at net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:10132)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10551)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10514)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1500(AgentBuilder.java:10280)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10964)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10902)
at java.security.AccessController.doPrivileged(java.base@10.0.1/Native Method)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10470)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
at sun.instrument.TransformerManager.transform(java.instrument@10.0.1/TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(java.instrument@10.0.1/InstrumentationImpl.java:560)
at java.lang.ClassLoader.defineClass1(java.base@10.0.1/Native Method)
at java.lang.ClassLoader.defineClass(java.base@10.0.1/ClassLoader.java:1009)
at java.security.SecureClassLoader.defineClass(java.base@10.0.1/SecureClassLoader.java:174)
at java.net.URLClassLoader.defineClass(java.base@10.0.1/URLClassLoader.java:545)
at java.net.URLClassLoader.access$100(java.base@10.0.1/URLClassLoader.java:83)
at java.net.URLClassLoader$1.run(java.base@10.0.1/URLClassLoader.java:453)
at java.net.URLClassLoader$1.run(java.base@10.0.1/URLClassLoader.java:447)
at java.security.AccessController.doPrivileged(java.base@10.0.1/Native Method)
at java.net.URLClassLoader.findClass(java.base@10.0.1/URLClassLoader.java:446)
at java.lang.ClassLoader.loadClass(java.base@10.0.1/ClassLoader.java:566)
- locked <0x00000007afb4bc50> (a java.lang.Object)
at java.lang.ClassLoader.loadClass(java.base@10.0.1/ClassLoader.java:499)
at org.gradle.internal.remote.internal.hub.MessageHub.addHandler(MessageHub.java:131)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection.addIncoming(MessageHubBackedObjectConnection.java:98)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.startReceivingTests(TestWorker.java:104)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:68)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:46)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:93)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:36)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:125)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:68)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:62)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:67)
This was weird having it as a separate project. Reduces the number of top level directories.
There weren’t any existing tests in dd-java-agent’s test directory, so this seemed like a good place to put them.
Shadow relocates are no longer needed because of our new bootstrapping
process.
It's no longer possible for agent dependencies to interfere with the
user's classpath.
The immediate reason for this change is a bug created in the Cassandra
instrumentation.
The Cassandra instrumentation references guava transitive deps from
the datastax driver. These references are re-written by shadow,
causing the instrumentation to reference 'datadog.agent.deps.google.*'
instead of the guava class.
By default, instrumentation is enabled, and can be disabled by default by overriding the `defaultEnabled` method. Instrumentation can also be disabled individually or enabled when default is disabled.
Java 9 is commented out because there are some test failures that will be investigated later.
Some tests had to be excluded from Java 7 because they don’t have a Java 7 compatible version.
Nikolay Martynov
Laplie Anderson
Tyler Benson
Andrew Kent
Tyler Benson
Andrew Kent