The issue we are trying to fix here seems to be a result of two concurrent issues.
Cassandra driver's used by atlas 1.1.0 brings in a old version of netty < 4.0.23.
In version 4.0.25 netty raised exception in `UniqueName` class for duplicate name
```
if (map.putIfAbsent(name, Boolean.TRUE) != null) {
throw new IllegalArgumentException(String.format("'%s' is already in use", name));
}
```
This changed in 4.0.26 where exception was not thrown, instead the duplicate key was just ignored.
```
protected UniqueName(String name) {
this.name = checkNotNull(name, "name");
id = nextId.incrementAndGet();
}
```
Now this alone would not be enough. The only way I was able to reproduce is if two different class loaders tries to load the
same class `AttributeKeys` twice.
I tried to look into Atlas source code (which is huge) and I could not find a clear path of how this would be possible.
Further more, looking at some build scripts that a user experiencing this issue provided,
it looks like they are doing everything correctly and there is nothing they should change.
In conclusion, I believe that even if this is not a very clean approach,
we should be resilient to this use case, when a customer using netty <4.0.25 in an
older version of Apache Atlas (or with some sort of class loader behavior).
Alpha version of Jetty 10 was released and it seems to be compiled for
java versions above 8 so tests fail on java8.
Limit latest dep Jetty tests to Jetty 9 for now.
Alpha version of Jetty 10 was released and it seems to be compiled for
java versions above 8 so tests fail on java8.
Limit latest dep Jetty tests to Jetty 9 for now.
By definition, a Flow generated by a SuperPool does not respect ordering
of requests and responses, and in fact will typically only rarely actually
behave in the fashion that the instrumentation expects. The previous implementation
would start a span for a given request before submitting it as input to the flow,
and close the span with whatever response is next emitted by the flow. This
request will rarely (if ever) be the actual response for the request that
started the span. For more info, see the official docs at
https://doc.akka.io/docs/akka-http/current/client-side/host-level.html#configuring-a-host-connection-pool
Additionally, compiling this instumentation against scala 2.11, and only
scala 2.11 can (and does) cause significant problems at runtime due to the
fact that Scala is explicitly not binary compatible across major versions.
Remove compiled Scala artifacts from the actual instrumentation. Scala
is not binary compatible across major versions, and having
AkkaHttpClientTransformFlow.scala be in the artifact causes problems
when using anything but Scala 2.11.
Having the AkkaHttpClientTransformFlow implementation be in pure java
utilizing the Akka-stream Java DSL ensures that this will work across
any given Scala major version.
For mongo, use description if set, fallback to db name.
Memcache doesn’t have a good “instance name” that would work as service name, so clear it out for now.
We already have a hack to wait for client span to close after the
request because it is closed on separate thread. This patch extends
that hack to handle cases when original request throws an exception.
This ensures that `dd-tracng-ot` gets included into shadow jar.
Before this patch it was included only 'by accident' via some
transitive dependencies of some instrumentations.
This test was not marked as `Specification` and therefor was not run
during builds. Some time ago some code got moved around and this test
got broken - so fixing it.
Move start/stop of servers in smoke tests from gradle code to test
code.
This gives more flexibility in terms of what we can do there.
This also allows us to run servers under test with same JVM as the
test itself - not same jvm as gradle which is currently locked to Java8
dd-jmx-collector
Stack Trace is:
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@10.0.1/Native Method)
- parking to wait for <0x00000007a9ce5448> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(java.base@10.0.1/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@10.0.1/AbstractQueuedSynchronizer.java:883)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@10.0.1/AbstractQueuedSynchronizer.java:915)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@10.0.1/AbstractQueuedSynchronizer.java:1238)
at java.util.concurrent.locks.ReentrantLock.lock(java.base@10.0.1/ReentrantLock.java:267)
at org.gradle.internal.remote.internal.hub.MessageHub$ChannelDispatch.dispatch(MessageHub.java:361)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy11.sendOutputEvent(Unknown Source)
at org.gradle.process.internal.worker.child.WorkerLogEventListener.onOutput(WorkerLogEventListener.java:36)
at jdk.internal.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@10.0.1/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@10.0.1/Method.java:564)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:230)
at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:149)
at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:140)
at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy10.onOutput(Unknown Source)
at org.gradle.internal.logging.sink.OutputEventTransformer.onOutput(OutputEventTransformer.java:104)
at org.gradle.internal.logging.sink.OutputEventRenderer.onOutput(OutputEventRenderer.java:420)
- locked <0x00000007aa223fc0> (a java.lang.Object)
at org.gradle.internal.logging.sink.OutputEventListenerManager$1.onOutput(OutputEventListenerManager.java:36)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onTextEvent(TextStreamOutputEventListener.java:57)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onOutput(TextStreamOutputEventListener.java:41)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$OutputEventDestination.onOutput(PrintStreamLoggingSystem.java:167)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$1.text(PrintStreamLoggingSystem.java:44)
at org.gradle.internal.io.LineBufferingOutputStream.flush(LineBufferingOutputStream.java:95)
at org.gradle.internal.io.LineBufferingOutputStream.write(LineBufferingOutputStream.java:80)
at java.io.OutputStream.write(java.base@10.0.1/OutputStream.java:113)
at java.io.PrintStream.write(java.base@10.0.1/PrintStream.java:559)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(java.base@10.0.1/StreamEncoder.java:233)
at sun.nio.cs.StreamEncoder.implFlushBuffer(java.base@10.0.1/StreamEncoder.java:312)
at sun.nio.cs.StreamEncoder.flushBuffer(java.base@10.0.1/StreamEncoder.java:104)
- locked <0x00000007afd03ac0> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flushBuffer(java.base@10.0.1/OutputStreamWriter.java:184)
at java.io.PrintStream.newLine(java.base@10.0.1/PrintStream.java:625)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at java.io.PrintStream.println(java.base@10.0.1/PrintStream.java:883)
- locked <0x00000007afd031a0> (a java.io.PrintStream)
at org.gradle.internal.io.LinePerThreadBufferingOutputStream.println(LinePerThreadBufferingOutputStream.java:203)
at datadog.slf4j.impl.SimpleLogger.write(SimpleLogger.java:318)
at datadog.slf4j.impl.SimpleLogger.log(SimpleLogger.java:295)
at datadog.slf4j.impl.SimpleLogger.info(SimpleLogger.java:480)
at org.datadog.jmxfetch.reporter.ConsoleReporter.doSendServiceCheck(ConsoleReporter.java:52)
at org.datadog.jmxfetch.reporter.Reporter.sendServiceCheck(Reporter.java:166)
at org.datadog.jmxfetch.App.sendServiceCheck(App.java:767)
at org.datadog.jmxfetch.App.processCollectionStatus(App.java:1161)
at org.datadog.jmxfetch.App.doIteration(App.java:491)
at org.datadog.jmxfetch.App.start(App.java:424)
at org.datadog.jmxfetch.App.run(App.java:219)
at datadog.trace.agent.jmxfetch.JMXFetch$1.run(JMXFetch.java:99)
at java.lang.Thread.run(java.base@10.0.1/Thread.java:844)
main
Stack Trace is:
java.lang.Thread.State: BLOCKED (on object monitor)
at org.gradle.internal.logging.sink.OutputEventRenderer.onOutput(OutputEventRenderer.java:420)
- waiting to lock <0x00000007aa223fc0> (a java.lang.Object)
at org.gradle.internal.logging.sink.OutputEventListenerManager$1.onOutput(OutputEventListenerManager.java:36)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onTextEvent(TextStreamOutputEventListener.java:57)
at org.gradle.internal.logging.services.TextStreamOutputEventListener.onOutput(TextStreamOutputEventListener.java:41)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$OutputEventDestination.onOutput(PrintStreamLoggingSystem.java:167)
at org.gradle.internal.logging.source.PrintStreamLoggingSystem$1.text(PrintStreamLoggingSystem.java:44)
at org.gradle.internal.io.LineBufferingOutputStream.flush(LineBufferingOutputStream.java:95)
at org.gradle.internal.io.LineBufferingOutputStream.write(LineBufferingOutputStream.java:80)
at java.io.OutputStream.write(java.base@10.0.1/OutputStream.java:113)
at java.io.PrintStream.write(java.base@10.0.1/PrintStream.java:559)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(java.base@10.0.1/StreamEncoder.java:233)
at sun.nio.cs.StreamEncoder.implFlushBuffer(java.base@10.0.1/StreamEncoder.java:312)
at sun.nio.cs.StreamEncoder.flushBuffer(java.base@10.0.1/StreamEncoder.java:104)
- locked <0x00000007a9c6b990> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flushBuffer(java.base@10.0.1/OutputStreamWriter.java:184)
at java.io.PrintStream.newLine(java.base@10.0.1/PrintStream.java:625)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at java.io.PrintStream.println(java.base@10.0.1/PrintStream.java:883)
- locked <0x00000007a9c6b060> (a java.io.PrintStream)
at org.gradle.internal.io.LinePerThreadBufferingOutputStream.println(LinePerThreadBufferingOutputStream.java:203)
at datadog.slf4j.impl.SimpleLogger.write(SimpleLogger.java:318)
at datadog.slf4j.impl.SimpleLogger.log(SimpleLogger.java:295)
at datadog.slf4j.impl.SimpleLogger.formatAndLog(SimpleLogger.java:355)
at datadog.slf4j.impl.SimpleLogger.debug(SimpleLogger.java:446)
at datadog.trace.agent.tooling.context.FieldBackedProvider$1$1$1.visitMethodInsn(FieldBackedProvider.java:202)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.onVisitMethodInsn(ExceptionTableSensitiveMethodVisitor.java:180)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.visitMethodInsn(ExceptionTableSensitiveMethodVisitor.java:167)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.utility.visitor.StackAwareMethodVisitor.visitMethodInsn(StackAwareMethodVisitor.java:302)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.jar.asm.MethodVisitor.visitMethodInsn(MethodVisitor.java:427)
at net.bytebuddy.jar.asm.ClassReader.readCode(ClassReader.java:2214)
at net.bytebuddy.jar.asm.ClassReader.readMethod(ClassReader.java:1283)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:688)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:400)
at net.bytebuddy.asm.Advice$Dispatcher$Inlining$Resolved$AdviceMethodInliner.apply(Advice.java:7379)
at net.bytebuddy.asm.Advice$AdviceVisitor.onAfterExceptionTable(Advice.java:9427)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.considerEndOfExceptionTable(ExceptionTableSensitiveMethodVisitor.java:49)
at net.bytebuddy.utility.visitor.ExceptionTableSensitiveMethodVisitor.visitLabel(ExceptionTableSensitiveMethodVisitor.java:62)
at net.bytebuddy.jar.asm.Label.accept(Label.java:357)
at net.bytebuddy.jar.asm.ClassReader.readCode(ClassReader.java:1823)
at net.bytebuddy.jar.asm.ClassReader.readMethod(ClassReader.java:1283)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:688)
at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:400)
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:3393)
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:1930)
at net.bytebuddy.dynamic.scaffold.inline.RedefinitionDynamicTypeBuilder.make(RedefinitionDynamicTypeBuilder.java:217)
at net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:10132)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10551)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10514)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1500(AgentBuilder.java:10280)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10964)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10902)
at java.security.AccessController.doPrivileged(java.base@10.0.1/Native Method)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10470)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
at sun.instrument.TransformerManager.transform(java.instrument@10.0.1/TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(java.instrument@10.0.1/InstrumentationImpl.java:560)
at java.lang.ClassLoader.defineClass1(java.base@10.0.1/Native Method)
at java.lang.ClassLoader.defineClass(java.base@10.0.1/ClassLoader.java:1009)
at java.security.SecureClassLoader.defineClass(java.base@10.0.1/SecureClassLoader.java:174)
at java.net.URLClassLoader.defineClass(java.base@10.0.1/URLClassLoader.java:545)
at java.net.URLClassLoader.access$100(java.base@10.0.1/URLClassLoader.java:83)
at java.net.URLClassLoader$1.run(java.base@10.0.1/URLClassLoader.java:453)
at java.net.URLClassLoader$1.run(java.base@10.0.1/URLClassLoader.java:447)
at java.security.AccessController.doPrivileged(java.base@10.0.1/Native Method)
at java.net.URLClassLoader.findClass(java.base@10.0.1/URLClassLoader.java:446)
at java.lang.ClassLoader.loadClass(java.base@10.0.1/ClassLoader.java:566)
- locked <0x00000007afb4bc50> (a java.lang.Object)
at java.lang.ClassLoader.loadClass(java.base@10.0.1/ClassLoader.java:499)
at org.gradle.internal.remote.internal.hub.MessageHub.addHandler(MessageHub.java:131)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection.addIncoming(MessageHubBackedObjectConnection.java:98)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.startReceivingTests(TestWorker.java:104)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:68)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:46)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:93)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:36)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:125)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:68)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:62)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:67)
Luca Abbati
Laplie Anderson
Pedro Silva
Nikolay Martynov
Laplie Anderson
Tyler Benson
David Pratt