Commit Graph

2362 Commits

Author SHA1 Message Date
Tyler Benson bd82166b58 Rename methods that don't require disambiguation. 2020-02-24 08:35:19 -08:00
Tyler Benson f70c35842d Move matchers to separate package and split matcher classes out 2020-02-24 08:31:12 -08:00
Nikolay Martynov 833f67816e
Merge pull request #1248 from DataDog/mar-kolya/reduce-superclass-matcher-exposure
Avoid calling superclass matcher where possible
2020-02-21 21:16:52 -05:00
Tyler Benson b27b30d913
Merge pull request #1247 from DataDog/tyler/bb-raw
Set ByteBuddy raw setting at startup
2020-02-21 16:30:24 -08:00
Nikolay Martynov b6d374e3a0 Avoid calling superclass matcher where possible
Saves about 100ms of startup time on test app
2020-02-21 15:08:39 -05:00
Tyler Benson ecf9bcc655
Make sure CallDepthThreadLocalMap is always reset (#1227)
Make sure CallDepthThreadLocalMap is always reset
2020-02-21 11:03:51 -08:00
Tyler Benson faccb0d483 Set ByteBuddy raw setting at startup
This reduces the complexity of TypeDefinitions.
```
Benchmark                                                             Mode  Cnt   Score   Error  Units
ClassRetransformingBenchmark.WithAgent.testTracedRetransform          avgt       22.450          ms/op
ClassRetransformingBenchmark.WithAgent.testUntracedRetransform        avgt        6.842          ms/op
ClassRetransformingBenchmark.WithAgentMaster.testTracedRetransform    avgt       23.188          ms/op
ClassRetransformingBenchmark.WithAgentMaster.testUntracedRetransform  avgt        7.009          ms/op
```
It also improved startup time by about 1 sec.
2020-02-21 10:35:20 -08:00
Nikolay Martynov 1f8acafc42
Merge pull request #1245 from DataDog/mar-kolya/single-ignores-matcher
Single ignores matcher
2020-02-21 12:42:18 -05:00
Tyler Benson e4d29e37ef
Update JMH benchmarks (#1242)
Update JMH benchmarks
2020-02-21 09:38:34 -08:00
Nikolay Martynov 00c268e6d8 Add equivalence check to TypeCacheKey 2020-02-21 12:24:53 -05:00
Nikolay Martynov d4c6d86e6d Do not use zero for hashcode
Seems like this may have odd sideeffects
2020-02-21 12:24:09 -05:00
Nikolay Martynov f736c425ff TypeCacheKey are different if hash codes are different
And this is very easy to check
2020-02-21 12:23:41 -05:00
Nikolay Martynov 9d7682f776 Fix hashcode calculation in TypeCacheKey
Using potentially very large number for a mod is probably not very effective
2020-02-21 12:22:51 -05:00
Nikolay Martynov 27842d8479 Ignore cglib packaged with springboot 2020-02-21 11:56:45 -05:00
Nikolay Martynov 0c659770cc Collect all ignore matchers into one
This should allow us to optimize this as a single function later
2020-02-21 11:48:40 -05:00
Nikolay Martynov b1d314aff1 Specify types in jaxrs mtcher to help Idea with type inference 2020-02-21 11:34:25 -05:00
Lev Priima 37bcb4e288 ClassLoaderMatcher switch to Guava Cache 2020-02-21 03:58:02 -08:00
Lev Priima 52e0eb9d37 SkipClassLoaderMatcher use str-switch in shouldSkipClass(ClassLoader) 2020-02-20 21:02:42 -08:00
Tyler Benson a4f34f4088 Latest tomcat release seems to have some issues
Instrumentation muzzled: [jsp, jsp-compile] -- datadog.trace.instrumentation.jsp.JasperJSPCompilationContextInstrumentation on jdk.internal.loader.ClassLoaders$AppClassLoader@2c13da15
-- datadog.trace.instrumentation.jsp.JSPDecorator:39 Missing method getServletContext#()Ljavax/servlet/ServletContext;
2020-02-20 20:35:25 -08:00
Tyler Benson 5e37ca02f1
Merge pull request #1240 from DataDog/tyler/executor-matcher-reorder
Reorder java-concurrent matchers
2020-02-20 17:54:39 -08:00
Tyler Benson 660921517c
Remove costly matchers from ignore list (#1233)
Remove costly matchers from ignore list
2020-02-20 17:52:43 -08:00
Tyler Benson e1964e526e Update JMH benchmarks
Benchmark is run with `./gradlew :dd-java-agent:benchmark:jmh` -- after updating the absolute paths to the various agents in `ClassRetransformingBenchmark`.  Unfortunately I was unable to have it launch correctly with a relative path.
2020-02-20 17:30:36 -08:00
Trask Stalnaker 8949d6c05f Make sure CallDepthThreadLocalMap is always reset 2020-02-20 16:53:57 -08:00
heathkd b7b46caaa8 create AgentBuilder instance with logging listeners only if debug mode is enabled 2020-02-20 16:29:27 -05:00
Tyler Benson b80e9857d2 Reorder java-concurrent matchers
Move the expensive matcher to last.

In theory this should help, but did not seem to make a significant difference in basic startup benchmarks.
2020-02-20 11:34:39 -08:00
Tyler Benson e4a454549d
Split interface and class "supertype matching" (#1232)
Split interface and class "supertype matching"
2020-02-20 11:08:39 -08:00
Nikolay Martynov f4d8b6b1f2 Avoid creating context provider for intrumentations that do not need that
This should reduce number of allocations
2020-02-20 08:28:25 -05:00
Laplie Anderson 43487fcfb5 Fix the logic 2020-02-19 21:22:17 -05:00
Laplie Anderson 92b9c07018 Add interfaces only code 2020-02-19 21:22:17 -05:00
Laplie Anderson 147b4a64f1 Add matcher for only concrete classes 2020-02-19 21:22:17 -05:00
Laplie Anderson d45ead1284 split matching into safeExtendsClass and safeHasInterface 2020-02-19 21:22:17 -05:00
Tyler Benson 6e4b55304e Add special handling for ignoring `javax.decorator.Decorator` annotated classes. 2020-02-19 16:24:23 -08:00
Tyler Benson 88aa31b63e Remove costly matchers from ignore list
Removing these matchers caused the time spent matching ignores to go from greater than 1 second to less than 100 ms.
2020-02-19 16:24:23 -08:00
Nikolay Martynov d49c08691e
Merge pull request #1229 from DataDog/mar-kolya/fix-jaxrs-matching
Fix JaxRs matching
2020-02-19 07:01:21 -05:00
Nikolay Martynov 02fb8d8888 Fix JaxRs matching
Paranthesys was missplaced leading to more complicated matcher than needed
2020-02-18 18:42:36 -05:00
Heather D'Souza a81f4dc223
Merge pull request #1192 from DataDog/heather.dsouza/scheduling_annotation
Add support for Spring Scheduling
2020-02-18 14:07:06 -05:00
Jean-Philippe Bempel efe8ee375b
Merge pull request #1226 from jpbempel/jpbempel/reuseKeys
Enable reuseKeys on WeakConcurrentMap
2020-02-18 19:09:05 +01:00
heathkd 917deb19be change imports to use bootstrap api package 2020-02-14 20:48:18 -05:00
heathkd f3537e1665 change module and file names to match version of earliest supported version of spring-context 2020-02-14 20:47:16 -05:00
heathkd 9eb51248cd address PR comments 2020-02-14 20:47:16 -05:00
heathkd b21e361591 add test for interval task and trigger task 2020-02-14 20:46:26 -05:00
heathkd cc1aaf5fc5 replace Trace annotation with regular API interaction 2020-02-14 20:46:26 -05:00
heathkd 68c323ac56 create ScheduledTasks bean and application context config 2020-02-14 20:46:00 -05:00
Nikolay Martynov a04cc15ce8 Fix some tests 2020-02-14 12:59:30 -05:00
Nikolay Martynov fd75575796 Disable peer hostname tag for http servers
Getting peer hostname makes DNS request
2020-02-14 11:25:07 -05:00
jean-philippe bempel 58b1cff62e Enable reuseKeys on WeakConcurrentMap
for each call to get method in WeakConcurrentMap, a LatentKey is
created. Considering the frequency of call it's significant.
Also, the classloader of the tracer can be considered as permanent
through the lifecycle of the JVM which is a prerequisite to be able
to reuse keys (otherwise classloader leak).
2020-02-14 14:52:59 +01:00
Tyler Benson 618780b62f
More general class loading instrumentation (#1218)
More general class loading instrumentation
2020-02-13 11:54:25 -08:00
Trask Stalnaker 94dd781e50 Better naming 2020-02-13 11:31:47 -08:00
Trask Stalnaker 4dfd9534e8 Add optimization for IBM JVM 2020-02-13 11:31:47 -08:00
Trask Stalnaker a0224e2ad7 Fix for test failure on IBM JVM 2020-02-13 11:31:47 -08:00
Trask Stalnaker bfc7a0de55 More general class loading instrumentation 2020-02-13 11:31:45 -08:00
Giuseppe Cafarelli f94f76488f Add test for elasticsearch 2 TransportActionListener 2020-02-13 14:34:21 +01:00
Nikolay Martynov ef9ebffc54 Simplify unknown compression handling 2020-02-12 15:08:25 -05:00
Nikolay Martynov ca9f623d05 Add profiling lz4 compression support
Gzip is still the default
2020-02-12 15:00:02 -05:00
Giuseppe Cafarelli 8f552993a6 Fix NPE on TransportActionListener for elasticsearch 2 2020-02-12 12:14:57 +01:00
Nikolay Martynov fd58187994 Support JFR profiling 2020-02-11 14:55:19 -05:00
Laplie Anderson 9023c6e1c7 Fix a couple tests using the wrong arguments 2020-02-11 11:44:46 -05:00
Laplie Anderson 01b0bebc93 minor formatting 2020-02-11 11:44:46 -05:00
Laplie Anderson 0cb0ae0a48 Less repetition in gradle files 2020-02-11 11:44:46 -05:00
Laplie Anderson c79fe3a459 Create shared internal jar 2020-02-11 11:44:46 -05:00
Laplie Anderson 3c0877e38a Create a shared parent classloader 2020-02-11 11:42:39 -05:00
Tyler Benson 032f22f5a6
Reduce tracer thread count by combining scheduled executors int… (#1200)
Reduce tracer thread count by combining scheduled executors into a single executor.
2020-02-11 08:18:59 -08:00
Tyler Benson 51bffa2e8a Move agent api classes to bootstrap package so they're not analyzed by muzzle
Also move jdbc classes to bootstrap to reduce size and complexity of those reference checkers.

These changes reduce the total file size of these instrumentation classes by 635k, which should also result in decent memory savings.
2020-02-10 15:09:15 -08:00
Tyler Benson 486d98135e Code review changes 2020-02-10 11:37:36 -08:00
Tyler Benson e5980d4812 Fix muzzle 2020-02-10 10:45:58 -08:00
Tyler Benson 75c7769192 Reduce tracer thread count by combining scheduled executors into a single executor.
Pulls out utility classes for reuse by other projects.

This also meant the dependency had to be bundled with dd-trace-ot since it isn't published as a separate dependency.
2020-02-10 10:45:58 -08:00
heathkd 7e62bca7fe limit hibernate latest dependency test to 5.x 2020-02-07 17:57:34 -05:00
Lev Priima 5265a43c6d Reduce use of Thread::setContextClassLoader 2020-02-06 22:53:04 -05:00
Tyler Benson 14b5576cd8
Merge pull request #1201 from DataDog/tyler/upgrade-okhttp
Upgrade OkHttp client to 3.12.8
2020-02-04 15:55:36 -08:00
Tyler Benson cab139e905 Upgrade OkHttp client to 3.12.8 2020-02-04 15:06:44 -08:00
Laplie Anderson a811c027ce
Merge pull request #1188 from DataDog/landerson/finatra
Finatra Instrumentation
2020-02-04 17:36:47 -05:00
Tyler Benson daae198b08 Set dispatcher span on request instead of clear
Clearing the span caused traces to be broken up and reported independently when calling forward/include.
2020-01-30 21:55:16 -05:00
Douglas Q Hawkins 53d32b4324
Merge pull request #1189 from DataDog/dougqh/type-caching2
Revised type cache
2020-01-30 18:47:02 -05:00
Laplie Anderson ed12af6994 Assume parent span is netty 2020-01-30 18:21:49 -05:00
dougqh faeb069424 Adjusting capacity check again 2020-01-30 18:08:49 -05:00
dougqh 235a6470fb googleJavaFormat 2020-01-30 17:45:40 -05:00
dougqh 0f095f0adb Final clean-up
- Removed unused method from earlier version
- Corrected previously overlooked comments that were remnant of prior version
2020-01-30 17:44:24 -05:00
Laplie Anderson 37a279069b Add server tests 2020-01-30 15:23:26 -05:00
Blinkingor 8d83172128 clazz -> segment rename 2020-01-29 18:41:29 +01:00
Blinkingor db485d10da final configString 2020-01-28 16:20:02 +01:00
Blinkingor 811051d111 Remove overlapping tests and fix failing unit test 2020-01-28 16:06:23 +01:00
Blinkingor 10b963d277 Codenarc fixes 2020-01-28 15:35:39 +01:00
Blinkingor cf876b5e67 Fix import order 2020-01-28 15:21:50 +01:00
Blinkingor 1e390984c1 Fix potential StackOverFlowError on regexp 2020-01-28 10:28:37 +01:00
dougqh 176f826a44 Adjusting approximateSize check to be more reliable 2020-01-27 17:44:39 -05:00
dougqh 4c7a0ba7a7 Fixing muzzle?
MuzzlePlugin groovy checks that no threads are spawned because this holds the ClassLoader live.

This was breaking with the caching change because the cache no longer uses the Cleaner service.

This caused a problem because the Thread behind the cleaner is created lazily when the first task is created, but without the cache the creation was delayed.

To solve this, I addressed the original cause of the leak.  The newly created Thread automatically inherits the contextClassLoader of its parent, but that's unnecessary for a cleaner thread.

So I changed the ThreadFactory for cleaner to explicitly null out the contextClassLoader.

We should probably null out contextClassLoader in other thread factories and also reduce our use of contextClassLoaders in general, but that will left to another PR.
2020-01-27 17:14:17 -05:00
dougqh d50f901f39 googleJavaFormat, codeNarcTest, and test reliability 2020-01-27 12:34:39 -05:00
dougqh fb871611b5 Replacing ID generation with WeakReference reuse
First pass at replacing ID generation with WeakReference reuse

In this first version, the Cache<ClassLoader, ID> was replaced with Cache<ClassLoader, WeakReference<ClassLoader>>.

The core cache is still of Cache<TypeCacheKey, TypePool.Resolution> and TypeCacheKey logically remains a composite key of ClassLoader, class name.

The removal of ID assignment means ID exhaustion is no longer na issue, so there's never a need to rebuild the cache.  For that reason, CacheInstance has removed and the core caching logic has been moved into DDCachingPoolStrategy.

While TypeCacheKey remains conceptually the same, the internals have changed somewhat.  The TypeCacheKey now has 3 core fields...
- loaderHash
- loadeRef
- class name

Since loader refs are recycled, the fast path for key equivalence can use reference equivalence of the reference objects.

This change ripples through the CacheProvider-s which also have to store loaderHash and loaderRef.

It may be worth going a step further and switching to a Cache<Loader, TypePool> as well.  That still avoid the creation of many WeakReference-s, since the underlying CacheProvider will hold a canonical WeakReference per ClassLoader.
2020-01-27 12:03:44 -05:00
dougqh cf877f67e5 Working around muzzle quirk
Muzzle doesn't like creation of SecureClassLoader-s, so switching to a URLClassLoader for my placeholder loader in tests
2020-01-27 09:47:36 -05:00
Brian Devins-Suresh 660041636a
Merge pull request #1190 from DataDog/devinsba/hibernate-null-entity-check
Add null check to hibernate decorator
2020-01-24 14:55:11 -05:00
Brian Devins-Suresh 3daad0009d Add null check to hibernate decorator 2020-01-24 10:58:52 -05:00
Laplie Anderson 8ff985afdb Cleaner way to skip netty executor's tasks 2020-01-23 12:04:56 -05:00
dougqh 984d77e44c googleJavaFormat & codeNarc 2020-01-23 11:15:39 -05:00
dougqh 726236bd64 Type cache overhaul
This change overhauls the core type cache

The new approach aims to achieve several things...
1 - cache is strictly bounded -- no variance for number of classes of ClassLoaders
2 - cache is significantly smaller
3 - cache doesn't compromise start-up time
4 - primary eviction policy isn't driven by time
5 - primary eviction policy isn't driven by GC

There are some slight compromises here.
In practice, start-up does increase slightly in a memory rich environment; however, start-up improves considerably in a memory poor environment.

The basic approcach is to have a single unified Guava cache for all ClassLoaders -- nominally keyed a composite of ClassLoader & class name

The ByteBuddy CacheProvider are simply thin wrappers around the Guava cache associated to a particular ClassLoader

However rather than having a large number of WeakReferences floating around.  The cache assigns an ID to each ClassLoader.

To further avoid, consuming memory the cache only preserves a small map of Loader / ID assignments.  This means a ClassLoader may have more than one active ID.

This introduce the possibility for ID exhaustion.  That unlikely case is handle by retiring the internal CacheInstance and starting anew.
2020-01-23 10:55:16 -05:00
Laplie Anderson 68e52497d6 Move to versioned folder name 2020-01-23 10:51:38 -05:00
dougqh 17af9b752c Fix typo in test name 2020-01-23 10:40:48 -05:00
Nikolay Martynov 950389587c Simplify groovy/scala/kotlin compile dependency hack 2020-01-23 09:40:15 -05:00
Laplie Anderson 4e58643bd0 Initial finatra instrumentation 2020-01-22 18:09:10 -05:00
Tyler Benson f9e43516ad Set timeouts on individual tests and increase overall test timeout. 2020-01-22 09:06:03 -08:00
Laplie Anderson 64ed09d76c
Merge pull request #1179 from DataDog/landerson/internal-jar-memory
Use streams directly in DatadogClassLoader
2020-01-16 10:51:42 -05:00
Laplie Anderson 6766e12597
Merge pull request #1173 from DataDog/landerson/weakmap-computeifabsent
Synchronized audit: WeakMap computeIfAbsent
2020-01-15 14:51:29 -05:00
Laplie Anderson 607c8f277e Don't copy to byte[], return input stream directly 2020-01-15 13:38:44 -05:00
Laplie Anderson 68ed1da9c6 Don't call `map.get()` in the `put(..)` case 2020-01-15 12:41:01 -05:00
Laplie Anderson a0b1cd4a75 WeakMap computeIfAbsent 2020-01-15 12:26:15 -05:00
Tyler Benson e31cbd6c89 Add Builder to DDTracer
Deprecate previous constructors.
2020-01-14 17:44:34 -08:00
Tyler Benson a52a1420db
Merge pull request #1165 from DataDog/tyler/servlet
Add servlet integrations for request and response dispatch back in.
2020-01-13 13:17:29 -08:00
Tyler Benson d42e72b683
Merge pull request #1171 from DataDog/tyler/bb-filter
Remove more declared method filtering for optimization
2020-01-13 13:05:28 -08:00
Tyler Benson ef6cd9c52c
Merge pull request #1170 from DataDog/tyler/couchbase
Adjust version compatibility to only 2.x
2020-01-13 13:04:57 -08:00
Tyler Benson 5fe26f004b Remove more declared method filtering for optimization
Same logic that was applied in #1169
2020-01-13 12:52:41 -08:00
Nikolay Martynov 6bf1e159b0
Merge pull request #1167 from DataDog/mar-kolya/upgrade-jmxfetch
Upgrade jmxfetch to latest version
2020-01-13 15:45:12 -05:00
Tyler Benson 77cdd210c6 Also update latestDepTest ranges. 2020-01-13 12:31:38 -08:00
Nikolay Martynov cb7faee63f Remove logging config from jmxfetch setup
It already uses slf4j
2020-01-13 15:23:04 -05:00
Tyler Benson 603155f56f
Merge pull request #1169 from DataDog/tyler/remove-filter
Optimize HasSuperMethodMatcher logic
2020-01-13 12:11:27 -08:00
Tyler Benson a0f0d45c74 Adjust version compatibility to only 2.x
They recently released 3.0.0 which is not compatible with our instrumentation.
2020-01-13 12:10:57 -08:00
Nikolay Martynov dcc5726797 Fix jmxfetch log level and log location config
Looks like this got lost in b505c60543
2020-01-13 15:05:45 -05:00
Nikolay Martynov d55d13e239 Exclude some dependencies from jmxfetch 2020-01-13 14:45:19 -05:00
Nikolay Martynov 8b5d89501b Upgrade jmxfetch to latest version 2020-01-13 14:19:03 -05:00
Tyler Benson 25397fd128 Fix formatting and add test. 2020-01-13 11:18:45 -08:00
Bruce Yu 13b84416b9 Blacklisting headers with -bin suffixes in GrpcExtractAdapter 2020-01-13 11:18:45 -08:00
Tyler Benson 49425e9963 Move request-response linking to main servlet advice
servlet-service is off by default, so we can't rely on it being called there.
2020-01-13 10:48:35 -08:00
Tyler Benson e440eba9a1 Enable Servlet request and response dispatcher instrumentation by default.
Change config for Servlet Filter and Service and JDBC DataSource instrumentation so it must be explicitly enabled (won't be enabled by accident if all of servlet is enabled).
2020-01-10 13:51:21 -08:00
Tyler Benson 43fbf28035 Revert "Remove experimental jdbc and servlet integrations until further evaluation"
This reverts commit 2432a92230.
2020-01-10 12:53:24 -08:00
Tyler Benson 7cb24f35c0 Better error handling for getClientInfo in JDBCDecorator
An exception there caused a span to be created but then never finished.
2020-01-08 15:10:54 -08:00
Tyler Benson 5e8af8439a Optimize HasSuperMethodMatcher logic
Instead of filtering then iterating, just iterate through everything and apply filter inline.

This will help avoid allocation for filter iterator and improve if early match is found.
2020-01-07 17:10:01 -08:00
Tyler Benson 03dd908fdb
Merge pull request #1158 from DataDog/tyler/aws-1-tagging
Add various tags for AWS SDK v1.x
2020-01-03 08:38:56 -08:00
Tyler Benson 4947dc3bd3 Add various tags for AWS SDK v1.x
This brings it inline with the v2 instrumentation.
2020-01-02 13:23:30 -08:00
Tyler Benson dacafe4217
Merge pull request #1157 from DataDog/pawel/rmi-add-client-and-server-info-to-decorators
[rmi] add instrumentation names to client and server decorators, remove default client service name
2020-01-02 11:41:40 -08:00
Laplie Anderson 6b6554996a
Merge pull request #1155 from DataDog/landerson/muzzle-scan-classloader-matcher
Add classloader matcher to muzzle scan
2019-12-30 10:12:59 -05:00
Pawel Chojnacki bbaf7278a6 [rmi] remove rmi client service name 2019-12-24 11:59:29 +01:00
Pawel Chojnacki a4b0dcbc9f [rmi] add instrumentation names to client and server decorators 2019-12-23 19:05:34 +01:00
Pawel Chojnacki 65c701dd51
Merge pull request #1128 from DataDog/pawel/add_rmi_support
Add RMI support
2019-12-23 14:38:46 +01:00
Pawel Chojnacki f71ba14016 [rmi] use simple thread local for propagating context
+ add more documentation
2019-12-23 13:25:14 +01:00
Tyler Benson 9020b74128
Merge pull request #1147 from DataDog/tyler/kafka-async
Add async propagation flags for kafka consumer.
2019-12-20 14:55:36 -08:00
Laplie Anderson f181fa721f Add classloader matcher to muzzle scan 2019-12-20 16:17:08 -05:00
Laplie Anderson 97b947919f Add a comment 2019-12-20 15:27:32 -05:00
Laplie Anderson 7338bbdd06 Register on JAX-RS client instead of builder 2019-12-20 13:15:04 -05:00
Pawel Chojnacki 3f21f6419d [rmi] Split context propagation to server and client instrumentation
+ Separate extra code from advices
 + cleanup helper definitions
2019-12-20 16:16:53 +01:00
Pawel Chojnacki c3308042d3 [rmi] use ObjectEndpoint#toString() to avoid need for reflection to be able to compare object identifiers. 2019-12-20 12:10:35 +01:00
Tyler Benson 32db01200c
Merge pull request #1131 from fujigon/feature/debugging
apply comma split logic to servlet request extract adapter
2019-12-19 09:06:12 -08:00
Pawel Chojnacki bb05700806 [rmi] Improve connection overhead and add additional metadata
- Cache response per connection to ensure as little communication overhead as possible is needed
 - change context payload serialization to use plain hashmap
   + add more logging
 - only set parent context on root entry point
   + use decorator to create spans
 - Use Instrumentation context to avoid reflection
 - separate integration parts into packages
2019-12-19 17:21:16 +01:00
Pawel Chojnacki 832605a01a [rmi] Add distributed context propagation
The flow for context propagation is as follows.

 * <p>We inject into StreamRemoteCall constructor used for invoking remote tasks and performs a
 * backwards compatible check to ensure if the other side is prepared to receive context propagation
 * messages then if successful sends a context propagation message
 *
 * <p>Context propagation consist of a Serialized HashMap with all data set by usual context
 * injection, which includes things like sampling priority, trace and parent id
 *
 * <p>As well as optional baggage items
 *
 * <p>On the other side of the communication a special Dispatcher is created when a message with
 * DD_CONTEXT_CALL_ID is received.
 *
 * <p>If the server is not instrumented first call will gracefully fail just like any other unknown
 * call. With small caveat that this first call needs to *not* have any parameters, since those will
 * not be read from connection and instead will be interpreted as another remote instruction, but
 * that instruction will essentially be garbage data and will cause the parsing loop to throw exception
 * and shutdown the connection which we do not want
2019-12-19 17:21:07 +01:00
Pawel Chojnacki 0ee80a0b95 [rmi] Instrumentation for RMI 1.2 and later
includes instrumentation of client and server endpoints
currently missing passing of execution context from client to server
2019-12-19 17:20:50 +01:00
Tyler Benson fb8f7af342
Merge pull request #1149 from DataDog/tyler/jdk-muzzle
Add option for muzzle validation on the specific JDK version
2019-12-19 08:14:54 -08:00
kfujita b707a3aab9 move multi-value aware logic to Extractor. 2019-12-19 13:33:08 +09:00
kfujita 0f3125498c add test case 2019-12-19 13:33:03 +09:00
kfujita a5e693bb92 assert fix same as JettyServlet3Test 2019-12-19 13:32:09 +09:00
Laplie Anderson dc0cbeb95d apply comma split logic to servlet request extract adapter 2019-12-19 13:32:09 +09:00
Tyler Benson 31b77cbd7b Capture error message even if no exception object provided.
This is currently only used by the GoogleHttpClient instrumentation, but may be used by customers too.
2019-12-18 13:49:24 -08:00
Tyler Benson ded28674d3 Add option for muzzle validation on the specific JDK version
This is still useful to validate various aspects of the integrations even if it doesn't need to check against maven.
2019-12-18 13:40:19 -08:00
Tyler Benson 7095ea3426 Add async propagation flags for kafka consumer. 2019-12-18 10:40:54 -08:00
Tyler Benson 96edbe8b8a Add better exception handling in Agent initialization
An invalid statsd hostname causes an exception for jmxfetch.  This should be an error, but not prevent the app from starting.

Also improved log variable name consistency.
2019-12-17 14:15:42 -08:00
Tyler Benson 99992df998 Fix tagging query string for additional servers
Add test to common test suite and adapt each test.
2019-12-17 09:33:27 -08:00
Tyler Benson a2b86e6b96 Pin kafka latestDepTest version to 2.3.x
2.4.0 introduces test failures:
```
Caused by: java.lang.NoClassDefFoundError: org.I0Itec.zkclient.ZkClient
```
when executing `compileLatestDepTestGroovy`.
2019-12-16 08:51:30 -08:00
Tyler Benson 839de614a5
Merge pull request #1136 from DataDog/tyler/tag-query
Capture full servlet URL in decorator
2019-12-12 15:16:38 -08:00
Laplie Anderson 86c3e80121 End Play 2.6 muzzle at 2.8 because its breaking 2019-12-12 17:47:21 -05:00
Tyler Benson b00cdee54a Capture full servlet URL in decorator
This is needed in order to tag the query string (when configured).

Full test suite will follow in future PR.
2019-12-12 13:47:40 -08:00
Brian Devins-Suresh cf3f04f509
Merge pull request #1129 from DataDog/brian.devinssuresh/jedis-3.0
Add support for jedis 3.0+
2019-12-11 11:22:16 -05:00
Laplie Anderson 8cae56219b Adding some implementation comments 2019-12-10 18:28:21 -05:00
Laplie Anderson f836f8976a Combine onControllerStart and onAbort 2019-12-10 18:07:26 -05:00
Laplie Anderson 73062199b0 Move constants to decorator 2019-12-10 14:00:31 -05:00
Laplie Anderson 5d2d19af75 Rename class since they instrument the context 2019-12-10 14:00:31 -05:00
Laplie Anderson 54ba905224 Add a Resteasy 3.1 test set 2019-12-10 14:00:31 -05:00
Laplie Anderson 498d2bd461 Add ContextRequestFilter instrumentation to JAX-RS 2019-12-10 14:00:31 -05:00
Tyler Benson 307e56714e
Merge pull request #1133 from DataDog/tyler/move-dropwizard-tests
Move dropwizard tests to dropwizard project
2019-12-10 10:51:42 -08:00
Tyler Benson 55fe3ba228 Move dropwizard tests to dropwizard project
This separate directory helps to avoid conflicts with JerseyTest.
ResourceTestRule usage in JerseyTest was causing some odd interactions which resulted in traces not being reported.
2019-12-10 10:08:04 -08:00
Laplie Anderson 7717dbf5d8
Merge pull request #1115 from trask/fix-sporadic-test-failure
Fix sporadic test failure in Elasticsearch53SpringRepositoryTest
2019-12-10 12:58:25 -05:00
Laplie Anderson fa14af40c3 Fix muzzle issue with async instrumentation 2019-12-06 18:18:34 -05:00
Brian Devins-Suresh c7329e4b6b Work around muzzle feature missing 2019-12-06 14:12:46 -05:00
Laplie Anderson d2db6b6fd9
Merge pull request #1098 from DataDog/tyler/jax-rs-annotation-inheritance
Support JAX-RS Annotation Inheritance
2019-12-06 13:49:46 -05:00
Brian Devins-Suresh c72f2d3b45 Fix muzzle test for 3.0 2019-12-06 13:45:05 -05:00
Brian Devins-Suresh 73eb217721 Make matchers more strict 2019-12-05 17:02:39 -05:00
Brian Devins-Suresh 2e5aff391d Use random ports for embedded redis 2019-12-05 16:23:22 -05:00
Brian Devins-Suresh 00fb29f528 Some clean up 2019-12-05 15:59:12 -05:00
Brian Devins-Suresh 7b92c40cbc Add support for jedis 3.0+ 2019-12-05 15:09:58 -05:00
Laplie Anderson 42ac5fc024 Remove synchronization and static helper map 2019-12-04 15:43:28 -05:00
Tyler Benson 57c3ec196f
Merge pull request #1104 from trask/normalize-tag-verification
Normalize tag verification
2019-12-02 10:58:03 -08:00
Trask Stalnaker ae0eb52cd7 Fix sporadic test failure 2019-11-27 10:31:59 -08:00
Trask Stalnaker 847a76cb04 Add missing tag verification 2019-11-22 10:59:47 -08:00
Trask Stalnaker 1987e86ebf Normalize tag verification order 2019-11-22 10:59:45 -08:00
Douglas Q Hawkins 2d87882724
Merge pull request #1112 from DataDog/dougqh/jmxfetch-config-resource-leak
Closing metricconfigs.txt
2019-11-22 13:51:50 -05:00
dougqh 75727840df Closing metricsconfigs.txt 2019-11-22 13:08:56 -05:00
Tyler Benson 10b0f35235
Merge pull request #1090 from DataDog/tyler/servlet-path
Add servlet.path tag
2019-11-22 09:18:14 -08:00
dougqh be9743b86b Eliminate class expression to Advice class
Changing Jasper instrumentation to not load the advice class.
This class was missed originally because it doesn't end in Advice.
2019-11-22 09:51:05 -05:00
Laplie Anderson f0bbd46097 Typo in merge conflict resolution 2019-11-21 13:07:02 -05:00
Laplie Anderson f1362a7d57
Merge branch 'master' into tyler/jax-rs-annotation-inheritance 2019-11-21 12:51:01 -05:00
Laplie Anderson 45c16a1c55 Create Play WS 2.1 Project 2019-11-20 18:13:41 -05:00
Zachary Groves c4cebc30c8
Merge pull request #1097 from DataDog/ZStriker19/remove_duplicate_okhttp_span_and_okhttp_separate_service
Remove duplicate Okhttp span and stop separate service creation of ok…
2019-11-20 17:01:18 -06:00
Laplie Anderson aa46e81e19 Copy instrumentation to jax-rs2 2019-11-20 15:38:17 -05:00
Laplie Anderson 75d66dee21 clean and fix tests 2019-11-20 13:14:35 -05:00
Laplie Anderson 7640e68337 The same classwalking is necessary for @Path 2019-11-20 13:07:04 -05:00
Laplie Anderson fd4e2d09e1 Fix method iterable logic. methodPath, httpMethod working 2019-11-20 10:18:00 -05:00
Laplie Anderson 99acb52589
Merge pull request #1103 from DataDog/landerson/load-generator
Simple load generator tool
2019-11-19 10:19:14 -05:00
dougqh 6b096c2240 Replacing Advice.class references
Replacing Advice.class references with string construction.

This stops the JVM from loading the Advice classes which are used as templates for byte buddy and doesn't ever need to be run directly.

This eliminates ~130 class loads at start-up -- and saves 0.5MiB in metaspace
2019-11-18 16:09:30 -05:00
dougqh 022257ee55 Fixing resource leak
To check for existence, switching from getResourceAsStream to getResource
2019-11-18 10:52:23 -05:00
ZStriker19 caaff02c87 Needed to run Java google format. 2019-11-15 15:30:39 -05:00
Tyler Benson 393a88585a First pass at supporting annotations inheritance for JAX-RS
[skip ci]
2019-11-15 11:53:16 -08:00
ZStriker19 c0ba1d3446 Changed test file permissions to match master. 2019-11-15 13:42:28 -05:00
ZStriker19 348cbc74c2 Changed okhttp file permissions back to original settings. 2019-11-15 13:29:51 -05:00
Tyler Benson a77045b9dc Add more complexity to jersey resource test
This demonstrates a matching failure in our current instrumentation.
2019-11-14 16:01:16 -08:00
ZStriker19 8fb32b5730 Remove duplicate Okhttp span and stop separate service creation of okhttp. Instead okhttp will now appear as part of the service that is using okhttp to make the call. 2019-11-14 17:29:28 -05:00
Laplie Anderson 509e655518 Move play-ws-1 and play-ws-2 into separate packages 2019-11-13 16:41:12 -05:00
Laplie Anderson 03b3fa6651
Merge pull request #1092 from DataDog/landerson/debug-instrumentation-names
Print all instrumentation names when muzzled
2019-11-13 16:36:40 -05:00
Laplie Anderson 251ceeb978 Print all instrumentation names when muzzled 2019-11-13 14:12:26 -05:00
Tyler Benson 53b4e5db19 fix test 2019-11-12 15:17:35 -08:00
Nikolay Martynov 2c4422aa04 Upgradle to gradle 6.0 2019-11-12 16:26:46 -05:00
Tyler Benson bff8303e18 Remove duplicate onRequest call.
This info should already be populated by the servlet framework.
2019-11-11 15:58:31 -08:00
Tyler Benson fdf2241e5f Add servlet.path tag 2019-11-11 15:54:21 -08:00
Laplie Anderson 88902046ce simple load generator tool 2019-11-07 16:50:23 -05:00
Nikolay Martynov 2789324cd5
Merge pull request #1084 from DataDog/mar-kolya/fix-agent-for-zulu8
fix agent for zulu8
2019-11-07 14:39:03 -05:00
Nikolay Martynov 8f80a42c4d Do not delay tracer on OpenJDK8 with JBOSS 2019-11-07 13:45:56 -05:00
Nikolay Martynov f371e391a8 Fix typo 2019-11-07 06:55:31 -05:00
Tyler Benson b002f0c190
Merge pull request #1085 from DataDog/tyler/remove-beta-integrations
Remove experimental jdbc and servlet integrations until further evaluation
2019-11-06 14:31:08 -08:00
Nikolay Martynov df6cb663d4 Fix test for case when tracer install is not delayed 2019-11-06 15:35:01 -05:00
Nikolay Martynov fdfe4b1dc3 More renames due to shadow clashes 2019-11-06 15:12:07 -05:00
Tyler Benson 2432a92230 Remove experimental jdbc and servlet integrations until further evaluation 2019-11-06 11:45:00 -08:00
Douglas Q Hawkins 599d1e0fab
Merge pull request #1082 from DataDog/dougqh/sisu-exclusion
Excluded classes generated by Eclipse Sisu
2019-11-06 14:41:36 -05:00
Nikolay Martynov 42cece67c2 Fix namespace clashes
Using `datadog.trace.agent` to hold agent class causes problems due to
shadowing into this package of other classes.
2019-11-06 13:58:20 -05:00
Nikolay Martynov 17155f7d57 Fix scala warning 2019-11-06 13:58:20 -05:00
Nikolay Martynov 15c1e67334 Fix agent on latest zulu8
The problem was that on zulu8 loading OkHttp touches JFR which in turn
touches log manager - which would break things like JBOSS.

The fix is to delay installing agent (and writer) until log manager
things have settled down - in way similar to jmxfetch.

Unfortunately for 'main' agent this turns out to be more involved
because of classloader shenanigans.
2019-11-06 10:09:50 -05:00
Nikolay Martynov 3715a3df53 Fix ClassLoadingTest
It should use real class to try to load
2019-11-06 09:57:09 -05:00
Nikolay Martynov 58d6103361 Allow big outputs in forked test processes 2019-11-06 09:38:53 -05:00
dougqh 270a440162 Excluded classes generated by Eclipse Sisu
This change excludes classes generated by Eclipse Sisu (part of Equinox DS).

These classes are created by https://github.com/eclipse/sisu.inject/blob/master/org.eclipse.sisu.inject/src/org/eclipse/sisu/space/CloningClassSpace.java
2019-11-05 15:20:52 -05:00
Nikolay Martynov 415a875dfc Add generic is-before-java9 check 2019-11-05 13:08:21 -05:00
Nikolay Martynov b31778a05e Do not use `printStackTrace()` 2019-11-05 13:08:21 -05:00
Nikolay Martynov f236c0e8e0 Reformat some comments 2019-11-05 13:08:21 -05:00
Nikolay Martynov acfa861e4a Fix some typos 2019-11-05 13:08:21 -05:00
Nikolay Martynov 7f5b270c67 Call class loading callbacks from onComplete
This should make things more clear
2019-11-05 13:08:21 -05:00
Tyler Benson 2d3b644c84
Merge pull request #1080 from DataDog/tyler/rename-beta-configs
Rename config key for beta/experimental instrumentation.
2019-11-05 08:44:33 -08:00
Tyler Benson 4f853f5f0c Rename config key for beta/experimental instrumentation.
Postfix the existing config names with `-beta` for `servlet` and `jdbc`.

To avoid any risk that the config was explicitly enabled for an unrelated reason resulting in unexpected behavior when upgrading to the latest version.
2019-11-04 13:41:33 -08:00
Laplie Anderson 4ba0ca0f9d
Merge pull request #1078 from DataDog/landerson/big-integer-ids
Change TraceId/SpanId to BigInteger
2019-11-04 16:34:09 -05:00
Laplie Anderson bc2978e8d8 Use groovy literals for BigInteger 2019-11-04 16:16:03 -05:00
Laplie Anderson a3cd46708b
Merge pull request #1075 from DataDog/landerson/wsclient
Play WS Client instrumentation
2019-11-04 13:57:46 -05:00
Laplie Anderson bf4587c23d Missed a few test helper methods 2019-11-04 12:49:35 -05:00
Laplie Anderson cfc19facf5 Change String ids to BigInteger ids 2019-11-04 11:21:32 -05:00
Nikolay Martynov 92f62ca954 Fix scala warning 2019-11-04 08:22:20 -05:00
Laplie Anderson 9e26da85be wsclient -> play-ws 2019-11-01 14:37:06 -04:00
Laplie Anderson 68f8607dbf Fix latestDepTest dependencies 2019-10-31 11:48:52 -04:00
Laplie Anderson 3a1d331525 WSClient instrumentation 2019-10-31 11:12:00 -04:00
Laplie Anderson 64d6d39e78
Merge pull request #1074 from DataDog/landerson/circular-redirect
Add "testCircularRedirects" option to HttpClientTest
2019-10-31 10:29:36 -04:00
Laplie Anderson 8a2594eeab Add "testCircularRedirects" option to HttpClientTest 2019-10-30 18:17:49 -04:00
Tyler Benson 8e48a70c6d Fix jax-rs Path annotation inheritance
Turns out the class level path annotation overrides parent classes, so we only need to find the nearest one.

We also need to look from the instance’s class, not from where the method is defined.

Commented out a test case because we still need to compile against java 7.  That case will be added back when we upgrade to java 8.
2019-10-30 11:39:06 -07:00
kfujita b0e7e6a3f9 - fix test compile setting
- code format with google formatter
2019-10-30 08:57:13 -07:00
kfujita d8bd12bef8 This patch fixes NPE in jaxrs instrument which occurs when we use interfaces with default method and its impl class.
- reproduce NPE error with JaxRs class and interfaces in JavaInterfaces
- fix the logic target class of method with @Path annotation to avoid NPE
- setting java 1.8 for jaxrs's test
2019-10-30 08:57:13 -07:00
Tyler Benson 607dd553d4 Attempt to remove span-ordering race condition with NettyClientTest 2019-10-28 09:50:56 -07:00
Tyler Benson e4bc90d15d
Merge pull request #1067 from DataDog/tyler/db-connection
Add span when calling DataSource.getConnection with a trace.
2019-10-28 08:41:55 -07:00
Tyler Benson a903f20dc4
Merge pull request #1050 from DataDog/tyler/servlet-depth
Add new integration for Servlet Filters, HttpServlet, and RequestDispatcher
2019-10-28 08:39:56 -07:00
Tyler Benson 6703bd96bb Fix servlet propagation for “tail-call” situations
Sometimes servlets will call another servlet by “resetting” the request object.  This maintains the attributes previously set, including the span.  We want to allow the trace to start if there’s no “active trace” even if there is a span attribute.
2019-10-25 17:01:47 -07:00
Tyler Benson a3a1467192 Add span when calling DataSource.getConnection with a trace.
Disabled by default, and only creates a span if existing trace detected.

To enable:
* System Property: `-Ddd.integration.jdbc.enabled=true`
* Environment Variable: `DD_INTEGRATION_JDBC_ENABLED=true`
(Or more specifically with `jdbc-datasource`.)
2019-10-25 16:02:39 -07:00
Tyler Benson 425156115c Add new integration for Servlet Filters, HttpServlet, and RequestDispatcher
Disabled by default, and only creates a span if existing trace detected.

To enable all of them:
* System Property: `-Ddd.integration.servlet.enabled=true`
* Environment Variable: `DD_INTEGRATION_SERVLET_ENABLED=true`

(They have independent configs as well.  If needed, view the source below.)
2019-10-25 14:47:48 -07:00
Tyler Benson df387b486a
Merge pull request #1064 from DataDog/tyler/spring-controller-refactor
Refactor JAX-RS Annotation and Spring Controller/Handler Instrumentation
2019-10-25 14:44:07 -07:00