Commit Graph

474 Commits

Author SHA1 Message Date
Nikolay Martynov 52e6f42f03 Codehale metrics provides servlets that we instrument 2020-03-03 11:45:22 -05:00
Nikolay Martynov bcec89895e Ignore couchbase packaged deps 2020-03-03 11:45:22 -05:00
Nikolay Martynov 4960e0630d Ignore carrotsearch.hppc and codahale.metrics 2020-03-03 11:45:22 -05:00
Nikolay Martynov 9d76281ddf Ignore jcommander 2020-03-03 11:45:22 -05:00
Nikolay Martynov 646bf5bb05 Ignore logback 2020-03-03 11:45:22 -05:00
Nikolay Martynov 31a4c9a427 Ignore kotlin runtime 2020-03-03 11:45:22 -05:00
Nikolay Martynov a59ab43efc Ignore org.objectweb.asm. 2020-03-03 11:45:22 -05:00
Nikolay Martynov dad9e72af7 Ignore more groovy classes 2020-03-03 11:45:22 -05:00
Nikolay Martynov fc5e2cc747 Ignore apache xml tools 2020-03-03 11:45:22 -05:00
Nikolay Martynov fe9cf09249 Ignore jackson and cglib 2020-03-03 11:45:22 -05:00
Nikolay Martynov 73abe16c26 Add a comment 2020-03-03 07:29:22 -05:00
Nikolay Martynov e22ce02762 Add test for ignore matchers for libraries 2020-03-02 16:37:39 -05:00
Nikolay Martynov a1a3957ca8 Rename variable 2020-03-02 16:37:39 -05:00
Nikolay Martynov b73bf22f2d
Merge pull request #1279 from DataDog/mar-kolya/make-profiling-test-more-resilient
Make profiling controller test more resilient to delays
2020-03-02 19:43:02 +01:00
Nikolay Martynov 2eb7aa7d20 Increase CleanerTest timeout to help IBM JVM to cope 2020-03-02 12:11:11 -05:00
Tyler Benson 80f13bc670 Add proper hashcode/equals methods to our matchers
I copied over the methods that would have been generated had the `HashCodeAndEqualsPlugin` actually been working.

Also updated the matcher tests to use the TypePool for more realistic values.
2020-02-28 14:13:18 -08:00
Tyler Benson 55276148b9
Merge pull request #1251 from DataDog/devinsba/fast-reference-matcher
Fail fast in the matcher, let the debug outputs use the cache
2020-02-28 09:56:45 -08:00
Nikolay Martynov cf5fe9feff Merge branch 'master' into mar-kolya/speedup-context-injection-matchers 2020-02-28 02:07:37 +01:00
Nikolay Martynov 3d4ab93888 Merge branch 'master' into mar-kolya/speedup-context-injection-matchers 2020-02-28 02:01:33 +01:00
Nikolay Martynov 0b196ed3ec Merge branch 'master' into mar-kolya/constantify-decorator-matcher 2020-02-28 01:41:02 +01:00
Lev Priima 885212ee36 ClassLoaderHasNoResourceMatcher: put to cache outside critical section 2020-02-27 13:12:23 -08:00
Tyler Benson 68e4bf8375
Merge pull request #1266 from DataDog/tyler/imply-not-interface
Imply not(isInterface()) call for expensive matchers
2020-02-27 12:54:15 -08:00
Tyler Benson 35e00686b3 Imply not(isInterface()) call for expensive matchers
Also remove from simple named("some.name") matchers to avoid metadata lookup.
2020-02-27 12:13:42 -08:00
dougqh 7c68238f6a Merge branch 'master' of github.com:DataDog/dd-trace-java 2020-02-27 14:50:11 -05:00
Tyler Benson 4e8e098330
Merge pull request #1257 from DataDog/tyler/caching-typedescription
Return TypePool.Resolution that caches TypeDescription calls.
2020-02-27 11:12:40 -08:00
Laplie Anderson 6ed5f56bb7
Merge pull request #1255 from DataDog/landerson/spring-blacklist
Add list of blacklisted spring classes
2020-02-27 14:07:42 -05:00
Nikolay Martynov 2f71ad850f Constantify non decorator matcher 2020-02-27 17:52:20 +01:00
Nikolay Martynov d53c138016 Make FieldBackedProvider inject fields only once 2020-02-27 16:24:01 +01:00
Nikolay Martynov 6e10eee6ee Use isEmpty 2020-02-27 15:53:48 +01:00
Nikolay Martynov 70bed6f4ac Remove javadoc from some inappropriate places 2020-02-27 15:53:47 +01:00
Nikolay Martynov 5a17e57e99 Provide `toString` for SafeExtendsClassMatcher 2020-02-27 15:53:46 +01:00
Nikolay Martynov 06b052c4d9 Minor cleanup 2020-02-27 15:53:45 +01:00
dougqh 7511dede5d Merge branch 'master' of github.com:DataDog/dd-trace-java 2020-02-26 15:57:39 -05:00
Tyler Benson f431d5285a Return TypePool.Resolution that caches
Caching focused on the expensive calls primarily used by our instrumentation.
2020-02-26 12:47:48 -08:00
Laplie Anderson 84c7be64b5 Add blacklist directly to global ignores 2020-02-26 11:59:21 -05:00
Brian Devins-Suresh 30e1d56023 Address comments 2020-02-26 09:14:49 -05:00
Laplie Anderson fa3f449ec4 Add list of blacklisted spring classes 2020-02-25 16:33:39 -05:00
Nikolay Martynov 96f74d0fef Compare loader hashes 2020-02-25 21:53:12 +01:00
Nikolay Martynov 82dd2aa1b3 Revert "Make sure that same classloaders get same weak ref"
This reverts commit 50793e5244.
2020-02-25 21:47:42 +01:00
Brian Devins-Suresh 815bd0e9f6 Fix comment 2020-02-25 10:31:52 -05:00
Brian Devins-Suresh 2441ca3332 No more arraylist(0) 2020-02-25 10:02:53 -05:00
dougqh 65c6d17df6 Merge branch 'master' of github.com:DataDog/dd-trace-java 2020-02-25 09:41:40 -05:00
Brian Devins-Suresh 3b78d4593f Don't need to collect mismatches in the critical path 2020-02-25 09:36:00 -05:00
Nikolay Martynov 50793e5244 Make sure that same classloaders get same weak ref 2020-02-25 14:32:35 +01:00
Nikolay Martynov aefcc477cb Simplify TypePoolCacheKey equals
We have to check string equivalence regardless of classloader state
2020-02-25 11:58:18 +01:00
Nikolay Martynov 31b5652d10 Remove reference check from TypeCacheKey 2020-02-25 10:19:47 +01:00
Nikolay Martynov 41db97ea0f Merge branch 'master' into mar-kolya/improve-hash-calclulation-cache-pool 2020-02-25 10:18:25 +01:00
Brian Devins-Suresh 5ebc13cafa Increase performance on java 8 2020-02-24 17:10:41 -05:00
Brian Devins-Suresh af67dfa720 Need to cache the boolean. The list can be taken later as debug is expected to be slower 2020-02-24 14:39:17 -05:00
Brian Devins-Suresh 75af71584c Refactor an agent usage, and fix logic issue 2020-02-24 14:39:17 -05:00
Brian Devins-Suresh e65177a143 Fail fast in the matcher, let the debug outputs use the cache 2020-02-24 14:39:17 -05:00
Tyler Benson c13cb2e60f Move a few other classes. 2020-02-24 08:35:19 -08:00
Tyler Benson 3559ffb21c Add unit tests for matchers 2020-02-24 08:35:19 -08:00
Tyler Benson bd82166b58 Rename methods that don't require disambiguation. 2020-02-24 08:35:19 -08:00
Tyler Benson f70c35842d Move matchers to separate package and split matcher classes out 2020-02-24 08:31:12 -08:00
Tyler Benson faccb0d483 Set ByteBuddy raw setting at startup
This reduces the complexity of TypeDefinitions.
```
Benchmark                                                             Mode  Cnt   Score   Error  Units
ClassRetransformingBenchmark.WithAgent.testTracedRetransform          avgt       22.450          ms/op
ClassRetransformingBenchmark.WithAgent.testUntracedRetransform        avgt        6.842          ms/op
ClassRetransformingBenchmark.WithAgentMaster.testTracedRetransform    avgt       23.188          ms/op
ClassRetransformingBenchmark.WithAgentMaster.testUntracedRetransform  avgt        7.009          ms/op
```
It also improved startup time by about 1 sec.
2020-02-21 10:35:20 -08:00
Nikolay Martynov 00c268e6d8 Add equivalence check to TypeCacheKey 2020-02-21 12:24:53 -05:00
Nikolay Martynov d4c6d86e6d Do not use zero for hashcode
Seems like this may have odd sideeffects
2020-02-21 12:24:09 -05:00
Nikolay Martynov f736c425ff TypeCacheKey are different if hash codes are different
And this is very easy to check
2020-02-21 12:23:41 -05:00
Nikolay Martynov 9d7682f776 Fix hashcode calculation in TypeCacheKey
Using potentially very large number for a mod is probably not very effective
2020-02-21 12:22:51 -05:00
Nikolay Martynov 27842d8479 Ignore cglib packaged with springboot 2020-02-21 11:56:45 -05:00
Nikolay Martynov 0c659770cc Collect all ignore matchers into one
This should allow us to optimize this as a single function later
2020-02-21 11:48:40 -05:00
Lev Priima 37bcb4e288 ClassLoaderMatcher switch to Guava Cache 2020-02-21 03:58:02 -08:00
Lev Priima 52e0eb9d37 SkipClassLoaderMatcher use str-switch in shouldSkipClass(ClassLoader) 2020-02-20 21:02:42 -08:00
Tyler Benson 660921517c
Remove costly matchers from ignore list (#1233)
Remove costly matchers from ignore list
2020-02-20 17:52:43 -08:00
heathkd b7b46caaa8 create AgentBuilder instance with logging listeners only if debug mode is enabled 2020-02-20 16:29:27 -05:00
Tyler Benson e4a454549d
Split interface and class "supertype matching" (#1232)
Split interface and class "supertype matching"
2020-02-20 11:08:39 -08:00
Nikolay Martynov f4d8b6b1f2 Avoid creating context provider for intrumentations that do not need that
This should reduce number of allocations
2020-02-20 08:28:25 -05:00
Laplie Anderson 43487fcfb5 Fix the logic 2020-02-19 21:22:17 -05:00
Laplie Anderson 92b9c07018 Add interfaces only code 2020-02-19 21:22:17 -05:00
Laplie Anderson 147b4a64f1 Add matcher for only concrete classes 2020-02-19 21:22:17 -05:00
Laplie Anderson d45ead1284 split matching into safeExtendsClass and safeHasInterface 2020-02-19 21:22:17 -05:00
Tyler Benson 6e4b55304e Add special handling for ignoring `javax.decorator.Decorator` annotated classes. 2020-02-19 16:24:23 -08:00
Tyler Benson 88aa31b63e Remove costly matchers from ignore list
Removing these matchers caused the time spent matching ignores to go from greater than 1 second to less than 100 ms.
2020-02-19 16:24:23 -08:00
Jean-Philippe Bempel efe8ee375b
Merge pull request #1226 from jpbempel/jpbempel/reuseKeys
Enable reuseKeys on WeakConcurrentMap
2020-02-18 19:09:05 +01:00
Nikolay Martynov fd75575796 Disable peer hostname tag for http servers
Getting peer hostname makes DNS request
2020-02-14 11:25:07 -05:00
jean-philippe bempel 58b1cff62e Enable reuseKeys on WeakConcurrentMap
for each call to get method in WeakConcurrentMap, a LatentKey is
created. Considering the frequency of call it's significant.
Also, the classloader of the tracer can be considered as permanent
through the lifecycle of the JVM which is a prerequisite to be able
to reuse keys (otherwise classloader leak).
2020-02-14 14:52:59 +01:00
Nikolay Martynov fd58187994 Support JFR profiling 2020-02-11 14:55:19 -05:00
Laplie Anderson 9023c6e1c7 Fix a couple tests using the wrong arguments 2020-02-11 11:44:46 -05:00
Laplie Anderson 3c0877e38a Create a shared parent classloader 2020-02-11 11:42:39 -05:00
Tyler Benson 032f22f5a6
Reduce tracer thread count by combining scheduled executors int… (#1200)
Reduce tracer thread count by combining scheduled executors into a single executor.
2020-02-11 08:18:59 -08:00
Tyler Benson 51bffa2e8a Move agent api classes to bootstrap package so they're not analyzed by muzzle
Also move jdbc classes to bootstrap to reduce size and complexity of those reference checkers.

These changes reduce the total file size of these instrumentation classes by 635k, which should also result in decent memory savings.
2020-02-10 15:09:15 -08:00
Tyler Benson 486d98135e Code review changes 2020-02-10 11:37:36 -08:00
Tyler Benson e5980d4812 Fix muzzle 2020-02-10 10:45:58 -08:00
Tyler Benson 75c7769192 Reduce tracer thread count by combining scheduled executors into a single executor.
Pulls out utility classes for reuse by other projects.

This also meant the dependency had to be bundled with dd-trace-ot since it isn't published as a separate dependency.
2020-02-10 10:45:58 -08:00
Lev Priima 5265a43c6d Reduce use of Thread::setContextClassLoader 2020-02-06 22:53:04 -05:00
dougqh faeb069424 Adjusting capacity check again 2020-01-30 18:08:49 -05:00
dougqh 235a6470fb googleJavaFormat 2020-01-30 17:45:40 -05:00
dougqh 0f095f0adb Final clean-up
- Removed unused method from earlier version
- Corrected previously overlooked comments that were remnant of prior version
2020-01-30 17:44:24 -05:00
dougqh 176f826a44 Adjusting approximateSize check to be more reliable 2020-01-27 17:44:39 -05:00
dougqh 4c7a0ba7a7 Fixing muzzle?
MuzzlePlugin groovy checks that no threads are spawned because this holds the ClassLoader live.

This was breaking with the caching change because the cache no longer uses the Cleaner service.

This caused a problem because the Thread behind the cleaner is created lazily when the first task is created, but without the cache the creation was delayed.

To solve this, I addressed the original cause of the leak.  The newly created Thread automatically inherits the contextClassLoader of its parent, but that's unnecessary for a cleaner thread.

So I changed the ThreadFactory for cleaner to explicitly null out the contextClassLoader.

We should probably null out contextClassLoader in other thread factories and also reduce our use of contextClassLoaders in general, but that will left to another PR.
2020-01-27 17:14:17 -05:00
dougqh d50f901f39 googleJavaFormat, codeNarcTest, and test reliability 2020-01-27 12:34:39 -05:00
dougqh fb871611b5 Replacing ID generation with WeakReference reuse
First pass at replacing ID generation with WeakReference reuse

In this first version, the Cache<ClassLoader, ID> was replaced with Cache<ClassLoader, WeakReference<ClassLoader>>.

The core cache is still of Cache<TypeCacheKey, TypePool.Resolution> and TypeCacheKey logically remains a composite key of ClassLoader, class name.

The removal of ID assignment means ID exhaustion is no longer na issue, so there's never a need to rebuild the cache.  For that reason, CacheInstance has removed and the core caching logic has been moved into DDCachingPoolStrategy.

While TypeCacheKey remains conceptually the same, the internals have changed somewhat.  The TypeCacheKey now has 3 core fields...
- loaderHash
- loadeRef
- class name

Since loader refs are recycled, the fast path for key equivalence can use reference equivalence of the reference objects.

This change ripples through the CacheProvider-s which also have to store loaderHash and loaderRef.

It may be worth going a step further and switching to a Cache<Loader, TypePool> as well.  That still avoid the creation of many WeakReference-s, since the underlying CacheProvider will hold a canonical WeakReference per ClassLoader.
2020-01-27 12:03:44 -05:00
dougqh cf877f67e5 Working around muzzle quirk
Muzzle doesn't like creation of SecureClassLoader-s, so switching to a URLClassLoader for my placeholder loader in tests
2020-01-27 09:47:36 -05:00
dougqh 984d77e44c googleJavaFormat & codeNarc 2020-01-23 11:15:39 -05:00
dougqh 726236bd64 Type cache overhaul
This change overhauls the core type cache

The new approach aims to achieve several things...
1 - cache is strictly bounded -- no variance for number of classes of ClassLoaders
2 - cache is significantly smaller
3 - cache doesn't compromise start-up time
4 - primary eviction policy isn't driven by time
5 - primary eviction policy isn't driven by GC

There are some slight compromises here.
In practice, start-up does increase slightly in a memory rich environment; however, start-up improves considerably in a memory poor environment.

The basic approcach is to have a single unified Guava cache for all ClassLoaders -- nominally keyed a composite of ClassLoader & class name

The ByteBuddy CacheProvider are simply thin wrappers around the Guava cache associated to a particular ClassLoader

However rather than having a large number of WeakReferences floating around.  The cache assigns an ID to each ClassLoader.

To further avoid, consuming memory the cache only preserves a small map of Loader / ID assignments.  This means a ClassLoader may have more than one active ID.

This introduce the possibility for ID exhaustion.  That unlikely case is handle by retiring the internal CacheInstance and starting anew.
2020-01-23 10:55:16 -05:00
Laplie Anderson 6766e12597
Merge pull request #1173 from DataDog/landerson/weakmap-computeifabsent
Synchronized audit: WeakMap computeIfAbsent
2020-01-15 14:51:29 -05:00
Laplie Anderson 68ed1da9c6 Don't call `map.get()` in the `put(..)` case 2020-01-15 12:41:01 -05:00
Laplie Anderson a0b1cd4a75 WeakMap computeIfAbsent 2020-01-15 12:26:15 -05:00
Tyler Benson e31cbd6c89 Add Builder to DDTracer
Deprecate previous constructors.
2020-01-14 17:44:34 -08:00