Commit Graph

731 Commits

Author SHA1 Message Date
Gary Huang bfc49e869e Make connections always close on test case completion. 2018-07-09 17:55:52 -04:00
Gary Huang 6bd7cf0675 Fixes #379.
Connection pools were creating proxy connection objects, so the proxy objects were cached. By unwrapping the proxies, the connection objects will be correctly cached and identified.

Refactored tests to use connection pools, and hsqldb connections only on statements (since they will not make a USER() query in a getMetaData call).
2018-07-09 17:55:52 -04:00
Andrew Kent 02a3e6a5d3 New Instrumentation API 2018-07-06 17:17:32 -04:00
Andrew Kent 33a139cdcb New Instrumentation outline 2018-06-28 14:54:14 -07:00
Andrew Kent f8e3ac3097 New instrumentation api outline 2018-06-28 14:54:14 -07:00
Andrew Kent 0360bd01ee
Merge pull request #378 from DataDog/ark/cassandra-fix
Remove most shadow relocates
2018-06-27 11:56:33 -07:00
Andrew Kent 3894f829bb Remove most shadow relocates
Shadow relocates are no longer needed because of our new bootstrapping
process.
It's no longer possible for agent dependencies to interfere with the
user's classpath.

The immediate reason for this change is a bug created in the Cassandra
instrumentation.
The Cassandra instrumentation references guava transitive deps from
the datastax driver. These references are re-written by shadow,
causing the instrumentation to reference 'datadog.agent.deps.google.*'
instead of the guava class.
2018-06-27 09:35:54 -07:00
Gary Huang cc7bb7943f
Merge pull request #354 from DataDog/gary/jsp-integration-branch
Generic JSP instrumentation with Tomcat Jasper compilation instrumentation.
2018-06-26 18:35:09 -04:00
Nikolay Martynov 8d6392fa94 Remove timeouts from lagom tests
It looks like server is started lazily and on laptop it may take over
5 seconds in parallel build. This means in may take long time on CI as
well.
It is in fact unikely that server will never return so adding timeout
introduces flakiness and doesn't really protect from any real-life
problems. Instead of hardcoding timeouts just rely on build eventually
giving up on its own one way or another.
2018-06-26 15:48:04 -04:00
Nikolay Martynov 44b63ea5ca Rename AkkaHttpInstrumentation to AkkaHttpServerInstrumentation
To be able to put client instrumentation in more consistent way
2018-06-26 15:41:59 -04:00
Nikolay Martynov 3ac866b0f9
Merge pull request #371 from DataDog/tyler/servlet-user
Set servlet’s user-principal as a tag when available
2018-06-26 13:41:13 -04:00
Andrew Kent 15c0f9a6b2 Akka-http-server renames 2018-06-26 09:55:32 -07:00
Nikolay Martynov a0d99e70fb Add tests to servlet intrumentation to verify that username is set correctly 2018-06-25 21:54:20 -04:00
Gary Huang 46e7036701 fix version scan and test versions 2018-06-25 16:54:45 -04:00
Tyler Benson 9899256381 Set servlet’s user-principal as a tag when available 2018-06-25 11:24:35 +10:00
Gary Huang eddb48f3e2 refactored tests, added parent-children relationship between spans in tests. Still need to figure out version scan and versions to support. 2018-06-22 18:24:06 -04:00
Gary Huang e90bcf624a add unit tests for jsp rendering (servicing requests), change to instrument generic jsp level rendering. 2018-06-22 18:24:06 -04:00
Gary Huang b1d06299cc JSP instrumentation
WIP: more unit tests to come
2018-06-22 18:24:06 -04:00
Nikolay Martynov 7405b7711b
Merge pull request #369 from DataDog/mar-kolya/increase-timeouts-in-url-connection-test
Increase timeouts in UrlConnection test
2018-06-22 17:17:04 -04:00
Nikolay Martynov 60a0722390 Increase timeouts in UrlConnection test
Before those timeouts where set to 10ms which legitimatelly can come
before server (even localhost) has a chance to reply with 'connection
refused'.

This should fix flaky test.
2018-06-22 16:57:31 -04:00
Nikolay Martynov 1fbb33d182 More fixes to Lettuce tests
Lettuce intrumentatio is implemented in a way that after operation has
been performed `span` is not closed syncronously - in fact this happens
on separate thread. This means `spans` for even syncronous operations
may be closed on opposite order.
This means that writing tests that pefrom two operations and expect
two traces is slightly more complicated. In many places we can just
avoid doing that by preparing necessary data in `setup`.

This fixes some of the false negatives in tests.
2018-06-22 16:44:58 -04:00
Nikolay Martynov 770f77a7ff
Merge pull request #367 from DataDog/mar-kolya/reduce-number-of-compile-workers
Reduce number of gradle workers during compile
2018-06-22 16:07:46 -04:00
Andrew Kent 8a666784a6 Close Jetty Handler scope 2018-06-22 11:53:39 -07:00
Nikolay Martynov 7882e4fe23 Make ES tests more deterministic by using listener pool with 1 thread
This should reduce number of false negatives in ES tests
2018-06-22 14:23:06 -04:00
Nikolay Martynov c050b4acd8
Merge pull request #366 from DataDog/mar-kolya/fixes-for-lettuce-tests
Fix some flakiness in Lettuce tests
2018-06-22 11:01:30 -04:00
Nikolay Martynov 389f65687a Fix some flakiness in Lettuce tests
It looks like automatic reconnection was enabled which lead to random
traces popping up in random places when Redis server was shutdown.

Also make sure that server persists only during single test to weedout
all inter-test dependencies.
2018-06-21 17:44:19 -04:00
Nikolay Martynov 79d7b9c062 JMS1/2: change resource names according to PR comments 2018-06-21 14:10:50 -04:00
Nikolay Martynov 37fdbf9188 JMS2: Fix `receiveNoWait` instrumentation and instrument `receive(timeout)` 2018-06-21 13:28:19 -04:00
Nikolay Martynov 3be5a78a53 JMS1: make resouce names more descriptive 2018-06-21 13:28:19 -04:00
Nikolay Martynov fe118adc6e JMS1: Fix `receiveNoWait` instrumentation and instrument `receive(timeout)` 2018-06-21 13:28:19 -04:00
Nikolay Martynov bb4026234d Refactor JMS1 integration tests 2018-06-21 13:28:19 -04:00
Nikolay Martynov beb91446c7 Increase Cassandra embedded server start up timeout
To try to make tests less prone to false positives.
2018-06-21 12:00:26 -04:00
Tyler Benson 83db4e8c4d
Merge pull request #352 from DataDog/tyler/netty
Netty HTTP client and server instrumentation
2018-06-21 14:11:12 +10:00
Nikolay Martynov 9cc99bafbf Simplify Jacoco configuration 2018-06-20 17:13:30 -04:00
Nikolay Martynov e070a9be4a Fix jacoco excluded classes concatenation 2018-06-20 17:05:16 -04:00
Nikolay Martynov 71a991fcaa Update jacoco exclusions to exclude things not hit during test runs 2018-06-20 17:01:01 -04:00
Nikolay Martynov da5cd0724b
Merge pull request #362 from DataDog/mar-kolya/enable-coverage-in-CI
Enable coverage checks in CI for java8
2018-06-20 15:15:49 -04:00
Nikolay Martynov 21d58768cb Lettuce: properly close connection instead of letting it being GCed 2018-06-19 22:41:58 -04:00
Andrew Kent 46b9cd35b9
Merge pull request #350 from DataDog/ark/lagom
akka-http instrumentation and lagom tests
2018-06-19 15:17:09 -07:00
Nikolay Martynov b0c2ab9b68 Lettuce: wait for trace after setup code
Otherwise that trace may land at some random point in time after and
cause flaky tests.
2018-06-19 15:15:29 -04:00
Andrew Kent 519f15bcc6
Merge pull request #357 from DataDog/ark/servlet-async
Close scope for async servlet requests
2018-06-17 23:44:24 -07:00
Tyler Benson c0da1f7093 Add version scan. 2018-06-18 12:52:42 +10:00
Tyler Benson 43e31eae80 Separate out instrumentation for netty 4.0 from 4.1 2018-06-18 12:52:42 +10:00
Tyler Benson 9b3df4ff9e Netty HTTP client and server instrumentation first pass
Doesn’t target right versions. Needs lots of testing.
2018-06-18 12:52:42 +10:00
Andrew Kent 93281f775a Clean up akkahttp-server/lagom tests 2018-06-15 14:30:23 -07:00
Andrew Kent 2cfd7bf911 Allow Unit tests to skip expected instrumentation errors. 2018-06-15 13:25:41 -07:00
Andrew Kent 7ac2bdfc22 Add akka-http to play 2.6 tests 2018-06-15 12:48:57 -07:00
Andrew Kent 5a77ac3749 akka-http sync and async tests 2018-06-15 12:48:57 -07:00
Andrew Kent 6b7dda3657 Instrument akka-http function handler instead of akka-streams 2018-06-15 12:48:57 -07:00
Andrew Kent 004b88f634 Initial Lagom Test 2018-06-15 12:48:57 -07:00
Andrew Kent 109f2cead4 Akka-http instrumentation and test 2018-06-15 12:48:57 -07:00
Nikolay Martynov d278cd48a7
Merge pull request #359 from DataDog/mar-kolya/spymemcached-instrumentation
Make spymemcached disabled by default
2018-06-15 15:29:52 -04:00
Nikolay Martynov 0e62ee8ec6 Make spymemcached instrumentation disabled by default 2018-06-15 14:25:51 -04:00
Nikolay Martynov f9cc33c7c8
Merge pull request #353 from DataDog/mar-kolya/spymemcached-instrumentation
Mar kolya/spymemcached instrumentation
2018-06-15 12:44:42 -04:00
Andrew Kent 6cd8be67d5 Close scope for async servlet requests 2018-06-14 20:23:54 -07:00
Andrew Kent f28445c45f CorrelationIdentifier for MDC frameworks. 2018-06-14 15:33:15 -07:00
Nikolay Martynov 1e4c88ffcb Merge branch 'master' into mar-kolya/spymemcached-instrumentation 2018-06-14 15:38:01 -04:00
Nikolay Martynov dd8f5ced07 Spymemcached: be slightly mode defensive in instumentation 2018-06-14 13:37:35 -04:00
Nikolay Martynov ba37ccc552 Spymemcached: use cache span type 2018-06-14 13:37:35 -04:00
Nikolay Martynov 940633b388 Spymemcached: Make abstract method in CompletionListener 2018-06-14 13:37:31 -04:00
Nikolay Martynov 2c9f8a2b71 Fix ES rest client latest dep tests by forcing 6.2.+ since our instrumentation doesnt work properly with 6.3.+ 2018-06-14 10:31:45 -04:00
Nikolay Martynov 30695555f9 Spymemcached: remove unnecessary unboxing and minor cleanup 2018-06-13 22:43:36 -04:00
Nikolay Martynov dafd52cb7a Spymemcached: add instumentation 2018-06-13 22:38:18 -04:00
Tyler Benson ff56389a56 Upgrade some versions. 2018-06-14 11:56:56 +10:00
Nikolay Martynov 63c4cdff87 Memcached testing: add support for testoingcontainers 2018-06-13 13:06:23 -04:00
Gary Huang adbab8692c
Merge pull request #336 from DataDog/gary/lettuce-integration-branch
lettuce 5 redis client integration
2018-06-13 09:18:01 -04:00
Nikolay Martynov 0fa972ec77 Merge branch 'master' into mar-kolya/slick-instrumentation 2018-06-09 00:06:17 -04:00
Gary Huang 429ee40f81 add workaround to avoid certain commands from crashing the trace agent. 2018-06-08 23:53:13 -04:00
Gary Huang fdc56c0e13 change the way classes are loaded so that classLoaderHasClasses would still work for the classes that are being instrumented and being matched at the same time. 2018-06-08 23:53:13 -04:00
Gary Huang 42b586badf remove recording of redis command arguments for now to avoid leaking sensitive data 2018-06-08 23:53:13 -04:00
Gary Huang 0bc9d9c08d remove string constants in advice class and comment fixes 2018-06-08 23:53:13 -04:00
Gary Huang b694ecdf79 add version scan and rename instrumentation classes 2018-06-08 23:53:13 -04:00
Gary Huang 3d61d839c8 make commands with no return values (void or Mono<Void>) to finish spans early in order to record command ran.
added support for cancellation of commands in async and reactive APIs.
since flux is a stream of events, when the stream is complete, the number of events emitted are recorded in the tag db.command.results.count
created helper class LettuceInstrumentationUtil
2018-06-08 23:53:13 -04:00
Gary Huang bfdf4c858a lettuce reactive api for mono publishers completed. 2018-06-08 23:53:13 -04:00
Gary Huang 989a259ec2 Fix code with suggestions from Tyler regarding previous commit.
no longer includes the command argument if the redis command being called is AUTH since it contains sensitive information.
2018-06-08 23:53:13 -04:00
Gary Huang ae1d9d7c52 complete async unit test and changes to implementation
using handleAsync with BiFunction because lettuce futures almost never throw exceptions on futures

improvement changes to come
2018-06-08 23:53:13 -04:00
Gary Huang 1213311418 lettuce redis client sync and async api instrumentation with sync unit tests complete 2018-06-08 23:53:13 -04:00
Tyler Benson c475ac688c Add additional Elasticsearch tests for Spring Data. 2018-06-09 10:59:08 +10:00
Tyler Benson 206de0fb54 Add some more missing helper classes 2018-06-09 10:59:08 +10:00
Tyler Benson c2771753b7 Suggested changes. 2018-06-08 16:38:51 +10:00
Nikolay Martynov 28c7c30af4 [HttpUrlConnection] Fix HEAD responses
Explain to ratpack that sending body for HEAD requests is not up to
spec [1] and seems to be confusing to clients. This resolves 'Keep-Alive'
mistery and makes disabling 'keepa-live' unnecessary.

[1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
2018-06-07 12:24:16 -04:00
Tyler Benson 9ff09b93e1 Refactor HttpUrlConnection Instrumentation
getOutputStream shouldn’t call getResponseCode, otherwise the request gets screwed up.
2018-06-07 16:25:38 +10:00
Nikolay Martynov 93a41b5064 Fix some typos 2018-06-06 20:47:49 -04:00
Nikolay Martynov 7a280d7e13 Fix some code formatting 2018-06-06 17:11:17 -04:00
Nikolay Martynov ca6fabfd64 Move code from `WrapAdviceUtils` to `DatadogWrapper` 2018-06-06 16:42:23 -04:00
Nikolay Martynov 5f82776fc5 Use assertTraces api in slick tests 2018-06-06 16:41:03 -04:00
Nikolay Martynov 30d640e7ee Provide way to use CLosure to assert tags values 2018-06-06 16:40:34 -04:00
Nikolay Martynov b1bc24dd67 Raname function in TraceAssert to match actual action 2018-06-06 16:40:12 -04:00
Nikolay Martynov d6aaee1bee Check for java concurrent classes prefixes match only if class was not whitelisted already 2018-06-06 15:51:30 -04:00
Nikolay Martynov bbcc2882ab Use test set instead of submodule for Slick tests 2018-06-06 15:49:39 -04:00
Nikolay Martynov 2e5007fbd8 Fix function name in TagsAssert 2018-06-06 10:43:23 -04:00
Nikolay Martynov f92af7d860 Fix Scala Slick instrumentation 2018-06-06 10:43:23 -04:00
Nikolay Martynov b8fdb4acd5 Added Scala Slick test that fails due to executor wrapping problems 2018-06-06 10:43:19 -04:00
Nikolay Martynov 3187f7350f Simplify Calldepththreadlocalmap to not use atomic integers 2018-06-06 09:31:25 -04:00
Tyler Benson d235618a6e Add instrumentation for Java’s UrlConnection
For HttpUrlConnection, trace propagation headers are added to the outgoing request.

Due to the awkward design of the API, the ability to instrument in a natural way was limited, thus the main points instrumented are the get{Input,Output}Stream methods.
2018-06-06 09:12:48 +10:00
Tyler Benson 979aa7838f Clear TEST_WRITER before waiting to ensure accurate timing
Otherwise the waitForTraces at the end might be triggered prematurely.
2018-06-05 14:49:17 +10:00
Tyler Benson 5fe58dc344
Merge pull request #338 from DataDog/tyler/es-meta
Add additional metadata to ES client calls.
2018-05-30 11:58:41 +10:00
Tyler Benson 682fb01350 Wait for initial traces before starting tests
to ensure a late reported trace doesn’t mess up the test assertions.
2018-05-30 11:40:31 +10:00
Tyler Benson b58a7db7a7 Add another helper used by Joiner. 2018-05-30 10:31:00 +10:00
Tyler Benson b5346e89a3 Begin 0.10.0 2018-05-29 18:41:37 +10:00
Tyler Benson cda86ca4ea Wait for cluster to be ready before starting test. 2018-05-29 15:40:05 +10:00
Tyler Benson f9ab729f92 Forgot to inject the helpers. 2018-05-25 20:27:57 +10:00
Tyler Benson de74eb1938 Add additional metadata to ES client calls. 2018-05-25 13:58:44 +10:00
Tyler Benson c03e894870
Merge pull request #329 from DataDog/tyler/elasticsearch-client
Elasticsearch Client Instrumentation
2018-05-24 09:03:20 +10:00
Andrew Kent d9dce43728
Merge pull request #332 from DataDog/ark/playtime
Enable Play, Hystrix, and JaxRS Client Instrumentation
2018-05-22 18:56:00 -07:00
Tyler Benson ec9ce10ca1 Limit version compatibility. 2018-05-23 11:29:13 +10:00
Tyler Benson 0cf8fef8d8 Elasticsearch Instrumentation 2018-05-23 11:29:13 +10:00
Tyler Benson 164689eeb3
Merge pull request #333 from DataDog/tyler/force-latest-tests
Fix version upgrading for latestDepTests
2018-05-23 11:21:28 +10:00
Andrew Kent d90de5741f Enable JaxRS Instrumentation 2018-05-22 18:02:30 -07:00
Andrew Kent 066aac761c Enable Hystrix 2018-05-22 18:01:43 -07:00
Andrew Kent c11de3003d Enable Play Instrumentation 2018-05-22 15:29:46 -07:00
Tyler Benson ce0ef409d8
Merge pull request #330 from DataDog/tyler/aws-versions
Fix AWS version ranges
2018-05-23 07:30:16 +10:00
Tyler Benson 03392f2138
Merge pull request #331 from DataDog/tyler/move-apache-httpclient-tests
Move Apache HttpClient tests to instrumentation
2018-05-23 07:28:04 +10:00
Andrew Kent 176b071300
Merge pull request #324 from DataDog/ark/default_service_name
Remove unknown feature-flag and use generic names as fallback
2018-05-22 14:19:50 -07:00
Andrew Kent c7a5bdcdbb Remove unknown feature-flag and use generic names as fallback 2018-05-22 14:01:08 -07:00
Tyler Benson 5d6fe43dc6 TestHttpServer is no longer used
Move the java version ignores.
2018-05-22 17:12:54 +10:00
Tyler Benson e481092f25 Avoid occasional false test failure. 2018-05-22 16:47:36 +10:00
Tyler Benson a3a1b08fa9 Fix AWS version ranges 2018-05-22 16:45:01 +10:00
Tyler Benson 3781a71e5f Fix version upgrading for latestDepTests 2018-05-22 15:14:11 +10:00
Andrew Kent b7570c7152
Merge pull request #319 from DataDog/ark/muzzle-integration-branch
muzzle integration branch
2018-05-21 11:32:00 -07:00
Andrew Kent 217dd411de Disable muzzle 2018-05-21 10:58:30 -07:00
Andrew Kent 1641c54448 Skip instrumenting slick thread pool 2018-05-18 11:17:27 -07:00
Andrew Kent 0bd7a62464 Flags, Fields, Methods, and builder logic for References 2018-05-18 10:45:05 -07:00
Andrew Kent bf70b3031b Cache classloaders for reference checking 2018-05-18 10:45:05 -07:00
Andrew Kent ad7f6a5e71 Generate at compile-time and lazy-load reference matchers 2018-05-18 10:45:05 -07:00
Andrew Kent 98b44c1dfb Remove ReplaceIsSafeVisitor 2018-05-18 10:45:05 -07:00
Andrew Kent 08df3cef35 Use bytebuddy plugin to add safety checks to instrumentation 2018-05-18 10:45:05 -07:00
Tyler Benson 024d414844 Move Apache HttpClient tests to instrumentation
And add latest test.
2018-05-18 12:01:12 +10:00
Gary Huang 71396aee01 Moved JDBC integration tests to become unit tests 2018-05-15 16:37:55 -04:00
Gary Huang 38dfe96eb0 Fixed java 9 JDBC integration test issue.
Refactored JDBCMaps's getDBInfo utlity function because JDBCMaps is in the bootstrap classloader, and the use of java.sql.* packages in getDBInfo is failing because java.sql.* packages are part of the platform classloader in java 9.
2018-05-15 16:18:55 -04:00
Gary Huang 4b71a21487 Fixed unknown database issue from JDBC Connection constructor exception
Client will now report correct database if JDBC connection is recovered from exception.
2018-05-15 16:18:55 -04:00
Tyler Benson b154f8366c Begin 0.8.1 2018-05-15 12:27:02 +10:00
Tyler Benson dc814aeb6d Fix tests. 2018-05-11 15:19:30 +10:00
Tyler Benson 46878d24d3 Name service based on servlet context
This only applies if a service name hasn’t been set or is empty.

This is particularly useful for environments that deploy multiple war files to the same app server.
2018-05-11 13:59:03 +10:00
Tyler Benson 632d9b8da4
Merge pull request #316 from DataDog/tyler/async-flag
Enable async tracking for some non-async frameworks
2018-05-11 09:30:17 +10:00
Tyler Benson d5d24c50c7 Replace @Unroll’s with a rule to always unroll. 2018-05-10 15:24:59 +10:00
Tyler Benson 9e5e48b96e Add a feature flag to gate the name change 2018-05-10 10:54:15 +10:00
Tyler Benson b9db79e251 Enable async tracking for some non-async frameworks
This will allow tracing of additional work being done inside the servlet context.
2018-05-09 10:40:50 +10:00
Tyler Benson 5e6259233e Remove some other unknowns. 2018-05-08 16:29:55 +10:00
Tyler Benson bae79a0c3a Rename unknown to database to reduce confusion 2018-05-08 16:16:02 +10:00
Tyler Benson 9606dd3467 Fix TraceConfig tests 2018-05-08 11:00:59 +10:00
Tyler Benson 2dd36c2b79 Give more distinct names and migrate test style 2018-05-08 10:03:22 +10:00
Tyler Benson c81615b255 Allow configuration of additional trace annotations
List can be set to empty if needed.
2018-05-07 13:27:23 +10:00
Tyler Benson 441bf59e80 Add tracing for other annotations and config
Allows for specifying additional traced methods via env var or sys prop config.
2018-05-07 13:27:23 +10:00
Tyler Benson c1ac920e97 Change from enum to object for map lookup key 2018-05-07 13:25:08 +10:00
Tyler Benson 122c482dd5 Queue should block. 2018-05-07 13:23:29 +10:00
Tyler Benson 9a52303376 Apply jacoco to every java project
Some projects have ignores/whitelists, but generally coverage is pretty high.

AFAIK, this doesn’t really capture instrumentation test coverage though.
2018-05-04 16:23:32 +10:00
Tyler Benson 7dfdc2d50e Missing autoservice annotation processor 2018-05-04 15:38:41 +10:00
Tyler Benson 114f088766 Fix TLS usage 2018-05-04 14:05:06 +10:00
Tyler Benson f35e2e6ebe Remove various warnings and other cleanup
Also upgrade some plugins.
2018-05-04 14:05:06 +10:00