Commit Graph

545 Commits

Author SHA1 Message Date
Eric Gribkoff 4ae7a37d7c
interop-testing: add health service to XdsTestServer (#6891) 2020-04-02 15:39:25 -07:00
Eric Gribkoff 99b6d0156e
buildscripts,interop-testing: Increase logging for xDS tests (#6818) 2020-03-11 20:34:26 -07:00
ZHANG Dapeng b348479ac8
interop-testing: set the right classpath for startscripts
Currently the classpath for all startscripts in interop-testing includes alpnagent and grpc-xds, even if a lot of startscripts don't need them. Made a change to only include alpnagent or grpc-xds when a startscript really needs it.
2020-03-06 11:55:45 -08:00
ZHANG Dapeng 7be75a0bcb
all: let interop test use shaded dependency correctly take 2
First rolls forward #6791, then fixes the issue of missing transitive dependency for the shadow configuration.
2020-03-04 16:10:46 -08:00
ZHANG Dapeng 1df7d7ea8e
Revert "all: let interop test use shaded dependency correctly (#6780)"
This reverts commit c5f48b8e38.  (#6780)

Revert because caused a regression in the ALTS tests. https://source.cloud.google.com/results/invocations/691d9965-fea1-487d-b606-352a5234039e/targets/grpc%2Fcore%2Fpull_request%2Flinux%2Fgrpc_interop_toprod/log

2020-03-01 20:02:12,491 Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang3/SystemUtils
at io.grpc.alts.CheckGcpEnvironment.isRunningOnGcp(CheckGcpEnvironment.java:69)
at io.grpc.alts.CheckGcpEnvironment.isOnGcp(CheckGcpEnvironment.java:44)
at io.grpc.alts.ComputeEngineChannelBuilder.(ComputeEngineChannelBuilder.java:62)
at io.grpc.alts.ComputeEngineChannelBuilder.forTarget(ComputeEngineChannelBuilder.java:72)
at io.grpc.alts.ComputeEngineChannelBuilder.forAddress(ComputeEngineChannelBuilder.java:77)
at io.grpc.testing.integration.TestServiceClient$Tester.createChannel(TestServiceClient.java:399)
at io.grpc.testing.integration.AbstractInteropTest.setUp(AbstractInteropTest.java:309)
at io.grpc.testing.integration.TestServiceClient.setUp(TestServiceClient.java:198)
at io.grpc.testing.integration.TestServiceClient.main(TestServiceClient.java:56)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang3.SystemUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 9 more
2020-03-02 10:22:50 -08:00
ZHANG Dapeng c5f48b8e38
all: let interop test use shaded dependency correctly (#6780) 2020-02-28 10:04:46 -08:00
Eric Gribkoff 161a26ecff
interop-testing: sync SimpleResponse definition with core and go (#6713) 2020-02-14 14:05:23 -08:00
ZhenLian 557da62305
netty: Upgrade Netty (4.1.45.Final) & tcnative (2.0.28.Final) 2020-02-06 10:03:14 -08:00
ZHANG Dapeng 258fe12f02
interop-testing: fix bug of xds test missing transitive dependency
This fixes a bug introduced in #6675 .

grpc-interop-testing should have added all dependency e.g. enovy that grpc-xds needs.
2020-02-05 16:47:13 -08:00
ZHANG Dapeng cd35a8153c
interop-testing: fix bug for xds dependency not published yet
In v1.27.0 release the grpc-interop-testing artifact in maven includes grpc-xds, but grpc-xds is not yet published. It should be removed from the dependency list in maven artifact.
2020-02-04 14:53:31 -08:00
ZHANG Dapeng d8250e9456
interop-testing: log RPC failures for XdsTestClient 2020-01-27 13:58:48 -08:00
Eric Gribkoff e5745a514f
interop-testing: use server hostname instead of id for xds test (#6639) 2020-01-24 14:03:49 -08:00
Eric Gribkoff d914e011b0
interop-testing: Add XdsTestClient and XdsTestServer (#6585) 2020-01-13 23:57:09 -08:00
Chengyuan Zhang b7ccc0d142
core, census, testing, interop-testing, android-interop-testing: move census dependency out of grpc-core (#6577)
Decouples grpc-core with census, while still preserve the default integration of census in grpc-core. Users wishing to enable census needs to add grpc-census to their runtime classpath.

- Created a grpc-census module:
    - Moved CensusStatsModule.java and CensusTracingModule.java into grpc-census from grpc-core. CensusModuleTests.java is also moved. They now belong to io.grpc.census package.
Moved DeprecatedCensusConstants.java into io.grpc.census.internal (is this necessary?) in grpc-census.
    - Created CensusStatsAccessor.java and CensusTracingAccessor.java, which are used to create census ClientInterceptor and ServerStreamTracer.Factory.
    - Everything in grpc-census are package private, except the accessor classes. They only publicly expose ClientInterceptor and ServerStreamTracer.Factory, no Census specific types are exposed.

- Use runtime reflection to load and apply census stats/tracing to channel/server builders, if grpc-census is found in runtime classpath.

- Removed special APIs on AbstractManagedChannelImplBuilder and AbstractServerImplBuilder for overriding census module. They are only used for testing. Now we changed tests to apply Census ClientInterceptor and ServerStreamTracer.Factory just as normal interceptor/stream tracer factory. Test writer is responsible for taking care of the ordering concerns of interceptors and stream tracer factories.
2020-01-13 14:35:29 -08:00
Tomo Suzuki 75f6fd8f10 Upgrade error_prone_annotations to 2.3.4 2020-01-03 14:50:34 -08:00
ZHANG Dapeng 8062406afc
interop-testing,core: interop test to get remote address attributes
Adding interop-test for getting remote server address from client interceptor. Also added this feature to inprocess transport.
2019-12-02 13:30:55 -08:00
Jihun Cho 7db873f1f6
Roll-forward of stub,compiler: generated stub extends Abstract{Async,Future,Blocking}Stub #6196 (#6458)
This reverts commit 2eb3f8c34e (#6317).
2019-11-25 09:41:16 -08:00
Carl Mastrangelo 40bcab5d12 core: use seconds in deadline exceeded string (#6341) 2019-11-05 13:59:20 -08:00
Jihun Cho 296440a4db interop-testing,benchmarks: publish tar, zip 2019-10-31 17:12:30 -07:00
Jihun Cho 2eb3f8c34e
stub,compiler: Rollback Abstract{Future,Blocking,Async}Stub (#6317)
rollback of #6304 & #6196
2019-10-22 13:27:30 -07:00
Jihun Cho 45d49a56cc
stub,compiler: generated stub extends Abstract{Async,Future,Blocking}Stub (#6196) 2019-10-17 14:49:24 -07:00
Jihun Cho e9ac1b4a76
all: update modules to wait until other module's sourceSet is available (#6232) 2019-10-02 15:05:44 -07:00
Carl Mastrangelo ab2aff48dc netty,okhttp,cronet: add option to use get/put when methods are safe/idempotent
This change adds two booleans to the ChannelBuilders to
allow transports to use get and put.   These are currently defaulted to
on, but unset on the method descriptors.   This change is 1/2 that will
allow the safe / idempotent bits to be set on generated proto code.
Part 2/2 will actually enable it.

The use case for this is for interceptors that implement caching logic.
They need to be able to access the safe/idempotent bits on the MD in
order to decide to how to handle the request, even if gRPC doesn't use
GET / PUT HTTP methods.
2019-09-25 14:57:17 -07:00
Eric Anderson 4215b80b81 Apply java plugin explicitly when needed 2019-09-13 09:42:17 -07:00
Eric Anderson 3b29f74271 Move ALPN Agent configuration to each project needing it 2019-09-13 09:42:17 -07:00
Eric Anderson 5b838e5284 Apply maven-publish plugin explicitly when needed 2019-09-13 09:42:17 -07:00
Eric Anderson 3c3a823a81 Swap to Gradle's Plugin DSL for much of build
Examples and android projects were left unchanged. They can be changed
later.

No plugin versions were changed, to make this as non-functional of a
change as possible. Upgrading Gradle to 5.6 was necessary for
pluginManagement in settings.gradle.
2019-09-13 09:42:17 -07:00
Igor Bernstein 271cbff1b8 core: Migrate to new OpenCensus method & status tags (#5996)
Fixes #5593 and supersedes #5601

Now that https://github.com/census-instrumentation/opencensus-java/pull/1854 has been merged & released as 0.21.0. We can start using the method & status tags.

Background:
Opencensus introduced new tags for status and method (https://github.com/census-instrumentation/opencensus-java/pull/1115). The old views that used those tags were deprecated and new views were created that used the new tags. However grpc-java wasn't updated to use the new tags due to concern of breaking existing metrics. This resulted in the old views being deprecated while the new views were broken (goomics #50).

https://github.com/census-instrumentation/opencensus-java/pull/1854 added a compatibility layer to opencensus that would remap new tags to old tags for old views. This should unblock grpc to switching to the new tags while allowing old views to still be populated. That commit was released as part of opencensus 0.21, which grpc currently uses
2019-08-01 10:37:04 -07:00
Jihun Cho 65109e6738
netty: Netty{Server,Channel}Builder requires all or none of ELG and ChannelType (#6014) 2019-07-26 09:25:02 -07:00
liym 47b11ab7a1 compiler: Use 'SERVICE_NAME' instead of duplicated '$Package$$service… (#5943)
* compiler: Use 'SERVICE_NAME' instead of duplicated '$Package$$service_name$'

* compiler: Align indentation

* Fix typo

* Add modified golden files and all re-generated code to meet Travis CI and Windows build requirements

See PR #5943

* Polishing
2019-07-24 10:37:13 -07:00
Carl Mastrangelo cc524f10d3
netty,interop-testing: increase timeouts on tests for TSAN 2019-07-19 15:24:19 -07:00
Carl Mastrangelo c3d7d74175
interop-testing: fix race in CascadingTest
ServerCall.close() is meant to only be accessed from a single thread, but in the test it is accessed from the callbacks of several client calls.  Synchronize access to sate TSAN.
2019-07-15 11:17:54 -07:00
Carl Mastrangelo d5e1a4bb5d
interop-testing: support just ServerBuilder types 2019-06-26 16:47:17 -07:00
Kun Zhang ddbaf743cc
core: attach debug information about stream to DEADLINE_EXCEEDED (#5892)
Works for #4740 

- Subclasses of `AbstractClientStream` include remote address in insight if available.
- `DelayedStream` adds buffered time, and the insight of real stream if it's set.
- `RetriableStream` insights outputs of Substreams.

Example error message:
```
deadline exceeded after 8112071ns. [buffered_nanos=24763, remote_addr=foo.test.google.fr/127.0.0.1:44749]
```
or
```
deadline exceeded after 8112071ns. [buffered_nanos=22344324763, waiting_for_connection]
```

This is related to #4776 but taking a more usage-specific approach.
2019-06-19 17:30:44 -07:00
Eric Anderson 675b37a027 Revert "interop-testing: Observe flow control in TestServiceImpl"
This reverts commit e795f14bed. It breaks some of
our internal tests. At the very least the cast fails because we are reusing the
service for something without going through the normal stub. There may also
have been a test hang, but it'd take more effort to figure out.
2019-06-14 15:17:22 -07:00
Eric Anderson e795f14bed interop-testing: Observe flow control in TestServiceImpl 2019-06-14 09:01:29 -07:00
Carl Mastrangelo 44ecdf3649
interop-testing: fix tests on Android 2019-06-07 15:01:21 -07:00
Eric Anderson ee5731cc18 interop-testing: Only set okhttp's sslSocketFactory for test CA
We want the interop client to be configured like a normal user would,
and a normal user wouldn't call sslSocketFactory to use the default
roots.
2019-06-06 10:30:10 -07:00
Carl Mastrangelo 9ef0e9fc1b
interop-testing: disable timeout when debugging 2019-06-05 23:39:43 -07:00
Kun Zhang 276b7d8512
interop-testing: create GrpclbLongLivedAffinityTestClient (#5817)
This is a long-running stand-alone test client for a specific customer that uses GRPCLB's pick_first mode.
2019-05-31 12:54:05 -07:00
Tim van der Lippe ad0893737e Migrate org.mockito.Matchers#any* to org.mockito.ArgumentMatchers
The former is deprecated and replaced by the latter in Mockito 2.
However, there is a functional difference: ArgumentMatchers will reject
`null` and check the type if the matcher specified a type (e.g.
`any(Class)` or `anyInt()`). `any()` will remain to accept anything.
2019-05-29 16:53:01 -07:00
Yang Song 46f34a513f core: Make OpenCensus tags non-propagating. (#5689)
The `method` and `status` shouldn't be propagated in the first place,
but in previous OpenCensus implementation all tags are propagating by
default. Now with the TagMetadata it may make sense to change them to
local tags.

This will be a breaking change to users who depend on the behavior that
these tags propagate through process boundaries.
2019-05-24 14:32:26 -07:00
Yang Song ea4e2bffec all: Upgrade OpenCensus versions. (#5657)
Also updated CensusModule to use the new helper methods ContextUtils.withValue() instead of directly manipulating the context keys. See census-instrumentation/opencensus-java#1864.
2019-05-08 16:05:37 -07:00
Carl Mastrangelo 0c304b1863
interop-testing: remove Truth method named() 2019-04-24 16:05:08 -07:00
Kun Zhang ba335f5e68
interop-test: add test case for "pick_first" picking behavior (#5554)
New fields are added to the test protos according to grpc/grpc-proto#51

This test needs to be run in an environment where "pick_first" or "grpclb" with child policy "pick_first" is configured.
2019-04-10 14:08:54 -07:00
ZHANG Dapeng 40526086eb
interopt-testing: move mockito from compile to testCompile (#5541)
Resolves #5540
2019-04-03 14:09:03 -07:00
apolcyn 1eb6fc523e interop-testing: implement compute engine channel creds test case in Java 2019-03-21 10:34:51 -07:00
Tim van der Lippe d35fbd7eee all: Update to Mockito 2
This is the public port of cl/238445847

Fixes #5319
2019-03-19 14:17:52 -07:00
Jiangtao Li 185cf3d047
alts: add ComputeEngineChannelBuilder to interop test (#5475)
* alts: add ComputeEngineChannelBuilder to interop test

* alts: rename to compute_engine_channel_creds
2019-03-13 18:50:08 -07:00
Eric Anderson b48b0ac1d4 all: Stop committing generated protobuf messages
This commit swaps to using a Sync task to place generated code in the
src/generated folder instead of the gradle-protobuf-plugin's
generatedFilesBaseDir. This provides much nicer results on failed
builds, and you will no longer see all the generated files deleted.

But at the same time the Sync task makes it easy to only copy the
grpc-generated code. This was not previously done because we were lazy
and using generatedFilesBaseDir, which made it difficult to treat the
services differently from the messages.
2019-03-05 16:28:55 -07:00
Kun Zhang 83b92cfc9f
core: pass transport attributes to ClientStreamTracer.Factory.newClientStreamTracer() (#5380)
This will be a new override.  The old override is now deprecated.

In order to pass new information without adding new overrides, I shoved most information
into an object called StreamInfo.  The Metadata is left out to draw attention because
it's mutable.

Motivation: this is needed for correctly supporting pick_first in GRPCLB.  GRPCLB needs to
add a token to the headers, and the token varies among servers.  With round_robin, GRPCLB
create a Subchannel for each server, thus can attach the token when the Subchannel is picked.
To implement pick_first, all server addresses will be put in a single Subchannel, we will
need to add the header in newClientStreamTracer(), by looking up the server address from
the transport attributes and deciding which token to add.
2019-02-21 11:13:51 -08:00
Eric Anderson eaca73473c
Upgrade to protobuf 3.6.1
For Bazel, we upgrade to protobuf 3.6.1.2 and javalite HEAD to fix
incompatibilities in newer Bazel releases.

compiler/Dockerfile is unused, so it was removed instead of being updated.

protoc no longer includes codegen for nano, so we remain on the older protoc
any time nano is used.

Protobuf now requires C++11 when compiling, so windows was swapped to
VC 14.
2019-02-07 13:40:53 -08:00
Carl Mastrangelo f6ec07d87d
core,netty: expose listening on multiple ports 2019-02-06 15:49:59 -08:00
Carl Mastrangelo 3a39b81cf5
all: remove java6 type args 2019-02-04 10:03:50 -08:00
ZHANG Dapeng 0dbab26bb4
all: fix lint 2019-02-01 17:06:27 -08:00
apolcyn 574c053ebb interop-testing: Add GoogleDefaultCreds test case for java 2019-02-01 16:49:54 -08:00
Eric Anderson 473a7d1ce7
interop-testing: Simply ProxyTest and shutdown handling 2019-01-30 16:23:26 -08:00
Eric Anderson c2f8d83663
gae-interop-testing: Remove jdk7 test
GAE Java 7 is dead as of January 16.
https://cloud.google.com/appengine/docs/deprecations/java7
2019-01-24 08:54:47 -08:00
Carl Mastrangelo d7659411eb
interop-testing: deflake MoreInProcessTest 2019-01-22 18:00:14 -08:00
Jihun Cho 08efd978d7
all: remove duplicated dependency (#5207) 2019-01-03 10:50:44 -08:00
Eric Anderson 330f45a2cd interop-testing: Disable flaky ProxyTest.smallLatency
Fixing the flakiness is tracked in #2951. The current flakiness is ~3/8
fail.
2018-12-14 16:01:39 -08:00
Thomas Broyer 2ffc46d6fa Update net.ltgt.errorprone to 0.6 and enable Error Prone on JDK 10+ 2018-12-13 10:17:06 -08:00
Eric Anderson 219c486d4e interop-testing: Sample to reduce ProxyTest flakiness
This also reduces the time of the bandwidth tests from ~6 seconds to ~2,
each (they have about 1s of overhead).

Fixes #2951. The flake is very frequent in #4980.
2018-12-13 10:02:18 -08:00
Eric Anderson de9bc0bcf1 Specify Locale to toLowerCase and lint fixes 2018-12-06 17:15:48 -08:00
Arnout Engelen 3dab7aed2f netty: Client-side support for h2c via Upgrade
Fixes #4518
2018-11-30 18:10:27 -06:00
Kun Zhang b5acbedd55
core: record real-time metrics to OpenCensus (updated to 1.18.0) (#5099)
Real-time metrics are total sent/received bytes and messages per
method, and are updated as the events occur rather than at the end of
RPCs.
2018-11-29 16:30:58 -08:00
Kun Zhang 2961857451
core: refactor flags in CensusStatsModule. (#5095)
There are currently three boolean flags, and there will be one more
soon.  Put them all in the top-level class instead of passing them as
arguments on lower levels.
2018-11-28 14:20:40 -08:00
Yang Song 09b13fecaa core: Update OpenCensus version to 0.17.0 (#4494) 2018-11-09 16:44:01 -08:00
Jihun Cho 51ab5b9432
interop-testing: update test proto to match grpc-proto. (#5003)
Update test proto to match stubby4 test server / grpc-proto repo.
- Deprecated PayloadType since it only provide 1 option.
- Change test cases to ignore deprecated field in Payload
2018-10-30 11:16:25 -07:00
Thomas Broyer 183e1f6735 all: update Error Prone to 2.3.2
This will allow enabling Error Prone on JDK 10+ (after
updating the net.ltgt.errorprone plugin), and is also a
prerequisite to that plugin update.

Also remove net.ltgt.apt plugin, as Gradle has native
support for annotationProcessor.
2018-10-19 13:08:36 -07:00
Kun Zhang fbfc3a40d0
core: add Grpc.TRANSPORT_ATTR_LOCAL_ADDR (#4906)
Resolves #4135
2018-10-03 16:43:37 -07:00
Carl Mastrangelo b0f423295b
all: use Java7 brackets 2018-09-14 13:52:29 -07:00
ZHANG Dapeng 7270938ebf
export CL: Fix misspelling. 2018-09-12 13:30:52 -07:00
Eric Gribkoff d2c0653b2e interop-testing: add very_large_request case to test client (#4816)
interop-testing: add very_large_request case to test client
2018-09-06 16:01:35 -07:00
Jiangtao Li 8d6ba2335a
alts: add Google Default Channel implementation (#4742)
alts: add Google Default Channel implementation
2018-08-27 15:27:35 -07:00
Eric Anderson 7374b78470 log exception to FINE 2018-08-17 17:25:32 -07:00
Eric Anderson 38af7ebf84 interop-testing: Clean up even if interrupted
Most environments won't trigger an interruption, but AppEngine will if
the request takes too long.
2018-08-17 17:25:32 -07:00
zpencer d1146e2015
interop-testing: timeoutOnSleepingServer should check that RPC finished (#4766)
If the RPC did not finish, then the remaining checks are red herrings.
2018-08-16 15:03:41 -07:00
Arnout Engelen 13aa02829e Correctly determine --use_alts default (#4538) 2018-08-14 13:44:43 -07:00
Eric Anderson 75b48b4dfe interop-testing: Add special_status_message test 2018-07-26 14:56:35 -07:00
Eric Anderson 5823fffa4c Upgrade to Truth 0.42
Truth 0.42 brings in some Java 8 bytecode, but they are only in
annotations. So we remove them for gae-java7, otherwise they cause the
build to fail with messages like:

> Unable to stage app: Class file is Java 8 but max supported is Java 7: com/google/auto/value/extension/memoized/Memoized.class in /usr/local/google/home/ejona/clients/grpc-java/gae-interop-testing/gae-jdk7/build/exploded-grpc-gae-interop-testing-jdk7/WEB-INF/lib/auto-value-annotations-1.6.2.jar
> Unable to stage app: Class file is Java 8 but max supported is Java 7: org/checkerframework/dataflow/qual/Pure$Kind.class in /tmpfs/src/github/grpc-java/gae-interop-testing/gae-jdk7/build/exploded-grpc-gae-interop-testing-jdk7/WEB-INF/lib/checker-qual-2.5.3.jar

I manually tested the interop client with Java 7 and it ran without
issue.

This fixes the issues experienced with Truth 0.42 before in #4664.
Related: google/truth#479
2018-07-23 15:53:12 -07:00
ZHANG Dapeng 8be92a1e0a
interop-testing: Fix clientStreaming interop test does not fail on missing trailers
If the server sends all response message but does not send the trailers, client throws
StatusRuntimeException: INTERNAL: Received unexpected EOS on DATA frame from server
2018-07-20 16:54:33 -07:00
Carl Mastrangelo 146b6006b3
compiler,stub: update RpcMethod docs and usage 2018-07-12 17:01:47 -07:00
jbingham-google ffcb3b964b compiler, stub: Rename inputType and outputType in @RpcMethod 2018-07-10 13:24:50 -07:00
jbingham-google 9229e30287 compiler, stub: Add @RpcMethod annotation
This annotation will enable Java APT to generate code.

Addresses part of #3173.
2018-07-06 17:02:01 -07:00
Eric Gribkoff 9f02381938
android-interop-testing,interop-testing: use AbstractInteropTest on Android (#4541) 2018-06-13 10:06:43 -07:00
ZHANG Dapeng 5ce10f0146
all: add gradle format checker
This PR adds an automatic gradle format checker and reformats all the *.gradle files. After this, new changes to *.gradle files will fail to build if not in good format, just like checkStyle failure.
2018-06-11 18:35:18 -07:00
ZHANG Dapeng 9d26c5c405
all: fix lint warnings 2018-06-11 12:57:27 -07:00
Eric Gribkoff d227852b30
okhttp: deprecate OkHttpChannelBuilder#negotiationType (#4533) 2018-06-05 12:27:31 -07:00
Carl Mastrangelo 4c4fda3e5d
stub: remove static Method descriptors and stabilize method accessors 2018-06-05 11:19:28 -07:00
Carl Mastrangelo 60a0b0c471
all: normalize copyright header 2018-05-03 14:55:21 -07:00
Carl Mastrangelo 26581562f0
interop-testing: normalize integration proto directories 2018-04-30 11:30:49 -07:00
zpencer 9ada30b25d
(low priority) core,netty,interop-testing: stabilize maxInboundMessageSize API (#4399)
On server side, `maxMessageSize` is deprecated for
`maxInboundMessageSize` to match the channel builder.

Update usages to use new setter.
2018-04-27 16:01:16 -07:00
zpencer 2347384256
GAE: call channel.shutdown() (#4274)
OkHttpClientInteropServlet and NettyClientInteropServlet both run the
@After method from AbstractInteropTest. Let's make sure we await
termination.

For the long lived channel test, do the cleanup in `destroy`.
2018-04-02 16:02:55 -07:00
Shohei Kamimori 156cc44c5f interop-testing: fix empty proto package name
Intended to resolve #4241
grpc-java has already diverged this proto file, so changing the
this package name is reasonable for now.
2018-03-29 10:37:53 -07:00
Eric Anderson 4e82e62eaa
Fix compilation in Java 9 2018-03-28 17:13:39 -07:00
Arnout Engelen 03a00aa8cf interop-testing: client compressed tests without probing (#4279) 2018-03-28 15:51:03 -07:00
zpencer 14003c14cc
build.gradle: bump protobuf plugin to 0.8.5 (#4101)
This update automatically adds generated sources and proto IDLs to the
`idea` plugin.
2018-03-26 17:29:55 -07:00
Eric Anderson 25f357699a
okhttp: Convert to internal ConnectionSpec eagerly
This allows ProGuard to remove OkHttp's ConnectionSpec in most cases,
saving about 40 methods. The savings won't be realized until
DEFAULT_CONNECTION_SPEC is removed.
2018-03-26 15:39:56 -07:00
Eric Anderson 7eab0d9468 netty: Add support for Conscrypt 2018-03-23 10:05:45 -07:00
Eric Anderson 0859d480e7 Move TlsTest from interop-testing to netty 2018-03-23 10:05:45 -07:00
Eric Gribkoff 52fedb624d
okhttp: support JDK9 ALPN (#4136) 2018-03-20 17:29:24 -07:00
Eric Anderson ca7b3dad11
interop-testing: Improve failure messages for ping_ping
Timeouts and Status returns can be common, and previously those cases
weren't that clear. For example, if there was a StatusRuntimeException
it would just print the status code and message, but not the causal
chain.
2018-03-07 13:28:41 -08:00
Eric Anderson 26719bdc5c Partially revert "core: change serverimpl,servercallimpl's internalclose to cancel stream (#4038)"
This partially reverts commit 48ca4527c1.
It leaves the changes to ServerCallImpl and test.

This also partially reverts "Lint fixes" commit
3002a23a0f which removed unused variables
which are now necessary again.

This is reverted for the combined result of two issues:
* Some users are testing that they get UNKNOWN when the service throws.
  That's not unreasonable given the behavior was well-publicised when it
  changed in v1.5. We should probably keep the UNKNOWN in some common
  cases (like the service threw immediately, before sending anything).
* The client could see CANCELLED instead of INTERNAL as had been
  intended. It's unclear as to why (I didn't investigate heavily). This
  behavior is visible in MoreInProcessTest and was overlooked during
  review.
2018-03-02 13:35:06 -08:00
zpencer 066ad3ceac
buildscripts,travis: fetch from mvn with retries (#4140)
A band aid for #3284, to make its symptoms less noticeable.
2018-03-01 19:11:24 -08:00
Carl Mastrangelo 7af2373a03
core,netty,okhttp,alts,inprocess: deprecate usePlaintext(boolean) 2018-02-28 08:53:14 -08:00
Jiangtao Li d8630f2521 alts: add ALTS to interop tests 2018-02-23 12:44:18 -08:00
Rama Chavali 48ca4527c1 core: change serverimpl,servercallimpl's internalclose to cancel stream (#4038)
The HTTP/2 error code will be INTERNAL_ERROR for all cancel statuses,
except for DEADLINE_EXCEEDED and CANCELLED, which are mapped to
CANCELLED.
2018-02-22 11:09:21 -08:00
Eric Gribkoff 6f9b4e87e1
compiler: avoid invoking experimental method in generated code 2018-02-08 11:25:38 -08:00
Kun Zhang df65bef881
core: delete outboundMessage() and inboundMessage() on StreamTracer. (#4014)
They were deprecated in 1.7.0.
2018-01-29 13:42:14 -08:00
Eric Gribkoff d0bbeced80
interop-testing: fix race in TestServiceImpl 2018-01-10 14:27:15 -08:00
Eric Anderson af0283477d Update ErrorProne to 2.2.0 and fix failures 2018-01-10 14:14:27 -08:00
Eric Anderson 4bc0c95d0b Update ErrorProne to 2.1.3 and fix failures
The fixes could have subtle side-effects, but I did take care when making them.
2018-01-09 12:40:55 -08:00
Eric Anderson 35a6bf863d Enable the Guava Beta Checker 2018-01-08 14:52:51 -08:00
zpencer 173ca5d332
cronet, grpc-lb, interop-testing: fix lints #3848 2017-12-07 16:09:43 -08:00
Eric Anderson 2d88269965 Update to Truth 0.36
Due to transitive dependencies, this also upgrades Guava to 22. However
Truth is only used in our tests, so our users should be unimpacted.
2017-12-04 13:48:00 -08:00
Kun Zhang 3f1d370e47
interop-testing: stop using MockableSpan. (#3823)
Interop tests use actual Census implementations, while MockableSpan is
compatible with the Census OSS implementation, it doesn't work with
the google internal implementation, thus the tests will fail.
2017-12-04 11:13:58 -08:00
Kun Zhang efcd134209
interop-testing: start a new server for each test method. (#3816)
By doing this we can isolate the Census records for each test, and
eliminate the trial-and-error workaround in AbstractInteropTest.

This is a preferred fix for #3777 and supersedes #3803
2017-12-01 13:08:03 -08:00
Carl Mastrangelo aee5fc4176
all: update to proto 3.5.0 2017-11-30 11:50:19 -08:00
Eric Anderson 2bde25d2d9 testing: Remove DeadlineSubject
The class is still used internally, so we move it to context's tests for
it to be reused. To avoid a circular dependency with context's tests
depending on core's tests, StaticTestingClassLoader was also moved to
context's tests.

This is driven by a need to modernize DeadlineSubject for newer versions
of Truth, but the newer versions of Truth update Guava. To avoid leaking
the Guava update to all users of grpc-testing, we're removing the
Subject. In our internal tests we can update the Truth dependency with
less issue.
2017-11-22 17:57:46 -08:00
Eric Anderson 6bab82eeb6 auth: Use async version of getRequestMetadata
This avoids using DelayedStream and a thread hop when the credentials
are known immediately.
2017-11-17 11:52:07 -08:00
Kun Zhang ca9a41a46e
core: record RPC upstarts to Census. (#3708)
RPC upstarts are counted into metrics
RPC_{CLIENT,SERVER}_STARTED_COUNT. In addition, RPC completions are
counted into metrics RPC_{CLIENT,SERVER}_FINISHED_COUNT.  From these
metrics, users will be able to derive count of RPCs that are currently
active.
2017-11-10 17:22:22 -08:00
sebright ef2ec94911 core: use new OpenCensus stats/tagging API. (#3647)
This commit updates gRPC core to use io.opencensus:opencensus-api and
io.opencensus:opencensus-contrib-grpc-metrics instead of
com.google.instrumentation:instrumentation-api for stats and tagging. The gRPC
Monitoring Service continues to use instrumentation-api.

The main changes affecting gRPC:

- The StatsContextFactory is replaced by three objects, StatsRecorder, Tagger,
  and TagContextBinarySerializer.

- The StatsRecorder, Tagger, and TagContextBinarySerializer are never null,
  but the objects are no-ops when the OpenCensus implementation is not
  available.

This commit includes changes written by @songy23 and @sebright.
2017-11-07 12:25:03 -08:00
Carl Mastrangelo 1bc7d76d3a
interop-testing: make max size tests more deterministic
This change removes some of the non deterministic size outputs for
proto.  Instead of serializing the message and measuring the size,
measure before serialization.  As long as the remote always returns
using the same encoder, this should be stable.
2017-11-06 12:34:10 -08:00
Eric Anderson f9f603570d Allow tests to run on IPv6-less machines
Our Travis-CI builds are failing with "Protocol family unavailable" due
to the usage of ::1. Although it's 2017 and we'd expect to have ipv6
_loopback_ anywhere that mattered, apparently that's not the case.

The tests now work equally well on IPv4-only and IPv6-only machines.
2017-11-02 13:32:34 -07:00
Kun Zhang d87ef74082
core: set sampled for local span per MethodDescriptor. (#3627)
This moves away from the global String-based Span name registry which
is not as flexible as we desire.

Also renamed the option name to be more accurate.  This is not
API-breaking because the origianl addition to MethodDescriptor and
code-gen didn't make it into the 1.7.0 release.
2017-11-01 16:46:05 -07:00
ZHANG Dapeng 8a9660c700
testing: move TestUtils.recordServerCallInterceptor internal 2017-11-01 09:37:42 -07:00
ZHANG Dapeng d6ff4c259f
testing: move io.grpc.testing.StreamRecorder to internal 2017-11-01 09:37:22 -07:00
Carl Mastrangelo 87c8791983
compiler: remove references to static fields 2017-10-31 15:24:48 -07:00
Carl Mastrangelo 30b59885b7
compiler: add methods for accessing method descriptors
* MethodDescriptor is lazy loaded, so protobuf loading only happens on demand.  This also means tracing registration happens  on demand.
* The names of the getters all being with `method`.  This makes it harder for autocomplete to pick them up.
* A new field is used, which matches the getter name.  Rather than make the new-getters reference the old-fields, make the old-fields reference the new getters.  This makes removal of the old-fields a simple operation.
* The getters may not be inlineable, but thats an easy fix if it ends up being a problem.  Not worth premature optimization (but is worth future work).

The expected timeline for this is adding this to the 1.8 cut, and deprecating the old-fields.  They will be removed in 1.9.
2017-10-30 10:30:34 -07:00
ZHANG Dapeng 7a4aa47070 core: remove unnecessary type params in ServerStreamTracer 2017-10-26 12:05:41 -07:00
Kun Zhang 9c817e5bb0 interop-test: disable message size check in interop-test client. (#3569)
The check fails in Java-Go interop test because of
grpc/grpc-go#1572

This is also needed for the internal gRPC-GFE test.

Resolves #3562
2017-10-12 17:14:16 -07:00
Kun Zhang 1e02b65c08 Fix interop_test client. 2017-10-10 13:46:51 -07:00
Kun Zhang 8edead0851 core: make in-process transport support StreamTracer
This is needed for both completeness and stats/tracing contexts propagation.

Stats recording with Census is intentionally disabled (#2284), while the rest of the Census-related logic work the same as on the other transports.
2017-10-09 10:14:36 -07:00
zpencer b07c70a09f gae-interop-testing: add GAE interop tests (#3535)
See `gae-interop-testing/README.md` for details on how to run the tests.
2017-10-06 10:44:58 -07:00
ZHANG Dapeng 80ac407c6c interop-testing: fix ErrorProne and Unused 2017-10-05 14:15:43 -07:00
Carl Mastrangelo 5e36a8deb5 all: upgrade to JUnit 4.12 2017-10-05 11:24:15 -07:00
zpencer 0d5943614e Make AbstractInteropTest runnable in GAE+jdk7 (#3533)
Avoid mocking and detect when a N/A test should be skipped.
2017-10-02 18:28:24 -07:00
Kun Zhang a6653bb135 core/compiler: register Span names for code-generated methods (take 2)
This is a more favorable approach than #3467. Doing the registration
in MethodDescriptor should allow us to deregister in case the
generated stub and its MethodDescriptors are garbage-collected
routinely, e.g., if they are loaded by a separate ClassLoader.
2017-09-25 15:38:48 -07:00
Eric Gribkoff 5ac9ad0c1f core: client supports inbound gzipped streams (#3403) 2017-09-25 09:01:53 -07:00
Kun Zhang 7e534ed704 core: record individual messages with sizes to Census/tracing (#3461)
Two methods, outboundMessageSent() and inboundMessageRead() are added to StreamTracer in order to associate individual messages with sizes. Both types of sizes are optional, as allowed by Census tracing.

Both methods accept a sequence number as the type ID as required by Census. The original outboundMesage() and inboundMessage() are also replaced by overrides that take the sequence number, to better match the new methods. The deprecation of the old overrides are tracked by #3460
2017-09-19 09:22:11 -07:00
Eric Anderson a3ff9cd784 all: Keep artifacts dependencyConvergence-clean
Maven Enforcer's dependencyConvergence is commonly used in Spring
projects, as it is inherited by all starter projects[1]. While I find
that option to be crazy and harmful (and would instead support
requireUpperBoundDeps), it does bother people.

1. https://github.com/spring-projects/spring-boot/blob/v1.5.6.RELEASE/spring-boot-starters/pom.xml#L94
2017-09-18 16:20:18 -07:00
Lukasz Strzalkowski 731bbefb17 core, compiler, protobuf: introduce MethodDescriptor#setSchemaDescriptor 2017-09-11 09:57:03 -07:00
Carl Mastrangelo 53f56a32f1 inprocess,core: add ManagedChannelBuilder and ServerBuilder factory hiders
* inprocess,core: add ManagedChannelBuilder and ServerBuilder factory hiders

Because the factory for Channels and Servers resides on the builder
itself, it is easy for subclasses to accidentally inherit the
factory.  This causes confusion, because calling a static method on
a specific class may result in a different class.

This change adds hiding static factories to each builder, and a test
to enforce that each subclass hides the factory.  The test lives in
the interop tests, because it has a classpath dependency on all the
existing transports.

Minor note: the test scans the classpath using a Beta Guava API.
The test can be disabled if the API goes away.
2017-09-06 15:21:17 -07:00
Carl Mastrangelo 24ff2748b7 all: update to proto 3.4.0 2017-08-25 11:25:36 -07:00
ZHANG Dapeng 23dfc84ccf interop-testing: fix dealineExceeded test flakiness by increasing time interval
Should resolve #1399
2017-08-24 14:27:49 -07:00
Eric Anderson 182164eafc compiler: Add option to disable version output
If the option becomes popular, we can just remove the version.
2017-08-23 12:37:40 -07:00
zpencer ab85c5ae76 Start 1.7.0 development cycle (#3357)
This bump changelist is applied a bit late with respect to the
1.6.0 branch cut. Look at the 1.6.0 to see the source of truth of
where it was cut. Do not assume it is the commit that precedes
this one.
2017-08-22 12:29:03 -07:00
Eric Gribkoff 15ca50ac7c interop-testing: larger timeout for HTTP/2 client 2017-08-10 13:48:25 -07:00
myPrecious f46280ae9f core: Java Refactor
Duplicate throws exception, and Javadoc issues: ManagedChannelImplTransportManagerTest.java has been deleted.
2017-08-09 09:11:19 -07:00
Eric Gribkoff 8585cd5e0d interop-testing: fix NPE on empty args to test service client 2017-08-07 17:20:54 -07:00
Kun Zhang 16f4de4636 core/stats: report message count metrics to Census. (#3312) 2017-08-04 15:58:21 -07:00
Carl Mastrangelo 02cb718767 testing,core: don't use mocks for stream tracers (#3305)
This is a big, but mostly mechanical change.  The newly added Test*StreamTracer classes are designed to be extended which is why they are non final and have protected fields.  There are a few notable things in this:

1.  verifyNoMoreInteractions is gone.   The API for StreamTracers doesn't make this guarantee.  I have recovered this behavior by failing duplicate calls.  This has resulted in a few bugs in the test code being fixed.

2.  StreamTracers cannot be mocked anymore.  Tracers need to be thread safe, which mocks simply are not.  This leads to a HUGE number of reports when trying to find real races in gRPC.

3.  If these classes are useful, we can promote them out of internal.  I just put them here out of convenience.
2017-08-03 16:59:06 -07:00
Eric Anderson c4f91272d2 all: Remove unused variables and squelch incorrect ErrorProne warning 2017-08-02 15:48:23 -07:00
Eric Gribkoff c0e010af8f interop-testing: implement compression interop tests 2017-07-31 16:08:23 -07:00
Kun Zhang 04e0450304 core: pass CallOptions to newClientStreamTracer(). (#3276)
Resolves #3256
2017-07-27 12:07:25 -07:00
Eric Anderson 924b0b2b00 Revert "interop-testing: overrideAuthority breaks JWT"
This reverts commit 57b9105c7f.

Issue #2682 is fixed, so we can revert the commit as planned. This
re-applies a previously-reverted modernization.
2017-07-19 13:54:32 -07:00
Ian Haken 677c84e1d4 okhttp: Add support for specifying a custom hostname verifier when using on OkHttpChannelBuilder. 2017-07-14 16:04:15 -07:00
ZHANG Dapeng ff0ad5fac3 testing: refactor part of TestUtils to internal
Moved the following APIs from `io.grpc.testing.TestUtils` to `io.grpc.internal.TestUtils`:

`InetSocketAddress testServerAddress(String host, int port)`
`InetSocketAddress testServerAddress(int port)`
`List<String> preferredTestCiphers()`
`File loadCert(String name)`
`X509Certificate loadX509Cert(String fileName)`
`SSLSocketFactory newSslSocketFactoryForCa(Provider provider, File certChainFile)`
`void sleepAtLeast(long millis)`

APIs not to be moved:

`ServerInterceptor recordRequestHeadersInterceptor()`
`ServerInterceptor recordServerCallInterceptor()`
2017-07-10 16:30:38 -07:00
ZHANG Dapeng 0a36c9d8a3 all: migrate .proto files from testing-proto to interop-testing
and provide a simple protobuf service for test in testing-proto instead.
2017-07-06 15:26:08 -07:00
Eric Gribkoff 7c60510794 all,interop-testing: update google-auth-library-oauth2-http (#3188) 2017-07-06 13:22:43 -07:00
Eric Anderson 361f0381be interop-testing: Support conscrypt at runtime with OkHttp
Conscrypt can thus be an alternative to Jetty ALPN for running these
OkHttp integration tests.
2017-06-30 17:07:38 -07:00
Eric Anderson 9052d4245f interop-testing: Skip Netty Jetty ALPN tests if it is unavailable 2017-06-30 16:25:31 -07:00
ZHANG Dapeng b7e50ddd05 testing: move NoopClientCall & NoopServerCall to internal
one step for #3105
2017-06-22 15:10:58 -07:00
ZHANG Dapeng 544ceded6b netty: rename HandlerSettings to InternalHandlerSettings 2017-06-19 09:04:04 -07:00
Eric Gribkoff a2a42e3396 interop-testing: add alpnagent to support okhttp in test client 2017-06-05 16:28:37 -07:00
Carl Mastrangelo 82fce837e4 core: don't return concrete type from AbstractServerImplBuilder 2017-06-01 14:38:05 -07:00
Carl Mastrangelo 166108a943 all: fix licence whitespace 2017-06-01 14:28:37 -07:00
Carl Mastrangelo 3bfd630bff all: update to Apache 2 licence
Also, update the authors.
2017-05-31 13:29:01 -07:00
Eric Anderson cefcd39396 Fix incorrect assertEquals argument ordering
The expected value should be first.
2017-05-24 08:47:02 -07:00
Eric Gribkoff 5dc8a124bf interop-testing,okhttp,testing: update tests to pass with ipv6 2017-05-16 17:01:05 -07:00
Kun Zhang be74e97b5d core: stop using static flags for Census control. (#2947)
Static mutable flags are evil.  Turned them to options on channel
builder.  Also restore the local stats recording by default, because
these flags were added with the concern of wire-compatibility, but not
local feature.
2017-04-26 10:50:55 -07:00
Kun Zhang 49bde5494b core: integrate instrumentation-java (Census) tracing (#2938)
Main implementation is in CensusTracingModule.

Also a few fix-ups in the stats implementation CensusStatsModule:

- Change header key name from grpc-census-bin to grpc-tags-bin
- Server does not fail on header parse errors. Uses the default instead.

Protect Census-based stats and tracing with static flags: `GrpcUtil.enableCensusStats` and `GrpcUtil.enableCensusTracing`. They keep those features disabled by default until they, especially their wire formats, are stabilized.
2017-04-25 13:53:29 -07:00
Eric Gribkoff b6bf252566 interop-testing: add cacheable_unary test 2017-04-21 10:22:05 -07:00
Kun Zhang 6618f9739e core: add inboundHeaders() to ClientStreamTracer. (#2921)
Also renamed headersSent to outboundHeaders
2017-04-17 14:01:34 -07:00
Carl Mastrangelo be61af42e9 core: use RESOURCE_EXHAUSTED for max message size failures 2017-04-13 08:35:48 -07:00
Kun Zhang 64938d3cb4 testing: remove negative asserts about stats. (#2894)
They were added in #2863. In TestServiceClient metricsExpected() returns
false because server-side stats is not available, but client-side stats
are there, thus these asserts would fail.

Resolves grpc/grpc/issues/10552
2017-04-10 14:23:26 -07:00
Kun Zhang 44cca5507d core: remove incorrect reporting of CLIENT_SERVER_ELAPSED_TIME. (#2891)
Per spec this metric should be calculated on the server and sent back to
the client, for which the mechanism is not currently defined. As it's
not a required metric, we remove the incorrect implementation for now.

Internal ref: b/37208451
2017-04-10 11:25:25 -07:00
Kun Zhang 903197b2aa core: StreamTracer (#2863)
Background
==========

LoadBalancer needs to track RPC measurements and status for
load-reporting.  We need to introduce a "Tracer" API for that.

Since such API is very close to the current
Census(instrumentation)-based stats reporting mechanism in terms of what
are recorded, we will migrate the Census-based stats reporting under the
new Tracer API.

Alternatives
============

We considered plumbing the LB-related information from the LoadBalancer
to the core, and recording those information along with the currently
recorded stats to Census. The LB-related information, such as LB_ID,
reason for dropping reqeusts etc, would be added to the Census
StatsContext as tags.

Since tags are held by StatsContext before eventually being recorded by
providing the measurements, and StatsContext is immutable, this would
require a way for LoadBalancer to override the StatsContext, which means
LoadBalancer API would has direct reference to the Census StatsContext.
This is undesirable because Census API is not stable yet.

Part of the LB-related information is whether the client has received
the initial headers from the server.  While such information can be
grabbed by implementing a ClientInterceptor, it must be recorded along
with other information such as LB_ID to be useful, and LB_ID is only
available in GrpclbLoadBalancer.

Bottom line, trying to use solely the Census StatsContext API to record
LB load information would require extra data plumbing channel between
ClientInterceptor, LoadBalancer and the gRPC core, as well as exposing
Census API on the gRPC API.  Even with those extensive changes, we are
yet to find a working solution. Therefore, we abandoned this idea and
propose this PR.

Summary of changes
==================

API summary
-----------
Introduce "StreamTracer" API, a callback interface for receiving stats
and tracing related updates concerning **a single stream**.
"ClientStreamTracer" and "ServerStreamTracer" add side-specific
events. A stream can have zero or more tracers and report to all of
them.

On the client-side, CallOptions now takes a list of
ClientStreamTracer.Factory. Opon creating a ClientStream, each of the
factory creates a ClientStreamTracer for the stream. This allows
ClientInterceptors to install its own tracer factories by overriding the
CallOptions.

Since StreamTracer only tracks the span of a stream, tracking of a
ClientCall needs to be done in a ClientInterceptor.  By installing its
own StreamTracer when a ClientCall is created, ClientInterceptor can
associate the updates for a Call with the updates for the Streams
created for that Call.  This is how we keep the existing Census
reporting mechanism in CensusStreamTracerModule.

On the server-side, ServerStreamTracer.Factory is added through the
ServerBuilder, and is used to create ServerStreamTracers for every
ServerStream.

The Tracer API supports propagation of stats/tracing information through
Context and metadata.  Both client-side and server-side tracer factories
have access to the headers object.  Client-side tracer relies on
interceptor to read the Context, while server-side tracer has
filterContext() method that can override the Context.

Implementation details
----------------------

Only real streams report stats.  Pseudo streams such as delayed stream,
failing stream don't report.  InProcess transport streams currently
don't report stats.

"StatsTraceContext" which used to receive updates from core and report
directly to Census (StatsContext), now delegates to the StreamTracers of
a stream.  On the client-side, the scope of a StatsTraceContext reduces
from ClientCall to a ClientStream to match the scope of StreamTracer.

The Census-specific logic that was in StatsTraceContext is moved into
CensusStreamTracerModule, which produces factories for StreamTracers
that report to Census.

Reporting with StatsTraceContext is moved out of the Channel/Call layer
into Transport/Stream layer, to match the scope change of
StatsTraceContext.

Bug fixed
----------------

The end of a server-side call was reported in ServerCallImpl's
ServerStreamListenerImpl.closed(), which was wrong.  Because closed()
receiving OK doesn't necessarily mean the RPC ended with OK.  Instead it
means the server has successfully sent the final status, which may be
non-OK, to the client.

Now the end report is done in both ServerStream.close(any Status) and
before calling ServerStreamListener.closed(non-OK).  Whichever happens
first is the reported status.

TODOs
=====

A follow-up change to the LoadBalancer API will add a
ClientStreamTracer.Factory to the PickResult to complete the API needed
by load-reporting.
2017-04-07 11:03:24 -07:00
Carl Mastrangelo 5c37a8360c core: cache Accept-Encoding headers (#2766)
* core: cache Accept-Encoding headers

This avoids rebuilding the raw bytes for each RPC.  The decode path
is not yet optimized to avoid pulling to much into a single commit.
Decompressors may still use invalid names, but this is equivalent to
previous behavior, as cleaing happens later in the caching.

Also, internal accessors on DecompressorRegistry are now hidden.

Before:
Benchmark                                                    (extraEncodings)    Mode      Cnt        Score    Error  Units
DecompressorRegistryBenchmark.marshalOld                                    0  sample   928744      124.104 ± 11.159  ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.00                   0  sample                84.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.50                   0  sample                94.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.90                   0  sample               107.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.95                   0  sample               114.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.99                   0  sample               202.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.999                  0  sample              4944.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.9999                 0  sample             12178.008           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p1.00                   0  sample           2056192.000           ns/op
DecompressorRegistryBenchmark.marshalOld                                    1  sample  1345050      150.123 ±  6.952  ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.00                   1  sample               109.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.50                   1  sample               127.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.90                   1  sample               142.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.95                   1  sample               152.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.99                   1  sample               243.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.999                  1  sample              4640.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.9999                 1  sample             11472.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p1.00                   1  sample           2101248.000           ns/op
DecompressorRegistryBenchmark.marshalOld                                    2  sample  1130903      175.846 ±  1.392  ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.00                   2  sample               131.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.50                   2  sample               148.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.90                   2  sample               164.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.95                   2  sample               174.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.99                   2  sample               311.000           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.999                  2  sample              6048.768           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.9999                 2  sample             12349.107           ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p1.00                   2  sample            112000.000           ns/op

After:
Benchmark                                                    (extraEncodings)    Mode      Cnt        Score   Error  Units
DecompressorRegistryBenchmark.marshalOld                                    0  sample  1095005       67.555 ± 5.529  ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.00                   0  sample                42.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.50                   0  sample                52.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.90                   0  sample                69.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.95                   0  sample                84.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.99                   0  sample               133.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.999                  0  sample              3324.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.9999                 0  sample             11056.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p1.00                   0  sample           1820672.000          ns/op
DecompressorRegistryBenchmark.marshalOld                                    1  sample  1437034       78.089 ± 0.723  ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.00                   1  sample                60.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.50                   1  sample                69.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.90                   1  sample                79.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.95                   1  sample                83.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.99                   1  sample                96.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.999                  1  sample              2728.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.9999                 1  sample             11104.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p1.00                   1  sample            105344.000          ns/op
DecompressorRegistryBenchmark.marshalOld                                    2  sample  1203782       95.213 ± 0.864  ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.00                   2  sample                68.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.50                   2  sample                85.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.90                   2  sample                98.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.95                   2  sample               101.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.99                   2  sample               119.000          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.999                  2  sample              3209.736          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p0.9999                 2  sample             11257.947          ns/op
DecompressorRegistryBenchmark.marshalOld:marshalOld·p1.00                   2  sample             63168.000          ns/op
2017-03-01 15:30:28 -08:00
Eric Anderson 437fafab1b Address review comments 2017-02-28 09:23:04 -08:00
Eric Anderson 2eeb5e3e9e all: Downgrade to Guava 19
Guava 20 introduced some overloading optimizations for Preconditions
that require using Guava 20+ at runtime. Unfortunately, Guava 20 removes
some things that is causing incompatibilities with other libraries, like
Cassandra. While the incompatibility did trigger some of those libraries
to improve compatibility for newer Guavas, we'd like to give the
community more time to work through it. See #2688

At this commit, we appear to be compatible with Guava 18+. It's not
clear if we want to actually "support" 18, but it did compile. Guava 17
doesn't have at least MoreObjects, directExecutor, and firstNotNull.
Guava 21 compiles without warnings, so it should be compatible with
Guava 22 when it is released.

One test method will fail with the upcoming Guava 22, but this won't
impact applications. I made MoreThrowables to avoid using any
known-deprecated Guava methods in our JARs, to reduce pain for those
stuck with old versions of gRPC in the future (July 2018).

In the stand-alone Android apps I removed unnecessary explicit deps
instead of syncing the version used.
2017-02-28 09:23:04 -08:00
Eric Anderson e67b6027af interop-testing: Remove useless tcnative configuration
It doesn't do anything, since tcnative has been specified as a normal
dependency for a while.
2017-02-27 17:12:07 -08:00
Eric Anderson 675080b208 all: Enable ErrorProne during compilation
ErrorProne provides static analysis for common issues, including
misused variables GuardedBy locks.

This increases build time by 60% for parallel builds and 30% for
non-parallel, so I've provided a way to disable the check. It is on by
default though and will be run in our CI environments.
2017-02-24 14:53:23 -08:00
Eric Anderson 9283b2b13f interop-testing: Remove useless Threads in test 2017-02-17 11:21:30 -08:00
Eric Anderson 72923dca87 all: prepare for ErrorProne's FutureReturnValueIgnored
Futures almost universally should be handled in some way when being
returned, either to receive the value or to cancel scheduled tasks to
prevent leaks.

Netty is a bit of a special case though, since it constantly returns
futures that you ignore (even adding a listener returns the "this"
future). So we want to suppress the warning for code using Netty instead
of trying to fix it. When we enable ErrorProne in the build, we should
start passing -Xep:FutureReturnValueIgnored:OFF in the compilerArgs.
2017-02-16 07:22:56 -08:00
Eric Gribkoff 48e1430909 interop-testing: fix flakes in Http2Client 2017-02-07 16:14:39 -08:00
Kun Zhang 7ab5e0e810 core: record server_elapsed_time on client (#2673)
It is defined as the time between the client sends out the headers, and the RPC finishes.
2017-02-03 13:29:06 -08:00
kpayson64 6fbe140959 interop-testing: Replace Mockito with StreamObserver for several tests (#2639) 2017-02-02 10:21:41 -08:00
Eric Anderson 57b9105c7f interop-testing: overrideAuthority breaks JWT
Commit 65e4d9f4 broke the jwt_token_creds. It is believed to be because
the JWT does not see the authority passed to overrideAuthority. So the
changes to interop-testing client are temporarily reverted here. Note
that this breaks GRPC_PROXY_EXP testing, so the incompatibility needs to
be resolved.

Solving #2682 will allow reverting this change.

Fixes grpc/grpc#9497
Fixes #2680
2017-02-01 08:38:53 -08:00
Eric Anderson 65e4d9f47a all: avoid DNS with GRPC_PROXY_EXP
In some environments DNS is not available and is performed by the
CONNECT proxy. Nothing "special" should need to be done for these
environments, but the previous support took shortcuts which knowingly
would not support such environments.

This change should fix both OkHttp and Netty. Netty's
Bootstrap.connect() resolved the name immediately whereas using
ChannelPipeline.connect() waits until the address reaches the end of the
pipeline. Netty uses NetUtil.toSocketAddressString() to get the name of
the address, which uses InetSocketAddress.getHostString() when
available.

OkHttp is still using InetSocketAddress.getHostName() which may issue
reverse DNS lookups. However, if the reverse DNS lookup fails, it should
convert the IP to a textual string like getHostString(). So as long as
the reverse DNS maps to the same machine as the IP, there should only be
performance concerns, not correctness issues. Since the DnsNameResolver
is creating unresolved addresses, the reverse DNS lookups shouldn't
occur in the common case.
2017-01-27 09:27:07 -08:00
Carl Mastrangelo 89bc2cd3b2 all: update to latest import ordering 2017-01-26 13:43:06 -08:00
Kun Zhang f088b81fc8 core: stop "testing" from depending on "core"'s test. (#2652)
Because "core"'s test source already depends on "testing", e.g.,
`core/src/test/java/io/grpc/internal/ServerCallImplTest.java` uses
`testing/src/main/java/io/grpc/internal/testing/StatsTestUtils.java`,
which forms a circular dependency.

This change moves the StatsContext setter accessors from "core"'s test
source to "testing".

Resolves #2651
2017-01-25 07:57:12 -08:00
Kun Zhang 737cd16a38 core: make StatsContextFactory setters protected (#2634)
These are only used in internal tests.  In production,
StatsContextFactory is loaded by the "Instrumentation" library and must
be one per process, thus we won't allow setting it on a per-channel or
per-server basis.
2017-01-20 17:20:44 -08:00
Eric Anderson f51316b84a testing: Move echo interceptors out of TestUtils
The interceptors are quite specific, and probably not helpful for
current testing strategies. They really are only useful for interop
testing. Moving them to interop-testing avoids them appearing to be in a
public API (even if that API is experimental).

No functional changes were made; just code movement.
2017-01-19 17:04:31 -08:00
Lukasz Strzalkowski 060eb45623 Rename attributes() to getAttributes() to make it consistent 2017-01-19 12:53:54 -08:00
Eric Anderson 1e99b299e1 all: ErrorProne fixes and avoid @Beta in Guava 2017-01-19 12:16:05 -08:00
Carl Mastrangelo 8e463ce1a8 core,stub: remove deprecated deadline methods 2017-01-10 15:54:38 -08:00
Kun Zhang d17a7b5bd4 core: abstract channel builder to accept LoadBalancer2 (#2583)
If a LoadBalancer2 is passed in, the builder will create ManagedChannelImpl2 instead of ManagedChannelImpl. This allows us to test the LBv2 classes on a large scale.
2017-01-10 15:30:12 -08:00
Carl Mastrangelo 8d49df28ee all: add max message size to client calls 2017-01-05 17:23:34 -08:00