Commit Graph

2801 Commits

Author SHA1 Message Date
zpencer 934c2e2fd7
core: clarify binlog API (#4362)
The interceptors are a part of the implementation and not meant for
callers.

Both methods for wrapping clients / servers can be `final`.
2018-04-19 17:32:45 -07:00
zpencer ec95999124
core: remove unneeded binlog tests for ManagedChannelImpl ServerImpl (#4361)
Replace the tests with tests that just make sure binary log is
installed. Better tests exist today in BinaryLogProviderTest to make
sure the actual logic is correct.
2018-04-19 16:55:32 -07:00
ZHANG Dapeng e67a61fa08
core: refactor ManagedChannelImplTest with a configurable builder
In previous code of `ManagedChannelImplTest`, new tests will have to add more and more custom args in `createChannel(...)`, and each time when a custom arg is needed all other default args need be repeatedly provided in `createChannel(...)`. In most cases those args are just builder attributes. So this PR makes ManagedChannelImplTest easier to stub by providing a builder field, then new tests will just set necessary builder attributes before calling `createChannel()`.
2018-04-19 10:06:39 -07:00
ZHANG Dapeng f6a032d6b2
core: forward toString for forwarding classes
Forward `toString()` method for forwarding classes to improve debug information.

For example, `ForwardingManagedChannel.toString()` will return something like:

```
ForwardingManagedChannel{delegate=ManagedChannelImpl{logId=tag-13247, target=localhost:8080}}
```
2018-04-19 10:05:48 -07:00
Carl Mastrangelo f754bff906
core: make JsonParser public 2018-04-18 18:49:08 -07:00
zpencer 30400e0e58
core: add sockets to subchannel stat (#4345)
This is info not being reported at the moment and is missing from the
display.
2018-04-16 11:33:28 -07:00
zpencer c04278ea07
services: turn channelz null futures into Status.UNIMPLEMENTED (#4346)
Then channelz GUI will take this into account. This is particularly
useful for InProcessTransport, where I have decided we do not need
special support for in channelz. The server and channel stats are
already sufficient.
2018-04-16 09:54:35 -07:00
zpencer 791a29f63b
services: socket options were erroneously ignored if socket has no stats (#4336)
This fixes listen sockets. It is ok to have no data but report socket
options.
2018-04-13 14:59:35 -07:00
Carl Mastrangelo e4502aca5d
stub: add docs clarifying readiness 2018-04-13 14:12:54 -07:00
zpencer 1d5ed78c8f
services: make channelz constructor static (#4334)
Right now this is inaccessible to services.
2018-04-13 14:10:50 -07:00
zpencer c50a57cd52
netty: fix visibility issues with InternalNettySocketSupport (#4335)
Previous version is not actually extendable from other packages.
2018-04-13 14:10:28 -07:00
Carl Mastrangelo 03465a7f45
core: don't warn on absent service config
Fixes #4333
2018-04-13 11:13:57 -07:00
ZHANG Dapeng a6811d5bb1
all: add toString() to Server to improve debug info 2018-04-13 09:41:09 -07:00
ZHANG Dapeng 0eafb8e9ca
core: compare doubles with compare instead of ==
Always not compare equality for doubles with `==`.
2018-04-13 09:39:59 -07:00
Eric Anderson 0a598c053c core: Fix lint, style, "good ideas" violations 2018-04-12 17:26:54 -07:00
ZHANG Dapeng 6d7e19cbe5
core: replumb RetryPolicy
- replumbed `RetryPolicy` with `MethodInfo` without breaking the existing `RetryPolicyTest`.
- moved `ServiceConfigInterceptor.MethodInfo.RetryPolicy` out as a top level class so that `RetriableStream` does not import `ServiceConfigInterceptor`.
2018-04-12 11:41:04 -07:00
Kun Zhang 3f63cd0ad4
core: always call StreamTracer.streamClosed() when stream is officially closed. (#4331)
Previously StreamTracer.streamClosed() is called in
ServerStream.close(), but it is not exactly when the stream is
officially closed.  ServerStreamListener.closed() is guaranteed to be
called and it is the official end of the stream.
2018-04-12 09:05:48 -07:00
zpencer 7c46bd9399
netty: prepare NettySocketSupport for internal import (#4311)
Delegate the actual heavy lifting to a helper class that can be easily swapped
at runtime.
2018-04-11 16:37:13 -07:00
zpencer 137e759fda
core, netty: allow InputStream based certs (#4316)
Allow ServerBuilder to read certs from InputStream, not just from a
File.
2018-04-11 16:14:58 -07:00
Shohei Kamimori d68b2cd74a protobuf: add test for ProtoUtils (#3838) 2018-04-06 10:10:08 -07:00
zpencer 8d3d26dda1
compiler: allow capitalized java keywords as rpc names (#4309)
previously `Import` fails because it gets turned into a method named
`import`. This PR makes the method be named `import_`, everything else
stays the same, such as the name in the method descriptor.

Note: `iMport` or `iMPORT` or any variation of capitalizations that is
not `Import` always worked. `Import` gets translated into `import`
because of our attempts to camel case the names.
2018-04-05 16:45:19 -07:00
Eric Anderson ecab86fad9 examples: upgrade protobuf-maven-plugin to 0.5.1
This was missed in 2094bb4d8
2018-04-05 14:14:32 -07:00
zpencer 9f92104b94
buildscripts: fix GAE dummy app deploy races (#4308)
This fixes a race where we can race with another deploy of the
'dummy' version.

Also, when we delete old versions, only delete versions beginning with
'kokoro'. This would make races even less likely because the
'dummy-default' should normally stick around forever.

This fixes this failure:
https://source.cloud.google.com/results/invocations/6b24aed4-7dc3-490a-87ea-c870d1435184/targets/grpc%2Fjava%2Fpresubmit%2Fgae-interop/log
2018-04-05 13:19:36 -07:00
zpencer 62cc2b1ae3
core,netty: add NettySocketSupport to populate TcpInfo (#4306)
NettySocketSupport is responsible for making the low level calls to
get and populate the TcpInfo structure.
2018-04-05 10:36:49 -07:00
Carl Mastrangelo 2df76cc710
core: apply service config 2018-04-04 17:45:12 -07:00
zpencer 49c8bdb60a
netty: fix getListenSockets race (#4301)
Move registration to separate future and wait for it.
2018-04-04 17:04:29 -07:00
Carl Mastrangelo 9ed84258aa
core: don't reschedule idle timer if it is already active
Benchmark results (3 runs each)

non direct

```
Before:
50.0%ile Latency (in nanos):		157471
90.0%ile Latency (in nanos):		185927
95.0%ile Latency (in nanos):		195135
99.0%ile Latency (in nanos):		218815
99.9%ile Latency (in nanos):		1188735
100.0%ile Latency (in nanos):		18333695
QPS:                           6126
50.0%ile Latency (in nanos):		160407
90.0%ile Latency (in nanos):		188551
95.0%ile Latency (in nanos):		197487
99.0%ile Latency (in nanos):		219575
99.9%ile Latency (in nanos):		390239
100.0%ile Latency (in nanos):		18338815
QPS:                           6106
50.0%ile Latency (in nanos):		157831
90.0%ile Latency (in nanos):		186439
95.0%ile Latency (in nanos):		195815
99.0%ile Latency (in nanos):		216951
99.9%ile Latency (in nanos):		281167
100.0%ile Latency (in nanos):		5384447
QPS:                           6235

After:
50.0%ile Latency (in nanos):		152255
90.0%ile Latency (in nanos):		180551
95.0%ile Latency (in nanos):		188943
99.0%ile Latency (in nanos):		209623
99.9%ile Latency (in nanos):		1184831
100.0%ile Latency (in nanos):		4351999
QPS:                           6313
50.0%ile Latency (in nanos):		153663
90.0%ile Latency (in nanos):		181671
95.0%ile Latency (in nanos):		189991
99.0%ile Latency (in nanos):		210495
99.9%ile Latency (in nanos):		278895
100.0%ile Latency (in nanos):		18283519
QPS:                           6300
50.0%ile Latency (in nanos):		152767
90.0%ile Latency (in nanos):		180839
95.0%ile Latency (in nanos):		189791
99.0%ile Latency (in nanos):		211719
99.9%ile Latency (in nanos):		280927
100.0%ile Latency (in nanos):		12231167
QPS:                           6381
```

direct:

```
Before:
50.0%ile Latency (in nanos):		133943
90.0%ile Latency (in nanos):		153671
95.0%ile Latency (in nanos):		163655
99.0%ile Latency (in nanos):		188871
99.9%ile Latency (in nanos):		235791
100.0%ile Latency (in nanos):		7864575
QPS:                           7134
50.0%ile Latency (in nanos):		131623
90.0%ile Latency (in nanos):		151863
95.0%ile Latency (in nanos):		162095
99.0%ile Latency (in nanos):		187719
99.9%ile Latency (in nanos):		234983
100.0%ile Latency (in nanos):		17836031
QPS:                           7250
50.0%ile Latency (in nanos):		131223
90.0%ile Latency (in nanos):		150823
95.0%ile Latency (in nanos):		161311
99.0%ile Latency (in nanos):		187719
99.9%ile Latency (in nanos):		237471
100.0%ile Latency (in nanos):		4416255
QPS:                           7273

After:
50.0%ile Latency (in nanos):		122751
90.0%ile Latency (in nanos):		140967
95.0%ile Latency (in nanos):		148911
99.0%ile Latency (in nanos):		173215
99.9%ile Latency (in nanos):		214823
100.0%ile Latency (in nanos):		18509823
QPS:                           7774
50.0%ile Latency (in nanos):		124507
90.0%ile Latency (in nanos):		145855
95.0%ile Latency (in nanos):		156623
99.0%ile Latency (in nanos):		183111
99.9%ile Latency (in nanos):		235679
100.0%ile Latency (in nanos):		18289663
QPS:                           7625
50.0%ile Latency (in nanos):		124295
90.0%ile Latency (in nanos):		145071
95.0%ile Latency (in nanos):		156439
99.0%ile Latency (in nanos):		183919
99.9%ile Latency (in nanos):		232447
100.0%ile Latency (in nanos):		3712383
QPS:                           7632
```
2018-04-04 16:02:02 -07:00
zpencer 276586a4fb
netty,services: fix param comment lint (#4304)
Fix linter complaint because comment does not match arg name.
2018-04-04 12:06:31 -07:00
Jorg Heymans 2094bb4d8e doc: upgrade protobuf-maven-plugin in README.md 2018-04-03 16:51:56 -07:00
Shohei Kamimori 8f01084bb3 core: add a `close` to InputBufferStream
Before:
`InputBufferStream.close()` does not close their buffer so the buffer will leak.

After:
Resolves #4198.
Override the `close` for closing their buffer.
2018-04-03 16:11:19 -07:00
Eric Anderson bace06fe9f
java_grpc_library: Add support for protobuf lite
gRPC's protobuf-lite auto-selects between full and lite protobuf based on the
value of crosstool_top. If the user is specifying their own
--android_crosstool_top, then it will not auto-detect correctly. One day,
platforms will fix problems like this, but for the moment it seems we get to
live with it.
2018-04-03 15:22:55 -07:00
zpencer b1d91b9f60
core,services: binary log should use real peer socket and call id (#4266)
The peer socket is read from TRANSPORT_ATTR_REMOTE_ADDR from the
stream attributes. We only log the peer on receive initial metadata.

The call id assumes census is available. The call ID read from the
context via SERVER_CALL_ID_CONTEXT_KEY on server side, and read from
CallOptions via CLIENT_CALL_ID_CALLOPTION_KEY on client side. The
value is copied from CONTEXT_SPAN_KEY which is set by census.

Pass around CallId with two longs, not a byte[].
2018-04-03 13:39:11 -07:00
zpencer 724e32fe57
core,netty,services: add server listen sockets to channelz proto service (#4220)
Server listen sockets differ from normal sockets in that they do not
have a remote address, do not have stats on calls started/failed/etc,
and do not have security info.
2018-04-03 10:05:47 -07:00
Eric Anderson d99d8d99d8 protobuf: fix @since for fromStatusAndTrailers
It was added in 1e9bd9a8 which was initially released in v1.11.0.
2018-04-03 09:16:56 -07:00
Eric Anderson 41e98ef72b SECURITY.md: Add reference to grpc-netty-shaded
Many people going to the troubleshooting section would be fixed by
swapping to grpc-netty-shaded, so we should call that out.
2018-04-03 08:45:03 -07:00
Eric Anderson ef030abe6b java_grpc_library: re-add grpc-java repo to targets
They were removed in 137c74d1 since it was believed they were unnecessary.
However, since they are in a macro and not a rule, they are relative to the
caller, not their definition.

Added building the examples to the kokoro CI. Note that this means the examples
are built twice: once in grpc-java's build and once in their own (because it
has a WORKSPACE). Given that the Bazel build is our fastest build, this
slowdown won't probably be an issue.
2018-04-03 08:42:40 -07:00
Shohei Kamimori 1d901a8de2 core: add tests for ReadableBuffers 2018-04-03 08:11:39 -07:00
David Brodsky 5b802de3fd okhttp: properly verify IPv6 address hosts (#4292)
Address mismatch between IPv6 address hosts derived from URIs and X509 subjectAltName extensions
2018-04-02 17:43:50 -07:00
zpencer 2347384256
GAE: call channel.shutdown() (#4274)
OkHttpClientInteropServlet and NettyClientInteropServlet both run the
@After method from AbstractInteropTest. Let's make sure we await
termination.

For the long lived channel test, do the cleanup in `destroy`.
2018-04-02 16:02:55 -07:00
zpencer 0067415811
core: document how to set up squid proxy for testing (#4285)
This makes it easier for a gRPC developer to validate the proxy code
using a real proxy.
2018-03-30 11:28:10 -07:00
zpencer 7c37f1d32b
core,netty,okhttp,services: expose socket options to channelz (#4228)
For okhttp, expose the standard options from the Socket object.

For netty, expose all the `io.netty.channel.ChannelOption`s of the
`channel.config()`.
2018-03-30 08:48:37 -07:00
Eric Anderson 2a95e38257 Update README to reference 1.11.0 2018-03-29 11:24:06 -07:00
Shohei Kamimori 156cc44c5f interop-testing: fix empty proto package name
Intended to resolve #4241
grpc-java has already diverged this proto file, so changing the
this package name is reasonable for now.
2018-03-29 10:37:53 -07:00
Eric Anderson 5780743be4 RELEASING.md: Push branch before tag
If the release branch (i.e., v1.11.x) was not up-to-date with the local
repo, then the push to update the branch will fail. Doing it first
allows you to notice, merge in the updates and fix the tag before
pushing it.

Also document pulling the most up-to-date version of the release branch
before creating the 'release' branch. That would reduce the amount of
time the release branch could be modified.
2018-03-29 09:48:35 -07:00
Eric Anderson 4e82e62eaa
Fix compilation in Java 9 2018-03-28 17:13:39 -07:00
ZHANG Dapeng bdecdaea22
netty: http2 server transport graceful shutdown sends 2 GOAWAYs
resolves #3442
2018-03-28 15:58:31 -07:00
Arnout Engelen 03a00aa8cf interop-testing: client compressed tests without probing (#4279) 2018-03-28 15:51:03 -07:00
Shohei Kamimori 5af2515f3e core,README: add docs about grpc-java-api-checker (#4182) 2018-03-28 15:34:18 -07:00
Eric Gribkoff 77e3b97f91 okhttp: add hpack test for max dynamic table size 2018-03-28 14:53:19 -07:00
Eric Gribkoff ecc36ac96d okhttp: include tests in third_party/okhttp 2018-03-28 14:53:19 -07:00