Commit Graph

2680 Commits

Author SHA1 Message Date
Eric Anderson ecab86fad9 examples: upgrade protobuf-maven-plugin to 0.5.1
This was missed in 2094bb4d8
2018-04-05 14:14:32 -07:00
zpencer 9f92104b94
buildscripts: fix GAE dummy app deploy races (#4308)
This fixes a race where we can race with another deploy of the
'dummy' version.

Also, when we delete old versions, only delete versions beginning with
'kokoro'. This would make races even less likely because the
'dummy-default' should normally stick around forever.

This fixes this failure:
https://source.cloud.google.com/results/invocations/6b24aed4-7dc3-490a-87ea-c870d1435184/targets/grpc%2Fjava%2Fpresubmit%2Fgae-interop/log
2018-04-05 13:19:36 -07:00
zpencer 62cc2b1ae3
core,netty: add NettySocketSupport to populate TcpInfo (#4306)
NettySocketSupport is responsible for making the low level calls to
get and populate the TcpInfo structure.
2018-04-05 10:36:49 -07:00
Carl Mastrangelo 2df76cc710
core: apply service config 2018-04-04 17:45:12 -07:00
zpencer 49c8bdb60a
netty: fix getListenSockets race (#4301)
Move registration to separate future and wait for it.
2018-04-04 17:04:29 -07:00
Carl Mastrangelo 9ed84258aa
core: don't reschedule idle timer if it is already active
Benchmark results (3 runs each)

non direct

```
Before:
50.0%ile Latency (in nanos):		157471
90.0%ile Latency (in nanos):		185927
95.0%ile Latency (in nanos):		195135
99.0%ile Latency (in nanos):		218815
99.9%ile Latency (in nanos):		1188735
100.0%ile Latency (in nanos):		18333695
QPS:                           6126
50.0%ile Latency (in nanos):		160407
90.0%ile Latency (in nanos):		188551
95.0%ile Latency (in nanos):		197487
99.0%ile Latency (in nanos):		219575
99.9%ile Latency (in nanos):		390239
100.0%ile Latency (in nanos):		18338815
QPS:                           6106
50.0%ile Latency (in nanos):		157831
90.0%ile Latency (in nanos):		186439
95.0%ile Latency (in nanos):		195815
99.0%ile Latency (in nanos):		216951
99.9%ile Latency (in nanos):		281167
100.0%ile Latency (in nanos):		5384447
QPS:                           6235

After:
50.0%ile Latency (in nanos):		152255
90.0%ile Latency (in nanos):		180551
95.0%ile Latency (in nanos):		188943
99.0%ile Latency (in nanos):		209623
99.9%ile Latency (in nanos):		1184831
100.0%ile Latency (in nanos):		4351999
QPS:                           6313
50.0%ile Latency (in nanos):		153663
90.0%ile Latency (in nanos):		181671
95.0%ile Latency (in nanos):		189991
99.0%ile Latency (in nanos):		210495
99.9%ile Latency (in nanos):		278895
100.0%ile Latency (in nanos):		18283519
QPS:                           6300
50.0%ile Latency (in nanos):		152767
90.0%ile Latency (in nanos):		180839
95.0%ile Latency (in nanos):		189791
99.0%ile Latency (in nanos):		211719
99.9%ile Latency (in nanos):		280927
100.0%ile Latency (in nanos):		12231167
QPS:                           6381
```

direct:

```
Before:
50.0%ile Latency (in nanos):		133943
90.0%ile Latency (in nanos):		153671
95.0%ile Latency (in nanos):		163655
99.0%ile Latency (in nanos):		188871
99.9%ile Latency (in nanos):		235791
100.0%ile Latency (in nanos):		7864575
QPS:                           7134
50.0%ile Latency (in nanos):		131623
90.0%ile Latency (in nanos):		151863
95.0%ile Latency (in nanos):		162095
99.0%ile Latency (in nanos):		187719
99.9%ile Latency (in nanos):		234983
100.0%ile Latency (in nanos):		17836031
QPS:                           7250
50.0%ile Latency (in nanos):		131223
90.0%ile Latency (in nanos):		150823
95.0%ile Latency (in nanos):		161311
99.0%ile Latency (in nanos):		187719
99.9%ile Latency (in nanos):		237471
100.0%ile Latency (in nanos):		4416255
QPS:                           7273

After:
50.0%ile Latency (in nanos):		122751
90.0%ile Latency (in nanos):		140967
95.0%ile Latency (in nanos):		148911
99.0%ile Latency (in nanos):		173215
99.9%ile Latency (in nanos):		214823
100.0%ile Latency (in nanos):		18509823
QPS:                           7774
50.0%ile Latency (in nanos):		124507
90.0%ile Latency (in nanos):		145855
95.0%ile Latency (in nanos):		156623
99.0%ile Latency (in nanos):		183111
99.9%ile Latency (in nanos):		235679
100.0%ile Latency (in nanos):		18289663
QPS:                           7625
50.0%ile Latency (in nanos):		124295
90.0%ile Latency (in nanos):		145071
95.0%ile Latency (in nanos):		156439
99.0%ile Latency (in nanos):		183919
99.9%ile Latency (in nanos):		232447
100.0%ile Latency (in nanos):		3712383
QPS:                           7632
```
2018-04-04 16:02:02 -07:00
zpencer 276586a4fb
netty,services: fix param comment lint (#4304)
Fix linter complaint because comment does not match arg name.
2018-04-04 12:06:31 -07:00
Jorg Heymans 2094bb4d8e doc: upgrade protobuf-maven-plugin in README.md 2018-04-03 16:51:56 -07:00
Shohei Kamimori 8f01084bb3 core: add a `close` to InputBufferStream
Before:
`InputBufferStream.close()` does not close their buffer so the buffer will leak.

After:
Resolves #4198.
Override the `close` for closing their buffer.
2018-04-03 16:11:19 -07:00
Eric Anderson bace06fe9f
java_grpc_library: Add support for protobuf lite
gRPC's protobuf-lite auto-selects between full and lite protobuf based on the
value of crosstool_top. If the user is specifying their own
--android_crosstool_top, then it will not auto-detect correctly. One day,
platforms will fix problems like this, but for the moment it seems we get to
live with it.
2018-04-03 15:22:55 -07:00
zpencer b1d91b9f60
core,services: binary log should use real peer socket and call id (#4266)
The peer socket is read from TRANSPORT_ATTR_REMOTE_ADDR from the
stream attributes. We only log the peer on receive initial metadata.

The call id assumes census is available. The call ID read from the
context via SERVER_CALL_ID_CONTEXT_KEY on server side, and read from
CallOptions via CLIENT_CALL_ID_CALLOPTION_KEY on client side. The
value is copied from CONTEXT_SPAN_KEY which is set by census.

Pass around CallId with two longs, not a byte[].
2018-04-03 13:39:11 -07:00
zpencer 724e32fe57
core,netty,services: add server listen sockets to channelz proto service (#4220)
Server listen sockets differ from normal sockets in that they do not
have a remote address, do not have stats on calls started/failed/etc,
and do not have security info.
2018-04-03 10:05:47 -07:00
Eric Anderson d99d8d99d8 protobuf: fix @since for fromStatusAndTrailers
It was added in 1e9bd9a8 which was initially released in v1.11.0.
2018-04-03 09:16:56 -07:00
Eric Anderson 41e98ef72b SECURITY.md: Add reference to grpc-netty-shaded
Many people going to the troubleshooting section would be fixed by
swapping to grpc-netty-shaded, so we should call that out.
2018-04-03 08:45:03 -07:00
Eric Anderson ef030abe6b java_grpc_library: re-add grpc-java repo to targets
They were removed in 137c74d1 since it was believed they were unnecessary.
However, since they are in a macro and not a rule, they are relative to the
caller, not their definition.

Added building the examples to the kokoro CI. Note that this means the examples
are built twice: once in grpc-java's build and once in their own (because it
has a WORKSPACE). Given that the Bazel build is our fastest build, this
slowdown won't probably be an issue.
2018-04-03 08:42:40 -07:00
Shohei Kamimori 1d901a8de2 core: add tests for ReadableBuffers 2018-04-03 08:11:39 -07:00
David Brodsky 5b802de3fd okhttp: properly verify IPv6 address hosts (#4292)
Address mismatch between IPv6 address hosts derived from URIs and X509 subjectAltName extensions
2018-04-02 17:43:50 -07:00
zpencer 2347384256
GAE: call channel.shutdown() (#4274)
OkHttpClientInteropServlet and NettyClientInteropServlet both run the
@After method from AbstractInteropTest. Let's make sure we await
termination.

For the long lived channel test, do the cleanup in `destroy`.
2018-04-02 16:02:55 -07:00
zpencer 0067415811
core: document how to set up squid proxy for testing (#4285)
This makes it easier for a gRPC developer to validate the proxy code
using a real proxy.
2018-03-30 11:28:10 -07:00
zpencer 7c37f1d32b
core,netty,okhttp,services: expose socket options to channelz (#4228)
For okhttp, expose the standard options from the Socket object.

For netty, expose all the `io.netty.channel.ChannelOption`s of the
`channel.config()`.
2018-03-30 08:48:37 -07:00
Eric Anderson 2a95e38257 Update README to reference 1.11.0 2018-03-29 11:24:06 -07:00
Shohei Kamimori 156cc44c5f interop-testing: fix empty proto package name
Intended to resolve #4241
grpc-java has already diverged this proto file, so changing the
this package name is reasonable for now.
2018-03-29 10:37:53 -07:00
Eric Anderson 5780743be4 RELEASING.md: Push branch before tag
If the release branch (i.e., v1.11.x) was not up-to-date with the local
repo, then the push to update the branch will fail. Doing it first
allows you to notice, merge in the updates and fix the tag before
pushing it.

Also document pulling the most up-to-date version of the release branch
before creating the 'release' branch. That would reduce the amount of
time the release branch could be modified.
2018-03-29 09:48:35 -07:00
Eric Anderson 4e82e62eaa
Fix compilation in Java 9 2018-03-28 17:13:39 -07:00
ZHANG Dapeng bdecdaea22
netty: http2 server transport graceful shutdown sends 2 GOAWAYs
resolves #3442
2018-03-28 15:58:31 -07:00
Arnout Engelen 03a00aa8cf interop-testing: client compressed tests without probing (#4279) 2018-03-28 15:51:03 -07:00
Shohei Kamimori 5af2515f3e core,README: add docs about grpc-java-api-checker (#4182) 2018-03-28 15:34:18 -07:00
Eric Gribkoff 77e3b97f91 okhttp: add hpack test for max dynamic table size 2018-03-28 14:53:19 -07:00
Eric Gribkoff ecc36ac96d okhttp: include tests in third_party/okhttp 2018-03-28 14:53:19 -07:00
Eric Gribkoff 1efb695399 okhttp: rename ambiguous variables in Hpack reader 2018-03-28 14:53:19 -07:00
Eric Gribkoff b82366159b okhttp: fix HPACK reader bug 2018-03-28 14:53:19 -07:00
zpencer 14003c14cc
build.gradle: bump protobuf plugin to 0.8.5 (#4101)
This update automatically adds generated sources and proto IDLs to the
`idea` plugin.
2018-03-26 17:29:55 -07:00
Eric Anderson 3bb6708eee services: Specify serialVersionUID for the dummy SocketAddress
This fixes a serial warning that breaks the build when building
_without_ ErrorProne. I have no clue why _disabling_ ErrorProne makes
the warning pop up, but whatever, the warning is legit.

I also gave it a name, since the anonymous $1 seems weird if we're
providing a serialVersionUID. Not to say we're actually supporting
serializing this object. But if we ever see a serialized form show up in
the future, having a more clear name would make it obvious what it is.
2018-03-26 16:57:09 -07:00
Eric Anderson 25f357699a
okhttp: Convert to internal ConnectionSpec eagerly
This allows ProGuard to remove OkHttp's ConnectionSpec in most cases,
saving about 40 methods. The savings won't be realized until
DEFAULT_CONNECTION_SPEC is removed.
2018-03-26 15:39:56 -07:00
zpencer 6b753bdc35
buildscripts: add explicit dependency on appengine SDK (#4269)
This fixes the following in GAE:
java.lang.ClassNotFoundException: com.google.appengine.api.ThreadManager
2018-03-26 15:34:17 -07:00
ZHANG Dapeng db64c36af2
core: no retry on dropped PickResult
Address #3553 for normal retry. Not covering the unimplemented hedging case.
2018-03-26 14:57:12 -07:00
Amit Desai 8fe313f598 Marking GrpcServerRule as final
Resolves #4239
2018-03-23 16:34:20 -07:00
Eric Anderson 5f0ee3a725
Bump to Guava 20
Fixes #4176
2018-03-23 15:45:05 -07:00
zpencer a5b55bb24c
netty,okhttp,testing: always set TRANSPORT_ATTR_REMOTE_ADDR (#4217)
Always set the remote address, no reason why this should be a TLS-only
feature. This is needed for channelz, and is especially useful in unit
tests where we are using plaintext.

This PR adds the attr for plaintext.
2018-03-23 15:44:40 -07:00
Ray Tsang ae42d666ad SECURITY.md: Added more troubleshooting instructions 2018-03-23 11:05:57 -07:00
Eric Anderson 7eab0d9468 netty: Add support for Conscrypt 2018-03-23 10:05:45 -07:00
Eric Anderson 0859d480e7 Move TlsTest from interop-testing to netty 2018-03-23 10:05:45 -07:00
zpencer 0955866d5a
core,services: implement binarylog Client and ServerInterceptor (#4174)
- To make unit tests easier, added the BinaryLogSinkWriter abstract
  class, which allows verifying high level arguments rather than low
  level protobufs.
- remove InputStream based marshaller (IDENTITY_MARSHALLER) in favor
  of byte[] because retries will need to call serialize multiple
  times. This is achieved with a special InputStream that lets us
  access the internal byte[] without copying.
- Change ByteBuffer based internal methods to use byte[]
- Update unit tests to reflect the above
2018-03-23 09:29:24 -07:00
Jan Tattermusch 92f95de3cb routeguide: reimplement distance calculation 2018-03-23 08:25:26 -07:00
zpencer 19a64cc336
buildscripts: build windows artifacts in kokoro (#4253)
We always build the artifacts, but only store them in placer via
kokoro if MVN_ARTIFACTS is set to a non empty value.

The 32 bit script runs tests as well as builds artifacts.
The 64 bit script only builds artifacts.
2018-03-22 17:56:59 -07:00
Kun Zhang a43845622f
grpclb: Cache Subchannels for 10 seconds after it is removed from the backendlist (#4238)
This is to conform with the GRPCLB spec. The spec doesn't (yet) define
the actual timeout. We choose 10 seconds here arbitrarily.  It may be
configurable in the future.

This will fix internal bug b/74410243
2018-03-22 17:30:02 -07:00
Eric Gribkoff 6160727274
README.md: fix maven central link (#4256) 2018-03-22 16:22:18 -07:00
Eric Gribkoff 5337728fa7
core,netty: client sends rst stream when server half-closes (#4222) 2018-03-22 12:38:42 -07:00
ZHANG Dapeng ca55b6f7e7
cronet: allow application to provide all threads 2018-03-21 15:04:31 -07:00
Carl Mastrangelo 7c69c3a867
core: disable jndi lookups 2018-03-21 12:55:54 -07:00