Commit Graph

2719 Commits

Author SHA1 Message Date
zpencer c6c20e9491
buildscripts: add linux job that release_artifacts can dep on (#4392)
* buildscripts: add linux job that release_artifacts can dep on

* rename linux -> linux_artifacts

* fix script name
2018-04-25 15:54:15 -07:00
Carl Mastrangelo 720d4fab69
all: print out diff of methods in android APK 2018-04-25 15:53:56 -07:00
Eric Gribkoff ef2a085acd
android-interop-testing: re-add min sdk version (#4393) 2018-04-25 14:48:35 -07:00
zpencer 1a2d076aed
core,netty,okhttp,services,testing: expose security info to channelz (#4300)
Pull the TLS info from the SSLSession object for TLS, and AltsContext for ALTS.
2018-04-25 14:38:09 -07:00
zpencer 218e944e16
buildscripts: initial kokoro config for auto releasing artifacts (#4391)
The script does nothing at the moment other than set up the kokoro
job.
2018-04-25 14:12:06 -07:00
zpencer 7cd2f5c3c0
buildscripts: sonatype uploader ussing CONF before it is set (#4388)
Set $CONF before we use it.
Also enable some defensive bash flags to catch errors.
Default value for USER and PASS
2018-04-25 12:54:49 -07:00
Carl Mastrangelo 6bdf5de342
alts: move alts protos to match proto package 2018-04-25 12:01:22 -07:00
Eric Gribkoff 3beb73eb46
android-interop-testing: update app dependencies (#4313) 2018-04-24 14:33:06 -07:00
zpencer 62e6e2de78
Start 1.13.0 development cycle (#4383) 2018-04-23 15:24:24 -07:00
Eric Gribkoff ba86a86c77
android: add AndroidChannelBuilder (#4172) 2018-04-23 11:49:36 -07:00
Eric Anderson a47266ea4a kokoro: Avoid --include-build for Android
Since 4369e8cd the --include-build just opens us up to trouble with
accidentally building the protoc plugin. Since we're going to do a
./gradlew install anyway, let's just wait until after that point for
building cronet.

This sort of problem was experienced while developing #4369.
2018-04-21 00:03:44 -07:00
Carl Mastrangelo 9f6270848b
benchmarks: remove unused proto import 2018-04-20 17:40:32 -07:00
zpencer 48b7c62b43
buildscripts: always keep mvn artifacts (#4372)
After searching for "artifacts retention policy" in the
kokoro-users group, I learned that there's a default 90 day
retention policy already in place.
2018-04-20 15:19:48 -07:00
Carl Mastrangelo c37ea15830
core: also expose parser on JsonParser 2018-04-20 13:25:21 -07:00
ZHANG Dapeng 74532acce0
core: use List instead of Set for drainedSubstreams
This improves latency performance (retry enabled) for NETTY transport by about 2us. For INPROCESS, NETTY_LOCAL, OKHTTP transports, the improvement seems much less than 2us.

```
before
Benchmark                                               (direct)  (transport)    Mode     Cnt         Score     Error  Units
TransportBenchmark.unaryCall1024                            true        NETTY  sample  260801     76566.536 ± 189.439  ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.00        true        NETTY  sample             60480.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.50        true        NETTY  sample             75264.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.90        true        NETTY  sample             85504.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.95        true        NETTY  sample             87424.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.99        true        NETTY  sample            100864.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.999       true        NETTY  sample            140800.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.9999      true        NETTY  sample            205547.469            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p1.00        true        NETTY  sample           3915776.000            ns/op 
TransportBenchmark.unaryCall1024                           false        NETTY  sample  208352     95865.619 ± 113.142  ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.00       false        NETTY  sample             72576.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.50       false        NETTY  sample             93568.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.90       false        NETTY  sample            105728.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.95       false        NETTY  sample            109568.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.99       false        NETTY  sample            124544.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.999      false        NETTY  sample            161792.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.9999     false        NETTY  sample            230520.448            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p1.00       false        NETTY  sample           3997696.000            ns/op
```

```
after
Benchmark                                               (direct)  (transport)    Mode     Cnt         Score     Error  Units
TransportBenchmark.unaryCall1024                            true        NETTY  sample  269471     74104.666 ± 182.514  ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.00        true        NETTY  sample             60992.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.50        true        NETTY  sample             70912.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.90        true        NETTY  sample             83584.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.95        true        NETTY  sample             85888.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.99        true        NETTY  sample            100224.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.999       true        NETTY  sample            142848.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.9999      true        NETTY  sample            489527.706            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p1.00        true        NETTY  sample           4300800.000            ns/op
TransportBenchmark.unaryCall1024                           false        NETTY  sample  216000     92468.337 ±  90.229  ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.00       false        NETTY  sample             68480.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.50       false        NETTY  sample             89472.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.90       false        NETTY  sample            103680.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.95       false        NETTY  sample            107008.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.99       false        NETTY  sample            120960.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.999      false        NETTY  sample            159232.000            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p0.9999     false        NETTY  sample            272076.493            ns/op
TransportBenchmark.unaryCall1024:unaryCall1024·p1.00       false        NETTY  sample           2662400.000            ns/op
```
2018-04-20 11:22:32 -07:00
Carl Mastrangelo c8aa9f70ca
services: move channelz proto into normalized directory structure 2018-04-20 10:53:45 -07:00
zpencer f5e8ec18b7
services: avoid static initialization for BinaryLog.java (#4363)
The code that uses this will create an instance.
2018-04-19 17:51:49 -07:00
Eric Anderson ddf77f59cc core: Fix experimental API issue for channel state API
The earlier issue was for the feature itself, not the stabilization.
2018-04-19 17:38:20 -07:00
zpencer 934c2e2fd7
core: clarify binlog API (#4362)
The interceptors are a part of the implementation and not meant for
callers.

Both methods for wrapping clients / servers can be `final`.
2018-04-19 17:32:45 -07:00
zpencer ec95999124
core: remove unneeded binlog tests for ManagedChannelImpl ServerImpl (#4361)
Replace the tests with tests that just make sure binary log is
installed. Better tests exist today in BinaryLogProviderTest to make
sure the actual logic is correct.
2018-04-19 16:55:32 -07:00
ZHANG Dapeng e67a61fa08
core: refactor ManagedChannelImplTest with a configurable builder
In previous code of `ManagedChannelImplTest`, new tests will have to add more and more custom args in `createChannel(...)`, and each time when a custom arg is needed all other default args need be repeatedly provided in `createChannel(...)`. In most cases those args are just builder attributes. So this PR makes ManagedChannelImplTest easier to stub by providing a builder field, then new tests will just set necessary builder attributes before calling `createChannel()`.
2018-04-19 10:06:39 -07:00
ZHANG Dapeng f6a032d6b2
core: forward toString for forwarding classes
Forward `toString()` method for forwarding classes to improve debug information.

For example, `ForwardingManagedChannel.toString()` will return something like:

```
ForwardingManagedChannel{delegate=ManagedChannelImpl{logId=tag-13247, target=localhost:8080}}
```
2018-04-19 10:05:48 -07:00
Carl Mastrangelo f754bff906
core: make JsonParser public 2018-04-18 18:49:08 -07:00
zpencer 30400e0e58
core: add sockets to subchannel stat (#4345)
This is info not being reported at the moment and is missing from the
display.
2018-04-16 11:33:28 -07:00
zpencer c04278ea07
services: turn channelz null futures into Status.UNIMPLEMENTED (#4346)
Then channelz GUI will take this into account. This is particularly
useful for InProcessTransport, where I have decided we do not need
special support for in channelz. The server and channel stats are
already sufficient.
2018-04-16 09:54:35 -07:00
zpencer 791a29f63b
services: socket options were erroneously ignored if socket has no stats (#4336)
This fixes listen sockets. It is ok to have no data but report socket
options.
2018-04-13 14:59:35 -07:00
Carl Mastrangelo e4502aca5d
stub: add docs clarifying readiness 2018-04-13 14:12:54 -07:00
zpencer 1d5ed78c8f
services: make channelz constructor static (#4334)
Right now this is inaccessible to services.
2018-04-13 14:10:50 -07:00
zpencer c50a57cd52
netty: fix visibility issues with InternalNettySocketSupport (#4335)
Previous version is not actually extendable from other packages.
2018-04-13 14:10:28 -07:00
Carl Mastrangelo 03465a7f45
core: don't warn on absent service config
Fixes #4333
2018-04-13 11:13:57 -07:00
ZHANG Dapeng a6811d5bb1
all: add toString() to Server to improve debug info 2018-04-13 09:41:09 -07:00
ZHANG Dapeng 0eafb8e9ca
core: compare doubles with compare instead of ==
Always not compare equality for doubles with `==`.
2018-04-13 09:39:59 -07:00
Eric Anderson 0a598c053c core: Fix lint, style, "good ideas" violations 2018-04-12 17:26:54 -07:00
ZHANG Dapeng 6d7e19cbe5
core: replumb RetryPolicy
- replumbed `RetryPolicy` with `MethodInfo` without breaking the existing `RetryPolicyTest`.
- moved `ServiceConfigInterceptor.MethodInfo.RetryPolicy` out as a top level class so that `RetriableStream` does not import `ServiceConfigInterceptor`.
2018-04-12 11:41:04 -07:00
Kun Zhang 3f63cd0ad4
core: always call StreamTracer.streamClosed() when stream is officially closed. (#4331)
Previously StreamTracer.streamClosed() is called in
ServerStream.close(), but it is not exactly when the stream is
officially closed.  ServerStreamListener.closed() is guaranteed to be
called and it is the official end of the stream.
2018-04-12 09:05:48 -07:00
zpencer 7c46bd9399
netty: prepare NettySocketSupport for internal import (#4311)
Delegate the actual heavy lifting to a helper class that can be easily swapped
at runtime.
2018-04-11 16:37:13 -07:00
zpencer 137e759fda
core, netty: allow InputStream based certs (#4316)
Allow ServerBuilder to read certs from InputStream, not just from a
File.
2018-04-11 16:14:58 -07:00
Shohei Kamimori d68b2cd74a protobuf: add test for ProtoUtils (#3838) 2018-04-06 10:10:08 -07:00
zpencer 8d3d26dda1
compiler: allow capitalized java keywords as rpc names (#4309)
previously `Import` fails because it gets turned into a method named
`import`. This PR makes the method be named `import_`, everything else
stays the same, such as the name in the method descriptor.

Note: `iMport` or `iMPORT` or any variation of capitalizations that is
not `Import` always worked. `Import` gets translated into `import`
because of our attempts to camel case the names.
2018-04-05 16:45:19 -07:00
Eric Anderson ecab86fad9 examples: upgrade protobuf-maven-plugin to 0.5.1
This was missed in 2094bb4d8
2018-04-05 14:14:32 -07:00
zpencer 9f92104b94
buildscripts: fix GAE dummy app deploy races (#4308)
This fixes a race where we can race with another deploy of the
'dummy' version.

Also, when we delete old versions, only delete versions beginning with
'kokoro'. This would make races even less likely because the
'dummy-default' should normally stick around forever.

This fixes this failure:
https://source.cloud.google.com/results/invocations/6b24aed4-7dc3-490a-87ea-c870d1435184/targets/grpc%2Fjava%2Fpresubmit%2Fgae-interop/log
2018-04-05 13:19:36 -07:00
zpencer 62cc2b1ae3
core,netty: add NettySocketSupport to populate TcpInfo (#4306)
NettySocketSupport is responsible for making the low level calls to
get and populate the TcpInfo structure.
2018-04-05 10:36:49 -07:00
Carl Mastrangelo 2df76cc710
core: apply service config 2018-04-04 17:45:12 -07:00
zpencer 49c8bdb60a
netty: fix getListenSockets race (#4301)
Move registration to separate future and wait for it.
2018-04-04 17:04:29 -07:00
Carl Mastrangelo 9ed84258aa
core: don't reschedule idle timer if it is already active
Benchmark results (3 runs each)

non direct

```
Before:
50.0%ile Latency (in nanos):		157471
90.0%ile Latency (in nanos):		185927
95.0%ile Latency (in nanos):		195135
99.0%ile Latency (in nanos):		218815
99.9%ile Latency (in nanos):		1188735
100.0%ile Latency (in nanos):		18333695
QPS:                           6126
50.0%ile Latency (in nanos):		160407
90.0%ile Latency (in nanos):		188551
95.0%ile Latency (in nanos):		197487
99.0%ile Latency (in nanos):		219575
99.9%ile Latency (in nanos):		390239
100.0%ile Latency (in nanos):		18338815
QPS:                           6106
50.0%ile Latency (in nanos):		157831
90.0%ile Latency (in nanos):		186439
95.0%ile Latency (in nanos):		195815
99.0%ile Latency (in nanos):		216951
99.9%ile Latency (in nanos):		281167
100.0%ile Latency (in nanos):		5384447
QPS:                           6235

After:
50.0%ile Latency (in nanos):		152255
90.0%ile Latency (in nanos):		180551
95.0%ile Latency (in nanos):		188943
99.0%ile Latency (in nanos):		209623
99.9%ile Latency (in nanos):		1184831
100.0%ile Latency (in nanos):		4351999
QPS:                           6313
50.0%ile Latency (in nanos):		153663
90.0%ile Latency (in nanos):		181671
95.0%ile Latency (in nanos):		189991
99.0%ile Latency (in nanos):		210495
99.9%ile Latency (in nanos):		278895
100.0%ile Latency (in nanos):		18283519
QPS:                           6300
50.0%ile Latency (in nanos):		152767
90.0%ile Latency (in nanos):		180839
95.0%ile Latency (in nanos):		189791
99.0%ile Latency (in nanos):		211719
99.9%ile Latency (in nanos):		280927
100.0%ile Latency (in nanos):		12231167
QPS:                           6381
```

direct:

```
Before:
50.0%ile Latency (in nanos):		133943
90.0%ile Latency (in nanos):		153671
95.0%ile Latency (in nanos):		163655
99.0%ile Latency (in nanos):		188871
99.9%ile Latency (in nanos):		235791
100.0%ile Latency (in nanos):		7864575
QPS:                           7134
50.0%ile Latency (in nanos):		131623
90.0%ile Latency (in nanos):		151863
95.0%ile Latency (in nanos):		162095
99.0%ile Latency (in nanos):		187719
99.9%ile Latency (in nanos):		234983
100.0%ile Latency (in nanos):		17836031
QPS:                           7250
50.0%ile Latency (in nanos):		131223
90.0%ile Latency (in nanos):		150823
95.0%ile Latency (in nanos):		161311
99.0%ile Latency (in nanos):		187719
99.9%ile Latency (in nanos):		237471
100.0%ile Latency (in nanos):		4416255
QPS:                           7273

After:
50.0%ile Latency (in nanos):		122751
90.0%ile Latency (in nanos):		140967
95.0%ile Latency (in nanos):		148911
99.0%ile Latency (in nanos):		173215
99.9%ile Latency (in nanos):		214823
100.0%ile Latency (in nanos):		18509823
QPS:                           7774
50.0%ile Latency (in nanos):		124507
90.0%ile Latency (in nanos):		145855
95.0%ile Latency (in nanos):		156623
99.0%ile Latency (in nanos):		183111
99.9%ile Latency (in nanos):		235679
100.0%ile Latency (in nanos):		18289663
QPS:                           7625
50.0%ile Latency (in nanos):		124295
90.0%ile Latency (in nanos):		145071
95.0%ile Latency (in nanos):		156439
99.0%ile Latency (in nanos):		183919
99.9%ile Latency (in nanos):		232447
100.0%ile Latency (in nanos):		3712383
QPS:                           7632
```
2018-04-04 16:02:02 -07:00
zpencer 276586a4fb
netty,services: fix param comment lint (#4304)
Fix linter complaint because comment does not match arg name.
2018-04-04 12:06:31 -07:00
Jorg Heymans 2094bb4d8e doc: upgrade protobuf-maven-plugin in README.md 2018-04-03 16:51:56 -07:00
Shohei Kamimori 8f01084bb3 core: add a `close` to InputBufferStream
Before:
`InputBufferStream.close()` does not close their buffer so the buffer will leak.

After:
Resolves #4198.
Override the `close` for closing their buffer.
2018-04-03 16:11:19 -07:00
Eric Anderson bace06fe9f
java_grpc_library: Add support for protobuf lite
gRPC's protobuf-lite auto-selects between full and lite protobuf based on the
value of crosstool_top. If the user is specifying their own
--android_crosstool_top, then it will not auto-detect correctly. One day,
platforms will fix problems like this, but for the moment it seems we get to
live with it.
2018-04-03 15:22:55 -07:00
zpencer b1d91b9f60
core,services: binary log should use real peer socket and call id (#4266)
The peer socket is read from TRANSPORT_ATTR_REMOTE_ADDR from the
stream attributes. We only log the peer on receive initial metadata.

The call id assumes census is available. The call ID read from the
context via SERVER_CALL_ID_CONTEXT_KEY on server side, and read from
CallOptions via CLIENT_CALL_ID_CALLOPTION_KEY on client side. The
value is copied from CONTEXT_SPAN_KEY which is set by census.

Pass around CallId with two longs, not a byte[].
2018-04-03 13:39:11 -07:00