Commit Graph

1818 Commits

Author SHA1 Message Date
Carl Mastrangelo ea3f506249 core: avoid allocating Iterators in EquivalentAddressGroup, which is called for each new RPC
Benchmarked with 3 runs of 4 forks
Before:
Benchmark                         (direct)  (transport)    Mode     Cnt       Score     Error  Units
TransportBenchmark.unaryCall1024      true        NETTY  sample  255593  156248.670 ± 563.514  ns/op
Benchmark                         (direct)  (transport)    Mode     Cnt       Score     Error  Units
TransportBenchmark.unaryCall1024      true        NETTY  sample  261443  152753.415 ± 500.957  ns/op
Benchmark                         (direct)  (transport)    Mode     Cnt       Score     Error  Units
TransportBenchmark.unaryCall1024      true        NETTY  sample  258978  154205.374 ± 453.808  ns/op

After:
Benchmark                         (direct)  (transport)    Mode     Cnt       Score     Error  Units
TransportBenchmark.unaryCall1024      true        NETTY  sample  262194  152313.101 ± 502.563  ns/op
Benchmark                         (direct)  (transport)    Mode     Cnt       Score     Error  Units
TransportBenchmark.unaryCall1024      true        NETTY  sample  264811  150809.474 ± 477.459  ns/op
Benchmark                         (direct)  (transport)    Mode     Cnt       Score     Error  Units
TransportBenchmark.unaryCall1024      true        NETTY  sample  263606  151501.729 ± 593.149  ns/op
2016-09-02 15:48:23 -07:00
Kun Zhang c4f7f5c4fd core: split Context into a separate grpc-context artifact.
The Context API is not particularly gRPC-specific, and will be used by
Census as its context propagation mechanism.

Removed all dependencies to make it easy for other libraries to depend
on.
2016-09-02 13:18:35 -07:00
Kun Zhang 58d78dd0aa core: add ServerTransportFilter
Called whenever a ServerTransport is ready and terminated.  Has the
ability to modify transport attributes, which ServerCall.attributes()
are based on.

Related changes:

- Attribute keys for remote address and SSL session are now moved from
ServerCall to a neutral place io.grpc.Grpc, because they can also be
used from ServerTransportFilter, and probably will be used on the
client-side too.  The old keys on ServerCall is marked deprecated and
are equivalent to the new keys.
- Added transportReady() to ServerTransportListener.

Resolves #2132
2016-08-31 14:45:15 -07:00
Carl Mastrangelo 48c6b3d398 all/tests: unmock ClientCall and ServerCall 2016-08-30 09:58:02 -07:00
Carl Mastrangelo 3bf8d94f02 core/tests: add missing RunWith annotation 2016-08-29 12:53:52 -07:00
Carl Mastrangelo 6d28a93e21 core: simplify timeout header processing
Changes slightly improve performance

Benchmark                    (serialized)    Mode     Cnt    Score    Error  Units
GrpcUtilBenchmark.encodeNew         1000n  sample  336623   51.718 ±  1.417  ns/op
GrpcUtilBenchmark.encodeNew         1000u  sample  236574   77.555 ± 20.875  ns/op
GrpcUtilBenchmark.encodeNew         1000m  sample  224392   71.155 ±  1.600  ns/op
GrpcUtilBenchmark.encodeNew         1000S  sample  229616   67.269 ±  2.037  ns/op
GrpcUtilBenchmark.encodeNew         1000M  sample  215301   70.282 ±  1.933  ns/op
GrpcUtilBenchmark.encodeNew         1000H  sample  225063   73.679 ± 20.430  ns/op
GrpcUtilBenchmark.encodeOld         1000n  sample  311832   85.519 ±  1.729  ns/op
GrpcUtilBenchmark.encodeOld         1000u  sample  291613   92.320 ±  1.732  ns/op
GrpcUtilBenchmark.encodeOld         1000m  sample  271871   93.447 ±  1.872  ns/op
GrpcUtilBenchmark.encodeOld         1000S  sample  234932  117.956 ± 16.810  ns/op
GrpcUtilBenchmark.encodeOld         1000M  sample  224636  124.310 ± 20.249  ns/op
GrpcUtilBenchmark.encodeOld         1000H  sample  226764  130.803 ± 19.211  ns/op
GrpcUtilBenchmark.parseNew          1000n  sample  320709   60.480 ±  1.303  ns/op
GrpcUtilBenchmark.parseNew          1000u  sample  316349   64.447 ± 13.673  ns/op
GrpcUtilBenchmark.parseNew          1000m  sample  318209   61.705 ±  2.580  ns/op
GrpcUtilBenchmark.parseNew          1000S  sample  319629   59.342 ±  1.758  ns/op
GrpcUtilBenchmark.parseNew          1000M  sample  305715   59.362 ±  1.489  ns/op
GrpcUtilBenchmark.parseNew          1000H  sample  314919   60.224 ±  1.563  ns/op
GrpcUtilBenchmark.parseOld          1000n  sample  279243   64.040 ±  1.510  ns/op
GrpcUtilBenchmark.parseOld          1000u  sample  278008   71.313 ± 13.620  ns/op
GrpcUtilBenchmark.parseOld          1000m  sample  272633   67.872 ±  2.967  ns/op
GrpcUtilBenchmark.parseOld          1000S  sample  280955   63.966 ±  2.490  ns/op
GrpcUtilBenchmark.parseOld          1000M  sample  257645   71.329 ±  2.117  ns/op
GrpcUtilBenchmark.parseOld          1000H  sample  282510   68.425 ± 17.650  ns/op
2016-08-26 17:32:41 -07:00
Carl Mastrangelo 8894769cdf stub: make more classes final 2016-08-26 15:39:43 -07:00
Eric Anderson 656e8ce37f core: Remove com.google.common.collect usages for Android
This reduces the number of methods gRPC brings in by ~450, which is
substantial. Each application will see different numbers though,
depending on their usage and their other dependencies.

A very rough (under) counting for number of methods included because of
gRPC in android-interop-test is 2746, and that is reduced to 2313 (-433)
by this change. That count includes grpc, guava, okhttp, okio, and nano.
The actual reduction of methods is 447, with the discrepency due to
reduction of methods in java.util and java.lang. Of the 433 removed
methods, 377 are from com.google.common.collect and 61 from
com.google.common.base. The removal costed an increase of 5 methods
(total 1671) within io.grpc itself.
2016-08-26 15:13:45 -07:00
nmittler f254fadfbe Add RunWith annotation to AutoWindowSizingOnTest 2016-08-26 07:18:57 -07:00
Carl Mastrangelo 7c847acf57 core: clarify EOS dataframe error came from server 2016-08-25 17:59:29 -07:00
Lukasz Strzalkowski 864158bdfb core: add tests for ResolvedServerInfo 2016-08-25 16:11:40 -07:00
Carl Mastrangelo 26a08cd75e core: add DoNotMock annotations 2016-08-25 11:14:43 -07:00
Kun Zhang 7909abf065 core: remove unused code path.
writeKnownLength() is never called with compressed == true, because in
most cases it's impossible to know the size of the compressed message.
2016-08-24 15:38:49 -07:00
Lukasz Strzalkowski 23652c5b03 core: change API of NameResolver#onUpdate
Instead of `List<List<ResolvedServerInfo>>`, `onUpdate` now takes
`List<ResolvedServerInfoGroup>` as an argument and every `ResolvedServerInfoGroup`
object can have `Attributes` attached to it which means that we can provide
attributes on each level:

  * root level via `onUpdate` argument (applies to all servers)
  * group level via property of `ResolvedServerInfoGroup` (applies to all servers
    in the group)
  * host level via property of `ResolvedServerInfo` (applies to a single server)
2016-08-24 15:14:57 -07:00
Lukasz Strzalkowski 577164c42f Rename Dummy load balancer to PickFirst 2016-08-23 09:15:56 -07:00
Lukasz Strzalkowski 5345be2ba0 core: Add equals/hashCode
Defines equality conditions for `Attributes` and `ResolvedServerInfo`.
2016-08-23 09:07:41 -07:00
Kun Zhang 63d54731cc Sync load_balancer.proto to latest version. 2016-08-22 15:53:34 -07:00
mfcripps 8a0c89a9f8 integration-tests: add integration tests for auto window sizing 2016-08-19 15:55:39 -07:00
Eric Anderson 092496bcc9 interop-testing: Remove hard-coded ports in ProxyTest 2016-08-19 12:58:12 -07:00
mfcripps d854da63d9 interop-tests: reduce flakiness of proxy tests 2016-08-19 12:46:20 -07:00
Eric Anderson 1f7fb044ab docs: Android examples moved for sed in RELEASING.md
examples/android/app/build.gradle is now
examples/android/helloworld/app/build.gradle and
examples/android/routeguide/app/build.gradle.

Since the number of files is getting a bit long, move it to a variable.
2016-08-19 11:12:09 -07:00
Eric Anderson 57b9dd467f Update README to reference 1.0.0 2016-08-19 09:45:34 -07:00
Kun Zhang 9d747bbbcd build: upgrade to protobuf-gradle-plugin 0.8.0 2016-08-18 13:06:18 -07:00
mfcripps 9ef13a4725 netty: add window resizing to netty handlers and unit tests 2016-08-17 16:21:30 -07:00
Carl Mastrangelo c47d948a47 protobuf: copy input data before decoding
CodedInputStream is risk averse in ways that hurt performance when
parsing large messages.  gRPC knows how large the input size is as it
is being read from the wire, and only tries to parse it once the entire
message has been read in.  The message is represented as chunks of
memory strung together in a CompositeReadableBuffer, and then wrapped
in a custom BufferInputStream.

When passed to Protobuf, CodedInputStream attempts to read data out
of this InputStream into CIS's internal 4K buffer.  For messages that
are much larger, CIS copies from the input in chunks of 4K and saved in
an ArrayList.  Once the entire message size is read in, it is re-copied
into one large byte array and passed back up.  This only happens for
ByteStrings and ByteBuffers that are read out of CIS.  (See
CIS.readRawBytesSlowPath for implementation).

gRPC doesn't need this overhead, since we already have the entire
message in memory, albeit in chunks.  This change copies the composite
buffer into a single heap byte buffer, and passes this (via
UnsafeByteOperations) into CodedInputStream.  This pays one copy to
build the heap buffer, but avoids the two copes in CIS.  This also
ensures that the buffer is considered "immutable" from CIS's point of
view.

Because CIS does not have ByteString aliasing turned on, this large
buffer will not accidentally be kept in memory even if only tiny fields
from the proto are still referenced.  Instead, reading ByteStrings out
of CIS will always copy.  (This copy, and the problems it avoids, can
be turned off by calling CIS.enableAliasing.)

Benchmark results will come shortly, but initial testing shows
significant speedup in throughput tests.  Profiling has shown that
copying memory was a large time consumer for messages of size 1MB.
2016-08-17 15:45:21 -07:00
mfcripps 09d663faf1 integration test: add a flowcontrolling proxy and integration tests 2016-08-17 15:33:13 -07:00
Xiao Hang 0d89bb4942 Update android example.
Show the full stacktrace when error happens and make the result text view scrollable.
2016-08-17 14:17:32 -07:00
Eric Anderson bcf6cc51ca Simplify commands in Android README
Comments should really use '#', since it is shell. Also, we avoid
telling users to clone the git repo since 1) this is basically implicit
already and 2) it encourages them to checkout master instead of using
the latest release. This is especially helpful when the document is
referenced from http://grpc.io/docs since they specify checking out the
latest release (which is much easier to maintain when using jekyll which
is not an option here).
2016-08-17 14:12:41 -07:00
Carl Mastrangelo 1285477133 all: add parameter name to checkNotNull
After debugging #2153, it would have been nice to know what the exact
parameter was that was null. This change adds a name for each
checkNotNull (and tries to normalized on static imports in order to
shorten lines)
2016-08-12 14:55:00 -07:00
Eric Anderson 55942fbd37 auth: Tests should end in Test, not Tests
We use globs internally to find tests, and weren't finding these two
tests. git ls-files confirmed these were the only two files ending in
Tests.
2016-08-11 13:37:34 -07:00
Kun Zhang 7a33fae8e8 core: fix a deadlock in TransportSet.
Resolves #2152
2016-08-10 09:47:25 -07:00
Xiao Hang 3879c6a8fa Update README.md 2016-08-09 14:05:49 -07:00
Eric Anderson c8648dc54e core: Remove usage of LogExceptionRunnable from Deadline
io.grpc should not be depending on anything from internal. Also, the
convenience method of Deadline is part of our public API and shouldn't
use LogExceptionRunnable because it would surprise our users.

Swapped to lower-case 'log' since the logger is not immutable.
2016-08-09 14:02:45 -07:00
Carl Mastrangelo 5384f9706a core: propagate name resolution errors 2016-08-08 17:53:58 -07:00
Eric Anderson ff7fb5ff52 jenkins: Fix Windows build to specify Studio version
If there are multiple versions available, cmake won't choose the Visual
Studio version selected by vsvars. So we have to explicitly specify the
generator to use.

This allows grpc-java to run on the shared Windows workers instead of
its own specialized instance.
2016-08-08 09:12:15 -07:00
Carl Mastrangelo 54f5c4ba89 benchmarks: add fork join pool executor to load server too 2016-08-04 17:34:41 -07:00
Carl Mastrangelo 130d3815cf benchmarks: use a fork-join pool to reduce executor contention 2016-08-04 16:49:10 -07:00
Eric Anderson 09d7a41946 examples: Explain reason for plainText() 2016-08-04 15:33:36 -07:00
Eric Anderson 7c44a4e111 docs: Clarify the relationship between Status*Exception 2016-08-04 15:33:31 -07:00
Eric Anderson d394cef775 core: Avoid wrapping Errors in RuntimeException 2016-08-04 14:25:40 -07:00
Eric Anderson f25f55a76f docs: Update README.md before tagging release 2016-08-04 13:55:29 -07:00
Carl Mastrangelo 1aadc6c223 benchmarks: update to using jmh 1.13 2016-08-04 13:04:49 -07:00
Xiao Hang 7267c0fbed Change routeguide example to use proto 3.0.0 2016-08-04 11:47:53 -07:00
Eric Anderson ab85d8751b Update to Gradle 2.14
Is faster and includes a security fix.
2016-08-04 10:03:57 -07:00
Xiao Hang 239d6ea299 Add android route guide example and move helloworld example to its own dir 2016-08-03 14:47:10 -07:00
Eric Anderson 7d464fcb02 inprocess: Avoid creating unnecessary threads
Implementations of ManagedClientTransport.start() are restricted from
calling the passed listener until start() returns, in order to avoid
reentrency problems with locks. For most transports this isn't a
problem, because they need additional threads anyway. InProcess uses no
additional threads naturally so ends up needing a thread just to
notifyReady. Now transports can just return a Runnable that can be run
after locks are dropped.

This was originally intended to be a performance optimization, but the
thread also causes nondeterminism because RPCs are delayed until
notifyReady is called. So avoiding the thread reduces needless fakes
during tests.
2016-08-02 13:16:36 -07:00
Eric Anderson a8700a7837 Begin consuming protobuf-lite artifact
Protobuf-lite since beta-4 is now more of a fork than a subset of
protobuf-java, which may cause us problems later since lite API is not
stable. Also, lite-generated code may now depend on APIs only in
protobuf-lite, so our users must depend on the protobuf-lite runtime.
Having all our users explicitly override the dependency is bothersome to
them and can easily only expose problems only after we do a release.

So now we are doing the dependency overriding; most users should "just
work" and pick up the correct protobuf artifact. I've confirmed the
exclusion is listed in the grpc-protobuf pom and "gradle dependencies"
and "mvn dependency:tree" do not include protobuf-lite for the examples.
Vanilla protobuf users are most likely to experience any breakage, which
should detect problems more quickly since we use protobuf-java more
frequently than protobuf-lite during development.

protobuf-lite does not include pre-generated code for the well-known
protos, so users will need to generate them themselves for the moment
(google/protobuf#1889).

Note that today changing deps does not noticeably reduce the method code
for our users, since ProGuard already is stripping most classes. The
difference in output is only a reduction of 3 classes and 6 methods for
the android example.
2016-08-01 15:51:11 -07:00
Jakob Buchgraber 21b4ce70ea netty: update to 4.1.4.Final 2016-08-01 21:16:45 +02:00
Eric Anderson 5b7a21ab03 okhttp: Remove wrong status description
The != should have been ==. However, it is provable that the exception
won't be null, but we want to make that fact obvious when auditing. So
we just fail if the exception is ever null.
2016-08-01 10:43:21 -07:00
Eric Anderson fb8402fcb1 stub: Avoid double-wrapping status
780b2696 caused all failures for blocked unary stubs to have a
StatusRuntimeException as the cause of the StatusRuntimeException, with
the two exceptions having almost the same status.
2016-08-01 09:53:57 -07:00