Commit Graph

478 Commits

Author SHA1 Message Date
Eric Anderson 4f4f8e40bf Remove Guava's Service from server transport
ServerImpl.start() now throws IOException to make the error explicit.
This was previously being papered over by wrapping the exception in
RuntimeException.
2015-04-16 11:28:04 -07:00
nmittler f920badc5e Upgrading to the latest Netty version. 2015-04-16 08:42:56 -07:00
Xudong Ma 883eb62de4 OkHttp: Temporally support multiple h2-xx protocol on client side.
Since the user provided SSLSocketFactory (especially in Android) may already do the handshake when creates the SSLSocket, and it may choose a different protocol name other than the one OkHttp is using.

Resolves #293
2015-04-16 09:04:19 +08:00
Eric Anderson 5b2e336b6f netty: Add option to set MAX_CONCURRENT_STREAMS 2015-04-13 13:46:36 -07:00
Eric Anderson 1e257c440e Remove unused local variable 2015-04-10 08:58:43 -07:00
Jakob Buchgraber e8afa3ca23 Fix bug where the stream id would not get incremented for buffered streams. 2015-04-09 13:04:39 -07:00
Xudong Ma 9d214637e6 Temporally support Http protocol name "h2-16" for OkHttp client.
Fixes #22 for real.

Manually tested with jdk8 by:
./run-test-server.sh
./run-test-client.sh --server_host_override=foo.test.google.fr --use_test_ca=true --use_okhttp=true
2015-04-09 13:50:18 +08:00
Eric Anderson fd7ba566aa netty: Use the bootstrap ClassLoader for ALPN/NPN
If ALPN.class isn't in the bootclasspath, we don't want to find it as it
won't work. Also, if someone has fiddled with ClassLoaders, we really
want to make sure we get the proper ALPN class.

Passing "null" to Class.forName() means "bootstrap class loader". In
fact, ClassLoader.getParent() says, "Some implementations may use null
to represent the bootstrap class loader."

We must load the Proxy in the bootstrap class loader as well, because
our class loader may not have access to ALPN.{Client,Server}Provider,
for the same reasons as above. Even if we have multiple instances of
gRPC (due to class loaders), combined they will only create two Proxy
classes: one for ALPN.ClientProvider and one for ALPN.ServerProvider.
2015-04-08 15:37:44 -07:00
Jakob Buchgraber 44f9904ab0 Replace 'internal' import with correct one.
The errornous import was introduced by my previous commit 5f70a12cfc
2015-04-08 12:54:30 -07:00
Xudong Ma f3ccdd99f1 Fixes Travis breakage 2015-04-08 13:00:09 +08:00
Jakob Buchgraber 5f70a12cfc Remove blocking on client stream creation and buffer RPC Calls for when the MAX_CONCURRENT_STREAMS limit is hit. Fixes #118 2015-04-07 17:06:21 -07:00
Jakob Buchgraber ce860f9003 fix compile error introduced by previous commit 2015-04-07 13:00:24 -07:00
Jakob Buchgraber 105964bfac Fix memory leak by adding the Http2StreamRemovalPolicy to the channel pipeline.
In benchmarks we would see grpc talking up tens of gigabytes of memory. A heap dump
revealed that streams would not get cleaned up and stick around in memory forever.
2015-04-07 12:56:27 -07:00
Xudong Ma 9aae6f65fa checkstyle change:
1. Adds <property name="separateLineBetweenGroups" value="true"/> to CustomImportOrder to enfore blank line between imports groups.
2. Uses checkstyle 6.5, which fixed a bug of "CustomImportOrder checks import sorting according to ASCII order instead of case-insensitive alphabetical order".
2015-04-07 08:27:48 +08:00
Eric Anderson 74c9b5ed02 Fix JavaDoc references to non-imported class
The ServerImpl import was removed because it wasn't used as far as
checkstyle was able to determine. However, it was being used to resolve
JavaDoc references. Instead of re-adding the import just make the
reference fully qualified to prevent the two systems from continuing to
disagree on whether it is needed.
2015-04-03 15:11:51 -07:00
Xudong Ma 2cad9e6000 Add "unsed import" check. 2015-04-03 16:55:01 +08:00
nmittler 11d0094e04 Upgrading to the latest Netty 4.1 branch. 2015-03-31 14:14:29 -07:00
Eric Anderson 9bd31daee6 Wait for handler registration 2015-03-26 12:35:31 -07:00
Eric Anderson e515c772cd netty: Status should be based on GOAWAY code
goingAway() is called before onGoAwayRead() in Netty:
b7f57223c1/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2ConnectionDecoder.java (L521)

The test before checked that the stream went away, but not that the
GOAWAY code influenced our Status, as UNAVAILABLE is the default
internally.

The UNAVAILABLE default has also been changed to include a message so
that we can determine where the Status came from in case it is triggered
again in the future.
2015-03-25 11:04:52 -07:00
nmittler 6407c18578 Default client connection window to 1MiB. 2015-03-24 10:29:20 -07:00
nmittler ab9f82756a Making connection and stream windows configurable for Netty. 2015-03-24 10:14:55 -07:00
Eric Anderson c3e8dae6ce Add checkstyle checking
The checkstyle.xml is a slightly modified version of the upstream Google
checkstyle configuration. All changes have comment describing them.

Lots of warnings were corrected. Examples is the only project that has
warnings still, as the necessary changes require some thought.
2015-03-16 10:53:13 -07:00
Jakob Buchgraber 54c31b0325 Fix compile error introduced by previous commit 2015-03-13 15:12:40 -07:00
Louis Ryan 3f7325ecf5 Allow switching between Netty 4 & 5 by restricting to intersection of interfaces 2015-03-13 14:57:55 -07:00
Jakob Buchgraber 0076243063 Add WritableBuffer interface for zero copy data writes. Fixes #8
WritableBuffer is a generic interface that allows to transfer data
from gRPC directly to the native transport's buffer implementation.
2015-03-13 13:00:44 -07:00
Louis Ryan 1216de6262 Add support to Netty builders for other channel types. Demonstrate and test use of this with local channels 2015-03-12 16:05:33 -07:00
nmittler d4d66a5fef Better error description onGoAwayRead 2015-03-12 08:16:52 -07:00
Jakob Buchgraber f822416f35 Use DEFAULT_WORKER_EVENT_LOOP_GROUP for both client and server. Fixes #82
+ Use only one thread for the boss eventloop group by default.
2015-03-11 14:31:51 -07:00
nmittler 2a425090d1 Migrating to Netty 4.1
Fixes #164
2015-03-11 13:18:27 -07:00
nmittler 54be11ef3c Updating to the latest Netty version. 2015-03-10 15:43:19 -07:00
Nathan Mittler f1b6f623c5 Merge pull request #166 from nmittler/doclint
Tightening up error message for GO_AWAY.
2015-03-09 13:49:03 -07:00
nmittler dfcfb7bca1 Tightening up error message for GO_AWAY.
The attempt here is to identify all causes of GO_AWAY and to ensure
there is a reasonable description to help understand the cause.

Fixes #163
2015-03-09 13:48:15 -07:00
nmittler 4deff027ba Proper buffer closure when receiving DATA with EOS
The Http2ClientStream should not close the buffer in this case since
it's already been given to the deframer and potentially to the user.

Added cleanup code to MessageDeframer and AbstractClientStream to make
sure that we free the Buffer when appropriate.
2015-03-09 13:28:32 -07:00
Jakob Buchgraber 4a2c0a5637 Have SendGrpcFrameCommand constructor take an AbstractStream object instead of a stream id.
As part of the effort to remove all blocking bits from the NettyClientStream with
this commit the SendGrpcFrameCommand now takes a stream object instead of the
stream id. This will be necessary as in a non blocking world the stream id might
not have yet been allocated when the SendGrpcFrameCommand gets instantiated.
2015-03-02 11:37:57 -08:00
Jakob Buchgraber 1961402104 Revert "Have SendGrpcFrameCommand constructor take an AbstractStream object instead of a stream id."
This reverts commit 0d78268b83.

Some unit tests aren't compiling due to this change.
2015-02-25 14:05:57 -08:00
Jakob Buchgraber 7ef1781016 Cleanup and Nitpicking
- Renamed 'eventGroup' property to 'group' as this what's used elsewhere.
- Moved assignment of the channel before the listener is added as currently
  there is a (theoretical) chance that the listener is executed before the assignment
  to channel happens, namely in case the Future is already done when the listener
  is added.
- Removed comment that seemed out of place / relict.
2015-02-25 12:28:25 -08:00
Jakob Buchgraber 0d78268b83 Have SendGrpcFrameCommand constructor take an AbstractStream object instead of a stream id.
As part of the effort to remove all blocking bits from the NettyClientStream with
this commit the SendGrpcFrameCommand now takes a stream object instead of the
stream id. This will be necessary as in a non blocking world the stream id might
not have yet been allocated when the SendGrpcFrameCommand gets instantiated.
2015-02-25 12:20:35 -08:00
Nathan Mittler 488f8a9989 Merge pull request #117 from nmittler/stream_removal_fix
Fixing memory leak in stream removal policy
2015-02-25 09:55:40 -08:00
nmittler 23f513c0d2 Some comment cleanup in the Netty builders. 2015-02-24 13:16:21 -08:00
nmittler cc170418ea Fixing memory leak in stream removal policy
The stream removal policy was never being added as a handler, so it was
never starting its periodic timer to remove streams.
2015-02-24 12:55:26 -08:00
Louis Ryan 425e3c0b7f Remove dead GrpcFramingUtil class 2015-02-23 17:44:23 -08:00
Louis Ryan 95c6f5082c Udate TODOs to use Github user names 2015-02-23 12:46:28 -08:00
Jakob Buchgraber c6c0a3bf69 Remove user prefix from NettyServerBuilder.userBossEventLoopGroup(). Fixes #73 2015-02-20 13:43:20 -08:00
Jakob Buchgraber 07d0917b94 Add ServerImpl#awaitTerminated() method. Fixes #80 2015-02-18 16:00:34 -08:00
Eric Anderson 74f231a6dd Remove Guava Service from ClientTransport
OkHttp no longer cancels all calls on shutdown, as we want to allow
graceful shutdown. Such cancelling behavior will likely be provided by
Channel in the future.
2015-02-12 15:20:22 -08:00
Eric Anderson 127270bd5f Fix shutdown race with negotiation
If the transport had started but negotiation had not completed then
previously channel would be null and shutdown would have no effect. Now
we set channel eagerly, but use a separate variable for determining if
it is safe for general use.
2015-02-10 12:47:53 -08:00
Eric Anderson 8aa79b39fc Improve thread safety of newStream()
OkHttp should now have a thread-safe implementation of newStream.
Previously 'lock' was not held when checking goAway and the checking in
AbstractClientTransport was redundant.

Netty was thread-safe, but it was very hard to tell what guarantees were
necessary and what guarantees each piece was providing.
2015-02-06 14:12:01 -08:00
Eric Anderson c84ef8332e Simplify connection callback handling in Netty
Channel is available immediately after connect(), so register callbacks
immediately instead of delaying.

Setting channel is now delayed until it is actually safe to use.
2015-02-05 13:35:49 -08:00
Eric Anderson e6a7b6c8a9 netty: Don't throw exception on double close 2015-02-04 15:46:28 -08:00
Eric Anderson f3a90cd42b netty: Cancel stream if interrupted during create
Previously streams were being partially orphaned if there was an
interruption during stream creation. To handle cancellation,
AbstractClientStream's cancel() had to be changed remove the
"optimization" otherwise, again, the stream would be orphaned.
2015-02-04 15:46:27 -08:00
nmittler 89b8d7ff47 Moving decompression to the channel thread. 2015-01-29 14:55:40 -08:00
nmittler f83145865a Removing all references to "stubby" 2015-01-27 11:25:25 -08:00
Eric Anderson aeeebb7cdb Remove Service API from ServerImpl
Fixes #21
2015-01-27 09:20:24 -08:00
Eric Anderson 0077e67274 Remove unused imports 2015-01-26 10:23:05 -08:00
Adrian Cole c4a43e6bdc Fixes api drift, addressing inbound flow control. 2015-01-23 08:34:07 -08:00
nmittler de3a13164f Changing gRPC Java inbound flow control model
The goal is to mirror the token-based approach used by the Reactive
Streams API.
2015-01-22 11:46:16 -08:00
ejona 4de2026492 Fix TODO attribution
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=84172357
2015-01-16 16:23:05 -08:00
simonma f589a4acbb Remove traces of gRPC v1 protocol vs gRPC v2 protocol.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=84087425
2015-01-16 11:03:38 -08:00
ejona 7235a396b8 Remove Service API from ChannelImpl
This change loses asynchronous notification of channel state-change and
a way to wait until the channel is actually connected. Both of these are
expected to be added back as part of a health API. The important
distinction from Service is that ChannelImpl never permanently fails and
can revert from being started to connecting again.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83875407
2015-01-15 13:42:49 -08:00
nathanmittler 11c363a7ae Disallowing message delivery after status has been delivered to the listener.
Summary of changes:

1) Merged the interfaces MessageDeframer2.Sink and DeframerListener into
MessageDeframer2.Listener. This simplifies the interface of
MessageDeframer2 quite a bit.

2) Added a deliveryPaused() handler to MessageDeframer2.Listener, which
is called by the deframer when there is not enough data to read/deliver
the next message.

3) Modified AbstractStream and AbstractClientStream to manage the timing
of when the closed() event is delivered to the listener. The
transportReportStatus ultimately controls this by creating a task to
close the listener. It either runs this task immediately or when the
next deliveryPaused() event occurs.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83620903
2015-01-09 17:00:25 -08:00
ejona c0f41920bd Remove gRPC v1 support.
No major refactorings/simplifications were done. Only gRPC v1 support
infrastructure was removed.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82737436
2015-01-08 14:43:23 -08:00
lryan b09c26b4dd Correctly handle disconnect notifications from Netty.
Previously the code close the HTTP2 streams prior to notifying the
application layer. This was the wrong order as the code depended on
enumerating the open streams to notify the application layer.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82733459
2015-01-08 14:43:22 -08:00
ejona fc30031fd1 Change default gRPC protocol to v2.
Any place that force-sets the protocol to 2 or assumes the old value
is now removed. Unfortunately, it seems InProcessTransportTest has
some non-obvious dependency on gRPC v1.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81597524
2015-01-08 14:43:18 -08:00
lryan 56e307fcb6 Add BSD license header to all source files
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81447025
2015-01-08 14:43:17 -08:00
lryan 7f55e8163a Allow use of a LocalChannel with Netty & HTTP2
Remove old in-process handling
Update tests and benchmarks

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81381986
2015-01-08 14:43:15 -08:00
zhangkun 2b116ef2cd Encode binary headers with Base64 on the wire, and requires all binary headers
have "-bin" suffix in their names.

Split Metadata.Marshaller into BinaryMarshaller and AsciiMarshaller.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81306135
2015-01-08 14:43:15 -08:00
zhangkun 8375cd00e8 GRPC Java clients will send the "te: trailers" header, and the server will
check for it, so that we can detect intermediate proxies that do not support
trailers.

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81084983
2015-01-08 14:43:13 -08:00
lryan c5e70c2310 Remove StreamState and use inboundPhase/outboundPhase instead
Remove synchronization on stateLock as we are not required to be thread safe
Add better toString for stream impls
Internal cleanup of various 'status' fields in AbstractClientStream
Remove 'stashTrailers' as we've already extracted status in layer above correctly

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80678356
2015-01-08 14:43:12 -08:00
nathanmittler be29ba57cc Upgrading to latest Netty.
Fixes race condition while shutting down.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80484055
2015-01-08 14:43:12 -08:00
ejona f96e7e8c6c Use real TLS security by default.
The client now checks certificates and performs hostname verification.
Tests check certificates, so the server uses a cert that the client
trusts.

Only the client portion of SslContextFactory was previously used.
Applications that want to ignore certificates (i.e., for testing) can
use io.netty.handler.ssl.util.InsecureTrustManagerFactory instead.

The MOE configuration was already failing to work, and so required the
simple mapping for examples in addition to what was needed for the new
certs.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80434148
2015-01-08 14:43:11 -08:00
nathanmittler d7f78773ea Allow stream state HALF_CLOSED_REMOTE in the gRPC Java client.
This is allowed in the gRPC v2 spec since trailers may be sent early to
convery error information.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80387029
2015-01-08 14:43:10 -08:00
zhangkun 054f595eaa Separate the default event loop groups for client and server, to prevent
contention between client and server in the same test process.
Name the threads of default thread pools.

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80326782
2015-01-08 14:43:10 -08:00
nathanmittler 5d953e840b Updating gRPC code to use the latest Netty with application-level flow control support.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80230204
2015-01-08 14:43:09 -08:00
lryan 669724a588 Rationalize the XXXStream classes to reduce code smear and make Netty/OkHTTP behave more consistently.
More cleanups to follow
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79574504
2015-01-08 14:43:06 -08:00
ejona 1553aabb2f Advertise h2-15 in ALPN negotiation.
We continue to support h2-14 to prevent having a flag-day. Flag-day
is unnecessary since h2-15 is pretty much the same as h2-14.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79369997
2015-01-08 14:43:05 -08:00
zhangkun a71d887661 Make all transport factories package-private in favor of channel builders.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79367593
2015-01-08 14:43:04 -08:00
zhangkun 7dc48e2787 SharedResourceHolder is a utility that manages expensive resources (thread
pools etc) that are shared among channels and servers as default values when
the application doesn't provide its own.

It uses reference counting to shut down resources with a delay.

Changed the channel and server builders to use it for default values of
executors and event loop groups.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78629204
2015-01-08 14:43:01 -08:00
nathanmittler 29cbef1f9b Renaming gRPC-java "newtransport" package to just "transport".
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78596663
2015-01-08 14:43:00 -08:00