Commit Graph

204 Commits

Author SHA1 Message Date
Kun Zhang f681b5f8be Move Marshaller into MethodDescriptor 2015-08-11 10:53:38 -07:00
Kun Zhang d2929cd1a3 Reorganize packages.
Reserve io.grpc for public API only, and all internal stuff in core to
io.grpc.internal, including the non-stable transport API.

Raise the netty/okhttp/inprocess subpackages one level up to io.grpc,
because they are public API and entry points for most users.

Details:

- Rename io.grpc.transport to io.grpc.internal;
- Move SharedResourceHolder and SerializingExecutor to io.grpc.internal
- Rename io.grpc.transport.{netty|okhttp|inprocess} to
  io.grpc.{netty|okhttp|inprocess}
2015-08-10 15:04:29 -07:00
Eric Anderson be965b86ab Remove extraneous shutdown condition in NettyClientTransport
We didn't do the extraneous check in notifyTerminated()...
2015-08-10 13:40:45 -07:00
Eric Anderson 6236968d4b netty: Provide useful information if connect fails
Previously you would get an error saying, "Transport failed during
protocol negotiation" and no further information.
2015-08-10 13:24:04 -07:00
Eric Anderson 2addeae2db NettyClientHandler should handle all exceptionCaught()s
If NettyClientHandler doesn't then the exception will propagate to the
end of the pipeline, get logged, and cause any open calls to hang.
2015-08-10 13:24:03 -07:00
Eric Anderson 849ed1b995 netty: Simplify ping cancellation
ChannelInactive should be called in all cases of channel going down, so
we only need to cancel ping there. Use goAwayStatus for the error, since
we will be putting the most effort into making that status useful.
2015-08-10 13:24:00 -07:00
Eric Anderson 017cdd2ae9 netty: Remove connectionError and just use goAwayStatus
When connectionError was set, goAwayStatus was also set, so we shouldn't
lose any errors.

NettyClientTransport doesn't really need a Throwable, it just needs a
Status. Passing a Status out of NettyClientHandler reduces the number of
places that need to do transport-specific translation of Throwables into
Status codes.
2015-08-10 13:23:27 -07:00
Eric Anderson e1c348c24a netty: Don't double-process ssl handshake failures
As described in SslHandler's documentation, handshakeFuture() and
SslHandshakeCompletionEvent are equivalent forms of learning of
handshake completion. Watching both causes double-logging and serves no
purpose.
2015-08-10 12:38:47 -07:00
Eric Anderson 8d38d03c2f netty: If negotiation fails, continue failing new writes
Otherwise new writes will be written to the channel and will fail in
some unhelpful way.

Logging was removed as we really want to propagate the failure back to
the application via Calls, which is done by failing the
CreateStreamCommand message. Propagating back to the application via
call removes uncontrollable log spam and is necessary anyway to inform
the application what sort of failure occurred in order to appropriately
to react.
2015-08-10 12:38:46 -07:00
Jack Coughlin 3eaa92bea6 Send an RST_STREAM frame on server deadline 2015-08-10 09:51:54 -07:00
Kun Zhang fc85a4085a Add more documentation for transports.
- Add package descriptions for transport, netty and okhttp.
- Describe transports (netty, okhttp and inprocess) in README
2015-08-06 17:24:42 -07:00
Carl Mastrangelo aebb58b200 Change awaitTerminated to awaitTermination 2015-08-06 13:52:17 -07:00
Xudong Ma 7d1e65c111 Switch ALPN/NPN to advertise only h2 2015-08-05 11:25:05 -07:00
Eric Anderson 41d875c7e3 Notify transportReady() in Netty 2015-08-03 16:50:05 -07:00
Eric Anderson 248f575a59 Make any lib-generated UNKNOWN have description
Note that even more importantly, this translates a RST_STREAM error code
to a gRPC status code. This is generally useful, but also necessary for
DEADLINE_EXCEEDED to be more reliable in 0eae0d9.

Fixes #687
2015-08-03 11:15:02 -07:00
Carl Mastrangelo bd8987af1a Add a status to Transport shutdown 2015-07-29 15:35:38 -07:00
Eric Anderson 26d77ecd2e Minor readability changes
Improved some consistency. writeHeaders was the only non-final
implementation method of ServerStream, even though it is really no
different than the others.
2015-07-23 10:57:00 -07:00
Carl Mastrangelo 0003e44886 Add simple server timeout support
Reintroduce throws

Add timeoutExecutor shutdown

Use a default future

Move timeout cancellation

Cancel the timeout in error cases
2015-07-15 09:33:16 -07:00
Jack Coughlin 3e26b993ce Enforce request deadline
Use a ScheduledExecutorService in the ChannelImpl to terminate the
request by closing the ClientStream with status DEADLINE_EXCEEDED
2015-07-14 16:47:45 -07:00
Kun Zhang 73acc73dbf Remove Method and switch its users to MethodDescriptor.
Resolves #511.

- In generated code, make CONFIG private and METHOD_* fields public.
  METHOD_* fields are MethodDescriptors now, users of the CONFIG field
  should switch to using the METHOD_* fields.
- Move MethodType into MethodDescriptor (#529).
- Unify the fully qualified method name. It is fully qualified service
  name + slash + short method name. It doesn't have the leading slash.
- HandlerRegistry switches the key from short method name to fully
  qualified method name.
2015-07-09 09:29:03 -07:00
nmittler a6c51e7e14 Fixing some compiler warnings. 2015-07-08 16:10:54 -07:00
nmittler 8c1d38a0d8 Adding default User-Agent for netty and okhttp. 2015-07-08 15:56:54 -07:00
nmittler efbb65522b Simplifying flow control window config for Netty.
Fixes #494
2015-07-08 15:43:03 -07:00
Kun Zhang d3c5b00827 Add CallOptions.
- Pass CallOptions to Channel.newCall() and
  ClientInterceptor.interceptCall().
- Remove timeout from AbstractStub.StubConfigBuilder and add deadline,
  which is stored in a CallOptions inside the stub.
- Deadline is in nanoseconds in the clock defined by System.nanoTime().
  It is converted to timeout before transmitting on the wire. Fail the
  call with DEADLINE_EXCEEDED if it's already expired.
2015-07-07 14:28:38 -07:00
Eric Anderson c7403127ea Revert swapping to the "canonical HTTP mapping"
The mapping is poorly suited for gRPC. C and Go don't even do any
mapping. We can improve the mapping in the future, but it is very
important that users don't start depending on the current mapping.

This change is "inspired by" the original code, but is even more
conservative.

Fixes #477
2015-06-25 12:15:34 -07:00
nmittler ae2b5bfda7 Exposing AbstractBufferingHandler so it can be used by custom protocol negotiators. 2015-06-24 13:59:33 -07:00
Jack Coughlin 77878a04ee Pass timeout header in ChannelImpl 2015-06-17 16:03:13 -07:00
nmittler 4322a43824 Throw error for Netty when Jetty ALPN not configured. 2015-06-17 14:19:55 -07:00
nmittler 9a6ded523a Cleanup of NettyClientTransportTest. 2015-06-16 13:11:03 -07:00
Eric Anderson 424c7e64c3 Avoid GCM in unit tests
GCM is very slow, and doesn't provide any benefit in unit tests. Even if
we were using tcnative and GCM is fast, using more available ciphers in
tests still makes sense. With this change building with Java 7 works
again, although that isn't the reason for the change.

On my machine with parallel building, it cuts full build time from
92 seconds to 39 seconds. For an incremental build after only changing
an interop test, the build time is cut from 73 seconds to 15 seconds.
2015-06-16 12:36:57 -07:00
nmittler cb486e461d Testing that buffered streams clean up properly upon disconnect. 2015-06-15 13:10:21 -07:00
nmittler 491aad2659 Changing Netty handlers to properly set initialSettings. 2015-06-11 15:06:20 -07:00
Josh Humphries 0d03f89467 add ping 2015-06-10 12:20:08 -04:00
nmittler 4ee2a6584a Allowing Netty TLS bootstrap handler to be sharable.
Fixes #504
2015-06-03 09:42:08 -07:00
Xudong Ma ffcc5f29c0 Remove unused imports.
to make CheckStyle happy.
2015-05-28 16:10:20 -07:00
nmittler 59f6f45cc4 Switching to Netty's ALPN support.
Also adding documentation for using gRPC with TLS-ALPN within Jetty.

Fixes #180
2015-05-28 13:04:10 -07:00
Louis Ryan 641fc288fc Add support for indeterminate length messages. This will make using GRPC easier for non-proto payload types.
Sync to head
2015-05-26 15:44:46 -07:00
Xudong Ma 6affc8dcd5 upgrade okhttp to 2.3 2015-05-22 16:45:13 -07:00
nmittler 65769c26b3 Making client-side negotiation more pluggable. 2015-05-20 14:37:16 -07:00
Eric Anderson 98c6355079 Disable Netty eventloop graceful termination
When shutting down the Netty event loop, we have already guaranteed that
all users of it are no longer running. Doing a shutdownGracefully is
just delaying graceful JVM termination by two seconds. This is very
noticeable for short-lived processes, like our integration tests.

We would actually also prefer to shutdown quickly and get a
RejectedExecutionException for any newly queued tasks, because that
would be a legitimate bug.

shutdown() is deprecated, thus we do shutdownGracefully with a timeout
of 0.
2015-05-15 18:19:57 -07:00
Louis Ryan d54911713a Implement writes to the channel using a dedicated write queue which allows for efficient flush
coalescing
2015-05-15 09:20:13 -07:00
nmittler 8f537e3ec6 updates to test native epoll 2015-05-14 15:33:55 -07:00
nmittler 5efbb6fe73 Allow 100 streams initially rather than 10.
The HTTP/2 spec suggests 100 as the recommended minimum for SETTINGS_MAX_CONCURRENT_STREAMS (https://tools.ietf.org/html/draft-ietf-httpbis-http2-17#section-6.5.2).  We should use this value as our default.
2015-05-13 14:22:46 -07:00
Eric Anderson 1787106cc7 isReady() should return false until stream allocated
isReady() can provide pushback while the call is in progress, but it
can also provide the pushback necessary when the client creates more
streams than permitted by MAX_CONCURRENT_STREAMS.

As part of this commit, OkHttp is now calling onReady() after call
creation (it previously never called onReady()).
2015-05-12 16:41:50 -07:00
nmittler c3fa600eae Fixing compiler warning. 2015-05-12 16:23:39 -07:00
nmittler d5727c7fcd Deferring stream creation until receiving SETTINGS from server.
Additionally:
- Fixed bug where the decoder was given the incorrect encoder.
- Adding proper logging class for client/server.
2015-05-12 15:46:32 -07:00
nmittler b6407c4a10 Only complete graceful shutdown after buffered streams complete.
Upgrading to to the latest Netty as well.
2015-05-07 08:36:22 -07:00
nmittler b5443833d9 Fixing build issue. 2015-05-07 07:50:33 -07:00
nmittler d8d8ebd783 Adding logging to NettyClientHandler. 2015-05-06 14:38:22 -07:00
nmittler 64df428e36 Properly removing buffered streams after receiving goAway 2015-05-06 14:24:29 -07:00
nmittler 7f73189e56 Always checking MAX_CONCURRENT_STREAMS in
BufferingHttp2ConnectionEncoder.

Currently we don't check this setting when handling a streamClosed()
event. If the setting has lowered prior to this event, the stream
creation could fail.
2015-05-05 19:29:34 -07:00
nmittler a45e0a4767 Cleaning up closing for Netty client/server. 2015-05-05 15:43:22 -07:00
nmittler ef6a2f02d4 Updating to the latest Netty version.
Also re-enabling the gracefulShutdown test now that Netty has been
fixed.
2015-05-05 15:39:21 -07:00
Eric Anderson 10fb20650d Produce combined JavaDoc, add links, exclude internals 2015-05-05 11:32:15 -07:00
Eric Anderson c8c478ef81 netty: Remove goAwayStatus when client closing
Client closing doesn't really many anything special, since it is still
fully-operational. We don't want a later goAwayStatus() from getting
squelched because we were shutting down.
2015-05-05 08:38:38 -07:00
nmittler 68cba971af Do not fail pending streams when sending GOAWAY.
If the HEADERS have been written to the buffering encoder we should
allow their creation to complete since the HTTP/2 spec does not prohibit
this.
2015-05-04 13:00:11 -07:00
nmittler 64176d5560 Adding outbound flow control for Netty. 2015-05-04 12:19:57 -07:00
Jakob Buchgraber 1af367c786 Remove all blocking from the NettyClientTransport. Fixes #116
Motivation:

We are currently blocking in NettyClientTransport.newStream(...) until the channel is active and/or the TLS Handshake is complete.
In certain cases this may lead to deadlock of the eventloop, see #116 for details.

Modifications:

Remove all blocking by buffering writes until the channel is ready to receive those i.e. it is active, TLS is negotiated or the HTTP to HTTP/2 upgrade was sucessfull.

Result:

No more blocking parts when using Netty on the client side.
2015-05-04 11:50:53 -07:00
Eric Anderson b6eb9d763d Don't close streams when sending GOAWAY
This reverts a change introduced in f920bad which caused all streams to
be closed when sending GOAWAY as part of graceful shutdown. This is
because lastKnownId() returns -1 when a GOAWAY has not been received.

The test doesn't pass, but at least appears to revert to the old
behavior. It is unknown if the test fails to pass due to client or
server, but given reports that the old code was working, we are thinking
that server-side GOAWAY handling is what is preventing the test from
passing.
2015-05-01 20:05:18 -07:00
Louis Ryan f2cba89e48 Update Netty to 9d70cf3 to pick up https://github.com/netty/netty/commit/8271c8a which eliminates
explicit flushing from Nettys HTTP2 codec.
2015-05-01 13:00:04 -07:00
Eric Anderson e05885d2aa Swap to Netty's SslContextBuilder
We provide a utility to configure the SslContext for our usage, which we
can change as necessary.
2015-05-01 08:42:12 -07:00
nmittler 7779b4fddf Cleaning up some warnings. 2015-04-30 13:57:38 -07:00
Louis Ryan d8d7908109 Avoid flushing CreateStreamCommand for calls where the client is known to send a payload immediately afterward.
Added simple JMH benchmark for Netty so improvements can be measured
2015-04-29 13:41:15 -07:00
nmittler 29c43c1c2d Upgrading to the latest Netty version. 2015-04-28 10:31:38 -07:00
Louis Ryan f679edcfcc Fix memeory leak in MessageFramer caused by allocation of 'empty' buffers never being released.
The buffers are not empty in reality as the allocators enforce a minimum size
2015-04-28 10:10:00 -07:00
Jakob Buchgraber 3a4f4b5a06 Add support for setting the connection and stream flow control window to NettyServerBuilder.
- Updated related comments in NettyServerBuilder and NettyChannelBuilder.
- Fixed related tests in NettyChannelBuilder.
2015-04-24 10:46:50 -07:00
Xudong Ma 2f4aa5a490 Use Status "UNKNOWN" instead of "INTERNAL" when Status.fromThrowable can not find a cause with a status. 2015-04-24 10:52:27 +08:00
Louis Ryan f7010f2eac Use message size as a hint to the allocator for buffer capacity.
Set upper and lower bounds for Netty & OkHttp allocators based on transport limitations and benchmark results.
Fix OkHttp OutboundFlowController to chunk payloads larger than frameWriter.maxDataLength
Allow OkHttp to allocate buffers to FrameWriter larger than max DATA length
2015-04-23 16:58:43 -07:00
nmittler 3ecbd0456a Upgrading to latest Netty version.
This is to resolve the issue of receiving frames for closed (and
therefore missing) streams.
2015-04-22 10:46:07 -07:00
Eric Anderson fc3e41674b Propagate explicit flushes through MessageFramer
MessageFramer allows queing of data and explicit flushing. Sinks
generally can benefit from knowing when they are required to flush, so
we now tell them when MessageFramer received a flush so they only have
to flush when required.
2015-04-21 10:29:08 -07:00
Eric Anderson 23aac9e9ec Fix netty closure check
During Service removal a condition was inverted but incompletely, which
caused the Netty server to never shutdown.
2015-04-16 12:29:24 -07:00
Eric Anderson 4f4f8e40bf Remove Guava's Service from server transport
ServerImpl.start() now throws IOException to make the error explicit.
This was previously being papered over by wrapping the exception in
RuntimeException.
2015-04-16 11:28:04 -07:00
nmittler f920badc5e Upgrading to the latest Netty version. 2015-04-16 08:42:56 -07:00
Xudong Ma 883eb62de4 OkHttp: Temporally support multiple h2-xx protocol on client side.
Since the user provided SSLSocketFactory (especially in Android) may already do the handshake when creates the SSLSocket, and it may choose a different protocol name other than the one OkHttp is using.

Resolves #293
2015-04-16 09:04:19 +08:00
Eric Anderson 5b2e336b6f netty: Add option to set MAX_CONCURRENT_STREAMS 2015-04-13 13:46:36 -07:00
Eric Anderson 1e257c440e Remove unused local variable 2015-04-10 08:58:43 -07:00
Jakob Buchgraber e8afa3ca23 Fix bug where the stream id would not get incremented for buffered streams. 2015-04-09 13:04:39 -07:00
Xudong Ma 9d214637e6 Temporally support Http protocol name "h2-16" for OkHttp client.
Fixes #22 for real.

Manually tested with jdk8 by:
./run-test-server.sh
./run-test-client.sh --server_host_override=foo.test.google.fr --use_test_ca=true --use_okhttp=true
2015-04-09 13:50:18 +08:00
Eric Anderson fd7ba566aa netty: Use the bootstrap ClassLoader for ALPN/NPN
If ALPN.class isn't in the bootclasspath, we don't want to find it as it
won't work. Also, if someone has fiddled with ClassLoaders, we really
want to make sure we get the proper ALPN class.

Passing "null" to Class.forName() means "bootstrap class loader". In
fact, ClassLoader.getParent() says, "Some implementations may use null
to represent the bootstrap class loader."

We must load the Proxy in the bootstrap class loader as well, because
our class loader may not have access to ALPN.{Client,Server}Provider,
for the same reasons as above. Even if we have multiple instances of
gRPC (due to class loaders), combined they will only create two Proxy
classes: one for ALPN.ClientProvider and one for ALPN.ServerProvider.
2015-04-08 15:37:44 -07:00
Jakob Buchgraber 44f9904ab0 Replace 'internal' import with correct one.
The errornous import was introduced by my previous commit 5f70a12cfc
2015-04-08 12:54:30 -07:00
Xudong Ma f3ccdd99f1 Fixes Travis breakage 2015-04-08 13:00:09 +08:00
Jakob Buchgraber 5f70a12cfc Remove blocking on client stream creation and buffer RPC Calls for when the MAX_CONCURRENT_STREAMS limit is hit. Fixes #118 2015-04-07 17:06:21 -07:00
Jakob Buchgraber ce860f9003 fix compile error introduced by previous commit 2015-04-07 13:00:24 -07:00
Jakob Buchgraber 105964bfac Fix memory leak by adding the Http2StreamRemovalPolicy to the channel pipeline.
In benchmarks we would see grpc talking up tens of gigabytes of memory. A heap dump
revealed that streams would not get cleaned up and stick around in memory forever.
2015-04-07 12:56:27 -07:00
Xudong Ma 9aae6f65fa checkstyle change:
1. Adds <property name="separateLineBetweenGroups" value="true"/> to CustomImportOrder to enfore blank line between imports groups.
2. Uses checkstyle 6.5, which fixed a bug of "CustomImportOrder checks import sorting according to ASCII order instead of case-insensitive alphabetical order".
2015-04-07 08:27:48 +08:00
Eric Anderson 74c9b5ed02 Fix JavaDoc references to non-imported class
The ServerImpl import was removed because it wasn't used as far as
checkstyle was able to determine. However, it was being used to resolve
JavaDoc references. Instead of re-adding the import just make the
reference fully qualified to prevent the two systems from continuing to
disagree on whether it is needed.
2015-04-03 15:11:51 -07:00
Xudong Ma 2cad9e6000 Add "unsed import" check. 2015-04-03 16:55:01 +08:00
nmittler 11d0094e04 Upgrading to the latest Netty 4.1 branch. 2015-03-31 14:14:29 -07:00
Eric Anderson 9bd31daee6 Wait for handler registration 2015-03-26 12:35:31 -07:00
Eric Anderson e515c772cd netty: Status should be based on GOAWAY code
goingAway() is called before onGoAwayRead() in Netty:
b7f57223c1/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2ConnectionDecoder.java (L521)

The test before checked that the stream went away, but not that the
GOAWAY code influenced our Status, as UNAVAILABLE is the default
internally.

The UNAVAILABLE default has also been changed to include a message so
that we can determine where the Status came from in case it is triggered
again in the future.
2015-03-25 11:04:52 -07:00
nmittler 6407c18578 Default client connection window to 1MiB. 2015-03-24 10:29:20 -07:00
nmittler ab9f82756a Making connection and stream windows configurable for Netty. 2015-03-24 10:14:55 -07:00
Eric Anderson c3e8dae6ce Add checkstyle checking
The checkstyle.xml is a slightly modified version of the upstream Google
checkstyle configuration. All changes have comment describing them.

Lots of warnings were corrected. Examples is the only project that has
warnings still, as the necessary changes require some thought.
2015-03-16 10:53:13 -07:00
Jakob Buchgraber 54c31b0325 Fix compile error introduced by previous commit 2015-03-13 15:12:40 -07:00
Louis Ryan 3f7325ecf5 Allow switching between Netty 4 & 5 by restricting to intersection of interfaces 2015-03-13 14:57:55 -07:00
Jakob Buchgraber 0076243063 Add WritableBuffer interface for zero copy data writes. Fixes #8
WritableBuffer is a generic interface that allows to transfer data
from gRPC directly to the native transport's buffer implementation.
2015-03-13 13:00:44 -07:00
Louis Ryan 1216de6262 Add support to Netty builders for other channel types. Demonstrate and test use of this with local channels 2015-03-12 16:05:33 -07:00
nmittler d4d66a5fef Better error description onGoAwayRead 2015-03-12 08:16:52 -07:00
Jakob Buchgraber f822416f35 Use DEFAULT_WORKER_EVENT_LOOP_GROUP for both client and server. Fixes #82
+ Use only one thread for the boss eventloop group by default.
2015-03-11 14:31:51 -07:00
nmittler 2a425090d1 Migrating to Netty 4.1
Fixes #164
2015-03-11 13:18:27 -07:00