When shutting down the Netty event loop, we have already guaranteed that
all users of it are no longer running. Doing a shutdownGracefully is
just delaying graceful JVM termination by two seconds. This is very
noticeable for short-lived processes, like our integration tests.
We would actually also prefer to shutdown quickly and get a
RejectedExecutionException for any newly queued tasks, because that
would be a legitimate bug.
shutdown() is deprecated, thus we do shutdownGracefully with a timeout
of 0.
isReady() can provide pushback while the call is in progress, but it
can also provide the pushback necessary when the client creates more
streams than permitted by MAX_CONCURRENT_STREAMS.
As part of this commit, OkHttp is now calling onReady() after call
creation (it previously never called onReady()).
BufferingHttp2ConnectionEncoder.
Currently we don't check this setting when handling a streamClosed()
event. If the setting has lowered prior to this event, the stream
creation could fail.
Client closing doesn't really many anything special, since it is still
fully-operational. We don't want a later goAwayStatus() from getting
squelched because we were shutting down.
Motivation:
We are currently blocking in NettyClientTransport.newStream(...) until the channel is active and/or the TLS Handshake is complete.
In certain cases this may lead to deadlock of the eventloop, see #116 for details.
Modifications:
Remove all blocking by buffering writes until the channel is ready to receive those i.e. it is active, TLS is negotiated or the HTTP to HTTP/2 upgrade was sucessfull.
Result:
No more blocking parts when using Netty on the client side.
This reverts a change introduced in f920bad which caused all streams to
be closed when sending GOAWAY as part of graceful shutdown. This is
because lastKnownId() returns -1 when a GOAWAY has not been received.
The test doesn't pass, but at least appears to revert to the old
behavior. It is unknown if the test fails to pass due to client or
server, but given reports that the old code was working, we are thinking
that server-side GOAWAY handling is what is preventing the test from
passing.
Set upper and lower bounds for Netty & OkHttp allocators based on transport limitations and benchmark results.
Fix OkHttp OutboundFlowController to chunk payloads larger than frameWriter.maxDataLength
Allow OkHttp to allocate buffers to FrameWriter larger than max DATA length
MessageFramer allows queing of data and explicit flushing. Sinks
generally can benefit from knowing when they are required to flush, so
we now tell them when MessageFramer received a flush so they only have
to flush when required.
ServerImpl.start() now throws IOException to make the error explicit.
This was previously being papered over by wrapping the exception in
RuntimeException.
Since the user provided SSLSocketFactory (especially in Android) may already do the handshake when creates the SSLSocket, and it may choose a different protocol name other than the one OkHttp is using.
Resolves#293
If ALPN.class isn't in the bootclasspath, we don't want to find it as it
won't work. Also, if someone has fiddled with ClassLoaders, we really
want to make sure we get the proper ALPN class.
Passing "null" to Class.forName() means "bootstrap class loader". In
fact, ClassLoader.getParent() says, "Some implementations may use null
to represent the bootstrap class loader."
We must load the Proxy in the bootstrap class loader as well, because
our class loader may not have access to ALPN.{Client,Server}Provider,
for the same reasons as above. Even if we have multiple instances of
gRPC (due to class loaders), combined they will only create two Proxy
classes: one for ALPN.ClientProvider and one for ALPN.ServerProvider.
In benchmarks we would see grpc talking up tens of gigabytes of memory. A heap dump
revealed that streams would not get cleaned up and stick around in memory forever.
1. Adds <property name="separateLineBetweenGroups" value="true"/> to CustomImportOrder to enfore blank line between imports groups.
2. Uses checkstyle 6.5, which fixed a bug of "CustomImportOrder checks import sorting according to ASCII order instead of case-insensitive alphabetical order".
The ServerImpl import was removed because it wasn't used as far as
checkstyle was able to determine. However, it was being used to resolve
JavaDoc references. Instead of re-adding the import just make the
reference fully qualified to prevent the two systems from continuing to
disagree on whether it is needed.
goingAway() is called before onGoAwayRead() in Netty:
b7f57223c1/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2ConnectionDecoder.java (L521)
The test before checked that the stream went away, but not that the
GOAWAY code influenced our Status, as UNAVAILABLE is the default
internally.
The UNAVAILABLE default has also been changed to include a message so
that we can determine where the Status came from in case it is triggered
again in the future.
The checkstyle.xml is a slightly modified version of the upstream Google
checkstyle configuration. All changes have comment describing them.
Lots of warnings were corrected. Examples is the only project that has
warnings still, as the necessary changes require some thought.
The Http2ClientStream should not close the buffer in this case since
it's already been given to the deframer and potentially to the user.
Added cleanup code to MessageDeframer and AbstractClientStream to make
sure that we free the Buffer when appropriate.
As part of the effort to remove all blocking bits from the NettyClientStream with
this commit the SendGrpcFrameCommand now takes a stream object instead of the
stream id. This will be necessary as in a non blocking world the stream id might
not have yet been allocated when the SendGrpcFrameCommand gets instantiated.
- Renamed 'eventGroup' property to 'group' as this what's used elsewhere.
- Moved assignment of the channel before the listener is added as currently
there is a (theoretical) chance that the listener is executed before the assignment
to channel happens, namely in case the Future is already done when the listener
is added.
- Removed comment that seemed out of place / relict.
As part of the effort to remove all blocking bits from the NettyClientStream with
this commit the SendGrpcFrameCommand now takes a stream object instead of the
stream id. This will be necessary as in a non blocking world the stream id might
not have yet been allocated when the SendGrpcFrameCommand gets instantiated.
OkHttp no longer cancels all calls on shutdown, as we want to allow
graceful shutdown. Such cancelling behavior will likely be provided by
Channel in the future.
If the transport had started but negotiation had not completed then
previously channel would be null and shutdown would have no effect. Now
we set channel eagerly, but use a separate variable for determining if
it is safe for general use.
OkHttp should now have a thread-safe implementation of newStream.
Previously 'lock' was not held when checking goAway and the checking in
AbstractClientTransport was redundant.
Netty was thread-safe, but it was very hard to tell what guarantees were
necessary and what guarantees each piece was providing.
Channel is available immediately after connect(), so register callbacks
immediately instead of delaying.
Setting channel is now delayed until it is actually safe to use.
Previously streams were being partially orphaned if there was an
interruption during stream creation. To handle cancellation,
AbstractClientStream's cancel() had to be changed remove the
"optimization" otherwise, again, the stream would be orphaned.
This change loses asynchronous notification of channel state-change and
a way to wait until the channel is actually connected. Both of these are
expected to be added back as part of a health API. The important
distinction from Service is that ChannelImpl never permanently fails and
can revert from being started to connecting again.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83875407
Summary of changes:
1) Merged the interfaces MessageDeframer2.Sink and DeframerListener into
MessageDeframer2.Listener. This simplifies the interface of
MessageDeframer2 quite a bit.
2) Added a deliveryPaused() handler to MessageDeframer2.Listener, which
is called by the deframer when there is not enough data to read/deliver
the next message.
3) Modified AbstractStream and AbstractClientStream to manage the timing
of when the closed() event is delivered to the listener. The
transportReportStatus ultimately controls this by creating a task to
close the listener. It either runs this task immediately or when the
next deliveryPaused() event occurs.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83620903
No major refactorings/simplifications were done. Only gRPC v1 support
infrastructure was removed.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82737436
Previously the code close the HTTP2 streams prior to notifying the
application layer. This was the wrong order as the code depended on
enumerating the open streams to notify the application layer.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82733459
Any place that force-sets the protocol to 2 or assumes the old value
is now removed. Unfortunately, it seems InProcessTransportTest has
some non-obvious dependency on gRPC v1.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81597524
have "-bin" suffix in their names.
Split Metadata.Marshaller into BinaryMarshaller and AsciiMarshaller.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81306135
check for it, so that we can detect intermediate proxies that do not support
trailers.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81084983
Remove synchronization on stateLock as we are not required to be thread safe
Add better toString for stream impls
Internal cleanup of various 'status' fields in AbstractClientStream
Remove 'stashTrailers' as we've already extracted status in layer above correctly
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80678356
The client now checks certificates and performs hostname verification.
Tests check certificates, so the server uses a cert that the client
trusts.
Only the client portion of SslContextFactory was previously used.
Applications that want to ignore certificates (i.e., for testing) can
use io.netty.handler.ssl.util.InsecureTrustManagerFactory instead.
The MOE configuration was already failing to work, and so required the
simple mapping for examples in addition to what was needed for the new
certs.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80434148
This is allowed in the gRPC v2 spec since trailers may be sent early to
convery error information.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80387029
contention between client and server in the same test process.
Name the threads of default thread pools.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80326782
We continue to support h2-14 to prevent having a flag-day. Flag-day
is unnecessary since h2-15 is pretty much the same as h2-14.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79369997
The TestService proto is temporarily supplied as a generated jar (until the open source protoc compiler supports grpc).
Copies of messages.proto, empty.proto, and message_set.proto are scrubbed and included in the source under integration-testing.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78711468
pools etc) that are shared among channels and servers as default values when
the application doesn't provide its own.
It uses reference counting to shut down resources with a delay.
Changed the channel and server builders to use it for default values of
executors and event loop groups.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78629204
nmittler
Louis Ryan
Xudong Ma
Eric Anderson
Jakob Buchgraber
Adrian Cole
zhangkun