Commit Graph

262 Commits

Author SHA1 Message Date
Eric Anderson 2b33598ec4 Don't overwrite error message if bad headers
If we notice something wrong with the headers, then we choose a nice
error message. But we are accidentally overwriting that message with
additional error details.
2015-04-30 09:59:24 -07:00
Louis Ryan d8d7908109 Avoid flushing CreateStreamCommand for calls where the client is known to send a payload immediately afterward.
Added simple JMH benchmark for Netty so improvements can be measured
2015-04-29 13:41:15 -07:00
Eric Anderson 2fb03b07dd Fix style violations added in 986d221
The unused AtomicInteger is caught by checkstyle so causes Travis-CI
to fail.
2015-04-29 08:49:28 -07:00
Louis Ryan 986d221eaa Fix issue where close can be called in a reentrant fashion by framer writing causing exception to be thrown by transport 2015-04-28 12:46:48 -07:00
Louis Ryan f679edcfcc Fix memeory leak in MessageFramer caused by allocation of 'empty' buffers never being released.
The buffers are not empty in reality as the allocators enforce a minimum size
2015-04-28 10:10:00 -07:00
Xudong Ma d0aad72441 okhttp: Clean up stream when error happens.
Resolves #279
2015-04-27 14:07:43 -07:00
Xudong Ma 2f4aa5a490 Use Status "UNKNOWN" instead of "INTERNAL" when Status.fromThrowable can not find a cause with a status. 2015-04-24 10:52:27 +08:00
Louis Ryan f7010f2eac Use message size as a hint to the allocator for buffer capacity.
Set upper and lower bounds for Netty & OkHttp allocators based on transport limitations and benchmark results.
Fix OkHttp OutboundFlowController to chunk payloads larger than frameWriter.maxDataLength
Allow OkHttp to allocate buffers to FrameWriter larger than max DATA length
2015-04-23 16:58:43 -07:00
Eric Anderson fc3e41674b Propagate explicit flushes through MessageFramer
MessageFramer allows queing of data and explicit flushing. Sinks
generally can benefit from knowing when they are required to flush, so
we now tell them when MessageFramer received a flush so they only have
to flush when required.
2015-04-21 10:29:08 -07:00
Eric Anderson 4f4f8e40bf Remove Guava's Service from server transport
ServerImpl.start() now throws IOException to make the error explicit.
This was previously being papered over by wrapping the exception in
RuntimeException.
2015-04-16 11:28:04 -07:00
nmittler f920badc5e Upgrading to the latest Netty version. 2015-04-16 08:42:56 -07:00
Eric Anderson e23f899491 Split protobuf into its own project
We don't want core to depend on protobuf.
2015-04-10 15:50:56 -07:00
Eric Anderson d077290c40 Fix warnings (JavaDoc and [deprecated]) 2015-04-09 18:05:31 -07:00
Kun Zhang 0d89d3d1bc Fix the issue where the intecepting call fails in start(), does not
call super.start(), and makes the subsequent use of other methods on the
call throw IllegalStateException.

Create ClientInterceptors.CheckedForwardingCall that handles exception
in start logic.

Create Forwarding[Server]Call[Listener] for generic decoration use
cases, with an abstract delegate() for flexibility.

Create SimpleForwarding[Server]Call[Listener] to replace now deprecated
forwarding classes.
2015-04-09 15:56:03 -07:00
Xudong Ma 9aae6f65fa checkstyle change:
1. Adds <property name="separateLineBetweenGroups" value="true"/> to CustomImportOrder to enfore blank line between imports groups.
2. Uses checkstyle 6.5, which fixed a bug of "CustomImportOrder checks import sorting according to ASCII order instead of case-insensitive alphabetical order".
2015-04-07 08:27:48 +08:00
Eric Anderson 45682dbc18 Hold lock while reading isThreadScheduled
isThreadScheduled is @GuardedBy("internalLock").
2015-04-06 14:13:49 -07:00
Eric Anderson d70d91b5d6 Improve JavaDoc for Status 2015-04-03 16:07:46 -07:00
Xudong Ma 2cad9e6000 Add "unsed import" check. 2015-04-03 16:55:01 +08:00
zhangkun83 966e120098 Add precondition to throw more informative exception when calling request() before start() has been called. 2015-03-26 10:54:59 -07:00
Thomas ten Cate 9175d9d79e Fix data corruption issue receiving payloads > 2kB
Also add OkHttpReadableBuffer unit test, and increase the test string
length to expose the bug.

Also use array equality assertions in the base test, instead of
comparisons whose return value is discarded.

Fixes #231.
2015-03-22 13:14:57 +01:00
Eric Anderson a6edc294fc Improve Status exception message
When the description is null, the exception message would be in the form
of "INTERNAL: null", which isn't very attractive, and makes it seem like
an error in itself. If the description is null, just use "INTERNAL".
2015-03-20 10:29:11 -07:00
Eric Anderson 78cde0ddc0 Move Status.toString next to other methods 2015-03-20 09:47:34 -07:00
nmittler 23972a25b9 Adding outbound flow control API for the transport API 2015-03-16 14:17:03 -07:00
nmittler 9f5614501d Adding outbound flow control API for Call/ServerCall 2015-03-16 12:59:15 -07:00
Eric Anderson 040007ec17 Add package statement to HttpUtilTest 2015-03-16 12:38:07 -07:00
Eric Anderson c3e8dae6ce Add checkstyle checking
The checkstyle.xml is a slightly modified version of the upstream Google
checkstyle configuration. All changes have comment describing them.

Lots of warnings were corrected. Examples is the only project that has
warnings still, as the necessary changes require some thought.
2015-03-16 10:53:13 -07:00
Jakob Buchgraber 0076243063 Add WritableBuffer interface for zero copy data writes. Fixes #8
WritableBuffer is a generic interface that allows to transfer data
from gRPC directly to the native transport's buffer implementation.
2015-03-13 13:00:44 -07:00
Eric Anderson 76d0955a6e Clean up warnings
-Xlint:-options is not available on some earlier JDK 7s, but won't fail
if unsupported. It prevents the warning wanting bootclasspath specified
since target/source is 1.6.
2015-03-12 17:37:32 -07:00
Eric Anderson acfe3def6e Stall MessageDeframer pro-actively
This allows sooner delivery of errors. We never needed to stop delivery
for unexpected EOS, but instead the application would have been required
to request() another message before delivering. Stalling MessageDeframer
sooner removes the need for the application to request another message
before noticing that the buffers are empty.
2015-03-12 11:28:19 -07:00
Eric Anderson 238a4cb620 Add two extra cases to make edge case testing more obvious. 2015-03-11 15:14:28 -07:00
Eric Anderson 8986fc3412 Reorganize tests. Tweak Javadoc formatting 2015-03-11 15:10:35 -07:00
Eric Anderson 0e904f08e0 Tweak Http2Error and add tests 2015-03-11 14:45:03 -07:00
nmittler 2a425090d1 Migrating to Netty 4.1
Fixes #164
2015-03-11 13:18:27 -07:00
Jorge Canizales ad5ae259bb Fixes javadoc errors 2015-03-10 09:24:06 -07:00
Nathan Mittler f1b6f623c5 Merge pull request #166 from nmittler/doclint
Tightening up error message for GO_AWAY.
2015-03-09 13:49:03 -07:00
nmittler dfcfb7bca1 Tightening up error message for GO_AWAY.
The attempt here is to identify all causes of GO_AWAY and to ensure
there is a reasonable description to help understand the cause.

Fixes #163
2015-03-09 13:48:15 -07:00
nmittler 4deff027ba Proper buffer closure when receiving DATA with EOS
The Http2ClientStream should not close the buffer in this case since
it's already been given to the deframer and potentially to the user.

Added cleanup code to MessageDeframer and AbstractClientStream to make
sure that we free the Buffer when appropriate.
2015-03-09 13:28:32 -07:00
nmittler 732cfc0d55 Disable Javadoc doclint on Java 8
It breaks the build.
2015-03-03 07:59:13 -08:00
zhangkun83 ef87818758 Polish javadoc for transport/ 2015-03-02 11:10:29 -08:00
Louis Ryan 46dd47f86b Cleanup Javadoc for Channel, ServerCall and their related classes. 2015-02-25 14:22:12 -08:00
Louis Ryan 425e3c0b7f Remove dead GrpcFramingUtil class 2015-02-23 17:44:23 -08:00
Louis Ryan 95c6f5082c Udate TODOs to use Github user names 2015-02-23 12:46:28 -08:00
Eric Anderson ee6c9d7b52 Add scary JavaDoc to Metadata.setAuthority() 2015-02-20 08:54:08 -08:00
Eric Anderson dbf7d529bd Make sure to close InputStream even if cancelled 2015-02-20 08:52:51 -08:00
Jakob Buchgraber 07d0917b94 Add ServerImpl#awaitTerminated() method. Fixes #80 2015-02-18 16:00:34 -08:00
Eric Anderson e26608fe91 Avoid MoreObjects for "old guava" compatibility
Some early grpc users on Android are using a very old Guava. They are
working on upgrading, but it will take time. This alone is not enough
for "old guava" compilibility; expect more to come.
2015-02-13 17:06:46 -08:00
Eric Anderson 74f231a6dd Remove Guava Service from ClientTransport
OkHttp no longer cancels all calls on shutdown, as we want to allow
graceful shutdown. Such cancelling behavior will likely be provided by
Channel in the future.
2015-02-12 15:20:22 -08:00
Eric Anderson db0d8ad8c4 Only run terminationRunnable once 2015-02-12 15:17:12 -08:00
nabeelmian d2e35b0d02 Added utility class, ForwardingServerListener, that forwards calls to
underlying ServerCall.Listener.

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=85910420
2015-02-09 13:09:19 -08:00
Eric Anderson 8aa79b39fc Improve thread safety of newStream()
OkHttp should now have a thread-safe implementation of newStream.
Previously 'lock' was not held when checking goAway and the checking in
AbstractClientTransport was redundant.

Netty was thread-safe, but it was very hard to tell what guarantees were
necessary and what guarantees each piece was providing.
2015-02-06 14:12:01 -08:00
Eric Anderson f3a90cd42b netty: Cancel stream if interrupted during create
Previously streams were being partially orphaned if there was an
interruption during stream creation. To handle cancellation,
AbstractClientStream's cancel() had to be changed remove the
"optimization" otherwise, again, the stream would be orphaned.
2015-02-04 15:46:27 -08:00
nmittler d0e883ac20 Fixing integration tests
Allowing MessageDeframer.request to be called after the deframer has
been closed.  The stub helpers blindly call request after receiving each
message.
2015-01-30 10:15:30 -08:00
nmittler 89b8d7ff47 Moving decompression to the channel thread. 2015-01-29 14:55:40 -08:00
nmittler f83145865a Removing all references to "stubby" 2015-01-27 11:25:25 -08:00
Eric Anderson aeeebb7cdb Remove Service API from ServerImpl
Fixes #21
2015-01-27 09:20:24 -08:00
nmittler 19052499f7 Removing Maven build 2015-01-27 08:57:37 -08:00
nmittler 02c953e5e0 Migrating run scripts to gradle. 2015-01-26 15:26:11 -08:00
Eric Anderson d0adadbc1f Use proto interface instead of concrete class 2015-01-26 10:23:05 -08:00
Eric Anderson ada32b0cb1 Optimize number of DATA frames for unary requests
Resolves #10
2015-01-22 16:45:20 -08:00
nmittler de3a13164f Changing gRPC Java inbound flow control model
The goal is to mirror the token-based approach used by the Reactive
Streams API.
2015-01-22 11:46:16 -08:00
ejona 4de2026492 Fix TODO attribution
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=84172357
2015-01-16 16:23:05 -08:00
simonma f589a4acbb Remove traces of gRPC v1 protocol vs gRPC v2 protocol.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=84087425
2015-01-16 11:03:38 -08:00
ejona 7235a396b8 Remove Service API from ChannelImpl
This change loses asynchronous notification of channel state-change and
a way to wait until the channel is actually connected. Both of these are
expected to be added back as part of a health API. The important
distinction from Service is that ChannelImpl never permanently fails and
can revert from being started to connecting again.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83875407
2015-01-15 13:42:49 -08:00
ejona 198a756b83 Add toString to Headers and Trailers
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83856622
2015-01-15 13:42:46 -08:00
simonma 828f941e99 Remove dependency on guava Lists for initializing list.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83808289
2015-01-15 13:42:43 -08:00
ejona 3a29c98d40 Don't overwrite transportError in DATA processing
There may already be a transportError, in which case we want to augment
it and not overwrite it.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83807747
2015-01-15 13:37:38 -08:00
nathanmittler 11c363a7ae Disallowing message delivery after status has been delivered to the listener.
Summary of changes:

1) Merged the interfaces MessageDeframer2.Sink and DeframerListener into
MessageDeframer2.Listener. This simplifies the interface of
MessageDeframer2 quite a bit.

2) Added a deliveryPaused() handler to MessageDeframer2.Listener, which
is called by the deframer when there is not enough data to read/deliver
the next message.

3) Modified AbstractStream and AbstractClientStream to manage the timing
of when the closed() event is delivered to the listener. The
transportReportStatus ultimately controls this by creating a task to
close the listener. It either runs this task immediately or when the
next deliveryPaused() event occurs.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83620903
2015-01-09 17:00:25 -08:00
yangg f8524a17b8 Add package name to the method string.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83353207
2015-01-08 14:43:24 -08:00
ejona c0f41920bd Remove gRPC v1 support.
No major refactorings/simplifications were done. Only gRPC v1 support
infrastructure was removed.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82737436
2015-01-08 14:43:23 -08:00
nathanmittler 164b734aa9 Adding gradle build for Java grpc
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82152044
2015-01-08 14:43:21 -08:00
nathanmittler 66ce6677b2 Updating version of gRPC maven build to 0.1.0-SNAPSHOT.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82151533
2015-01-08 14:43:20 -08:00
ejona fc30031fd1 Change default gRPC protocol to v2.
Any place that force-sets the protocol to 2 or assumes the old value
is now removed. Unfortunately, it seems InProcessTransportTest has
some non-obvious dependency on gRPC v1.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81597524
2015-01-08 14:43:18 -08:00
lryan 56e307fcb6 Add BSD license header to all source files
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81447025
2015-01-08 14:43:17 -08:00
zhangkun 2b116ef2cd Encode binary headers with Base64 on the wire, and requires all binary headers
have "-bin" suffix in their names.

Split Metadata.Marshaller into BinaryMarshaller and AsciiMarshaller.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81306135
2015-01-08 14:43:15 -08:00
zhangkun 63fc64761a Inspect content type from trailers when there are no headers.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81291102
2015-01-08 14:43:14 -08:00
ejona fb09bba456 Don't leak decompressor in gRPC v2.
We don't use the decompressor at all in gRPC v2, but it may still have
resources that should be close()d.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81271990
2015-01-08 14:43:14 -08:00
zhangkun 8375cd00e8 GRPC Java clients will send the "te: trailers" header, and the server will
check for it, so that we can detect intermediate proxies that do not support
trailers.

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81084983
2015-01-08 14:43:13 -08:00
zhangkun 4332c2f56e Change the HTTP code -> GRPC Status mapping to comply with http canonical mapping
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80767939
2015-01-08 14:43:13 -08:00
lryan c5e70c2310 Remove StreamState and use inboundPhase/outboundPhase instead
Remove synchronization on stateLock as we are not required to be thread safe
Add better toString for stream impls
Internal cleanup of various 'status' fields in AbstractClientStream
Remove 'stashTrailers' as we've already extracted status in layer above correctly

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80678356
2015-01-08 14:43:12 -08:00
ejona f96e7e8c6c Use real TLS security by default.
The client now checks certificates and performs hostname verification.
Tests check certificates, so the server uses a cert that the client
trusts.

Only the client portion of SslContextFactory was previously used.
Applications that want to ignore certificates (i.e., for testing) can
use io.netty.handler.ssl.util.InsecureTrustManagerFactory instead.

The MOE configuration was already failing to work, and so required the
simple mapping for examples in addition to what was needed for the new
certs.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80434148
2015-01-08 14:43:11 -08:00
zhangkun 054f595eaa Separate the default event loop groups for client and server, to prevent
contention between client and server in the same test process.
Name the threads of default thread pools.

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80326782
2015-01-08 14:43:10 -08:00
nathanmittler 63271f4729 Fixing a compatibility issue between the GFE and gRPC java client for the V2 protocol.
The GFE should not be sending DATA frames with endOfStream=true.
However to make things work in the short-term, if we receive one we
still need to process the DATA frame.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80243944
2015-01-08 14:43:09 -08:00
nathanmittler 5d953e840b Updating gRPC code to use the latest Netty with application-level flow control support.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=80230204
2015-01-08 14:43:09 -08:00
ejona 01152e093d Fix workaround in place for GFE's lack of trailers.
A recent refactoring moved code so our previous workaround stopped
producing any effect.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79965647
2015-01-08 14:43:08 -08:00
ejona 0f56c47ac8 Handle exceptions thrown by the application
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79895899
2015-01-08 14:43:08 -08:00
lryan 1f2a1869e5 Fix NullPointerException in AbstractServerStream when deframer reports error and fix error message propagation
Improve detail of deframer error reporting to client

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79799152
2015-01-08 14:43:07 -08:00
zhangkun aa0b7cc9af Forbid calling next more than once in interceptors.
Will throw IllegalStateException.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79717262
2015-01-08 14:43:07 -08:00
lryan 4073a8a1a7 Remove redundant field from AbstractClientStream
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79592498
2015-01-08 14:43:07 -08:00
lryan 669724a588 Rationalize the XXXStream classes to reduce code smear and make Netty/OkHTTP behave more consistently.
More cleanups to follow
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79574504
2015-01-08 14:43:06 -08:00
ejona ff669ac9c2 More precise Builder generics
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79430607
2015-01-08 14:43:06 -08:00
zhangkun 776ff86687 Eliminate ForwardingChannel and switch all use cases to client interceptors.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79360027
2015-01-08 14:43:04 -08:00
zhangkun 48e734d7e3 Dedicated ClientInterceptor interface and utilities.
This will replace the channel-wrapping technique as the official
solution of intercepting calls on the client side. It is done in the
same manner as ServerInterceptor.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79280263
2015-01-08 14:43:04 -08:00
ejona 9a148a5022 Remove outbound flow control interface.
We know we don't like the API, but don't yet have the new one in place.
Remove the old one now instead of waiting further, because we are
getting more users who are more likely to try to start using the API.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=79090165
2015-01-08 14:43:03 -08:00
nathanmittler ffc18f6c57 Adding MOE configuration for grpc_java.
The TestService proto is temporarily supplied as a generated jar (until the open source protoc compiler supports grpc).

Copies of messages.proto, empty.proto, and message_set.proto are scrubbed and included in the source under integration-testing.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78711468
2015-01-08 14:43:02 -08:00
zhangkun 7dc48e2787 SharedResourceHolder is a utility that manages expensive resources (thread
pools etc) that are shared among channels and servers as default values when
the application doesn't provide its own.

It uses reference counting to shut down resources with a delay.

Changed the channel and server builders to use it for default values of
executors and event loop groups.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78629204
2015-01-08 14:43:01 -08:00
nathanmittler 29cbef1f9b Renaming gRPC-java "newtransport" package to just "transport".
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78596663
2015-01-08 14:43:00 -08:00
simonma 3c986bbcdb Move protobuf code dependencies to net/stubby/proto
Remove needless proto dependencies from tests

-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78593107
2015-01-08 14:42:59 -08:00
nathanmittler 0304b3d6b2 Cleaning up various compiler warnings in preparation for opensource.
overview:
1) Lots of @SuppressWarnings :)

2) Remove dependencies on StandardCharsets.XXX (which is Java 7)

3) Moved testing/utils/ssl/* to .../stubby/util so that the netty transport doesn't depend directly on the testing module.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78460727
2015-01-08 14:42:59 -08:00
ejona df9867def9 Support early-OK from Server.
The semantics in gRPC changed where server sending OK before client
half-closes is now permitted.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78404390
2015-01-08 14:42:58 -08:00
ejona 0a585333ce Pre-serialize status codes, for efficiency
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78401520
2015-01-08 14:42:58 -08:00