Commit Graph

323 Commits

Author SHA1 Message Date
Carl Mastrangelo 88945e00f3 Make ServerImpl Use the common Scheduled Executor Service 2015-08-03 08:53:30 -07:00
Carl Mastrangelo 89db769d2d Rename sendPayload to sendMessage 2015-07-31 15:28:06 -07:00
Carl Mastrangelo d2b1b37ed7 Add a transport ready for use with retry 2015-07-31 15:24:44 -07:00
Carl Mastrangelo 14e774130d Rename onPayload to onMessage 2015-07-31 15:11:18 -07:00
Eric Anderson 8fe667676c Remove serializable distinction for Metadata
This does make use of the fact we are no longer using Multimap. Doing
entries() for ArrayListMultimap must create a new Map.Entry for every
entry. Since we are now using HashMap, we are able to use entries() with
no extra cost.

Merging particular Keys no longer needs to deserialize.
2015-07-30 14:09:21 -07:00
Eric Anderson a2292faf43 Optimize Metadata for reduced allocation count
Multimap creates at least one new object for every access, because all
the objects it returns are "live" and when mutated they need to update
the multimap's size. Many common operations thus require at least an
object allocation per key.

Note that previously remove() was non-functional as it removed the wrong
type from the multimap. The type system did not catch this because
remove() is passed an Object for all collection types.

The return type of removeAll() was changed to Iterable to prevent the
need of converting to Key if the caller doesn't consume the return
value.

Although it appears serialize() is now more expensive in terms of
allocations because it first accumulates into an ArrayList, the memory
usage is approximately the same since Multimap.values() makes an
Iterator for each key. The new code would allocate fewer bytes overall
and in fewer allocations, but the older code retained less memory while
processing. If we want to optimize serialize() we can track the number
of entries without needing to do any wrapping like Multimap does. I
didn't bother because the ArrayList is a fraction of the cost compared
to actually serializing the values.
2015-07-30 14:08:48 -07:00
Eric Anderson 50e28c2ca0 Make docs more strict for byte[] methods in Metadata 2015-07-30 14:07:04 -07:00
Eric Anderson 5698ccaee6 Fix invalid data sharing in Metadata.merge() 2015-07-30 14:07:03 -07:00
Carl Mastrangelo bd8987af1a Add a status to Transport shutdown 2015-07-29 15:35:38 -07:00
Carl Mastrangelo 4d2b3e3d06 Remove Intrinsic locking in ChannelImpl. 2015-07-28 07:16:03 -07:00
Carl Mastrangelo 6f7c5143f1 Stop using intrinsic locks in ServerImpl 2015-07-27 15:40:03 -07:00
Eric Anderson eba12fb514 Add missing generics to Context internals 2015-07-23 13:25:39 -07:00
Eric Anderson afdbe19937 Minor fixes/improvements
Cancel was the only method for implementing ClientStream that was not
final, even though it isn't really any different from the other methods.
2015-07-23 11:13:50 -07:00
Eric Anderson 5abe321b81 Fix deframing error handling
CANCELLED is certainly not the right status code. Communicating the
exception to the client removes the need for logging, which also makes
it more clear which call experienced the problem.
2015-07-23 11:13:48 -07:00
Eric Anderson 26d77ecd2e Minor readability changes
Improved some consistency. writeHeaders was the only non-final
implementation method of ServerStream, even though it is really no
different than the others.
2015-07-23 10:57:00 -07:00
Carl Mastrangelo 9a5c733ce9 Move CallImpl 2015-07-23 09:33:48 -07:00
Eric Anderson d11e9be127 Add in-process transport
Resolves #518
2015-07-22 15:54:37 -07:00
Eric Anderson 0df3d5e72a Add ClientInterceptors "inside" ChannelImpl
Fixes #538
2015-07-22 15:49:06 -07:00
Eric Anderson 3ce15b50b2 Reverse interceptor execution order
The previous order was unintuitive as the following would execute in the
reverse order:

Channel channel;
channel = ClientInterceptors.intercept(channel, interceptor1,
                                                interceptor2);
// vs
channel = ClientInterceptors.intercept(channel, interceptor1);
channel = ClientInterceptors.intercept(channel, interceptor2);

After this change, they have equivalent behavior. With this change,
there are no more per-invocation allocations and so calling 'next' twice
is no longer prohibited.

Resolves #570
2015-07-22 15:45:58 -07:00
Carl Mastrangelo 7b63909caf removed unused field 2015-07-22 15:22:01 -07:00
Carl Mastrangelo fe21e2799a Added copyright 2015-07-22 15:20:30 -07:00
Carl Mastrangelo 2eaeacafe8 Added basic unit test and reorg how onready calls are made
Forgot to add this last file

updated method name

Remove unused function

Remove helper function for threshold edge detection

Remove helper function for threshold edge detection

Re make listener abstract
2015-07-22 14:45:23 -07:00
Eric Anderson f81ed8e87b Clarify/improve transport interface requirements 2015-07-22 09:30:45 -07:00
Eric Anderson 7d22c09b2c Add Call state-checking enforcement to ChannelImpl and ServerImpl
The CallImpls in ChannelImpl and ServerImpl implement the Call
interfaces; they should be the ones ensuring that inappropriate calling
of methods is handled as the interface describes.
2015-07-22 09:30:44 -07:00
Eric Anderson dff29759a7 Fix reentrancy bug in ServerImpl during shutdown
Previously if the transport shut down immediately and during the
shutdown() call, the ServerImpl would never become terminated.
2015-07-22 09:30:42 -07:00
Eric Anderson 0eae0d9264 Make DEADLINE_EXCEEDED more reliable
The client can race with the server in cancelling due to deadline. If
server cancels we don't get DEADLINE_EXCEEDED, so double-check on
client-side to reduce the chances of losing the race.

Generally we expect the client to lose the race because of coarse timer
granularity for timer expirary. This change does little to help if the
server's clock runs noticably "fast" relative to the client.
2015-07-22 09:30:41 -07:00
Eric Anderson f48bc3b3eb Specify locale for toLowerCase in Metadata 2015-07-21 16:56:00 -07:00
Louis Ryan f6121165f9 Implementation of context API to allow for propagation of a limited amount of state through the
call stack and across thread boundaries.

Strongly modeled after the Go context API https://blog.golang.org/context with support for
- cancellation propagation & cancellation listeners
- typed value binding
- timeout/deadline

The major difference with Go is that ThreadLocal is used for propagation instead of parameter
passing as this is considered more idiomatic for Java.
2015-07-21 10:22:59 -07:00
Kun Zhang a6585e36ed Replace DeferredInputStream with interface Drainable.
- Rename flushTo() to drainTo().
- Remove flushTo() from DeferredNanoProtoInputStream (which is renamed
  to NanoProtoInputStream), because the optimization is not implemented.
- Rename DeferredProtoInputStream to ProtoInputStream.

 #529
2015-07-20 17:04:49 -07:00
Carl Mastrangelo 0003e44886 Add simple server timeout support
Reintroduce throws

Add timeoutExecutor shutdown

Use a default future

Move timeout cancellation

Cancel the timeout in error cases
2015-07-15 09:33:16 -07:00
Jack Coughlin 3e26b993ce Enforce request deadline
Use a ScheduledExecutorService in the ChannelImpl to terminate the
request by closing the ClientStream with status DEADLINE_EXCEEDED
2015-07-14 16:47:45 -07:00
Carl Mastrangelo 3ee5b5a752 Try to simplify server method definition 2015-07-09 14:06:47 -07:00
Kun Zhang 73acc73dbf Remove Method and switch its users to MethodDescriptor.
Resolves #511.

- In generated code, make CONFIG private and METHOD_* fields public.
  METHOD_* fields are MethodDescriptors now, users of the CONFIG field
  should switch to using the METHOD_* fields.
- Move MethodType into MethodDescriptor (#529).
- Unify the fully qualified method name. It is fully qualified service
  name + slash + short method name. It doesn't have the leading slash.
- HandlerRegistry switches the key from short method name to fully
  qualified method name.
2015-07-09 09:29:03 -07:00
nmittler a6c51e7e14 Fixing some compiler warnings. 2015-07-08 16:10:54 -07:00
nmittler 8c1d38a0d8 Adding default User-Agent for netty and okhttp. 2015-07-08 15:56:54 -07:00
Eric Anderson 35ff624eb2 Metadata.Key's name is non-null; don't check for null
There is a Preconditions.checkNotNull guaranteeing name is not null.
2015-07-08 15:10:02 -07:00
Eric Anderson d27cbc8aa3 Improve docs to describe close as being last method called
This isn't changing any of the semantics we already had, but more
informing users of the guarantees we provide.
2015-07-08 15:08:26 -07:00
Kun Zhang 7fff088e19 Fix a javadoc warning 2015-07-07 16:02:47 -07:00
Kun Zhang d3c5b00827 Add CallOptions.
- Pass CallOptions to Channel.newCall() and
  ClientInterceptor.interceptCall().
- Remove timeout from AbstractStub.StubConfigBuilder and add deadline,
  which is stored in a CallOptions inside the stub.
- Deadline is in nanoseconds in the clock defined by System.nanoTime().
  It is converted to timeout before transmitting on the wire. Fail the
  call with DEADLINE_EXCEEDED if it's already expired.
2015-07-07 14:28:38 -07:00
Eric Anderson 30455fd2f4 Remove unused variables 2015-07-06 16:27:22 -07:00
Eric Anderson a479c9116a Move timeout marshaller tests to ChannelImplTest
Now that ChannelImplTest exists, it makes more sense for them to be
there, since the implementation is in ChannelImpl.
2015-06-26 12:19:42 -07:00
Eric Anderson 45da9c5766 Add a few ChannelImpl tests and improve error status 2015-06-26 12:19:41 -07:00
Eric Anderson 9a2db9d7d9 Make ChannelImpl.obtainActiveTransport's fast path lock-free
Resolves #479
2015-06-26 12:19:41 -07:00
Eric Anderson c7403127ea Revert swapping to the "canonical HTTP mapping"
The mapping is poorly suited for gRPC. C and Go don't even do any
mapping. We can improve the mapping in the future, but it is very
important that users don't start depending on the current mapping.

This change is "inspired by" the original code, but is even more
conservative.

Fixes #477
2015-06-25 12:15:34 -07:00
Jack Coughlin 77878a04ee Pass timeout header in ChannelImpl 2015-06-17 16:03:13 -07:00
David Connelly 15104cdc69 Make sure we enter terminated state when transport server has been shut down and all server transports have been closed. Previously, we had only been checking if server transports were closed. Also, make sure termination cleanup is performed at most once with an AssertionError if checkForTermination() called when server has already been terminated 2015-06-15 13:04:11 -07:00
Josh Humphries 0d03f89467 add ping 2015-06-10 12:20:08 -04:00
Kun Zhang 2ee4d0228d Rename Call to ClientCalls.
Other classes are already following the convention that ClientFoo for
client-side, and ServerFoo for server-side. Call has been the black
sheep of the family.

- Call -> ClientCall
- Calls -> ClientCalls
- ForwardingCall* -> ForwardingClientCall*
2015-06-04 16:39:25 -07:00
Louis Ryan 641fc288fc Add support for indeterminate length messages. This will make using GRPC easier for non-proto payload types.
Sync to head
2015-05-26 15:44:46 -07:00
Xudong Ma 0d480879e5 okhttp: outbound flow control.
Fixes #371
2015-05-21 23:05:54 -07:00
Eric Anderson eb85d697c7 Set likely final size of array in toHttp2Headers
We know the size won't be more than serializedHeaders.length, is
unlikely to be fewer, and very unlikely to be substantially fewer. This
prevents us from needing to resize the array in all cases.

Resolves #321
2015-05-19 09:58:17 -07:00
Eric Anderson b69c59ce5f Decrease ArrayDeque memory in SerializingExecutor
By default, ArrayDeque will be of size 16, which is an overkill for most
calls. "4" is not a magical number itself, but seems a better guess than
16 since we do have some knowledge of how much it will contain.

Resolves #320
2015-05-19 09:55:25 -07:00
Eric Anderson 1787106cc7 isReady() should return false until stream allocated
isReady() can provide pushback while the call is in progress, but it
can also provide the pushback necessary when the client creates more
streams than permitted by MAX_CONCURRENT_STREAMS.

As part of this commit, OkHttp is now calling onReady() after call
creation (it previously never called onReady()).
2015-05-12 16:41:50 -07:00
nmittler 518b7dbf7c Slight performance improvment for MutableHandlerRegistryImpl 2015-05-12 09:56:45 -07:00
Eric Anderson 3462eb0e72 Handle OkHttp throwing exception during start()
Ideally OKHttp wouldn't do blocking I/O during start(), but it does and
fixing it is non-trivial. OkHttp can either throw an exception when it
encounters an error during start or it can shut itself down. Both
require changes in ChannelImpl, so we just choose to keep OkHttp's
current behavior and deal with it in ChannelImpl.
2015-05-07 18:52:21 -07:00
Eric Anderson 4e82a11311 Make Channel/Server abstract classes
Abstract classes allow us greater ability to change them over time. They
previously were interfaces to allow us to use Guava's AbstractService.
2015-05-07 16:41:52 -07:00
nmittler 1fa11cea1f Adding default implementation of onReady in Call and ServerCall 2015-05-07 10:52:07 -07:00
nmittler 64176d5560 Adding outbound flow control for Netty. 2015-05-04 12:19:57 -07:00
Louis Ryan f2cba89e48 Update Netty to 9d70cf3 to pick up https://github.com/netty/netty/commit/8271c8a which eliminates
explicit flushing from Nettys HTTP2 codec.
2015-05-01 13:00:04 -07:00
Eric Anderson 4a05869db7 Replace Operation*Exception with Status*Exception
Operation is a term no longer used in gRPC. StatusException seems clear
and is concise. Moved out of Status class to remove stuttering.

The return types of as*Exception() is now explicitly the
Status.*Exception type.
2015-05-01 08:43:37 -07:00
nmittler 7779b4fddf Cleaning up some warnings. 2015-04-30 13:57:38 -07:00
Eric Anderson 2b33598ec4 Don't overwrite error message if bad headers
If we notice something wrong with the headers, then we choose a nice
error message. But we are accidentally overwriting that message with
additional error details.
2015-04-30 09:59:24 -07:00
Louis Ryan d8d7908109 Avoid flushing CreateStreamCommand for calls where the client is known to send a payload immediately afterward.
Added simple JMH benchmark for Netty so improvements can be measured
2015-04-29 13:41:15 -07:00
Eric Anderson 2fb03b07dd Fix style violations added in 986d221
The unused AtomicInteger is caught by checkstyle so causes Travis-CI
to fail.
2015-04-29 08:49:28 -07:00
Louis Ryan 986d221eaa Fix issue where close can be called in a reentrant fashion by framer writing causing exception to be thrown by transport 2015-04-28 12:46:48 -07:00
Louis Ryan f679edcfcc Fix memeory leak in MessageFramer caused by allocation of 'empty' buffers never being released.
The buffers are not empty in reality as the allocators enforce a minimum size
2015-04-28 10:10:00 -07:00
Xudong Ma d0aad72441 okhttp: Clean up stream when error happens.
Resolves #279
2015-04-27 14:07:43 -07:00
Xudong Ma 2f4aa5a490 Use Status "UNKNOWN" instead of "INTERNAL" when Status.fromThrowable can not find a cause with a status. 2015-04-24 10:52:27 +08:00
Louis Ryan f7010f2eac Use message size as a hint to the allocator for buffer capacity.
Set upper and lower bounds for Netty & OkHttp allocators based on transport limitations and benchmark results.
Fix OkHttp OutboundFlowController to chunk payloads larger than frameWriter.maxDataLength
Allow OkHttp to allocate buffers to FrameWriter larger than max DATA length
2015-04-23 16:58:43 -07:00
Eric Anderson fc3e41674b Propagate explicit flushes through MessageFramer
MessageFramer allows queing of data and explicit flushing. Sinks
generally can benefit from knowing when they are required to flush, so
we now tell them when MessageFramer received a flush so they only have
to flush when required.
2015-04-21 10:29:08 -07:00
Eric Anderson 4f4f8e40bf Remove Guava's Service from server transport
ServerImpl.start() now throws IOException to make the error explicit.
This was previously being papered over by wrapping the exception in
RuntimeException.
2015-04-16 11:28:04 -07:00
nmittler f920badc5e Upgrading to the latest Netty version. 2015-04-16 08:42:56 -07:00
Eric Anderson e23f899491 Split protobuf into its own project
We don't want core to depend on protobuf.
2015-04-10 15:50:56 -07:00
Eric Anderson d077290c40 Fix warnings (JavaDoc and [deprecated]) 2015-04-09 18:05:31 -07:00
Kun Zhang 0d89d3d1bc Fix the issue where the intecepting call fails in start(), does not
call super.start(), and makes the subsequent use of other methods on the
call throw IllegalStateException.

Create ClientInterceptors.CheckedForwardingCall that handles exception
in start logic.

Create Forwarding[Server]Call[Listener] for generic decoration use
cases, with an abstract delegate() for flexibility.

Create SimpleForwarding[Server]Call[Listener] to replace now deprecated
forwarding classes.
2015-04-09 15:56:03 -07:00
Xudong Ma 9aae6f65fa checkstyle change:
1. Adds <property name="separateLineBetweenGroups" value="true"/> to CustomImportOrder to enfore blank line between imports groups.
2. Uses checkstyle 6.5, which fixed a bug of "CustomImportOrder checks import sorting according to ASCII order instead of case-insensitive alphabetical order".
2015-04-07 08:27:48 +08:00
Eric Anderson 45682dbc18 Hold lock while reading isThreadScheduled
isThreadScheduled is @GuardedBy("internalLock").
2015-04-06 14:13:49 -07:00
Eric Anderson d70d91b5d6 Improve JavaDoc for Status 2015-04-03 16:07:46 -07:00
Xudong Ma 2cad9e6000 Add "unsed import" check. 2015-04-03 16:55:01 +08:00
zhangkun83 966e120098 Add precondition to throw more informative exception when calling request() before start() has been called. 2015-03-26 10:54:59 -07:00
Thomas ten Cate 9175d9d79e Fix data corruption issue receiving payloads > 2kB
Also add OkHttpReadableBuffer unit test, and increase the test string
length to expose the bug.

Also use array equality assertions in the base test, instead of
comparisons whose return value is discarded.

Fixes #231.
2015-03-22 13:14:57 +01:00
Eric Anderson a6edc294fc Improve Status exception message
When the description is null, the exception message would be in the form
of "INTERNAL: null", which isn't very attractive, and makes it seem like
an error in itself. If the description is null, just use "INTERNAL".
2015-03-20 10:29:11 -07:00
Eric Anderson 78cde0ddc0 Move Status.toString next to other methods 2015-03-20 09:47:34 -07:00
nmittler 23972a25b9 Adding outbound flow control API for the transport API 2015-03-16 14:17:03 -07:00
nmittler 9f5614501d Adding outbound flow control API for Call/ServerCall 2015-03-16 12:59:15 -07:00
Eric Anderson 040007ec17 Add package statement to HttpUtilTest 2015-03-16 12:38:07 -07:00
Eric Anderson c3e8dae6ce Add checkstyle checking
The checkstyle.xml is a slightly modified version of the upstream Google
checkstyle configuration. All changes have comment describing them.

Lots of warnings were corrected. Examples is the only project that has
warnings still, as the necessary changes require some thought.
2015-03-16 10:53:13 -07:00
Jakob Buchgraber 0076243063 Add WritableBuffer interface for zero copy data writes. Fixes #8
WritableBuffer is a generic interface that allows to transfer data
from gRPC directly to the native transport's buffer implementation.
2015-03-13 13:00:44 -07:00
Eric Anderson 76d0955a6e Clean up warnings
-Xlint:-options is not available on some earlier JDK 7s, but won't fail
if unsupported. It prevents the warning wanting bootclasspath specified
since target/source is 1.6.
2015-03-12 17:37:32 -07:00
Eric Anderson acfe3def6e Stall MessageDeframer pro-actively
This allows sooner delivery of errors. We never needed to stop delivery
for unexpected EOS, but instead the application would have been required
to request() another message before delivering. Stalling MessageDeframer
sooner removes the need for the application to request another message
before noticing that the buffers are empty.
2015-03-12 11:28:19 -07:00
Eric Anderson 238a4cb620 Add two extra cases to make edge case testing more obvious. 2015-03-11 15:14:28 -07:00
Eric Anderson 8986fc3412 Reorganize tests. Tweak Javadoc formatting 2015-03-11 15:10:35 -07:00
Eric Anderson 0e904f08e0 Tweak Http2Error and add tests 2015-03-11 14:45:03 -07:00
nmittler 2a425090d1 Migrating to Netty 4.1
Fixes #164
2015-03-11 13:18:27 -07:00
Jorge Canizales ad5ae259bb Fixes javadoc errors 2015-03-10 09:24:06 -07:00
Nathan Mittler f1b6f623c5 Merge pull request #166 from nmittler/doclint
Tightening up error message for GO_AWAY.
2015-03-09 13:49:03 -07:00
nmittler dfcfb7bca1 Tightening up error message for GO_AWAY.
The attempt here is to identify all causes of GO_AWAY and to ensure
there is a reasonable description to help understand the cause.

Fixes #163
2015-03-09 13:48:15 -07:00
nmittler 4deff027ba Proper buffer closure when receiving DATA with EOS
The Http2ClientStream should not close the buffer in this case since
it's already been given to the deframer and potentially to the user.

Added cleanup code to MessageDeframer and AbstractClientStream to make
sure that we free the Buffer when appropriate.
2015-03-09 13:28:32 -07:00
nmittler 732cfc0d55 Disable Javadoc doclint on Java 8
It breaks the build.
2015-03-03 07:59:13 -08:00
zhangkun83 ef87818758 Polish javadoc for transport/ 2015-03-02 11:10:29 -08:00