After debugging #2153, it would have been nice to know what the exact
parameter was that was null. This change adds a name for each
checkNotNull (and tries to normalized on static imports in order to
shorten lines)
780b2696 caused all failures for blocked unary stubs to have a
StatusRuntimeException as the cause of the StatusRuntimeException, with
the two exceptions having almost the same status.
It was withCallCredentials on the stub to avoid confusion with channel
credentials (which don't exist in Java at this time, but do in other
languages), and having the method names be different doesn't add value.
onReady/isReady previously could disagree causing a sort of deadlock
where the application isn't sending because grpc said not to, but won't
be informed to send via onReady later.
This is a stack trace from inprocessTransportOutboundFlowControl. The
line numbers are from this commit but with the changes to DelayedStream
disabled:
at io.grpc.internal.DelayedStream.isReady(DelayedStream.java:306)
(That is isReady returning false because fallThrough == false)
at io.grpc.internal.ClientCallImpl.isReady(ClientCallImpl.java:382)
at io.grpc.stub.ClientCalls$CallToStreamObserverAdapter.isReady(ClientCalls.java:289)
at io.grpc.stub.ClientCallsTest$8$1.run(ClientCallsTest.java:403)
(And yet that was the onReady callback, and it won't be called again)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onReady(ClientCalls.java:377)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$4.runInContext(ClientCallImpl.java:481)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:52)
at io.grpc.internal.SerializeReentrantCallsDirectExecutor.execute(SerializeReentrantCallsDirectExecutor.java:65)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.onReady(ClientCallImpl.java:478)
at io.grpc.internal.DelayedStream$DelayedStreamListener.onReady(DelayedStream.java:366)
at io.grpc.inprocess.InProcessTransport$InProcessStream$InProcessServerStream.request(InProcessTransport.java:284)
at io.grpc.internal.ServerCallImpl.request(ServerCallImpl.java:99)
at io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.request(ServerCalls.java:345)
at io.grpc.stub.ClientCallsTest.inprocessTransportOutboundFlowControl(ClientCallsTest.java:432)
Fixes#1932
This reverts commit 3df1446deb.
The commit was adding to the difficulty of integration for testing. By
itself it isn't bad, so this is a temporary revert until the many other
commits are absorbed and then it will be reapplied.
This does have a manual edit for ClientCallsTest.
The test has found a legitimate flake caused by a race in DelayedStream
related to onReady. Disable it until the fix arrives to prevent CI
failure noise.
If the timoutout expires, the test will almost certainly fail now due to
responses not matching expected responses, but it isn't as obvious why
the test fails. Swapping to ArrayList prints the entire list when they
don't match, which also makes it easier to diagnose.
This is, in part, to help diagnose #1932
Introduce CallCredentials as a first-class option to allow applications
to set per-call credentials into headers for outgoing RPCs. This will
supersede ClientAuthInterceptor. It has access to more
information (e.g., transport attributes, MethodDescriptor) and allow
results to be returned asynchronously, e.g., from a blocking I/O, which
was problemantic with ClientAuthInterceptor.
This change updates the behavior of the core compression semantics. Previously,
if the codec was "identity", nothing was set on the wire. This is allowed by
the spec, but doesn't match what wrapped languages do.
Additionally, the interop tests will now attempt to honor the requested
compression.
Resolves#1221
Add ClientCall.cancel(String, Throwable) and deprecate
ClientCall.cancel(). Will delete cancel() after all known third-party
overriders have switched to overriding the new one.
- Made CallOptions use the Deadline type instead of a
long to represent a deadline.
- Added new methods CallOptions.withDeadline(Deadline) and
AbstractStub.withDeadline(Deadline). The methods are
marked experimental, as the Deadline class is marked
experimental. These methods are meant to replace
CallOptions.withDeadlineNanoTime(Long) and
AbstractStub.withDeadlineNanoTime(Long), which have
been deprecated.
- Updated CallOptions.toString() to include all fields.
while still allowing stubs to be used.
Currently these APIs are only useful on the server side, a separate proposal will be made for adding client support
An example of usage is in TestServiceImpl.streamingOutputCallManualFlowControl which listens for flow-control 'onReady' events and produces messages to the client.
Added some unit testing for ServerCalls
Although the changes were determined automatically, they were manually
applied to the codebase.
ClientCalls actually has a bug fix, since the suggestion to add
interrupt() made it obvious that interrupted() was inappropriate.
This was already being done in one case, but should have been done for
the other occurrances of InterruptedException. Before the
RuntimeException is just asking to be a bug since application code will
commonly only catch StatusRuntimeException.
This reduces the necessary number of threads in the application executor
and provides a small improvement in latency (~15μs, which is normally in
the noise, but would be a 5% improvement).
Benchmark (direct) (transport) Mode Cnt Score Error Units
Before:
TransportBenchmark.unaryCall1024 true INPROCESS avgt 10 1566.168 ± 13.677 ns/op
TransportBenchmark.unaryCall1024 false INPROCESS avgt 10 35769.532 ± 2358.967 ns/op
After:
TransportBenchmark.unaryCall1024 true INPROCESS avgt 10 1813.778 ± 19.995 ns/op
TransportBenchmark.unaryCall1024 false INPROCESS avgt 10 18568.223 ± 1679.306 ns/op
The benchmark results are exactly what we would expect, assuming that
half of the benefit of direct is on server and half on client:
1566 + (35769 - 1566) / 2 = 18668 ns --vs-- 18568 ns
It is expected that direct=true would get worse, because
SerializingExecutor is now used instead of
SerializeReentrantCallsDirectExecutor plus the additional cost of
ThreadlessExecutor.
In the future we could try to detect the ThreadlessExecutor and ellide
Serializ*Executor completely (as is possible for any single-threaded
executor). We could also optimize the queue used in ThreadlessExecutor
to be single-producer, single-consumer. I don't expect to do those
optimizations soon, however.
ServerCall already had "headers must be sent before any messages, which
must be sent before closing," but the implementation did not enforce it
and our async server handler didn't obey.
The benefit of forcing sending headers first is that it removes the only
implicit call in our API and interceptors dealing just with metadata
don't need to override sendMessage. The implicit behavior was bug-prone
since it wasn't obvious you were forgetting that headers may not be
sent.
This makes the reconfiguration code more concise.
- Remove configureNewStub().
- Add mutation methods withDeadlineNanoTime(), withChannel() etc that
returns the reconfigured stub.
- Remove blockingClientStreamingCall() which is not used, and we don't
actually want that API.
- Rename duplexStreamingCall() to asyncDuplexStreamingCall() to align
with other async methods.
- In unary call and client streaming call, do not request for additional
response after the first response.
This gives us more flexibility in API changes in the future.
Unary call and server streaming call should call the flow-control method
call.request() only once. Previously it was called whenever a request
arrives, which is wrong. Now it's fixed.
Resolves#436
Resolves#511.
- In generated code, make CONFIG private and METHOD_* fields public.
METHOD_* fields are MethodDescriptors now, users of the CONFIG field
should switch to using the METHOD_* fields.
- Move MethodType into MethodDescriptor (#529).
- Unify the fully qualified method name. It is fully qualified service
name + slash + short method name. It doesn't have the leading slash.
- HandlerRegistry switches the key from short method name to fully
qualified method name.
- Pass CallOptions to Channel.newCall() and
ClientInterceptor.interceptCall().
- Remove timeout from AbstractStub.StubConfigBuilder and add deadline,
which is stored in a CallOptions inside the stub.
- Deadline is in nanoseconds in the clock defined by System.nanoTime().
It is converted to timeout before transmitting on the wire. Fail the
call with DEADLINE_EXCEEDED if it's already expired.
10 years exceeds the maximum for some systems at the moment. Change to 1
year to workaround such systems while they get changed. And since "have
the default be a large number instead of not present" is a temporary fix
anyway.
Delaying creating the exception produces a much more useful exception.
The status code, description, and exception type are all identical. The
only difference in the backtrace.
Stubs don't have any timeout. However, MethodDescriptor does and
requires a timeout. We really want "no timeout," which is infinite, but
we use 10 years as the next best thing. CallOptions will end up fixing
this hack as MethodDescriptor will no longer have timeout.
Before, the hard-coded 1s didn't matter, because nobody was observing
it. Since 77878a0 it is now being sent to servers, and some servers
enforce it. Oops.
Previously, it always threw a RuntimeExecutionException with a
StatusRuntimeException within (since the only callers of setException()
provide a StatusRuntimeException).
Resolves#507
Other classes are already following the convention that ClientFoo for
client-side, and ServerFoo for server-side. Call has been the black
sheep of the family.
- Call -> ClientCall
- Calls -> ClientCalls
- ForwardingCall* -> ForwardingClientCall*
call super.start(), and makes the subsequent use of other methods on the
call throw IllegalStateException.
Create ClientInterceptors.CheckedForwardingCall that handles exception
in start logic.
Create Forwarding[Server]Call[Listener] for generic decoration use
cases, with an abstract delegate() for flexibility.
Create SimpleForwarding[Server]Call[Listener] to replace now deprecated
forwarding classes.
The checkstyle.xml is a slightly modified version of the upstream Google
checkstyle configuration. All changes have comment describing them.
Lots of warnings were corrected. Examples is the only project that has
warnings still, as the necessary changes require some thought.
Summary of changes:
1) Merged the interfaces MessageDeframer2.Sink and DeframerListener into
MessageDeframer2.Listener. This simplifies the interface of
MessageDeframer2 quite a bit.
2) Added a deliveryPaused() handler to MessageDeframer2.Listener, which
is called by the deframer when there is not enough data to read/deliver
the next message.
3) Modified AbstractStream and AbstractClientStream to manage the timing
of when the closed() event is delivered to the listener. The
transportReportStatus ultimately controls this by creating a task to
close the listener. It either runs this task immediately or when the
next deliveryPaused() event occurs.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83620903
The TestService proto is temporarily supplied as a generated jar (until the open source protoc compiler supports grpc).
Copies of messages.proto, empty.proto, and message_set.proto are scrubbed and included in the source under integration-testing.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78711468
overview:
1) Lots of @SuppressWarnings :)
2) Remove dependencies on StandardCharsets.XXX (which is Java 7)
3) Moved testing/utils/ssl/* to .../stubby/util so that the netty transport doesn't depend directly on the testing module.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78460727
Change from /Service.method to /Service/method.
quote from the spec:
Method (required): a path identifying the operation to perform on the
specified host, conforming to RFC 3986 sec 3.3. E.g
“/calendar.v1/ListEvents”
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=78288091
Fixes some propagation issues for trailers too
Adds some more testing for metadata exchange
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=77979593
This CL also:
- Removes the OkHTTP server implementation
- Switches NanoTest and Http2OkHttpTest to use Netty server. These tests are currently @Suppressed as OkHttp is not yet draft 14 compliant. Simon is fixing
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=76998151
- Creates and passes a transport instance to ServerListener.transportCreated().
- Keeps the "/" prefix of the fully qualified method name when passing it to the handler
registry.
- Adds necessary "this." when accessing a member variable in ServerCalls.
- BlockingResponseStream.buffer should be added with BlockingResponseStream.this as
as the mark of end of data.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=76922440
ServerServiceDefinition that connects the server with application-provided
service implementation.
Introduces a class Method, that holds the invariables of a method that are
inferred from the proto file, to be reused for creating MethodDescriptor
for client code and ServerServiceDefinition for server code in the generated
class.
Adds ServerCalls that contains the functionalities that is used by the
bindService() method in the generated class. It minimizes the logic in the
generated code.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=76741797
- Remove transport.proto and move status codes into Status.java with a little refactoring to make
status easier & more precise to use
- Move DeferredProtoInputStream into a proto subpackage
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=76392172
Remove 'context' from interfaces/tests/framers/...
Next phases
- Switch the wire format (ESF needs to be done in near lock-step)
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=75164642
Introduces Header and uses it for propagating text-only header values over existing transports
Leaves Context & wire format otherwise unchanged
Next phases
- Remove context from interfaces
- Switch the wire format (ESF needs to be done in near lock-step)
Interface changes are relatively light
Headers class is functional but not optimal
All serialization is done as string until transports expose interface for binary headers
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=75050265
The primary functionality is an interceptor that authenticates calls using oauth. However, most of the changes are for an integration test to make sure the code works with ESF.
This is based on work by lryan and simonma.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=72567773
transports that can deliver them.
This unblocks sending OAuth2 headers natively
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=71118741
Session is now (properly) implementing transport API, so ChannelImpl has some
testing.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=69074094
Call does not accept InputStream length because Marshaller makes it too hard
for the Stub to determine the length.
SessionCall is updated to implement changes to Call, but it is replaced in the
next CL.
You can see what glue is necessary to join the Transport and Stub APIs. The
ugliest pain is dealing with flow control callback. Some of that code is
unrelated and necessary because the Channel uses a separate thread to run
Call.Listener. Other parts will be necessary because Channel must intervene
during Transport callbacks to handle retry logic.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=69020275
It was an oversight that newstub was not in third_party when committed.
Renaming of package was preformed via a script.
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=68416211
Carl Mastrangelo
ZHANG Dapeng
Eric Anderson
nmittler
Louis Ryan
Kun Zhang
elandau
Lukasz Strzalkowski
Łukasz Strzałkowski
Steven Parkes
buchgr
Xudong Ma
yangg