This is lower-level than the existing AbstractTransportTest. It should
be used by all transports, but InProcessTransport is the only one as
part of this commit.
DelayedClientTransport.PendingStream will override cancel(), which has a
clearer semantic.
Also permitting all status codes except OK in ClientStream.cancel(),
instead of just 4 codes.
This PR finishes the refactoring started by #1395. Resolves#1342.
Major changes:
- All interfaces that return `ListenableFuture<T>` now return `T`
- Stop passing `null` for transports after shut down. Pass
`FailingClientTransport` instead. This simplifies the interface
semantics of interfaces that return a transport, as well as their
callers because they no longer need to check for `null`.
- Add two methods to `TransportManager`:
- `createInterimTransport()` takes the place of `BlankFutureProvider`
in `LoadBalancer` implementations.
- `createFailingTransport()` takes the place of errors in the `Future`
when `LoadBalancer` implementations propagate errors to the caller.
- `createInterimTransport()` creates a `DelayedClientTransport` tracked
by `ManagedChannelImpl`, which will not terminate until all
`DelayedClientTransport`s, in addition to the `TransportSet`s, are
terminated.
- Removed the transport argument from the ready and shutdown callbacks,
because if `LoadBalancer` was given a delayed transport earlier, it
will get the real transport's shutdown event, thus won't be able to
match the two through identity comparison, which beats the purpose of
passing the transport.
When running :grcp-interop-testing:test, Travis has been hanging on OS X
or flaking on Linux with:
> Process 'Gradle Test Executor 4' finished with non-zero exit value 137
Exit code 137 indicates SIGKILL (128 + 9) and is most likely caused by
the JVM being killed by the kernel's OOM killer.
The limit in .travis.yml is 2x what was necessary to do a parallel
build. The main test memory limit in build.gradle is well above 16m
which is necessary for the tests. Interop-testing is well above 64m
which is necessary for interop-testing, but we use 1.5g to help prevent
timeouts on Travis.
Protobuf and protobuf-nano each have one tests that decodes a proto >64M
in size, which prevents them from running with less than 512m and 768m,
respectively.
Codecov.io provides patch-based code coverage, so you can easily know
how many of the added lines are covered. It also has a more useful UI.
Unfortunately, the percentage it reports does not include partially-
covered lines--those with uncovered conditions. Thus the reported
percentage is about 6% lower than the coverage we've been looking at
previously. Because of this alone, I don't expect to remove coveralls
support soon.
Use the bash script instead of python module since pip isn't available
by default on Travis OS X.
jacocoTestReport uses mustRunAfter (contrary to the docs; see
https://issues.gradle.org/browse/GRADLE-2960) to make sure it runs after
all tests, only if testing is taking place. We would like
:grpc-all:jacocoTestReport to behave the same way. Without it, we would
need two separate invocations of gradle (adding ~1m to Travis run) in
order to prevent getting random results depending on what tests just so
happened to have been run.
Although the changes were determined automatically, they were manually
applied to the codebase.
ClientCalls actually has a bug fix, since the suggestion to add
interrupt() made it obvious that interrupted() was inappropriate.
This was already being done in one case, but should have been done for
the other occurrances of InterruptedException. Before the
RuntimeException is just asking to be a bug since application code will
commonly only catch StatusRuntimeException.
Always return a completed future from `TransportSet`. If a (real) transport has not been created (e.g., in reconnect back-off), a `DelayedClientTransport` will be returned.
Eventually we will get rid of the transport futures everywhere, and have streams always __owned__ by some transports.
DelayedClientTransport
----------------------
After we get rid of the transport future, this is what `ClientCallImpl` and `LoadBalancer` get when a real transport has not been created yet. It buffers new streams and pings until `setTransport()` is called, after which point all buffered and future streams/pings are transferred to the real transport.
If a buffered stream is cancelled, `DelayedClientTransport` will remove it from the buffer list, thus #1342 will be resolved after the larger refactoring is complete.
This PR only makes `TransportSet` use `DelayedClientTransport`. Follow-up changes will be made to allow `LoadBalancer.pickTransport()` to return null, in which case `ManagedChannelImpl` will give `ClientCallImpl` a `DelayedClientTransport`.
Changes to ClientTransport shutdown semantics
---------------------------------------------
Previously when shutdown() is called, `ClientTransport` should not accept newStream(), and when all existing streams have been closed, `ClientTransport` is terminated. Only when a transport is terminated would a transport owner (e.g., `TransportSet`) remove the reference to it.
`DelayedClientTransport` brings about a new case: when `setTransport()` is called, we switch to the real transport and no longer need the delayed transport. This is achieved by calling `shutdown()` on the delayed transport and letting it terminate. However, as the delayed transport has already been handed out to users, we would like `newStream()` to keep working for them, even though the delayed transport is already shut down and terminated.
In order to make it easy to manage the life-cycle of `DelayedClientTransport`, we redefine the shutdown semantics of transport:
- A transport can own a stream. Typically the transport owns the streams
it creates, but there may be exceptions. `DelayedClientTransport` DOES
NOT OWN the streams it returns from `newStream()` after `setTransport()`
has been called. Instead, the ownership would be transferred to the
real transport.
- After `shutdown()` has been called, the transport stops owning new
streams, and `newStream()` may still succeed. With this idea,
`DelayedClientTransport`, even when terminated, will continue
passing `newStream()` to the real transport.
- When a transport is in shutdown state, and it doesn't own any stream,
it then can enter terminated state.
ManagedClientTransport / ClientTransport
----------------------------------------
Remove life-cycle interfaces from `ClientTransport`, and put them in its subclass - `ManagedClientTransport`, with the same idea that we have `Channel` and `ManagedChannel`. Only the one who creates the transport will get `ManagedClientTransport` thus is able to start and shutdown the transport. The users of transport, e.g., `LoadBalancer`, can only get `ClientTransport` thus are not alter its state. This change clarifies the responsibility of transport life-cycle management.
Fix TransportSet shutdown semantics
-----------------------------------
Currently, if `TransportSet.shutdown()` has been called, it no longer create new transports, which is wrong.
The correct semantics of `TransportSet.shutdown()` should be:
- Shutdown all transports, thus stop new streams being created on them
- Stop `obtainActiveTransport()` from returning transports
- Streams that already created, including those buffered in delayed transport, should continue. That means if delayed transport has buffered streams, we should let the existing reconnect task continue.
com.android.tools.build:gradle:1.1.0 doesn't work with Gradle 2.10, but
1.5.0 does.
I also bumped the protobuf-gradle-plugin to be the same as the version
used in the README and our primary build.gradle.
Having Handler implement ChannelInboundHandler is overspecifying and
unnecessary, as the code compiles fine just by changing "implements
ChannelInboundHandler" to "implements ChannelHandler".
PlaintextHandler was swapped to ChannelHandlerAdapter instead of
ChannelDuplexHandler because it just needs the ChannelHandler methods.
Louis Ryan
Eric Anderson
Kun Zhang
Carl Mastrangelo
Xiao Hang
William Thurston
nmittler
Lukasz Strzalkowski
Trask Stalnaker