It's more reasonable and convenient to return a `Runnable` for `commit()` method because we want to execute part of the commit under a lock and run part of it out of the lock.
I think the idle state transitions are correct. I have looked at them and tried tracing a few paths through. This doesn't go full idle because the name resolver doesn't need to be restarted. Also, the LB transition happens inside of a NR callback, so it would be odd to have the NR terminate itself upon successful resolution. (Might this cause recursion? I think it may).
Spies are really magical and easily produce unexpected results. Using them in
tests can easily yield tests that don't do what you think they do. Delegation
is much safer when possible.
Delegation doesn't work when methods `return true`, final methods, and with
restricted visibility, though. So CensusModulesTest and
MaxConnectionIdleManagerTest are left as-is.
As delayedTransport `shutdown` will make retry attempts that are in backoff unable to create new stream. So wrap delayedTransport with `retriableTransport`, which will keep uncommitted retriable streams alive when `shutdown`.
Using META-INF for loading the version is broken on Android and
frequently broken when gRPC is shaded. Having hard-coded strings be
replaced on version bumps has been working well.
Fixes#2098
`ManagedChannelImpl` and `OobChannelq now both have a package
private field `ChannelStats`. There will be an internal access
API coming in a follow up PR.
The channelz service must not live in io.grpc.internal, and channelz
needs to be able to get the identifier of the entities it
tracks. Since io.grpc can not refer to io.grpc.internal, the LogId
must be moved out of internal.
With an approach using a variant of `DelayedStream` - `RetriableStream`.
`RetriableStream` is a logical stream per call that can create a sequence of `substreams`, the physical streams for the retry attempts. It could also create a family of substreams in parallel in hedging case.
`RetriableStream` buffers, delays and executes stream operations before `passThrough`, where `passThrough` is meant by `commit` as well as the buffer `drained` by the winning substream.
This PR only implements buffering messages and replaying buffered messages when retry.
Retry policy, hedging, transparent retry, backoff, max buffer size and APIs that users can enable retry are not included in this PR.
Each MethodDescriptor will have a binary log, though the log may
be a noop log. The binary log class is a skeleton class at the
moment, but does contain the max header and message length
info. The limits are determined by parsing the shell variable
GRPC_BINARY_LOG_CONFIG.
Previously it was 20s. 20s isn't _bad_, but may be too aggressive.
Having it longer doesn't hurt us too much; we just need some bound.
Since the MAX_BACKOFF on client-side is 120s, this will have client and
server match, except for jitter. Although I will note that the
client-side value covers more work than the server-side, such as
attempting to connect to multiple hosts.
The class is still used internally, so we move it to context's tests for
it to be reused. To avoid a circular dependency with context's tests
depending on core's tests, StaticTestingClassLoader was also moved to
context's tests.
This is driven by a need to modernize DeadlineSubject for newer versions
of Truth, but the newer versions of Truth update Guava. To avoid leaking
the Guava update to all users of grpc-testing, we're removing the
Subject. In our internal tests we can update the Truth dependency with
less issue.
handshakeTimeout is unnecessary for InProcess, and the scheduling is causing
Thread creation that is breaking restrictive test environments. Those
environments are mostly broken already because client-side will try to create
Threads as well, but they are currently lucking out that the exception on
client-side doesn't break much.
RPC upstarts are counted into metrics
RPC_{CLIENT,SERVER}_STARTED_COUNT. In addition, RPC completions are
counted into metrics RPC_{CLIENT,SERVER}_FINISHED_COUNT. From these
metrics, users will be able to derive count of RPCs that are currently
active.
This mainly copies documentation from other places, like StreamObserver
and ClientCall, but does fix some missing important threading notes.
Fixes#3413
The method name passed to MethodDescriptor does not include the leading
'/'. If it does, on the wire it will actually cause two slashes. This
has been this way for a _long_ time, but in tests that ignore the method
name or use the same MethodDescriptor no client and server the extra /
"works fine." But it's misleading, so let's remove it.
Only bump the counter from AbstractServerStream.TransportState, and hole punch
from AbstractServerStream to TransportState when the application calls close.
This diff does not actually change any behaviors yet, that will come
in the next diff along with unit tests for those new behaviors. This
diff's goal is only to change the method signatures so future diffs
are cleaner.
This commit updates gRPC core to use io.opencensus:opencensus-api and
io.opencensus:opencensus-contrib-grpc-metrics instead of
com.google.instrumentation:instrumentation-api for stats and tagging. The gRPC
Monitoring Service continues to use instrumentation-api.
The main changes affecting gRPC:
- The StatsContextFactory is replaced by three objects, StatsRecorder, Tagger,
and TagContextBinarySerializer.
- The StatsRecorder, Tagger, and TagContextBinarySerializer are never null,
but the objects are no-ops when the OpenCensus implementation is not
available.
This commit includes changes written by @songy23 and @sebright.
Counters are bumped when a message is completely written. If a
part of a message is still buffered and not yet flushed, we will
not increment the stats.
This moves away from the global String-based Span name registry which
is not as flexible as we desire.
Also renamed the option name to be more accurate. This is not
API-breaking because the origianl addition to MethodDescriptor and
code-gen didn't make it into the 1.7.0 release.
In relative order of importance:
* Don't acquire lock when picking subchannel.
* Use O(1) lookup for updating channel state
* Use non synchronized ref instead of AtomicReference
* Dont store size in picker.
* make class final
* remove test that was not valid
This is needed for both completeness and stats/tracing contexts propagation.
Stats recording with Census is intentionally disabled (#2284), while the rest of the Census-related logic work the same as on the other transports.
This is a more favorable approach than #3467. Doing the registration
in MethodDescriptor should allow us to deregister in case the
generated stub and its MethodDescriptors are garbage-collected
routinely, e.g., if they are loaded by a separate ClassLoader.
Two methods, outboundMessageSent() and inboundMessageRead() are added to StreamTracer in order to associate individual messages with sizes. Both types of sizes are optional, as allowed by Census tracing.
Both methods accept a sequence number as the type ID as required by Census. The original outboundMesage() and inboundMessage() are also replaced by overrides that take the sequence number, to better match the new methods. The deprecation of the old overrides are tracked by #3460
* core: add finalizer checks for ManagedChannels
Cleaning up channels is something users should do. To promote this
behavior, add a log message to indicate that the channel has not
been properly cleaned.
This change users WeakReferences to avoid keeping the channel
alive and retaining too much memory. Only the id and the target
are kept. Additionally, the lost references are only checked at
JVM shutdown and on new channel creation. This is done to avoid
Object finalizers.
The test added checks to see that the message is logged. Since
java does not allow forcing of a GC cycle, this code is best
effort, giving up after about a second. A custom log filter is
added to hook the log messages and check to see if the correct
one is present. Handlers are not used because they are
hierarchical, and would be annoying to restore their state after
the test.
The other tests in the file contribute a lot of bad channels. This
is reasonable, because they aren't real channels. However, it does
mean that less than half of them are being cleaned up properly.
After trying to fix a few, it is too hard to do. It would only
serve to massively complicate the tests.
Instead, this code just keeps track of how many it wasn't able to
clean up, and ignores them for the test. They are still logged,
because really they should be closed.
* inprocess,core: add ManagedChannelBuilder and ServerBuilder factory hiders
Because the factory for Channels and Servers resides on the builder
itself, it is easy for subclasses to accidentally inherit the
factory. This causes confusion, because calling a static method on
a specific class may result in a different class.
This change adds hiding static factories to each builder, and a test
to enforce that each subclass hides the factory. The test lives in
the interop tests, because it has a classpath dependency on all the
existing transports.
Minor note: the test scans the classpath using a Beta Guava API.
The test can be disabled if the API goes away.
The benchmarks should be close to the code they're benchmarking, like
we do with tests.
This includes a bugfix to SerializingExecutorBenchmark to let it run.
The io.grpc.benchmarks.netty benchmarks in benchmarks/ depend on
ByteBufOutputMarshaller from benchmarks's main, so they were not moved.
While the code had correctly determined full threads were available, the
call to MoreExecutors returned a request thread factory, which has
limitations.
Note that Async stub users may not be able to call GAE APIs in
callbacks. This is because the threads aren't request threads. They can
override the individual call's executor with
com.google.appengine.api.ThreadManager.currentRequestThreadFactory() in
an interceptor via callOptions.withExecutor().
Fixes#3296
The assertions are actually wrong and fail every time. It doesn't
cause test failures because SharedResourceHolder calls them in a
scheduled executor because of its delayed close feature.
It's better to remove them, rather than leaving them there deceiving
us.
This aligns with shutdownNow(), which is already accepting a status.
The status will be propagated to application when RPCs failed because
of transport shutdown, which will become useful information for debug.
This commit aligns the naming of the Bazel Maven jars with the names
used by Bazel's migration-tooling project:
https://github.com/bazelbuild/migration-tooling
Unfortunately, we can't fix @com_google_protobuf_java because it's
required by Bazel itself.
Fixes#3328
null class loader means to use the bootstrap class loader, which would
normally make sense. However, Robolectric tests run on OpenJDK and
provide the Android environment as part of the application, so "Android"
won't be present in the bootstrap class loader.
This is a big, but mostly mechanical change. The newly added Test*StreamTracer classes are designed to be extended which is why they are non final and have protected fields. There are a few notable things in this:
1. verifyNoMoreInteractions is gone. The API for StreamTracers doesn't make this guarantee. I have recovered this behavior by failing duplicate calls. This has resulted in a few bugs in the test code being fixed.
2. StreamTracers cannot be mocked anymore. Tracers need to be thread safe, which mocks simply are not. This leads to a HUGE number of reports when trying to find real races in gRPC.
3. If these classes are useful, we can promote them out of internal. I just put them here out of convenience.
If onTransportActive ran while SendPing was already scheduled, we would
schedule another SendPing, which seems fine, but the server might observe
us sending pings too quickly, and make us GOAWAY.
Fixes#3274.
As discussed in #1914, we need CallCredentials and MoreCallCredentials
to be stable, but there's less of a strong need for the contents of
CallCredentials to be stable. We're willing to commit to the name,
without needing to commit to the plumbing.
Coupled with the similar change on server-side, this removes the need for a
thread when using Netty. For InProcess and OkHttp, it would allow us to let the
user to provide the scheduler for tests or application-wide thread sharing.
ZHANG Dapeng
Eric Anderson
zpencer
Carl Mastrangelo
Shohei Kamimori
Kun Zhang
Eric Gribkoff
kenji yoshida
David Cowden
ramaraochavali
sebright
Devin Smith
pieterjanpintens
Lukasz Strzalkowski
John Hume
Paul Gross
myPrecious
Stephen Haberman