The method name passed to MethodDescriptor does not include the leading
'/'. If it does, on the wire it will actually cause two slashes. This
has been this way for a _long_ time, but in tests that ignore the method
name or use the same MethodDescriptor no client and server the extra /
"works fine." But it's misleading, so let's remove it.
Only bump the counter from AbstractServerStream.TransportState, and hole punch
from AbstractServerStream to TransportState when the application calls close.
This diff does not actually change any behaviors yet, that will come
in the next diff along with unit tests for those new behaviors. This
diff's goal is only to change the method signatures so future diffs
are cleaner.
This commit updates gRPC core to use io.opencensus:opencensus-api and
io.opencensus:opencensus-contrib-grpc-metrics instead of
com.google.instrumentation:instrumentation-api for stats and tagging. The gRPC
Monitoring Service continues to use instrumentation-api.
The main changes affecting gRPC:
- The StatsContextFactory is replaced by three objects, StatsRecorder, Tagger,
and TagContextBinarySerializer.
- The StatsRecorder, Tagger, and TagContextBinarySerializer are never null,
but the objects are no-ops when the OpenCensus implementation is not
available.
This commit includes changes written by @songy23 and @sebright.
Counters are bumped when a message is completely written. If a
part of a message is still buffered and not yet flushed, we will
not increment the stats.
This moves away from the global String-based Span name registry which
is not as flexible as we desire.
Also renamed the option name to be more accurate. This is not
API-breaking because the origianl addition to MethodDescriptor and
code-gen didn't make it into the 1.7.0 release.
In relative order of importance:
* Don't acquire lock when picking subchannel.
* Use O(1) lookup for updating channel state
* Use non synchronized ref instead of AtomicReference
* Dont store size in picker.
* make class final
* remove test that was not valid
This is needed for both completeness and stats/tracing contexts propagation.
Stats recording with Census is intentionally disabled (#2284), while the rest of the Census-related logic work the same as on the other transports.
This is a more favorable approach than #3467. Doing the registration
in MethodDescriptor should allow us to deregister in case the
generated stub and its MethodDescriptors are garbage-collected
routinely, e.g., if they are loaded by a separate ClassLoader.
Two methods, outboundMessageSent() and inboundMessageRead() are added to StreamTracer in order to associate individual messages with sizes. Both types of sizes are optional, as allowed by Census tracing.
Both methods accept a sequence number as the type ID as required by Census. The original outboundMesage() and inboundMessage() are also replaced by overrides that take the sequence number, to better match the new methods. The deprecation of the old overrides are tracked by #3460
* core: add finalizer checks for ManagedChannels
Cleaning up channels is something users should do. To promote this
behavior, add a log message to indicate that the channel has not
been properly cleaned.
This change users WeakReferences to avoid keeping the channel
alive and retaining too much memory. Only the id and the target
are kept. Additionally, the lost references are only checked at
JVM shutdown and on new channel creation. This is done to avoid
Object finalizers.
The test added checks to see that the message is logged. Since
java does not allow forcing of a GC cycle, this code is best
effort, giving up after about a second. A custom log filter is
added to hook the log messages and check to see if the correct
one is present. Handlers are not used because they are
hierarchical, and would be annoying to restore their state after
the test.
The other tests in the file contribute a lot of bad channels. This
is reasonable, because they aren't real channels. However, it does
mean that less than half of them are being cleaned up properly.
After trying to fix a few, it is too hard to do. It would only
serve to massively complicate the tests.
Instead, this code just keeps track of how many it wasn't able to
clean up, and ignores them for the test. They are still logged,
because really they should be closed.
* inprocess,core: add ManagedChannelBuilder and ServerBuilder factory hiders
Because the factory for Channels and Servers resides on the builder
itself, it is easy for subclasses to accidentally inherit the
factory. This causes confusion, because calling a static method on
a specific class may result in a different class.
This change adds hiding static factories to each builder, and a test
to enforce that each subclass hides the factory. The test lives in
the interop tests, because it has a classpath dependency on all the
existing transports.
Minor note: the test scans the classpath using a Beta Guava API.
The test can be disabled if the API goes away.
The benchmarks should be close to the code they're benchmarking, like
we do with tests.
This includes a bugfix to SerializingExecutorBenchmark to let it run.
The io.grpc.benchmarks.netty benchmarks in benchmarks/ depend on
ByteBufOutputMarshaller from benchmarks's main, so they were not moved.
While the code had correctly determined full threads were available, the
call to MoreExecutors returned a request thread factory, which has
limitations.
Note that Async stub users may not be able to call GAE APIs in
callbacks. This is because the threads aren't request threads. They can
override the individual call's executor with
com.google.appengine.api.ThreadManager.currentRequestThreadFactory() in
an interceptor via callOptions.withExecutor().
Fixes#3296
The assertions are actually wrong and fail every time. It doesn't
cause test failures because SharedResourceHolder calls them in a
scheduled executor because of its delayed close feature.
It's better to remove them, rather than leaving them there deceiving
us.
This aligns with shutdownNow(), which is already accepting a status.
The status will be propagated to application when RPCs failed because
of transport shutdown, which will become useful information for debug.
This commit aligns the naming of the Bazel Maven jars with the names
used by Bazel's migration-tooling project:
https://github.com/bazelbuild/migration-tooling
Unfortunately, we can't fix @com_google_protobuf_java because it's
required by Bazel itself.
Fixes#3328
null class loader means to use the bootstrap class loader, which would
normally make sense. However, Robolectric tests run on OpenJDK and
provide the Android environment as part of the application, so "Android"
won't be present in the bootstrap class loader.
This is a big, but mostly mechanical change. The newly added Test*StreamTracer classes are designed to be extended which is why they are non final and have protected fields. There are a few notable things in this:
1. verifyNoMoreInteractions is gone. The API for StreamTracers doesn't make this guarantee. I have recovered this behavior by failing duplicate calls. This has resulted in a few bugs in the test code being fixed.
2. StreamTracers cannot be mocked anymore. Tracers need to be thread safe, which mocks simply are not. This leads to a HUGE number of reports when trying to find real races in gRPC.
3. If these classes are useful, we can promote them out of internal. I just put them here out of convenience.
If onTransportActive ran while SendPing was already scheduled, we would
schedule another SendPing, which seems fine, but the server might observe
us sending pings too quickly, and make us GOAWAY.
Fixes#3274.
As discussed in #1914, we need CallCredentials and MoreCallCredentials
to be stable, but there's less of a strong need for the contents of
CallCredentials to be stable. We're willing to commit to the name,
without needing to commit to the plumbing.
Coupled with the similar change on server-side, this removes the need for a
thread when using Netty. For InProcess and OkHttp, it would allow us to let the
user to provide the scheduler for tests or application-wide thread sharing.
This is only implementing how subchannel state changes are hooked up to channel state APIs. Debates in state semantics and transition rules for loadbalanced channel such as #2873 are not addressed. Any changes in state semantics and transition rules in the future can be easily adapted.
For Netty, this reduces the number of threads necessary for servers (although
until channel is converted, actual number of threads isn't impacted) and
naturally reduces contention and timeout latency.
For InProcess, this gets us closer to allowing applications to provide all
executors, which is especially useful during tests.
Class.forName(String) is understood by ProGuard, removing the need for
manual ProGuard configuration and allows ProGuard to rename the provider
classes. Previously the provider classes could not be renamed.
Fixes#2633
Previously anyone could create metadata keys with a leading ":", due to not having a way to prevent it. This change effectively only allows internal code to make use of this.
* refactor SubchannelImpl -> AbstractSubchannel
The old `SubchannelImpl` is actually not implementing anything, and the old `SubchannelImplImpl` has a weird name.
currentThread().getContextClassLoader() was originally used to match
ServiceLoader's default. ServiceLoader is mostly used with SPIs that
exist in the bootclasspath and so the class's ClassLoader would not
work. But in gRPC we have a library in the application's ClassLoader.
Context ClassLoader has been causing problems in any case more than one
application ClassLoader exists, like in Servlet containers, Android, and
plugins. To my knowledge, in every case the context class loader is
different from the provider's class loader, the context class loader has
caused breakage, not success. In addition, since we use static fields
for storing the provider results, using anything but the provider's
class loader is asking to have results that vary simply depending on
which thread called gRPC first.
Fixes#2375
It is misnamed if it is @ExperimentalApi. Even being @Internal is
strange, since nothing in io.grpc actually uses it and so it could
actually be in io.grpc.internal.
I'm not trying to fix the API, but just make it slightly more
consistent.
Previously ClientCallImpl's stream listener would call
stream.setDecompressor(), but this has always been a bit strange as the
only case where the call listener calls the stream and forms a bit of a
loop. It also turned out to be racy in the presence of
DelayedClientStream since DelayedClientStream does not guarantee that
the listener has processed before returning.
Now we let the stream handle decompressor selection itself. Compressor
selection on client and server and decompressor selection on server
remain unchanged. Nothing prevents them from being changed, other than
it is currently unnecessary to fix the severe compressionTest flake.
Fixes#2865Fixes#2157
Bazel third party dependencies are specified in repositories.bzl which
gives the consumer the ability to opt-out of any dependencies they use
directly in their own project.
Fixes#2756
This adds server-wide interceptors that applies to all call handlers.
Because ServerCallHandler is acquired per request, and can be dynamicly
provided by the fallback registry, the interceptors have to be installed
on a per-request basis. This adds a few object allocations per request,
which is acceptable.
Today JumpToApplicationThreadServerStreamListener leaks server state by transmitting details about uncaught StatusRuntimeException throwables to the client. This is a security problem.
This PR ensures that uncaught exceptions always close the ServerCall without leaking any state information. Users running in a trusted environment who want to transmit error details can install the TransmitStatusRuntimeExceptionInterceptor.
fixes#2189
Interning strings isn't needed much any more since keys are now
compared by byte value. Also, intern is slow (about 150ns per)
which affects code that creates keys often. Lastly, older
versions of Java don't garbage collect interned strings, which
lowers the applications stability.
The current implementation has a bug where certain methods are not forwarded to the delegate.
This is essentially the same as e4f1f39 which was merged to the v1.4.x branch. This PR uses the new license header.
Fixes#3061
The current check in ServerCallImpl is theoretically unsafe (#3059). Move that check into the stub, and expand the unit tests to cover other interesting edge cases on the server side:
client sends one, but zero requests received at onHalfClose
client sends one, but > 1 requests received at onHalfClose
server sends one, but zero responses sent at onComplete
server sends one, but > 1 responses sent via onNext
fixes#2243fixes#3059
Add some example tests for easier fields in AbstractManagedChannelImplBuilder.
Many fields are no longer Nullable, in order to move logic from construction to
mutation, which eases testing and simplifies cross-class interactions.
The nameResolverFactory comment starting "Avoid loading the provider unless
necessary" was outdated and has not been a concern since #2071 which swapped to
a hard-coded list on Android.
We don't want people to be desensitized to the Internal annotation.
NameResolverProvider should have probably been ExperimentalApi from the
start, but it was copied from ManagedChannelProvider.
Theoretically, ManagedChannelProvider could be marked ExperimentalApi as
well, but there are no known use cases for doing so since making an
alternative ManagedChannel implementation is quite difficuilt and we
aren't aware of anybody interested in doing so. That isn't the case for
NameResolverProvider; NameResolverProvider is core to 'targets' being
useful.
Calling CancellableContext#close() multiple times does not cause problems, but is unnecessary here. The return value of getListener() is a ServerStreamListenerImpl, which already calls CancellableContext#cancel() inside of the finally block of ServerStreamListener#closed().
When a cancellation happens, the ServerCall and Context get notified. Rather than serializing on the normal work queue (which may be doing user computation), we should execute the notification immediately, thereby allowing the user computation to see the cancellation.
This allows avoiding re-creating connections unnecessarily. For
NameResolvers that support notifications this avoids stampeding herds
when possible, since all clients may receive the notification at
approximately the same time.
This allows avoiding re-creating connections unnecessarily. For
NameResolvers that support notifications this avoids stampeding herds
when possible, since all clients may receive the notification at
approximately the same time.
Eric Anderson
zpencer
sebright
Eric Gribkoff
Kun Zhang
ZHANG Dapeng
Carl Mastrangelo
Devin Smith
pieterjanpintens
Lukasz Strzalkowski
John Hume
Paul Gross
myPrecious
Stephen Haberman
Bogdan Drutu
Simon Horlick
Łukasz Strzałkowski
Yang Song
William Thurston