Also updated CensusModule to use the new helper methods ContextUtils.withValue() instead of directly manipulating the context keys. See census-instrumentation/opencensus-java#1864.
Having two Helpers (the other being LoadBalancer.Helper) is proven to
be confusing, and the one in NameResolver is fundamentally different
from the one from LoadBalancer, as the latter mutates the states while
the former doesn't. Renaming it to Args make it less awkward to
expose it to LoadBalancer, which is needed for creating NameResolvers
for per-locality routing in the XdsLoadBalancer.
As we are now endorsing the wrapping of ClientStreamTracers by
providing ForwardingClientStreamTracer, there is a need for altering
StreamInfo, especially CallOptions before it's passed onto the
delegate. A Builder class and a toBuilder() provides a robust way
to copy the rest of the fields.
This is a breaking change for anybody who creates StreamInfo, which is
unlikely in non-test code, because StreamInfo was added as late as
1.20.0.
NameResolverRegistry takes on all the logic previously in
NameResolverProvider. But it also allows manual registration of
NameResolvers, which is useful when the providers have complex
construction or need objects injected into them.
This also avoids a circular dependency during class loading since
previously loading any Provider searched for all Providers via
ClassLoader since ClassLoader handling was static within the parent
class.
Fixes#5562
io.grpc has fewer dependencies than io.grpc.internal. Moving it to a
separate artifact lets users use the API without bringing in the deps.
If the library has an optional dependency on grpc, that can be quite
convenient.
We now version-pin both grpc-api and grpc-core, since both contain
internal APIs.
I had to change a few tests in grpc-api to avoid FakeClock. Moving
FakeClock to grpc-api was difficult because it uses
io.grpc.internal.TimeProvider, which can't be moved since it is a
production class. Having grpc-api's tests depend on grpc-core's test
classes would be weird and cause a circular dependincy. Having
grpc-api's tests depend on grpc-core is likely possible, but weird and
fairly unnecessary at this point. So instead I rewrote the tests to
avoid FakeClock.
Fixes#1447
Resolves#5497
## Motivation
In hierarchical `LoadBalancer`s (e.g., `XdsLoadBalancer`) or wrapped `LoadBalancer`s (e.g., `HealthCheckingLoadBalancerFactory`, the top-level `LoadBalancer` receives `Subchannel` state updates from the Channel impl, and they almost always pass it down to its children `LoadBalancer`s.
Sometimes the children `LoadBalancer`s are not directly created by the parent, thus requires whatever API in the middle to also pass Subchannel state updates, complicating that API. For example, the proposed [`RequestDirector`](https://github.com/grpc/grpc-java/issues/5496#issuecomment-476008051) includes `handleSubchannelState()` solely to plumb state updates to where they are used. We also see this pattern in `HealthCheckingLoadBalancerFactory`, `GrpclbState` and `SubchannelPool`.
Another minor issue is, the parent `LoadBalancer` would need to intercept the `Helper` passed to its children to map Subchannels to the children `LoadBalancer`s, so that it pass updates about relevant Subchannels to the children. Otherwise, a child `LoadBalancer` may be surprised by seeing Subchannel not created by it, and it's not efficient to broadcast Subchannel updates to all children.
## API Proposal
We will pass a `SubchannelStateListener` when creating a `Subchannel` to accept state updates, those updates could be directly passed to where the `Subchannel` is created, skipping the explicit chaining in the middle.
Also define a first-class object `CreateSubchannelArgs` to pass arguments for the reasons below:
1. It may avoid breakages when we add new arguments to `createSubchannel()`. For example, a `LoadBalancer` may wrap `Helper` and intercept `createSubchannel()` in a hierarchical case. It may not be interested in all arguments. Passing a single `CreateSubchannelArgs` will not break the parent `LoadBalancer` if we add new fields later.
2. This also reduces the eventual size of Helper interface, as the convenience `createSubchannel()` that accepts one EAG instead of a List is no longer necessary, since that convenience is moved into `CreateSubchannelArgs`.
```java
interface SubchannelStateListener {
void onSubchannelState(Subchannel subchannel, ConnectivityStateInfo newState);
}
abstract class LoadBalancer.Helper {
abstract Subchannel createSubchannel(CreateSubchannelArgs args);
}
final class CreateSubchannelArgs {
final List<EquivalentAddressGroup> getAddresses();
final Attributes getAttributes();
final SubchannelStateListener getStateListener();
final class Builder () {
...
}
}
```
The new `createSubchannel()` must be called from synchronization context, as a step towards #5015.
## How the new API helps
Most hierarchical `LoadBalancer`s would just let the listener from the child `LoadBalancer`s directly pass through to gRPC core, which is less boilerplate than before.
Without any effort by the parent, each child will only see updates for the Subchannels it has created, which is clearer and more efficient.
If a parent `LoadBalancer` does want to look at or alter the Subchannel state updates for its delegate (like in `HealthCheckingLoadBalancerFactory`), it can still do so in the wrapping `LoadBalancer.Helper` passed to the delegate by intercepting the `SubchannelStateListener`.
## Migration implications
Existing `LoadBalancer` implementations will continue to work, while they will see deprecation warnings when compiled:
- The old `LoadBalancer.Helper#createSubchannel` variants are now deprecated, but will work until they are deleted. They create a `SubchannelStateListener` that delegates to `LoadBalancer#handleSubchannelState`.
- `LoadBalancer#handleSubchannelState` is now deprecated, and will throw if called and the implementation doesn't override it. It will be deleted in a future release.
The migration for most `LoadBalancer` implementation would be moving the logic from `LoadBalancer#handleSubchannelState` into a `SubchannelStateListener`.
This class is used in other places than just NameResolver.Helper. It
should not be an inner class of Helper.
Strictly speaking this is an API-breaking change. However, this is
part of the service config error handling API that hasn't been done
yet. Nobody has a legitimate reason to use it.
We assumed that we were the only caller. Turns out there are
forwarding NameResolver too. Implementing the old override will give
them time to migrate to the new API.
Resolves#5556
Both the config and the error need to be passed to the managed channel so it can decide to keep or reject the config. Passing only the parsed object means the managed channel cannot implement the error handling gRFC.
This code is highly experimental, so we can change it at will later. This PR is to go ahead and try it out.
Perfmark task calls are added to the client and server calls public APIs, recording when the calls begin and end. They use separate scope IDs (roughly these are Thread IDs) for the listener and the call due to no synchronization between them. However, they both use the same PerfTags, which allows them to be associated.
In the future, we can plumb the tag down into the stream to include deeper information about whats going on in a call.
Make the "logged" method be consistent, and refer to the public logging class name and method. This makes the log statements return the same class name used to set the log level.
Before:
```
190306 13:29:39.290:D 1 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Channel for 'localhost:10000' created
190306 13:29:39.414:D 1 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Exiting idle mode
190306 13:29:39.622:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Resolved address: [[[/127.0.0.1:10000]/{}]], config={}
190306 13:29:39.623:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Address resolved: [[[/127.0.0.1:10000]/{}]]
190306 13:29:39.624:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Subchannel<3>] Subchannel for [[[/127.0.0.1:10000]/{}]] created
```
After:
```
190306 13:49:15.654:D 1 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Channel for 'localhost:10000' created
190306 13:49:15.772:D 1 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Exiting idle mode
190306 13:49:15.995:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Resolved address: [[[/127.0.0.1:10000]/{}]], config={}
190306 13:49:15.995:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Address resolved: [[[/127.0.0.1:10000]/{}]]
190306 13:49:15.997:D 18 [io.grpc.ChannelLogger.log] [Subchannel<3>] Subchannel for [[[/127.0.0.1:10000]/{}]] created
190306 13:49:15.999:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Child Subchannel created
```
Make sure the config for grpclb is passed to the GrpclbLoadBalancer, which will support two child policies -- "round_robin" (default) and "pick_first".
Previously the presence of balancer addresses would dictate "grpclb" policy, despite of the service config. Service config will now take precedence instead.
Implement config parsing logic in GrpclbLoadBalancer. Per offline discussions with @markdroth and @ejona86, we will ignore configuration errors for now. The more appropriate config error handling is upcoming.
This was added for the potential use case of needing to resolve target
names (of the same scheme as the top-level channel's target's) in the
LoadBalancer. Now actual use cases come up in xDS that we need to
resolve fully-qualified target strings with arbitrary schemes. This
method has never been used and won't fit future uses because it's too
restrictive.
This is needed for GRPCLB pick_first support, which needs to attach
tokens to headers, and the tokens are per server. In pick_first, all
addresses are in a single Subchannel, thus the LoadBalancer needs to
know which backend is used for a new stream.
This can be used by xds LoadBalancer to create a channel to the XDS
traffic director, as the service config will only specify the target
name of the balancer.
This PR only adds the interface to unblock the xds work.
Implementation would take some time thus will come later.
The LoadBalancingConfig message, which looks like
```json
{
"policy_name" : {
"config_key1" : "config_value1",
"config_key2" : "config_value2"
}
}
```
appears multiple times. It gets super tedious and confusing to handle, because both the whole config and the value (in the above example is `{ "config_key1" : "config_value1" }`) are just `Map<String, Object>`, and each user needs to do the following validation:
1. The whole config must have exactly one key
2. The value must be a map
Here I define `LbConfig` that holds the policy name and the config value, and a method in `ServiceConfigUtil` that converts the parsed JSON format into `LbConfig`.
There is also multiple cases where you need to handle a list of configs (top-level balancing policy, child and fallback policies in xds, grpclb child policies). I also made another helper method in `ServiceConfigUtil` to convert them into `List<LbConfig>`.
Found and fixed a bug in the xds code, where the top-level balancer should pass the config value (excluding the policy name), not the whole config to the child balancers. Search for "supported_1_option" in the diff to see it in the tests.
This will be a new override. The old override is now deprecated.
In order to pass new information without adding new overrides, I shoved most information
into an object called StreamInfo. The Metadata is left out to draw attention because
it's mutable.
Motivation: this is needed for correctly supporting pick_first in GRPCLB. GRPCLB needs to
add a token to the headers, and the token varies among servers. With round_robin, GRPCLB
create a Subchannel for each server, thus can attach the token when the Subchannel is picked.
To implement pick_first, all server addresses will be put in a single Subchannel, we will
need to add the header in newClientStreamTracer(), by looking up the server address from
the transport attributes and deciding which token to add.
Resolves#2649
As a prerequisite, added `getSynchronizationContext()` to `NameResolver.Helper`.
`DnsNameResolver` has gone through a small refactor around the `Resolve` runnable, which makes it a little simpler.
This reverts commit d47379947f.
This caused test failures internally, where gRPC failed with
"IllegalArgumentException: Invalid trace name". Not only was this
failure unexpected, it was also weird that it failed with an
IllegalArgumentException instead of the normal StatusRuntimeException.
This allows users to upgrade to newer versions of Guava without breaking
the build due to missing transitive dependencies. Without it, when
increasing the Guava version to 27.0+ you see errors like:
stub/src/main/java/io/grpc/stub/ClientCalls.java:487: error: cannot access InternalFutureFailureAccess
private static final class GrpcFuture<RespT> extends AbstractFuture<RespT> {
^
class file for com.google.common.util.concurrent.internal.InternalFutureFailureAccess not found
Note that using a runtime_dep does not fix the compilation error.
Long-term we will want our users to migrate to
jvm_maven_import_external. However, if we use it ourselves, it will
effectively require our users to use it immediately themselves. It has
low penetration today, including lack of usage by tools like
johnynek/bazel-deps.
Fixes#5214
Context: [#4159 (comment)](https://github.com/grpc/grpc-java/issues/4159#issuecomment-415827641)
`Attributes` is appropriate for plumbing optional objects, especially useful for a long plumbing path where components in the middle may not care or see all objects in the container. It's not the case for the `params` on `newNewResolver()`. Both the default port and the proxy detector are guaranteed to be there and the plumbing path is very short. In this case, a first-class object is more appropriate and easier to use.
The `Helper` will also have `getSynchronizationContext()` (#2649) and a method to parse and validate service config. We we also considering merging `Listener` into the `Helper`, to make `NameResolver` match the `LoadBalancer` API.
Combine the public `ProxyParameters` and the internal `ProxySocketAddress` as `HttpConnectProxiedSocketAddress`. The more specific name signifies the type of the proxy we currently support, and makes room for other proxy types (e.g., SOCKS) in the future. The combination simplifies NameResolver implementation.
Introduce `ProxiedSocketAddress` as the base class that is returned by `ProxyDetector`, mainly for clarification and documentation. Added documentation about proxy in general on `ProxyDetector`.
- defined XdsLbState, playing a similar role to GrpclbState
- there are two modes of XdsLbState: STANDARD and CUSTOM
- on `XdsLoadBalancer.handleResolvedAddressGroups()`, the `xdsLoadBalancer` will update the `xdsLbState` based on the lb config in the attributes passed in
In Mockito 2, if a method expects a primitive type, but an any(<Primitive>.class) matcher is used in its place, it will throw an error. To prepare for this upcoming breakage, change
all existing any(<Primitive>.class) matchers to use the correct any<Primitive>() matcher.
This fixes two races: a data race where scheduledRetry is accessed
by cancel() and confusion where scheduledRetry could be set to null by
the schedule()d runnable before it is set by the schedule() return
value.
Although it seems these races can't actually cause problems due to other
conditions/constraints, it's hard to reason about. So let's plug these
preemptively, even if we can't add tests that trigger them.
ScheduledHedging was not specific to hedging, so can now be reused for
this retry case. It was renamed to avoid being misleading.
This avoids a memory leak when the channel itself participates in a
reference cycle (e.g., when an interceptor retains a reference to an
Android app's context). With the current implementation, the static
`ManagedChannelOrphanWrapper.refs` map will keep the channel reachable
and prevent the ref cycle from being GCed.
SRV has not yet been enabled in a release. Since work is rapidly
underway to replace GRPC-LB with a service config+XDS-based solution,
there's now thoughts that we won't ever enable grpclb by default (but
may allow it to be automatically enabled when using GoogleDefaultChannel
or similar). Since things are being worked out, disable it.
This effectively reverts c729a0f.
This is the 3rd step of #4901
- The deprecated `CC.applyRequestMetadata(... Attributes ...)` is now **replaced** by the new API `CC.applyRequestMetadata(... CC.MetadataApplier ...)` transformed from `CC2.applyRequestMetadata(... CC2.MetadataApplier ...)`.
- The Attributes keys in `CallCredentials` were deprecated, and now deleted.
- The deprecated interface `CC.MetadataApplier` is **replaced** by an equivalent abstract class.
- `CallCredentials2` is now marked as deprecated, while keeping its interface intact so that it won't break current implementations that are still on `CallCredentials2`.
- From this point on, implementations should do a one-line change from `extends CallCredentials2` to `extends CallCredentials`
- `GoogleAuthLibraryCallCredentials` is kept as `CallCredentials2` for now, as there is an internal consumer that expects it to be `CallCredentials2`.
We've been on newer versions of Guava for a while now; these no longer
do anything.
Reworded the comment for Stopwatch.createUnstarted(), because it is not
safe (it doesn't matter if the method isn't marked Beta; you have to use
Ticker), except for the fact it is only used in our tests.
* makes Census tracing factories at the end of the user added ones
* makes more vars in AbstractServerImplBuilder package private
* annotates methods in ASIB to be clearer
* simplifies several of the setters to be single line
* Makes the generics on the Tracer factories proper
Currently ManagedChannelImpl will interpret empty address list from
NameResolver as an error, and LoadBalancer will NEVER receive an empty
list from handleResolvedAddressGroups(). There is a case in the
request-routing design where a LoadBalancer only receives service
config with which it constructs children NameResolver/LoadBalancer
pairs, thus no address is expected at this level.
canHandleEmptyAddressListFromNameResolution() is a signal to Channel
(and to the parent LoadBalancer in case of hierachical LoadBalancers)
about whether it accepts empty address lists. The default is false,
which is the current behavior.
The logic is currently duplicated in ManagedChannelImpl and
AutoConfiguredLoadBalancer. The one in ManagedChannelImpl will be
removed once we delete ManagedChannelBuilder.loadBalancerFactory() as
AutoConfiguredLoadBalancer will always be the top-level LoadBalancer
by then.
Previously, overwriting an existing Key would cause the original CallOptions instance to also be mutated.
See #5142
Also adds a regression test for this issue.
ManagedChannelBuilder.loadBalancingFactory() overrides the proper
policy selection logic implemented in AutoConfiguredLoadBalancer, thus
has problems in cases where NameResolver returns balancer addresses,
because custom LoadBalancers normally don't differentiate between
normal server addresses with balancer addresses. The policy selection
logic will filter out balancer addresses.
When service owner turns on grpclb through service config, it
shouldn't break existing clients that don't have grpclb in their
classpath.
Resolves#4602
There are currently three boolean flags, and there will be one more
soon. Put them all in the top-level class instead of passing them as
arguments on lower levels.
Raise visibility of AutoConfiguredLoadBalancerFactory as internal
tests need to access it from a different package.
Rename HealthCheckingLoadBalancerFactory.LoadBalancerImpl to
*.HealthCheckingLoadBalancer so that its toString() output is more
informative.
This is needed for internal issue b/119247688.
A particular test that runs GRPC Android build in a non-Android
environment failed because RoundRobinLoadBalancerProvider was deleted
by ProGuard but the service-loader META-INF file still referred to it,
causing a loading failure.
This could be fixed by adding RoundRobinLoadBalancerProvider to the
hard-coded list, which is recognized by ProGuard then it will keep the
class.
However, we don't expect anyone to use RoundRobinLoadBalancerProvider
on Android, including the class on Android would increase code size,
which Android apps are sensitive to. Hence we move
RoundRobinLoadBalancerProvider to a different package (util), which is
built as a separate artifact internally which Android users usually
don't depend on. (Note that in open-source util is in the same artifact as core,
which is unfortunately).
Remove unused variables and prefer ArrayDeque to LinkedList. The swap to
Queue from Deque was just to make it more obvious what the usage was,
since the original swap to Deque was to avoid the same LinkedList lint
violation (3d51756d).
Introduce ChannelLogger, which is a utility provided to LoadBalancer implementations (potentially NameResolvers too) for recording events to channel trace. This is immediately required by client-side health checking (#4932, https://github.com/grpc/proposal/blob/master/A17-client-side-health-checking.md) to record an error about disabling health checking. It is also useful for any LoadBalancer implementations to record important information.
ChannelLogger implementation is backed by the internal ChannelTracer/Channelz. Because Channelz limits the number of retained events, and events are lost once the process ends, I have expanded it to also log Java logger. This would provide a "last resort" in cases where there are too many events or off-line investigation is needed. All logs are prefixed with logId so that they can be easily associated with the involved Channel/Subchannel.
To prevent log spamming, the logs are all at FINE level or below so that they are not visible by default. They are logged to ChannelLogger's logger, so that user can have precise control.
There are also more verbose information that may not fit in ChannelTracer, but can be useful for debugging. It's desirable that these logs are associated with logId, but they currently manually include the logId, which is cumbersome and may result in inconsistency. For this use case, I added the DEBUG level for ChannelLogger, which formats the log in the same way as other levels, while not recorded to Channelz.
I have converted most logging and channel tracer recording in the Channel implementation and LoadBalancers.
The ManagedChannelImpl change prevents any LB initialization failure
from producing a useless exception like:
java.lang.NullPointerException
at io.grpc.internal.ManagedChannelImpl.shutdownNameResolverAndLoadBalancer(ManagedChannelImpl.java:321)
at io.grpc.internal.ManagedChannelImpl.panic(ManagedChannelImpl.java:738)
at io.grpc.internal.ManagedChannelImpl$1.uncaughtException(ManagedChannelImpl.java:144)
Instead, now it will have the expected panic behavior of an INTERNAL
Status with a proper cause.
Since the NPE in AutoConfiguredLoadBalancerFactory wouldn't mean much to
users, it now has a more explicit message.
Following the [spec](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md) on duplicate header names:
**Custom-Metadata** header order is not guaranteed to be preserved except for values with duplicate header names. Duplicate header names may have their values joined with "," as the delimiter and be considered semantically equivalent. Implementations must split Binary-Headers on "," before decoding the Base64-encoded values.
Because otherwise the user logic around Subchannel creation will
likely to race with handleSubchannelState().
Will log a warning if LoadBalancer.Helper.createSubChannel() is called
outside of the SynchronizationContext.
Adds SynchronizationContext.throwIfNotInThisSynchronizationContext()
to facilitate this warning. It can also be used by LoadBalancer
implementations to make it a requirement.
LoadBalancerProvider is the interface that extends LoadBalancer.Factory. LoadBalancerRegistry is the one that loads the providers through service loader, and allows users to access providers through their names.
pick_first and round_robin balancer factories, which are experimental public API are now deprecated. Their providers are internal, as they are accessible by policy name.
AutoConfiguredLoadBalancerFactory is modified to access implementations purely by their names, thus hard-coded class names are no longer needed, and it can support arbitrary policy selected by service config.
Ran
Yang Song
Kun Zhang
Carl Mastrangelo
NickUfer
Eric Anderson
Jihun Cho
Chengyuan Zhang
ZHANG Dapeng
Nick Hill
Tim van der Lippe
Nguyen Quang Huy
Dan
Bogdan Drutu
Michael Plump
kenji yoshida
Eric Gribkoff
Ignacio del Valle Alles
David Hoover
Grant Oakley
ST-DDT
zpencer
Rodrigo Queiro