This add perfmark annotations in some key places, notably on transport/application boundaries, and thread hop locations. Perfmark records to a thread-local buffer the events that happen in each thread. Perfmark is disabled by default, and will compile to a noop unless Perfmark.setEnabled is invoked. This should make it free when disable, and pretty fast when it is enabled.
It is important that started tasks are ended, so several places in our code are moved to either try-finally blocks, or moved into a private method. I realize this is ugly, but I think it is manageable. In the future, we can look at making an agent or compiler plugin that simplifies the recording.
Linking between threads is done with a Link object, which is created on the "outbound" task, and used on the "inbound" task. This is slightly more verbose, and does has a small amount of runtime overhead, even when disabled. (for null checks, slightly higher memory usage, etc.) I think this is okay to, because it makes other optimizations much easier.
Previously, AutoConfiguredLoadBalancer was also handling it, but it
doesn't trigger retries. By returning true for
canHandleEmptyAddressListFromNameResolution(),
AutoConfiguredLoadBalancer effectively by-passed the empty-result
handling logic in ManagedChannelImpl, thus resolution retries were
never triggered.
This change requires AutoConfiguredLoadBalancer to stop being a
LoadBalancer, for its tryHandleResolvedAddresses(). It doesn't
cause any trouble because AutoConfiguredLoadBalancer has become
less and less like a LoadBalancer during the service config changes.
As mentioned in 5188[1], the default used with the enableKeepAlive API
conflicted with the default server enforcement. Instead of fixing it,
remove it. These APIs were deprecated in v1.3.0 in April 2017.
1. https://github.com/grpc/grpc-java/issues/5188#issuecomment-482269303
The `method` and `status` shouldn't be propagated in the first place,
but in previous OpenCensus implementation all tags are propagating by
default. Now with the TagMetadata it may make sense to change them to
local tags.
This will be a breaking change to users who depend on the behavior that
these tags propagate through process boundaries.
The old createSubchannel() doesn't require being called from
sync-context, thus its implementation schedules start() on
sync-context because start() has such requirement. However, if a
LoadBalancer call createSubchannel() in sync-context, start() is
queued and only called after the control exits the sync-context. If
createSubchannel() is immediately followed by other Subchannel methods
that requires start(), such as requestConnection(), they will fail.
This will be a very common issue as most LoadBalancers call
createSubchannel() in the sync-context.
This fix splits out the real work of start() into internalStart() which
is called by the old createSubchannel().
Previously PickResult's Subchannel must be the actual implementation
returned from the Channel's Helper, and Channel would cast it to the
implementation class in order to use it. This will be broken if
Subchannel is wrapped in the case of hierarchical LoadBalancers.
getInternalSubchannel() is the guaranteed path for the Channel to get
the InternalSubchannel implementation. It is friendly for wrapping.
Background: #5676
The pick_first policies in core and grpclb previously would call
Subchannel.requestConnection() from data-path. They now will schedule
that call in the sync-context to avoid the warning. They will only
call it for the first pick of each picker, to prevent storming the
sync-context.
This is a revised version of #5503 (62b03fd), which was rolled back in f8d0868. The newer version passes SubchannelStateListener to Subchannel.start() instead of SubchannelCreationArgs, which allows us to remove the Subchannel argument from the listener, which works as a solution for #5676.
LoadBalancers that call the old createSubchannel() will get start() implicitly called with a listener that passes updates to the deprecated LoadBalancer.handleSubchannelState(). Those who call the new createSubchannel() will have to call start() explicitly.
GRPCLB code is still using the old API, because it's a pain to migrate the SubchannelPool to the new API. Since CachedSubchannelHelper is on the way, it's easier to switch to it when it's ready. Keeping
GRPCLB with the old API would also confirm the backward compatibility.
Contrary to #5736, we will still keep the sync-context requirement of
requestConnection(), because it prevents API fragmentation.
PickFirstLoadBalancer is the only known violator. We will fix it on
master, but we don't want to make that change on 1.21.x because the
release is soon. We simply remove the warning in this release so that
users won't be annoyed.
This supersedes #5736
We will require Subchannel.requestConnection() to be called from
sync-context (#5722), but SubchannelPicker.requestConnection() is
currently calling it with the assumption of thread-safety. Actually
SubchannelPicker.requestConnection() is called already from
sync-context by ChannelImpl, it makes more sense to move this method
to LoadBalancer where all other methods are sync-context'ed, rather than
making SubchannelPicker.requestConnection() sync-context'ed and fragmenting
the SubchannelPicker API because pickSubchannel() is thread-safe.
C++ also has the requestConnection() equivalent on their LoadBalancer
interface.
We check for idle mode the first time we try newStream(), but failed to when
newStream races with reprocess(). This would normally be a very rare race,
except when you consider that AbstractChannelBuilder will call
managedChannel.enterIdle() when the network changes.
Fixes#5729
I see more cases of wrapping Helper and Subchannel during the work of
XdsLoadBalancer, we will require that all methods that involve mutable
state to be called from the Synchronization Context. We will start
logging warnings first, and make them throw in a future release.
Helper.createSubchannel() is already doing so. This change adds
warnings to the other eligible methods.
https://github.com/grpc/grpc-java/issues/5015
This follows the sort of changes we've done in the past, where the '2'
implies "version 2." We can end up reclaiming the original name if we
wish in the future.
The main reason for this change is to avoid changing to Observer since
the rest of io.grpc consistently uses Listener.
Also updated CensusModule to use the new helper methods ContextUtils.withValue() instead of directly manipulating the context keys. See census-instrumentation/opencensus-java#1864.
Having two Helpers (the other being LoadBalancer.Helper) is proven to
be confusing, and the one in NameResolver is fundamentally different
from the one from LoadBalancer, as the latter mutates the states while
the former doesn't. Renaming it to Args make it less awkward to
expose it to LoadBalancer, which is needed for creating NameResolvers
for per-locality routing in the XdsLoadBalancer.
As we are now endorsing the wrapping of ClientStreamTracers by
providing ForwardingClientStreamTracer, there is a need for altering
StreamInfo, especially CallOptions before it's passed onto the
delegate. A Builder class and a toBuilder() provides a robust way
to copy the rest of the fields.
This is a breaking change for anybody who creates StreamInfo, which is
unlikely in non-test code, because StreamInfo was added as late as
1.20.0.
NameResolverRegistry takes on all the logic previously in
NameResolverProvider. But it also allows manual registration of
NameResolvers, which is useful when the providers have complex
construction or need objects injected into them.
This also avoids a circular dependency during class loading since
previously loading any Provider searched for all Providers via
ClassLoader since ClassLoader handling was static within the parent
class.
Fixes#5562
io.grpc has fewer dependencies than io.grpc.internal. Moving it to a
separate artifact lets users use the API without bringing in the deps.
If the library has an optional dependency on grpc, that can be quite
convenient.
We now version-pin both grpc-api and grpc-core, since both contain
internal APIs.
I had to change a few tests in grpc-api to avoid FakeClock. Moving
FakeClock to grpc-api was difficult because it uses
io.grpc.internal.TimeProvider, which can't be moved since it is a
production class. Having grpc-api's tests depend on grpc-core's test
classes would be weird and cause a circular dependincy. Having
grpc-api's tests depend on grpc-core is likely possible, but weird and
fairly unnecessary at this point. So instead I rewrote the tests to
avoid FakeClock.
Fixes#1447
Resolves#5497
## Motivation
In hierarchical `LoadBalancer`s (e.g., `XdsLoadBalancer`) or wrapped `LoadBalancer`s (e.g., `HealthCheckingLoadBalancerFactory`, the top-level `LoadBalancer` receives `Subchannel` state updates from the Channel impl, and they almost always pass it down to its children `LoadBalancer`s.
Sometimes the children `LoadBalancer`s are not directly created by the parent, thus requires whatever API in the middle to also pass Subchannel state updates, complicating that API. For example, the proposed [`RequestDirector`](https://github.com/grpc/grpc-java/issues/5496#issuecomment-476008051) includes `handleSubchannelState()` solely to plumb state updates to where they are used. We also see this pattern in `HealthCheckingLoadBalancerFactory`, `GrpclbState` and `SubchannelPool`.
Another minor issue is, the parent `LoadBalancer` would need to intercept the `Helper` passed to its children to map Subchannels to the children `LoadBalancer`s, so that it pass updates about relevant Subchannels to the children. Otherwise, a child `LoadBalancer` may be surprised by seeing Subchannel not created by it, and it's not efficient to broadcast Subchannel updates to all children.
## API Proposal
We will pass a `SubchannelStateListener` when creating a `Subchannel` to accept state updates, those updates could be directly passed to where the `Subchannel` is created, skipping the explicit chaining in the middle.
Also define a first-class object `CreateSubchannelArgs` to pass arguments for the reasons below:
1. It may avoid breakages when we add new arguments to `createSubchannel()`. For example, a `LoadBalancer` may wrap `Helper` and intercept `createSubchannel()` in a hierarchical case. It may not be interested in all arguments. Passing a single `CreateSubchannelArgs` will not break the parent `LoadBalancer` if we add new fields later.
2. This also reduces the eventual size of Helper interface, as the convenience `createSubchannel()` that accepts one EAG instead of a List is no longer necessary, since that convenience is moved into `CreateSubchannelArgs`.
```java
interface SubchannelStateListener {
void onSubchannelState(Subchannel subchannel, ConnectivityStateInfo newState);
}
abstract class LoadBalancer.Helper {
abstract Subchannel createSubchannel(CreateSubchannelArgs args);
}
final class CreateSubchannelArgs {
final List<EquivalentAddressGroup> getAddresses();
final Attributes getAttributes();
final SubchannelStateListener getStateListener();
final class Builder () {
...
}
}
```
The new `createSubchannel()` must be called from synchronization context, as a step towards #5015.
## How the new API helps
Most hierarchical `LoadBalancer`s would just let the listener from the child `LoadBalancer`s directly pass through to gRPC core, which is less boilerplate than before.
Without any effort by the parent, each child will only see updates for the Subchannels it has created, which is clearer and more efficient.
If a parent `LoadBalancer` does want to look at or alter the Subchannel state updates for its delegate (like in `HealthCheckingLoadBalancerFactory`), it can still do so in the wrapping `LoadBalancer.Helper` passed to the delegate by intercepting the `SubchannelStateListener`.
## Migration implications
Existing `LoadBalancer` implementations will continue to work, while they will see deprecation warnings when compiled:
- The old `LoadBalancer.Helper#createSubchannel` variants are now deprecated, but will work until they are deleted. They create a `SubchannelStateListener` that delegates to `LoadBalancer#handleSubchannelState`.
- `LoadBalancer#handleSubchannelState` is now deprecated, and will throw if called and the implementation doesn't override it. It will be deleted in a future release.
The migration for most `LoadBalancer` implementation would be moving the logic from `LoadBalancer#handleSubchannelState` into a `SubchannelStateListener`.
This class is used in other places than just NameResolver.Helper. It
should not be an inner class of Helper.
Strictly speaking this is an API-breaking change. However, this is
part of the service config error handling API that hasn't been done
yet. Nobody has a legitimate reason to use it.
We assumed that we were the only caller. Turns out there are
forwarding NameResolver too. Implementing the old override will give
them time to migrate to the new API.
Resolves#5556
Both the config and the error need to be passed to the managed channel so it can decide to keep or reject the config. Passing only the parsed object means the managed channel cannot implement the error handling gRFC.
This code is highly experimental, so we can change it at will later. This PR is to go ahead and try it out.
Perfmark task calls are added to the client and server calls public APIs, recording when the calls begin and end. They use separate scope IDs (roughly these are Thread IDs) for the listener and the call due to no synchronization between them. However, they both use the same PerfTags, which allows them to be associated.
In the future, we can plumb the tag down into the stream to include deeper information about whats going on in a call.
Make the "logged" method be consistent, and refer to the public logging class name and method. This makes the log statements return the same class name used to set the log level.
Before:
```
190306 13:29:39.290:D 1 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Channel for 'localhost:10000' created
190306 13:29:39.414:D 1 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Exiting idle mode
190306 13:29:39.622:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Resolved address: [[[/127.0.0.1:10000]/{}]], config={}
190306 13:29:39.623:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Address resolved: [[[/127.0.0.1:10000]/{}]]
190306 13:29:39.624:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Subchannel<3>] Subchannel for [[[/127.0.0.1:10000]/{}]] created
```
After:
```
190306 13:49:15.654:D 1 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Channel for 'localhost:10000' created
190306 13:49:15.772:D 1 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Exiting idle mode
190306 13:49:15.995:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Resolved address: [[[/127.0.0.1:10000]/{}]], config={}
190306 13:49:15.995:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Address resolved: [[[/127.0.0.1:10000]/{}]]
190306 13:49:15.997:D 18 [io.grpc.ChannelLogger.log] [Subchannel<3>] Subchannel for [[[/127.0.0.1:10000]/{}]] created
190306 13:49:15.999:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Child Subchannel created
```
Make sure the config for grpclb is passed to the GrpclbLoadBalancer, which will support two child policies -- "round_robin" (default) and "pick_first".
Previously the presence of balancer addresses would dictate "grpclb" policy, despite of the service config. Service config will now take precedence instead.
Implement config parsing logic in GrpclbLoadBalancer. Per offline discussions with @markdroth and @ejona86, we will ignore configuration errors for now. The more appropriate config error handling is upcoming.
This was added for the potential use case of needing to resolve target
names (of the same scheme as the top-level channel's target's) in the
LoadBalancer. Now actual use cases come up in xDS that we need to
resolve fully-qualified target strings with arbitrary schemes. This
method has never been used and won't fit future uses because it's too
restrictive.
This is needed for GRPCLB pick_first support, which needs to attach
tokens to headers, and the tokens are per server. In pick_first, all
addresses are in a single Subchannel, thus the LoadBalancer needs to
know which backend is used for a new stream.
This can be used by xds LoadBalancer to create a channel to the XDS
traffic director, as the service config will only specify the target
name of the balancer.
This PR only adds the interface to unblock the xds work.
Implementation would take some time thus will come later.
The LoadBalancingConfig message, which looks like
```json
{
"policy_name" : {
"config_key1" : "config_value1",
"config_key2" : "config_value2"
}
}
```
appears multiple times. It gets super tedious and confusing to handle, because both the whole config and the value (in the above example is `{ "config_key1" : "config_value1" }`) are just `Map<String, Object>`, and each user needs to do the following validation:
1. The whole config must have exactly one key
2. The value must be a map
Here I define `LbConfig` that holds the policy name and the config value, and a method in `ServiceConfigUtil` that converts the parsed JSON format into `LbConfig`.
There is also multiple cases where you need to handle a list of configs (top-level balancing policy, child and fallback policies in xds, grpclb child policies). I also made another helper method in `ServiceConfigUtil` to convert them into `List<LbConfig>`.
Found and fixed a bug in the xds code, where the top-level balancer should pass the config value (excluding the policy name), not the whole config to the child balancers. Search for "supported_1_option" in the diff to see it in the tests.
This will be a new override. The old override is now deprecated.
In order to pass new information without adding new overrides, I shoved most information
into an object called StreamInfo. The Metadata is left out to draw attention because
it's mutable.
Motivation: this is needed for correctly supporting pick_first in GRPCLB. GRPCLB needs to
add a token to the headers, and the token varies among servers. With round_robin, GRPCLB
create a Subchannel for each server, thus can attach the token when the Subchannel is picked.
To implement pick_first, all server addresses will be put in a single Subchannel, we will
need to add the header in newClientStreamTracer(), by looking up the server address from
the transport attributes and deciding which token to add.
Resolves#2649
As a prerequisite, added `getSynchronizationContext()` to `NameResolver.Helper`.
`DnsNameResolver` has gone through a small refactor around the `Resolve` runnable, which makes it a little simpler.
This reverts commit d47379947f.
This caused test failures internally, where gRPC failed with
"IllegalArgumentException: Invalid trace name". Not only was this
failure unexpected, it was also weird that it failed with an
IllegalArgumentException instead of the normal StatusRuntimeException.
Context: [#4159 (comment)](https://github.com/grpc/grpc-java/issues/4159#issuecomment-415827641)
`Attributes` is appropriate for plumbing optional objects, especially useful for a long plumbing path where components in the middle may not care or see all objects in the container. It's not the case for the `params` on `newNewResolver()`. Both the default port and the proxy detector are guaranteed to be there and the plumbing path is very short. In this case, a first-class object is more appropriate and easier to use.
The `Helper` will also have `getSynchronizationContext()` (#2649) and a method to parse and validate service config. We we also considering merging `Listener` into the `Helper`, to make `NameResolver` match the `LoadBalancer` API.
Combine the public `ProxyParameters` and the internal `ProxySocketAddress` as `HttpConnectProxiedSocketAddress`. The more specific name signifies the type of the proxy we currently support, and makes room for other proxy types (e.g., SOCKS) in the future. The combination simplifies NameResolver implementation.
Introduce `ProxiedSocketAddress` as the base class that is returned by `ProxyDetector`, mainly for clarification and documentation. Added documentation about proxy in general on `ProxyDetector`.
- defined XdsLbState, playing a similar role to GrpclbState
- there are two modes of XdsLbState: STANDARD and CUSTOM
- on `XdsLoadBalancer.handleResolvedAddressGroups()`, the `xdsLoadBalancer` will update the `xdsLbState` based on the lb config in the attributes passed in
In Mockito 2, if a method expects a primitive type, but an any(<Primitive>.class) matcher is used in its place, it will throw an error. To prepare for this upcoming breakage, change
all existing any(<Primitive>.class) matchers to use the correct any<Primitive>() matcher.
This fixes two races: a data race where scheduledRetry is accessed
by cancel() and confusion where scheduledRetry could be set to null by
the schedule()d runnable before it is set by the schedule() return
value.
Although it seems these races can't actually cause problems due to other
conditions/constraints, it's hard to reason about. So let's plug these
preemptively, even if we can't add tests that trigger them.
ScheduledHedging was not specific to hedging, so can now be reused for
this retry case. It was renamed to avoid being misleading.
This avoids a memory leak when the channel itself participates in a
reference cycle (e.g., when an interceptor retains a reference to an
Android app's context). With the current implementation, the static
`ManagedChannelOrphanWrapper.refs` map will keep the channel reachable
and prevent the ref cycle from being GCed.
SRV has not yet been enabled in a release. Since work is rapidly
underway to replace GRPC-LB with a service config+XDS-based solution,
there's now thoughts that we won't ever enable grpclb by default (but
may allow it to be automatically enabled when using GoogleDefaultChannel
or similar). Since things are being worked out, disable it.
This effectively reverts c729a0f.
This is the 3rd step of #4901
- The deprecated `CC.applyRequestMetadata(... Attributes ...)` is now **replaced** by the new API `CC.applyRequestMetadata(... CC.MetadataApplier ...)` transformed from `CC2.applyRequestMetadata(... CC2.MetadataApplier ...)`.
- The Attributes keys in `CallCredentials` were deprecated, and now deleted.
- The deprecated interface `CC.MetadataApplier` is **replaced** by an equivalent abstract class.
- `CallCredentials2` is now marked as deprecated, while keeping its interface intact so that it won't break current implementations that are still on `CallCredentials2`.
- From this point on, implementations should do a one-line change from `extends CallCredentials2` to `extends CallCredentials`
- `GoogleAuthLibraryCallCredentials` is kept as `CallCredentials2` for now, as there is an internal consumer that expects it to be `CallCredentials2`.
We've been on newer versions of Guava for a while now; these no longer
do anything.
Reworded the comment for Stopwatch.createUnstarted(), because it is not
safe (it doesn't matter if the method isn't marked Beta; you have to use
Ticker), except for the fact it is only used in our tests.
* makes Census tracing factories at the end of the user added ones
* makes more vars in AbstractServerImplBuilder package private
* annotates methods in ASIB to be clearer
* simplifies several of the setters to be single line
* Makes the generics on the Tracer factories proper
Currently ManagedChannelImpl will interpret empty address list from
NameResolver as an error, and LoadBalancer will NEVER receive an empty
list from handleResolvedAddressGroups(). There is a case in the
request-routing design where a LoadBalancer only receives service
config with which it constructs children NameResolver/LoadBalancer
pairs, thus no address is expected at this level.
canHandleEmptyAddressListFromNameResolution() is a signal to Channel
(and to the parent LoadBalancer in case of hierachical LoadBalancers)
about whether it accepts empty address lists. The default is false,
which is the current behavior.
The logic is currently duplicated in ManagedChannelImpl and
AutoConfiguredLoadBalancer. The one in ManagedChannelImpl will be
removed once we delete ManagedChannelBuilder.loadBalancerFactory() as
AutoConfiguredLoadBalancer will always be the top-level LoadBalancer
by then.
Previously, overwriting an existing Key would cause the original CallOptions instance to also be mutated.
See #5142
Also adds a regression test for this issue.
ManagedChannelBuilder.loadBalancingFactory() overrides the proper
policy selection logic implemented in AutoConfiguredLoadBalancer, thus
has problems in cases where NameResolver returns balancer addresses,
because custom LoadBalancers normally don't differentiate between
normal server addresses with balancer addresses. The policy selection
logic will filter out balancer addresses.
When service owner turns on grpclb through service config, it
shouldn't break existing clients that don't have grpclb in their
classpath.
Resolves#4602
There are currently three boolean flags, and there will be one more
soon. Put them all in the top-level class instead of passing them as
arguments on lower levels.
Carl Mastrangelo
ZHANG Dapeng
Ran
Eric Anderson
Jihun Cho
Kun Zhang
Yang Song
Chengyuan Zhang
NickUfer
Nick Hill
Tim van der Lippe
Nguyen Quang Huy
Dan
Bogdan Drutu
Michael Plump
kenji yoshida
Eric Gribkoff
Ignacio del Valle Alles
David Hoover
Grant Oakley
ST-DDT