The problem: GrpclbState tracks Subchannels' states as a mutable
attribute in Subchannel.getAttributes(). However, GrpclbState only
update this attribute for the Subchannels its managing. For those
cached in SubchannelPool, their state attributes are stale. When they
are given back to GrpclbState, IDLE state is assumed. As a result, if
a Subchannel is READY when it's reclaimed from the pool, it will not
be picked.
To fix that, this change expands SubchannelPool interface to handle
Subchannel state updates, which GrpclbState will call. SubchannelPool
saves the latest state and delivers it when it's returned to
GrpclbState by scheduling a call to handleSubchannelState() in the
SynchronizationContext, so that GrpclbState will take the latest state
as if it was just reported from the Channel.
Make the "logged" method be consistent, and refer to the public logging class name and method. This makes the log statements return the same class name used to set the log level.
Before:
```
190306 13:29:39.290:D 1 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Channel for 'localhost:10000' created
190306 13:29:39.414:D 1 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Exiting idle mode
190306 13:29:39.622:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Resolved address: [[[/127.0.0.1:10000]/{}]], config={}
190306 13:29:39.623:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Channel<1>: (localhost:10000)] Address resolved: [[[/127.0.0.1:10000]/{}]]
190306 13:29:39.624:D 17 [io.grpc.internal.ChannelTracer.logOnly] [Subchannel<3>] Subchannel for [[[/127.0.0.1:10000]/{}]] created
```
After:
```
190306 13:49:15.654:D 1 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Channel for 'localhost:10000' created
190306 13:49:15.772:D 1 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Exiting idle mode
190306 13:49:15.995:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Resolved address: [[[/127.0.0.1:10000]/{}]], config={}
190306 13:49:15.995:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Address resolved: [[[/127.0.0.1:10000]/{}]]
190306 13:49:15.997:D 18 [io.grpc.ChannelLogger.log] [Subchannel<3>] Subchannel for [[[/127.0.0.1:10000]/{}]] created
190306 13:49:15.999:D 18 [io.grpc.ChannelLogger.log] [Channel<1>: (localhost:10000)] Child Subchannel created
```
The PICK_FIRST mode puts all backend addresses in a single Subchannel. There are a few points where it's different from the default ROUND_ROBIN mode:
1. PICK_FIRST doesn't eagerly connect to backends like ROUND_ROBIN does. Instead, it requests for connections when the Subchannel is picked.
2. PICK_FIRST adds tokens to the headers via a different code path (`TokenAttachingTracerFactory`) than ROUND_ROBIN
3. For simple implementation, when the mode is changed by service config when the LoadBalancer is working, we will shut down `GrpclbState` and starts a new one with the new mode. All connections will be closed during the transition. We don't expect this to happen in practice given the specific use case of PICK_FIRST.
Make sure the config for grpclb is passed to the GrpclbLoadBalancer, which will support two child policies -- "round_robin" (default) and "pick_first".
Previously the presence of balancer addresses would dictate "grpclb" policy, despite of the service config. Service config will now take precedence instead.
Implement config parsing logic in GrpclbLoadBalancer. Per offline discussions with @markdroth and @ejona86, we will ignore configuration errors for now. The more appropriate config error handling is upcoming.
This commit swaps to using a Sync task to place generated code in the
src/generated folder instead of the gradle-protobuf-plugin's
generatedFilesBaseDir. This provides much nicer results on failed
builds, and you will no longer see all the generated files deleted.
But at the same time the Sync task makes it easy to only copy the
grpc-generated code. This was not previously done because we were lazy
and using generatedFilesBaseDir, which made it difficult to treat the
services differently from the messages.
This was added for the potential use case of needing to resolve target
names (of the same scheme as the top-level channel's target's) in the
LoadBalancer. Now actual use cases come up in xDS that we need to
resolve fully-qualified target strings with arbitrary schemes. This
method has never been used and won't fit future uses because it's too
restrictive.
This is needed for GRPCLB pick_first support, which needs to attach
tokens to headers, and the tokens are per server. In pick_first, all
addresses are in a single Subchannel, thus the LoadBalancer needs to
know which backend is used for a new stream.
This can be used by xds LoadBalancer to create a channel to the XDS
traffic director, as the service config will only specify the target
name of the balancer.
This PR only adds the interface to unblock the xds work.
Implementation would take some time thus will come later.
The LoadBalancingConfig message, which looks like
```json
{
"policy_name" : {
"config_key1" : "config_value1",
"config_key2" : "config_value2"
}
}
```
appears multiple times. It gets super tedious and confusing to handle, because both the whole config and the value (in the above example is `{ "config_key1" : "config_value1" }`) are just `Map<String, Object>`, and each user needs to do the following validation:
1. The whole config must have exactly one key
2. The value must be a map
Here I define `LbConfig` that holds the policy name and the config value, and a method in `ServiceConfigUtil` that converts the parsed JSON format into `LbConfig`.
There is also multiple cases where you need to handle a list of configs (top-level balancing policy, child and fallback policies in xds, grpclb child policies). I also made another helper method in `ServiceConfigUtil` to convert them into `List<LbConfig>`.
Found and fixed a bug in the xds code, where the top-level balancer should pass the config value (excluding the policy name), not the whole config to the child balancers. Search for "supported_1_option" in the diff to see it in the tests.
Also pass in the binary correctly as a tool. This is required for #5383.
I have tested the build with older Bazel versions and this doesn't
appear to affect compatibility, ie grpc-java continues to be compatible
with Bazel >=0.19.
For security, we should change http into https links.
Co-Authored-By: Nguyen Van Trung [trungnvfet@outlook.com](mailto:trungnvfet@outlook.com)
Signed-off-by: Nguyen Quang Huy [huynq0911@gmail.com](mailto:huynq0911@gmail.com)
In testing, I discovered that Alpine does in fact work. Update the
documentation with the error you might see if you are simply missing a
package (for Googling) as well as fixes.
JDK 11 is LTS and superceded JDK 9 and 10. Nobody should really be using JDK 9
(just like the already untested JDK 10) so let's not waste the CPU testing it.
As long as we support JDK 8 and 11 it is unlikely we break JDK 9, though.
added fallback handling
in addition:
- made XdsLbState not abstract for now
- did not include graceful swapping balancers when service config change, for now just shutdown the old one and use the new one.
This will be a new override. The old override is now deprecated.
In order to pass new information without adding new overrides, I shoved most information
into an object called StreamInfo. The Metadata is left out to draw attention because
it's mutable.
Motivation: this is needed for correctly supporting pick_first in GRPCLB. GRPCLB needs to
add a token to the headers, and the token varies among servers. With round_robin, GRPCLB
create a Subchannel for each server, thus can attach the token when the Subchannel is picked.
To implement pick_first, all server addresses will be put in a single Subchannel, we will
need to add the header in newClientStreamTracer(), by looking up the server address from
the transport attributes and deciding which token to add.
Resolves#2649
As a prerequisite, added `getSynchronizationContext()` to `NameResolver.Helper`.
`DnsNameResolver` has gone through a small refactor around the `Resolve` runnable, which makes it a little simpler.
...and move the `close()` method to ProtocolNegotiator rather than Handler.
Since this is a breaking change (for people who ignored our `@Internal` annotations), I wanted to make both changes in the same PR so as to fix them both at the same time.
* Import envoy proto file to the latest internal version, which has correct java proto options. (The PGV proto, `validate.proto`, doesn't have the correct and up-to-date java_package proto option yet, but as long as we don't use those generated classes, it seems fine.)
* Stop modifying java proto options by import.sh.
* Apply shadow plugin when publishing.
Eric Anderson
Kun Zhang
Carl Mastrangelo
AmiDavidW
ZHANG Dapeng
Liam Miller-Cushon
Rodrigo Queiro
Rodrigo Queiro
Nguyen Quang Huy
sanjaypujare
Jihun Cho
Eric Gribkoff
David Hoover
Dan