`io.grpc.Uri` is an implementation of RFC 3986 tailored for grpc-java's
needs. It lifts some of the limitations of `java.net.URI` that currently
prevent us from resolving target URIs like `intent:#Intent;...` See
#12244 for more.
Marked `@Internal` for now but the plan is to eventually use this to
replace `java.net.URI` in our public APIs such as NameResolver.Factory.
Allow users to specify custom HTTP headers when connecting through
an HTTP CONNECT proxy. This extends HttpConnectProxiedSocketAddress
with an optional headers field (Map<String, String>), which is
converted to Netty's HttpHeaders in the protocol negotiator.
This change is fully backward-compatible. Existing code without
headers continues to work as before.
Fixes#9826
implementing [gRFC A68](https://github.com/grpc/proposal/blob/master/A68-random-subsetting.md)
This change contains:
1. Usage of `murmur3_128` hashing algorithm from Guava library.
2. Implementation of `RandomSubsettingLoadBalancer` and
`RandomSubsettingLoadBalancerProvider` classes and integration into the
`util` project.
Since envoy extensions does not support `random_subsetting` LB policy
yet, xDS related changes will be introduced later. Envoy PR
[here](https://github.com/envoyproxy/envoy/pull/41758).
Fixes https://github.com/grpc/grpc-java/issues/12444
This PR addresses a bug in the `NameResolver.Listener` to
`NameResolver.Listener2` bridge affecting custom NameResolver
implementations using Listener.
The bridge in `NameResolver.start(Listener)` at
https://github.com/grpc/grpc-java/blob/master/api/src/main/java/io/grpc/NameResolver.java#L100
unconditionally calls `getValue()` on the `StatusOr`, throwing
`java.lang.IllegalStateException: No value present.` when the result
contains an error.
This was identified when upgrading from gRPC `v1.63.3` to `v1.75.0`.
The bug occurs due to `DnsNameResolver`'s error handling changes between
versions:
- `v1.63.3`: Errors reported via `Listener.onError()`
(https://github.com/grpc/grpc-java/blob/v1.63.x/core/src/main/java/io/grpc/internal/DnsNameResolver.java#L319)
- `v1.75.0`: Errors passed via `Listener2.onResult2()` with a
ResolutionResult containing either addresses OR an error
(https://github.com/grpc/grpc-java/blob/master/core/src/main/java/io/grpc/internal/DnsNameResolver.java#L322)
This PR updates the bridge to check whether `ResolutionResult` contains
addresses or an error. It passes the error via `onError` and addresses
via `onAddresses`.
**Reproducing the Issue**
The `startOnOldListener_resolverReportsError` test reproduces a similar
issue. It creates a custom `NameResolver` that reports errors through
the `ResolutionResult` like the `DNSNameResolver` in `v1.75.0`, passes
an old Listener to `resolver.start`, which triggers the bridge code
path.
Without the fix, the bridge calls `getValue()` on the error containing
`StatusOr`, throwing `IllegalStateException: No value present`.
With the fix, the bridge checks `hasValue()` first and correctly routes
to `listener.onError()` when appropriate. This ensures backward
compatibility for `Listener` implementations when resolvers report
errors via `ResolutionResult`.
It's discouraged in modern Bazel to use native rules and will make it
difficult for modules dependent on grpc-java to fully migrate to new
versions of Bazel (like grpc-kotlin).
For example, try building this repo using:
```bash
$ bazelisk build ... --incompatible_autoload_externally=
```
It will fail.
**IMPORTANT**: Now you still see errors after this commit, but it's
better because the errors come from `@protobuf`, not `@grpc-java`.
Modifying existing Attributes was virtually guaranteed to allocate
within build(), just because `base` was not considered for the new map
size. Discard was also allocation-heavy because it often created a new
map. Using a regular copy-on-write approach is enough to avoid the
unnecessary allocations in both cases.
This was noticed in a profile that included xds's
AddressFilter.setPathFilter(), where Attributes.Builder.build()
allocated 6x the memory of Attributes.Builder.data().
b/435208946#comment41
Fix HashSet / HashMap initializations to have sufficient capacity allocated based on the number of keys to be inserted, without which it would always lead to a rehash / resize operation.
LoadBalancers shouldn't be called after shutdown(), but RingHashLb could
have enqueued work to the SynchronizationContext that executed after
shutdown(). This commit fixes problems discovered when auditing all LBs
usage of the syncContext for that type of problem.
Similarly, PickFirstLb could have requested a new connection after
shutdown(). We want to avoid that sort of thing too.
RingHashLb's test changed from CONNECTING to TRANSIENT_FAILURE to get
the latest picker. Because two subchannels have failed it will be in
TRANSIENT_FAILURE. Previously the test was using an older picker with
out-of-date subchannelView, and the verifyConnection() was too imprecise
to notice it was creating the wrong subchannel.
As discovered in b/430347751, where ClusterImplLb was seeing a new
subchannel being called after the child LB was shutdown (the shutdown
itself had been caused by RingHashConfig not implementing equals() and
was fixed by a8de9f07ab, which caused ClusterResolverLb to replace its
state):
```
java.lang.NullPointerException
at io.grpc.xds.ClusterImplLoadBalancer$ClusterImplLbHelper.createClusterLocalityFromAttributes(ClusterImplLoadBalancer.java:322)
at io.grpc.xds.ClusterImplLoadBalancer$ClusterImplLbHelper.createSubchannel(ClusterImplLoadBalancer.java:236)
at io.grpc.util.ForwardingLoadBalancerHelper.createSubchannel(ForwardingLoadBalancerHelper.java:47)
at io.grpc.util.ForwardingLoadBalancerHelper.createSubchannel(ForwardingLoadBalancerHelper.java:47)
at io.grpc.internal.PickFirstLeafLoadBalancer.createNewSubchannel(PickFirstLeafLoadBalancer.java:527)
at io.grpc.internal.PickFirstLeafLoadBalancer.requestConnection(PickFirstLeafLoadBalancer.java:459)
at io.grpc.internal.PickFirstLeafLoadBalancer.acceptResolvedAddresses(PickFirstLeafLoadBalancer.java:174)
at io.grpc.xds.LazyLoadBalancer$LazyDelegate.activate(LazyLoadBalancer.java:64)
at io.grpc.xds.LazyLoadBalancer$LazyDelegate.requestConnection(LazyLoadBalancer.java:97)
at io.grpc.util.ForwardingLoadBalancer.requestConnection(ForwardingLoadBalancer.java:61)
at io.grpc.xds.RingHashLoadBalancer$RingHashPicker.lambda$pickSubchannel$0(RingHashLoadBalancer.java:440)
at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:96)
at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:128)
at io.grpc.xds.client.XdsClientImpl$ResourceSubscriber.onData(XdsClientImpl.java:817)
```
ClusterResolverLb gets the NameResolverRegistry from
LoadBalancer.Helper, so a new API was added in NameResover.Args to
propagate the same object to the name resolver tree.
RetryingNameResolver was exposed to xds. This is expected to be
temporary, as the retrying is being removed from ManagedChannelImpl and
moved into the resolvers. At that point, DnsNameResolverProvider would
wrap DnsNameResolver with a similar API to RetryingNameResolver and xds
would no longer be responsible.
Contributes to the gRFC A74 effort.
https://github.com/grpc/proposal/blob/master/A74-xds-config-tears.md
The alternative to using Mockito's ArgumentMatcher is to use Hamcrest.
However, Hamcrest did not impress me. ArgumentMatcher is trivial if you
don't care about the error message.
This fixes a pre-existing issue where ConfigSelector.releaseCluster
could revert the LB config back to using cluster manager after releasing
all RPCs using a cluster have committed.
Co-authored-by: Larry Safran <lsafran@google.com>
In e08b9db20 we added `@DoNotCall` annotations to some call sites, but
Bazel used an older version of ErrorProne that complained at times it
shouldn't. The minimum version of Bazel we test/support is now Bazel 6,
well past Bazel 3.4+.
This avoids the dependency on animalsniffer-annotations. grpc-api, and
particularly grpc-context, are used many low-level places and it is
beneficial for them to be very low dependency. This brings grpc-context
back to zero-dependency.
grpc-binder's upcoming AndroidIntentNameResolver needs to know the target Android user so it can resolve target URIs in the correct place. Unfortunately, Android's built in intent:// URI scheme has no way to specify a user and in fact the android.os.UserHandle object can't reasonably be encoded as a String at all.
We solve this problem by extending NameResolver.Args with the same type-safe and domain-specific Key<T> pattern used by CallOptions, Context and CreateSubchannelArgs. New "custom" arguments could apply to all NameResolvers of a certain URI scheme, to all NameResolvers producing a particular type of java.net.SocketAddress, or even to a specific NameResolver subclass.
In 61f19d707a I swapped the signatures to use the version catalog. But I
failed to preserve the `@signature` extension and it all seemed to
work... But in fact all the animalsniffer tasks were completing as
SKIPPED as they lacked signatures. The build.gradle changes in this
commit are to fix that while still using version catalog.
But while it was broken violations crept in. Most violations weren't
too important and we're not surprised went unnoticed. For example, Netty
with TLS has long required the Java 8 API
`setEndpointIdentificationAlgorithm()`, so using `Optional` in the same
code path didn't harm anything in particular. I still swapped it to
Guava's `Optional` to avoid overuse of `@IgnoreJRERequirement`.
One important violation has not been fixed and instead I've disabled the
android signature in api/build.gradle for the moment. The violation is
in StatusException using the `fillInStackTrace` overload of Exception.
This problem [had been noticed][PR11066], but we couldn't figure out
what was going on. AnimalSniffer is now noticing this and agreeing with
the internal linter. There is still a question of why our interop tests
failed to notice this, but given they are no longer running on pre-API
level 24, that may forever be a mystery.
[PR11066]: https://github.com/grpc/grpc-java/pull/11066
This was noticed because of a CallOptionsTest flake that had a
surprising error:
```
expected : 59.983387319
but was : 59.983387319
outside tolerance in seconds: 0.01
```
When forwarding from Listener onAddresses to Listener2 continue to use onResult and not onResult2 because the latter requires to be called from within synchronization context and it breaks existing code that didn't need to do so when using the old Listener interface.
It is the `Executor appExecutor` that should be given an asynchronous
task, not `CallCredentials.MetadataApplier applier`.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Combined success / error status passed via ResolutionResult to the NameResolver.Listener2 interface's onResult2 method - Addresses in the success case or address resolution error in the failure case now get set in ResolutionResult::addressesOrError by the internal name resolvers.
* Change PickFirstLeafLoadBalancer to only have 1 subchannel at a time if environment variable GRPC_SERIALIZE_RETRIES == true.
Cache serializingRetries value so that it doesn't have to look up the flag every time.
Clear the correct task when READY in processSubchannelState and move the logic to cancelScheduledTasks
Cleanup based on PR review
remove unneeded checks for shutdown.
* Fix previously broken tests
* Shutdown previous subchannel when run off end of index.
* Provide option to disable subchannel retries to let PFLeafLB take control of retries.
* InternalSubchannel internally goes to IDLE when sees TF when reconnect is disabled.
Remove an extra index.increment in LeafLB
Bazel had the dependency added because of #5046, where Guava was
depending on it as compile-only and Bazel build have "unknown enum
constant" warnings. Guava now has a compile dependency on j2objc, so
this workaround is no longer needed. There are currently no version skew
issues in Gradle, which was the only usage.
This reverts commit 9ba2f9dec5.
It causes a channel panic due to unimplemented onResult2().
```
java.lang.UnsupportedOperationException: Not implemented.
at io.grpc.NameResolver$Listener2.onResult2(NameResolver.java:257)
at io.grpc.internal.DnsNameResolver$Resolve.lambda$run$0(DnsNameResolver.java:334)
at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:94)
at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:126)
at io.grpc.internal.DnsNameResolver$Resolve.run(DnsNameResolver.java:333)
```
b/356669977
They share very little code, and we really don't want RoundRobinLb to be
public and non-final. Originally, WRR was expected to share much more
code with RR, and even delegated to RR at times. The delegation was
removed in 111ff60e. After dca89b25, most of the sharing has been moved
out into general-purpose tools that can be used by any LB policy.
FixedResultPicker now has equals to makes it as a EmptyPicker
replacement. RoundRobinLb still uses EmptyPicker because fixing its
tests is a larger change. OutlierDetectionLbTest was changed because
FixedResultPicker is used by PickFirstLeafLb, and now RoundRobinLb can
squelch some of its updates for ready pickers.
John Cormie