Commit Graph

790 Commits

Author SHA1 Message Date
Terry Wilson 53a2d50695
xds: always update priority LB connectivity state (#9527)
Removes the option of skipping the update of the priority LB state when
the failover timer is pending.

This consistency facilitates a future change weher we delay child LB
status updates if the priority LB is performing an update. The upcoming
priority LB policy gRFC also does not require this update to ever be
skipped.
2022-09-08 08:49:07 -07:00
Terry Wilson d4cd926c96
core: Enable outlier detection by default. (#9479) 2022-08-23 11:32:10 -07:00
Terry Wilson 9ed5a1bbbf
xds: Fix outlier detection env flag name. (#9462) 2022-08-21 16:13:31 -07:00
Eric Anderson 618a4de705
xds: CHANNEL_ID hash should be random for the channel
The log id is an incrementing value starting from 0. That means the same
binary on two different machines will have the same hashes for each
consecutive Channel. That was not at all the intension of CHANNEL_ID.
From gRFC A42:

> This can be used in similar situations to where Envoy uses
> connection_properties to hash on the source IP address.
2022-08-18 17:27:15 -07:00
Terry Wilson 81abb21e7f
xds: Configure outlier detection. (#9456)
Enables the new OutlierDetectionLoadBalancer when outlier detection is enabled
in the xDS cluster configuration.
2022-08-18 16:06:17 -07:00
Larry Safran 01d5bd47cb
Cleanup some of the warnings across the code base (#9445)
No logic changes, just cleans up warnings to make spotting real problems easier.

Remove "public" declarations on interfaces
Remove duplicate semicolons (Java lines ending in ";;")
Remove unneeded import
Change non-javadoc comment to not start with "/**"
Remove unneeded explicit type declarations from generics
Fix broken javadoc links
2022-08-15 11:06:31 -07:00
Eric Anderson 61f19d707a
Swap Animalsniffer to Java 8 and Android 19
Also added missing signatures. Swapping to version catalog will make
this process easier in the future.
2022-08-10 12:41:57 -07:00
Eric Anderson 2b50e405b1 core: Remove LB refreshNameResolver check
It's been 17 months since the check was introduced, which is plenty for
the migration. Leaving ignoreRefreshNameResolutionCheck() in-place to
let users delete their call sites. We'll remove the method after a few
releases.

Fixes #9409
2022-08-10 10:26:15 -07:00
Eric Anderson db320cefc1
repositories.bzl: Use valid target name for services/xds
This fixes builds including dependencies from Maven that use
io.grpc:grpc-services or io.grpc:grpc-xds. It resolves this error:
```
no such target '@io_grpc_grpc_java//services:services': target 'services' not declared in package 'services' defined by services/BUILD.bazel and referenced by '@maven//:io_grpc_grpc_services'
```

Fixes #9419
2022-08-02 10:28:44 -07:00
Sergii Tkachenko e26b6ee1d7
xds: Refactor and document `ClientXdsClient.handleResourceUpdate()` (#9377)
- Reduce nesting level by using `continue`
- Rearrange the order when it's possible to bail out early
- Add comments explaining what case it is, and the logic behind it
2022-07-29 16:55:27 -07:00
Terry Wilson 7665d3850b
Revert "Add LoadBalancer.acceptResolvedAddresses() (#8742)" (#9414)
This reverts commit 70a29fbfe3.
2022-07-28 12:15:26 -07:00
Terry Wilson 70a29fbfe3
Add LoadBalancer.acceptResolvedAddresses() (#8742)
Introduces a new acceptResolvedAddresses() to the LoadBalancer.

This will now be the preferred way to handle addresses from the NameResolver. The existing handleResolvedAddresses() will eventually be deprecated.

The new method returns a boolean based on the LoadBalancers ability to use the provided addresses. If it does not accept them, false is returned. LoadBalancer implementations using the new method should no longer implement the canHandleEmptyAddressListFromNameResolution(), which will eventually be removed, along with handleResolvedAddresses().

Backward compatibility will be maintained so existing load balancers using handleResolvedAddresses() will continue to work.

Additionally the previously deprecated handleResolvedAddressGroups() method is removed.
2022-07-26 09:23:37 -07:00
yifeizhuang 5f1a1d4f35
service: CalMetricRecorder.recordCallMetric is deprecated, use CalMetricRecorder.recordRequestCostMetric (#9410) 2022-07-25 13:10:29 -07:00
sanjaypujare 1fe3ed9b53
xds: use predefined XdsLbPolicies constants instead of string literals for consistency (#9408) 2022-07-25 22:35:08 +05:30
yifeizhuang 027d36eee7
xds: xdsNameResolver match channel overrideAuthority in virtualHost matching (#9405) 2022-07-22 12:41:16 -07:00
Terry Wilson 4850ad219e
xds: ClusterManager LB state/picker update fix (#9404)
* xds: ClusterManager LB state/picker update fix

Correctly set the currentState and curentPicker when the
child LB updates balancing state
2022-07-21 13:54:40 -07:00
Eric Anderson 0e45e04041
Avoid accidental locale-sensitive String.format()
%s is fairly safe (requires a Formattable to use Locale), so %d is the
main risk item. Places that really didn't need to use String.format()
were converted to plain string concatenation. Logging locations were
generally converted to using the log infrastructure's delayed
formatting, which is generally locale-sensitive but we're okay with
that. That wasn't done in okhttp, however, because Android frequently
doesn't use MessageFormat so we'd lose the parameters. Everywhere else
was explicitly defined to be Locale.US, to be consistent independent of
the default system locale.
2022-07-19 14:41:34 -07:00
yifeizhuang 756fdf3f2c
service: make the orca MetricReport a top level experimental class (#9382) 2022-07-18 13:23:55 -07:00
yifeizhuang 6609f11f48
xds: do not expose orca proto in ORCA api (#9366)
The fix avoids shaded dependency of orca protos
2022-07-15 09:41:50 -07:00
Terry Wilson 49f555192d
xds: cluster manager to delay picker updates (#9365)
Do not perform picker updates while handling new addresses even if child
LBs request it. Assure that a single picker update is done.
2022-07-13 15:54:49 -07:00
Eric Anderson 9cd17ce3a7 Fix Gradle UP-TO-DATE checking for all tasks
The two checker tasks run quickly so don't gain much from UP-TO-DATE,
but it is convenient to not see them in the noise (checkUpperBoundDeps
in particular). Gradle only performs UP-TO-DATE checks (on the inputs)
if the task has both inputs and outputs defined.

The biggest saving was for distZip/distTar/shadowDistZip/shadowDistTar
which were using the same name for the non-shadow and shadow versions.
Thus the output file would always be out-of-date because it had been
rewritten and was invalid. This is worrisome because we could have
"randomly" been using the shadow Zip/Tar at times and the non-shadow
ones at others, although I think in practice the shadow tasks always run
last and so those are the files we'd see. Changing the classifier avoids
the colliding file names. These tasks took ~7 seconds, so incremental
builds are considerably shorter now.
2022-07-11 11:06:10 -07:00
Eric Anderson 19ad4467db Service config parse failures should be UNAVAILABLE
INVALID_ARGUMENT is propagated to the data plane if no previous config
is available. INVALID_ARGUMENT is reserved for application use; LBs
should pretty much use UNAVAILABLE exclusively.

While most of the changes are in xds, there do not appear to be likely
xds code paths that would propagate a bad status to the data plane.
Internal policies either don't use parseLoadBalancingPolicyConfig() and
instead have their configuration objects constructed directly or are
constructed transitively through the cluster manager which uses INTERNAL
if there's a child failure. There was a worrisome hole before this
commit for StatusRuntimeExceptions received by the cluster manager, but
the audit didn't find any locations throwing such an exception.
User-selected policies produce a NACK and are protected from the
existing xds client watcher paths. The worst that appears could happen
is the channel could panic (which uses INTERNAL) if a bug let a bad
configuration through.
2022-07-08 15:49:12 -07:00
Sergii Tkachenko ac23d33d72
xds: implement ignore_resource_deletion server feature (#9339)
As defined in the gRFC [A53: Option for Ignoring xDS Resource Deletion](https://github.com/grpc/proposal/blob/master/A53-xds-ignore-resource-deletion.md).

This includes semi-related changes:
* Refactor ClientXdsClientTestBase: extract verify methods for golden resources
* Parameterize ClientXdsClientV2Test and ClientXdsClientV3Test with ignoreResourceDeletion enabled and disabled
* Add FORCE_INFO and FORCE_WARNING levels to XdsLogLevel
2022-07-08 13:09:38 -07:00
Eric Anderson 0ff9f37b9e Use Gradle's task configuration avoidance APIs
This can avoid creating an additional 736 tasks (previously 502 out of
1591 were not created). That's not all that important as the build time
is essentially the same, but this lets us see the poor behavior of the
protobuf plugin in our own project and increase our understanding of how
to avoid task creation when developing the plugin. Of the tasks still
being created, protobuf is the highest contributor with 165 tasks,
followed by maven-publish with 76 and appengine with 53. The remaining
59 are from our own build, but indirectly caused by maven-publish.
2022-07-08 12:16:40 -07:00
Eric Anderson 3de7e74c57
xds: Build third-party protos in separate build step
This dramatically shortens build time, even for full builds. A full
assemble of xds on my laptop goes from 1m 46s to 33s at least because
errorprone is disabled for the protos.
2022-07-07 07:26:38 -07:00
Eric Anderson 3e09ea0068 xds: Fail RPCs with error details when resources are deleted
Previously if LDS/RDS were missing or improperly configured RPCs would
fail with "UNAVAILABLE: NameResolver returned no usable address errors".
That is very confusing and not helpful for debugging.

Ideally we'd also include the node id in this error message, but that's
a bit more involved and this is a huge improvement even without it.

b/237539851
2022-07-06 11:03:42 -07:00
yifeizhuang dc8954d442
xds: eds reuse priority names for the same existing locality (#9287) 2022-06-24 16:15:44 -07:00
Terry Wilson 7bd0797496
all: Update netty to 4.1.77.Final and netty_tcnative to 2.0.53.Final (#9027)
all: Update netty to 4.1.77.Final and netty_tcnative to 2.0.53.Final

Also switches to a non-release version of rules_jvm_external to allow Bazel build to work with artifact classifiers.
2022-06-24 10:47:27 -07:00
Terry Wilson 79c607e5ac
xds: weighted target to delay picker updates while updating children (#9306) 2022-06-24 10:29:22 -07:00
sanjaypujare 75aeccd385
xds: when delegate server throws on start communicate the error to statusListener (#9277) 2022-06-14 17:53:38 -07:00
Eric Anderson b06942d63b Use Gradle's version catalog
This moves our depedencies into a plain file that can be read and
updated by tooling. While the current tooling is not particularly better
than just using gradle-versions-plugin, it should put us on better
footing. gradle-versions-plugin is actually pretty nice, but will be
incompatible with Gradle 8, so we need to wait a bit to see what the
future holds.

Left libraries as an alias for libs to reduce the commit size and make
it easier to revert if we don't end up liking this approach.

We're using Gradle 7.3.3 where it was an incubating fetaure. But in
Gradle 7.4 is became stable.
2022-06-14 14:04:10 -07:00
Eric Anderson 5b825e8b17 Remove classifiers from ext.libraries
Classifier can't be specified in version catalog.
2022-06-14 14:04:10 -07:00
sanjaypujare 428b541114
xds: delete the permanent error logic in processing LDS updates in XdsServerWrapper (#9268) 2022-06-13 12:44:35 -07:00
Terry Wilson c589441bde
xds: Support least_request LB in LoadBalancingPolicy (#9262) 2022-06-10 16:00:39 -07:00
Eric Anderson 65bf2dc61b xds: Fix hash gen in RingHash when generating more than 10 hashes 2022-06-09 15:50:56 -07:00
yifeizhuang deea4e737a
xds: fix NullPointerException for unknown xdstp resources (#9245) 2022-06-06 14:36:01 -07:00
Eric Anderson 4d00e0cb02 xds: Remove unused Bouncy Castle dependency 2022-06-06 12:29:43 -07:00
Terry Wilson f8da92e176
xds: Update xDS protos (#9223) 2022-05-27 15:56:31 -07:00
Terry Wilson 69d4e71309
Enable xDS custom LB config by default. (#9214) 2022-05-26 17:03:14 -07:00
Terry Wilson e2f7e676cf
xds: Make a copy of Metadata for thread safety. (#9212) 2022-05-25 15:57:24 -07:00
Terry Wilson de3388bd21
xds: Custom LB configs to support UDPA TypeStruct (#9198)
* xds: Custom LB configs to support UDPA TypeStruct

The legacy com.github.udpa.udpa.type.v1.TypedStruct proto should be supported in addition to the current com.github.udpa.udpa.type.v1.TypedStruct one.

Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2022-05-24 13:55:02 -07:00
Terry Wilson 48ea07efce
xds: Use weighted_target LB provider in wrr_locality (#9195)
Fixes a bug where WrrLocalityLoadBalancer would use the endpoint picking policy provider instead of WeightedTargetLoadBalancerProvider.

Also adds a test to fake control plane integration test that caught this bug. The test scaffolding is also updated to have the test server echo all client headers back in the response.

The test load balancer in the test is an almost straight copy of: https://github.com/grpc/grpc-java/blob/master/interop-testing/src/test/java/io/grpc/testing/integration/RpcBehaviorLoadBalancerProviderTest.java
2022-05-24 13:51:12 -07:00
Eric Anderson f9e1c0f37f
xds: Protect xdstp processing with federation env var
There are still some cases for xdstp processing, but they are to percent
encoding replacement strings. Those seem better to leave running since
it looks like it they could be triggered even with federation disabled
in the bootstrap processing.
2022-05-23 08:16:11 -07:00
yifeizhuang c39b3583a1
orca: change metric recorder api to: clear/set/put/remove/setAll (#9180) 2022-05-18 10:54:19 -07:00
Eric Anderson ea09d3eebd Add Bazel build support for xds, googleapis to flesh out maven_install
Not updating the example WORKSPACE because it doesn't have any
Bazel-enabled build that depends on xds and so doesn't need the
additional repository dependencies.

Fixes #9162
2022-05-16 10:05:35 -07:00
cfredri4 1d4e82e9ae
xds: Allow unspecified listener traffic direction to work with Istio (#9173)
* Allow unspecified listener traffic direction to work with Istio
2022-05-16 09:35:55 -07:00
Terry Wilson 8a84611d9d
xds: Update the env variable to enable custom LB config (#9170) 2022-05-13 15:18:02 -07:00
Sergii Tkachenko 23ab7d0309 xds: ignore routes with unsupported optional cluster specifiers
Support for the is_optional logic in Cluster Specifier Plugins:
if unsupported Cluster Specifier Plugin is optional,
don't NACK, and skip any routes that point to it.
2022-05-13 13:03:04 -07:00
Terry Wilson c6bfce034f
xds: Check for env variable before doing custom LB config (#9165) 2022-05-12 12:57:11 -07:00
Terry Wilson 36d1d5fe45
xsd: wrr_locality to not propagate locality weight attribute (#9158)
WrrLocalityLoadBalancer should remove the locality weights attribute from Resolved addresses after using the information. Not propagating this attribute will make it impossible for another child wrr_locality from working. This is not a supported situation and this change make the failure happen earlier and to be more obvious as to the cause.
2022-05-12 12:51:21 -07:00