Commit Graph

1013 Commits

Author SHA1 Message Date
Kurt Alfred Kluever 06135a0745 Migrate from the deprecated `Charsets` constants (in Guava) to the `StandardCharsets` constants (in the JDK)
cl/658539667
2024-08-05 13:31:08 -07:00
Sergii Tkachenko c29763d886
xds: Import RLQS protos (#11418)
Imports the protos of Rate Limiting Quota Service (RLQS) and Rate
Limit Quota HTTP Filter.

Note: the list below only shows the new top-level protos, and excludes
their direct and transitional dependencies (those from import
statements).

#### RLQS Imports
- Service — envoy/service/rate_limit_quota/v3/rlqs.proto
  (Service): 7b8a304
- HTTP Filter —
  envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto:
  49c77c4

#### CEL Imports
- Initial third-party repo setup: 99a64bd
- Parsed CEL Expression: cel/expr/syntax.proto: 99a64bd
- Parsed and type-checked CEL Expression: cel/expr/checked.proto:
  99a64bd


#### Required typed_config extensions
##### `bucket_matchers` predicate input
- `HttpAttributesCelMatchInput` —
  xds/type/matcher/v3/http_inputs.proto: 54924e0
- `HttpRequestHeaderMatchInput` —
  envoy/type/matcher/v3/http_inputs.proto: 49c77c4

##### `bucket_matchers` predicate custom_match
- `CelMatcher` — xds/type/matcher/v3/cel.proto: 54924e0
2024-08-02 10:35:29 -07:00
Eric Anderson 9bc1a93f6e xds: Add test that uses real DnsNR with ClusterResolverLB
This can detect failures like the UnsupportedOperationException from
ebffb0a6.
2024-08-02 07:26:11 -07:00
Eric Anderson dc83446d98 xds: Stop extending RR in WRR
They share very little code, and we really don't want RoundRobinLb to be
public and non-final. Originally, WRR was expected to share much more
code with RR, and even delegated to RR at times. The delegation was
removed in 111ff60e. After dca89b25, most of the sharing has been moved
out into general-purpose tools that can be used by any LB policy.

FixedResultPicker now has equals to makes it as a EmptyPicker
replacement. RoundRobinLb still uses EmptyPicker because fixing its
tests is a larger change. OutlierDetectionLbTest was changed because
FixedResultPicker is used by PickFirstLeafLb, and now RoundRobinLb can
squelch some of its updates for ready pickers.
2024-07-31 13:32:49 -07:00
Sergii Tkachenko 0017c98f6b
xds: cncf/xds proto sync to 2024-07-24 (#11417)
`cncf/xds`: Sync protos to the latest imported version
cncf/xds@024c85f (commit 2024-07-23, cl/655545156).

Should be a noop, just a routine xDS proto update to make upcoming
RLQS-related imports simpler, see related #11401.

Note that CEL is only added as a bazel dependency as now it's required
to build cncf/xds. Actual third-party source import will be done in
the follow up PR, where RLQS dependencies are added to the import
scripts.
2024-07-30 12:17:49 -07:00
Jiajing LU 448ec4f37e
xds: XdsClient should unsubscribe on last resource (#11264)
Otherwise, the server will continue sending updates and if we
re-subscribe to the last resource, the server won't re-send it. Also
completely remove the per-type state, as it could only add confusion.
2024-07-30 08:46:01 -07:00
Sergii Tkachenko 96a788a349
xds: Envoy proto sync to 2024-07-06 (#11401)
`envoyproxy/envoy`: Sync protos to the latest imported version
ab911ac2ff
(commit 2024-07-06, cl/651956889).

Should be a noop, just a routine xDS proto update to make upcoming
RLQS-related imports simpler.
2024-07-29 09:18:18 -07:00
Eric Anderson 786523dca4 xds: WRR rr_fallback should trigger with one endpoint weight
From gRFC A58:
> When less than two subchannels have load info, all subchannels will
> get the same weight and the policy will behave the same as round_robin
2024-07-25 11:14:21 -07:00
maleo 6dd6ca9f90 Remove udpa aa repo alias for xds 2024-07-11 10:47:23 -07:00
Eric Anderson 0454b9e328 xds: Avoid switchTo in PriorityLb 2024-07-03 10:32:38 -07:00
Eric Anderson dfb22ba97c xds: Avoid switchTo in ClusterImplLb and ClusterResolverLb 2024-07-03 10:32:38 -07:00
Eric Anderson 749b2e0abc util: Avoid switchTo in OutlierDetectionLb 2024-07-03 10:32:38 -07:00
Eric Anderson 2c49cc4197 xds: Avoid switchTo in WrrLocalityLb and WeightedTargetLb 2024-07-03 10:32:38 -07:00
yifeizhuang 9e2145970a
doc: add document on WRR configure examples from service config and xds (#11344) 2024-06-27 16:55:38 -07:00
Larry Safran 7a53fa8bc1
Dualstack interop testing enablement (#11231)
* Have java test server listen on appropriate address based upon new optional flag "address_type"
2024-06-27 16:37:48 -07:00
Eric Anderson e7c3803b5a
xds: Remove unused opencensus-proto dependency
opencensus-proto is old generated code, which is not compatible with
protobuf-java 4.27.2 and may not be fixed since the project is dead.
Since it is unused, I think this doesn't cause any trouble for
downstream users trying to use protobuf-java 4.x. Related to #11015.
2024-06-26 15:30:31 -07:00
Eric Anderson c540993aaa
bazel: Upgrade and fix import issues with envoy_api
Bazel was not updated as part of feab4e54. Also, `(( X++ ))` exits 1 if
X is zero. The patch to data-plane-api was upstreamed in
https://github.com/envoyproxy/envoy/pull/34759
2024-06-17 14:58:14 -07:00
Eric Anderson df8cfe9ddc Create gcp-csm-observability 2024-05-29 14:40:44 -07:00
Eric Anderson 75fa441fc9
xds: Plumb the Cluster's filterMetadata to RPCs
This will be used by CSM observability, and may get exposed to further
uses in the future.
2024-05-24 15:08:36 -07:00
Eric Anderson fea577c804
core: Exit idle mode when delayed transport is in use
8844cf7b8 triggered a regression where a new RPC wouldn't cause the
channel to exit idle mode, if an RPC was still progressing on an old
transport. This was already possible previously, but was racy.
8844cf7b8 made it less racy and more obvious.

The two added `exitIdleMode()` calls in this commit are companions to
those in `enterIdleMode()`, which detect whether the channel should
immediately exit idle mode.

Noticed in cl/635819804.
2024-05-23 14:45:38 -07:00
Vindhya Ningegowda 77a1e77e11
xds, rls: Experimental metrics are disabled by default (#11196)
Experimental metrics (i.e WRR and RLS metrics) are disabled by default. Users are expected to explicitly enable while configuring metrics.
2024-05-10 17:46:58 -07:00
Eric Anderson 7a663f633c api: Hide internal metric APIs
Some APIs were marked experimental but had internal APIs in their
surface. These were all changed to internal. And then the internal APIs
were mostly hidden from generated documentation.

All these APIs will eventually become public and maybe even stable. But
they need some iteration before we're ready for others to start using
them.
2024-05-08 10:24:24 -07:00
Larry Safran 59b189bf91
Change HappyEyeballs and new pick first LB flags default value to false (#11120)
* Change HappyEyeballs flag default value to false since some G3 users are seeing problems.
Put the flag logic in a common place for PickFirstLeafLoadBalancer & WRR's test.

* Set expected requestConnection count based on whether happy eyeballs is enabled or not

* Disable new PickFirstLB

* Fix test expectations to handle both new and old PF LB paths.
2024-05-08 10:08:23 -07:00
Eric Anderson 45a91bd035 xds: Add WRR metric test with real channel 2024-05-08 07:50:09 -07:00
Terry Wilson 2bc4306940
xds: Include locality label in WRR metrics (#11170) 2024-05-07 11:40:03 -07:00
hakusai22 6ec744f2a0
Fix various typos (#11144) 2024-05-06 20:29:44 -07:00
Eric Anderson 077dcbf90f xds: Plumb locality in xds_cluster_impl and weighted_target
As part of gRFC A78:

> To support the locality label in the WRR metrics, we will extend the
> `weighted_target` LB policy (see A28) to define a resolver attribute
> that indicates the name of its child. This attribute will be passed
> down to each of its children with the appropriate value, so that any
> LB policy that sits underneath the `weighted_target` policy will be
> able to use it.

xds_cluster_impl is involved because it uses the child names in the
AddressFilter, which must match the names used by weighted_target.
Instead of using Locality.toString() in multiple policies and assuming
the policies agree, we now have xds_cluster_impl decide the locality's
name and pass it down explicitly. This allows us to change the name
format to match gRFC A78:

> If locality information is available, the value of this label will be
> of the form `{region="${REGION}", zone="${ZONE}",
> sub_zone="${SUB_ZONE}"}`, where `${REGION}`, `${ZONE}`, and
> `${SUB_ZONE}` are replaced with the actual values. If no locality
> information is available, the label will be set to the empty string.
2024-05-03 12:42:31 -07:00
Terry Wilson 35a171bc1d
xds: include the target label to WRR metrics (#11141) 2024-05-01 15:20:38 -07:00
Eric Anderson 4c78a9746c
Plumb optional labels from LB to ClientStreamTracer
As part of gRFC A78:

> To support the locality label in the per-call metrics, we will provide
> a mechanism for LB picker to add optional labels to the call attempt
> tracer.
2024-04-29 16:30:51 -07:00
Terry Wilson 06df25b65d
core,xds: Metrics recording in WRR LB (#11129)
Adds the recording of the four metrics documented in:

https://github.com/grpc/proposal/blob/master/A78-grpc-metrics-wrr-pf-xds.md#weighted-round-robin-lb-policy
2024-04-26 15:59:49 -07:00
Eric Anderson 9de8e44384 util: Remove deactivation and GracefulSwitchLb from MultiChildLb
It is easy to manage these things outside of MultiChildLb and it makes
the shared code easier and use less memory. In particular, we don't want
to use many instances of GracefulSwitchLb in virtually every policy
simply because it was needed in one or two cases.
2024-04-22 07:48:49 -07:00
Eric Anderson 7f0a1910d3 xds: Directly manage deactivation in cluster manager 2024-04-22 07:48:49 -07:00
Eric Anderson 61bf21e2a1 xds: Swap RingHashLb to use lazy child, instead of deactivation 2024-04-22 07:48:49 -07:00
Eric Anderson 32d48ae89a bazel: Fix formatting with buildifier
Keys and dependencies are sorted.
2024-04-04 11:50:07 -07:00
Kannan J 097a46b761 Use empty string instead of null for endpoint identification algorithm to disable server hostname verification, since null value gets ignored in Sun's SSLEngine implementation. 2024-03-28 16:58:48 -07:00
David Burns 00649913b0
bazel: Use the `artifact` macro for loading maven deps
The recommended way to load dependencies from `rules_jvm_external`
is to make use of the `@maven` workspace, and the most readable
way of doing that is to use the `artifact` macro provides.

This removes the need to generate the "compat" namespaces, which
`rules_jvm_external` provided for backwards compatibility with
older releases. This change also sets things up for supporting
`bzlmod`: this requires all workspaces accessed by a library to
be named "up front" in the `MODULE.bazel` file. This way, the
only repo that needs to be exported is `@maven`, rather than the
current huge list.
2024-03-28 14:33:32 -07:00
Eric Anderson e6305930de Specify a locale for upper/lower case conversions
None of these conversions should use the arbitrary system locale. Error
Prone will help prevent these getting introduced in the future.

Fixes #10372
2024-03-27 15:58:34 -07:00
Larry Safran 51f811df86
Enable Happy Eyeballs by default (#11022)
* Flip the flag

* Fix test flakiness where IPv6 was not considered loopback
2024-03-21 16:59:54 -07:00
Larry Safran 38f968fafb
Have EDS resource parse the additional addresses from envoy message (#11011)
* Have EDS resource parse the additional addresses from envoy message
* Update respositories.bzl to point to current grpc-proto instead of a 2021 version.
* Update respositories.bzl to point to recent cncf/xds and envoyproxy/data-plane-api
* Add cncf_upda to repositories.bzl
2024-03-15 12:26:21 -07:00
Sergii Tkachenko 0d749c5943
xds: Stabilize CsdsService (#11003)
To make it stable, this PR hides protobuf from being exposed via the
API.

Note: this breaks ABI of `CsdsService.streamClientStatus` and
`CsdsService.fetchClientStatus`, but these methods should not
normally be called by the user.

Closes #8016.
2024-03-11 16:14:05 -07:00
Larry Safran d1c406bd23
Prepare to switch flag to use new PickFirstLeafLoadBalancer by default (#10998)
* Fix PickFirstLeafLoadBalancer and tests to work when it is used.
* Actually use EAG attributes for subchannels.
2024-03-11 14:12:56 -07:00
Eric Anderson 9ee5e9f008 xds: Move node id logging out of xds.client
This removes a grpc-ism environment variable. Note that the logger is
still registered under XdsClientImpl. That could maybe change, but it is
a bit unclear what it should become and it seemed better for this to
have no behavior changes.
2024-03-06 07:25:54 -08:00
Eric Anderson 42b2cbdec3 xds: Move LR and RLS experimental flags to where they are used
That's better for code organization and also removes some grpc-isms from
XdsResourceType.
2024-03-06 07:25:33 -08:00
Eric Anderson 85e52cd113 xds: Remove WRR and PF experimental flags
They have been on by default for a good while, and seem stable. This
also removes some grpc-isms from XdsResourceType.
2024-03-05 13:21:53 -08:00
Eric Anderson 27824469bf xds: Provide default XdsResourceType.extractResourceName impl
This method is only needed sometimes, and with time will be needed less
and less. Don't require new types to implement it, instead relying on
control planes to use the new approach.
2024-03-05 11:30:35 -08:00
Anirudh Ramachandra 867e469404
xds: Support retrieving names from wrapped resource containers (#10975)
The xDS library only honored names retrieved from the inner resource
containers, but for wrapped resources the outer layer could contain the
required name. This commit prefers the name on the wrapped container
over the inner resource name.
2024-03-05 07:22:18 -08:00
Eric Anderson ac62c8b055 Fix tests and warnings on Java 17
SelfSignedCertificate is not available on Java 17 because
OpenJdkSelfSignedCertGenerator is not available. This only impacted
tests.

AccessController is being removed, and these locations are doing simple
reflection which is unlikely to require it even when a security policy
is in effect. There's other places we do reflection without the
AccessController, so either no security policies care or the users can
update their policies to allow it.
2024-02-29 16:55:46 -08:00
Sergii Tkachenko feab4e5449
xds: Get rid of xDS v2 dependencies (#10968)
xDS v2 support was dropped about a year ago, but the xds package still
had a few xDS v2 usages. This PR:

- Removes all leftover usages of xDS v2 classes in gprc-xds
- Removes all imported xDS v2 protos and their leaf dependencies:
- Removes xDS v2 generated services
- Makes minor improvements to the xds import script output

### Before
```sh
# Imported 154 protos.
❯ find . -iname "*xds*.jar" -exec du -h {} \; | col -x
  13M ./build/libs/grpc-xds-1.63.0-SNAPSHOT-original.jar
  6.1M ./build/libs/grpc-xds-1.63.0-SNAPSHOT-sources.jar
  388K ./build/libs/grpc-xds-1.63.0-SNAPSHOT-javadoc.jar
  14M ./build/libs/grpc-xds-1.63.0-SNAPSHOT.jar

```

### After
```sh
# Imported 86 protos.
❯ find . -iname "*xds*.jar" -exec du -h {} \; | col -x
  9.1M ./build/libs/grpc-xds-1.63.0-SNAPSHOT-original.jar
  4.1M ./build/libs/grpc-xds-1.63.0-SNAPSHOT-sources.jar
  388K ./build/libs/grpc-xds-1.63.0-SNAPSHOT-javadoc.jar
  9.1M ./build/libs/grpc-xds-1.63.0-SNAPSHOT.jar ```

Reduction:
- Number of protos: 44%
- Jar size: 35%
2024-02-29 10:33:18 -08:00
Eric Anderson 9b53bcaf66 xds: Increase timeouts in XdsClientFederationTest
`isolatedResourceDeletions()` has failed with a timeout waiting on
onChanged when running under TSAN. TSAN can slow things down, so let's
increase the timeout to ensure it isn't just timeout flake.
2024-02-27 11:28:24 -08:00
Larry Safran 8087977c0b
Move xds classes for Stubby to xds.client package (#10912)
* Move bootstrap, XdsClient, load reporting, XdsLogger and XdsResourceType to xds.client package.
2024-02-26 16:41:16 -08:00