Commit Graph

886 Commits

Author SHA1 Message Date
Terry Wilson 810d8cab09
xds: pick_first LB configuration (#10181)
Configure pick_first load balancing if the xDS PickFirst LB config
message is used in the Cluster load balancing config.
2023-05-17 10:19:29 -07:00
Eric Anderson 29b8483fd6
Use test fixtures instead of sourceSets.test.output
This avoids the (often missing) evaluationDependsOn and fixes using
results from other projects without propagating those through
Configuration. It also reduces the number of useless classes pulled in
by down-stream tests, reducing the probability of rebuilds.

The expectation of fixtures is they help testing down-stream code that
use the classes in main. That applies to all the classes here except for
FakeClock and StaticTestingClassLoader. It would also apply to many
internal classes in grpc-testing, but let's consider cleaning that up
future work.
2023-05-16 12:10:13 -07:00
Eric Anderson 2cc24a9bda gradle: Remove version pinning from POMs
The pinning is unreliable in Maven and ignored by Gradle. I'm not at all
convinced that we are pinning/not pinning in appropriate projects. The
pinning also serves less of a purpose since we started encouraging the
BOM and grpc-netty-shaded. Netty's HTTP/2 API has also become somewhat
stable compared to its earlier history. If we notice an up-tick in
version skew, we can reinstate it.

The pinning is annoying in the build.gradle code and causes Maven/Gradle
to download the version list once a day, which can be troublesome to
users unaware of how to tell the tools to work offline.

It also opens our users to platform issues like seen in #10043
and #10086 where Maven Central's version list was incorrectly generated.
Or like #9664 where Gradle Plugin's repository caches packages from
JCenter but the version list is not as cachable so exposed us to JCenter
instability.

This fixes #8357, by way of "we think we won't worry any more." See
90db93b9 when it was originally introduced. And issues
like #8337, #3634.
2023-05-15 13:08:40 -07:00
Terry Wilson 793fde840a
xds: Import latest Envoy protos (#10167) 2023-05-11 09:44:43 -07:00
Daniel Liu 6e486cba48
import cncf/xds using import.sh script (#10160) 2023-05-10 12:42:26 -07:00
Eric Anderson 02a4cb5c69 Fix throwing LBProvider.parseLoadBalancingConfig() impls
LoadBalancers in general should never throw, but
parseLoadBalancingConfig() in particular has a return value to
communicate the error. Throwing can be a bit unpredictable, but at its
most trivial form causes a channel panic. There's no reason to throw
explicitly and calls to JsonUtil have to be protected by a try-catch
because it can throw.
2023-05-10 12:42:05 -07:00
yifeizhuang 6f804331f7
xds: remove _experimental in wrr policy name (#10162) 2023-05-10 11:02:13 -07:00
Eric Anderson 74b515ecf7 Migrate many usages of TestUtils.loadCert() to the public TlsTesting
TlsTesting.loadCert() is a public API and so should be preferred over
our internal utility. It avoids creating a temp file that has to be
deleted by a shutdown hook. Usages that needed a file were not migrated.
2023-05-09 17:01:31 -07:00
Eric Anderson f229aed538 core: Hide RR's ReadyPicker and EmptyPicker
ReadyPicker hasn't been necessary since 111ff60e, when we stopped
calling super.pickSubchannel(). EmptyPicker was only used in tests, and
we can just compare the class name instead of doing an instanceof check.

Unfortunately, calling getClass() caused Java to start casting the
return value of pickerCaptor.getValue() based on its generics. Captors
don't verify the type they capture, so using any type other than
SubchannelPicker for the pickerCaptor is misleading and hides a cast.
2023-05-09 13:25:19 -07:00
Eric Anderson 847ea7cfc9 Upgrade Mockito to 3.12.4
MockitoAnnotations.initMocks() is deprecated.
2023-05-08 16:39:42 -07:00
Eric Anderson 180b3e9092 xds: Avoid default locale and reset()
Found via linter
2023-05-05 15:23:18 -07:00
Eric Anderson 94dec7c020 xds: Avoid WRR config synchronization 2023-05-05 14:43:26 -07:00
Terry Wilson b5800edd76
Importing latest xDS protos from envoyproxy/envoy (#10123) 2023-05-03 11:57:52 -07:00
sanjaypujare f24a5a7d0d
xds: handle the handlerRemoved callback to skip updateSslContext processing (#10118)
* xds: handle the handlerRemoved callback to skip updateSslContext processing
     In handlerAdded we submit a callback to updateSslContext but before the
     callback is executed the handler could be removed (e.g. bad connection)
     in which case the callback should skip all of the processing.
     Also added a unit test to check there is no exception.
2023-05-03 07:41:15 -07:00
yifeizhuang b249a5340e
services: allow cpu utilization greater than 1 (#10107) 2023-04-28 11:59:10 -07:00
yifeizhuang fe53346697
xds: fix flaky xds test (#10109) 2023-04-28 11:17:49 -07:00
chenwei321 c5b825aa7d
xds: fix matchRoute in RoutingUtils and clean up implementations in XdsNameResolver. (#10095) 2023-04-27 13:28:44 -07:00
yifeizhuang c4c9939887
xds: enable wrr env variable (#10100) 2023-04-25 10:15:24 -07:00
Terry Wilson a384c16e8c
xds,googleapis: Default GRPC_EXPERIMENTAL_XDS_FEDERATION to true (#10093) 2023-04-24 11:01:32 -07:00
Terry Wilson 68b67b616e
xds: Allow child of cluster_impl LB to change (#10091)
Under normal conditions the child LB of `ClusterImplLoadBalancer` does
not fluctuate, based on the field used to configure load balancing in
the xDS `Cluster` proto it is either:

1. `WrrLocalityLoadBalancer` if the newer `load_balancing_policy` field
   is used
2. `WeightedTargetLoadBalancer` if the legacy `lb_policy` field is used

`ClusterImplLoadBalancer` currently assumes that this child does not
change and so does not change the child LB when the name resolver sends an
update. If the control plane does switch to using a different field for
LB config, that update will have an LB config meant for the other child
LB type. This will result in a ClassCastException and a channel panic.

To address this, `ClusterImplLoadBalancer` will now use
`GracefulSwitchLoadBalancer` and makes sure if the child policy changes
the correct LB implementation is switched to.
2023-04-24 11:00:56 -07:00
yifeizhuang 8b9ae27cdf
xds: remove environmental variables (#10055) 2023-04-19 17:26:26 -07:00
yifeizhuang 111ff60e1c
xds: fix wrr stuck in rr mode (#10061) 2023-04-18 16:39:51 -07:00
yifeizhuang e12baed5a9
services: rename to (clear)setQpsMetric (#10031) 2023-04-11 15:18:41 -07:00
Terry Wilson 5082b4c02a
xds: Add dummy LRS service for control plane tests (#10038) 2023-04-11 13:56:08 -07:00
Terry Wilson 1e028c404d
xds: Wait for sync context before assertions in federation test (#10021) 2023-04-10 14:05:01 -07:00
Daniel Liu 5201e49ce1
services,orca: update backend metrics support to allow for server-wide metrics recording (per-call and OOB) (#9902)
Also added input range validation.
2023-04-10 11:45:04 -07:00
Terry Wilson 18e274de65
xds: Synchronize access to test control plane collections (#10012)
Fixes #9938
2023-04-04 14:34:11 -07:00
Terry Wilson 6d75fca23f
xds: Distinct LoadStatManagers (#10009)
Currently the code maintains one LoadStatsManager2 that collects all
stats. The problem with this is that in a federation situation there
will be multiple LrsClients that will be periodically picking up stats
from the manager and sending them to their respective control planes.
This creates a first-come-first-serve situation where the stats get
randomly distributed across the control planes.

This change creates separate LoadStatsManagers dedicated to their own
control planes, thus assuring no stats will get lost.
2023-04-04 11:29:17 -07:00
Terry Wilson ec9b8e0d61
xds: Correctly start LRS clients in federation situations (#10000)
xds: Correctly start LRS clients in federation situations

The old code used a single member variable to indicate if load reporting
had already been started by XdsClientImpl. This boolean was used to
avoid starting a LoadReportClient more than twice. This works fine with
a single control plane server.

The problem occurs in federation situations where there is more than one
control plane and thus more than one LoadReportClient. Once the first
LoadReportClient is started, the member variable boolean is flipped to
true and no other LoadReportClients would be started.

This change removes the boolean member variable and relies on the fact
that starting an already started LoadReportClient is a no-op.
2023-04-03 18:35:48 -07:00
yifeizhuang bbe5a0227d
xds: fix flaky wrr test (#10004) 2023-04-03 09:14:44 -07:00
Terry Wilson 56e4ce49bd
Rename AbstractXdsClient to ControlPlaneClient (#9934)
AbstractXdsClient is not abstract.
2023-03-06 10:35:32 -08:00
Larry Safran 95a44e0592
Move retrieval of elapsed time in handleRpcStreamClosed to make TSan happy (#9923)
* In `handleRpcStreamClosed()`, move retry handling to before the call to `xdsResponseHandler.handleStreamClosed()` so that TSan doesn't report a race condition that is completely meaningless.

fixes #9920
2023-03-02 15:28:37 -08:00
yifeizhuang c367b267c6
xds, wrr: randomize the initial deadline in the scheduler (#9922) 2023-03-02 15:17:54 -08:00
yifeizhuang c7f556a9a6
xds: exclude generated error prone (#9921) 2023-03-02 09:41:31 -08:00
Benjamin Peterson ae6c506f96
all: fix build with errorprone 2.18 (#9886)
errorprone cannot be updated past 2.10 because later versions do not support Java 8.

Fixes https://github.com/grpc/grpc-java/issues/9916.
2023-03-01 13:45:18 -08:00
yifeizhuang 8d12baa447
xds: add weighted round robin LB policy support (#9873) 2023-02-27 10:34:51 -08:00
Terry Wilson b481f34855
xds: deletion only to watchers of same control plane (#9896)
When XdsClient learns that a control plane no longer tracks a resource,
it should only notify watchers associated with that control plane.

This matters in control plane federation cases when more than one
control plane is in use.
2023-02-17 06:34:30 -08:00
yifeizhuang 4ef0200f6b
xds: import envoy proto for WRR (#9898) 2023-02-16 15:23:28 -08:00
Larry Safran 19eab29f8d
compiler: Generate interfaces for services to implement (#9688)
Introduce an AsyncService interface in the generated code and move the methods from <service>ImplBase to default implementation of the interface.
* update pom files to allow java 1.8
* Add a bindService(<service>Async) method
* Change TestServiceImpl to use the interface and include a bind method instead of extending TestServiceImplBase.
2023-02-15 10:33:44 -08:00
yifeizhuang f6a0028fe5
orca: support remove listener in OrcaOobUtil (#9881) 2023-02-08 09:50:32 -08:00
Larry Safran 54c1f37093
xds:Allow big cluster total weight (#9864)
* xds:  allow sum of cluster weights above MAX_INT up to max of unsigned int.

* Define nextLong(long bound) method in FakeRandom for WeightedRandomPickerTest.
2023-02-03 10:53:50 -08:00
yifeizhuang 6119f6ec94
services: add qps in orca api (#9866) 2023-02-02 15:16:27 -08:00
yifeizhuang b7164f0791
xds: import cncf/xds (#9862) 2023-01-30 10:42:11 -08:00
Larry Safran 501ca8f7b4
xds: Update logic so that an error being reported when stream is closed gets propagated to subscribers (#9827)
* Stop setting waitForReady in XdsClient's AbstractXdsClient.
* Handle bad URL cleanly.  

Fix test cases to deal with asynchronous flow.
2023-01-24 18:31:50 -08:00
chenwei321 b2895198c3
xds: fixed RouteConfiguration not supporting contain and stringMatcher (#9845) 2023-01-23 11:49:56 -05:00
Eric Anderson ce86090322 xds: Include unknown type in channel logger warning
This makes the log statement more useful and easier to determine whether
this was expected behavior.
2023-01-12 11:04:20 -08:00
yifeizhuang 82ca98f338
xds:fix cancel xds watcher accidentally removes the url (#9809)
Fix a bug. When any of the xds subscribers for a resource has the last watcher cancelled, the bug will accidentally remove that resource type from the map, which make xds stream not accepting response update for that resource type entirely(pass through, no ACK/NACK will send).
2023-01-11 15:33:48 -08:00
Chris Povirk c0de130ac6 Change ArgumentCaptor.forClass(null) to ArgumentCaptor.forClass(Foo.class)
The former will [soon throw NullPointerException][notnull].

[notnull]: fe1cb2de09 (diff-8d274a9bda2d871524d15bbfcd6272bd893a47e6b1a0b460d82a8845615f26daR31)
2023-01-09 07:15:23 -08:00
Eric Anderson d17a2db4bd Upgrade to Checkstyle 8.28
Trying to upgrade Gradle to 7.6 improved the checkstyle plugin such that
it appears to have been running in new occasions. That in turn exposed
us to https://github.com/checkstyle/checkstyle/issues/5088. That bug was
fixed in 8.28, which also fixed lots of other bugs. So now we have
better checking and some existing volations needed fixing. Since the
code style fixes generated a lot of noise, this is a pre-fix to reduce
the size of a Gradle upgrade.

I did not upgrade past 8.28 because at some point some other bugs were
introduced, in particular with the Indentation module. I chose the
oldest version that had the particular bug impacting me fixed. Upgrading
to this old-but-newer version still makes it easier to upgrade to a
newer version in the future.
2023-01-05 17:07:04 -08:00
yifeizhuang d07ecbe037
xds: refactor LoadReportClient with xds v2 dropped (#9788) 2023-01-05 12:49:23 -08:00