Commit Graph

6195 Commits

Author SHA1 Message Date
Eric Anderson f768c4222b Remove build usages of Jetty ALPN
It wasn't actually being used. Since Java 8u252 in early 2020 we've been
using ALPN from the JDK. The Jetty ALPN Agent has been a noop.

We do keep the Jetty ALPN support in the code and tests, but we don't
have the infrastructure to actually run it.
2024-02-23 15:27:33 -08:00
Eric Anderson 569956e022 xds: Pre-add fallback to xds client pool accessor
When we implement A71, we're no longer going to have a single xds
client, but instead one per channel target. Add that parameter now, even
though it is unused, to avoid managing the (internal) API breakage when
we implement fallback.
2024-02-23 14:12:46 -08:00
Eric Anderson c7e87d2180 buildscripts: Use Java 11 on Windows CI
This version of Java is old, but not as old as the Java 8 version.
2024-02-23 14:12:11 -08:00
Eric Anderson bfc0f959cd xds: Avoid nonexistent DNS resolution in XdsClientImplV3Test
The DNS lookups are taking considerable time on the Windows CI (~11s),
which causes the test to time out:

```
Wanted but not invoked:
ldsResourceWatcher.onError(<any>);
-> at io.grpc.xds.XdsClientImplTestBase.sendToNonexistentHost(XdsClientImplTestBase.java:3733)
Actually, there were zero interactions with this mock.

	at io.grpc.xds.XdsClientImplTestBase.sendToNonexistentHost(XdsClientImplTestBase.java:3733)
```

The ARM build, which uses an emulator, has had this test succeed, so the
failure seems unrelated to CPU usage. We want to avoid external I/O
anyway during tests, so removing the DNS lookup is good.

The TSAN comment referenced XdsClientImplTestBase.sendToNonexistentHost,
but the test no longer calls fakeClock.forwardTime so the comment was
out-of-date. Change the comment to make clear the race involved.
2024-02-23 11:44:06 -08:00
Eric Anderson d7628a3aba
xds: Fix flow control data race in ControlPlaneClient
As discovered by TSAN, the adsStream field is not synchronized.
```
WARNING: ThreadSanitizer: data race (pid=1625)
  Read of size 4 at 0x00009b66fc88 by thread T23 (mutexes: write M0):
    #0 io.grpc.xds.ControlPlaneClient.isReady()Z ControlPlaneClient.java:203
    #1 io.grpc.xds.ControlPlaneClient.readyHandler()V ControlPlaneClient.java:211
    #2 io.grpc.xds.ControlPlaneClient$AdsStream.onReady()V ControlPlaneClient.java:328
    #3 io.grpc.xds.GrpcXdsTransportFactory$EventHandlerToCallListenerAdapter.onReady()V GrpcXdsTransportFactory.java:145
    #4 io.grpc.PartialForwardingClientCallListener.onReady()V PartialForwardingClientCallListener.java:44
    #5 io.grpc.ForwardingClientCallListener.onReady()V ForwardingClientCallListener.java:23
    #6 io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onReady()V ForwardingClientCallListener.java:40
    #7 io.grpc.PartialForwardingClientCallListener.onReady()V PartialForwardingClientCallListener.java:44
    #8 io.grpc.ForwardingClientCallListener.onReady()V ForwardingClientCallListener.java:23
    #9 io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onReady()V ForwardingClientCallListener.java:40
    #10 io.grpc.internal.DelayedClientCall$DelayedListener.onReady()V DelayedClientCall.java:497
    #11 io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamOnReady.runInternal()V ClientCallImpl.java:781
    #12 io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamOnReady.runInContext()V ClientCallImpl.java:772
    #13 io.grpc.internal.ContextRunnable.run()V ContextRunnable.java:37
    #14 io.grpc.internal.SerializingExecutor.run()V SerializingExecutor.java:133
    #15 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V ThreadPoolExecutor.java:1130
    #16 java.util.concurrent.ThreadPoolExecutor$Worker.run()V ThreadPoolExecutor.java:630
    #17 java.lang.Thread.run()V Thread.java:830
    #18 (Generated Stub) <null>

  Previous write of size 4 at 0x00009b66fc88 by thread T4 (mutexes: write M1, write M2, write M3, write M4, write M5):
    #0 io.grpc.xds.ControlPlaneClient$AdsStream.cleanUp()V ControlPlaneClient.java:424
    #1 io.grpc.xds.ControlPlaneClient$AdsStream.close(Ljava/lang/Exception;)V ControlPlaneClient.java:418
    #2 io.grpc.xds.ControlPlaneClient$1.run()V ControlPlaneClient.java:130
    #3 io.grpc.SynchronizationContext.drain()V SynchronizationContext.java:94
    #4 io.grpc.SynchronizationContext.execute(Ljava/lang/Runnable;)V SynchronizationContext.java:126
    #5 io.grpc.xds.XdsClientImpl.shutdown()V XdsClientImpl.java:207
    #6 io.grpc.xds.SharedXdsClientPoolProvider$RefCountedXdsClientObjectPool.returnObject(Ljava/lang/Object;)Lio/grpc/xds/XdsClient; SharedXdsClientPoolProvider.java:144
    #7 io.grpc.xds.SharedXdsClientPoolProvider$RefCountedXdsClientObjectPool.returnObject(Ljava/lang/Object;)Ljava/lang/Object; SharedXdsClientPoolProvider.java:102
    #8 io.grpc.xds.XdsClientFederationTest.cleanUp()V XdsClientFederationTest.java:86
```
2024-02-22 14:43:29 -08:00
Sergii Tkachenko de7e649e04
(minor) Fix a typo in ReflectableMarshaller docstring (#10942)
`it's` -> `its`
2024-02-21 13:29:51 -08:00
Eric Anderson e19f1f15a9 examples: Fix file paths in debug example README 2024-02-21 08:34:19 -08:00
Eric Anderson 071a784795
Change GAE interop tests to use java11 runtime (#10933)
The Java 8 runtime is end of support. Leaving this a gae-jdk8 for now.
The gae-jdk8 was because AppEngine changed dramatically from Java 7 to
Java 8. Nowadays the versions are more in line with OpenJDK and not very
different from each other.

Fixes #10925
2024-02-20 10:52:22 -08:00
David Ankin 0d39c2c701
implement ability to skip generation of javax annotation (#10927)
* commit to implement feedback from #10786 to partially fix #9179
2024-02-16 17:49:20 -08:00
Eric Anderson 3994b15fea buildscripts: Avoid vswhere crashing Gradle
I'm trying to upgrade to a newer Windows Kokoro image, but the new one
has an old vswhere installed that breaks Gradle. Our old image doesn't
have vswhere at all. If vswhere isn't found, this rename prints some
errors, but the bat script continues executing. So this change is
compatible with both the older and newer image.
2024-02-16 15:12:14 -08:00
Eric Anderson f4cc166f18 xds: Copy data in least request to avoid picker data race
In 0d39bf50 the ReadyPicker was changed holding List<Subchannel> to
List<ChildLbState>, but ChildLbState mutates over time and is not
synchronized. We want the picker to have a snapshot of the data, so copy
the data from ChildLbState instead of using it directly.

Unfortunately the tests depended on the ChildLbState a bit, so we need
to save the EAG only to use it in tests. That's okay for now, but in the
future we'll probably want to remove that unnecessary memory usage.
2024-02-16 13:44:25 -08:00
Eric Anderson a231d80756 Remove semi-circular dependency between core and util
Add the 'fake' dependency to grpc-netty instead of grpc-core.
grpc-okhttp already depends on grpc-util and probably would be fine
without round_robin on Android.

There's not actually a circular dependency, but some tools can't handle
the compile vs runtime distinction. Such tools are broken, but fixes
have been slow and this approach works with no real downfalls.

Works around #10576 #10701
2024-02-16 12:31:54 -08:00
Alex Panchenko 257d1c2db4 servlet: introduce ServletServerBuilder.buildServlet() 2024-02-16 12:21:42 -08:00
Benjamin Peterson a68399a9b6
netty: improve server handling of writes to reset streams (#10258)
* netty: improve server handling of writes to reset streams

A server stream can be reset by the client while server writes are still queued. After the stream is reset, the netty connection will forget the stream object. The `NettyServerHandler` must deal with that situation. `sendResponseHandlers` already had some code to do that. This change standardizes that code and adds it to `sendGrpcFrame`. This fixes a potential bug where a `SendGrpcFrameCommand` with `endOfStream=true` would raise an `AssertionError` if written to a reset stream. (This bug is not currently reachable because `endOfStream=false` for all server `SendGrpcFrameCommand` objects.)

* Do not call into the encoder when we know the stream is gone.
2024-02-16 11:08:09 -08:00
Alex Panchenko 2f025b2b24
servlet: missing @Rule annotation in GrpcServletSmokeTest (#10922) 2024-02-16 10:29:45 -08:00
Larry Safran 67dfe49ae6 Fix compile failure 2024-02-15 21:28:37 -08:00
Terry Wilson eba699ad16
rls: Adding extra debug logs (#10902) 2024-02-15 15:23:36 -08:00
Larry Safran 044749706a
util:MultiChildLoadBalancer cleanup (#10780)
* add final, change method permissions, add javadoc, cleanup unneeded, move updateOverallBalancingState to ClusterManagerLB and make it abstract

* Restructure to eliminate the flags as protected methods

* Move methods around so that the candidates for override are near the top.

* Reorder picker methods lower
2024-02-15 14:12:40 -08:00
John Cormie f20c853c40
Factor out MainThreadScheduledExecutorService. (#10916)
So it can be used by other robolectric tests.
2024-02-15 11:11:43 -08:00
Terry Wilson 08f8ed1bc3 util: MultiChildLoadBalance.shutdown() log to FINE
The log level of this method is set to INFO, which is too spammy. Bring
it down to the FINE level.
2024-02-13 14:12:33 -08:00
Eric Anderson 7787673992 xds: Replace isEquivalentTo with equals in LeastRequest
This is similar to the changes to round robin in dca89b25.
2024-02-13 08:29:31 -08:00
Eric Anderson 7e72413233 okhttp: Move HostnameVerifier tests to TlsTest
Http2OkHttp is now unnecessary, as Http2Test tests OkHttp client to
Netty server. receivedDataForFinishedStream() was the only remaining
unique test and it seems already covered by AbstractInteropTest these
days.
2024-02-12 19:40:33 -08:00
Eric Anderson 92463f62bf interop-testing: Avoid Netty-specific APIs to creating channels
We prefer to test using the stable APIs, as they are what our users
should be using. Http2Client continues using NettyChannelBuilder because
it is intended to test grpc-netty.
2024-02-12 07:52:31 -08:00
Eric Anderson 372a535615 auth: Add support for Retryable interface
Retryable was added in google-auth-library 1.5.3 to make clear the
situations that deserve a retry of the RPC. Upgrading to that caused
problems because of transitive dependency issues syncing into Google so
it was reverted in 369f87be. google-auth-library 1.11.0 changed the
approach to avoid the transitive dependency updates. cl/601545581
upgraded to 1.22.0 inside Google. Bump to that version and swap away
from the imprecise IOException heuristic. go/auth-correct-retry

Fixes #6808
2024-02-09 15:15:18 -08:00
yifeizhuang ff34d51c73
api: stabilize server.getListenerSockets (#10910) 2024-02-09 14:40:54 -08:00
Anirudh Ramachandra 608bb8499c
Mark couple of helper functions in XdsClient as public. (#10871) 2024-02-09 12:53:03 -08:00
Anirudh Ramachandra 52b11c1d08
Expose the getOrCreate method via the InternalSharedXdsClientPoolProvider. This is needed for internal users to both set the bootstrap and interact with the XdsClient via the shared object pool (#10872) 2024-02-09 12:50:26 -08:00
Eric Anderson d568a8dc19 kokoro: Avoid using Kokoro's env_vars setting
It makes it hard to run ephemeral jobs, and we don't actually need it
here because we aren't changing the setting with manual runs.
2024-02-09 12:47:16 -08:00
Larry Safran 6847219657
Start 1.63.0 development cycle (#10906) 2024-02-08 13:11:38 -08:00
Eric Anderson 95b847e799 interop-testing: Use separate event loops in RetryTest
The RetryTest was flaky, and it seems to have been caused by the client
and server getting assigned to the same event loop. Separating the two
reduces the flake rate from ~3% to less than 0.1% (no flakes in a 1000).

While I was here fixing the executors, I reduced the number of threads
created and shut down the threads after they are no longer used. This
had no impact to the flake rate (no flakes in 1000).
2024-02-08 07:26:31 -08:00
Larry Safran 7ba0718bb9
Change log level from WARNING to FINER for expected exception (#10899)
* Change log level from WARNING to FINER.  Fixes #10816
2024-02-07 17:31:27 -08:00
Larry Safran 5faab7f47e
Revert "Fix retries that timeout hanging forever. (#10855)" (#10901)
This reverts commit c0a9d315d3.
2024-02-07 15:05:21 -08:00
yifeizhuang 03decafa1f
XdsClient is experimental (#10876) 2024-02-06 17:43:30 -08:00
yifeizhuang f6d9221b65
xds: hide TlsContextManager in XdsResourceType.Args (#10894) 2024-02-06 15:41:24 -08:00
Sergii Tkachenko 4f7ec131ec
xds: Googleapis proto sync to 2023-01-10 (#10896)
Sync googleapis protos to
googleapis/googleapis@114a745b28 for
consistency with Envoy and cncf/xds.

The same version is used in [envoy]
(62e7c59374/api/bazel/repository_locations.bzl (L69))
and [cncf/xds]
(0fa0005c9c/bazel/repository_locations.bzl (L23))
since Jan 2023.

Function-wise, this is a noop.
2024-02-05 18:52:55 -08:00
Sergii Tkachenko 68334a019a
xds: Envoy proto sync to 2024-01-24 (#10895)
`envoyproxy/envoy`: Sync protos to the latest imported version
147e6b9523
(commit 2024-01-24, cl/604403196).

Should be a noop, just a routine xDS proto update to make upcoming
RLQS-related imports simpler.
2024-02-05 17:21:51 -08:00
Larry Safran 374dbe9461
Fix flaky retry tests (#10887)
* Reorder tracing and actually closing listener to eliminate test flakiness
* Use real value rather than mock for flaky test
2024-02-05 10:54:55 -08:00
Anirudh Ramachandra 3202370684
Allow users to start watching xDS resources (#10864) 2024-02-01 09:21:00 -08:00
yifeizhuang a97f21b61e
xds: fix NPE in wrr in TF state (#10868) 2024-02-01 09:06:46 -08:00
Larry Safran c0a9d315d3
Fix retries that timeout hanging forever. (#10855)
Fixes #10336
2024-01-31 13:15:35 -08:00
yifeizhuang 8d280c97e3
xds: move filterRegistry and loadBalancerRegistry out of XdsResourceType.Args (#10843) 2024-01-31 09:45:37 -08:00
yifeizhuang 20abea47bc
xds: move tlsContextManager (#10859)
Minor refactor to the tlsContextManager to not expose itself on the xdsClientImpl constructor.
This is to allow people who plugins xdsTransportFactory to use the API easily.
2024-01-30 12:59:33 -08:00
Mateus Azis c2a3792a04
binder: Create a new RobolectricBinderSecurityTest (#10835)
Exercises async security policies codepaths.
2024-01-30 11:36:53 -08:00
Larry Safran ac428c14fd
Relax some timeouts that have been flaky for the automated CI testing. (#10858)
* Relax some timeouts that have been flaky for the automated CI testing.
2024-01-29 17:43:21 -08:00
Larry Safran ae5fdd5faa
Add check for empty addresses and if not there disable ipV6. (#10848)
* Add check for empty addresses and if not there dissable ipV6.
2024-01-29 17:20:11 -08:00
Larry Safran 7f4c16e068
internal:Happy eyeballs (#10731)
* implement happy eyeballs

---------

Co-authored-by: tonyjongyoonan <tonyjan@google.com>
2024-01-29 12:50:58 -08:00
yifeizhuang 8e1cc943b0
xds: change controlPlaneClient and loadReportClient to use xdsTransportFactory (#10829) 2024-01-25 17:05:12 -08:00
Anirudh Ramachandra 6d96e6588e
Allow users outside of io.grpc.xds package to create custom xDS resources (#10834)
Currently few of the interfaces needed to define and start a watch for a xDS resource are package private, which can't be used externally outside of io.grpc.xds. Exposing them outside allows users to define their own custom resources and start a watch along with the default supported resources.

Also as part of this change, move an Exception defined in the XdsClientImpl into XdsResourceType. As XdsClientImpl is an implementation package, it makes more sense to expose it via the XdsResourceType class.
2024-01-24 19:48:52 -08:00
Mateus Azis 3e8e56feea
Make AsyncSecurityPolicy.checkAuthorizationAsync public. (#10836)
I initially omitted the visibility modifier because this class began as an interface. Since it moved to an abstract class, we must make it public so it can be overriden by subclasses in the integrator's packages.

Part of #10566.
2024-01-22 15:54:48 -08:00
Mateus Azis 9b9e53359f
Add unit test coverage for PendingAuthListener. (#10806)
Part of #10566.
2024-01-22 15:06:16 -08:00