Commit Graph

1952 Commits

Author SHA1 Message Date
Laglangyue 52e65ec0d8
minor: remove the unnecessary final,static (#11098) 2024-04-18 15:26:01 -07:00
Sergii Tkachenko e490273edd
netty: Handle write queue promise failures (#11016)
Handles Netty write frame failures caused by issues in the Netty
itself.

Normally we don't need to do anything on frame write failures because
the cause of a failed future would be an IO error that resulted in
the stream closure.  Prior to this PR we treated these issues as a
noop, except the initial headers write on the client side.

However, a case like netty/netty#13805 (a bug in generating next
stream id) resulted in an unclosed stream on our side. This PR adds
write frame future failure handlers that ensures the stream is
cancelled, and the cause is propagated via Status.

Fixes #10849
2024-04-16 16:27:51 -07:00
David Burns 00649913b0
bazel: Use the `artifact` macro for loading maven deps
The recommended way to load dependencies from `rules_jvm_external`
is to make use of the `@maven` workspace, and the most readable
way of doing that is to use the `artifact` macro provides.

This removes the need to generate the "compat" namespaces, which
`rules_jvm_external` provided for backwards compatibility with
older releases. This change also sets things up for supporting
`bzlmod`: this requires all workspaces accessed by a library to
be named "up front" in the `MODULE.bazel` file. This way, the
only repo that needs to be exported is `@maven`, rather than the
current huge list.
2024-03-28 14:33:32 -07:00
Larry Safran 4ef1baddd2
core: Remove useless ExperimentalApi annotation from pick first
PickFirstLeafLoadBalancer is in internal and also isn't public.
2024-03-28 08:07:53 -07:00
Eric Anderson 569b426d62 core: Improve clarity of RpcProgress meanings 2024-03-27 16:11:26 -07:00
Kannan J 2c5f0c22cd
core: Transition to CONNECTING immediately when exiting idle
The name resolver takes some time before it returns addresses. While waiting the channel will be IDLE instead of the proper CONNECTING. This generally doesn't matter since RPCs behave similarly for IDLE and CONNECTING, but is confusing for users when watching channel.getState() closely.

Fixes #10517.
2024-03-27 11:40:13 -07:00
Larry Safran bdb623031f
Fix retry race condition that can lead to double decrementing inFlightSubStreams and so miss calling closed (#11026) 2024-03-22 10:01:58 -07:00
yifeizhuang b3ffb5078d
Start 1.64.0 development cycle (#11030) 2024-03-22 09:32:10 -07:00
Eric Anderson 3abab95e75
core: Provide DEADLINE_EXCEEDED insights for context deadline
We provided extra details when the RPC is killed by CallOptions'
Deadline, but didn't do the same for Context.

To avoid duplicating code, things were restructured, including the
threading. There are more code flows now, but I think the
multi-threading came out more obvious and less error-prone. I didn't
change the status when the deadline is already expired, because the
text is shared with DelayedClientCall and AbstractInteropTest doesn't
distinguish between the two cases.

This is a roll-forward that avoids a NPE when cancel() is called
without an earlier call to start().

As seen at b/300991330
2024-03-21 17:02:06 -07:00
Larry Safran 51f811df86
Enable Happy Eyeballs by default (#11022)
* Flip the flag

* Fix test flakiness where IPv6 was not considered loopback
2024-03-21 16:59:54 -07:00
James Duong 2c83ef0632
Allow configuration of the queued byte threshold at which a Stream is considered not ready (#10977)
* Allow the queued byte threshold for a Stream to be ready to be configurable

- on clients this is exposed by setting a CallOption
- on servers this is configured by calling a method on ServerCall or ServerStreamListener
2024-03-21 15:37:26 -07:00
Terry Wilson 68eb639b1c
Revert "core: Provide DEADLINE_EXCEEDED insights for context deadline" (#11024)
This reverts commit 0e31ac9303.
2024-03-19 10:06:01 -07:00
Eric Anderson 0e31ac9303 core: Provide DEADLINE_EXCEEDED insights for context deadline
We provided extra details when the RPC is killed by CallOptions'
Deadline, but didn't do the same for Context.

To avoid duplicating code, things were restructured, including the
threading. There are more code flows now, but I think the
multi-threading came out more obvious and less error-prone. I didn't
change the status when the deadline is already expired, because the
text is shared with DelayedClientCall and AbstractInteropTest doesn't
distinguish between the two cases.

As seen at b/300991330
2024-03-15 10:44:44 -07:00
Larry Safran 36e9f0dfac
core: Eliminate NPE seen in PickFirstLeafLoadBalancer (#11013)
ref b/329420531
2024-03-13 20:53:34 -07:00
Larry Safran 8a9ce990b0
Enable new PF by default (#11002) 2024-03-12 14:50:40 -07:00
Larry Safran 27a3d8e278
Eliminate validation so that code path is more heavily exercised (#11005)
* Eliminate validation so that code path is more heavily exercised
2024-03-12 13:41:03 -07:00
Larry Safran d1c406bd23
Prepare to switch flag to use new PickFirstLeafLoadBalancer by default (#10998)
* Fix PickFirstLeafLoadBalancer and tests to work when it is used.
* Actually use EAG attributes for subchannels.
2024-03-11 14:12:56 -07:00
Eric Anderson ac62c8b055 Fix tests and warnings on Java 17
SelfSignedCertificate is not available on Java 17 because
OpenJdkSelfSignedCertGenerator is not available. This only impacted
tests.

AccessController is being removed, and these locations are doing simple
reflection which is unlikely to require it even when a security policy
is in effect. There's other places we do reflection without the
AccessController, so either no security policies care or the users can
update their policies to allow it.
2024-02-29 16:55:46 -08:00
Eric Anderson af117e9764 core: Don't use real time when comparing deadline in ClientCallImplTest
We can just compare the Deadline instances instead of asserting that
very little time has passed during the test. Real time probably still
matters in the test, but only insofar that the deadline is not expired
by the time ClientCallImpl sees it.

This fixes a test failure seen in the emulated aarch64 CI. Note that the
message says "ns" when it should say "ms", but this change deletes the
code with that typo.

```
java.lang.AssertionError: timeout: 548 ns
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at io.grpc.internal.ClientCallImplTest.assertTimeoutBetween(ClientCallImplTest.java:1102)
	at io.grpc.internal.ClientCallImplTest.contextDeadlineShouldBePropagatedToStream(ClientCallImplTest.java:828)
```
2024-02-27 12:54:58 -08:00
Eric Anderson a231d80756 Remove semi-circular dependency between core and util
Add the 'fake' dependency to grpc-netty instead of grpc-core.
grpc-okhttp already depends on grpc-util and probably would be fine
without round_robin on Android.

There's not actually a circular dependency, but some tools can't handle
the compile vs runtime distinction. Such tools are broken, but fixes
have been slow and this approach works with no real downfalls.

Works around #10576 #10701
2024-02-16 12:31:54 -08:00
Larry Safran 6847219657
Start 1.63.0 development cycle (#10906) 2024-02-08 13:11:38 -08:00
Larry Safran 5faab7f47e
Revert "Fix retries that timeout hanging forever. (#10855)" (#10901)
This reverts commit c0a9d315d3.
2024-02-07 15:05:21 -08:00
Larry Safran 374dbe9461
Fix flaky retry tests (#10887)
* Reorder tracing and actually closing listener to eliminate test flakiness
* Use real value rather than mock for flaky test
2024-02-05 10:54:55 -08:00
Larry Safran c0a9d315d3
Fix retries that timeout hanging forever. (#10855)
Fixes #10336
2024-01-31 13:15:35 -08:00
Larry Safran 7f4c16e068
internal:Happy eyeballs (#10731)
* implement happy eyeballs

---------

Co-authored-by: tonyjongyoonan <tonyjan@google.com>
2024-01-29 12:50:58 -08:00
John Cormie 09acf2f6b7
transports can send a msg larger than their flow control window (#10842) 2024-01-22 10:30:32 -08:00
Larry Safran 4b2e5eddd2
api:Provide a default implementation for new method added to ManagedTranport.Listener to support ClientTransportFilters (#10795)
* Provide a default implementation for new method added to ManagedTransport.Listener to support ClientTransportFilters
* Relax test constraint to reduce flakiness due to timing.
* Add test for listener.filterTransport.
2024-01-04 14:19:13 -08:00
joybestourous 91d15ce4e6
Add ClientTransportFilter (#10646)
* Add ClientTransportFilter
2024-01-03 10:45:22 -08:00
Sergii Tkachenko 7692a9f5db
Start 1.62.0 development cycle (#10792) 2024-01-03 10:08:36 -08:00
Vladimir Gordiychuk 0f21574bb1
core: reduce CompositeReadableBuffer allocation (#3279)
* core: reduce CompositeReadableBuffer allocation

Add ability for CompositeReadableBuffer read specified length bytes
to another CompositeReadableBuffer instead of create temp
CompositeReadableBuffer.

fixes #3278
---------

Co-authored-by: Larry Safran <107004254+larry-safran@users.noreply.github.com>
2023-12-28 14:12:52 -08:00
James Roper 67b67f8382
Fix outbound message size checking (#10739)
This fixes two bugs in outbound message size checking:

* When thet checke failed, the thrown StatusRuntimeException with a status code
  of RESOURCE_EXHAUSTED was been caught and rewrapped in another
  StatusRuntimeException but this time with status code UNKNOWN.
* This applies the max outbound message size check to messages prior to, and
  after compression, since compression of a message that is smaller than the
  maximum send size can result in a larger message that exceeds the maximum
  send size.
2023-12-18 11:02:51 -08:00
Eric Anderson d6830d7f99
Change many api deps to implementation deps
These look pretty fair now, mostly only exposing grpc-api and
annotations as api dependencies.
2023-12-15 15:14:29 -08:00
Terry Wilson 8c4f4e08d9
Move Noop{Client,Server}Call from testing to api (#10753)
To help reduce cross dependencies, we move these test classes to the test
fixtures of grpc-api.
2023-12-14 23:00:30 -08:00
Eric Anderson 225a7d30c9 core: Remove test dependency on grpclb
It was just unnecessary, and is isn't great having lower-level tests
depend on higher-level packages.
2023-12-14 09:26:58 -08:00
Eric Anderson 201893f5e7 core: Improve language for leaked channel error
Originally you had to confirm that awaitTermination() returned true, but
that was annoying and useless, especially after calling shutdownNow().
The behavior was changed in ce2ae1fb because the awaitTermination()
detection logic could prevent the channel from getting garbage
collected.

Fixes #10732
2023-12-12 16:25:40 -08:00
Larry Safran e562a0558b
Remove Unused experimental API ManagedChannelBuilder.enableFullStreamDecompression. (#10744) 2023-12-12 14:56:22 -08:00
yifeizhuang 7567894e67
dual stack: generic health checking (#10623)
This change has health checking consumer (new pick first) to install a listener through and health checking producer (outlier detection and client health checking) producing health checks. Health notification chain is built reusing the previous connectivity state chain.
Pickfirst installs the health listener, and is capable of detecting when no health checking producer is installed in the system. In that case, it sets health status to be READY so that health system is no-op.
2023-12-11 17:49:56 -08:00
Eric Anderson 3b92333890 core: Detect NameResolverProviders passed as Factories
This may help some to move closer to Providers. It especially helps
cases where `NameResolverFactory`s aren't returning `InetSocketAddress`,
as it allows them to override `getProducedSocketAddressTypes()`, which
will now fail starting in 15fc70be.
2023-11-21 10:56:55 -08:00
Eric Anderson 32d85d8309 Start 1.61.0 development cycle 2023-11-15 07:41:12 -08:00
Vindhya Ningegowda d5544bbb02
opentelemetry : Implement metrics (#10593)
Adds a new module grpc-opentelemetry that integrates OpenTelemetry and focuses on metrics.

OpenTelemetry APIs are used for instrumenting metrics collection. Users are expected to provide SDK with implementations.

If no SDK is passed, by default gRPC uses OpenTelemetry.noop().
2023-11-13 10:58:16 -08:00
Larry Safran dfdd50bc79
xds:Make Ring Hash LB a petiole policy (#10610)
* Update picker logic per A61 that it no longer pays attention to the first 2 elements, but rather takes the first ring element not in TF and uses that.
---------
Pulled in by rebase:
Eric Anderson  (android: Remove unneeded proguard rule 44723b6)
Terry Wilson (stub: Deprecate StreamObservers b5434e8)
2023-11-09 13:46:52 -08:00
Sergii Tkachenko 8b4b14ac40
core: add description to Status.UNKNOWN in ServerImpl's #internalClose (#10643)
This is currently the only place where we return Status.UNKNOWN with no description, which makes is harder to debug and differentiate from statuses originated from non-grpc sources.

This PR enriches ServerImpl's #internalClose `Status.UNKNOWN` with description `Application error processing RPC`.
2023-11-06 12:31:30 -08:00
sanjaypujare 15fc70be2a
core, netty, okhttp: implement new logic for nameResolverFactory API in channelBuilder (#10590)
* core, netty, okhttp: implement new logic for nameResolverFactory API in channelBuilder
fix ManagedChannelImpl to use NameResolverRegistry instead of NameResolverFactory
fix the ManagedChannelImplBuilder and remove nameResolverFactory

* Integrate target parsing and NameResolverProvider searching

Actually creating the name resolver is now delayed to the end of
ManagedChannelImpl.getNameResolver; we don't want to call into the name
resolver to determine if we should use the name resolver.

Added getDefaultScheme() to NameResolverRegistry to avoid needing
NameResolver.Factory.
---------

Co-authored-by: Eric Anderson <ejona@google.com>
2023-11-03 09:57:59 -07:00
Terry Wilson 9888a54abd
lb: acceptResolvedAddresses() to return Status (#10636)
Instead of a boolean, we now return a Status object. Status.OK
represents accepted addresses and other non-acceptance. This allows the
LB to provide more information about why a set of addresses were not
acceptable.

The status will later be sent to the name resolver as well to allow it
to also better react to to bad addresses.
2023-11-03 09:02:46 -07:00
Sergii Tkachenko a294b27d52
core: Deprecate ForwardingChannelBuilder (#10587)
Deprecate `ForwardingChannelBuilder` in favor of `ForwardingChannelBuilder2`.
2023-11-02 10:58:20 -07:00
Larry Safran 0d39bf5018
lb:Implement LeastRequestLB as a petiole policy and restore RR and WRR (#10584)
* Change LeastRequest, Round Robin and WeightedRoundRobin into petiole policies
2023-10-16 16:40:20 -07:00
amirhadadi 8c0545564f
Avoid flushing headers when the server returns a single response (#9314)
* core,netty: avoid flushing the headers on the server side when the server has a single response.
2023-10-09 09:43:10 -07:00
Terry Wilson bf9ccc68f5 Start 1.60.0 development cycle 2023-10-04 10:29:43 -07:00
Sergii Tkachenko 0e03654add core: Remove temporary AbstractManagedChannelImplBuilder
This breaks the ABI of the classes listed below.

Users that recompiled  their code using grpc-java [`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) (released on
Feb 23, 2021) and later, ARE NOT AFFECTED.

Users that compiled their source using grpc-java earlier than
[`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) need to
recompile when upgrading to grpc-java `v1.59.0`. Otherwise the code
will fail on runtime with `NoSuchMethodError`. For example, code:

```java
NettyChannelBuilder.forTarget("localhost:100").maxRetryAttempts(2);
```

Will fail with

> `java.lang.NoSuchMethodError: 'io.grpc.internal.AbstractManagedChannelImplBuilder
  io.grpc.netty.NettyChannelBuilder.maxRetryAttempts(int)'`

**Affected classes**

Class `AbstractManagedChannelImplBuilder` is deleted, and no longer in
the class hierarchy of the channel builders:
- `io.grpc.netty.NettyChannelBuilder`
- `io.grpc.okhttp.OkhttpChannelBuilder`
- `grpc.cronet.CronetChannelBuilder`
2023-10-03 10:35:16 -07:00
Sergii Tkachenko 050ae18e2a core: Remove temporary AbstractServerImplBuilder
This breaks the ABI of the classes listed below.

Users that recompiled  their code using grpc-java [`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) (released on
Feb 23, 2021) and later, ARE NOT AFFECTED.

Users that compiled their source using grpc-java earlier than
[`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) need to
recompile when upgrading to grpc-java `v1.59.0`. Otherwise the code
will fail on runtime with `NoSuchMethodError`. For example, code:

```java
NettyServerBuilder.forPort(80).directExecutor();
```

Will fail with

> `java.lang.NoSuchMethodError: 'io.grpc.internal.AbstractServerImplBuilder
  io.grpc.netty.NettyServerBuilder.directExecutor()'`

**Affected classes**

Class `AbstractServerImplBuilder` is deleted, and no longer in the
class hierarchy of the server builders:
- `io.grpc.netty.NettyServerBuilder`
- `io.grpc.inprocess.InProcessServerBuilder`
2023-10-03 10:35:16 -07:00