Commit Graph

1932 Commits

Author SHA1 Message Date
Larry Safran 6847219657
Start 1.63.0 development cycle (#10906) 2024-02-08 13:11:38 -08:00
Larry Safran 5faab7f47e
Revert "Fix retries that timeout hanging forever. (#10855)" (#10901)
This reverts commit c0a9d315d3.
2024-02-07 15:05:21 -08:00
Larry Safran 374dbe9461
Fix flaky retry tests (#10887)
* Reorder tracing and actually closing listener to eliminate test flakiness
* Use real value rather than mock for flaky test
2024-02-05 10:54:55 -08:00
Larry Safran c0a9d315d3
Fix retries that timeout hanging forever. (#10855)
Fixes #10336
2024-01-31 13:15:35 -08:00
Larry Safran 7f4c16e068
internal:Happy eyeballs (#10731)
* implement happy eyeballs

---------

Co-authored-by: tonyjongyoonan <tonyjan@google.com>
2024-01-29 12:50:58 -08:00
John Cormie 09acf2f6b7
transports can send a msg larger than their flow control window (#10842) 2024-01-22 10:30:32 -08:00
Larry Safran 4b2e5eddd2
api:Provide a default implementation for new method added to ManagedTranport.Listener to support ClientTransportFilters (#10795)
* Provide a default implementation for new method added to ManagedTransport.Listener to support ClientTransportFilters
* Relax test constraint to reduce flakiness due to timing.
* Add test for listener.filterTransport.
2024-01-04 14:19:13 -08:00
joybestourous 91d15ce4e6
Add ClientTransportFilter (#10646)
* Add ClientTransportFilter
2024-01-03 10:45:22 -08:00
Sergii Tkachenko 7692a9f5db
Start 1.62.0 development cycle (#10792) 2024-01-03 10:08:36 -08:00
Vladimir Gordiychuk 0f21574bb1
core: reduce CompositeReadableBuffer allocation (#3279)
* core: reduce CompositeReadableBuffer allocation

Add ability for CompositeReadableBuffer read specified length bytes
to another CompositeReadableBuffer instead of create temp
CompositeReadableBuffer.

fixes #3278
---------

Co-authored-by: Larry Safran <107004254+larry-safran@users.noreply.github.com>
2023-12-28 14:12:52 -08:00
James Roper 67b67f8382
Fix outbound message size checking (#10739)
This fixes two bugs in outbound message size checking:

* When thet checke failed, the thrown StatusRuntimeException with a status code
  of RESOURCE_EXHAUSTED was been caught and rewrapped in another
  StatusRuntimeException but this time with status code UNKNOWN.
* This applies the max outbound message size check to messages prior to, and
  after compression, since compression of a message that is smaller than the
  maximum send size can result in a larger message that exceeds the maximum
  send size.
2023-12-18 11:02:51 -08:00
Eric Anderson d6830d7f99
Change many api deps to implementation deps
These look pretty fair now, mostly only exposing grpc-api and
annotations as api dependencies.
2023-12-15 15:14:29 -08:00
Terry Wilson 8c4f4e08d9
Move Noop{Client,Server}Call from testing to api (#10753)
To help reduce cross dependencies, we move these test classes to the test
fixtures of grpc-api.
2023-12-14 23:00:30 -08:00
Eric Anderson 225a7d30c9 core: Remove test dependency on grpclb
It was just unnecessary, and is isn't great having lower-level tests
depend on higher-level packages.
2023-12-14 09:26:58 -08:00
Eric Anderson 201893f5e7 core: Improve language for leaked channel error
Originally you had to confirm that awaitTermination() returned true, but
that was annoying and useless, especially after calling shutdownNow().
The behavior was changed in ce2ae1fb because the awaitTermination()
detection logic could prevent the channel from getting garbage
collected.

Fixes #10732
2023-12-12 16:25:40 -08:00
Larry Safran e562a0558b
Remove Unused experimental API ManagedChannelBuilder.enableFullStreamDecompression. (#10744) 2023-12-12 14:56:22 -08:00
yifeizhuang 7567894e67
dual stack: generic health checking (#10623)
This change has health checking consumer (new pick first) to install a listener through and health checking producer (outlier detection and client health checking) producing health checks. Health notification chain is built reusing the previous connectivity state chain.
Pickfirst installs the health listener, and is capable of detecting when no health checking producer is installed in the system. In that case, it sets health status to be READY so that health system is no-op.
2023-12-11 17:49:56 -08:00
Eric Anderson 3b92333890 core: Detect NameResolverProviders passed as Factories
This may help some to move closer to Providers. It especially helps
cases where `NameResolverFactory`s aren't returning `InetSocketAddress`,
as it allows them to override `getProducedSocketAddressTypes()`, which
will now fail starting in 15fc70be.
2023-11-21 10:56:55 -08:00
Eric Anderson 32d85d8309 Start 1.61.0 development cycle 2023-11-15 07:41:12 -08:00
Vindhya Ningegowda d5544bbb02
opentelemetry : Implement metrics (#10593)
Adds a new module grpc-opentelemetry that integrates OpenTelemetry and focuses on metrics.

OpenTelemetry APIs are used for instrumenting metrics collection. Users are expected to provide SDK with implementations.

If no SDK is passed, by default gRPC uses OpenTelemetry.noop().
2023-11-13 10:58:16 -08:00
Larry Safran dfdd50bc79
xds:Make Ring Hash LB a petiole policy (#10610)
* Update picker logic per A61 that it no longer pays attention to the first 2 elements, but rather takes the first ring element not in TF and uses that.
---------
Pulled in by rebase:
Eric Anderson  (android: Remove unneeded proguard rule 44723b6)
Terry Wilson (stub: Deprecate StreamObservers b5434e8)
2023-11-09 13:46:52 -08:00
Sergii Tkachenko 8b4b14ac40
core: add description to Status.UNKNOWN in ServerImpl's #internalClose (#10643)
This is currently the only place where we return Status.UNKNOWN with no description, which makes is harder to debug and differentiate from statuses originated from non-grpc sources.

This PR enriches ServerImpl's #internalClose `Status.UNKNOWN` with description `Application error processing RPC`.
2023-11-06 12:31:30 -08:00
sanjaypujare 15fc70be2a
core, netty, okhttp: implement new logic for nameResolverFactory API in channelBuilder (#10590)
* core, netty, okhttp: implement new logic for nameResolverFactory API in channelBuilder
fix ManagedChannelImpl to use NameResolverRegistry instead of NameResolverFactory
fix the ManagedChannelImplBuilder and remove nameResolverFactory

* Integrate target parsing and NameResolverProvider searching

Actually creating the name resolver is now delayed to the end of
ManagedChannelImpl.getNameResolver; we don't want to call into the name
resolver to determine if we should use the name resolver.

Added getDefaultScheme() to NameResolverRegistry to avoid needing
NameResolver.Factory.
---------

Co-authored-by: Eric Anderson <ejona@google.com>
2023-11-03 09:57:59 -07:00
Terry Wilson 9888a54abd
lb: acceptResolvedAddresses() to return Status (#10636)
Instead of a boolean, we now return a Status object. Status.OK
represents accepted addresses and other non-acceptance. This allows the
LB to provide more information about why a set of addresses were not
acceptable.

The status will later be sent to the name resolver as well to allow it
to also better react to to bad addresses.
2023-11-03 09:02:46 -07:00
Sergii Tkachenko a294b27d52
core: Deprecate ForwardingChannelBuilder (#10587)
Deprecate `ForwardingChannelBuilder` in favor of `ForwardingChannelBuilder2`.
2023-11-02 10:58:20 -07:00
Larry Safran 0d39bf5018
lb:Implement LeastRequestLB as a petiole policy and restore RR and WRR (#10584)
* Change LeastRequest, Round Robin and WeightedRoundRobin into petiole policies
2023-10-16 16:40:20 -07:00
amirhadadi 8c0545564f
Avoid flushing headers when the server returns a single response (#9314)
* core,netty: avoid flushing the headers on the server side when the server has a single response.
2023-10-09 09:43:10 -07:00
Terry Wilson bf9ccc68f5 Start 1.60.0 development cycle 2023-10-04 10:29:43 -07:00
Sergii Tkachenko 0e03654add core: Remove temporary AbstractManagedChannelImplBuilder
This breaks the ABI of the classes listed below.

Users that recompiled  their code using grpc-java [`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) (released on
Feb 23, 2021) and later, ARE NOT AFFECTED.

Users that compiled their source using grpc-java earlier than
[`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) need to
recompile when upgrading to grpc-java `v1.59.0`. Otherwise the code
will fail on runtime with `NoSuchMethodError`. For example, code:

```java
NettyChannelBuilder.forTarget("localhost:100").maxRetryAttempts(2);
```

Will fail with

> `java.lang.NoSuchMethodError: 'io.grpc.internal.AbstractManagedChannelImplBuilder
  io.grpc.netty.NettyChannelBuilder.maxRetryAttempts(int)'`

**Affected classes**

Class `AbstractManagedChannelImplBuilder` is deleted, and no longer in
the class hierarchy of the channel builders:
- `io.grpc.netty.NettyChannelBuilder`
- `io.grpc.okhttp.OkhttpChannelBuilder`
- `grpc.cronet.CronetChannelBuilder`
2023-10-03 10:35:16 -07:00
Sergii Tkachenko 050ae18e2a core: Remove temporary AbstractServerImplBuilder
This breaks the ABI of the classes listed below.

Users that recompiled  their code using grpc-java [`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) (released on
Feb 23, 2021) and later, ARE NOT AFFECTED.

Users that compiled their source using grpc-java earlier than
[`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) need to
recompile when upgrading to grpc-java `v1.59.0`. Otherwise the code
will fail on runtime with `NoSuchMethodError`. For example, code:

```java
NettyServerBuilder.forPort(80).directExecutor();
```

Will fail with

> `java.lang.NoSuchMethodError: 'io.grpc.internal.AbstractServerImplBuilder
  io.grpc.netty.NettyServerBuilder.directExecutor()'`

**Affected classes**

Class `AbstractServerImplBuilder` is deleted, and no longer in the
class hierarchy of the server builders:
- `io.grpc.netty.NettyServerBuilder`
- `io.grpc.inprocess.InProcessServerBuilder`
2023-10-03 10:35:16 -07:00
Larry Safran bc784c0ef9
Revert "Change Round Robin and WeightedRoundRobin into petiole policies (#10528)" (#10575)
This reverts commit e1334eae7b.
2023-09-26 17:31:58 -07:00
Larry Safran e1334eae7b
Change Round Robin and WeightedRoundRobin into petiole policies (#10528)
* Change Round Robin and WeightedRoundRobin into petiole policies
2023-09-15 10:27:36 -07:00
yifeizhuang 6335e0be3e
core: DEADLINE_EXCEEDED gives hints for slow resolver (#10545) 2023-09-08 15:09:18 -07:00
Eric Anderson d4a163461a netty: Touch ByteBuf when message framing has been decoded
When a memory leak occurs, it is really helpful to have access records
to understand where the buffer was being held when it leaked. retain()
when we create the NettyReadableBuffer already creates an access record
the ByteBuf, so here we track when the ByteBuf is passed to another
thread.

See #8330
2023-09-07 13:15:22 -07:00
Stephane Landelle cb2b68e39d core: only try to resolve InternalCensusStatsAccessor once 2023-09-06 11:12:04 -07:00
Terry Wilson 36e0af66fe
core: de-expermentalize pick first config parsing (#10531) 2023-08-30 15:40:34 -07:00
Larry Safran 92174be3df
Log warning message when server side gets exception writing message to stream and allow multiple closes (#10513)
* Use internalClose instead of close when sendMessage has a RuntimeException.
* Change argument to internalClose to a Throwable instead of a Status.
* Rename internalClose to handleInternalError
2023-08-24 16:40:23 -07:00
yifeizhuang 9424f8b4c7
Start 1.59.0 development cycle (#10519) 2023-08-24 11:25:28 -07:00
Tony An fba7835de1
new pick first policy, architectural change (#10354) 2023-08-14 15:23:40 -07:00
sanjaypujare 41552bfd9a
all: generate automatic module name in the manifest (#10413) 2023-07-25 09:00:11 -07:00
yifeizhuang e179212672
core: fix retriablestream deadlock (#10386) 2023-07-21 13:48:37 -07:00
Sergii Tkachenko 419767f968
Upgrade dependencies post v1.57.x branch cut (#10359)
### Dependency updates

#### Update successfully to the latest
- [x] `androidx.core:core 1.10.0 -> 1.10.1`
- [x] `com.google.api.grpc:proto-google-common-protos 2.17.0 -> 2.22.0`
- [x] `com.google.cloud:google-cloud-logging 3.14.5 -> 3.15.5`
- [x] `com.google.truth:truth 1.0.1 -> 1.1.5`
- [x] `com.puppycrawl.tools:checkstyle 8.28 -> 10.12.1`
- [x] `org.robolectric:robolectric 4.9.2 -> 4.10.3`
- [x] Auto-value
   - [x] `com.google.auto.value:auto-value 1.10.1 -> 1.10.2`
   - [x] `com.google.auto.value:auto-value-annotations 1.10.1 -> 1.10.2`
- [x] Protobuf
   - [x] `com.google.protobuf:protobuf-java 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protobuf-java-util 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protobuf-javalite 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protoc 3.22.3 -> 3.23.4`
- [x] Errorprone
   - [x] `com.google.errorprone:error_prone_annotations 2.18.0 -> 2.20.0`
   - [x] `com.google.errorprone:error_prone_core 2.18.0 -> 2.20.0`
   - ~`libs.checkstylejava8 = com.puppycrawl.tools:checkstyle 9.3 -> 10.12.1`~ -- pinned to last version supporting java8, update not needed

#### Updated to non-latest
- [x] `com.squareup.okio:okio 1.17.5 ->` ~`3.4.0`~ `2.10.0` -- updating to 3.x failed due to them [introducing gradle multiplatform artifacts](https://github.com/square/okio/blob/master/CHANGELOG.md#version-320). Error in [the comment below](https://github.com/grpc/grpc-java/pull/10359#issuecomment-1632853307.).
- [x] `org.checkerframework:checker-qual 3.33.0 ->` ~`3.36.0`~ -- removed, no longer needed
- [x] Mockito - updated to `4.11.0`. Versions 5.x break some tests. Errors in [the comment below](https://github.com/grpc/grpc-java/pull/10359#issuecomment-1632834435).
   - `org.mockito:mockito-android 3.12.4 ->` ~`5.4.0`~ `4.11.0`
   - `org.mockito:mockito-core 3.12.4 ->` ~`5.4.0`~ `4.11.0`

#### Not updated
- Cronet -- upgrade failed, created a blocker ticket https://github.com/grpc/grpc-java/issues/10396
   - ~`org.chromium.net:cronet-api 108.5359.79 -> 113.5672.61`~
   - ~`org.chromium.net:cronet-embedded 108.5359.79 -> 113.5672.61`~
- Google auth -- upgrade blocked by https://github.com/grpc/grpc-java/pull/9118
   - ~`com.google.auth:google-auth-library-credentials 1.4.0 -> 1.19.0`~
   - ~`com.google.auth:google-auth-library-oauth2-http 1.4.0 -> 1.19.0`~
- Guava -- not upgraded: 32.1.0 [broke gradle modules](https://github.com/google/guava/releases/tag/v32.1.0)
   - ~`libs.guava = com.google.guava:guava 32.0.1-android -> 32.1.1-jre`~
   - ~`libs.guava.jre = com.google.guava:guava 32.0.1-jre -> 32.1.1-jre`~
   - ~`libs.guava.testlib = com.google.guava:guava-testlib 32.0.1-android -> 32.1.1-jre`~


### checkForUpdates results
```
> Task :checkForUpdates
androidx.core:core 1.10.0 -> 1.10.1
com.google.auto.value:auto-value 1.10.1 -> 1.10.2
com.google.auto.value:auto-value-annotations 1.10.1 -> 1.10.2
org.checkerframework:checker-qual 3.33.0 -> 3.36.0
com.puppycrawl.tools:checkstyle 8.28 -> 10.12.1
org.chromium.net:cronet-api 108.5359.79 -> 113.5672.61
org.chromium.net:cronet-embedded 108.5359.79 -> 113.5672.61
com.google.errorprone:error_prone_annotations 2.18.0 -> 2.20.0
com.google.errorprone:error_prone_core 2.18.0 -> 2.20.0
com.google.errorprone:error_prone_core 2.10.0 -> 2.20.0
com.google.api.grpc:proto-google-common-protos 2.17.0 -> 2.22.0
com.google.auth:google-auth-library-credentials 1.4.0 -> 1.19.0
com.google.auth:google-auth-library-oauth2-http 1.4.0 -> 1.19.0
com.google.cloud:google-cloud-logging 3.14.5 -> 3.15.5
com.google.guava:guava 32.0.1-android -> 32.1.1-jre
com.google.guava:guava 32.0.1-jre -> 32.1.1-jre
com.google.guava:guava-testlib 32.0.1-android -> 32.1.1-jre
org.mockito:mockito-android 3.12.4 -> 5.4.0
org.mockito:mockito-core 3.12.4 -> 5.4.0
com.squareup.okio:okio 1.17.5 -> 3.4.0
com.google.protobuf:protobuf-java 3.22.3 -> 3.23.4
com.google.protobuf:protobuf-java-util 3.22.3 -> 3.23.4
com.google.protobuf:protobuf-javalite 3.22.3 -> 3.23.4
com.google.protobuf:protoc 3.22.3 -> 3.23.4
org.robolectric:robolectric 4.9.2 -> 4.10.3
com.google.truth:truth 1.0.1 -> 1.1.5
```
2023-07-21 10:15:05 -07:00
Larry Safran 864d30caed
Handle header with errors and endStream = true (#10384)
* Eliminate NPE by skipping further processing when stream is defined, but doesn't have a property for streamKey (header processing identified an error)

Fixes #10364

* Add unit test for missing content type
2023-07-20 15:31:46 -07:00
sanjaypujare dea7fa09ff
core, services, util: export util from core and move gson dep to services for the checkUpperBoundDeps error (#10390) 2023-07-20 10:20:10 -07:00
sanjaypujare 0f5f07f876
core, inprocess, util: move inprocess and util code into their own new artifacts grpc-inprocess and grpc-util (#10362)
* core, inprocess, util: move inprocess and util code into their own new artifacts grpc-inprocess and grpc-util
2023-07-17 11:45:31 -07:00
Anirudh Ramachandra ac35ab67f2
xds: Encode the service authority in XdsNameResolver (#10207)
Encode the service authority before passing it into gRPC util in the xDS name resolver to handle xDS requests which might contain multiple slashes. Example: xds:///path/to/service:port.

As currently the underlying Java URI library does not break the encoded authority into host/port correctly simplify the check to just look for '@' as we are only interested in checking for user info to validate the authority for HTTP.

This change also leads to few changes in unit tests that relied on this check for invalid authorities which now will be considered valid.

Just like #9376, depending on Guava packages such as URLEscapers or PercentEscapers leads to internal failures(Ex: Unresolvable reference to com.google.common.escape.Escaper from io.grpc.internal.GrpcUtil). To avoid these issues create an in house version that is heavily inspired by grpc-go/grpc.
2023-07-14 12:54:18 -07:00
Larry Safran 0aaa2e0434
Bump version (#10378)
* Start 1.58.0 development cycle
2023-07-13 13:06:21 -07:00
Terry Wilson 78cf1c39cf
core: Apply RetryingNameResolver in ManagedChannelImpl (#10371)
Wrapping the DnsNameResolver in DnsNameResolverProvider can cause
problems to external name resolvers that delegate to a DnsResolver
already wrapped in RetryingNameResolver. ManagedChannelImpl would
end up wrapping these name resolvers again, causing an exception
later from a RetryingNameResolver safeguard that checks for double
wrapping.
2023-07-12 10:14:50 -07:00
Stephane Landelle 1e30cb6601
core,grpclb: Resolve isAndroid only once on class loading (#10345)
Motivation:

When multiple NameResolvers are created, the Classloader is scanned every time trying to figure out if the Platform is Android. This expensive work could be done only once.

Modification:

Cache isAndroid resolution in a constant.

Result:

Less expensive multiple NameResolvers instantiation.
2023-07-06 13:42:43 -07:00
sanjaypujare 2be568d70b
core: stabilize MutableHandlerRegistry by removing ExperimentalApi annotation (#10348) 2023-07-05 16:07:37 -07:00