Commit Graph

1907 Commits

Author SHA1 Message Date
Larry Safran 0d39bf5018
lb:Implement LeastRequestLB as a petiole policy and restore RR and WRR (#10584)
* Change LeastRequest, Round Robin and WeightedRoundRobin into petiole policies
2023-10-16 16:40:20 -07:00
amirhadadi 8c0545564f
Avoid flushing headers when the server returns a single response (#9314)
* core,netty: avoid flushing the headers on the server side when the server has a single response.
2023-10-09 09:43:10 -07:00
Terry Wilson bf9ccc68f5 Start 1.60.0 development cycle 2023-10-04 10:29:43 -07:00
Sergii Tkachenko 0e03654add core: Remove temporary AbstractManagedChannelImplBuilder
This breaks the ABI of the classes listed below.

Users that recompiled  their code using grpc-java [`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) (released on
Feb 23, 2021) and later, ARE NOT AFFECTED.

Users that compiled their source using grpc-java earlier than
[`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) need to
recompile when upgrading to grpc-java `v1.59.0`. Otherwise the code
will fail on runtime with `NoSuchMethodError`. For example, code:

```java
NettyChannelBuilder.forTarget("localhost:100").maxRetryAttempts(2);
```

Will fail with

> `java.lang.NoSuchMethodError: 'io.grpc.internal.AbstractManagedChannelImplBuilder
  io.grpc.netty.NettyChannelBuilder.maxRetryAttempts(int)'`

**Affected classes**

Class `AbstractManagedChannelImplBuilder` is deleted, and no longer in
the class hierarchy of the channel builders:
- `io.grpc.netty.NettyChannelBuilder`
- `io.grpc.okhttp.OkhttpChannelBuilder`
- `grpc.cronet.CronetChannelBuilder`
2023-10-03 10:35:16 -07:00
Sergii Tkachenko 050ae18e2a core: Remove temporary AbstractServerImplBuilder
This breaks the ABI of the classes listed below.

Users that recompiled  their code using grpc-java [`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) (released on
Feb 23, 2021) and later, ARE NOT AFFECTED.

Users that compiled their source using grpc-java earlier than
[`v1.36.0`]
(https://github.com/grpc/grpc-java/releases/tag/v1.36.0) need to
recompile when upgrading to grpc-java `v1.59.0`. Otherwise the code
will fail on runtime with `NoSuchMethodError`. For example, code:

```java
NettyServerBuilder.forPort(80).directExecutor();
```

Will fail with

> `java.lang.NoSuchMethodError: 'io.grpc.internal.AbstractServerImplBuilder
  io.grpc.netty.NettyServerBuilder.directExecutor()'`

**Affected classes**

Class `AbstractServerImplBuilder` is deleted, and no longer in the
class hierarchy of the server builders:
- `io.grpc.netty.NettyServerBuilder`
- `io.grpc.inprocess.InProcessServerBuilder`
2023-10-03 10:35:16 -07:00
Larry Safran bc784c0ef9
Revert "Change Round Robin and WeightedRoundRobin into petiole policies (#10528)" (#10575)
This reverts commit e1334eae7b.
2023-09-26 17:31:58 -07:00
Larry Safran e1334eae7b
Change Round Robin and WeightedRoundRobin into petiole policies (#10528)
* Change Round Robin and WeightedRoundRobin into petiole policies
2023-09-15 10:27:36 -07:00
yifeizhuang 6335e0be3e
core: DEADLINE_EXCEEDED gives hints for slow resolver (#10545) 2023-09-08 15:09:18 -07:00
Eric Anderson d4a163461a netty: Touch ByteBuf when message framing has been decoded
When a memory leak occurs, it is really helpful to have access records
to understand where the buffer was being held when it leaked. retain()
when we create the NettyReadableBuffer already creates an access record
the ByteBuf, so here we track when the ByteBuf is passed to another
thread.

See #8330
2023-09-07 13:15:22 -07:00
Stephane Landelle cb2b68e39d core: only try to resolve InternalCensusStatsAccessor once 2023-09-06 11:12:04 -07:00
Terry Wilson 36e0af66fe
core: de-expermentalize pick first config parsing (#10531) 2023-08-30 15:40:34 -07:00
Larry Safran 92174be3df
Log warning message when server side gets exception writing message to stream and allow multiple closes (#10513)
* Use internalClose instead of close when sendMessage has a RuntimeException.
* Change argument to internalClose to a Throwable instead of a Status.
* Rename internalClose to handleInternalError
2023-08-24 16:40:23 -07:00
yifeizhuang 9424f8b4c7
Start 1.59.0 development cycle (#10519) 2023-08-24 11:25:28 -07:00
Tony An fba7835de1
new pick first policy, architectural change (#10354) 2023-08-14 15:23:40 -07:00
sanjaypujare 41552bfd9a
all: generate automatic module name in the manifest (#10413) 2023-07-25 09:00:11 -07:00
yifeizhuang e179212672
core: fix retriablestream deadlock (#10386) 2023-07-21 13:48:37 -07:00
Sergii Tkachenko 419767f968
Upgrade dependencies post v1.57.x branch cut (#10359)
### Dependency updates

#### Update successfully to the latest
- [x] `androidx.core:core 1.10.0 -> 1.10.1`
- [x] `com.google.api.grpc:proto-google-common-protos 2.17.0 -> 2.22.0`
- [x] `com.google.cloud:google-cloud-logging 3.14.5 -> 3.15.5`
- [x] `com.google.truth:truth 1.0.1 -> 1.1.5`
- [x] `com.puppycrawl.tools:checkstyle 8.28 -> 10.12.1`
- [x] `org.robolectric:robolectric 4.9.2 -> 4.10.3`
- [x] Auto-value
   - [x] `com.google.auto.value:auto-value 1.10.1 -> 1.10.2`
   - [x] `com.google.auto.value:auto-value-annotations 1.10.1 -> 1.10.2`
- [x] Protobuf
   - [x] `com.google.protobuf:protobuf-java 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protobuf-java-util 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protobuf-javalite 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protoc 3.22.3 -> 3.23.4`
- [x] Errorprone
   - [x] `com.google.errorprone:error_prone_annotations 2.18.0 -> 2.20.0`
   - [x] `com.google.errorprone:error_prone_core 2.18.0 -> 2.20.0`
   - ~`libs.checkstylejava8 = com.puppycrawl.tools:checkstyle 9.3 -> 10.12.1`~ -- pinned to last version supporting java8, update not needed

#### Updated to non-latest
- [x] `com.squareup.okio:okio 1.17.5 ->` ~`3.4.0`~ `2.10.0` -- updating to 3.x failed due to them [introducing gradle multiplatform artifacts](https://github.com/square/okio/blob/master/CHANGELOG.md#version-320). Error in [the comment below](https://github.com/grpc/grpc-java/pull/10359#issuecomment-1632853307.).
- [x] `org.checkerframework:checker-qual 3.33.0 ->` ~`3.36.0`~ -- removed, no longer needed
- [x] Mockito - updated to `4.11.0`. Versions 5.x break some tests. Errors in [the comment below](https://github.com/grpc/grpc-java/pull/10359#issuecomment-1632834435).
   - `org.mockito:mockito-android 3.12.4 ->` ~`5.4.0`~ `4.11.0`
   - `org.mockito:mockito-core 3.12.4 ->` ~`5.4.0`~ `4.11.0`

#### Not updated
- Cronet -- upgrade failed, created a blocker ticket https://github.com/grpc/grpc-java/issues/10396
   - ~`org.chromium.net:cronet-api 108.5359.79 -> 113.5672.61`~
   - ~`org.chromium.net:cronet-embedded 108.5359.79 -> 113.5672.61`~
- Google auth -- upgrade blocked by https://github.com/grpc/grpc-java/pull/9118
   - ~`com.google.auth:google-auth-library-credentials 1.4.0 -> 1.19.0`~
   - ~`com.google.auth:google-auth-library-oauth2-http 1.4.0 -> 1.19.0`~
- Guava -- not upgraded: 32.1.0 [broke gradle modules](https://github.com/google/guava/releases/tag/v32.1.0)
   - ~`libs.guava = com.google.guava:guava 32.0.1-android -> 32.1.1-jre`~
   - ~`libs.guava.jre = com.google.guava:guava 32.0.1-jre -> 32.1.1-jre`~
   - ~`libs.guava.testlib = com.google.guava:guava-testlib 32.0.1-android -> 32.1.1-jre`~


### checkForUpdates results
```
> Task :checkForUpdates
androidx.core:core 1.10.0 -> 1.10.1
com.google.auto.value:auto-value 1.10.1 -> 1.10.2
com.google.auto.value:auto-value-annotations 1.10.1 -> 1.10.2
org.checkerframework:checker-qual 3.33.0 -> 3.36.0
com.puppycrawl.tools:checkstyle 8.28 -> 10.12.1
org.chromium.net:cronet-api 108.5359.79 -> 113.5672.61
org.chromium.net:cronet-embedded 108.5359.79 -> 113.5672.61
com.google.errorprone:error_prone_annotations 2.18.0 -> 2.20.0
com.google.errorprone:error_prone_core 2.18.0 -> 2.20.0
com.google.errorprone:error_prone_core 2.10.0 -> 2.20.0
com.google.api.grpc:proto-google-common-protos 2.17.0 -> 2.22.0
com.google.auth:google-auth-library-credentials 1.4.0 -> 1.19.0
com.google.auth:google-auth-library-oauth2-http 1.4.0 -> 1.19.0
com.google.cloud:google-cloud-logging 3.14.5 -> 3.15.5
com.google.guava:guava 32.0.1-android -> 32.1.1-jre
com.google.guava:guava 32.0.1-jre -> 32.1.1-jre
com.google.guava:guava-testlib 32.0.1-android -> 32.1.1-jre
org.mockito:mockito-android 3.12.4 -> 5.4.0
org.mockito:mockito-core 3.12.4 -> 5.4.0
com.squareup.okio:okio 1.17.5 -> 3.4.0
com.google.protobuf:protobuf-java 3.22.3 -> 3.23.4
com.google.protobuf:protobuf-java-util 3.22.3 -> 3.23.4
com.google.protobuf:protobuf-javalite 3.22.3 -> 3.23.4
com.google.protobuf:protoc 3.22.3 -> 3.23.4
org.robolectric:robolectric 4.9.2 -> 4.10.3
com.google.truth:truth 1.0.1 -> 1.1.5
```
2023-07-21 10:15:05 -07:00
Larry Safran 864d30caed
Handle header with errors and endStream = true (#10384)
* Eliminate NPE by skipping further processing when stream is defined, but doesn't have a property for streamKey (header processing identified an error)

Fixes #10364

* Add unit test for missing content type
2023-07-20 15:31:46 -07:00
sanjaypujare dea7fa09ff
core, services, util: export util from core and move gson dep to services for the checkUpperBoundDeps error (#10390) 2023-07-20 10:20:10 -07:00
sanjaypujare 0f5f07f876
core, inprocess, util: move inprocess and util code into their own new artifacts grpc-inprocess and grpc-util (#10362)
* core, inprocess, util: move inprocess and util code into their own new artifacts grpc-inprocess and grpc-util
2023-07-17 11:45:31 -07:00
Anirudh Ramachandra ac35ab67f2
xds: Encode the service authority in XdsNameResolver (#10207)
Encode the service authority before passing it into gRPC util in the xDS name resolver to handle xDS requests which might contain multiple slashes. Example: xds:///path/to/service:port.

As currently the underlying Java URI library does not break the encoded authority into host/port correctly simplify the check to just look for '@' as we are only interested in checking for user info to validate the authority for HTTP.

This change also leads to few changes in unit tests that relied on this check for invalid authorities which now will be considered valid.

Just like #9376, depending on Guava packages such as URLEscapers or PercentEscapers leads to internal failures(Ex: Unresolvable reference to com.google.common.escape.Escaper from io.grpc.internal.GrpcUtil). To avoid these issues create an in house version that is heavily inspired by grpc-go/grpc.
2023-07-14 12:54:18 -07:00
Larry Safran 0aaa2e0434
Bump version (#10378)
* Start 1.58.0 development cycle
2023-07-13 13:06:21 -07:00
Terry Wilson 78cf1c39cf
core: Apply RetryingNameResolver in ManagedChannelImpl (#10371)
Wrapping the DnsNameResolver in DnsNameResolverProvider can cause
problems to external name resolvers that delegate to a DnsResolver
already wrapped in RetryingNameResolver. ManagedChannelImpl would
end up wrapping these name resolvers again, causing an exception
later from a RetryingNameResolver safeguard that checks for double
wrapping.
2023-07-12 10:14:50 -07:00
Stephane Landelle 1e30cb6601
core,grpclb: Resolve isAndroid only once on class loading (#10345)
Motivation:

When multiple NameResolvers are created, the Classloader is scanned every time trying to figure out if the Platform is Android. This expensive work could be done only once.

Modification:

Cache isAndroid resolution in a constant.

Result:

Less expensive multiple NameResolvers instantiation.
2023-07-06 13:42:43 -07:00
sanjaypujare 2be568d70b
core: stabilize MutableHandlerRegistry by removing ExperimentalApi annotation (#10348) 2023-07-05 16:07:37 -07:00
Terry Wilson 8192ff6103
core: ManagedChannelImpl to always use RetryingNameResolver (#10328)
ManagedCahnnelImpl did not make sure to use a RetryingNameResolver if
authority was not overriden. This was not a problem for DNS name
resolution as the DNS name resolver factory explicitly returns a
RetryingNameResolver. For polling name resolvers that do not do this in
their factories (like the grpclb name resolver) this meant not having retry
at all.
2023-06-29 13:47:23 -07:00
sanjaypujare 4d2c3aac0e
context, all: move Context classes to grpc-api (#10313)
* context, all: move Context classes to grpc-api
clean up grpc-context since it has no source code: only add dep on grpc-api
add exclusion for all transitive deps of grpc-api - only guava
exclude grpc-context as a dependency from grpc-alts because all context code is in grpc-api now
api: 1.7 as target Java version for Context source-set of grpc-api

* core, census: fix the issues with android project pulling in old grpc-context version


* api,context: make changes to bazel build files to account for context code moving from context to api
2023-06-28 18:28:30 -07:00
Eric Anderson f46793e82d Remove unneeded dependency excludes
Since 44847bf4e, when we upgraded our JUnit version, the JUnit
exclusions have probably not been necessary. e0ac97c4f upgraded
Robolectric to a version that had the auto.service problem fixed.
2023-06-23 11:18:10 -07:00
Larry Safran f1de820c19
Change delay for hedging retry after a non-fatal error to be 0 to match the gRFC A6. (#10293)
Fixes #10145
2023-06-21 14:53:02 -07:00
Sergii Tkachenko 17b2df78ec
Start 1.57.0 development cycle (#10229) 2023-05-30 18:02:40 -07:00
Daniel Liu 5a27e3ead2
services, xds, orca: support EPS in client-side WRR (#10177) 2023-05-26 12:52:42 -07:00
Eric Anderson cb42405217 testing: Move some internal classes to testFixtures
These two classes are precisely the purpose of test fixtures. Move them
close to the classes they make easier to use in tests.
2023-05-25 14:03:42 -07:00
Terry Wilson dd2dc21de2
api: Stabilize io.grpc.CallCredentials (#10208) 2023-05-22 09:59:53 -07:00
Eric Anderson 29b8483fd6
Use test fixtures instead of sourceSets.test.output
This avoids the (often missing) evaluationDependsOn and fixes using
results from other projects without propagating those through
Configuration. It also reduces the number of useless classes pulled in
by down-stream tests, reducing the probability of rebuilds.

The expectation of fixtures is they help testing down-stream code that
use the classes in main. That applies to all the classes here except for
FakeClock and StaticTestingClassLoader. It would also apply to many
internal classes in grpc-testing, but let's consider cleaning that up
future work.
2023-05-16 12:10:13 -07:00
Eric Anderson 02a4cb5c69 Fix throwing LBProvider.parseLoadBalancingConfig() impls
LoadBalancers in general should never throw, but
parseLoadBalancingConfig() in particular has a return value to
communicate the error. Throwing can be a bit unpredictable, but at its
most trivial form causes a channel panic. There's no reason to throw
explicitly and calls to JsonUtil have to be protected by a try-catch
because it can throw.
2023-05-10 12:42:05 -07:00
Eric Anderson f229aed538 core: Hide RR's ReadyPicker and EmptyPicker
ReadyPicker hasn't been necessary since 111ff60e, when we stopped
calling super.pickSubchannel(). EmptyPicker was only used in tests, and
we can just compare the class name instead of doing an instanceof check.

Unfortunately, calling getClass() caused Java to start casting the
return value of pickerCaptor.getValue() based on its generics. Captors
don't verify the type they capture, so using any type other than
SubchannelPicker for the pickerCaptor is misleading and hides a cast.
2023-05-09 13:25:19 -07:00
Eric Anderson 847ea7cfc9 Upgrade Mockito to 3.12.4
MockitoAnnotations.initMocks() is deprecated.
2023-05-08 16:39:42 -07:00
Terry Wilson 3c01bfe77f
core: Optional address shuffle in PickFirstLoadBalancer (#10110)
If provided with the new PickFirstLoadBalancerConfig,
PickFirstLoadBalancer will shuffle the list of addresses it receives
from the name resolver.

PickFirstLoadBalancerProvider will now support the new config
if enabled by an env variable.
2023-04-28 15:50:18 -07:00
Terry Wilson fbc8679f7a
core: Sticky TRANSIENT_FAILURE in PickFirstLoadBalancer (#10106)
When the subchannel is transitioning from TRANSIENT_FAILURE to either
IDLE or CONNECTING we will not update the LB state. Additionally, if
the subchannel becomes idle we request a new connection so that the
subchannel will keep on trying to establish a connection.
2023-04-27 15:43:27 -07:00
Eric Anderson 8ca99f661d core: Pass Locality to toLowerCase() in GrpcUtil
We pretty much never want to use the system locality, across all our
code.
2023-04-24 10:53:12 -07:00
Eric Anderson 27edab007d Start 1.56.0 development cycle 2023-04-21 15:56:00 -07:00
Larry Safran 7255c8dfa8
retries:Remove early commit for transparent retries with none remaining. (#10066)
* retries:Remove early commit for transparent retries when no retries or no hedging remain.

Fixes #10011
2023-04-19 13:42:59 -07:00
Carl Mastrangelo 4e3ee4471e
all: use PerfMark.traceTask (#9950)
* all: use PerfMark.traceTask

* make linter happy
2023-04-18 10:46:54 -07:00
yifeizhuang 14ba959545
census: add delayed name resolution tracing annotation (#10044) 2023-04-14 11:44:50 -07:00
Terry Wilson f2b976266e Revert "core: ForwardingLoadBalancer to forward acceptResolvedAddresses() (#10050)"
This reverts commit 086edf0f20.
2023-04-14 10:12:33 -07:00
Terry Wilson 086edf0f20
core: ForwardingLoadBalancer to forward acceptResolvedAddresses() (#10050) 2023-04-13 18:23:11 -07:00
yifeizhuang fc4410f159
api, census: add new pendingStreamCreated on clientStreamTracer and new tracer annotation (#10014) 2023-04-11 13:49:33 -07:00
Eric Anderson 8d98e5ff7f core: Fix NPE race during hedging
The problem was one hedge was committed before another had drained
start(). This was not testable because HedgingRunnable checks whether
scheduledHedgingRef is cancelled, which is racy, but there's no way to
deterministically trigger either race.

The same problem couldn't be triggered with retries because only one
attempt will be draining at a time. Retries with cancellation also
couldn't trigger it, for the surprising reason that the noop stream used
in cancel() wasn't considered drained.

This commit marks the noop stream as drained with cancel(), which allows
memory to be garbage collected sooner and exposes the race for tests.
That then showed the stream as hanging, because inFlightSubStreams
wasn't being decremented.

Fixes #9185
2023-04-11 13:10:41 -07:00
Terry Wilson 3332199106
Start 1.55.0 development cycle (#9939) 2023-03-07 18:14:31 -08:00
Benjamin Peterson 4b6853b6dd
errorprone: enable UnnecessaryAnonymousClass (#9927)
The Java 7 compatibility rationale no longer holds.
2023-03-03 15:16:17 -08:00