Commit Graph

5981 Commits

Author SHA1 Message Date
Larry Safran f90656293f
Mark MultiChildLoadBalancer as Internal. (#10481)
* Mark MultiChildLoadBalancer as Internal.  Cannot move to the internal package because of its use of classes in the util package.

* Exclude MultiChildLoadBalancer from javadoc generation.

* Fix javadoc creation.
2023-08-15 17:33:23 -07:00
Eric Anderson b5d7f1394d xds: Fix import scripts deleting the wrong files, use of trap, and full git clone
The scripts used `git rev-parse --show-toplevel` so it appeared they
could be used from any directory. But references to "GIT_BASE_DIR"
weren't absolute, so it did matter the starting directory. And it
mattered in a big way for xds/import.sh as if you ran it from the
grpc-java directory it would delete the xds directory in grpc-java, not
third_party.

The trap that deleted the GIT_BASE_DIR was very broken. In addition to
potentially deleting the wrong directory, it was unnecessary because
that directory was in tmpdir. But you can only have one trap per signal,
so this unnecessary trap disabled the trap that deleted tmpdir.

The script needed a full clone because it needed to check out a specific
commit. To work with --depth 1 you have to use some convoluted syntax.
But just downloading a tar.gz is easy and seems should work fine on Mac.

protoc-gen-validate/import.sh didn't have the trap problem, but seemed
to have drifted from the other scritps. All the scripts were synced to
match.
2023-08-15 16:54:32 -07:00
Mohan Li 5850de2f5f
pick_first: de-experiment pick first (#10475) 2023-08-15 15:48:24 -07:00
Eric Anderson 9585bc948a build.gradle: Modernize japicmp confiuration
Even though we don't really use japicmp, the configuration was resolving
artifacts even when the task was not executed. After some clean up, the
configuration looks more ordinary.
2023-08-14 17:01:38 -07:00
Eric Anderson 849186ac35
examples: Add pre-serialized-message example (#10112)
This came out of the question #9707, and could be useful to others.
2023-08-14 17:00:29 -07:00
Tony An fba7835de1
new pick first policy, architectural change (#10354) 2023-08-14 15:23:40 -07:00
Eric Anderson 778c209751 java_grpc_library.bzl: Allow toolchain to use annotation processors
While not easy to use because java_grpc_library() uses a fixed
toolchain, it is possible for downstream users to apply a patch to the
repo to add their own annotation processors. This feature was added
inside Google so exporting it reduces the diff between internal and
external and causes no harm.

cl/280287611
2023-08-14 12:51:18 -07:00
Eric Anderson 47a84c48c7 RELEASING.md: Prefix grpc/grpc commit with [interop]
They require this style.
2023-08-14 10:55:49 -07:00
Eric Anderson f8baa9ca1d Upgrade protobuf-java and protoc to 3.24.0 2023-08-14 09:20:39 -07:00
Mohan Li cebb4659a1
test: allow set request/response size in interop soak test (#10465) 2023-08-10 10:30:41 -07:00
Eric Anderson 3b61799f73 okhttp: Add OkHttpServerProvider
This allows okhttp to service the Grpc.newServerBuilderForPort() API.
Note that, unlike Netty, it will throw if you try to use
ServerBuilder.forPort().

This fixes android-interop-testing which was broken by c9864a119.
2023-08-09 13:58:34 -07:00
Larry Safran a0d8f2eb31
Create a shared LB base class for LBs with multiple children (#10448)
* Create a shared LB base class for LBs with multiple children and change ClusgterManagerLoadBalancer to use it.
2023-08-08 14:36:27 -07:00
Terry Wilson 4453ce7eb6
util: Outlier detection tracer delegation (#10459)
OutlierDetectionLoadBalancer did not delegate calls to an existing
ClientStreamTracer from the tracer it installed. This change has the OD
tracer delegate all calls to the underlying one.
2023-08-08 14:35:44 -07:00
Eric Anderson c9864a1198 android-interop-testing: Remove usage of Netty for grpc server
We have the OkHttp server these days, so we don't need to use Netty. Use
the generic API instead of hard-coding OkHttp.

We've seen some recent interop failures. We aren't entirely sure what is
going on, but we have seen some Netty usages in logcat. Since we don't
even want Netty on Android, just get rid of it and even if it doesn't
help with the failures things are better dependency-wise.
2023-08-08 14:30:32 -07:00
Tony An 40bff673c0
xds: replace random with atomic sequence in WRR (#10458) 2023-08-07 16:19:50 -07:00
Eric Anderson 4049f89e13
README.md: Revert reference to protobuf 3.23.4 (#10430)
The README describes the released version of gRPC. But the most recent
release doesn't use protobuf 3.23.4. Telling people to use protoc 3.23.4
is asking for breakages because they will be running with protobuf-java
3.22.3, which is a downgrade.

The bump to 3.23.4 was incorrectly done in #10359.
2023-08-03 10:53:38 -07:00
Eric Anderson 1638ce39f8 .github: Use checkout@v3 for Bazel
The other jobs were swapped to v3 in 38311e8730. Bazel was added later
with the older version.

This fixes the warning:
```
The following actions uses node12 which is deprecated and will be forced to run on node16: actions/checkout@v2. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
```
2023-08-03 07:28:22 -07:00
Eric Anderson 1e5480847a Update README etc to reference 1.57.1 2023-08-02 14:56:26 -07:00
Eric Anderson f3f4ed4ef3 Upgrade to Gradle 8.2.1 and upgrade plugins
Most changes are migrating from conventions to the equivalent
extensions. JMH, AppEngine, and Jib plugins trigger future compatibility
warnings with `--warning-mode all`.

The movement of configurations was to allow sourceSets to create the
configurations and then we just configure them. When configurations were
before sourceSets, we'd implicitly create the configuration.

The examples were _not_ updated to the newer Gradle, although the
non-Android examples work with the newer Gradle. The Android examples
use an older Android Gradle Plugin which will need to be upgraded first.
https://github.com/grpc/grpc-java/issues/10445
2023-08-02 13:29:44 -07:00
Zach Reyes 6d48271fa4
buildscripts: Add bootstrap generator to cron job that runs nightly (#10440) 2023-08-02 13:04:56 -07:00
Eric Anderson b67a29af3b all: Copy mustRunAfter from each project's JacocoTestReport
While JacocoReportBase.getExecutionData() is properly annotated with
`@InputFiles` and is a FileCollection, which can propagate dependencies,
the JacocoPlugin's configuration of the task [goes through a
File][barefile], which doesn't propagate task dependency information.
This is overcome in the plugin by mustRunAfter. This may have been on
purpose to allow jacocoTestReport to report on the subset of tests
already run, but is atypical these days.

Because of that, we need to copy the mustRunAfter from the other
JacocoTestReports. The dependsOn weren't doing anything because there
were no dependsOn; that configuration is quite old so it's hard to say
when they stopped functioning.

This fixes warnings like:
```
  - Gradle detected a problem with the following location: 'grpc-java/core/build/jacoco/test.exec'. Reason: Task ':grpc-all:jacocoTestReport' uses this output of task ':grpc-core:test' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.6/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: 'grpc-java/testing/build/resources/main'. Reason: Task ':grpc-all:jacocoTestReport' uses this output of task ':grpc-testing:processResources' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.6/userguide/validation_problems.html#implicit_dependency for more details about this problem.
```

I was able to trigger the warnings by:
```
./gradlew clean
./gradlew :grpc-core:test
./gradlew :grpc-all:jacocoTestReport
./gradlew :grpc-core:test :grpc-all:jacocoTestReport
```

[barefile]: bc4029063c/subprojects/jacoco/src/main/java/org/gradle/testing/jacoco/tasks/JacocoReportBase.java (L162-L168)
2023-08-02 10:41:28 -07:00
sanjaypujare a589eacc8f
stub: remove 2 deprecated methods from MetadataUtils (#10443) 2023-08-01 11:00:23 -07:00
Eric Anderson 1dee7efca4 Avoid accidental Java 9+ dependency with --release
In d654707 we swapped compiling the uploaded artifacts to Java 11. This
caused ABI issues with ByteBuffer, like clear() returning ByteBuffer
instead of Buffer.

There are source-level approaches to avoid the accidental ABI dependency
on Java 11, but we have no tool able to detect such breakages.
We use Animalsniffer for similar cases, but it fails to detect these[1].
Since we have no tool, source-level approaches can't gain the necessary
confidence that all incompatibility fixes have been resolved.

Java has had javac-level ways to address this, but they used to require
setting bootclasspath. Since Java 9, though, they made it easier and we
can use --release, which does exactly what we need.

Fixes #10432

1. https://github.com/mojohaus/animal-sniffer/issues/77
2023-07-31 17:51:49 -07:00
Halvard Skogsrud 4dc786b5be
Fix locality logging (#10423)
The bootstrapping code currently does not log zone and subZone from locality correctly, and only logs region. This commit fixes the logging message format.
2023-07-31 17:44:16 -07:00
Eric Anderson 8121e8118e api: Link to gRFCs A8/A9 for keepalive and related settings
This is to give users a place to look to learn more.
2023-07-31 16:46:41 -07:00
Tony An d9a464e725
Context.Storage deprecated method attach() removed, doAttach() made abstract (#10379) 2023-07-31 16:38:01 -07:00
Tony An 3c19325587
removed assert for flakey static stride scheduler with multiple threads (#10437)
on master branch
2023-07-31 14:03:00 -07:00
Larry Safran 944de93f79
Minor cleanup in releasing instructions (#10426) 2023-07-30 20:06:48 -07:00
Larry Safran 621fbad78c
Reduce test flakiness (#10429) 2023-07-28 14:55:37 -07:00
Larry Safran 31d5b40672
Update README etc to reference 1.57.0 (#10427) 2023-07-28 14:44:13 -07:00
Kun Zhang c5c37ac51c
binder/test: Stop running Robolectric tests on APIs < 19 (#10385)
This is a port of internal cl/547560711, but with a change to call
isDeviceOwner that requires sdk 18 unconditioally
2023-07-26 10:44:56 -07:00
sanjaypujare 41552bfd9a
all: generate automatic module name in the manifest (#10413) 2023-07-25 09:00:11 -07:00
Larry Safran 08a44b9224
Handle header with errors and endStream = true (#10384) (#10412)
* Eliminate NPE by skipping further processing when stream is defined, but doesn't have a property for streamKey (header processing identified an error)

Fixes #10364

* Add unit test for missing content type
2023-07-24 14:33:07 -07:00
Eric Anderson a77e1a240a Downgrade Netty to 4.1.93-Final
Our benchmarks saw a serious performance decrease with the upgrade from
Netty 4.1.88 to 4.1.94. The problem was tracked down to a single PR in
4.1.94, so we avoid 4.1.94 for now.

Fixes #10401
2023-07-21 17:41:41 -07:00
yifeizhuang e179212672
core: fix retriablestream deadlock (#10386) 2023-07-21 13:48:37 -07:00
Larry Safran afa4d6dac8
Have rls's LRU Cache rely on cleanup process to remove expired entries (#10400)
* Add test for multiple targets with cache expiration.
2023-07-21 12:12:19 -07:00
Sergii Tkachenko 419767f968
Upgrade dependencies post v1.57.x branch cut (#10359)
### Dependency updates

#### Update successfully to the latest
- [x] `androidx.core:core 1.10.0 -> 1.10.1`
- [x] `com.google.api.grpc:proto-google-common-protos 2.17.0 -> 2.22.0`
- [x] `com.google.cloud:google-cloud-logging 3.14.5 -> 3.15.5`
- [x] `com.google.truth:truth 1.0.1 -> 1.1.5`
- [x] `com.puppycrawl.tools:checkstyle 8.28 -> 10.12.1`
- [x] `org.robolectric:robolectric 4.9.2 -> 4.10.3`
- [x] Auto-value
   - [x] `com.google.auto.value:auto-value 1.10.1 -> 1.10.2`
   - [x] `com.google.auto.value:auto-value-annotations 1.10.1 -> 1.10.2`
- [x] Protobuf
   - [x] `com.google.protobuf:protobuf-java 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protobuf-java-util 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protobuf-javalite 3.22.3 -> 3.23.4`
   - [x] `com.google.protobuf:protoc 3.22.3 -> 3.23.4`
- [x] Errorprone
   - [x] `com.google.errorprone:error_prone_annotations 2.18.0 -> 2.20.0`
   - [x] `com.google.errorprone:error_prone_core 2.18.0 -> 2.20.0`
   - ~`libs.checkstylejava8 = com.puppycrawl.tools:checkstyle 9.3 -> 10.12.1`~ -- pinned to last version supporting java8, update not needed

#### Updated to non-latest
- [x] `com.squareup.okio:okio 1.17.5 ->` ~`3.4.0`~ `2.10.0` -- updating to 3.x failed due to them [introducing gradle multiplatform artifacts](https://github.com/square/okio/blob/master/CHANGELOG.md#version-320). Error in [the comment below](https://github.com/grpc/grpc-java/pull/10359#issuecomment-1632853307.).
- [x] `org.checkerframework:checker-qual 3.33.0 ->` ~`3.36.0`~ -- removed, no longer needed
- [x] Mockito - updated to `4.11.0`. Versions 5.x break some tests. Errors in [the comment below](https://github.com/grpc/grpc-java/pull/10359#issuecomment-1632834435).
   - `org.mockito:mockito-android 3.12.4 ->` ~`5.4.0`~ `4.11.0`
   - `org.mockito:mockito-core 3.12.4 ->` ~`5.4.0`~ `4.11.0`

#### Not updated
- Cronet -- upgrade failed, created a blocker ticket https://github.com/grpc/grpc-java/issues/10396
   - ~`org.chromium.net:cronet-api 108.5359.79 -> 113.5672.61`~
   - ~`org.chromium.net:cronet-embedded 108.5359.79 -> 113.5672.61`~
- Google auth -- upgrade blocked by https://github.com/grpc/grpc-java/pull/9118
   - ~`com.google.auth:google-auth-library-credentials 1.4.0 -> 1.19.0`~
   - ~`com.google.auth:google-auth-library-oauth2-http 1.4.0 -> 1.19.0`~
- Guava -- not upgraded: 32.1.0 [broke gradle modules](https://github.com/google/guava/releases/tag/v32.1.0)
   - ~`libs.guava = com.google.guava:guava 32.0.1-android -> 32.1.1-jre`~
   - ~`libs.guava.jre = com.google.guava:guava 32.0.1-jre -> 32.1.1-jre`~
   - ~`libs.guava.testlib = com.google.guava:guava-testlib 32.0.1-android -> 32.1.1-jre`~


### checkForUpdates results
```
> Task :checkForUpdates
androidx.core:core 1.10.0 -> 1.10.1
com.google.auto.value:auto-value 1.10.1 -> 1.10.2
com.google.auto.value:auto-value-annotations 1.10.1 -> 1.10.2
org.checkerframework:checker-qual 3.33.0 -> 3.36.0
com.puppycrawl.tools:checkstyle 8.28 -> 10.12.1
org.chromium.net:cronet-api 108.5359.79 -> 113.5672.61
org.chromium.net:cronet-embedded 108.5359.79 -> 113.5672.61
com.google.errorprone:error_prone_annotations 2.18.0 -> 2.20.0
com.google.errorprone:error_prone_core 2.18.0 -> 2.20.0
com.google.errorprone:error_prone_core 2.10.0 -> 2.20.0
com.google.api.grpc:proto-google-common-protos 2.17.0 -> 2.22.0
com.google.auth:google-auth-library-credentials 1.4.0 -> 1.19.0
com.google.auth:google-auth-library-oauth2-http 1.4.0 -> 1.19.0
com.google.cloud:google-cloud-logging 3.14.5 -> 3.15.5
com.google.guava:guava 32.0.1-android -> 32.1.1-jre
com.google.guava:guava 32.0.1-jre -> 32.1.1-jre
com.google.guava:guava-testlib 32.0.1-android -> 32.1.1-jre
org.mockito:mockito-android 3.12.4 -> 5.4.0
org.mockito:mockito-core 3.12.4 -> 5.4.0
com.squareup.okio:okio 1.17.5 -> 3.4.0
com.google.protobuf:protobuf-java 3.22.3 -> 3.23.4
com.google.protobuf:protobuf-java-util 3.22.3 -> 3.23.4
com.google.protobuf:protobuf-javalite 3.22.3 -> 3.23.4
com.google.protobuf:protoc 3.22.3 -> 3.23.4
org.robolectric:robolectric 4.9.2 -> 4.10.3
com.google.truth:truth 1.0.1 -> 1.1.5
```
2023-07-21 10:15:05 -07:00
Larry Safran 9f78b2bd3c
Revert "Change the default for staleAge to be maxAge - 1 minute rather than maxage (unless maxAge is < 2 minutes) for the RLS configuration from proto. (#10397)" (#10399)
This reverts commit 56d1c42c80.
2023-07-20 15:32:35 -07:00
Larry Safran 864d30caed
Handle header with errors and endStream = true (#10384)
* Eliminate NPE by skipping further processing when stream is defined, but doesn't have a property for streamKey (header processing identified an error)

Fixes #10364

* Add unit test for missing content type
2023-07-20 15:31:46 -07:00
Larry Safran 56d1c42c80
Change the default for staleAge to be maxAge - 1 minute rather than maxage (unless maxAge is < 2 minutes) for the RLS configuration from proto. (#10397) 2023-07-20 10:43:08 -07:00
sanjaypujare dea7fa09ff
core, services, util: export util from core and move gson dep to services for the checkUpperBoundDeps error (#10390) 2023-07-20 10:20:10 -07:00
Eric Anderson 3f60746a98
servlet: Remove unused dependency on grpc-util in tests
Also convert to testFixtures, like we did elsewhere.

grpc-testing had previously been listed, but because of the missing
comma, it was not an actual dependency.
2023-07-20 07:57:11 -07:00
Eric Anderson b526a68622 Restrict checkUpperBoundDeps to library dependencies
Guava 32.1.0 added Gradle Module Metadata to their artifact (they still
use Maven for their build). Because of that, we're now seeing less
normal dependencies that were confusing checkUpperBoundDeps.  Obviously
'null' is not the version we were looking for.

We aren't upgrading to Guava 32.1.x yet as there's still other problems.

```
Execution failed for task ':grpc-census:checkUpperBoundDeps'.
> Maven version skew: com.google.guava:guava-parent (32.1.1-jre != null) Bad version dependency path: [project ':grpc-census', com.google.guava:guava:32.1.1-android] Run './gradlew :grpc-census:dependencies --configuration runtimeClasspath' to diagnose
```
2023-07-19 17:17:05 -07:00
Sergii Tkachenko e3016cf7f0
Stabilize setExtensionRegistry() of ProtoLiteUtils and ProtoUtils (#10392)
Closes #1787
2023-07-19 15:38:42 -07:00
Sergii Tkachenko 313c343729
Stabilize setMessageCompression() (#10393)
Closes #1703
2023-07-19 15:38:32 -07:00
Eric Anderson fefbd12c7e Remove unnecessary thisUsesUnstableApi from CallCredentials impls
This method has been unnecessary/deprecated since dd2dc21d.
2023-07-18 13:38:24 -07:00
Eric Anderson e8f36fa522 all: Remove grpc-context from grpc-all
grpc-context is empty, so isn't needed. Including it also causes the
warning:
```
> Task :grpc-all:javadoc
Resolution of the configuration :grpc-context:compileClasspath was attempted from a context different than the project context. Have a look at the documentation to understand why this is a problem and how it can be resolved. This behaviour has been deprecated and is scheduled to be removed in Gradle 8.0. See https://docs.gradle.org/7.6/userguide/viewing_debugging_dependencies.html#sub:resolving-unsafe-configuration-resolution-errors for more details.
```
2023-07-18 11:20:18 -07:00
sanjaypujare 0f5f07f876
core, inprocess, util: move inprocess and util code into their own new artifacts grpc-inprocess and grpc-util (#10362)
* core, inprocess, util: move inprocess and util code into their own new artifacts grpc-inprocess and grpc-util
2023-07-17 11:45:31 -07:00
Anirudh Ramachandra ac35ab67f2
xds: Encode the service authority in XdsNameResolver (#10207)
Encode the service authority before passing it into gRPC util in the xDS name resolver to handle xDS requests which might contain multiple slashes. Example: xds:///path/to/service:port.

As currently the underlying Java URI library does not break the encoded authority into host/port correctly simplify the check to just look for '@' as we are only interested in checking for user info to validate the authority for HTTP.

This change also leads to few changes in unit tests that relied on this check for invalid authorities which now will be considered valid.

Just like #9376, depending on Guava packages such as URLEscapers or PercentEscapers leads to internal failures(Ex: Unresolvable reference to com.google.common.escape.Escaper from io.grpc.internal.GrpcUtil). To avoid these issues create an in house version that is heavily inspired by grpc-go/grpc.
2023-07-14 12:54:18 -07:00
Larry Safran 0aaa2e0434
Bump version (#10378)
* Start 1.58.0 development cycle
2023-07-13 13:06:21 -07:00