Commit Graph

5262 Commits

Author SHA1 Message Date
Penn (Dapeng) Zhang f987de7497 xds: migrate EnvoyServerProtoData.Listener data types to AutoValue 2022-02-10 08:21:21 -08:00
ZHANG Dapeng da617e6ecd
rls: fix service name and method name separation
Should consistently use the last '/' in the full method name as the separator between service name and method name in MethodDescriptor.
2022-02-09 15:36:08 -08:00
nafeabd b35506b144
netty: Include both x86 and Arm64 epoll as dependency for grpc-netty-shaded 2022-02-09 13:21:36 -08:00
ZHANG Dapeng dc6eacccbf
core: limit total number of local-only transparent retries
Limit the total number of local-only transparent retries per RPC for the moment to mitigate any potential bug that would trigger infinite loop of transparent retries. If the limit is exceeded, fail the RPC.
2022-02-09 13:20:07 -08:00
Eric Anderson 7a35e3bbde android: fix code style of onBlockedStatusChanged() 2022-02-09 10:28:32 -08:00
ZHANG Dapeng 39c8f4e584
rls: migrate data types to AutoValue
Refactor to use `@AutoValue` for data types. This reduces human mistakes on `equals()`, `hashCode()`, and `toString()` while we are constantly adding and changing member fields of the data type.
2022-02-08 21:40:56 -08:00
DNVindhya 5b6ee1c176
observability: implement GrpcLogRecord proto (#8882)
* observability: implement GrpcLogRecord proto
- add configureProtoCompilation() to build.gradle for compiling protos
2022-02-08 10:20:47 -08:00
ZHANG Dapeng a1c41e3d30
xds/federation: fix percent encoding on server side
Overlooked in #8857 on server side. Since `XdsNameResolver.percentEncodePath()` will be also used for server side, I moved the method to `XdsClient`.
2022-02-07 13:29:09 -08:00
Eric Anderson 7a9ceacafc
benchmarks: Modernize client to use target and credentials
This allows using LoadClient with xDS.

The changes to SocketAddressValidator are a bit hacky, but we really
don't care about cleanliness there. Eventually on client-side, we should
be deleting the unix special case entirely, as we'll have a unix name
resolver.

Fixes #8877
2022-02-07 13:11:04 -08:00
sanjaypujare f0a7132fbe
xds: fix the validation code to accept new-style CertificateProviderPluginInstance wherever used (#8892) 2022-02-07 11:43:17 -08:00
ZHANG Dapeng 431fb0255f
core/netty: infinite local-only transparent retry for netty (#8878)
In core, add a new enum element to `RpcProgress` for the case that the stream is closed even before anything leaves the client. `RetriableStream` will do unlimited transparent retry for this type of `RpcProgress` since they are local-only.

In netty, call `tranportReportStatus()` for pending streams on failure.

Also fixes #8394
2022-02-04 22:12:03 -08:00
yifeizhuang 467985e958
add kokoro config for basic tests xds k8s (#8888) 2022-02-04 11:28:13 -08:00
Sergii Tkachenko c46d2c276a
core: test JsonUtil.getObject with a map containing a null value (#8881)
Verifies the behavior of JsonUtil.getObject when the map contains a null value for a given key.

Note: this may be incorrect behavior. Issue to track the investigation: #8883.
2022-02-04 09:56:26 -08:00
sanjaypujare a661515421
observability: revert previous visibility changes made in the provider APIs (#8889)
change visibility back to protected for certain methods of Providers
2022-02-04 08:38:46 -08:00
litclimbing ca4a1d8ca3
android: fix for app coming to foreground
When an app goes to the background, onBlockedStatusChanged is called with true and then called with false when it comes back to the foreground. The function onAvailable isn't called in this case and the connection wasn't being reset. Closes #8850

I noticed the comment that this is used for API versions 24+ but onBlockedStatusChanged was added in 29. I'm not sure if some kind of guard needs to be added or not.
https://developer.android.com/reference/android/net/ConnectivityManager.NetworkCallback#onBlockedStatusChanged(android.net.Network,%20boolean)
2022-02-03 15:56:43 -08:00
ZHANG Dapeng 7308d92034
rls: support routeLookupChannelServiceConfig in RLS lb config
Implementing the latest change for RLS lb config.

```
The configuration for the LB policy will be of the following form:

{
  "routeLookupConfig": <JSON form of RouteLookupConfig proto>,
  "routeLookupChannelServiceConfig": {...service config JSON...},
  "childPolicy": [
    {"<policy name>": {...child policy config...}}
  ],
  "childPolicyConfigTargetFieldName": "<name of field>"
}
```

>If the routeLookupChannelServiceConfig field is present, we will pass the specified service config to the RLS control plane channel, and we will disable fetching service config via that channel's resolver.
2022-02-01 14:08:15 -08:00
sanjaypujare bd156f98d6
observability: implement and integrate LoggingServerProvider into Observability (#8879) 2022-02-01 10:05:59 -08:00
sanjaypujare 881f747b7e
observability: implement Observability.grpcInit() and LoggingChannelProvider (#8872) 2022-01-31 10:37:54 -08:00
yifeizhuang 5635c6cb44 Update README etc to reference 1.44.0 2022-01-27 20:24:11 -08:00
Zhouyihai Ding 46563b75bf
core: include cause when logging Status in InternalSubchannel (#8846)
It would be good to print Cause when the transport is shutdown and has throwable exception messages.

The current log doesn't have this information for debugging:
`SHUTDOWN with UNAVAILABLE(io exception Channel Pipeline: [HttpProxyHandler$HttpClientCodecWrapper#0, HttpProxyHandler#0, TsiHandshakeHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0])`
2022-01-27 11:07:14 -08:00
sanjaypujare 5be09ec215
observability: create the grpc-observability artifact (#8870) 2022-01-27 09:58:19 -08:00
ZHANG Dapeng 7c49e5657f
rls: fix RLS lb name
The lb name of RLS lb should be "rls_experimental" instead of "rls-experimental", using underscore like "round_robin".
2022-01-25 12:02:28 -08:00
ZHANG Dapeng c59cc11e7a
Update RELEASING.md to clarify a step in tagging process 2022-01-25 12:01:59 -08:00
John Cormie 128324540f
binder: Fix a ServiceConnection leak (#8861)
Closes #8726
2022-01-24 19:54:21 -08:00
ZHANG Dapeng b29c3ec021
xds/federation: validate and canonify resource name
On reading a new `xdstp`: resource name, do a validation on the URI and canonify the query params.
2022-01-24 16:55:43 -08:00
ZHANG Dapeng 1231ce686e
xds/federation: fix percent encode
Fix percent encoding to comply with [RFC-3986 section 3.3](https://datatracker.ietf.org/doc/html/rfc3986#section-3.3) as specified in [gRFC A47](367ba33a0a/A47-xds-federation.md).
2022-01-24 10:55:19 -08:00
ZHANG Dapeng 6b0009d850
xds/federation: allow ConfigSource to have its self field set
Adopting the change in the [spec](367ba33a0a/A47-xds-federation.md (xds-api-changes)):

>Currently, for the ConfigSource fields in the LDS resource that points to the RDS resource and in the CDS resource that points to the EDS resource, gRPC requires the ConfigSource to have its ads field set. As part of supporting federation, gRPC will now also allow the ConfigSource to have its self field set. Both fields will have the same meaning.
2022-01-21 18:44:45 -08:00
Eric Anderson 41f2ad2540 RELEASING.md: Use git merge-base instead of cherry for backport summary
git cherry is too conservative in determining backports. Showing all
commits between the branch point and the release is more reliable.
2022-01-20 11:06:44 -08:00
ZHANG Dapeng 07567eebe6
xds: XdsNameResolver change to support RouteAction with RLS plugin
Implementation of the xDS Resolver section of the design http://go/grpc-rls-in-xds/view#heading=h.wkxepad0knu
2022-01-19 12:55:22 -08:00
Erik Johansson a35336c15f
xds: implement least_request load balancing policy (#8739)
Implements least_request_experimental as defined by
[A48](https://github.com/grpc/proposal/blob/master/A48-xds-least-request-lb-policy.md)

These tests are mostly just a copy of
RoundRobinLoadBalancerTest.
The main difference is currently in the pickerLeastRequest test case.
All other tests should be the same.
2022-01-19 10:14:24 -08:00
Eric Anderson 2c5a9e2aed
xds: Handle negative random numbers in c2p resolver
This was noticed because Mockito can't mock Random in Java 17, so it was
replaced with actual Random. But when doing that change it exposed that
negative numbers would cause the id to have a double '-'.
2022-01-18 12:38:04 -08:00
ZHANG Dapeng d1e0be6919
all: fix various gradle build warnings 2022-01-18 10:18:16 -08:00
Penn (Dapeng) Zhang cf4cd65707 Revert "all: clean up code related to android api level less than 19"
This reverts commit 3ad4d9bfb7.
2022-01-18 10:14:50 -08:00
Penn (Dapeng) Zhang 3179bc3be0 Revert "use charset from StandardCharsets instead of 'Charset.forName' (#8779)"
This reverts commit a74a3ad834.
2022-01-18 10:14:50 -08:00
ZHANG Dapeng d28f718c84
xds: PriorityLoadBalancer should treat IDLE in the same way as READY (#8837) 2022-01-18 09:58:30 -08:00
Kurt Alfred Kluever d5f22b77b2 netty: Assign the result of a @CheckReturnValue'ed constructor to an unused variable
This fixes a soon-to-be compile error via ErrorProne.

Alternatively, we could use assertThrows() instead of
@Test(expected = ...), but grpc doesn't yet require Java 8.
2022-01-18 09:27:25 -08:00
markb74 d853414ba3
Update javadoc for AndroidComponentAddress. (#8725)
Be more explicit that "packagename" is the application package name.
2022-01-18 09:22:59 -08:00
Eric Anderson 7cf048eb28
Drop Java 7 support
Oracle's Premier Support for Java 7 ended in July 2019. Per gRFC P5,
dropping support for the only release. Android is able to desugar many
Java 8 language features.
2022-01-18 07:17:51 -08:00
John Cormie 25531d6257
binder: Invoke onTransportReady() in a round-robin fashion. (#8835)
Also call onTransportReady() only if isReady() still holds by the time
we get to a given Inbound. This dramatically reduces timeouts and
improves throughput when flow control has kicked in.

This approach is still not completely fair since each ongoing call might
consume a different amount of window on its turn, but because of the way
Outbound#writeMessageData() and BlockPool already work, everyone gets to
send at least 16kb.
2022-01-14 14:06:14 -08:00
Penn (Dapeng) Zhang e279479908 android-interop-testing: update androidTest/AndroidManifest.xml for multidex 2022-01-14 13:38:29 -08:00
Penn (Dapeng) Zhang 14feae81b3 android-interop-testing: migrate AndroidJUnit4 runner 2022-01-14 13:38:29 -08:00
ZHANG Dapeng 9ee0ac208b
testing: remove opencensus dependency from grpc-testing (#8833)
`io.grpc.internal.testing.StatsTestUtils` in `grpc-testing` is only used internally by `grpc-interop-testing` and unit tests. The opencensus dependency does not need to be exposed to `grpc-interop-testing` maven artifact.
2022-01-14 10:13:43 -08:00
Eric Anderson 39cc44e38c
kokoro: Pretty test results
Previously, only Windows had the plumbing to rename test results for
the Kokoro result viewers to pretty-print.

macos.cfg was the only CI that lacked a corresponding .sh, which maked
unix.sh harder to reason about. Created macos.sh so that unix.sh is now
just a helper script and will not be called directly by Kokoro.

We now avoid "gradle clean" to avoid wiping results. Still clean compiler
since we do re-run the build multiple times with varying platforms.
Shouldn't be necessary, but "just in case" since I want this commit to
be low risk. This improves Windows to produce detailed results even
if the CI was successful.
2022-01-12 15:44:01 -08:00
ZHANG Dapeng 7a23fb27fe
rls: fix child lb leak when client channel is shutdown (#8750)
When client channel is shutting down, the RlsLoadBalancer is shutting down. However, the child loadbalancers of RlsLoadBalancer are not shut down. This is causing the issue b/209831670
2022-01-12 14:58:44 -08:00
yifeizhuang 26f0d611db
Start 1.45.0 development cycle (#8825) 2022-01-12 12:08:22 -08:00
Eric Anderson 58a7ace6ac
Bump ErrorProne to 2.10.0
Previous versions of error prone were incompatible with Java 17 javac.

In grpc-api, errorprone is now api dependency because it is on a public
API.  I was happy to see that Gradle failed the build without the dep
change, although the error message wasn't super clear as to the cause.

It seems that previously -PerrorProne=false did nothing. I'm guessing
this is due to a behavior change of Gradle at some point. Swapping to
using the project does build without errorProne, although the build
fails with Javac complaining certain classes are unavailable. It's
unclear why. It doesn't seem to be caused by the error-prone plugin.
I've left it failing as a pre-existing issue.

ClientCalls/ServerCalls had Deprecated removed from some methods because
they were only deprecated in the internal class, not the API. And with
Deprecated, InlineMeSuggester complained.

I'm finding InlineMeSuggester to be overzealous, complaining about
package-private methods. In time we may figure out how to use it better,
or we may request changes to the checker in error-prone.
2022-01-12 12:06:27 -08:00
Sergii Tkachenko 7c4fe69dfd
xds: fix a concurrency issue in CSDS ClientStatus responses (#8795)
* xds: fix a concurrency issue in CSDS ClientStatus responses

Fixes an issue with ClientXdsClient.getSubscribedResourcesMetadata()
executed out of shared synchronization context, and leading to:

- each individual config dump containing outdated data when
  an xDS resource is updated during CsdsService preparing the response
- config dumps for different services being out-of-sync with each
  other when any of the related xDS resources is updated during
  CsdsService preparing the response

The fix replaces getSubscribedResourcesMetadata(ResourceType type)
with atomic getSubscribedResourcesMetadataSnapshot() returning
a snapshot of all resources for each type as they are
at the moment of a CSDS request.
2022-01-11 17:45:24 -08:00
Eric Anderson 69671e152f
interop-testing: Fix script rewrite for Gradle 7.2
e0dca93c broke the interop-testing script for unix because Gradle
changed the scripts for
https://github.com/gradle/gradle/security/advisories/GHSA-6j2p-252f-7mw8

The solution here looks weird, but we are inserting the replacement
string into a single-quoted string, so we stop that string, start a
double-quoted string to allow the variable replacement, and then resume
the previous string.
2022-01-11 09:43:41 -08:00
sanjaypujare 56e9321d8a
android-interop-testing: add the linter error message ignore annotation in the source xml file (#8821) 2022-01-11 09:12:46 -08:00
Eric Anderson 9e8375850b
build.gradle: Add missing dependsOn for generated code sync
When messing with error prone for another commit, Gradle started
producing a clear warning the dependsOn was missing. But the warning
was not reliable. However, even when no warning was printed it is clear
the task was broken.
2022-01-10 14:59:19 -08:00