Commit Graph

5393 Commits

Author SHA1 Message Date
yifeizhuang c48b01a46c
interop-test:sync grpc-proto and fix typo (#9233) 2022-06-03 09:31:32 -07:00
yifeizhuang 3624d59b18
interop-test: orca test change gradle dependency to xds non-shaded to allow bazel run (#9230)
Co-authored-by: Eric Anderson <ejona@google.com>
2022-06-03 09:31:00 -07:00
Eric Anderson a206cda1a8 Change Attributes.Key debug strings to reference the API of the key
Users appear to be doing `attributes.toString()` to find keys they are
interested in and then unable to find the name of the Key in our API.
They workaround the problem by scanning through `attributes.keys()`
looking for the key of interest. This is an abuse of the keys() API and
unnecessary user friction. They'd happily use the API if they just knew
where to find it.

I added internal to some strings to make it clear that you shouldn't go
looking to use it. There were many strings I didn't change. I focused on
keys most likely to be seen by users, which meant keys in grpc-api and
keys that are available via transport attributes.

See https://github.com/grpc/grpc-java/issues/1764#issuecomment-1139250061
2022-06-02 16:11:02 -07:00
Sergii Tkachenko 1b620847ad Update README etc to reference 1.47.0, protobuf plugin 0.8.18 2022-06-02 12:02:16 -07:00
Jeremy Volkman 84edc33239
Update to handle rename of java_names.h to names.h in protobuf upstream (#9218) 2022-05-31 15:08:10 -07:00
Vladimir Chizhov 0490ced91c
Bump GSON to 2.9.0 (#9215) 2022-05-31 15:07:22 -07:00
Terry Wilson f8da92e176
xds: Update xDS protos (#9223) 2022-05-27 15:56:31 -07:00
apolcyn 6dbd1d8f58
interop-testing: bump client op timeout from 5 to 15 seconds (#9211)
JVM startup costs that happen when dependencies are loaded for the
first time can consume a lot of time (we've occasionally observed
around ~5 seconds of CPU time); this causes frequent test flakes
with xds (google-c2p) when using the current 5 second deadline.
Increasing to 15 seconds should give enough time.
2022-05-27 14:36:54 -07:00
Terry Wilson 69d4e71309
Enable xDS custom LB config by default. (#9214) 2022-05-26 17:03:14 -07:00
Eric Anderson 0ebf2f76c2 examples/xds: Remove unnecessary includeBuild ..
Previously examples-xds depended on the normal hello-world, as it used
the same classes. But since b6601ba273 it has had its own classes and
not had a dependency on `:examples.
2022-05-26 11:11:56 -07:00
yifeizhuang 45dd17c799
examples: add examples-orca (#9204) 2022-05-26 10:23:43 -07:00
Terry Wilson e2f7e676cf
xds: Make a copy of Metadata for thread safety. (#9212) 2022-05-25 15:57:24 -07:00
Lidi Zheng d6a23e4116 interop-test: Remove duplicated GCE tests 2022-05-25 15:56:29 -07:00
Terry Wilson fc406e8612
xds: Properly assign picker. (#9201)
* xds: Properly assign picker.

Fixes a bug where the picker was not getting used in RpcBehaviorLoadBalancer.
2022-05-25 12:40:12 -07:00
Terry Wilson de3388bd21
xds: Custom LB configs to support UDPA TypeStruct (#9198)
* xds: Custom LB configs to support UDPA TypeStruct

The legacy com.github.udpa.udpa.type.v1.TypedStruct proto should be supported in addition to the current com.github.udpa.udpa.type.v1.TypedStruct one.

Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2022-05-24 13:55:02 -07:00
Terry Wilson 48ea07efce
xds: Use weighted_target LB provider in wrr_locality (#9195)
Fixes a bug where WrrLocalityLoadBalancer would use the endpoint picking policy provider instead of WeightedTargetLoadBalancerProvider.

Also adds a test to fake control plane integration test that caught this bug. The test scaffolding is also updated to have the test server echo all client headers back in the response.

The test load balancer in the test is an almost straight copy of: https://github.com/grpc/grpc-java/blob/master/interop-testing/src/test/java/io/grpc/testing/integration/RpcBehaviorLoadBalancerProviderTest.java
2022-05-24 13:51:12 -07:00
yifeizhuang 01ab821a06
interop-test: add orca test case (#9079)
add interop testing `orca_per_rpc` and `orca_oob`
2022-05-24 13:34:01 -07:00
Terry Wilson 62119b2a2e
interop-testing: Update comment to round_robin. (#9196) 2022-05-24 10:45:07 -07:00
Eric Anderson f9e1c0f37f
xds: Protect xdstp processing with federation env var
There are still some cases for xdstp processing, but they are to percent
encoding replacement strings. Those seem better to leave running since
it looks like it they could be triggered even with federation disabled
in the bootstrap processing.
2022-05-23 08:16:11 -07:00
Terry Wilson 4a5f6adf73
interop-testing: LB that adds the rpc-behavior to calls (#9186) 2022-05-20 12:17:58 -07:00
Eric Anderson 5bb721e217
interop-testing: Support syncing protos from grpc-proto
Two main incompatibilities existed in the copy of protos in grpc-proto:
no SimpleContext and an Empty method argument was replaced with a
message. "Context" is a very old word for "Metadata" back from the days
before the current gRPC protocol. We don't need that message in
particular, and well-known protos actually works in Protobuf Lite these
days, so we can swap to wrappers.proto's StringValue and don't need to
upstream a change to grpc-proto. The argument problem is fixed just by
changing the type in the Java code.

With the incompatibilities fixed, do a sync from grpc-proto and include
interop-testing.
2022-05-18 14:28:20 -07:00
Sergii Tkachenko 9cb3c6e688 Start 1.48.0 development cycle 2022-05-18 13:57:28 -07:00
yifeizhuang c39b3583a1
orca: change metric recorder api to: clear/set/put/remove/setAll (#9180) 2022-05-18 10:54:19 -07:00
Grant Oakley 0c5863f867
binder: Add SecurityPolicies#anyOf(), similar to allOf() (#9147) 2022-05-17 10:40:17 -07:00
John Cormie 3f19dfaf4a
binder: Avoid an ISE from asAndroidAppUri() (#9169) 2022-05-16 17:33:16 -07:00
Ovidiu Tirla a88f19fb4b
Expose SecurityLevel on server-side (#8943) 2022-05-16 17:26:10 -07:00
Eric Anderson c61b4af33d benchmarks: Raise timeout for aarch64
runUnaryBlockingClosedLoop is failing after 10.3s, which means 5.3s was
probably spent loading the LoadWorker. That means things are likely
indeed slow enough that 5s isn't enough time to wait for the server to
start. A successful execution of runUnaryBlockingClosedLoop takes
12.1 seconds, which again points to general slow execution.
2022-05-16 13:20:04 -07:00
Eric Anderson b5e78d548d api: Ignore ClassCastExceptions for hard-coded providers
This was observed in the Bazel/Blaze build where io.grpc.util is a
separate target from the rest of core. During the build of a library
SecretRoundRobinLoadBalancerProvider was not on the classpath, and the
library was later included into a binary using grpc-core from Maven
Central which includes SecretRoundRobinLoadBalancerProvider.

```
java.util.ServiceConfigurationError: Provider io.grpc.util.SecretRoundRobinLoadBalancerProvider$Provider could not be instantiated java.lang.ClassCastException: class io.grpc.util.SecretRoundRobinLoadBalancerProvider$Provider cannot be cast to some.app.aaa.aab
```
2022-05-16 13:19:46 -07:00
Eric Anderson ba57a1d893 api: Add connection management APIs to ServerBuilder
These APIs were added to NettyServerBuilder for gRFC A8 and A9. They are
important enough that they shouldn't require using the perma-unstable
transport API to access. This change also allows using these methods
with grpc-netty-shaded.

Fixes #8991
2022-05-16 13:02:09 -07:00
Michael Lumish 78067cbace
xds: Update rpc-behavior handling in interop server 2022-05-16 10:07:22 -07:00
Eric Anderson ea09d3eebd Add Bazel build support for xds, googleapis to flesh out maven_install
Not updating the example WORKSPACE because it doesn't have any
Bazel-enabled build that depends on xds and so doesn't need the
additional repository dependencies.

Fixes #9162
2022-05-16 10:05:35 -07:00
cfredri4 1d4e82e9ae
xds: Allow unspecified listener traffic direction to work with Istio (#9173)
* Allow unspecified listener traffic direction to work with Istio
2022-05-16 09:35:55 -07:00
Andrei Kandratovich 2c33e39f5d
binder: Respect requested message limits within a single MessageProducer 2022-05-16 09:23:16 -07:00
Terry Wilson 8a84611d9d
xds: Update the env variable to enable custom LB config (#9170) 2022-05-13 15:18:02 -07:00
Sergii Tkachenko 23ab7d0309 xds: ignore routes with unsupported optional cluster specifiers
Support for the is_optional logic in Cluster Specifier Plugins:
if unsupported Cluster Specifier Plugin is optional,
don't NACK, and skip any routes that point to it.
2022-05-13 13:03:04 -07:00
Terry Wilson c6bfce034f
xds: Check for env variable before doing custom LB config (#9165) 2022-05-12 12:57:11 -07:00
Terry Wilson 36d1d5fe45
xsd: wrr_locality to not propagate locality weight attribute (#9158)
WrrLocalityLoadBalancer should remove the locality weights attribute from Resolved addresses after using the information. Not propagating this attribute will make it impossible for another child wrr_locality from working. This is not a supported situation and this change make the failure happen earlier and to be more obvious as to the cause.
2022-05-12 12:51:21 -07:00
yifeizhuang 0fd8a6fcea
xds: add OrcaOobHelperWrapper.setListener(), allow receiving reports per subchannel (#9143)
remove OrcaOobHelperWrapper layer. Use OrcaOobUtil.updateListener() to set OrcaOobReportListener per each subchannel, not per helper. OrcaOobReportListener is per helper+subchannel unique.

Orca stats are created when creating helper.createSubchannels(), overriding subchannel attributes to store orcaState in the orcaHelper created subchannels
2022-05-12 10:50:58 -07:00
Terry Wilson 1ee93758cd
xds: More nuanced LoadBalancerConfigFactory tests (#9161)
Also some refactoring.
2022-05-11 23:01:04 -07:00
yifeizhuang b771403273
xds: separate orca APIs and implementations (#9144)
moved orcaMetrics to service project and renamed to MetricRecorder, added internal accessor. Maybe we can combine metricsRecorder and callMetricRecorder, they looks almost the same things.
OrcaServiceImpl depends on metricRecorder, not visa versa.
2022-05-11 15:54:05 -07:00
Terry Wilson f68be9f87c
xsd: Fix some lint errors. (#9159) 2022-05-11 09:37:24 -07:00
sanjaypujare 8f1dab0470
gpc-observability: add configuration for metrics and tracing, sampler and sampler_rate and reading from file (#9151) 2022-05-11 08:20:12 -07:00
Sergii Tkachenko 743b1ede11
grpclb: Include META-INF/services to //grpclb:grpclb (#9156) 2022-05-10 09:02:33 -07:00
Eric Anderson af56f1fc7d Revert "bom: Remove protoc-gen-grpc-java"
This reverts commit 4a84c6fa96. The BOM
was usable for protoc-gen-grpc-java using dependencyManagement for the
buildscript. See conversation on #9020.
2022-05-09 10:28:32 -07:00
Eric Anderson a8bd0b8c92 netty: Simplify and increase compat of UdsNettyChannelProvider
The transport should be usable with non-`unix:` name resolvers. As long
as the name resolver returns the correct socket address type, things
should work fine.
2022-05-09 08:37:31 -07:00
Terry Wilson c20904d681
xds: Use wrr_locality LB and support load_balancing_policy in Cluster (#9141)
Instead of providing round robin or least request configurations directly, ClientXdsClient now wraps them in a WRR locality config.

ClusterResolverLoadBalancer passes this configuration directly to PriorityLoadBalancer to use as the endpoint LB policy it provides to ClusterImplLoadBalancer. A new ResolvedAddresses attribute is also set that has all the locality weights. This is needed by WrrLocalityLoadBalancer when it configures WeightedTargetLoadBalancer.

Renames the LegacyLoadBalancerConfigFactory to just LoadBalancerConfigFactory and gives it responsibility for both the legacy and the new LB config mechanism.

The new configuration mechanism is explained in gRFC A52: https://github.com/grpc/proposal/pull/298
2022-05-06 14:54:10 -07:00
DNVindhya 15ecc0714e
gcp-observability: add grpc-census as a dependency and update opencensus version (#9140) 2022-05-03 20:52:34 -07:00
Eric Anderson de7db565a3
benchmarks: Propagate errors in LoadWorkerTest startup
Also clean up resources at the end of test.
2022-05-03 16:55:08 -07:00
sanjaypujare 41c027c11b
netty: implement UdsNameResolver and UdsNettyChannelProvider (#9113)
* netty: implement UdsNameResolver and UdsNettyChannelProvider
When the scheme is "unix:" we get the UdsNettyChannelProvider to
create a NettyChannelBuilder with DomainSocketAddress type and
other related params needed for UDS sockets
2022-05-02 16:41:50 -07:00
Eric Anderson cb61a5e284 benchmarks: Increase timeout of LoadWorkerTest
This should fix test failures on aarch64.
```
expected to be less than: 0.0
but was                 : 0.0
	at app//io.grpc.benchmarks.driver.LoadWorkerTest.assertWorkOccurred(LoadWorkerTest.java:198)
	at app//io.grpc.benchmarks.driver.LoadWorkerTest.runUnaryBlockingClosedLoop(LoadWorkerTest.java:90)
```

runUnaryBlockingClosedLoop() has been failing but the other tests
suceeding. The failure is complaining that getCount() == 0, which means
no RPCs completed. The slowest successful test has a mean RPC time of
226 ms (the unit was logged incorrectly) and comparing to x86 tests
runUnaryBlockingClosedLoop() is ~2x as slow because it executes first.
So this is probably _barely_ failing and 4 attempts instead of 3 would
be sufficient. While the test tries to wait for 10 RPCs to complete, it
seems likely it is stopping early even for the successful runs on
aarch64. There are 4 concurrent RPCs, so to get 10 RPCs we need to wait
for 3 batches of RPCs to complete which would be 1346 ms (5 loops)
assuming a 452 ms mean latency. Bumping timeout by 10x to give lots of
headroom.
2022-05-02 13:09:34 -07:00