Commit Graph

4024 Commits

Author SHA1 Message Date
Chengyuan Zhang 043ab93b3b
xds: fix bug of return resolution error twice in xDS resolver (#6671) 2020-02-11 09:48:53 -08:00
Eric Anderson 255e5feb24 Sync grpc-proto to 1ff78907
This noticed that load_balancer.proto had local changes introduced
in #6549. This was not noticed by Bazel because grpclb was not using
the io_grpc_grpc_proto repository. These issues have been fixed.
2020-02-10 12:32:39 -08:00
Eric Gribkoff 84b10350eb buildscripts,interop-testing: Add xDS tests on GCP (#6662) 2020-02-09 19:43:11 -08:00
Georg Welzel 60bc74620f
census: Set SpanKind on Client/Server traces (#6680) 2020-02-07 09:48:29 -08:00
ZhenLian 557da62305
netty: Upgrade Netty (4.1.45.Final) & tcnative (2.0.28.Final) 2020-02-06 10:03:14 -08:00
ZHANG Dapeng 258fe12f02
interop-testing: fix bug of xds test missing transitive dependency
This fixes a bug introduced in #6675 .

grpc-interop-testing should have added all dependency e.g. enovy that grpc-xds needs.
2020-02-05 16:47:13 -08:00
ZHANG Dapeng cd35a8153c
interop-testing: fix bug for xds dependency not published yet
In v1.27.0 release the grpc-interop-testing artifact in maven includes grpc-xds, but grpc-xds is not yet published. It should be removed from the dependency list in maven artifact.
2020-02-04 14:53:31 -08:00
sanjaypujare 461d30adfb
netty: fix server and client handlers to check the correct alpn list (#6603) 2020-02-04 13:03:34 -08:00
Eric Anderson 5b726c07fe alts: _Actually_ use Conscrypt when available
Previously the check for Conscrypt would always fail because CONSCRYPT
was guaranteed to be null.
2020-02-03 17:38:54 -08:00
Chengyuan Zhang e0ee52cc22
grpclb: fix lint warnings (#6670) 2020-02-03 10:34:30 -08:00
Chengyuan Zhang 295b64b5ff
grpclb: expose balancer address related attributes in internal accessor (#6669) 2020-01-31 17:50:04 -08:00
Chengyuan Zhang 26bff62ff3
grpclb: internal accessor for balancer address related attribute keys (#6667)
Creates an internal accessor for attribute keys in grpclb package that is used by name resolver implementations to set balancer addresses as name resolution result attributes.
2020-01-31 15:44:30 -08:00
Chengyuan Zhang be27d0ba8b
Revert "github: add Github workflow for Gradle wrapper validation (#6634)" (#6668)
This reverts commit 6548c6ff5e.
2020-01-31 15:32:44 -08:00
Chengyuan Zhang 6548c6ff5e
github: add Github workflow for Gradle wrapper validation (#6634) 2020-01-31 15:04:20 -08:00
Chengyuan Zhang 2734e22ef5
context, stub: check result of CountDownLatch.await() in test (#6664) 2020-01-31 15:03:22 -08:00
Jihun Cho 307f7d0e09
grpclb,xds: fix lint warnings (#6666) 2020-01-31 11:32:26 -08:00
ZHANG Dapeng 66f08c71b1
core,xds: clean up xds config parsing from service config util 2020-01-31 11:31:48 -08:00
Eric Anderson 2e12b53f2f
examples: Add XDS client example 2020-01-31 11:13:44 -08:00
Chengyuan Zhang c0f37e59ab
core, grpclb: change policy selection strategy for Grpclb policy (take one: eliminate special logic for deciding grpclb policy in core) (#6637)
First take for grpclb selection stabilization: 

1. Changed DnsNameResolver to return balancer addresses as a GrpcAttributes.ATTR_LB_ADDRS attribute in ResolutionResult, instead of among the addresses.

2. AutoConfiguredLoadBalancerFactory decides LB policy solely based on parsed service config without looking at resolved addresses. Behavior changes:
  - If no LB policy is specified in service config, default to pick_first, even if there exist balancer addresses (in attributes).
  - If grpclb specified but not available and no other specified policies available, it will fail without fallback to round_robin.

3. GrpclbLoadBalancer populates balancer addresses from ResolvedAddresses's attribute (GrpclbConstants.ATTR_LB_ADDRS) instead of sieving from addresses.
2020-01-31 10:41:43 -08:00
Chengyuan Zhang 3e6a77a7ef
xds: precise logic for selecting the virtual host in RDS responses (#6661)
Implements the precise logic for choosing the virtual host in RouteConfiguration of RDS responses. Specifically, fixes logic for domain search order. Minor fix for checking match field in RouteConfiguration. See RouteConfiguration Proto section in gRPC Client xDS API Flow design doc for specification.
2020-01-30 16:24:18 -08:00
ZHANG Dapeng 06b983567e
xds: rename LookasideLb to EdsLoadBalancer and XdsLoadBalancer2 to XdsLoadBalancer
This is a follow up cleanup of #6656 .
2020-01-30 13:19:14 -08:00
Elliotte Rusty Harold b3aea8193f
all: update google auth libraries (#6652) 2020-01-30 12:59:55 -08:00
Liam Miller-Cushon 084f5bc4bd java_grpc_library: Inline find_java_toolchain and find_java_runtime_toolchain
These methods were used to migrate the Java toolchains to use toolchain
resolution. Now that the migration is complete, the toolchain providers
can be used directly.
2020-01-30 12:55:40 -08:00
Jihun Cho 2acca6ea3b
all: fix lint warnings (#6658) 2020-01-30 10:00:39 -08:00
ZHANG Dapeng f04c4921d9
xds: add EdsLoadBalancerProvider to shortcut CDS to EDS flow skipping fallback
The `EdsLoadBalancerProvider` provides `LookasideLb` (Will rename `LookasideLb` to `EdsLoadBalancer` in future, but kept the name now to show better diff) with no-op callbacks for fallback.

- `CdsLoadBalancer` will load `EdsLoadBalancerProvider/LookasideLb` directly skipping fallback.

- The EDS-only flow is unchanged, still loading `XdsLoadBalancerProvider/XdsLoadBalancer2`, keeping current fallback behavior and producing horrible error message when both the primary and fallback policy fail.
2020-01-30 09:49:22 -08:00
Chengyuan Zhang 4ad3acb8f2
xds: print xDS responses nicely with protobuf JsonFormat (#6654)
This change implements a mechanism for printing xDS responses, which contains com.google.protobuf.Any type fields in proto messages, in human-readable format.
2020-01-29 09:58:47 -08:00
ZHANG Dapeng 2eccdb8337
Revert "xds: better error handling to avoid RPC hangup"
This reverts commit a223263134.
2020-01-28 15:36:32 -08:00
ZHANG Dapeng 64e827d996
grpclb,xds: fix parseLoadBalancingConfigPolicy error message format 2020-01-28 13:08:05 -08:00
Roman Leventov 589a645a38 stub,examples: Clarify CallStreamObserver's Javadoc (#6561)
* Clarify CallStreamObserver's Javadoc
* Remove unnecessary AtomicBoolean and clarify a comment in ManualFlowControlServer
2020-01-28 11:32:09 -08:00
Chengyuan Zhang 4ad3acc1d4
xds: remove special logic for unsubscribing last CDS resource (#6647) 2020-01-28 10:37:10 -08:00
ZHANG Dapeng 1ed538f253
all: log picker when updating balancing state
Let `ManagedChannelImpl` log picker update when updating balancing state.
2020-01-28 09:55:04 -08:00
ZHANG Dapeng 14e63fea61
xds: use resolvedAddresses.getLoadBalancingPolicyConfig() to get LB config 2020-01-27 15:47:25 -08:00
Jihun Cho 85eb92418c
core: fix ATTR_LOAD_BALANCING_CONFIG has service config instead of raw lb config (#6648) 2020-01-27 14:36:53 -08:00
ZHANG Dapeng d8250e9456
interop-testing: log RPC failures for XdsTestClient 2020-01-27 13:58:48 -08:00
Chengyuan Zhang e9882ec78d
xds: fix bug of xDS resolver parsing service config incorrectly (#6640)
The ResolutionResult returned by resolver should have its ServiceConfig set with a parsed service config, ins
2020-01-24 16:12:24 -08:00
Eric Gribkoff e5745a514f
interop-testing: use server hostname instead of id for xds test (#6639) 2020-01-24 14:03:49 -08:00
Jihun Cho 751faa6faa
core: promote ServiceConfigErrorHandling (#6633) 2020-01-23 17:59:27 -08:00
Eric Anderson 5b7f5b8c3b examples: Allow passing target and simplify lifecycle
The target can be provided on the command line to avoid needing to recompile
the example just to change where the server is located. We use a target instead
of addresses as that is the approach we have wanted to encourage for a while
since it allows choosing alternative name resolvers.

We typically encourage injecting Channels, not ManagedChannels, which has the
added benefit of simplifying the example. Less indirection makes for a better
example.

Swapping to target string could be done to examples-tls and examples-gauth as
well, but it would be much more invasive to the tls example and the gauth
example would need proper testing after the change.
2020-01-23 13:13:54 -08:00
ZHANG Dapeng 27f12a90f3
xds: refactor LookasideLb for EDS-only usecase and integrate load stats report
- Replace XdsComms2 with XdsClientImpl
- Enable/Disable load report stats with `XdsClient` APIs.

Testing strategy:
- Use real XdsClientImp for EDS-only because the balancer creates the XdsClient by itself. The state of the XdsClientImpl will be the actual state as real.
- Use mock XdsClient for non EDS-only case because the XdsClient in resolved addresses attributes is supposed to be a stateful XdsClient with some pre-existing CDS state, so creating a brand new real XdsClientImp in test can not simulate the same state. In this case only verify interaction with XdsClient APIs.
- Use a `LocalityStoreFactory` to verify interaction with `LocalityStore` APIs. However, this can not cover any interaction with the `Helper` and `LoadStatsStore` inputs of `LocalityStoreFactory.newLocalityStore(Helper, LoadBalancerRegistry, LoadStatsStore)`, so some basic non-exhaustive tests are added to cover the gap.

The testing strategy is imperfect but is a trade-off considering load stats report is very hard to test here and LocalityStore/real balancing behavior is too much to be exhaustively tested in `LookasideLb`.
2020-01-22 17:05:18 -08:00
Jihun Cho b1209b7ed5
core: fix AutoConfiguredLBFactory populates wrong type (#6600) 2020-01-22 16:52:43 -08:00
ZHANG Dapeng a223263134
xds: better error handling to avoid RPC hangup
This change will fail application RPC immediately if XdsClient encounters any error instead of retrying or getting to fallback silently.

There could be optimization if the channel is currently READY while XdsClient stream just closed due to connection error, in which case we could still be using the current available subchannels while retrying, but this requires the LB knows the semantics of error status from the XdsClient. This optimization is not worth the effort for now.
2020-01-22 15:38:38 -08:00
Chengyuan Zhang ee661d45eb
xds: notify all watchers when RPC stream is closed by server (#6629) 2020-01-22 14:27:06 -08:00
Eric Anderson 9cf45e960b xds: Truth Lint fixes 2020-01-22 11:26:00 -08:00
ZHANG Dapeng 1f1ccf2641
xds: enable keepalive for XDS channel
XDS clients will use a keepalive time of 5 minutes.
2020-01-21 17:03:37 -08:00
Eric Gribkoff 42f2e7f733
core: change catch blocks for Android <19 (#6611)
Caught by lint warning during import
2020-01-21 14:38:10 -08:00
Kun Zhang 74cde7e8b4
netty: add an internal option to disable native buffer (#6619)
This is needed for internal rollout where the native memory usage from netty makes task more prone to exceeding memory limits.
2020-01-21 11:37:02 -08:00
Eric Anderson e7d7c5bde4
examples: Add hostname example
This is placed in its own directory since it depends on grpc-services.
2020-01-17 15:50:49 -08:00
Chengyuan Zhang 57bc1910e4
xds: incorporate initial fetch timeout as resource not found for xDS protocol (#6570)
Use timeout to conclude resource not exist in xDS protocol.

RDS and EDS protocols are quasi-incremental, each response may not include all the requested resources that present on server side. The way to conclude a requested resource not exist is to use a timeout. In Envoy, this timeout is defined as initial fetch timeout, which is set up at the time client starts subscribing to some resource and disarmed at the time client receives update for that resource. In gRPC's implementation, we set this timeout to be constant 15 seconds, instead of getting its value from ConfigSource proto message.

Initial fetch timeout was initially considered to be not required for LDS and CDS. But gRPC is trying to avoid the temporary inconsistency in the case of racing request/response.

After resource fetch timers are fired, some resources are known to be absent for sure. XdsClient manages its knowledge for resources that are known to be present or absent with caches.
2020-01-17 14:50:20 -08:00
ZHANG Dapeng b8474d61c9
netty: fix a race for channelz at server transport creation
A race condition was reported by user in #6601:

`ServerImpl.start()` calls `NettyServer.start()` while holding `ServerImpl.lock`. `NettyServer.start()` awaits a submitted runnable in eventloop. However, this pending runnable may never be executed because the eventloop might be executing some other task, like `ServerListenerImpl.transportCreated()`, that is trying to acquire `ServerImpl.lock` causing a deadlock.

This PR resolves the particular issue reported in #6601 for server with a single port, but `NettyServer` (https://github.com/grpc/grpc-java/blob/v1.26.0/netty/src/main/java/io/grpc/netty/NettyServer.java#L251) and `ServerImpl` (https://github.com/grpc/grpc-java/blob/v1.26.0/core/src/main/java/io/grpc/internal/ServerImpl.java#L184) in general still have the same potential risk of deadlock, which need further fix.
2020-01-16 11:53:37 -08:00
Chengyuan Zhang cb4a7fb2de
alts: do not exclude opencensus-api from google-oauth2's transitive dependencies (#6607) 2020-01-15 12:33:57 -08:00