Commit Graph

4140 Commits

Author SHA1 Message Date
Chengyuan Zhang e9882ec78d
xds: fix bug of xDS resolver parsing service config incorrectly (#6640)
The ResolutionResult returned by resolver should have its ServiceConfig set with a parsed service config, ins
2020-01-24 16:12:24 -08:00
Eric Gribkoff e5745a514f
interop-testing: use server hostname instead of id for xds test (#6639) 2020-01-24 14:03:49 -08:00
Jihun Cho 751faa6faa
core: promote ServiceConfigErrorHandling (#6633) 2020-01-23 17:59:27 -08:00
Eric Anderson 5b7f5b8c3b examples: Allow passing target and simplify lifecycle
The target can be provided on the command line to avoid needing to recompile
the example just to change where the server is located. We use a target instead
of addresses as that is the approach we have wanted to encourage for a while
since it allows choosing alternative name resolvers.

We typically encourage injecting Channels, not ManagedChannels, which has the
added benefit of simplifying the example. Less indirection makes for a better
example.

Swapping to target string could be done to examples-tls and examples-gauth as
well, but it would be much more invasive to the tls example and the gauth
example would need proper testing after the change.
2020-01-23 13:13:54 -08:00
ZHANG Dapeng 27f12a90f3
xds: refactor LookasideLb for EDS-only usecase and integrate load stats report
- Replace XdsComms2 with XdsClientImpl
- Enable/Disable load report stats with `XdsClient` APIs.

Testing strategy:
- Use real XdsClientImp for EDS-only because the balancer creates the XdsClient by itself. The state of the XdsClientImpl will be the actual state as real.
- Use mock XdsClient for non EDS-only case because the XdsClient in resolved addresses attributes is supposed to be a stateful XdsClient with some pre-existing CDS state, so creating a brand new real XdsClientImp in test can not simulate the same state. In this case only verify interaction with XdsClient APIs.
- Use a `LocalityStoreFactory` to verify interaction with `LocalityStore` APIs. However, this can not cover any interaction with the `Helper` and `LoadStatsStore` inputs of `LocalityStoreFactory.newLocalityStore(Helper, LoadBalancerRegistry, LoadStatsStore)`, so some basic non-exhaustive tests are added to cover the gap.

The testing strategy is imperfect but is a trade-off considering load stats report is very hard to test here and LocalityStore/real balancing behavior is too much to be exhaustively tested in `LookasideLb`.
2020-01-22 17:05:18 -08:00
Jihun Cho b1209b7ed5
core: fix AutoConfiguredLBFactory populates wrong type (#6600) 2020-01-22 16:52:43 -08:00
ZHANG Dapeng a223263134
xds: better error handling to avoid RPC hangup
This change will fail application RPC immediately if XdsClient encounters any error instead of retrying or getting to fallback silently.

There could be optimization if the channel is currently READY while XdsClient stream just closed due to connection error, in which case we could still be using the current available subchannels while retrying, but this requires the LB knows the semantics of error status from the XdsClient. This optimization is not worth the effort for now.
2020-01-22 15:38:38 -08:00
Chengyuan Zhang ee661d45eb
xds: notify all watchers when RPC stream is closed by server (#6629) 2020-01-22 14:27:06 -08:00
Eric Anderson 9cf45e960b xds: Truth Lint fixes 2020-01-22 11:26:00 -08:00
ZHANG Dapeng 1f1ccf2641
xds: enable keepalive for XDS channel
XDS clients will use a keepalive time of 5 minutes.
2020-01-21 17:03:37 -08:00
Eric Gribkoff 42f2e7f733
core: change catch blocks for Android <19 (#6611)
Caught by lint warning during import
2020-01-21 14:38:10 -08:00
Kun Zhang 74cde7e8b4
netty: add an internal option to disable native buffer (#6619)
This is needed for internal rollout where the native memory usage from netty makes task more prone to exceeding memory limits.
2020-01-21 11:37:02 -08:00
Eric Anderson e7d7c5bde4
examples: Add hostname example
This is placed in its own directory since it depends on grpc-services.
2020-01-17 15:50:49 -08:00
Chengyuan Zhang 57bc1910e4
xds: incorporate initial fetch timeout as resource not found for xDS protocol (#6570)
Use timeout to conclude resource not exist in xDS protocol.

RDS and EDS protocols are quasi-incremental, each response may not include all the requested resources that present on server side. The way to conclude a requested resource not exist is to use a timeout. In Envoy, this timeout is defined as initial fetch timeout, which is set up at the time client starts subscribing to some resource and disarmed at the time client receives update for that resource. In gRPC's implementation, we set this timeout to be constant 15 seconds, instead of getting its value from ConfigSource proto message.

Initial fetch timeout was initially considered to be not required for LDS and CDS. But gRPC is trying to avoid the temporary inconsistency in the case of racing request/response.

After resource fetch timers are fired, some resources are known to be absent for sure. XdsClient manages its knowledge for resources that are known to be present or absent with caches.
2020-01-17 14:50:20 -08:00
ZHANG Dapeng b8474d61c9
netty: fix a race for channelz at server transport creation
A race condition was reported by user in #6601:

`ServerImpl.start()` calls `NettyServer.start()` while holding `ServerImpl.lock`. `NettyServer.start()` awaits a submitted runnable in eventloop. However, this pending runnable may never be executed because the eventloop might be executing some other task, like `ServerListenerImpl.transportCreated()`, that is trying to acquire `ServerImpl.lock` causing a deadlock.

This PR resolves the particular issue reported in #6601 for server with a single port, but `NettyServer` (https://github.com/grpc/grpc-java/blob/v1.26.0/netty/src/main/java/io/grpc/netty/NettyServer.java#L251) and `ServerImpl` (https://github.com/grpc/grpc-java/blob/v1.26.0/core/src/main/java/io/grpc/internal/ServerImpl.java#L184) in general still have the same potential risk of deadlock, which need further fix.
2020-01-16 11:53:37 -08:00
Chengyuan Zhang cb4a7fb2de
alts: do not exclude opencensus-api from google-oauth2's transitive dependencies (#6607) 2020-01-15 12:33:57 -08:00
sanjaypujare 1b5d61f18f
xds: mark Sds-Ssl threadfactory daemon to allow apps to exit (#6608) 2020-01-15 11:10:19 -08:00
Eric Anderson cf89b4b53d examples: Bump Gradle to 5.6.2
The rest of the build went to Gradle 5.6.2 in 3c3a823a, but that commit
failed to update the examples. This brings the two back in sync.
2020-01-15 09:30:03 -08:00
Eric Anderson ca56aa30d4 Start 1.28.0 development cycle 2020-01-14 15:10:05 -08:00
sanjaypujare 04cf90a9ab
xds: support for updating upstreamTlsContext and SslContextProvider, also release object in SdsProtocolNegotiators (#6599) 2020-01-14 13:59:28 -08:00
ZHANG Dapeng 066e72dc5d
core/util: Go to TRANSIENT_FAILURE on initial resolution error for GracefulSwitchLoadBalancer (#6598)
Go to TRANSIENT_FAILURE immediately instead of NOOP if GracefulSwitchLoadBalancer receives resolution error before switching to any delegate.

In most natural usecase, the `gracefulSwitchLb` is a child balancer of some `parentLb`, and the `gracefulSwitchLb` switches to a new `delegateLb` when `parentLb.handleResolvedAddressGroup()`. If the  `parentLb` receives a resolution error before receiving any resolved addresses, it should go to  TRANSIENT_FAILURE. In this case, it will be more convenient if the initial  `gracefulSwitchLb` can go to TRANSIENT_FAILURE directly.
2020-01-14 09:54:48 -08:00
Eric Gribkoff d914e011b0
interop-testing: Add XdsTestClient and XdsTestServer (#6585) 2020-01-13 23:57:09 -08:00
Aleksei Kandybaev 18e099d9d3 netty: fix ENHANCE_YOUR_CALM debug data logging (#6596) 2020-01-13 14:59:23 -08:00
Chengyuan Zhang b7ccc0d142
core, census, testing, interop-testing, android-interop-testing: move census dependency out of grpc-core (#6577)
Decouples grpc-core with census, while still preserve the default integration of census in grpc-core. Users wishing to enable census needs to add grpc-census to their runtime classpath.

- Created a grpc-census module:
    - Moved CensusStatsModule.java and CensusTracingModule.java into grpc-census from grpc-core. CensusModuleTests.java is also moved. They now belong to io.grpc.census package.
Moved DeprecatedCensusConstants.java into io.grpc.census.internal (is this necessary?) in grpc-census.
    - Created CensusStatsAccessor.java and CensusTracingAccessor.java, which are used to create census ClientInterceptor and ServerStreamTracer.Factory.
    - Everything in grpc-census are package private, except the accessor classes. They only publicly expose ClientInterceptor and ServerStreamTracer.Factory, no Census specific types are exposed.

- Use runtime reflection to load and apply census stats/tracing to channel/server builders, if grpc-census is found in runtime classpath.

- Removed special APIs on AbstractManagedChannelImplBuilder and AbstractServerImplBuilder for overriding census module. They are only used for testing. Now we changed tests to apply Census ClientInterceptor and ServerStreamTracer.Factory just as normal interceptor/stream tracer factory. Test writer is responsible for taking care of the ordering concerns of interceptors and stream tracer factories.
2020-01-13 14:35:29 -08:00
Chengyuan Zhang 1cefe851e1
Upgrade robolectric version to 4.3.1 (#6595) 2020-01-13 11:39:05 -08:00
sanjaypujare 04924d07b3
xds: implement alpnProtocols based on list from xDS (#6594) 2020-01-10 17:02:06 -08:00
Jihun Cho 641d74f34f
grpclb: support explicit fallback from LB (#6549) 2020-01-10 13:54:43 -08:00
Chengyuan Zhang e6d15a6d04
doc: add instruction for installing Android SDK in COMPILING.md (#6593) 2020-01-10 09:58:28 -08:00
sanjaypujare dba09163de
netty: remove 'grpc-exp' from the list of next-protocol-versions in ALPN (#6592) 2020-01-09 16:41:20 -08:00
sanjaypujare bee375f205
xds: suppress hostname check but keep trust check in the delegated X509TrustManagerImpl (#6589) 2020-01-09 16:40:43 -08:00
Eric Anderson 8cf0196548
Add Cronet to main build
Building now requires an Android SDK unless you specify
-PskipAndroid=true
2020-01-09 15:20:41 -08:00
sanjaypujare 6517ac8a61
xds: add support for static and combined validation context and enhanced loggging (#6586) 2020-01-08 16:41:16 -08:00
ZHANG Dapeng d03a746d02
xds: rename experimental_cds policy name to cds_experimental
Incorporating comment https://github.com/grpc/grpc-java/pull/6504#discussion_r363463413. Currently there is no cross-language resolver sending CDS config so there is no impact of the change.
2020-01-08 15:38:53 -08:00
Tomo Suzuki 75f6fd8f10 Upgrade error_prone_annotations to 2.3.4 2020-01-03 14:50:34 -08:00
Eric Anderson bcda439317 core: Simplify ServiceConfigUtil via utility methods
This should be functionally equivalent. More can be done, but wanted all the
changes to be "obvious" when reading them.
2020-01-02 11:17:21 -08:00
Eric Anderson c606519a5d bazel: Support maven_install
maven_install is strongly superior to previous forms of grabbing dependencies
from Maven as it computes the appropriate versions to use from the full
transitive dependencies of all libraries used by an application. It also has
much less boilerplate and includes dependencies with generated targets.

In the future we will drop the jvm_maven_import_external usages and require
maven_install, at which point we can swap to using the `@maven' repository and
no longer depend on compat_repositories.

Fixes #5359
2019-12-30 12:08:42 -08:00
Eric Anderson 6a5ee19695 examples: Suggest Bazel's http_repository instead of git_repository
http_repository is preferred by Bazel over git_repository.
2019-12-30 12:08:42 -08:00
Eric Anderson 69f9029c9a repositories.bzl: Remove omit* args in favor of existing_rule() check
Using existing_rule() is now the preferred way of avoiding re-defining
repositories. The function names were changed to match the name of the
repository they add. Normally we would inline all the functions, but that's
unnecessary churn since the repositories will mostly be replaced with
maven_install() in the future.
2019-12-30 12:08:42 -08:00
Graeme Morgan d3c77f2d87 all: Add suppressions for GuardedBy violations
This supports releasing a new version of GuardedBy which finds more mistakes than it used to.

Filed #6578 to try to clean up the suppressions.
2019-12-30 10:17:05 -08:00
ZhenLian 19805233ef netty: Add Executor When Creating SslContext
This PR is to add one more Executor parameter when creating the SslContext.
In Netty, we already have this implementation for passing Executor when creating SslContext: netty/netty#8847. This extra Executor is used to take some time-consuming tasks when doing SSL handshake. However, in current gRPC implementation, we are not using this API.

In this PR, the relevant changes are:

1. get the executorPool from ChannelBuilder or ServerBuilder

2. pass the executorPool all the way down to ClientTlsHandler

3. fill executorPool in when creating SslHandler
2019-12-30 08:33:16 -08:00
Eric Anderson dd8165b2a7 api: Add ExperimentalApi to Metadata.BinaryStreamMarshaller
This was missed in d1078591 and the API has not yet been released.
2019-12-27 13:33:46 -08:00
Jiangtao Li d82140cdbc
alts: change errr status to permission deny if alts context not found (#6562) 2019-12-27 12:41:51 -08:00
Tomo Suzuki f07b79b11d proto-google-common-protos:1.17.0 (#6572) 2019-12-27 11:55:05 -08:00
Kun Zhang d227520388 xds: fix lint warnings in test code (#6559) 2019-12-27 11:14:51 -08:00
Jihun Cho 90969b3471
core: make service config error handling optional implementation (#6564) 2019-12-26 15:07:20 -08:00
ZHANG Dapeng 6b04fc18f3
xds: rename XdsClientRef to XdsClientPool 2019-12-18 13:44:35 -08:00
Eric Gribkoff 91853036e6
Update README etc to reference 1.26.0 (#6548) 2019-12-18 13:40:28 -08:00
Nathan H. Leung 9c9163d453 context: update javadoc for withValue(s) to indicate withValue can be chained (#6545) 2019-12-18 13:33:31 -08:00
Jiangtao Li 04e1c9d44a
alts: add client authorization util library (#6529)
* alts: add client authorization util library
2019-12-18 10:54:59 -08:00
Eric Anderson 274bf62e04 bazel: Fix renamed javalite target
This was missed from 2d592642a, because the select hid the failure.
2019-12-18 10:24:14 -08:00