Commit Graph

4658 Commits

Author SHA1 Message Date
Eric Anderson 03d04f1944 api: Reword retry javadoc to make clear service config is source of configuration
Multiple users have tried things like
`mcb.enableRetry().maxRetryAttempts(3)` and been confused when no
retries were performed. Providing a reference to the gRFC and
`defaultServiceConfig()` should greatly increase the clarity of how to
use the method.
2021-01-13 11:36:08 -08:00
Yifei Zhuang dac8347063
interop-testing: allow multiple port in test server (#7799) 2021-01-13 09:47:26 -08:00
Chengyuan Zhang b66d182bb9
api: delete LoadBalancer.Helper APIs that had been deprecated for a long time (#7793) 2021-01-11 15:25:35 -08:00
Chengyuan Zhang ca7f1e4cee
xds: slightly refactor and improve tests for cluster_manager LB policy (#7773) 2021-01-11 10:49:39 -08:00
Chengyuan Zhang 32173975e0
Revert "netty: add exporting SSL/TLS master key log feature (#7724)" (#7792)
This reverts commit 9bc05fba67.
2021-01-08 15:45:58 -08:00
huangqiangxiong 9bc05fba67
netty: add exporting SSL/TLS master key log feature (#7724)
Enable this feature by setting the system property
   -Dio.netty.ssl.masterKeyHandler=true
or
   System.setProperty(SslMasterKeyHandler.SYSTEM_PROP_KEY, "true");
The keys will be written to the log named "io.netty.wireshark" in
the warnning level. To export the keys to a file, you can configure
log factory like: (with log4j.xml for example)
<appender name="key-file" class="org.apache.log4j.RollingFileAppender">
	<param name="file" value="d:/keyfile.txt"/>
	<layout class="org.apache.log4j.PatternLayout">
		<param name="ConversionPattern" value="%m%n"/>
	</layout>
</appender>
<category name="io.netty.wireshark">
	<priority value="DEBUG" />
	<appender-ref ref="key-file" />
</category>

Wireshark can analyze the messages gRPC over TLS with this
key log file.

close #7199
2021-01-08 13:11:24 -08:00
sanjaypujare 2dba43c727
xds: reorder the shutdown of delegate and xds-client for better semantics of ServerWrapperForXds (#7791) 2021-01-08 12:02:09 -08:00
Chengyuan Zhang 18772e2470
xds: wire up re-resolution requests from LB policies for DNS clusters and bypass requests from LB policies for EDS clusters (#7769)
This change wires up the refreshNameResolution() API for triggering DNS re-resolution in ClusterResolverLoadBalancer, which provides a way for downstream LB policies to request a resolution refresh and only re-resolution requests from LB policies for the DNS cluster should trigger the DNS refresh.
2021-01-08 01:08:13 -08:00
Chengyuan Zhang 2755afeaa5
xds: clean up verbose getters and builders on XdsClient interface (#7784)
Eliminate getters and builders for xDS resource data. They are (should be effectively) immutable and mostly only used for internal implementations. Cleaning up getters and builders significantly reduces the verbosity.
2021-01-07 14:55:05 -08:00
Chengyuan Zhang ff52893c79
android-interop-testing: fix wrong instrumented test runner (#7789) 2021-01-07 11:20:24 -08:00
Chengyuan Zhang 70dd45ae70
xds: fix LB policy address and balancing state update propagations (#7772)
Delaying handleResolvedAddresses() for propagating configs to the child LB policy can be problematic. For example, if channel shutdown has been enqueued when calling child policy's handleResolvedAddresses() is being enqueued (e.g., receiving updates from XdsClient), it should not be executed. Otherwise, subchannels may be created by LBs that have already been shut down.

This change fixes LB config propagations in LB policies that manage a group of child LBs and delay the propagation for avoiding reentrancy. LB policies will always directly propagate child LB config/addresses updates directly. On the other hand, upcalls from child LB policies for balancing state updates will be queued and executed later.
2021-01-06 18:25:03 -08:00
sanjaypujare f788eec9e0
xds: multiple changes needed for PSM security GA as discussed (#7777)
* xds: multiple changes needed for GA:
- check to allow XdsServerBuilder.build() only once
- add transportBuilder() to XdsServerBuilder
- remove "grpc/server" hardcoding
- reorder the shutdown of delegate and xdsClient as per new design
2021-01-06 16:44:34 -08:00
Elliotte Rusty Harold 43d2e53a2a
alts: Remove dependency on Apache commons lang 2021-01-06 10:05:44 -08:00
Chengyuan Zhang edfc0d4d02
android-interop-testing: downgrade target SDK version to 26 and eliminate androidx dependencies (#7781)
androidx.appcompat is not ready to be used internally. This change downgrades android-interop-testing's target SDK version to 26 and use android.support's appcompat instead.
2021-01-05 17:41:08 -08:00
Yifei Zhuang 53da588dd1
Move multiple-port ServerImpl to NettyServer (#7674)
Change InternalServer to handle multiple addresses and implemented in NettyServer.
It makes ServerImpl to have a single transport server, and this single transport server (NettyServer) will bind to all listening addresses during bootstrap. (#7674)
2021-01-05 13:24:16 -08:00
ZHANG Dapeng ccef406f89
api: fix LoadBalancer javadoc 2021-01-05 13:12:06 -08:00
Chengyuan Zhang 4548cc5632
xds: fix regression for deleting EDS resources referenced by unchanged CDS resources (#7778)
A bug was introduced in the previous change for processing CDS responses. It mistakenly deleted EDS resources valid still referenced by CDS resources. EDS resource names either appear as the edsServiceName for CDS resources or has the same name as the CDS resources whose edsServiceName is null.

In the previous change, EDS resources having the same name as CDS resources are not retained. This caused LB subtrees for those EDS resources are shut down mistakenly, leading to RPC fails.
2021-01-05 12:08:40 -08:00
Eric Gribkoff f2f3bbeb5d
Start 1.36.0 development cycle (#7770) 2020-12-30 17:14:04 -08:00
Chengyuan Zhang 73fe68eeca
xds: support getting logical DNS and aggregate cluster configurations from CDS responses (#7696)
This change adds support for receiving and extracting configurations for logical DNS clusters and aggregate clusters.
2020-12-30 16:30:35 -08:00
Chengyuan Zhang 64d2bf1256
android, cronet, android-interop-testing: bump Android versions (#7740)
Bumps target SDK version of grpc-android, grpc-cronet, grpc-android-interop-testing to API 29. Major related changes are:

- Migrated android.support to androidx, which is required for Android Q.
- android.net.NetworkInfo is deprecated in SDK 29, suppressed compiler warnings as it is needed for supporting old Android versions.
- Robolectric requires Java 9+ for Android 29, which causes unit tests in grpc-cronet fail. Added annotation to emulate Android 28 for grpc-cronet's test.
- Upgraded Android dependencies accordingly in android-interop-testing.
2020-12-29 18:05:09 -08:00
Eric Anderson 8359d0b710
netty: Upgrade to Netty 4.1.52 and tcnative 2.0.34
The tiny cache size was removed from the bytebuf allocator and so was
deprecated. TLSv1.3 was enabled by the upgrade, which fails mTLS
connections at different times. Conscrypt is incompatible with the
default TrustManager when TLSv1.3 is enabled so we explicitly disable
TLSv1.3 when Conscrypt is used for the moment.
2020-12-29 15:20:46 -08:00
Chengyuan Zhang cddc1a500c
xds: decouple xds channel creation and bootstrapping (#7764)
This change fixes the problem of mismatched lifecycle of the xDS channel and XdsClient. Reading the bootstrap will determine and create the ChannelCredentials for each specified xDS server. An exception will be thrown if any xDS server specifies some channel_creds type that is not supported, not just for the first server (which is the only one to be used now). Reading the bootstrap also determines the xDS protocol version. The xDS channel will have the same lifecycle as the XdsClient instance: an xDS channel is created at the first call of getObject() and is shut down at the same time as the XdsClient is shutting down. A new xDS channel will be created when the ObjectPool creates a new XdsClient instance.
2020-12-28 18:39:14 -08:00
sanjaypujare 67ad78608c
xds: fix typo in a log message (#7762) 2020-12-27 18:10:45 -08:00
Eric Anderson 20197d36ce core: Include wait-for-ready in deadline exceeded insights
There was a report from a user in b/176088054 that experienced an RPC
waiting_for_connection that failed after 45 minutes due to deadline.
That would be quite normal for wait-for-ready, but because we didn't
include that detail in the status we have to do code inspection to
determine if wait-for-ready was enabled.
2020-12-23 13:37:21 -08:00
Eric Anderson b3899087d0 api: Improve code clarity by using Arrays.copyOf()
As noticed by a linter
2020-12-23 13:08:01 -08:00
sanjaypujare a4aa955d66
xds: don't add ServerTlsHandler if current handler is removed - also fix logging. (#7755) 2020-12-23 12:58:53 -08:00
ZHANG Dapeng 7d77f64773
compiler: remove some of the static imports in codegen (#7751)
Resolves #7741 
Some of the static methods in generated code have the same method name but different package name, such `ClientCalls.asyncClientStreamingCall` and `ServerCalls.asyncClientStreamingCall`. It's less readable using static import than using full-qualified method name in-place.
2020-12-23 11:28:03 -08:00
Sergii Tkachenko e5f57adb7f
buildscripts: xDS Kubernetes Interop tests buildscript 2020-12-23 09:52:01 -05:00
ZHANG Dapeng 90d61178a3
all: ChannelCredentials.withoutBearerTokens() and LoadBalancer.Helper API change (#7748)
API change (See go/grpc-rls-callcreds-to-server):

- Add `ChannelCredentials.withoutBearerTokens()`
- Add `createResolvingOobChannelBuilder(String, ChannelCredentials)`, `getChannelCredentials()` and `getUnsafeChannelCredentials()` for `LoadBalancer.Helper`

This PR does not include the implementation of `createResolvingOobChannelBuilder(String, ChannelCredentials)`.
2020-12-22 22:48:39 -08:00
sanjaypujare e530e10648
xds: import latest envoy and udpa protos to get new fields (#7747) 2020-12-22 12:22:40 -08:00
Chengyuan Zhang 5212e3c6df
xds: implement xds_cluster_resolver LB policy (#7685)
Implementation of the xDS cluster_resolver LB policy. It will replace the existing EdsLoadBalancer2.

The cluster_resolver LB policy supports discovering endpoints for aggregate clusters. Its config contains a list of DiscoveryMechanisms for each underlying cluster (the CDS LB policy will flatten an aggregate cluster to a list of underlying clusters), with each represents the mechanism (via EDS or DNS) to be used for resolving endpoints of each underlying cluster. Endpoints in underlying cluster will be resolved independently, but endpoint addresses and the priority/locality structure for each underlying cluster will be combined together before passing down to the child LB policy (aka, Priority LB policy).
2020-12-22 12:16:30 -08:00
Chengyuan Zhang 1e49ac665a
buildscript: ensure dependency convergence (#7745)
Run grpc-all:dependency task in Kokoro. For some unknown reason, building grpc-all doesn't reliably catch dependency conflicts.
2020-12-22 09:09:11 -08:00
Yifei Zhuang a67d816f4b
Revert " check pending stream completion at delayed transport lifecycle (#7720)" (#7744)
This reverts commit 90850128a6.
Alternative solution: alternative #7743
2020-12-21 15:15:00 -08:00
Chengyuan Zhang 0fb2667c96
all: Fix opencensus-api dependency conflict (#7739)
We depend on 0.28.0 while oauth2 depends on 0.24.0. This change replaces oauth2's opencensus-api dependency with our own.
2020-12-21 12:02:17 -08:00
Yifei Zhuang 90850128a6
check pending stream completion at delayed transport lifecycle (#7720)
add onTransferComplete() at delayedStream and wait for all pending streams to complete transfer when shutting down delayedClientTransport
2020-12-21 11:56:51 -08:00
Eric Anderson da939ca762 stub: Stop linking to Guava Javadoc
This resolves the following warning when building with JDK 8 introduced
in 9c5427fd4:
javadoc: warning - Error fetching URL: https://google.github.io/guava/releases/30.0-android/api/docs/

Guava is now building their Javadoc with JDK 11. JDK 11 swapped from
producing package-list to element-list, and stopped creating
package-list entirely. This file is what Javadoc uses to cross-link
documentation, and so it no longer works on the JDK 8 build, even though
the files have virtually the same contents.

ListenableFuture was the only reason we were including the Guava
Javadoc; let's just drop the link instead of worrying about this
silliness.
2020-12-17 17:42:42 -08:00
Eric Anderson ec70b64610
Bump Gradle and plugin versions
Android plugins weren't touched, since they will need a lot more testing
when doing so.
2020-12-17 15:02:27 -08:00
Eric Anderson 499694e9da android-interop-testing: ErrorProne should ignore generated R class
This silences many MutablePublicArray warnings that we can't do anything
about.
2020-12-17 09:13:27 -08:00
Eric Anderson 9c5427fd44
Bump versions for lots of dependencies
I didn't touch Protobuf and Netty; we upgrade those individually. Below
are issues I encountered that caused me to not upgrade (further).

Guava 30.1-android fails to build with Android without enabling
desugaring. https://github.com/google/guava/issues/5358

Robolectric 4.4 breaks AndroidChannelBuilderTest.
https://github.com/grpc/grpc-java/issues/7731

Opencensus 0.28.1+ is incompatible with gRPC.
https://github.com/census-instrumentation/opencensus-java/issues/2069
https://github.com/grpc/grpc-java/issues/7732

Truth now defines the asm dependency as "compile" although it is still
optional. But asm appears to have accidentally included incorrect gradle
module metadata in their release (I see they've disabled the metadata on
master) which make gradle think it requires Java 8. We could asm
everywhere, but that's is annoying. It seems likely this will resolve
itself.

Mockito can be upgraded to 3.4.0, but it deprecates initMocks, which
causes more code churn than I wanted in this commit. I still
synchronized the example versions on 3.4.0, though, as it was already
being used in some examples and the examples don't use initMocks.
2020-12-17 09:04:09 -08:00
Eric Anderson d10d4a6a65 Update README etc to reference 1.34.1 2020-12-15 12:45:28 -08:00
Chengyuan Zhang 9ead2c7c28
xds: move subchannel TLS context attaching code to cluster_impl LB policy (#7709)
This change moves the xDS security implementation that attaches an SSLContextProviderSupplier as EAG attributes from CDS LB policy to cluster_impl LB policy. It is similar to how DropOverload and circuit breakers work. This change assumes the UpstreamTlsContext in an CDS response is configured for underlying clusters in the context of supporting aggregate clusters. The UpstreamTlsContext configuration is obtained from CdsUpdate, then it is passed to the child EDS LB policy, where it is embedded into the cluster_impl LB policy config that the EDS LB policy generates.
2020-12-14 17:08:51 -08:00
ZHANG Dapeng 821ec65f2e
rls: cleanup and minor enhancement for rls logging
Cleanup `toString()` for cache entries, and print more debug information about cache entry when `pickSubchannel()`. This will be more helpful to debug.
2020-12-11 11:45:33 -08:00
Eric Anderson 60d3e7c536
core: On unexpected EOS, mention whether the frame was empty
Empty DATA frames with EOS tell a stronger tale as to where the server
may have its bug.
2020-12-10 14:54:33 -08:00
Eric Anderson 4be68f3287
core: Don't leak CallCredentials into OOB channels
The addition of CompositeChannelCredentials allowed CallCredentials to
be passed to the ManagedChannel itself. But the implementation was buggy
and used the call creds for out-of-band channels as well, which is
inappropriate since they have a different authority.

This also fixes a bug where resolving OOB channels would have CallCreds
duplicated; that wasn't noticed or important because we don't use
CallCreds in OOB channels.

Fixes #7643
2020-12-10 11:49:29 -08:00
sanjaypujare 20fc907b21
xds: fix ServerXdsClient to return subscribed resources only for LDS (#7689) 2020-12-09 17:42:12 -08:00
赵延 f5865d5bf2
examples: Modify the way of getHedgingServiceConfig in io.grpc.examples.hedging.HedgingHelloWorldClient (#7700) 2020-12-09 16:21:23 -08:00
Eric Anderson 26e88d9326 CONTRIBUTING.md: Fix out-of-date CLA documentation
The CLA system swapped a while back and this was left behind.
2020-12-09 14:32:57 -08:00
Sergii Tkachenko d9becc74d8
interop: add channelz to xds interop test server running in non-secure mode 2020-12-09 11:06:52 -05:00
Eric Anderson 8ce6355e89 buildscripts: Combine android builds together
Previously the android projects were separate from the main build and
each other. For quite a while now they have been integrated in the main
project. There's no longer any need to build each separately.
2020-12-07 15:01:51 -08:00
Chengyuan Zhang 71a3c55177
buildscripts: add missing CI coverage for examples (#7708)
Adds CI coverage for building example/android/strictmode and examples/example-jwt-auth. Also cleans up existing Gradle and Maven build command in the CIs.
2020-12-07 14:43:32 -08:00