Commit Graph

4996 Commits

Author SHA1 Message Date
Elliotte Rusty Harold 43d2e53a2a
alts: Remove dependency on Apache commons lang 2021-01-06 10:05:44 -08:00
Chengyuan Zhang edfc0d4d02
android-interop-testing: downgrade target SDK version to 26 and eliminate androidx dependencies (#7781)
androidx.appcompat is not ready to be used internally. This change downgrades android-interop-testing's target SDK version to 26 and use android.support's appcompat instead.
2021-01-05 17:41:08 -08:00
Yifei Zhuang 53da588dd1
Move multiple-port ServerImpl to NettyServer (#7674)
Change InternalServer to handle multiple addresses and implemented in NettyServer.
It makes ServerImpl to have a single transport server, and this single transport server (NettyServer) will bind to all listening addresses during bootstrap. (#7674)
2021-01-05 13:24:16 -08:00
ZHANG Dapeng ccef406f89
api: fix LoadBalancer javadoc 2021-01-05 13:12:06 -08:00
Chengyuan Zhang 4548cc5632
xds: fix regression for deleting EDS resources referenced by unchanged CDS resources (#7778)
A bug was introduced in the previous change for processing CDS responses. It mistakenly deleted EDS resources valid still referenced by CDS resources. EDS resource names either appear as the edsServiceName for CDS resources or has the same name as the CDS resources whose edsServiceName is null.

In the previous change, EDS resources having the same name as CDS resources are not retained. This caused LB subtrees for those EDS resources are shut down mistakenly, leading to RPC fails.
2021-01-05 12:08:40 -08:00
Eric Gribkoff f2f3bbeb5d
Start 1.36.0 development cycle (#7770) 2020-12-30 17:14:04 -08:00
Chengyuan Zhang 73fe68eeca
xds: support getting logical DNS and aggregate cluster configurations from CDS responses (#7696)
This change adds support for receiving and extracting configurations for logical DNS clusters and aggregate clusters.
2020-12-30 16:30:35 -08:00
Chengyuan Zhang 64d2bf1256
android, cronet, android-interop-testing: bump Android versions (#7740)
Bumps target SDK version of grpc-android, grpc-cronet, grpc-android-interop-testing to API 29. Major related changes are:

- Migrated android.support to androidx, which is required for Android Q.
- android.net.NetworkInfo is deprecated in SDK 29, suppressed compiler warnings as it is needed for supporting old Android versions.
- Robolectric requires Java 9+ for Android 29, which causes unit tests in grpc-cronet fail. Added annotation to emulate Android 28 for grpc-cronet's test.
- Upgraded Android dependencies accordingly in android-interop-testing.
2020-12-29 18:05:09 -08:00
Eric Anderson 8359d0b710
netty: Upgrade to Netty 4.1.52 and tcnative 2.0.34
The tiny cache size was removed from the bytebuf allocator and so was
deprecated. TLSv1.3 was enabled by the upgrade, which fails mTLS
connections at different times. Conscrypt is incompatible with the
default TrustManager when TLSv1.3 is enabled so we explicitly disable
TLSv1.3 when Conscrypt is used for the moment.
2020-12-29 15:20:46 -08:00
Chengyuan Zhang cddc1a500c
xds: decouple xds channel creation and bootstrapping (#7764)
This change fixes the problem of mismatched lifecycle of the xDS channel and XdsClient. Reading the bootstrap will determine and create the ChannelCredentials for each specified xDS server. An exception will be thrown if any xDS server specifies some channel_creds type that is not supported, not just for the first server (which is the only one to be used now). Reading the bootstrap also determines the xDS protocol version. The xDS channel will have the same lifecycle as the XdsClient instance: an xDS channel is created at the first call of getObject() and is shut down at the same time as the XdsClient is shutting down. A new xDS channel will be created when the ObjectPool creates a new XdsClient instance.
2020-12-28 18:39:14 -08:00
sanjaypujare 67ad78608c
xds: fix typo in a log message (#7762) 2020-12-27 18:10:45 -08:00
Eric Anderson 20197d36ce core: Include wait-for-ready in deadline exceeded insights
There was a report from a user in b/176088054 that experienced an RPC
waiting_for_connection that failed after 45 minutes due to deadline.
That would be quite normal for wait-for-ready, but because we didn't
include that detail in the status we have to do code inspection to
determine if wait-for-ready was enabled.
2020-12-23 13:37:21 -08:00
Eric Anderson b3899087d0 api: Improve code clarity by using Arrays.copyOf()
As noticed by a linter
2020-12-23 13:08:01 -08:00
sanjaypujare a4aa955d66
xds: don't add ServerTlsHandler if current handler is removed - also fix logging. (#7755) 2020-12-23 12:58:53 -08:00
ZHANG Dapeng 7d77f64773
compiler: remove some of the static imports in codegen (#7751)
Resolves #7741 
Some of the static methods in generated code have the same method name but different package name, such `ClientCalls.asyncClientStreamingCall` and `ServerCalls.asyncClientStreamingCall`. It's less readable using static import than using full-qualified method name in-place.
2020-12-23 11:28:03 -08:00
Sergii Tkachenko e5f57adb7f
buildscripts: xDS Kubernetes Interop tests buildscript 2020-12-23 09:52:01 -05:00
ZHANG Dapeng 90d61178a3
all: ChannelCredentials.withoutBearerTokens() and LoadBalancer.Helper API change (#7748)
API change (See go/grpc-rls-callcreds-to-server):

- Add `ChannelCredentials.withoutBearerTokens()`
- Add `createResolvingOobChannelBuilder(String, ChannelCredentials)`, `getChannelCredentials()` and `getUnsafeChannelCredentials()` for `LoadBalancer.Helper`

This PR does not include the implementation of `createResolvingOobChannelBuilder(String, ChannelCredentials)`.
2020-12-22 22:48:39 -08:00
sanjaypujare e530e10648
xds: import latest envoy and udpa protos to get new fields (#7747) 2020-12-22 12:22:40 -08:00
Chengyuan Zhang 5212e3c6df
xds: implement xds_cluster_resolver LB policy (#7685)
Implementation of the xDS cluster_resolver LB policy. It will replace the existing EdsLoadBalancer2.

The cluster_resolver LB policy supports discovering endpoints for aggregate clusters. Its config contains a list of DiscoveryMechanisms for each underlying cluster (the CDS LB policy will flatten an aggregate cluster to a list of underlying clusters), with each represents the mechanism (via EDS or DNS) to be used for resolving endpoints of each underlying cluster. Endpoints in underlying cluster will be resolved independently, but endpoint addresses and the priority/locality structure for each underlying cluster will be combined together before passing down to the child LB policy (aka, Priority LB policy).
2020-12-22 12:16:30 -08:00
Chengyuan Zhang 1e49ac665a
buildscript: ensure dependency convergence (#7745)
Run grpc-all:dependency task in Kokoro. For some unknown reason, building grpc-all doesn't reliably catch dependency conflicts.
2020-12-22 09:09:11 -08:00
Yifei Zhuang a67d816f4b
Revert " check pending stream completion at delayed transport lifecycle (#7720)" (#7744)
This reverts commit 90850128a6.
Alternative solution: alternative #7743
2020-12-21 15:15:00 -08:00
Chengyuan Zhang 0fb2667c96
all: Fix opencensus-api dependency conflict (#7739)
We depend on 0.28.0 while oauth2 depends on 0.24.0. This change replaces oauth2's opencensus-api dependency with our own.
2020-12-21 12:02:17 -08:00
Yifei Zhuang 90850128a6
check pending stream completion at delayed transport lifecycle (#7720)
add onTransferComplete() at delayedStream and wait for all pending streams to complete transfer when shutting down delayedClientTransport
2020-12-21 11:56:51 -08:00
Eric Anderson da939ca762 stub: Stop linking to Guava Javadoc
This resolves the following warning when building with JDK 8 introduced
in 9c5427fd4:
javadoc: warning - Error fetching URL: https://google.github.io/guava/releases/30.0-android/api/docs/

Guava is now building their Javadoc with JDK 11. JDK 11 swapped from
producing package-list to element-list, and stopped creating
package-list entirely. This file is what Javadoc uses to cross-link
documentation, and so it no longer works on the JDK 8 build, even though
the files have virtually the same contents.

ListenableFuture was the only reason we were including the Guava
Javadoc; let's just drop the link instead of worrying about this
silliness.
2020-12-17 17:42:42 -08:00
Eric Anderson ec70b64610
Bump Gradle and plugin versions
Android plugins weren't touched, since they will need a lot more testing
when doing so.
2020-12-17 15:02:27 -08:00
Eric Anderson 499694e9da android-interop-testing: ErrorProne should ignore generated R class
This silences many MutablePublicArray warnings that we can't do anything
about.
2020-12-17 09:13:27 -08:00
Eric Anderson 9c5427fd44
Bump versions for lots of dependencies
I didn't touch Protobuf and Netty; we upgrade those individually. Below
are issues I encountered that caused me to not upgrade (further).

Guava 30.1-android fails to build with Android without enabling
desugaring. https://github.com/google/guava/issues/5358

Robolectric 4.4 breaks AndroidChannelBuilderTest.
https://github.com/grpc/grpc-java/issues/7731

Opencensus 0.28.1+ is incompatible with gRPC.
https://github.com/census-instrumentation/opencensus-java/issues/2069
https://github.com/grpc/grpc-java/issues/7732

Truth now defines the asm dependency as "compile" although it is still
optional. But asm appears to have accidentally included incorrect gradle
module metadata in their release (I see they've disabled the metadata on
master) which make gradle think it requires Java 8. We could asm
everywhere, but that's is annoying. It seems likely this will resolve
itself.

Mockito can be upgraded to 3.4.0, but it deprecates initMocks, which
causes more code churn than I wanted in this commit. I still
synchronized the example versions on 3.4.0, though, as it was already
being used in some examples and the examples don't use initMocks.
2020-12-17 09:04:09 -08:00
Eric Anderson d10d4a6a65 Update README etc to reference 1.34.1 2020-12-15 12:45:28 -08:00
Chengyuan Zhang 9ead2c7c28
xds: move subchannel TLS context attaching code to cluster_impl LB policy (#7709)
This change moves the xDS security implementation that attaches an SSLContextProviderSupplier as EAG attributes from CDS LB policy to cluster_impl LB policy. It is similar to how DropOverload and circuit breakers work. This change assumes the UpstreamTlsContext in an CDS response is configured for underlying clusters in the context of supporting aggregate clusters. The UpstreamTlsContext configuration is obtained from CdsUpdate, then it is passed to the child EDS LB policy, where it is embedded into the cluster_impl LB policy config that the EDS LB policy generates.
2020-12-14 17:08:51 -08:00
ZHANG Dapeng 821ec65f2e
rls: cleanup and minor enhancement for rls logging
Cleanup `toString()` for cache entries, and print more debug information about cache entry when `pickSubchannel()`. This will be more helpful to debug.
2020-12-11 11:45:33 -08:00
Eric Anderson 60d3e7c536
core: On unexpected EOS, mention whether the frame was empty
Empty DATA frames with EOS tell a stronger tale as to where the server
may have its bug.
2020-12-10 14:54:33 -08:00
Eric Anderson 4be68f3287
core: Don't leak CallCredentials into OOB channels
The addition of CompositeChannelCredentials allowed CallCredentials to
be passed to the ManagedChannel itself. But the implementation was buggy
and used the call creds for out-of-band channels as well, which is
inappropriate since they have a different authority.

This also fixes a bug where resolving OOB channels would have CallCreds
duplicated; that wasn't noticed or important because we don't use
CallCreds in OOB channels.

Fixes #7643
2020-12-10 11:49:29 -08:00
sanjaypujare 20fc907b21
xds: fix ServerXdsClient to return subscribed resources only for LDS (#7689) 2020-12-09 17:42:12 -08:00
赵延 f5865d5bf2
examples: Modify the way of getHedgingServiceConfig in io.grpc.examples.hedging.HedgingHelloWorldClient (#7700) 2020-12-09 16:21:23 -08:00
Eric Anderson 26e88d9326 CONTRIBUTING.md: Fix out-of-date CLA documentation
The CLA system swapped a while back and this was left behind.
2020-12-09 14:32:57 -08:00
Sergii Tkachenko d9becc74d8
interop: add channelz to xds interop test server running in non-secure mode 2020-12-09 11:06:52 -05:00
Eric Anderson 8ce6355e89 buildscripts: Combine android builds together
Previously the android projects were separate from the main build and
each other. For quite a while now they have been integrated in the main
project. There's no longer any need to build each separately.
2020-12-07 15:01:51 -08:00
Chengyuan Zhang 71a3c55177
buildscripts: add missing CI coverage for examples (#7708)
Adds CI coverage for building example/android/strictmode and examples/example-jwt-auth. Also cleans up existing Gradle and Maven build command in the CIs.
2020-12-07 14:43:32 -08:00
Eric Anderson f13471d68d buildscripts: Check for out-of-date codegen after android builds
Issues like that fixed in 05048cf3 should have failed the CI, like it
does for the rest of the build (on both Travis and linux_artifacts).
2020-12-07 11:56:06 -08:00
Eric Anderson 3f1563434d xds: Add missing relocations and jacoco exclusions
We have been leaking classes since grpc-xds's inception. That's no good.

Lists are now sorted and use the same package names so they can be
easily compared. Excluding things like com/github/** is too aggressive,
and is unclear what is being targeted.
2020-12-07 11:42:24 -08:00
sanjaypujare 71606479ed
interop: add channelz and reflection support to xds interop test client and server (#7701)
* interop: add channelz support to xds interop test client and server

* add reflection

Co-authored-by: Sergii Tkachenko <sergiitk@google.com>
2020-12-07 10:48:36 -08:00
Eric Anderson ee9286a0f2 buildscripts: Fix grpc-java-artifacts indentation
Re-sort packages, now that the tabs aren't skewing the sorting.

I'm quite confident I had fixed this already, but I had multiple copies
of this file on multiple machines and must have fixed the wrong copy.
2020-12-04 16:42:26 -08:00
ZHANG Dapeng af29777686
all: migrate to interceptor-based config selector
This is migration following #7610
2020-12-04 11:11:57 -08:00
Chengyuan Zhang 0be913467d
xds: move drop support and circuit breaking into xds_cluster_impl LB policy (#7631)
Implementation for cluster_impl LB policy as per C2P design. This policy sits in the middle of priority LB policy and weighted_target LB policy, its major functionality is to handle drops and apply circuit breakers.

The idea is to move implementations of handling drops and circuit breakers from the EDS LB policy (will be split to cluster_resolver LB policy and cluster_impl LB policy) to cluster_impl LB policy.
2020-12-04 10:45:56 -08:00
Eric Anderson 486d82a7e1 Build Linux artifacts using CentOS 7
CentOS 6 is dead and no longer has update servers. CentOS 7 is older
than Debian 9 (oldstable), so binaries hopefully work on both. More
testing is necessary, but everything's broken now, so this is better
than nothing.

We stop using protoc-artifacts because now the container is
straight-forward enough that we can just use our own. Previously the
"devtoolset" stuff made us want to share the container.
2020-12-03 22:56:31 -08:00
ZHANG Dapeng 90e3dcf1ae
xds: provide more debug info for priority lb error-picker
When all priorities fail, update the overall picker with the picker from the last priority. This still does not provide the full detail why all priorities fail but is better than nothing being provided currently.
2020-12-03 14:07:24 -08:00
Eric Anderson 814e36b541 alts: Limit number of concurrent handshakes to 32 2020-12-03 13:51:50 -08:00
Chengyuan Zhang 7dc8ab1c6e
xds: print xDS request messages with JsonFormat (#7679)
Currently xDS response messages are printed with protobuf JsonFormat as they may contain Any-typed messages. JsonFormat internally uses Gson for generating human-readable message representations. The default Gson instance escapes HTML characters, which causes characters like '=' to be printed as '\u003d'. There is nothing can be done on our side to disable this as it is protobuf's internal implementation.

To make the same content consistent in both request and response messages consistent, we use JsonFormat to print both.
2020-12-01 15:57:18 -08:00
Chengyuan Zhang 9b276aa73a Update README etc to reference 1.34.0 2020-12-01 15:47:37 -08:00
Chengyuan Zhang 05048cf3e2
interop-testing: update proto generated service files (#7682) 2020-11-30 22:49:00 -08:00