Commit Graph

5399 Commits

Author SHA1 Message Date
Sergii Tkachenko 461b10a907 services: add support for channelz.GetServer() 2021-02-01 19:33:16 -05:00
Martin Schaub 4c5559d54f Make clearer that AltsAuthContext is internal by renaming 2021-02-01 16:31:33 -08:00
Sergii Tkachenko 7f3ddca30f xds: sort FILES in import.sh scripts alphabetically 2021-02-01 18:43:41 -05:00
ZHANG Dapeng 92e7fd370b
core: user is responsible to override authority for resolvingOobChannelBuilder
ManagedChannelImpl should not override authority for createResolvingOobChannel(target, creds), because ManagedChannelImpl does not know what target and creds are.
2021-01-29 09:50:59 -08:00
yifeizhuang ef76337f5c
core: add more delayedStream tests (#7843)
Add more delayedStream tests related to #7750, where we changed to call realStream.start() synchronously with setting realStream.
2021-01-29 09:42:10 -08:00
ZHANG Dapeng 9bb9fef6b0
rls: use channel creds to create resolvingOobChannel 2021-01-29 09:29:39 -08:00
ZHANG Dapeng 9437783838
core: enhance ManagedChannelBuilder.overrideAuthority()
Enhance `ManagedChannelBuilder.overrideAuthority()` to make it impossible to use a different authority to a backend by wrapping ClientTransportFactory.newClientTransport() and setting ClientTransportOptions’ authority. To avoid confusing the LB policy, it would need to keep the original addresses to return during `Subchannel.getAddresses()`

The class `OverrideAuthorityNameResolverFactory` is deleted and its logic is moved into `ManagedChannelImpl`.
2021-01-29 09:29:06 -08:00
sanjaypujare 64676198c5
xds: implement all string matcher types for SAN list matching (#7801) 2021-01-29 08:32:17 +05:30
ZHANG Dapeng 45a151810c
all: implement Helper.createResolvingOobChannelBuilder(target, creds)
- Add APIs to `ClientTransportFactory`:
```java
public interface ClientTransportFactory {
  /**
   * Swaps to a new ChannelCredentials with all other settings unchanged. Returns null if the
   * ChannelCredentials is not supported by the current ClientTransportFactory settings.
   */
  SwapChannelCredentialsResult swapChannelCredentials(ChannelCredentials channelCreds);

  final class SwapChannelCredentialsResult {
    final ClientTransportFactory transportFactory;
    @Nullable final CallCredentials callCredentials;
  }
}
```

- Add `ChannelCredentials` to constructor args of `ManagedChannelImplBuilder`:
 ```java
public ManagedChannelImplBuilder(
      String target, @Nullable ChannelCredentials channelCreds, @Nullable CallCredentials callCreds, ...)
  ```
2021-01-28 09:49:53 -08:00
ZHANG Dapeng a6df2b2ff4
xds: parse HttpFault filter from LDS/RDS response 2021-01-27 20:45:18 -08:00
Chengyuan Zhang 09869528f8
xds: fix bug of missing cluster_service_name conversion in v2 ClusterStats (#7849) 2021-01-27 18:18:22 -08:00
Chengyuan Zhang 14a38ef9b5
interop-testing: implement test client support for xDS timeout test (#7837)
Changes the xDS interop test client to support timeout test.

- Synced xDS test proto messages with grpc-proto.
- Changed RpcConfig to be the configuration for per test method type. Added timeoutSec for its deadline configuration.
- Changed accumulated stats to include RPC status instead of just succeeded/failed.
2021-01-27 17:09:01 -08:00
yifeizhuang ac2ead70b4
core: delay CallCredentialsApplyingTransport shutdown until metadataApplier finalized (#7813)
Improve the CallCredentialsApplyingTransport shutdown lifecycle management. Right now CallCredentialsApplyingTransport shutdown the delegated real transport too early. It should be waiting for the metadataAppliers to finish because they may execute asynchronously. In addition, there is no shutdown check on CallCredentialsApplyingTransport for newStream(). The degraded lifecycle implementation may cause RejectionExecutionException, or accepting new RPCs after the underlying transport is already closed during channel shutdown.

We added listener on metadataApplier to notify completion, a magic counter to track the pending metadataApplier for delaying shutdown, also added shutdown check for newStream().
2021-01-26 12:01:16 -08:00
Eric Anderson dbd903c018
core: Rewrite builder class signatures to avoid internal class
This provides us a path forward with #7211 (hiding
AbstractManagedChannelImplBuilder and AbstractServerImplBuilder) while
providing users a migration path to manage the ABI breakage (#7552). We
do a .class hack so that recompiling avoids the internal class reference
yet the old methods are still available.

Leaving the classes as-is causes javac to compile two versions of each
method, one returning the public class (e.g. ServerBuilder) and one
returning the internal class (e.g., AbstractServerImplBuilder). However,
we rewrite the signature that is used at compile time so that new
compilations will not reference internal-returning methods.

This is intended to be temporary, just to give a migration path. Once we
have given users some time to recompile we will remove this rewriting
and change the generics to use public classes.
2021-01-25 17:29:12 -08:00
Doug Fawley ec270a7757
interop-testing: support rpc sleep behavior for xDS test server (#7836) 2021-01-25 10:14:37 -08:00
Eric Gribkoff 0ffd942fd2
buildscripts: increase xds job timeouts (#7831) 2021-01-25 04:33:01 -08:00
Srini Polavarapu 2a62de0559
Add link to CVE process (#7832) 2021-01-22 12:42:05 -08:00
Chengyuan Zhang 92364c89c7
xds: support reading bootstrap config directly from env var or system property values (#7826)
This change supports reading the bootstrap config from the following places (in order):

- If GRPC_XDS_BOOTSTRAP exists then use its value as the name of the bootstrap file. If the file is missing or the contents of the file are malformed, return an error.

- Else, if io.grpc.xds.bootstrap exists then use its value as the name of the bootstrap file. If the file is missing or the contents are malformed, return an error.

- Else, if GRPC_XDS_BOOTSTRAP_CONFIG exists then use its value as the bootstrap config. If the value is malformed, return an error.

- Else, if io.grpc.xds.bootstrapValue exists then use its value as the bootstrap config. If the value is malformed, return an error.
2021-01-22 00:26:45 -08:00
Nick Hill 2072df9be6
Random acts of garbage reduction
I noticed some opportunities to reduce allocations on hot paths
2021-01-21 10:45:31 -08:00
ZHANG Dapeng ae82c41032
xds: import fault proto 2021-01-21 09:56:57 -08:00
Eric Anderson 7b8105e100
core: DelayedStream should start() real stream immediately
DelayedClientTransport needs to avoid becoming terminated while it owns
RPCs. Previously DelayedClientTransport could terminate when some of its
RPCs had their realStream but realStream.start() hadn't yet been called.

To avoid that, we now make sure to call realStream.start()
synchronously with setting realStream. Since start() and the method
calls before start execute quickly, we can run it in-line. But it does
mean we now need to split the Stream methods into "before start" and
"after start" categories for queuing.

Fixes #6283
2021-01-20 15:01:49 -08:00
Sergii Tkachenko cd2c168f5b examples: Highlight the suggestion to checkout a tag 2021-01-20 15:40:51 -05:00
ZHANG Dapeng 150f994315
buildscript: fix execution path of xds.sh in xds_v3.sh 2021-01-20 09:21:36 -08:00
Chengyuan Zhang a857b5c911
xds: refactor Bootstrapper (#7810)
Split Bootstrapper into Bootstrapper interface and BootstrapperImpl for the concrete implementation. The implementation is cleaned up for better readability and testing. 

Note after this change, the environment variable/system property for setting the path to bootstrap file will be read statically at the time BootstrapImpl class is loaded, instead of when bootstrap() is called. Since in either case the bootstrap file is read for at most once (when the first Channel is created and its resolver starts), the change shouldn't have behavior change from the gRPC client's perspective.
2021-01-19 19:00:15 -08:00
ZHANG Dapeng c1f9658a72
buildscripts: fix buildscript unbound variable error 2021-01-19 18:33:55 -08:00
ZHANG Dapeng 56fc8b8626
buildscripts: add xds_v3 buildscripts 2021-01-19 15:01:21 -08:00
Chengyuan Zhang 29753f2009
xds: google_default should use TLS if address contains no cluster name (#7818)
Fixes bug introduced by 4130c5a1b8. TLS should be selected for addresses without cluster name attributes, even if grpc-xds is in classpath.
2021-01-19 10:16:06 -08:00
Jan Tattermusch b5a0d14da7 testing: Avoid trying Conscrypt on ARM with OpenJDK
This allows io.grpc.testing.integration.Http2OkHttpTest to pass on ARM.

Currently Concrypt doesn't offer an ARM version of the native
components for OpenJDK, so running Http2OkHttpTest ends up with a
failure when ran on an ARM machine:
java.lang.UnsatisfiedLinkError: no conscrypt_openjdk_jni-linux-aarch_64 in java.library.path
2021-01-19 09:27:01 -08:00
Jan Tattermusch 76c5a651a1
Add a simple way to run grpc-java test inside emulated ARM64 docker container 2021-01-19 09:05:22 -08:00
Chengyuan Zhang 1b23cf4f39
xds: fix lint warnings (#7817) 2021-01-15 17:03:39 -08:00
Chengyuan Zhang 4130c5a1b8
alts, xds: backend handshake protocol selection support for xDS in directpath (#7783)
Attaches an attribute on endpoint addresses resolved/discovered using xDS plugin. The attribute indicates whether the endpoint address is a direct Google service endpoint or a CFE. This lets the GoogleDefault credentials choose between ALTS (direct Google service endpoint) and TLS (CFE).

Due to dependency relation between grpc-xds and grpc-alts, GoogleDefault credentials will use the attribute key defined in grpc-xds reflectively.
2021-01-15 16:51:57 -08:00
Kun Zhang 23d279660c
grpclb: keep RR Subchannel state in TRANSIENT_FAILURE until becoming READY (#7816)
If all RR servers are unhealthy, it's possible that at least one
connection is CONNECTING at every moment which causes RR to stay in
CONNECTING. It's better to keep the TRANSIENT_FAILURE state in that
case so that fail-fast RPCs can fail fast.

The same changes have been made for RoundRobinLoadBalancer in #6657
2021-01-15 15:19:52 -08:00
Chengyuan Zhang 9016cf55d7
xds: make XdsAttributes really public internal (#7815)
Change XdsAttributes to InternalXdsAttributes. It is public internal and may be accessed out of grpc-xds.
2021-01-15 12:54:00 -08:00
Chengyuan Zhang 458b0e4447
xds: throw UnsupportedOperation for unimplemented XdsClient interface methods (#7814)
Letting unimplemented XdsClient methods throw avoids misuses.
2021-01-15 11:26:34 -08:00
Chengyuan Zhang 727b17bbcb
xds: delete old CDS and EDS LB policies (#7811)
Deleted old CDS and EDS LB policies as we've migrated to new LB hierachy support new fetures.
2021-01-15 00:06:26 -08:00
Chengyuan Zhang b01dfb9d3c
xds: promote CdsLoadBalancer2 (#7807)
This change migrates to the new LB hierarchy of CDS -> cluster_resolver -> priority -> cluster_impl -> ...
2021-01-14 15:45:18 -08:00
Chengyuan Zhang a584baf86a
xds: implement new CDS LB policy for supporting aggregate clusters (#7722)
The new CDS LB policy supports discovering aggregate clusters and logical DNS clusters. If the cluster given by its config is an aggregate cluster, it recursively resolves into a list of underlying clusters. It creates a cluster_resolver LB policy as its child policy that receives a list of DiscoveryMechanisms with each containing cluster-level configurations for a single non-aggregate cluster.
2021-01-14 13:17:55 -08:00
Chengyuan Zhang ffcc360ba9
core: further clean up leftovers in ManagedChannelImpl's LoadBalancer.Helper and Subchannel implementations (#7806) 2021-01-14 10:41:36 -08:00
Eric Gribkoff 3b03d317cd
Update README etc to reference 1.35.0 (#7804) 2021-01-13 19:34:33 -08:00
Chengyuan Zhang 389c5403e9
core: make subchannel creation timing restriction stricter (#7790)
Throw for subchannel creation if the channel is being shutting down and the delayed transport is terminated (aka, all retry calls has been finished). This enforces load balancer implementations to avoid creating subchannels after being shut down.
2021-01-13 17:01:02 -08:00
Yifei Zhuang ff59104b62
Revert "interop-testing: allow multiple port in test server (#7799)" (#7805)
This reverts commit dac8347063.

Changing from Grpc. newServerBuilder to NettyServerBuilder breaks interop-test with credentials.
2021-01-13 15:08:42 -08:00
Eric Anderson 03d04f1944 api: Reword retry javadoc to make clear service config is source of configuration
Multiple users have tried things like
`mcb.enableRetry().maxRetryAttempts(3)` and been confused when no
retries were performed. Providing a reference to the gRFC and
`defaultServiceConfig()` should greatly increase the clarity of how to
use the method.
2021-01-13 11:36:08 -08:00
Yifei Zhuang dac8347063
interop-testing: allow multiple port in test server (#7799) 2021-01-13 09:47:26 -08:00
Chengyuan Zhang b66d182bb9
api: delete LoadBalancer.Helper APIs that had been deprecated for a long time (#7793) 2021-01-11 15:25:35 -08:00
Chengyuan Zhang ca7f1e4cee
xds: slightly refactor and improve tests for cluster_manager LB policy (#7773) 2021-01-11 10:49:39 -08:00
Chengyuan Zhang 32173975e0
Revert "netty: add exporting SSL/TLS master key log feature (#7724)" (#7792)
This reverts commit 9bc05fba67.
2021-01-08 15:45:58 -08:00
huangqiangxiong 9bc05fba67
netty: add exporting SSL/TLS master key log feature (#7724)
Enable this feature by setting the system property
   -Dio.netty.ssl.masterKeyHandler=true
or
   System.setProperty(SslMasterKeyHandler.SYSTEM_PROP_KEY, "true");
The keys will be written to the log named "io.netty.wireshark" in
the warnning level. To export the keys to a file, you can configure
log factory like: (with log4j.xml for example)
<appender name="key-file" class="org.apache.log4j.RollingFileAppender">
	<param name="file" value="d:/keyfile.txt"/>
	<layout class="org.apache.log4j.PatternLayout">
		<param name="ConversionPattern" value="%m%n"/>
	</layout>
</appender>
<category name="io.netty.wireshark">
	<priority value="DEBUG" />
	<appender-ref ref="key-file" />
</category>

Wireshark can analyze the messages gRPC over TLS with this
key log file.

close #7199
2021-01-08 13:11:24 -08:00
sanjaypujare 2dba43c727
xds: reorder the shutdown of delegate and xds-client for better semantics of ServerWrapperForXds (#7791) 2021-01-08 12:02:09 -08:00
Chengyuan Zhang 18772e2470
xds: wire up re-resolution requests from LB policies for DNS clusters and bypass requests from LB policies for EDS clusters (#7769)
This change wires up the refreshNameResolution() API for triggering DNS re-resolution in ClusterResolverLoadBalancer, which provides a way for downstream LB policies to request a resolution refresh and only re-resolution requests from LB policies for the DNS cluster should trigger the DNS refresh.
2021-01-08 01:08:13 -08:00
Chengyuan Zhang 2755afeaa5
xds: clean up verbose getters and builders on XdsClient interface (#7784)
Eliminate getters and builders for xDS resource data. They are (should be effectively) immutable and mostly only used for internal implementations. Cleaning up getters and builders significantly reduces the verbosity.
2021-01-07 14:55:05 -08:00