Commit Graph

553 Commits

Author SHA1 Message Date
sanjaypujare 38cba5c8dd
xds: add all validations related to security as described in A29 gRFC (#8331) 2021-07-25 22:51:50 -07:00
ZHANG Dapeng f3642422b4
xds: support xds retry policy (#8304) 2021-07-22 12:04:06 -07:00
yifeizhuang 4c1272febd
api: use <scheme,provider> map in nameResoverRegistry (#8323)
An improvement that makes name resolver provider scheme matching more explicit in name resolver registry.
2021-07-21 10:03:55 -07:00
ZHANG Dapeng 9ed444ea2a
xds: add hint of fault injection to injected failures (#8326) 2021-07-14 19:37:25 -07:00
sanjaypujare 629748da61
xds: fix the race condition in SslContextProviderSupplier's updateSslContext and close (#8294) 2021-07-09 10:48:18 -07:00
sanjaypujare 3965315039
xds: implement filter-chain uniqueness check as per grfc A36 (#8295) 2021-07-08 17:22:43 -07:00
Eric Anderson 0cabf5672a compiler: Add GrpcGenerated annotation to generated class
This can be used by annotation processors to avoid processing the
gRPC-generated code. The normal Generated annotation only has SOURCE
retention, so isn't available to annotation processors.

I don't include the service name within the annotation as that assumes
we'll never have need for any other type of generated class. If there's
a request for exposing service name via an annotation in the future, we
can make an RpcService annotation or the like.

Fixes #8158
2021-07-02 22:11:40 -07:00
Eric Anderson f93cfe5add xds: Delete unused ScheduledExecutorService management code
In 02ff64fa2 the SharedResourceHolder.get() was removed and it became
dead code.
2021-06-29 11:33:19 -05:00
Eric Anderson 4814d975a5
xds: Avoid NPE for no filter chain match on server-side 2021-06-29 09:32:37 -07:00
yifeizhuang 3aa871b7de
xds: remove cell based rbac engine (#8277) 2021-06-25 11:20:11 -07:00
sanjaypujare b118a590c8
xds: remove unused SDS code (#8282) 2021-06-23 20:58:22 -07:00
sanjaypujare e4ab8287d0
xds: get rid of legacy SDS and file watching code (#8276) 2021-06-23 11:13:19 -07:00
Chengyuan Zhang 9a8bc10f51
xds: unify client and server handling HttpConnectionManager (#8228)
Enables parsing HttpConnectionManager filter for the server side TCP listener, with the same codepath for handling it on the client side. Major changes include:

- Remodeled LdsUpdate with HttpConnectionManager. Now LdsUpdate is an oneof of HttpConnectionManager (for client side) or Listener (for server side). Each of Listener's FiliterChain contains an HttpConnectionManager (required).
Refactored code for validating and parsing the TCP Listener (for server side), put it into ClientXdsClient. The common part of validating/parsing HttpConnectionManager is reused/shared for client side.
- Included the name of FilterChain in the parsed form. As specified by the API, each FilterChain has a unique name. If the name is not provided by the control plane, a UUID is used. FilterChain names can be used for bookkeeping a set of FilterChain easily (e.g., used as map key).
- Added methods isSupportedOnClients() and isSupportedOnServers() to the Filter interface. Parsing the top-level HttpFilter requires knowing if the HttpFilter implementation is supported for the target usage (client-side or server-side). Note, parsing override HttpFilter configs does not need to know whether the config is used for an HttpFilter that is only supported for the client-side or server side.
- Added a new kind of Route: Route with non-forwarding action. Updated the XdsNameResolver being able to handle Route with non-forwarding action: if such a Route is matched to an RPC, that RPC is failed. Note, it is possible that XdsNameResolver receives xDS updates with all Routes with non-forwarding action. That is, the service config will not reference any cluster. Such case can be handled by cluster_manager LB policy's LB config parser: the parser returns the error to Channel and the Channel will handle it as error service config.
2021-06-18 11:57:36 -07:00
yifeizhuang 84eb285742
xds: add override rbacfilter type url RbacPerProto (#8262) 2021-06-15 16:50:50 -07:00
yifeizhuang c8ba601529
xds: add rbac http filter (#8251) 2021-06-14 12:54:07 -07:00
Chengyuan Zhang 91948b2606
xds: fix lint (#8248) 2021-06-09 14:57:26 -07:00
Chengyuan Zhang d41094944c
xds: equally weight endpoints within locality if endpoint-level weight unspecified (#8245)
Use a multiplier of 1 for endpoints with endpoint-level load balancing weight unspecified when computing weights for mixing-locality load balancing. Therefore, if a locality has endpoints without endpoint-level load balancing weight, they are weighted equally within the locality.
2021-06-09 12:04:17 -07:00
yifeizhuang b7f3fddc76
xds, rbac: implement rbac engine (#8168) 2021-06-08 14:45:11 -07:00
Chengyuan Zhang fa4b980e07
xds: use defaults for unspecified ring_hash_lb_config values (#8237)
Sets ring_hash LB config to its default values (min_ring_size = 1024 and max_ring_size = 8M) if not given by the control plane. This applies to both parsing RingHashLbConfig from xDS proto and parsing RingHashConfig from the JSON config (currently not used). If the values are given by the control plane, they are validated such that min_ring_size is not less than max_ring_size and do not exceed the 8M limit.
2021-06-07 14:26:50 -07:00
Chengyuan Zhang e51a17574f
xds: append a random number to C2P generated node id (#8239)
Adding a random number to the xDS stream node id helps debugging for distinguishing between different clients.
2021-06-07 11:01:04 -07:00
sanjaypujare 4209c8d8cc
xds: close SslContexrProviderSupplier when the CDS LoadBalancer is shut down to prevent leakage (#8240) 2021-06-07 10:53:57 -07:00
sanjaypujare d8d378454f
xds: remove XdsChannelBuilder and related code that uses old/unsupported interfaces (#8231) 2021-06-03 10:07:05 -07:00
sanjaypujare 087d7bc7d5
xds: move the unsupported filterChainMatch matchers to the ranking stage for correct outcomes (#8219) 2021-06-02 10:10:58 -07:00
sanjaypujare 54b4e93927
xds: replace PriorityHeap with simpler logic that keeps track of top matches (#8225) 2021-06-02 10:09:42 -07:00
Chengyuan Zhang a589c2c68f
xds: fix order of processing resolution errors with original cluster ordering (#8224)
When aggregating the endpoint resolution errors of the list of clusters in ClusterResolverLoadBalancer, clusters should be processed in its original order as received in the LB config. The last cluster's error is used as the overall error status.
2021-06-01 11:22:24 -07:00
sanjaypujare bfcba82dd5
xds: remove MeshCaCertificateProvider and DynamicReloadingCertificate{Provider (#8214) 2021-05-26 19:35:51 -07:00
sanjaypujare 328071bbce
xds: replace DownstreamTlsContext by SslContextProviderSupplier in the Listener (#8205) 2021-05-26 14:42:47 -07:00
ZHANG Dapeng 6aeeba805f
xds: enhance delay injection error message on DEADLINE_EXCEEDED (#8185)
When an RPC is injected with a delay and then fails with DEADLINE_EXCEEDED (partially) due to the delay, it could confuse users if the error message does not mention the existence of the delay injection, because end users normally are not the same people who configured fault injection policy in control plane.
2021-05-26 14:35:45 -07:00
Chengyuan Zhang bbc5f61abb
xds: use load assignment endpoint address in Cluster as the DNS hostname for LOGICAL_DNS (#8151)
Fixes the source of hostname used for DNS resolution in the cluster_resolver LB policy for LOGICAL_DNS clusters. The change includes:

- parse the single endpoint address from the embedded Cluster resource in CDS responses as the DNS hostname for LOGICAL_DNS cluster and include it in CdsUpdate being notified to the CDS LB policy.
- propagate the DNS hostname to the cluster_resolver LB policy via its LB config (DiscoveryMechanism for LOGICAL_DNS cluster).
- cluster_resolver LB policy takes the DNS hostname from the DiscoveryMechanism for LOGICAL_DNS cluster and use it as the name for DNS resolution.
2021-05-26 12:02:18 -07:00
yifeizhuang 2239dd717c
tsan, xds: fix data race (#8206) 2021-05-25 13:35:09 -07:00
sanjaypujare 5b1c3fa12c
xds: shutDown the scheduledExecutorService when the provider is shutdown (#8198) 2021-05-24 12:45:01 -07:00
sanjaypujare 869b395ec0
xds: ignore unknown SAN name type instead of throwing exception (#8183) 2021-05-19 11:48:11 -07:00
Chengyuan Zhang 86465b3399
xds: cluster_resolver LB policy should wait until all clusters being resolved before propagating endpoints to child LB policy (#8176)
Do not propagate partial endpoint discovery results to the child LB policy of cluster_resolver LB policy. This could avoid premature RPC failures when connections to resolved endpoints fail while there are other unresolved endpoints. Also, endpoints should be attempted in the order of clusters they belong to: endpoints from a lower-priority cluster should not be used before endpoints from a higher-priority cluster are attempted. Most importantly, it should not fallback to use DNS-resolved endpoints before all EDS-resolved endpoints failed.
2021-05-18 13:14:37 -07:00
Chengyuan Zhang 413deb7f0c
xds: implement PriorityChildConfig toString() (#8173) 2021-05-12 16:01:40 -07:00
Chengyuan Zhang 2335eb5b63
xds: eliminate test verification for nondeterministic behaviors (#8172)
When the ring_hash LB policy enters TRANSIENT_FAILURE, it tries to connect one of the IDLE subchannels. Which subchannel to be connected to is non-deterministic, it just choose the first one from the subchannels map.

The existing test creates 4 subchannels, brings down 2 of them to let ring_hash LB policy enter TRANSIENT_FAILURE. But which one fo the remaining two subchannels to be kicked off connection is nondeterministic. This introduces trouble for verifying the behavior. This change simplifies the test, to only create 3 subchannels so that there is only one single subchannel remaining in IDLE after bringing the other two down. We are able to easily verify the behavior of ring_hash LB policy requesting connection for that one subchannel.
2021-05-12 14:17:21 -07:00
sanjaypujare e59604b7ce
xds: add null reference checks in SslContextProviderSupplier (#8169) 2021-05-12 10:27:44 -07:00
Eric Anderson e08b9db208
Use @DoNotCall for static methods in Builders that throw
Since static methods are pseudo-inherited by Builder implementations but
are trivially accidentally used, we re-define static methods in each
builder to make them behave more like the caller would expect. However,
not all the methods actually work; some just throw because the caller
was certainly not getting what they would expect.

Annotating with `@DoNotCall` can expose the problems at compile time
instead of runtime. While `@Deprecated` would also be an option, it is a
bit harder to figure out the ramifications and whether we want to go
that route.

This change was suggested by a lint tool for XdsServerBuilder and it
seems appropriate so I applied it to the other similar cases I could
find.
2021-05-12 10:12:52 -07:00
Chengyuan Zhang f4fe466fb0
xds: lazily and only parse headers with matchers matching the key (#8163)
In normal cases, we only have a few header matchers but the number of headers can be completely up to the application. Indexing headers eagerly parses all headers, even for those with no matcher matching the key. We should only parse header values for those with key matching the header matcher (aka, only call Metadata.get() with key that has some matcher looking for).
2021-05-11 14:20:02 -07:00
Chengyuan Zhang dbc5786c30
xds: ring_hash self recover from TRANSIENT_FAILURE by attempting to connect one subchannel (#8144)
Kicks off connection for one of IDLE subchannels (if exist) when the ring_hash LB policy is reporting TRANSIENT_FAILURE to its upstream.

While the ring_hash policy is reporting TRANSIENT_FAILURE, it will not be getting any pick requests from the priority policy. However, because the ring_hash policy does not attempt to reconnect to subchannels unless it is getting pick requests, it will need special handling to ensure that it will eventually recover from TRANSIENT_FAILURE state once the problem is resolved. Specifically, it will make sure that it is attempting to connect (after applicable backoff period) to at least one subchannel at any given time.
2021-05-11 01:58:57 -07:00
sanjaypujare 0c2d8edc4c
xds: refactor TlsContextManager related code to remove dependency on Bootstrapper (#8150) 2021-05-10 13:13:26 -07:00
Chengyuan Zhang 7b09056aa4
xds: use a standalone Context for xDS control plane RPCs (#8153)
Control plane RPCs are independent of application RPCs, they can stand for completely different lifetime. So the context for making application RPCs should not be propagated to control plane RPCs. This change makes control plane RPCs use the ROOT Context.
2021-05-07 18:00:47 -07:00
sanjaypujare c9e327d42f
xds: extend SslContextProviderSupplier to DowmstreamTlsContext for server side (#8146) 2021-05-04 22:19:15 -07:00
Chengyuan Zhang fcaf9a9583
xds: ignore balancing state update from downstream after LB shutdown (#8134)
LoadBalancers should not propagate balancing state updates after itself being shutdown.

For LB policies that maintain a group of child LB policies with each having its independent lifetime, balancing state update propagations from each child LB policy can go out of the lifetime of its parent easily, especially for cases that balancing state update is put to the back of the queue and not propagated up inline.

For LBs that are simple pass-through in the middle of the LB tree structure, it isn't a big issue as its lifecycle would be the same as its child. Transitively, It would behave correctly as long as its downstream is doing in the right way.

This change is a sanity cleanup for LB policies that maintain multiple child LB policies to preserve the invariant that further balancing state updates from their child policies will not get propagated.
2021-05-04 15:56:56 -07:00
Chengyuan Zhang ee000f0dc1
xds: throw away subchannel references after ring_hash is shutdown (#8140)
Similar to 368c43aec4. 

Clean up subchannels after the RingHashLoadBalancer itself is shutdown to prevent further balancing state updates being propagated to the upstream.

Note this should not be considered as a fix for any problem anybody is noticing. Upstreams of RingHashLoadBalancer should not rely on this, it should still have its own logic for maintaining the lifecycle of downstream LB and ignore invalid upcalls when necessary.
2021-05-04 13:35:37 -07:00
Eric Anderson 16eb5a47ec Stabilize ChannelCredentials
Some of the experimental API annotations were changed to other issues or
became `@Internal` to match their related APIs.

Fixes #7479
2021-05-03 16:22:43 -07:00
Eric Anderson d42f3b8fcb Stabilize ServerCredentials
Some of the experimental API annotations were changed to other issues or
became @Internal to match their related APIs.

Fixes #7621
2021-05-03 16:10:24 -07:00
Chengyuan Zhang 4a339e41ba
xds: fix URI creation used to instantiate DNS name resolver (#8129)
When creating the URI using Channel authority for instantiating a DNS resolver in the cluster_resolver LB policy, a "dns" scheme needs to be manually attached and the Channel authority would be used as the URI path (same as creating Channel with target). Otherwise, the Channel authority will just be used as the scheme and causing name resolver not found.

The change also handles name resolver lookup more defensively. Although it should not happen, if there does have bug causing DNS resolver not being able to be loaded, the cluster_resolver LB policy propagates the INTERNAL error to upstream.
2021-04-30 18:10:40 -07:00
sanjaypujare 02ff64fa21
xds: use singleton XdsClient for server side (#8130) 2021-04-30 09:52:56 -07:00
Chengyuan Zhang 5d99bb07b8
xds: pretty print ClusterConfig message (#8128)
Adds ClusterConfig message descriptor to message printer.
2021-04-29 18:04:03 -07:00
Chengyuan Zhang 42d7fba1b8
xds: implement toString() for pickers to visualize selectable hosts (#8123)
Implements toString() for the wrapping SubchannelPickers so that we are able to see how hosts are selected when sending out RPCs.
2021-04-28 15:16:30 -07:00