Commit Graph

5042 Commits

Author SHA1 Message Date
yifeizhuang 38a554c23a
xds: implement RBAC gRFC misc cases (#8518) 2021-09-16 16:12:52 -07:00
yifeizhuang fcf13952bb
xds, rbac: build per route serverInterceptor for httpConfig (#8524) 2021-09-16 12:35:09 -07:00
Eric Anderson 9d9d8ec66b
xds: Fix test compilation for confused javac
The internal build fails with "reference to assertThat is ambiguous". It
isn't clear why the internal build fails while the external one is okay,
but it is clear that the wildcard T return of readOutbound() is probably
confusing things as javac is considering assertThat(BigDecimal) as a
possible match.

The T return type is a hidden, convenience cast. We force the type
passed to assertThat() to be Object to avoid any ambiguity.
2021-09-16 12:09:15 -07:00
sanjaypujare 49842d2af1
xds: add hashCode and equals back to SslContextProviderSupplier (#8528) 2021-09-15 15:46:22 -07:00
Eric Anderson 7669656725 Revert "netty: Requests with Connection header are malformed"
This reverts commit 6e89919e32.

This was found to break a test proxy. We'll work on fixing the proxy and
then roll this forward again.
2021-09-15 15:20:40 -07:00
Eric Anderson 43b507160f xds: Drain old server connections on Listener updates
This is necessary to make sure all connections are using the new
configuration.
2021-09-15 10:08:28 -07:00
Eric Anderson 5307b69c9e netty: Allow protocol negotiators to shut down transport, with grace period
This will be used for draining old connections when xDS configuration
changes.
2021-09-15 10:08:28 -07:00
Eric Anderson 122b3b2f7c
netty: Support Host header on server-side
We want to know the single, unambiguous authority for the request. If
there is no authority, we use host instead. While authority would be
most typical for HTTP/2, requests proxied from HTTP/1 may use host
instead of authority.

This is generally useful, but the impetus is RBAC. See gRFC A41.
2021-09-15 09:40:56 -07:00
ZHANG Dapeng 3b237339c7
core: discard outbound content-length header (#8522)
Since netty version v4.1.67, content-lenght header validation will be enforced. So once grpc upgrades netty to that version or above, RPCs with invalid content-length header will fail.

Some libraries such as HTTP to gRPC adapters blindly copy all HTTP headers to gRPC metadata, but the content-length header is one of those that shouldn't be forwarded because gRPC uses different encoding. This mistake has already been in existence for a long time.

Discard outbound content-length headers in gRPC, so that users who encounter invalid content-length issue when upgrading grpc-java version on server/client side would be able to workaround by upgrading grpc-java on client/server side as well without fixing the HTTP adapter.
2021-09-13 17:15:45 -07:00
Terry Wilson 876f56e2ea
api: Stabilize the Status.asException() call. (#8520)
Removes the ExperimentalApi annotation from this call.

Contributes to: #4683
2021-09-13 14:54:25 -07:00
Eric Anderson 6e89919e32
netty: Requests with Connection header are malformed
Although this is part of HTTP/2 and should have already been handled
already, it was noticed as part of RBAC work to avoid matching
hop-by-hop headers. See gRFC A41.

Also add a warning if creating Metadata.Key for "Connection". Use this
to try to help diagnose a client if it happens to blindly copy headers
from HTTP/1, as PROTOCOL_ERROR is hard to debug.
2021-09-13 11:30:19 -07:00
ZHANG Dapeng 7c6f53ab79
all: add internal API to disable retry stats (#8510)
Resolves b/197648853 for internal performance regression. Reporting retry stats caused significant amount of performance overhead internally.
2021-09-13 09:12:04 -07:00
ZHANG Dapeng 9ff54059d8
xds: populate envoy RetryPolicy with no retryOn to resolver (#8511)
Envoy RetryPolicy with empty retryOn should not be ignored as no retry config when selecting Route config. Therefore, if xDS update for a route contains a RetryPolicy that has no RetryOn value that we support, but the virtual host config does, xds client should choose the Envoy RetryPolicy from the route (even with no RetryOn), rather than choosing the one from virtual host, and try to convert it into grpc RetryPolicy, and end up with no retry.
2021-09-13 08:31:00 -07:00
ZHANG Dapeng 7a65c74283
xds: apply valid resources while NACKing update (#8506)
Implementing [gRFC A46](https://github.com/grpc/proposal/pull/260)
2021-09-11 21:57:47 -07:00
yifeizhuang 7ad7876e99
fix header matcher for null value (#8503) 2021-09-09 12:15:27 -07:00
yifeizhuang a6df9de7bb
xds: add terminal http filter verification, remove lame route filter, add hcm as terminal network filter verification (#8342)
* xds: add terminal filter verification, remove lame route filter

* move last filter check inline

* add server validate terminal filter
2021-09-09 09:55:27 -07:00
Benjamin Peterson 67d5f1b0d6 stub: update CallStreamObserver stabilization issue 2021-09-09 09:53:47 -07:00
yifeizhuang be7aa50441
xds: referenciate server routing config (#8491)
* routing config ref

* atomic ref virtual host list

* Revert "routing config ref"

This reverts commit cbcad5744f.

* test: noop config non-static, better validation
2021-09-08 18:32:26 -07:00
Eric Anderson 9870db1f47 stub: Document that noop onCancelHandler is useful
setOnCancelHandler tells gRPC that the application is handling
cancellation. But it's fine to have noop behavior within the handler
itself if the application doesn't need it. It is just a way to opt-in to
the more recent no-exception-from-onNext behavior. Let's mention this
use-case in the docs to make it more obvious it is a possibility.

Came up as part of #8409.
2021-09-08 16:48:20 -07:00
sanjaypujare 22603810b9
xds: use the new cert-provider instances if present (#8494) 2021-09-08 16:06:21 -07:00
sanjaypujare f71eedff40
xds: remove hashCode() and equals() for SslContextProviderSupplier (#8496) 2021-09-08 15:38:26 -07:00
ZhenLian fb00463001
fix a flaky test in advanced TLS (#8474)
* fix a flaky test in advanced tls
2021-09-08 11:43:23 -07:00
Eric Anderson 1f1396f3f0 Start 1.42.0 development cycle 2021-09-08 09:10:20 -07:00
ZHANG Dapeng cd346832ba
rls: migrate deprecated server/path to extraKeys (#8469)
The [`server` and `path` fields](https://github.com/grpc/grpc-java/blob/v1.40.1/rls/src/main/proto/grpc/lookup/v1/rls.proto#L25-L32) in `RouteLookupRequest` are deprecated. Instead, we will send the server/path information in side of [`key_map`](https://github.com/grpc/grpc-java/blob/v1.40.1/rls/src/main/proto/grpc/lookup/v1/rls.proto#L45).

The keys for the server, service and method in the `key_map` will be the _values_ of `host`, `service`, `method` fields respectively in [`extraKeys`](https://github.com/grpc/grpc-java/blob/v1.40.1/rls/src/main/proto/grpc/lookup/v1/rls_config.proto#L69) in RlsConfig.

We will also include all entries in the [`constantKey`](https://github.com/grpc/grpc-java/blob/v1.40.1/rls/src/main/proto/grpc/lookup/v1/rls_config.proto#L80) in RlsConfig into `RouteLookupRequest`.


Other changes:

- Add AutoValue library for ExtraKeys class, just like data classes used in grpc-xds. Will migrate other data classes to AutoValue as well.
- Not to keep `targetType` field in the route lookup request data class, because we always use "grpc" as targetType.
2021-09-07 21:32:33 -07:00
sanjaypujare 5dc6e0ca54
xds: update Envoy protos to a later revision for the new CertificateProvider definitions (#8490) 2021-09-07 14:27:49 -07:00
Sergii Tkachenko 6cd911757a
census: make internal linter happy
TODO is preferred to FIXME.
2021-09-03 13:26:43 -07:00
ZHANG Dapeng 5475cf12bb
xds: fix parsing retryOn values (#8477)
- Envoy ignores white spaces in `retryOn` field
https://github.com/envoyproxy/envoy/blob/v1.19.1/source/common/router/retry_state_impl.cc#L166

  We should do the same.

- Envoy ignores unsupported values https://github.com/envoyproxy/envoy/blob/v1.19.1/source/common/router/config_impl.cc#L89-L90
  and we should do the same.
2021-09-03 12:47:38 -07:00
sanjaypujare 4828698bec
xds: enable PSM security by default (#8478) 2021-09-03 12:38:26 -07:00
Sergii Tkachenko a91cc85dfd
Revert "core/auth: Remove CallCredentials2 (#8464)"
This reverts commit 7cde473efa.
2021-09-02 17:20:20 -07:00
Brice Jaglin 62fafe7eda core: clarify exception message
Reformulate message to highlight that SizeEnforcingInputStream is
applied on the message size of the message after decompression.
2021-09-02 15:18:00 -07:00
Daniel Zou ffebe231c0
netty-shaded: Rename the directory of netty shaded resources to avoid collisions 2021-09-02 18:12:10 -04:00
zpencer 0838b73674
netty: remove unneeded TransportTracer null checks 2021-09-02 12:01:44 -07:00
ZHANG Dapeng 07747c59a2
xds: Fix WeakReference bug in SharedCallCounterMap (#8466)
Fixes #8397.
#8397 is caused by mistakenly clearing up a map entry right after the entry is recreated after gc. Reproduced in regression test.
2021-09-02 10:25:15 -07:00
ZHANG Dapeng 2faa748797
census: Fix retry stats data race (#8459)
There is data race in `CensusStatsModule. CallAttemptsTracerFactory`:

If client call is cancelled while an active stream on the transport is not committed, then a [noop substream](https://github.com/grpc/grpc-java/blob/v1.40.0/core/src/main/java/io/grpc/internal/RetriableStream.java#L486) will be committed and the active stream will be cancelled. Because the active stream cancellation triggers the stream listener closed() on the _transport_ thread, the closed() method can be invoked concurrently with the call listener onClose(). Therefore, one `CallAttemptsTracerFactory.attemptEnded()` can be called concurrently with `CallAttemptsTracerFactory.callEnded()`, and there could be data race on RETRY_DELAY_PER_CALL. See also the regression test added.

The same data race can happen in hedging case when one of hedges is committed and completes the call, other uncommitted hedges would cancel themselves and trigger their stream listeners closed() on the transport_thread concurrently. 

Fixing the race by recording RETRY_DELAY_PER_CALL once both the conditions are met: 
- callEnded is true 
- number of active streams is 0.
2021-09-02 10:24:22 -07:00
Anuraag Agrawal 522b37bc3b
Fix drift in MessageFramer comment (#8427) 2021-09-02 08:56:56 -07:00
sanjaypujare b0b250024f
xds: fix implementation to comply with gRFC for security (#8468) 2021-09-01 10:49:33 -07:00
Terry Wilson 7cde473efa
core/auth: Remove CallCredentials2 (#8464)
- Removes CallCredentials2
- Removes CallCredentials2ApplyingTest
- Adds two tests from CallCredentials2ApplyingTest to CallCredentialsApplyingTest
- Updates GoogleAuthLibraryCallCredentials to extend from CallCredentials instead of CallCredentials2
2021-09-01 09:49:20 -07:00
Sergii Tkachenko 4fa612ae3d
xds: fix java style 2021-08-31 16:45:37 -07:00
Lidi Zheng 40f70ca3c1
Change to a non-workload-identity GKE cluster (#8461)
Part of grpc/grpc#27189 and b/198291728.

By disabling the workload identity, we should be able to run tests faster and avoid future IAM policy size issue.

Kokoro run: https://fusion2.corp.google.com/invocations/b52b1684-47de-406d-a9f6-644909755f34/targets
2021-08-31 10:35:51 -07:00
Eric Anderson 5cc94a5488
stub: Document StreamObserver is an async API
Missing docs were brought up in #8423
2021-08-30 11:15:05 -07:00
yifeizhuang b3ef588520
Fix Java Style (#8458) 2021-08-27 16:35:23 -07:00
apolcyn 137bdaa868
interop-testing: add soak test cases to test service client 2021-08-27 15:30:43 -07:00
Kurt Alfred Kluever f3337f28ce
stub: Add @InlineMe to deprecated gRPC APIs (#8457)
Read more @ https://errorprone.info/docs/inlineme
2021-08-27 14:11:06 -07:00
yifeizhuang 0f6380b470
xds: server side xDS routing and config application (#8318)
Added routing config discovery for HCM in LdsUpdate in XdsServerWrapper. This can be LDS inline or through RDS. Deal with inflight SslContextProviderSupplier resource handling. Discovered routing config is updated to FilterChainSelectorRef.
Added routing config data field in FilterChainSelector. Filter chain matching would resulting in setting a new attribute key for server routing config. Filter chain matching logics mostly not changed.
Installed ConfigApplyingInterceptor in XdsServerWrapper's delegateBuilder. It fetches server routing config attribute set above. It does routing match and creates server interceptors for the http filters as a result.
2021-08-27 13:30:47 -07:00
Kurt Alfred Kluever 46d47d52d9
Update error-prone to the latest release (2.9.0) (#8456)
required as a prerequisite to using `@InlineMe.`
2021-08-27 11:24:27 -07:00
Terry Wilson df4ac5973c
core: Exit idle mode in enterIdle() if there are pending calls or delayed transport.
This change assures that if there are only calls in real transport the
channel will remain in idle mode. Idle mode will be exited if there
are calls in delayed transport to allow them to be processed.
2021-08-26 14:42:27 -07:00
Alexander Polcyn f1b699bbf1 Update default XDS server name in C2P resolver 2021-08-26 13:57:19 -07:00
ZhenLian 3cb0696b1f
advancedtls: change enum to use UPPER_SNAKE_CASE (#8446) 2021-08-25 16:13:09 -07:00
ZHANG Dapeng 8a5694b7f8
Update README etc to reference 1.40.1 (#8448) 2021-08-25 10:58:42 -07:00
yifeizhuang 48219d902a
fix import warning (#8441) 2021-08-24 16:33:12 -07:00