Commit Graph

829 Commits

Author SHA1 Message Date
yifeizhuang 221ee494d9 move max connection idle manager to core 2022-08-25 12:53:11 -07:00
Larry Safran 01d5bd47cb
Cleanup some of the warnings across the code base (#9445)
No logic changes, just cleans up warnings to make spotting real problems easier.

Remove "public" declarations on interfaces
Remove duplicate semicolons (Java lines ending in ";;")
Remove unneeded import
Change non-javadoc comment to not start with "/**"
Remove unneeded explicit type declarations from generics
Fix broken javadoc links
2022-08-15 11:06:31 -07:00
Eric Anderson 61f19d707a
Swap Animalsniffer to Java 8 and Android 19
Also added missing signatures. Swapping to version catalog will make
this process easier in the future.
2022-08-10 12:41:57 -07:00
Eric Anderson 0e45e04041
Avoid accidental locale-sensitive String.format()
%s is fairly safe (requires a Formattable to use Locale), so %d is the
main risk item. Places that really didn't need to use String.format()
were converted to plain string concatenation. Logging locations were
generally converted to using the log infrastructure's delayed
formatting, which is generally locale-sensitive but we're okay with
that. That wasn't done in okhttp, however, because Android frequently
doesn't use MessageFormat so we'd lose the parameters. Everywhere else
was explicitly defined to be Locale.US, to be consistent independent of
the default system locale.
2022-07-19 14:41:34 -07:00
Eric Anderson 0ff9f37b9e Use Gradle's task configuration avoidance APIs
This can avoid creating an additional 736 tasks (previously 502 out of
1591 were not created). That's not all that important as the build time
is essentially the same, but this lets us see the poor behavior of the
protobuf plugin in our own project and increase our understanding of how
to avoid task creation when developing the plugin. Of the tasks still
being created, protobuf is the highest contributor with 165 tasks,
followed by maven-publish with 76 and appengine with 53. The remaining
59 are from our own build, but indirectly caused by maven-publish.
2022-07-08 12:16:40 -07:00
Eric Anderson e96d04774b okhttp: Add server implementation 2022-07-01 12:38:01 -07:00
Eric Anderson fad38f49f1
Fix Channelz window reporting
Both Netty and OkHttp had local and remote windows flipped. Also, the
comment in OkHttp about "not tracked" wasn't true, so make it more
accurate and provide a _hint_ of what the window may be instead of just
-1.
2022-06-27 12:29:57 -07:00
Terry Wilson 7bd0797496
all: Update netty to 4.1.77.Final and netty_tcnative to 2.0.53.Final (#9027)
all: Update netty to 4.1.77.Final and netty_tcnative to 2.0.53.Final

Also switches to a non-release version of rules_jvm_external to allow Bazel build to work with artifact classifiers.
2022-06-24 10:47:27 -07:00
Kurt Alfred Kluever 1ed4b778e0 Tidy up @CheckReturnValue / @CanIgnoreReturnValue annotations
Includes pushing @CanIgnoreReturnValue annotations down from the
class-level to method-level.
2022-06-17 14:57:42 -07:00
Eric Anderson b06942d63b Use Gradle's version catalog
This moves our depedencies into a plain file that can be read and
updated by tooling. While the current tooling is not particularly better
than just using gradle-versions-plugin, it should put us on better
footing. gradle-versions-plugin is actually pretty nice, but will be
incompatible with Gradle 8, so we need to wait a bit to see what the
future holds.

Left libraries as an alias for libs to reduce the commit size and make
it easier to revert if we don't end up liking this approach.

We're using Gradle 7.3.3 where it was an incubating fetaure. But in
Gradle 7.4 is became stable.
2022-06-14 14:04:10 -07:00
Eric Anderson 5b825e8b17 Remove classifiers from ext.libraries
Classifier can't be specified in version catalog.
2022-06-14 14:04:10 -07:00
Eric Anderson ba57a1d893 api: Add connection management APIs to ServerBuilder
These APIs were added to NettyServerBuilder for gRFC A8 and A9. They are
important enough that they shouldn't require using the perma-unstable
transport API to access. This change also allows using these methods
with grpc-netty-shaded.

Fixes #8991
2022-05-16 13:02:09 -07:00
Eric Anderson a8bd0b8c92 netty: Simplify and increase compat of UdsNettyChannelProvider
The transport should be usable with non-`unix:` name resolvers. As long
as the name resolver returns the correct socket address type, things
should work fine.
2022-05-09 08:37:31 -07:00
sanjaypujare 41c027c11b
netty: implement UdsNameResolver and UdsNettyChannelProvider (#9113)
* netty: implement UdsNameResolver and UdsNettyChannelProvider
When the scheme is "unix:" we get the UdsNettyChannelProvider to
create a NettyChannelBuilder with DomainSocketAddress type and
other related params needed for UDS sockets
2022-05-02 16:41:50 -07:00
sanjaypujare 538db03d56
api: add support for SocketAddress types in ManagedChannelProvider (#9076)
* api: add support for SocketAddress types in ManagedChannelProvider
also add support for SocketAddress types in NameResolverProvider
Use scheme in target URI to select a NameRseolverProvider and get
that provider's supported SocketAddress types.
implement selection in ManagedChannelRegistry of appropriate
ManagedChannelProvider based on NameResolver's SocketAddress types
2022-04-22 09:10:55 -07:00
Eric Anderson 4a0fe99f8a
netty: Support pseudo headers in all GrpcHttp2RequestHeaders methods
The previous code assumed that only gRPC would be using these methods.
But twice now Netty has made a change (generally relating to security)
that used a method for pseudo headers that previously wasn't supported.
Let's stop the whack-a-mole and just implement them all.

This restores compatibility with Netty 4.1.75.Final. Fixes #8981
2022-03-22 07:39:48 -07:00
nafeabd b35506b144
netty: Include both x86 and Arm64 epoll as dependency for grpc-netty-shaded 2022-02-09 13:21:36 -08:00
ZHANG Dapeng 431fb0255f
core/netty: infinite local-only transparent retry for netty (#8878)
In core, add a new enum element to `RpcProgress` for the case that the stream is closed even before anything leaves the client. `RetriableStream` will do unlimited transparent retry for this type of `RpcProgress` since they are local-only.

In netty, call `tranportReportStatus()` for pending streams on failure.

Also fixes #8394
2022-02-04 22:12:03 -08:00
sanjaypujare a661515421
observability: revert previous visibility changes made in the provider APIs (#8889)
change visibility back to protected for certain methods of Providers
2022-02-04 08:38:46 -08:00
sanjaypujare bd156f98d6
observability: implement and integrate LoggingServerProvider into Observability (#8879) 2022-02-01 10:05:59 -08:00
ZHANG Dapeng d1e0be6919
all: fix various gradle build warnings 2022-01-18 10:18:16 -08:00
Kurt Alfred Kluever d5f22b77b2 netty: Assign the result of a @CheckReturnValue'ed constructor to an unused variable
This fixes a soon-to-be compile error via ErrorProne.

Alternatively, we could use assertThrows() instead of
@Test(expected = ...), but grpc doesn't yet require Java 8.
2022-01-18 09:27:25 -08:00
Eric Anderson 58a7ace6ac
Bump ErrorProne to 2.10.0
Previous versions of error prone were incompatible with Java 17 javac.

In grpc-api, errorprone is now api dependency because it is on a public
API.  I was happy to see that Gradle failed the build without the dep
change, although the error message wasn't super clear as to the cause.

It seems that previously -PerrorProne=false did nothing. I'm guessing
this is due to a behavior change of Gradle at some point. Swapping to
using the project does build without errorProne, although the build
fails with Javac complaining certain classes are unavailable. It's
unclear why. It doesn't seem to be caused by the error-prone plugin.
I've left it failing as a pre-existing issue.

ClientCalls/ServerCalls had Deprecated removed from some methods because
they were only deprecated in the internal class, not the API. And with
Deprecated, InlineMeSuggester complained.

I'm finding InlineMeSuggester to be overzealous, complaining about
package-private methods. In time we may figure out how to use it better,
or we may request changes to the checker in error-prone.
2022-01-12 12:06:27 -08:00
sanjaypujare d7f951a9d8
all: update netty to 4.1.72.Final and tcnative to 2.0.46.Final (#8780) 2022-01-07 15:34:59 -08:00
Eric Anderson d44de5069d
Bump to Gradle 6.9 and update plugins
These changes make the build compatible with Gradle 7, except for
Android which requires plugin updates.

I removed animalsniffer from binder because it did nothing (as there
were no signatures) and it was failing after setting toolVersion. It
failed because animalsniffer is only compatible with java plugin. After
this change I put the withId(animalsniffer) loading inside the
withId(java) to avoid a plugin ordering failure. That made it safe again
for binder to load animalsniffer, but it is still best to remove the
plugin from binder as it is misleading.

I did not upgrade Android plugin versions as newer versions (even 3.6)
require dealing with androidx (#8421).
2022-01-07 09:54:50 -08:00
Eric Anderson b6eafbe695 netty: Add system property to disable Connection header check
A user has a proxy that is sending "Connection: close", which is against
the HTTP/2 spec, but will take time to fix.

Fixes #8674
2021-11-09 09:26:18 -08:00
Benjamin Peterson 1fe62dd417
Fix sentences with a missing "be". (#8613) 2021-10-20 15:14:10 -07:00
Sergii Tkachenko 0376de15b8
Fix AbstractManagedChannelImplBuilder#maxInboundMessageSize(int) ABI (#8607)
In refactoring described in #7211, the implementation of #maxInboundMessageSize(int)
(and its corresponding field) were pulled down from internal AbstractManagedChannelImplBuilder
to concrete classes that actually enforce this setting. For the same reason, it wasn't ported
to ManagedChannelImplBuilder (the #delegate()).

Then AbstractManagedChannelImplBuilder was brought back to fix ABI backward compatibility,
and temporarily turned into a ForwardingChannelBuilder, ref PR #7564. Eventually it will
be deleted, after a period with "bridge" ABI solution introduced in #7834.

However, restoring AbstractManagedChannelImplBuilder unintentionally made ABI of
pre-refactoring builds expect it to be a method of AbstractManagedChannelImplBuilder,
and not concrete classes, ref #8313.

The end goal is to keep #maxInboundMessageSize(int) only in concrete classes that enforce it.
To fix method's ABI, we temporary reintroduce it to the original layer it was removed from:
AbstractManagedChannelImplBuilder. This class' only intention is to provide short-term
ABI compatibility. Once we move forward with dropping the ABI, both fixes are no longer
necessary, and both will perish with removing AbstractManagedChannelImplBuilder.
2021-10-14 17:25:06 -07:00
ZHANG Dapeng 28f2647aaf
core: move closed check from Stream.isReady() to Call.isReady() (#8566)
This fixes data race described in #8565.

We are doubtful whether checking closed in isReady() is necessary (#3201 might be a requirement), but it was easier to just maintain the existing behavior than think heavily about it.
2021-09-29 09:42:59 -07:00
Eric Anderson 192688f1f2 netty: Requests with Connection header are malformed
Although this is part of HTTP/2 and should have already been handled
already, it was noticed as part of RBAC work to avoid matching
hop-by-hop headers. See gRFC A41.

Also add a warning if creating Metadata.Key for "Connection". Use this
to try to help diagnose a client if it happens to blindly copy headers
from HTTP/1, as PROTOCOL_ERROR is hard to debug.

This rolls-forward 6e89919 after it was reverted in 7669656, now that
the test proxy has been fixed.
2021-09-27 12:59:15 -07:00
ZhenLian 838438cedb
AdvancedTls: add functions to load credentials from static files (#8525)
* AdvancedTls: add functions to load credentials from static files
2021-09-17 09:45:41 -07:00
Eric Anderson 7669656725 Revert "netty: Requests with Connection header are malformed"
This reverts commit 6e89919e32.

This was found to break a test proxy. We'll work on fixing the proxy and
then roll this forward again.
2021-09-15 15:20:40 -07:00
Eric Anderson 5307b69c9e netty: Allow protocol negotiators to shut down transport, with grace period
This will be used for draining old connections when xDS configuration
changes.
2021-09-15 10:08:28 -07:00
Eric Anderson 122b3b2f7c
netty: Support Host header on server-side
We want to know the single, unambiguous authority for the request. If
there is no authority, we use host instead. While authority would be
most typical for HTTP/2, requests proxied from HTTP/1 may use host
instead of authority.

This is generally useful, but the impetus is RBAC. See gRFC A41.
2021-09-15 09:40:56 -07:00
Eric Anderson 6e89919e32
netty: Requests with Connection header are malformed
Although this is part of HTTP/2 and should have already been handled
already, it was noticed as part of RBAC work to avoid matching
hop-by-hop headers. See gRFC A41.

Also add a warning if creating Metadata.Key for "Connection". Use this
to try to help diagnose a client if it happens to blindly copy headers
from HTTP/1, as PROTOCOL_ERROR is hard to debug.
2021-09-13 11:30:19 -07:00
ZHANG Dapeng 7c6f53ab79
all: add internal API to disable retry stats (#8510)
Resolves b/197648853 for internal performance regression. Reporting retry stats caused significant amount of performance overhead internally.
2021-09-13 09:12:04 -07:00
ZhenLian fb00463001
fix a flaky test in advanced TLS (#8474)
* fix a flaky test in advanced tls
2021-09-08 11:43:23 -07:00
Daniel Zou ffebe231c0
netty-shaded: Rename the directory of netty shaded resources to avoid collisions 2021-09-02 18:12:10 -04:00
zpencer 0838b73674
netty: remove unneeded TransportTracer null checks 2021-09-02 12:01:44 -07:00
ZhenLian 3cb0696b1f
advancedtls: change enum to use UPPER_SNAKE_CASE (#8446) 2021-08-25 16:13:09 -07:00
yifeizhuang 48219d902a
fix import warning (#8441) 2021-08-24 16:33:12 -07:00
Eric Anderson 8026ccde4b netty: Don't use old-style classpath for shadow plugin
Seems it was introduced unnecessarily in dc74a31b. This also removes the
jcenter reference which is a repository that no longer receives updates.
2021-08-18 10:04:21 -07:00
ZhenLian 2c2ebaebd5
advancedtls: adding AdvancedTlsX509TrustManager and AdvancedTlsX509KeyManager (#8175)
* add advanced TLS classes and tests
2021-08-17 16:13:30 -07:00
Nick Ufer 9dd0c66929 netty: removes TODO in test for NettyServer 2021-08-05 11:27:27 -07:00
ZHANG Dapeng 860e97d12a
all: API refactoring in preparation to support retry stats (#8355)
Rebased PR #8343 into the first commit of this PR, then (the 2nd commit) reverted the part for metric recording of retry attempts. The PR as a whole is mechanical refactoring. No behavior change (except that some of the old code path when tracer is created is moved into the new method `streamCreated()`).

The API change is documented in go/grpc-stats-api-change-for-retry-java
2021-07-31 18:33:02 -07:00
Eric Anderson b2764595e6 netty: Refine workaround for Netty header processing for transparent retries
Nginx and C core don't do graceful GOAWAY and retries have matured such
that transparent retries may soon be on by default. Refining the
workaround thus can reduces error rate for users.

Fixes #8310
2021-07-30 16:42:21 -07:00
Eric Anderson a282019902
netty: Use charset=utf-8, not encoding=utf-8 for error pages
This has been wrong since the introduction of the code in df357cb8.
Noticed as part of https://github.com/grpc/grpc-go/pull/4491 . The error
text is generally ASCII, so this probably doesn't matter much.
2021-07-19 13:32:11 -07:00
ZHANG Dapeng 4f09073e0f
all: remove 2-arg ClientStreamListener.closed()
We used to have two ClientStreamListener.closed() methods. One is simply calling the other with default arg. This doubles debugging (e.g. #7921) and sometimes unit testing work. Deleting the 2-arg method to cleanup.

This PR is purely refactoring.
2021-06-29 10:27:03 -07:00
Eric Anderson 2258d2e3e9 netty-shaded: Use compile instead of runtime dep in pom for core
This is essentially re-adding #5901 after its behavior was "reverted"
in #8242. At the time grpc-core was the only dependency, so this change
only applies to grpc-core to make it a tiny bit easier to swap to
runtime scope in the future.
2021-06-18 07:48:04 -07:00
Daniel Zou dc74a31be1
Modify netty-shaded resources to reference shaded class names (#8258) 2021-06-16 07:56:48 -07:00