Commit Graph

588 Commits

Author SHA1 Message Date
Arnout Engelen 3dab7aed2f netty: Client-side support for h2c via Upgrade
Fixes #4518
2018-11-30 18:10:27 -06:00
Kun Zhang b5acbedd55
core: record real-time metrics to OpenCensus (updated to 1.18.0) (#5099)
Real-time metrics are total sent/received bytes and messages per
method, and are updated as the events occur rather than at the end of
RPCs.
2018-11-29 16:30:58 -08:00
Nicholas DiPiazza 7c05127cbc netty: Add to "An established connection was aborted by the software in your host machine" QUIET_ERRORS 2018-11-13 14:13:36 -08:00
Rodrigo Queiro 8481943866 Add missing j2objc dependency to Bazel build
This avoids a warning when building artifacts that depend on Guava.

Fixes #5046.
2018-11-13 13:39:35 -08:00
Jihun Cho b78036daaa
netty: finalize maxMessageSize deprecation in NettyChannelBuilder. (#5054) 2018-11-13 10:59:15 -08:00
ZHANG Dapeng bff008fbc8
core: Emit bin-headers with unpadded encoding
Following the [grpc PROTOCOL-HTTP2 spec](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md)
"Note that HTTP2 does not allow arbitrary octet sequences for header values so binary header values must be encoded using Base64 as per https://tools.ietf.org/html/rfc4648#section-4. Implementations MUST accept padded and un-padded values and should emit un-padded values. "
2018-11-09 13:39:01 -08:00
ZHANG Dapeng 85b244bb41
core,netty,testing: Support dup headers joined with commas
Following the [spec](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md) on duplicate header names:

**Custom-Metadata** header order is not guaranteed to be preserved except for values with duplicate header names. Duplicate header names may have their values joined with "," as the delimiter and be considered semantically equivalent. Implementations must split Binary-Headers on "," before decoding the Base64-encoded values.
2018-11-01 16:17:05 -07:00
Thomas Broyer 183e1f6735 all: update Error Prone to 2.3.2
This will allow enabling Error Prone on JDK 10+ (after
updating the net.ltgt.errorprone plugin), and is also a
prerequisite to that plugin update.

Also remove net.ltgt.apt plugin, as Gradle has native
support for annotationProcessor.
2018-10-19 13:08:36 -07:00
Eric Anderson b7c3d276c9 core: Add maxInboundMetadataSize to builders
This is a rename of the pre-existing Netty builder method, so aliases
were added to the Netty builders.

Fixes #4050. This API was a minor rename to the pre-existing Netty API,
so has already undergone API review and thus is not ExperimentalApi.
2018-10-18 14:34:03 -07:00
Eric Anderson 0fbc1153bd testing: Add transport tests for maxInboundMetadataSize 2018-10-18 14:34:03 -07:00
Carl Mastrangelo 52d6a6e680
netty: fix local socket bind 2018-10-17 18:34:03 -07:00
Eric Anderson 959323be97
Add build support for Java 11
It appears everything was already working on Java 11, except
build-specific and testing issues. Updating to Netty 4.1.30 (#4940)
probably fixed the last true Java 11 incompatibility.

Fixes #4933
2018-10-17 11:49:30 -07:00
Kun Zhang c528df8ae8
core: add internal Subchannel.asChannel() (#4950)
Returns a Channel that allows a LoadBalancer to make auxiliary RPCs on already-established application connections. We need this to implement client-side health-checking (#4932)

See comments on the API for its semantics.

Notable changes:

- Transports are modified to use InUseStateAggregator so that they can exclude RPCs made on Subchannel.asChannel() when reporting in-use state for idle mode.
- OobChannel shares the same Executor as Subchannel.asChannel(). Because the latter is not a ManagedChannel and doesn't have life-cycle, thus can't determine when to return the Executor to a pool, the Executor is now returned only when ManagedChannelImpl is terminated.
2018-10-15 15:39:21 -07:00
Jihun Cho 1d52d8e62a netty: suppress linter warning about Ignoring returned Futures 2018-10-12 16:18:17 -07:00
Eric Anderson 4c5dabf705 netty: Don't return null ServerTransportListener in tests
Null is never allowed, so will cause exceptions and not behave properly.
2018-10-11 17:17:42 -07:00
Kun Zhang 861f9147ed
core: add CallCredentials2 and deprecate CallCredentials' old interface (#4902)
This is the first step of smoothly changing the CallCredentials API.

Security level and authority are parameters required to be passed to
applyRequestMetadata(). This change wraps them, along with
MethodDescriptor and the transport attributes to RequestInfo, which is
more clear to the implementers.

ATTR_SECURITY_LEVEL is moved to the internal GrpcAttributes and
annotated as TransportAttr, because transports are required to set it,
but no user is actually reading them from
{Client,Server}Call.getAttributes().

ATTR_AUTHORITY is removed, because no transport is overriding it.

All involved interfaces are changed to abstract classes, as this will
make further API changes smoother.

The CallCredentials name is stabilized, thus we first introduce
CallCredentials2, ask CallCredentials implementations to migrate to
it, while GRPC accepting both at the same time, then replace
CallCredentials with CallCredentials2.
2018-10-10 21:45:56 -07:00
Carl Mastrangelo fc908e2dcc
netty: expose setting a local socket address 2018-10-08 14:14:09 -07:00
Kun Zhang fbfc3a40d0
core: add Grpc.TRANSPORT_ATTR_LOCAL_ADDR (#4906)
Resolves #4135
2018-10-03 16:43:37 -07:00
Eric Anderson 99a2cac07a netty: Add ProtocolNegotiator.close
This notifies the negotiator when it will no longer be used, allowing it
to clean up any resources.
2018-10-03 08:48:35 -07:00
Eric Anderson 09737fea7a netty: Remove TransportCreationParamsFilterFactory
Now there is a clear lifetime of ProtocolNegotiator.
2018-10-03 08:48:35 -07:00
Eric Anderson acf80d63b0
netty: Allow specifying ProtocolNegotatiorFactory directly to Channels
This will be the replacement for TransportCreationParamsFilterFactory
and matches somewhat what used to be done and what is done on
server-side.
2018-09-24 16:09:36 -07:00
Carl Mastrangelo b0f423295b
all: use Java7 brackets 2018-09-14 13:52:29 -07:00
Eric Anderson 7efbc36c72 netty: Initialize ProtocolNegotiators eagerly
This simplifies the construction paradigm and leads to the eventual
removal of TransportCreationParamsFilterFactory. The eventual end goal
is to be able to shut down ProtocolNegotiators as is necessary for ALTS.

The only reason the initialization was delayed was for 'authority', so
we now plumb the authority through GrpcHttp2ConnectionHandler.
2018-09-12 08:16:54 -07:00
ZHANG Dapeng c8975e987b
all: fix lint warnings in import 2018-09-11 16:24:44 -07:00
Eric Anderson 98630f1870 netty,okhttp: Enable TransportTest.flowControlPushBack
It appears to be stable now. Ran for 1000s of times. I do see some
general flakiness in TransportTest, but it applies to the tests in
general and isn't specific to this one test. It is:

```
org.mockito.exceptions.verification.WantedButNotInvoked:
Wanted but not invoked:
listener.transportReady();
-> at io.grpc.internal.testing.AbstractTransportTest.startTransport(AbstractTransportTest.java:1815)
Actually, there were zero interactions with this mock.
```

This flake is not seen often because it occurs less frequently when
running all the tests (~.1% vs 1%). One of the early tests must warm
something up to make it less likely.
2018-09-10 17:45:20 -07:00
zpencer 2fca42feb9
all: prepend internal classes with Internal (#4826)
This is a safer way to hide the classes, because they will not appear
in public targets for some build configurations.
2018-09-05 18:48:42 -07:00
zpencer 4d366ce978
all: move Channelz to io.grpc as InternalChannelz (#4797)
This is an API used to coordinate across packages and must live in
`io.grpc`.

Prepending `Internal` makes it easier to detect and hide this class
from public visibility when using certain build tools.

fixes #4796
2018-09-04 16:52:01 -07:00
Eric Anderson 3792242ae1
alts: Use grpc-netty-shaded instead of grpc-netty
There's no good way to provide users of ALTS a choice between grpc-netty
and grpc-netty-shaded. Since Netty is not exposed through the ALTS API
surface, we opt for the shaded version as it has fewer deployment
issues. However, this also means that we _can't_ expose any Netty API,
like EventLoopGroup.
2018-08-15 17:39:20 -07:00
Kun Zhang 5d4fe28260
netty/internal: add InternalNettyChannelBuilder.buildTransportFactory(). (#4757)
This is needed internally for building netty transports outside of a channel.
2018-08-14 13:04:33 -07:00
Spencer Fang dccae9de33 netty: fix unused variable lint 2018-08-09 13:31:23 -07:00
Eric Anderson ed4b5f30f4 netty: Remove option to pass promise to WriteQueue
Passing a promise to WriteQueue was only misused to add a listener on
the promise before issuing the write. Although in this case the listener
ordering will be "random" because listeners are being added from two
different threads, in general we always want to add a listener after the
write returns to let any lower-level listeners be registered first.

Future work can resolve the "random" listener order by passing the
listener to the WriteQueue and adding the listener from the event loop.
2018-07-30 11:34:58 -07:00
Eric Anderson d66ba24272 Upgrade Shadow and Animalsniffer plugins
This avoids warnings of Gradle 5 deprecation.
2018-07-26 13:43:50 -07:00
Eric Anderson 32d50eb3b3 netty: Fix regression in Java 9 ALPN support
Enable testing on Java 9+ in TlsTest, to prevent future regressions.

Fixes #4620
2018-07-10 09:14:49 -07:00
Eric Anderson b92ea68647 netty: Propagate EAG attributes to ProtocolNegotiator
This lets the NameResolver/LB coordinate with the negotiator, like is
necessary with ALTS on GCP.
2018-07-09 13:00:17 -07:00
Eric Anderson 9d6241eedc Propagate EquivalentAddressGroup attributes to transports
Most of the changes are changing the signature of newClientTransport.
Since this is annoying, I choose to introduce a ClientTransportOptions
object to avoid the churn in the future.

With ClientTransportOptions in place, there's only a few lines necessary
of plumbing for the Attributes: add the field to ClientTransportOptions
and populate it in InternalSubchannel. There are no consumers of the
field in this commit.
2018-07-09 13:00:17 -07:00
Eric Anderson dd57b667cc
Fix unused variables
Unused variables in tests were deleted. The unused variable in Netty
was a future that needed completing; that was a bug.
2018-07-04 07:32:57 -07:00
ZHANG Dapeng b244988572
netty,okhttp: make rpc followed by racy GOAWAY transparent-retry-able
A new RPC starts with the following steps:

1. Pick a READY transport
2. the READY transport calls `transport.newStream()`
3. the new stream calls `stream.start()`
4. `stream.start()` invokes or enqueus `writeHeaders()` (or for GET request, noop)

A racy GOAWAY could happen between 3 and 4, and by the retry spec, the RPC should be transparent-retry-able in this case. For Netty and OkHttp transport implementation, before step 4, (even if step 1, 2, and 3 excluding 4 are made atomic,) the http2-stream for the RPC is not created, so the current transparent retry logic does not apply and need fix.

Of course, if step 1, 2, and 3 including 4 are made atomic, and not with GET, there will be no such problem.
2018-06-28 12:52:17 -07:00
ZHANG Dapeng 5ce10f0146
all: add gradle format checker
This PR adds an automatic gradle format checker and reformats all the *.gradle files. After this, new changes to *.gradle files will fail to build if not in good format, just like checkStyle failure.
2018-06-11 18:35:18 -07:00
Eric Anderson e41e054776 Propagate CallCredentials.ATTR_SECURITY_LEVEL from transports
Previously no transport provided the key so CallCredentials would always
see the security as NONE.
2018-05-24 09:51:45 -07:00
Kun Zhang 68a462e447
noop: resolve lint warnings found at import. (#4496) 2018-05-23 11:58:19 -07:00
ZHANG Dapeng f5f560ad36
all: TimeProvider to use nanos rather than millis
This is the same practice as #2833
2018-05-21 12:44:06 -07:00
Eric Anderson 2d75686a88 netty: Fix Javadoc reference to Channelz.Security
This fixes the warning:
`Tag @link: reference not found: Channelz.Security`

Javadoc `@link` is simplistic in its processing of '.' and thinks if a
dot exists it means it is part of the package name. You're forced to use
the full name of nested classes.
2018-05-17 10:32:15 -07:00
ZHANG Dapeng bf4a00c6de
context/core/netty: Add @CheckReturnValue to Context
By adding inner class annotations without introducing external dependencies.
2018-05-15 13:14:30 -07:00
zpencer 247a76ed93
core,netty,okhttp: make toString more consistent for channelz (#4434)
Use MoreObjects.toStringHelper and use only the log id's long value,
because the class name is already present in the toStringHelper.
2018-05-07 11:38:16 -07:00
Carl Mastrangelo 60a0b0c471
all: normalize copyright header 2018-05-03 14:55:21 -07:00
zpencer 9ada30b25d
(low priority) core,netty,interop-testing: stabilize maxInboundMessageSize API (#4399)
On server side, `maxMessageSize` is deprecated for
`maxInboundMessageSize` to match the channel builder.

Update usages to use new setter.
2018-04-27 16:01:16 -07:00
zpencer 1a2d076aed
core,netty,okhttp,services,testing: expose security info to channelz (#4300)
Pull the TLS info from the SSLSession object for TLS, and AltsContext for ALTS.
2018-04-25 14:38:09 -07:00
zpencer c50a57cd52
netty: fix visibility issues with InternalNettySocketSupport (#4335)
Previous version is not actually extendable from other packages.
2018-04-13 14:10:28 -07:00
ZHANG Dapeng a6811d5bb1
all: add toString() to Server to improve debug info 2018-04-13 09:41:09 -07:00
zpencer 7c46bd9399
netty: prepare NettySocketSupport for internal import (#4311)
Delegate the actual heavy lifting to a helper class that can be easily swapped
at runtime.
2018-04-11 16:37:13 -07:00