* okhttp: add verbose logging for OkHttp HTTP/2 frames to include brief frame content.
* fix small details and typo issues
* fix branch condition (it's a typo) for logging the whole buffer
* remove redudent log level checking, always not log entire buffer
* convert ByteString to String to avoid printing 'hex=' prefix
* add test for OkHttpFrameLogger for inbound http frames
* refactor OkHttpFrameLogger's help method
* add Constructor for testing and missing logHeaders
* add http/2 framelogger test for outbound side
* use ArgumentMatchers instead of Matchers
* replace assertTrue with Truth.assertThat
* assert buffer log with exact length match instead of <=
This is needed for GRPCLB pick_first support, which needs to attach
tokens to headers, and the tokens are per server. In pick_first, all
addresses are in a single Subchannel, thus the LoadBalancer needs to
know which backend is used for a new stream.
This will be a new override. The old override is now deprecated.
In order to pass new information without adding new overrides, I shoved most information
into an object called StreamInfo. The Metadata is left out to draw attention because
it's mutable.
Motivation: this is needed for correctly supporting pick_first in GRPCLB. GRPCLB needs to
add a token to the headers, and the token varies among servers. With round_robin, GRPCLB
create a Subchannel for each server, thus can attach the token when the Subchannel is picked.
To implement pick_first, all server addresses will be put in a single Subchannel, we will
need to add the header in newClientStreamTracer(), by looking up the server address from
the transport attributes and deciding which token to add.
Context: [#4159 (comment)](https://github.com/grpc/grpc-java/issues/4159#issuecomment-415827641)
`Attributes` is appropriate for plumbing optional objects, especially useful for a long plumbing path where components in the middle may not care or see all objects in the container. It's not the case for the `params` on `newNewResolver()`. Both the default port and the proxy detector are guaranteed to be there and the plumbing path is very short. In this case, a first-class object is more appropriate and easier to use.
The `Helper` will also have `getSynchronizationContext()` (#2649) and a method to parse and validate service config. We we also considering merging `Listener` into the `Helper`, to make `NameResolver` match the `LoadBalancer` API.
Combine the public `ProxyParameters` and the internal `ProxySocketAddress` as `HttpConnectProxiedSocketAddress`. The more specific name signifies the type of the proxy we currently support, and makes room for other proxy types (e.g., SOCKS) in the future. The combination simplifies NameResolver implementation.
Introduce `ProxiedSocketAddress` as the base class that is returned by `ProxyDetector`, mainly for clarification and documentation. Added documentation about proxy in general on `ProxyDetector`.
This will reduce the method count, as okhttp's copy of ConnectionSpec will no
longer be retained when ProGuarded. It was deprecated in 25f357699, 10 months
ago.
We've been on newer versions of Guava for a while now; these no longer
do anything.
Reworded the comment for Stopwatch.createUnstarted(), because it is not
safe (it doesn't matter if the method isn't marked Beta; you have to use
Ticker), except for the fact it is only used in our tests.
Optimize OkHttp transport's memory use by getting rid of queuing writes in
AsyncFrameWriter. If any write is pending due to connection issue or by flow
control, AsyncFrameWriter can use at least 8K per each task (task includes
buffer) even if the actual payload is very small. To merge pending writes,
Async mechanism is moved from AsyncFrameWriter to AsyncSink (AsyncSink
is used by okio's FrameWriter). AsyncSink is still relying on okio's buffer to
decide merging writes or not.
Resolves#4860
Following the [spec](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md) on duplicate header names:
**Custom-Metadata** header order is not guaranteed to be preserved except for values with duplicate header names. Duplicate header names may have their values joined with "," as the delimiter and be considered semantically equivalent. Implementations must split Binary-Headers on "," before decoding the Base64-encoded values.
This will allow enabling Error Prone on JDK 10+ (after
updating the net.ltgt.errorprone plugin), and is also a
prerequisite to that plugin update.
Also remove net.ltgt.apt plugin, as Gradle has native
support for annotationProcessor.
This is a rename of the pre-existing Netty builder method, so aliases
were added to the Netty builders.
Fixes#4050. This API was a minor rename to the pre-existing Netty API,
so has already undergone API review and thus is not ExperimentalApi.
Returns a Channel that allows a LoadBalancer to make auxiliary RPCs on already-established application connections. We need this to implement client-side health-checking (#4932)
See comments on the API for its semantics.
Notable changes:
- Transports are modified to use InUseStateAggregator so that they can exclude RPCs made on Subchannel.asChannel() when reporting in-use state for idle mode.
- OobChannel shares the same Executor as Subchannel.asChannel(). Because the latter is not a ManagedChannel and doesn't have life-cycle, thus can't determine when to return the Executor to a pool, the Executor is now returned only when ManagedChannelImpl is terminated.
This is the first step of smoothly changing the CallCredentials API.
Security level and authority are parameters required to be passed to
applyRequestMetadata(). This change wraps them, along with
MethodDescriptor and the transport attributes to RequestInfo, which is
more clear to the implementers.
ATTR_SECURITY_LEVEL is moved to the internal GrpcAttributes and
annotated as TransportAttr, because transports are required to set it,
but no user is actually reading them from
{Client,Server}Call.getAttributes().
ATTR_AUTHORITY is removed, because no transport is overriding it.
All involved interfaces are changed to abstract classes, as this will
make further API changes smoother.
The CallCredentials name is stabilized, thus we first introduce
CallCredentials2, ask CallCredentials implementations to migrate to
it, while GRPC accepting both at the same time, then replace
CallCredentials with CallCredentials2.
It appears to be stable now. Ran for 1000s of times. I do see some
general flakiness in TransportTest, but it applies to the tests in
general and isn't specific to this one test. It is:
```
org.mockito.exceptions.verification.WantedButNotInvoked:
Wanted but not invoked:
listener.transportReady();
-> at io.grpc.internal.testing.AbstractTransportTest.startTransport(AbstractTransportTest.java:1815)
Actually, there were zero interactions with this mock.
```
This flake is not seen often because it occurs less frequently when
running all the tests (~.1% vs 1%). One of the early tests must warm
something up to make it less likely.
This is an API used to coordinate across packages and must live in
`io.grpc`.
Prepending `Internal` makes it easier to detect and hide this class
from public visibility when using certain build tools.
fixes#4796
Most of the changes are changing the signature of newClientTransport.
Since this is annoying, I choose to introduce a ClientTransportOptions
object to avoid the churn in the future.
With ClientTransportOptions in place, there's only a few lines necessary
of plumbing for the Attributes: add the field to ClientTransportOptions
and populate it in InternalSubchannel. There are no consumers of the
field in this commit.
A new RPC starts with the following steps:
1. Pick a READY transport
2. the READY transport calls `transport.newStream()`
3. the new stream calls `stream.start()`
4. `stream.start()` invokes or enqueus `writeHeaders()` (or for GET request, noop)
A racy GOAWAY could happen between 3 and 4, and by the retry spec, the RPC should be transparent-retry-able in this case. For Netty and OkHttp transport implementation, before step 4, (even if step 1, 2, and 3 excluding 4 are made atomic,) the http2-stream for the RPC is not created, so the current transparent retry logic does not apply and need fix.
Of course, if step 1, 2, and 3 including 4 are made atomic, and not with GET, there will be no such problem.
This PR adds an automatic gradle format checker and reformats all the *.gradle files. After this, new changes to *.gradle files will fail to build if not in good format, just like checkStyle failure.
This allows ProGuard to remove OkHttp's ConnectionSpec in most cases,
saving about 40 methods. The savings won't be realized until
DEFAULT_CONNECTION_SPEC is removed.
Always set the remote address, no reason why this should be a TLS-only
feature. This is needed for channelz, and is especially useful in unit
tests where we are using plaintext.
This PR adds the attr for plaintext.
Changes:
- `ClientStreamListener.onClose(Status status, RpcProgress rpcProgress, Metadata trailers)` added.
- `AbstractClientStream.transportReportStatus(Status status, RpcProgress rpcProgress, boolean stopDelivery, Metadata trailers)` added
- `ClientCallImpl.ClientStreamListenerImpl` will ignore the arg `rpcProgress` (non retry)
- `RetriableStream.SubListener` will handle `rpcProgress` and decide if transparent retry.
- `NettyClientHandler` and `OkHttpClientTransport` will pass `RpcProgress.REFUSED` to client stream listener for later stream ids when received GOAWAY, or for stream received a RST_STREAM frame with REFUSED code.
- All other files are just a result of refactoring.
ProxyDetector is now responsible for resolving the proxy's
`InetSocketAddress`, and `ProxyParameters` asserts that the address is
resolved. The results are plumbed through using a `PairSocketAddress`,
which is a special `SocketAddress`.
If a proxy should be used but the proxy can not be resolved, we the
`DnsNameResolver` will re-attempt the resolution later.
Remove the unit test testing for unresolved proxy addresses, since
it's no longer applicable.
Transport ststistics should really be a child member of SocketStats.
While we're at it, let's add the local and remote SocketAddress to
SocketStats, with a test.
The channelz service must not live in io.grpc.internal, and channelz
needs to be able to get the identifier of the entities it
tracks. Since io.grpc can not refer to io.grpc.internal, the LogId
must be moved out of internal.
The method name passed to MethodDescriptor does not include the leading
'/'. If it does, on the wire it will actually cause two slashes. This
has been this way for a _long_ time, but in tests that ignore the method
name or use the same MethodDescriptor no client and server the extra /
"works fine." But it's misleading, so let's remove it.
This diff does not actually change any behaviors yet, that will come
in the next diff along with unit tests for those new behaviors. This
diff's goal is only to change the method signatures so future diffs
are cleaner.
Counters are bumped when a message is completely written. If a
part of a message is still buffered and not yet flushed, we will
not increment the stats.
Our Travis-CI builds are failing with "Protocol family unavailable" due
to the usage of ::1. Although it's 2017 and we'd expect to have ipv6
_loopback_ anywhere that mattered, apparently that's not the case.
The tests now work equally well on IPv4-only and IPv6-only machines.
Jihun Cho
Chengyuan Zhang
Tim van der Lippe
Carl Mastrangelo
Kun Zhang
Eric Anderson
Eric Gribkoff
Ignacio del Valle Alles
ZHANG Dapeng
Rodrigo Queiro
Thomas Broyer
Grant Oakley
zpencer
David Brodsky
kenji yoshida
ramaraochavali