Commit Graph

1195 Commits

Author SHA1 Message Date
ZHANG Dapeng 6b9e090d70
core: rm unnecessary InternalNameResolverProvider 2018-03-13 10:21:40 -07:00
Eric Anderson 11c8bcde1b Start 1.12.0 development cycle 2018-03-13 09:04:25 -07:00
ZHANG Dapeng 7852b34781
core: fix channel.getState(true) will not reconnect
resolves #4170
2018-03-12 14:13:52 -07:00
ZHANG Dapeng a83f67a706
core,netty,okhttp: Transparent retry
Changes:

- `ClientStreamListener.onClose(Status status, RpcProgress rpcProgress, Metadata trailers)` added.

- `AbstractClientStream.transportReportStatus(Status status, RpcProgress rpcProgress, boolean stopDelivery, Metadata trailers)` added

- `ClientCallImpl.ClientStreamListenerImpl` will ignore the arg `rpcProgress` (non retry)

- `RetriableStream.SubListener` will handle `rpcProgress` and decide if transparent retry.

- `NettyClientHandler` and `OkHttpClientTransport` will pass `RpcProgress.REFUSED` to client stream listener for later stream ids when received GOAWAY, or for stream received a RST_STREAM frame with REFUSED code.

- All other files are just a result of refactoring.
2018-03-12 14:12:46 -07:00
Eric Gribkoff c6fe4deb33
core: set delayedTransport picker to null in idle mode (#4207) 2018-03-12 12:21:39 -07:00
zpencer 402c1740fa
core,okhttp,netty,alts,testing: Plumb proxy resolved addr to transports (#4137)
ProxyDetector is now responsible for resolving the proxy's
`InetSocketAddress`, and `ProxyParameters` asserts that the address is
resolved. The results are plumbed through using a `PairSocketAddress`,
which is a special `SocketAddress`.

If a proxy should be used but the proxy can not be resolved, we the
`DnsNameResolver` will re-attempt the resolution later.

Remove the unit test testing for unresolved proxy addresses, since
it's no longer applicable.
2018-03-12 11:15:40 -07:00
zpencer f56ac76b35
core,services: add ChannelzService class (#4205)
This implements the methods to expose the stats as a gRPC
service. GetServerSockets is still unimplemented and will require a
follow up change.
2018-03-10 14:58:34 -08:00
zpencer 0678499965
core: express children of channels as WithLogId for channelz (#4201)
For channelz ref objects, we need the id as well as the name. By
returning a `WithLogId`, we can return the original object so that we
may call `toString` on it.
2018-03-09 15:49:15 -08:00
zpencer c3c8bc7ab6
core: keep root channels separate from subchannels in channelz (#4200)
It is a channelz requirement that subchannels and channels are
separate, and we can not retrieve a root channel by asking for a
subchannel and vice versa.
2018-03-09 14:44:41 -08:00
zpencer 921b668ad5
core,netty,okhttp,testing: separate local vs remote stream start times for channelz (#4194)
The channelz spec states that the two must be separate.
2018-03-08 16:57:12 -08:00
zpencer f0eb6d2102
core,netty,okhttp,testing: nest TransportStats inside SocketStats for channelz (#4190)
Transport ststistics should really be a child member of SocketStats.
While we're at it, let's add the local and remote SocketAddress to
SocketStats, with a test.
2018-03-08 14:56:04 -08:00
Eric Anderson c59ddc52a1 core: Internal is not necessary for LoadBalancers
With the advent of LoadBalancer2 (now just LoadBalancer) the need to use
internal APIs was removed; there's now first-class APIs exposed for
LoadBalancer use.

The old LoadBalancer API interacted with io.grpc.internal.ClientTransport
directly.
2018-03-08 13:31:31 -08:00
Eric Anderson 19cd32fb66 core: Remove redundancy in JSON parser 2018-03-08 09:20:34 -08:00
Carl Mastrangelo 8b9d179798
core: split JSON parsing into its own file 2018-03-07 17:57:48 -08:00
Eric Gribkoff eef9add54c
core: rename prepareToLoseNetwork() to enterIdle() (#4179) 2018-03-07 15:51:13 -08:00
Eric Gribkoff ae1fb9467c
core: use exponential backoff for name resolution (#4105) 2018-03-05 21:46:02 -08:00
Eric Anderson 26719bdc5c Partially revert "core: change serverimpl,servercallimpl's internalclose to cancel stream (#4038)"
This partially reverts commit 48ca4527c1.
It leaves the changes to ServerCallImpl and test.

This also partially reverts "Lint fixes" commit
3002a23a0f which removed unused variables
which are now necessary again.

This is reverted for the combined result of two issues:
* Some users are testing that they get UNKNOWN when the service throws.
  That's not unreasonable given the behavior was well-publicised when it
  changed in v1.5. We should probably keep the UNKNOWN in some common
  cases (like the service threw immediately, before sending anything).
* The client could see CANCELLED instead of INTERNAL as had been
  intended. It's unclear as to why (I didn't investigate heavily). This
  behavior is visible in MoreInProcessTest and was overlooked during
  review.
2018-03-02 13:35:06 -08:00
Eric Anderson f865b31f5f core: Disable ManagedChannelImpl orphan check
Some internal tests are broken and noticing the warning logs. They need
to be fixed before this is enabled.
2018-03-02 12:29:33 -08:00
Eric Anderson e392af1efd Partially revert "core: Tracking NameResolverProvider being experimental"
This partially reverts commit c07ad68cbd.

The PARAMS_DEFAULT_PORT in NameResorverProvider is the preferred name.
The hope is still to kill NameResolver.Factory.
2018-03-02 12:29:08 -08:00
Eric Gribkoff 3268082f01
core: prefer getHostString() to getHostName() in ProxyDetectorImpl (#4165) 2018-03-02 12:11:54 -08:00
ZHANG Dapeng c07ad68cbd
core: Tracking NameResolverProvider being experimental 2018-03-01 14:20:49 -08:00
zpencer fcc8ea5950
core: load binarylog class using serviceprovider util (#4161)
Use the common util rather than rolling our own here.
2018-03-01 10:15:15 -08:00
Eric Anderson 3002a23a0f Lint fixes 2018-02-28 10:20:08 -08:00
Carl Mastrangelo 7af2373a03
core,netty,okhttp,alts,inprocess: deprecate usePlaintext(boolean) 2018-02-28 08:53:14 -08:00
Eric Anderson 1b5aadf068 core: Disable panic mode
It is now discovering legitimate errors in some tests that need to be
resolved first. Follow progress at #3293.
2018-02-27 14:03:34 -08:00
zpencer 09d305c3fc
core: add server transports to channelz (#4149)
A previous PR added client transports, this PR adds server
transports.
2018-02-27 10:02:44 -08:00
zpencer 4dc1b50e1a
core: add ServerImpl to channelz (#4147)
This allows servers to be identified by ids.
2018-02-27 08:22:25 -08:00
ZHANG Dapeng 39decadaf6
core: allow application to provide all threads - inprocess channel 2018-02-26 17:12:04 -08:00
zpencer 00d1805f78
core: register client transports to channelz (#4146)
This makes client side transports visible to channelz when created,
and removed when terminated.
2018-02-26 16:41:09 -08:00
Eric Anderson ce84c2b227 core: Add missing @RunWith test annotation (#4142)
For ManagedChannelOrphanWrapperTest
2018-02-26 14:06:43 -08:00
zpencer b71b1084de
core: register channels, subchannels, oobchannels with channelz (#4035)
Channels, subchannels and oobchannels are added at creation and
removed at termination. Same goes for subchannels and
oobchannels. Everything is registered as a hard reference. This means
channels that are not properly cleaned up using `awaitTermination`
will remain visible in channelz.
2018-02-26 09:45:19 -08:00
ZHANG Dapeng 4bac7d7084
core: add InProcessServerBuilder.generateName() 2018-02-23 16:44:10 -08:00
ZHANG Dapeng e31851a0eb
core: remove runDunTasks with filter
In `FakeClock` we have `runDueTasks()` (without filter), `numPendingTasks(filter)`, and `getPendingTasks(filter)` which are useful. The `runDueTasks(filter)` method seems not a right way to test.
2018-02-22 12:46:24 -08:00
zpencer 9dc4ded597
core: move ManagedChannelImpl orphan check to wrapper class (#4093)
okhttp and netty libraries will retain hard references to
ManagedChannelImpl via its non static nested classes. Moving the
orphan check to a wrapper class solves this problem.

- Added ForwardingManagedChannel + test
2018-02-22 11:57:26 -08:00
Rama Chavali 48ca4527c1 core: change serverimpl,servercallimpl's internalclose to cancel stream (#4038)
The HTTP/2 error code will be INTERNAL_ERROR for all cancel statuses,
except for DEADLINE_EXCEEDED and CANCELLED, which are mapped to
CANCELLED.
2018-02-22 11:09:21 -08:00
zpencer 887217e57b
core,services: fix binarylog unit tests (#4109)
- The unsafe operation was reverted in code, so the test should not
  use it either
- The variable naming is tripping internal linters
2018-02-22 10:08:12 -08:00
Kun Zhang 46c1133a1e
core: add panic mode for ManagedChannelImpl (#4023)
Channel enters this mode whenever there is an uncaught throwable from
its ChannelExecutor, which is where most channel internal states are
mutated, such as load-balancing.

In panic mode, the channel will always report TRANSIENT_FAILURE as its
state, and will fail RPCs with an INTERNAL error code with the
uncaught throwable as the cause, which is helpful for investigating
bugs within gRPC and 3rd-party LoadBalancer implementations.

## Change to existing behavior
Previously if `LoadBalancer` throws in `handleResolvedAddressGroups()`, it would
be routed back to `LoadBalancer.handleNameResolutionError()`. Now it will make
the channel panic.

## Internal refactors
- Refactored out `shutdownNameResolverAndLoadBalancer()`, called from three code paths: `enterIdleMode()`, `delayedTransport.transportTerminated()`, and `panic()`.
- Refactored out `updateSubchannelPicker()`, called from both `updateBalancingState()` and `panic()`.
2018-02-22 09:27:08 -08:00
ZHANG Dapeng 38c84ed9ef
core: temporarily disable retry when stats or tracing is enabled
To avoid breakage, temporarily disable retry when stats or tracing is enabled.

This still agrees with the description of the javadoc of `ManagedChannelBuilder.enableRetry()`

```java
/**
   * Enables the retry mechanism provided by the gRPC library.
   *
   * <p>This method may not work as expected for the current release because retry is not fully
   * implemented yet.
   *
   * @return this
   * @since 1.11.0
   */
  @ExperimentalApi("https://github.com/grpc/grpc-java/issues/3982")
  public T enableRetry()
```
2018-02-21 15:04:36 -08:00
Eric Anderson b78c5cfcf2 core: Improve status descriptions in MessageDeframer
This removes the debug string added in 5a4794f2c because the issue was resolved
and the debug string is confusing for Netty, producing results like:
io.grpc.netty.NettyClientTransport$3: Frame size 216695976 exceeds maximum: 4194304

This makes it seem like it's a HTTP/2 frame problem and not a gRPC message
problem. Although it is clearer once you see the maximum is 4 MB, this already
bit me in #4086.

In general, we should talk about "messages" instead of "frames" when possible
as that's what is most comprehensible to the consumer of the description. I
also make sure to mention "gRPC" messages/frames because otherwise the message
is ambiguous.
2018-02-21 14:20:12 -08:00
zpencer 7d47ac0c12
core: Clarify ownership of InputStreams wrt Marshaller and Stream (#3419)
It is the responsibility of the caller to close the `InputStream`
returned by `Marshaller`.

`Stream` takes ownership of any `InputStream`s passed in.

`DelayedStream` and the streams of `InProcessTransport` already
handle ownership transfer, but `AbstractStream` needs to call
`close`.
2018-02-20 09:57:57 -08:00
zpencer 5f7b64a250
core,services: lock down visibility of BinaryLogSink (#4100)
- This class should not be a part of the public API
- Update ServiceProvidersTest to verify package private services can be
  loaded with the utility.
2018-02-20 09:54:40 -08:00
zpencer bde2ba2444
services: introduce BinaryLogSinkProvider (#3917)
The `BinaryLog` will write `GrpcLogEntry`s to the sink, which is
intended as a pluggable interface. It is the sink's
responsibility to send the binlog protos to disk, to a remote
logging service, etc.
2018-02-16 13:47:31 -08:00
ZHANG Dapeng df4048d98d
core: retry part 5 continued, throttle policy 2018-02-15 13:15:04 -08:00
Carl Mastrangelo 2f2881756f
core: remove lint warning 2018-02-14 13:10:55 -08:00
Eric Anderson 3480a08e70 core: ConnectivityStateManager is always enabled
There have been no callers of disable() since commit 1c7421be7.
2018-02-13 08:56:43 -08:00
zpencer 890da706bb
Revert "core: tests for forwarding {Server,Client}{Call, CallListener} (#4058)" (#4070)
This reverts commit 77397b9dd0.
2018-02-12 12:21:24 -08:00
Eric Gribkoff 6ee6eae5a0
core: add prepareToLoseNetwork() method to ManagedChannel 2018-02-09 11:05:09 -08:00
zpencer 77397b9dd0
core: tests for forwarding {Server,Client}{Call, CallListener} (#4058) 2018-02-09 10:56:39 -08:00
zpencer 5d5e8c6ad5
core: Install the ServerInterceptor from BinaryLogProvider (#3879) 2018-02-09 10:56:04 -08:00
zpencer 011c7629fd
core: split binlog tests into two in ManagedChannelImplTest (#4049) 2018-02-09 10:52:07 -08:00
zpencer 35f0d15291
core: add subchannel stats (#3967) 2018-02-09 10:50:11 -08:00
zpencer d2c7e33f3e
core: pull out reusable forwarding test, refactoring forwarding tests (#3480) 2018-02-08 16:31:23 -08:00
ZHANG Dapeng bc3e4e9a85
core: Remove incorrect `@GuardedBy` annotations
This change is being made in preparation for fixing a bug in the enforcement
of `@guardedby`.

Non-static inner classes can refer to locks declared by their enclosing
instances. Static member types (static classes, interfaces) cannot.
2018-02-08 14:29:59 -08:00
zpencer f8c7f963ef
core,netty,okhttp: use ServiceProviders for ManagedChannelProvider, ServerProvider (#4047) 2018-02-08 08:57:52 -08:00
ZHANG Dapeng c735bb3385
core: retry part 6, client options - maxAttempts 2018-02-07 18:00:56 -08:00
ZHANG Dapeng e33d39cf93
core: retry part 6, client options - disable retry 2018-02-07 18:00:36 -08:00
ZHANG Dapeng ad62cc2775
core: retry part 5 continued, logic on retry policy
Implement logic for making retry decision using RetryPolicy.
2018-02-07 18:00:04 -08:00
Eric Gribkoff de54a4cb49
core: remove @Internal annotation on @Internal and @ExperimentalApi 2018-02-07 14:40:25 -08:00
Ryan Michela 48fb9ff55f core: Add toString() for MethodDescriptor and ServiceDescriptor (#4013) 2018-02-07 10:30:54 -08:00
zpencer 71cc0e37a8
core: remove unused test resouce files from NameResolverProviderTest (#4048) 2018-02-06 17:37:42 -08:00
zpencer ca40d55e00
core: extract provider class out of ServiceProvidersTest (#4042) 2018-02-05 15:17:57 -08:00
Snow Pettersen 2c8bc71fb7 core: Resolve address of GRPC_PROXY_EXP hostname
Since this address is never resolved gRPC fails to connect with an
UnresolvedAddressException when the env variable is specified. This
should resolve the address before we attempt to proxy to it.
2018-02-01 15:08:42 -08:00
ZHANG Dapeng dc95465f6a
core: retry part 5, add RetryPolicy data object 2018-01-31 11:25:50 -08:00
zpencer 141a1d23ee
core: install the binary logging client interceptor (#3937) 2018-01-31 09:33:12 -08:00
Eric Gribkoff 2f05c23063
Start 1.11.0 development cycle 2018-01-30 15:13:55 -08:00
Eric Anderson d792a72ea1 Revert "core: make ManagedChannel honor Service config LB "
This reverts commit 98aa69af72.

The GSON dependency is noticable on Android for no benefit (currently).
Reverting until we figure out what we want to do.
2018-01-30 13:38:45 -08:00
Eric Anderson fbdc217b46 Revert "core: handle long dns txt records properly, parse service config, and add tests"
This reverts commit b01609572a.

The GSON dependency is noticable on Android for no benefit (currently).
Reverting until we figure out what we want to do.
2018-01-30 13:38:45 -08:00
zpencer 47019325db
core: PartialForwardingServerCall should forward getAuthority (#4009) 2018-01-29 14:02:21 -08:00
zpencer a85b0e1047
core: remove spy() usage in ServerImplTest (#4010) 2018-01-29 14:02:00 -08:00
Kun Zhang df65bef881
core: delete outboundMessage() and inboundMessage() on StreamTracer. (#4014)
They were deprecated in 1.7.0.
2018-01-29 13:42:14 -08:00
zpencer 0465bb5aeb
core,testing: deprecate passing ServerCall to StatsTraceContext (#3912)
Instead, pass a ServerCallInfo object containing the interesting bits
of info. This lets us modify the call handler for binary logging, but
still provide the original info to the StatsTraceContext API.
2018-01-26 16:51:45 -08:00
Bogdan Drutu af936919f3 Update opencensus to 0.11.0 and remove extra dependencies. 2018-01-26 13:04:26 -08:00
Shohei Kamimori 0c21032221 core: change retention policies to CLASS
`@Internal` and `@ExperimentalApi` annotations are not visible at compile time
of application (not grpc-java) because their retention policies are `SOURCE`.
So, change retention policies to `CLASS`.
2018-01-26 13:01:55 -08:00
zpencer 199a5203c5
core: move ChannelStats and TransportStats to io.grpc.internal.Channelz (#4008) 2018-01-25 23:24:49 -08:00
zpencer b109595ad3
core: move Instrumented, LogId, WithLogId to io.grpc.internal as public (#3995) 2018-01-25 17:19:00 -08:00
zpencer 2a93e6b92f
core: NameResolverProvider should use ServiceProviders util (#4005) 2018-01-25 17:18:42 -08:00
zpencer 2996207c20
core: generifiy ServiceProvider class (#3886)
Moving all the common code among different service provider
classes to one place.
2018-01-25 14:12:09 -08:00
zpencer 36670c1fcd
core: disable jndi in gae jdk7 (#4000) 2018-01-25 11:32:19 -08:00
ZHANG Dapeng cd3c9e245b
core: retry part 4, retry attempt header 2018-01-23 17:28:52 -08:00
ZHANG Dapeng 671834fd9e
core: retry part 3: use call executor and sceduled executor service
Use call executor and scheduled executor service to schedule and run `retry()`.

Backoff amount computation from retry policy and testcases during backoff will be added in future PRs.
2018-01-23 10:56:19 -08:00
ZHANG Dapeng cf4a38ed45
core: retry part 2, buffer size limit
Implement buffer size counting with ClientStreamTracer and buffer size limit following the spce https://github.com/grpc/proposal/blob/master/A6-client-retries.md#memory-management-buffering
2018-01-23 10:35:59 -08:00
Carl Mastrangelo b01609572a
core: handle long dns txt records properly, parse service config, and add tests 2018-01-22 16:16:52 -08:00
Eric Anderson a54ff9cb54 core: Remove unused ServiceConfiguration
It was going to be used, but then the design changed. It's now
unnecessary.
2018-01-18 10:56:52 -08:00
zpencer c51b9271fb
core: rename io.grpc.internal.ChannelTracer to CallTracer (#3973)
This class will be reused to count ServerCall stats.
2018-01-17 14:04:09 -08:00
ZHANG Dapeng 358ea0c15d
core: retry part I followup, refactor commit()
It's more reasonable and convenient to return a `Runnable` for `commit()` method because we want to execute part of the commit under a lock and run part of it out of the lock.
2018-01-17 09:12:56 -08:00
Eric Anderson efcbef58e3 core: add missing separator in user-agent
The separator was accidentally removed in 1bbe126b4
2018-01-17 08:17:36 -08:00
zpencer 32ad2311ad
core: channel tracer should report target name and ConnectivityState (#3969)
This addresses regular channels and oob channels.
2018-01-16 10:05:30 -08:00
Carl Mastrangelo 98aa69af72
core: make ManagedChannel honor Service config LB
I think the idle state transitions are correct. I have looked at them and tried tracing a few paths through. This doesn't go full idle because the name resolver doesn't need to be restarted. Also, the LB transition happens inside of a NR callback, so it would be odd to have the NR terminate itself upon successful resolution. (Might this cause recursion? I think it may).
2018-01-12 16:56:54 -08:00
Eric Anderson bd7080337b core: Mention limitations of connectivity state API
These limitations are by design, but may be non-obvious.
2018-01-12 12:21:50 -08:00
Eric Anderson ba8063e7b0 all: Prefer mock+delegatesTo() over Mockito.spy()
Spies are really magical and easily produce unexpected results. Using them in
tests can easily yield tests that don't do what you think they do. Delegation
is much safer when possible.

Delegation doesn't work when methods `return true`, final methods, and with
restricted visibility, though. So CensusModulesTest and
MaxConnectionIdleManagerTest are left as-is.
2018-01-11 09:32:54 -08:00
Eric Anderson af0283477d Update ErrorProne to 2.2.0 and fix failures 2018-01-10 14:14:27 -08:00
zpencer 16cec947c5
core: MethodDescriptor.toBuilder() should copy schemaDescriptor (#3941) 2018-01-10 13:15:25 -08:00
Eric Anderson 4bc0c95d0b Update ErrorProne to 2.1.3 and fix failures
The fixes could have subtle side-effects, but I did take care when making them.
2018-01-09 12:40:55 -08:00
Eric Anderson 35a6bf863d Enable the Guava Beta Checker 2018-01-08 14:52:51 -08:00
Spencer Fang 3809f676ee core: make TransportStats package-private
Limited access will be granted via InternalTransportStats.
2018-01-04 15:35:04 -08:00
ZHANG Dapeng 450a983552
core: rm unused variable and method 2018-01-02 10:06:10 -08:00
ZHANG Dapeng 66119463ad
core: fix java ErrorProne 2017-12-28 13:06:58 -08:00
zpencer e7b2089c85
core: ManagedChannel should not depend on internal (#3898) 2017-12-28 11:04:11 -08:00
ZHANG Dapeng 40453c744f
core: fix unimplemented API in ManageChannel
Two APIs were added to ManageChannel. This would be API breaking if they are still abstract.
2017-12-27 13:30:12 -08:00
Shohei Kamimori b1d62b70c4 core: remove unused variables 2017-12-20 10:27:16 -08:00