Commit Graph

1347 Commits

Author SHA1 Message Date
Kun Zhang cc5e3c19df
core: ForwardingLoadBalancerHelper (#4911)
This will be used by LoadBalancer plugins that delegates to another,
which is what the new request routing (go/grpc-request-routing-design)
requires.  This will also be used to wrap LoadBalancers to add
client-side health-checking functionality.
2018-10-06 12:36:35 -07:00
Kun Zhang fbfc3a40d0
core: add Grpc.TRANSPORT_ATTR_LOCAL_ADDR (#4906)
Resolves #4135
2018-10-03 16:43:37 -07:00
Carl Mastrangelo 6b7fa40378
core: name anonymous classes in ManagedChannel for clear stacktraces 2018-10-03 14:09:00 -07:00
Kun Zhang ebbf8005be
doc: organize Attributes with annotations. (#4892)
* doc: organize Attributes Keys with annotations.

Keys are annotated with the following annotations:

1. Grpc.TransportAttr: transport attributes returned by
{Client,Server}Call.getAttributes().

2. NameResolver.ResolutionResultAttr: attributes passed as the
argument of NameResolver.Listener.onAddresses() and
LoadBalancer.handleResolvedAddressGroups()

3. EquivalentAddressGroup.Attr: attributes from
EquivalentAddressGroups.

* Expand the usage of annotations to Attributes variables.
2018-10-01 10:11:01 -07:00
zpencer 2fae9a3a97
core: permanently store authority at channel creation (#4886)
Getting the authority must not rely on the name resolver being
non-null, because that can trivially happen if the channel is shut
down.
2018-09-28 16:10:14 -07:00
Jesse Wilson 8b16899bc1 Upgrade to Guava 26.0-android and jsr305 3.0.2 2018-09-28 13:23:55 -07:00
zpencer da87ffb329
core,services: v1 binlog (#4846)
Log using new proto definition

- Remove io.grpc.BinaryLog.CallId because a call ID is now an AtomicLong
- Add the concept of "always included" and "never included" metadata
  keys. This is needed because grpc-status-details-bin is already
  logged in the binlog msg, and we will log grpc-trace-bin for the
  census info.
- unit tests are effectively rewritten
2018-09-27 13:19:24 -07:00
Spencer Fang cc09eab9af core: fix channelz import on AutoConfiguredLoadBalancerFactory
The original PR was stale when merged.
2018-09-19 14:32:16 -07:00
ZHANG Dapeng 0afc10c2d6
core: channel tracing to log lb policy changes 2018-09-19 11:36:33 -07:00
Carl Mastrangelo b0f423295b
all: use Java7 brackets 2018-09-14 13:52:29 -07:00
Nick Hill ed709ff9ff core: remove redundant SubchannelPicker refreshes in RoundRobinLoadBalancer
* Remove redundant SubchannelPicker refreshes in RoundRobinLoadBalancer

- Ensure active subchannel list and round-robin index is only
regenerated/refreshed when it changes
- Make it so that Subchannels exist in subchannels map iff their state
!= SHUTDOWN
- Add EmptyPicker class since logic for this case is disjoint from the
non-empty case

* remove explicit initialization of boolean ready field

per @carl-mastrangelo's review comment

* minor restructuring to make logic clearer; more explanatory comments

* move some checks inside updateBalancingState method for clarity

* store current state and picker in RRLB, only update when new one is diff

* some more simplification/refactoring; improve test coverage

- remove now redundant check in handleSubchannelState

- collapse getAggregatedState() and getAggregatedError() into
handleBalancingState()

- have both pickers extend new RoundRobinPicker, move
areEquivalentPickers() logic into RoundRobinPicker.isEquivalentTo()

- extend unit tests to cover some additional cases

* Address latest review comments from @zhangkun83

- Use explicit check for non-empty list instead of assert
- Change EmptyPicker.status to be non-nullable
- Further test coverage improvement including explicit picker comparison
tests

* use EMPTY_OK instead of Status.OK for initial empty picker
2018-09-12 15:06:46 -07:00
ZHANG Dapeng c8975e987b
all: fix lint warnings in import 2018-09-11 16:24:44 -07:00
zpencer 95fd47d747
core, services: remove census from binary logs (#4845)
The exact census span behavior wrt gRPC is not yet defined, so let's
punt on tight integration.

It may be fine to log grpc-trace-bin on server side because it is a
key visible to the application.
2018-09-10 16:35:08 -07:00
creamsoup 189991012b
core: DnsNameResolver caches refresh (#4812)
DnsNameResolver caches refresh using java security property networkaddress.cache.ttl.

Resolves #4745
2018-09-06 16:58:34 -07:00
zpencer 2fca42feb9
all: prepend internal classes with Internal (#4826)
This is a safer way to hide the classes, because they will not appear
in public targets for some build configurations.
2018-09-05 18:48:42 -07:00
zpencer 4d366ce978
all: move Channelz to io.grpc as InternalChannelz (#4797)
This is an API used to coordinate across packages and must live in
`io.grpc`.

Prepending `Internal` makes it easier to detect and hide this class
from public visibility when using certain build tools.

fixes #4796
2018-09-04 16:52:01 -07:00
creamsoup bbacd164f9
skip populating trace header if no census impl is available. (#4805) 2018-08-30 14:45:11 -07:00
ZHANG Dapeng 6037659dd7
core: make HedgingPolicy final
Just as RetryPolicy is final.
2018-08-28 17:09:05 -07:00
Eric Gribkoff 67ee4b6a8f
core: switch to Java 7 source and bytecode (#4801)
javac can produce code that invokes Object.requireNonNull when instantiating
an inner class using a instance variable. See
https://bugs.openjdk.java.net/browse/JDK-8202137
2018-08-27 20:48:57 -07:00
Kun Zhang 28d44ae46d
Start 1.16.0 development cycle (#4803) 2018-08-27 17:21:46 -07:00
Eric Anderson 7b126b00a0 all: Swap to Java 7 source and bytecode
Core and OkHttp are left with Java 6 for the moment. Once we resolve
their issues they could be bumped as well.

Updates #3961
2018-08-27 15:29:03 -07:00
ZHANG Dapeng e9b6568450
core: plumb hedging policy
This is only API plumbing for hedging, following exactly the same way as its retry counterpart, so it is almost trivial.
2018-08-23 14:38:10 -07:00
Carl Mastrangelo 3f05a6e331
core: minor cleanup of NameResolverProvider
* Make the list of providers an immutable List
* Make obvious that the list is statically initialized
* Add documentation for when methods were added.
* Use RuntimeException, rather than IllegalStateException.
2018-08-20 12:59:55 -07:00
Carl Mastrangelo 7fe49f9b52
core: add ability to create stackless status exceptions 2018-08-17 11:40:11 -07:00
zpencer a48b090dc6
core: Handle null ProxySelector (#4762)
ProxySelector.getDefault() can return null

Fixes #4677
2018-08-16 15:04:33 -07:00
creamsoup ba4db45e71 Fix errror message when DNS name is invalid. (#4751)
It used to throw NPE, since URI.create creates URI with null hostname. Now it
thorws IllegalArgumentException for invalid DNS name, NPE for null name.
2018-08-13 16:55:20 -07:00
Eric Anderson 3cfc5af4f1 core: Avoid implicit requestConnection in PickFirst
This makes the behavior more clear.
2018-08-13 09:41:06 -07:00
Spencer Fang 9f477b27bb core: fix unused variable lint 2018-08-09 13:31:23 -07:00
Eric Gribkoff 79b24709b9
all: update animalsniffer to Java 7 and add Android 14 (#4727) 2018-08-09 09:09:21 -07:00
Carl Mastrangelo 67352081f8
core: revert warning about `Status.asException(null)`
There seem to be some users converting from StatusRuntimeException
to StatusException using the following paradigm:

sre.getStatus().toException(sre.getTrailers())

Since there isn't a viable alternative, revert the warning.
2018-08-06 11:44:52 -07:00
Kun Zhang 2aa02fd869
core: remove unused fullMethodName field from CensusStatsModule. (#4731) 2018-08-03 09:16:29 -07:00
Carl Mastrangelo 9895e243b1
inprocess: prevent null names, and define socket address equality 2018-08-02 17:28:36 -07:00
Eric Anderson b64cde1488 Encourage using grpc-netty-shaded instead of grpc-netty
grpc-netty is still really useful, but for most users who aren't doing
anything advanced using grpc-netty-shaded is much safer from a
dependency basis.

grpc-netty-shaded has seen more usage and has shown itself to be stable
and reduce the number of conflicts due to Netty versions.
2018-08-02 16:58:59 -07:00
Tom Leach b9d1bb8b8b core: initialize round robin load balancer picker to random index (#4462)
RoundRobinLoadBalancerFactory creates a new Picker instance every time the set of provided address groups changes or the connection state of subchannels associated with existing address groups changes. In certain scenarios, such as deployment/replacement of the target service cluster, this can lead to high churn of Picker objects. Given that each new Picker's subchannel index is initialized to zero, in these scenarios requests can end up getting disproportionately routed through subchannels (and hence server nodes) which are earlier in the list of address groups.

At Netflix we have measured that some service nodes end up taking 3-4x the load that of other nodes during deployment.

This commit randomizes the start index of the RoundRobinLoadBalancerFactory.Picker which eliminates this behavior.
2018-08-01 13:15:17 -07:00
Carl Mastrangelo 85448189ee
all: add tracking issues for all experimental APIs and make it required
Additionally, make Status*Exception.getTrailers() non experimental
2018-07-31 15:28:48 -07:00
Carl Mastrangelo bcbc6ae8fd
core: stabilize Status Exceptions API, and warn on null trailers in Status 2018-07-27 15:53:44 -07:00
Carl Mastrangelo 9b200eb7be
core: add flags for selectively enabling grpclb 2018-07-27 15:24:43 -07:00
ZHANG Dapeng 658e73a69e
core: let channel tracing log service config changes
This is the implementation of the second one of the changes to the channel tracing spec
grpc/proposal#89
2018-07-25 10:42:00 -07:00
ZHANG Dapeng 03d85ee53e
doc: clarify channel state API unimplemented until v1.6.1
Resolves #4624
2018-07-24 14:01:24 -07:00
Carl Mastrangelo 8da06a8bc4
all: remove unneeded deps on errorprone 2018-07-23 17:51:44 -07:00
zpencer 67073c921e
core: turn PairSocketAddress into ProxySocketAddress (#4649)
This internal class will only ever be used to plumb proxy info.
2018-07-23 17:43:26 -07:00
George Gensure 19b2a17801 core: Don't close in TSREI in cancelled contexts (#4596)
Prevent multiple effective close calls either by successful completion
of a cancel or complete notification, or through successive exceptions
handled within a single call.
2018-07-23 14:39:39 -07:00
ZHANG Dapeng e1865b565d
core: channel tracing log only when number of backends changed between zeor and nonzero
This is to implement one of changes made in the spec
https://github.com/grpc/proposal/pull/89/files
2018-07-23 14:30:38 -07:00
Eric Gribkoff 877b1a198f
core: enterIdleMode() exits idle if still in use (#4630) 2018-07-23 09:45:06 -07:00
Carl Mastrangelo 64d272ae7c
core: make service config errors recoverable 2018-07-20 17:47:03 -07:00
Eric Anderson 15a5ba2698 Remove DoNotMock annotation in favor of JavaDoc
DoNotMock was removed from error_prone_annotations in 2.1.3, because
there was no enforcement mechanism (which is in google/error-prone#572).
Guava and Trust also depend on error_prone_annotations and are beginning
to use newer versions, so our usage of DoNotMock is causing diamond
dependency problems. This allows us to update to 2.2.0.

The annotations were useful internally; we're solving that in cl/205294089.
2018-07-20 14:03:55 -07:00
zpencer 4335445d98
Start 1.15.0 development cycle (#4650) 2018-07-18 10:48:04 -07:00
Carl Mastrangelo 146b6006b3
compiler,stub: update RpcMethod docs and usage 2018-07-12 17:01:47 -07:00
Eric Anderson 9222dc5d01 core: Add missing @Nullable annotations to ClientTransportOptions
Convert the checkNotNull into our more standard line while I'm cleaning
it up.
2018-07-09 17:21:51 -07:00
Eric Anderson 9d6241eedc Propagate EquivalentAddressGroup attributes to transports
Most of the changes are changing the signature of newClientTransport.
Since this is annoying, I choose to introduce a ClientTransportOptions
object to avoid the churn in the future.

With ClientTransportOptions in place, there's only a few lines necessary
of plumbing for the Attributes: add the field to ClientTransportOptions
and populate it in InternalSubchannel. There are no consumers of the
field in this commit.
2018-07-09 13:00:17 -07:00
Eric Anderson 2b48210b73 grpclb: Plumb attributes for OOB and backend channels
These attributes can be used by ALTS-specific code to determine whether
ALTS or TLS should be used.
2018-07-09 11:25:49 -07:00
Eric Anderson dd57b667cc
Fix unused variables
Unused variables in tests were deleted. The unused variable in Netty
was a future that needed completing; that was a bug.
2018-07-04 07:32:57 -07:00
Yang Song 02ad99e1f5 core: Avoid compare default TagContext in thread local with empty. (#4613) 2018-07-03 15:28:05 -07:00
ZHANG Dapeng 6dad047126
core: add jndi timeout and dirContext.close 2018-07-02 17:36:05 -07:00
Eric Anderson e3ff1ade07
core: Add support for List<EAG> in Subchannels
This avoids the needs to flatten to EAGs for cases like PickFirst,
making the Attributes in EAGs able to be used in communication with
core. See #4302 for some discussion on the topic.
2018-07-02 14:00:05 -07:00
Carl Mastrangelo 39d2dd0eaa
core: make InternalHandlerRegistry a HandlerRegistry 2018-06-29 13:01:25 -07:00
Nick Hill b2dd6ae7f0 util: Improve RoundRobinLoadBalancer scalability with stickiness
- Rework stickiness picker logic to be non-blocking
- Stash `Subchannel` ref in an attribute rather than dedicated map
2018-06-29 09:49:23 -07:00
Marco Ferrer ac55604527 fixed typo in round robin factory name const
Fixes: #4598
2018-06-29 09:16:30 -07:00
Eric Anderson 3e67a5be75 core: Avoid unnecessary unchecked cast in DnsNameResolver 2018-06-28 15:47:06 -07:00
Carl Mastrangelo a74bb350b7
core: always do SRV record lookup in DnsNameResolver
Instead of failing after a a missing A/AAAA record, this change
makes the resolver keep going and try out SRV records too.  This
is needed for use with ALTS, and is part of the gRPCLB spec.

This change also moved the JNDI code to a separate, reflectively
loaded file.  This makes it easy to exclude the file and not worry
about the missing class references on Android.  Additionally, if
javax.naming might be available on Android, this allows it to be
loaded.  A key side effect of this is that DnsNameResolver is
smaller, and more cleanly tested.
2018-06-26 16:15:52 -07:00
Carl Mastrangelo 81da3eb95b
core: make Auto config load balancer not depend on service config
Also, add some tests
2018-06-26 13:36:02 -07:00
Doug Lawrie ffd0f9feca inprocess: add a toString for InProcessSocketAddress
The motivation here is in some cases we log the remote-addr that is set in the gRPC call attributes, and have to special case this type to support inprocess servers.
2018-06-22 10:32:42 -07:00
Kun Zhang 15786520f9
grpclb: use exponential back-off for retries of balancer RPCs (#4525) 2018-06-12 14:04:45 -07:00
Grant Oakley defb955f3a core: add clarification to ClientCall javadoc
* Improve ClientCall documentation. Clarify that some methods of ClientCall may be invoked from any thread. Adds @throws clauses to request().
2018-06-12 10:23:01 -07:00
ZHANG Dapeng 5ce10f0146
all: add gradle format checker
This PR adds an automatic gradle format checker and reformats all the *.gradle files. After this, new changes to *.gradle files will fail to build if not in good format, just like checkStyle failure.
2018-06-11 18:35:18 -07:00
Carl Mastrangelo 8f51c27319
core: fix lint 2018-06-06 17:14:58 -07:00
apolcyn 6dbe3921ed core: fix SRV record parsing
* only unquote TXT data, so as to not remove spaces from SRV data
2018-06-04 18:59:15 -07:00
ZHANG Dapeng c796901aca
core: channel tracing to log events for channel state change 2018-06-04 17:38:26 -07:00
ZHANG Dapeng 8a3e623be7
Start 1.14.0 development cycle 2018-06-04 16:08:31 -07:00
Bogdan Drutu 2a127ce3e7 core: update opencensus to 0.12.3. (#4530)
Changed calls to deprecated methods to the newly added methods.
2018-06-04 15:27:27 -07:00
ZHANG Dapeng a0794d5a68
core: Implement channel tracing for some events
Includes 
- API plumbing for channel tracing `+ManagedChannelBuilder.maxTraceEvents()`
- trace logging for channel/subchannel creation events and name resolving events.

Trace logging for all other events will be added in subsequent PRs.
2018-06-01 07:33:37 -07:00
zpencer 25a2ec1ef3
core: fix ServerBuilder.setBinaryLog javadoc (#4514)
The javadoc should refer to server not channel.
2018-05-31 18:14:44 -07:00
ZHANG Dapeng 608ce3d344
core: unit test for channel shutdown during retry backoff 2018-05-24 17:18:05 -07:00
zpencer ff9aa39bb6
core,services: fix lints (#4504)
- Type parameters must be upper camel case
- constant case field must be final
2018-05-24 11:12:30 -07:00
Eric Anderson e41e054776 Propagate CallCredentials.ATTR_SECURITY_LEVEL from transports
Previously no transport provided the key so CallCredentials would always
see the security as NONE.
2018-05-24 09:51:45 -07:00
zpencer 27439876f2
services,core: make BinaryLogSink visible, add setter (#4503)
* make BinaryLogSink public
* add io.grpc.BinaryLog to server/channel builder
2018-05-23 20:49:09 -07:00
zpencer 1f99fcdc82
move io.grpc.BinaryLogProvider to io.grpc.services (#4501)
Add internal accessors for ServerInterceptors and ClientInterceptors because some helpers were pkg private
Fix tests that were once creating BinaryLogProvider instances, they should now only create io.grpc.BinaryLog instances
2018-05-23 17:04:48 -07:00
zpencer 1aec994475
io.grpc.BinaryLog is now the fully functional API (#4498)
classes in internal now use the io.grpc.BinaryLog API,
io.grpc.BinaryLogProvider is an implementation of the API and will be
moved to io.grpc.services.
2018-05-23 15:58:37 -07:00
ZHANG Dapeng 6f2980e979
core: fix Attributes hashCode 2018-05-23 13:27:03 -07:00
Kun Zhang 68a462e447
noop: resolve lint warnings found at import. (#4496) 2018-05-23 11:58:19 -07:00
ZHANG Dapeng 451c412354
core: fix client does not detect truncated message
Resolves #3264
2018-05-21 13:51:15 -07:00
Jianwei Mao 10291d5ccc core: fix missing comment for headers param 2018-05-21 13:10:58 -07:00
ZHANG Dapeng f5f560ad36
all: TimeProvider to use nanos rather than millis
This is the same practice as #2833
2018-05-21 12:44:06 -07:00
zpencer faffb09f0a
core: remove io.grpc core dependency on census (#4461)
io.grpc (core) does not strictly require census. Move the usages and
and remove census from BUILD.bazel .
2018-05-18 14:56:45 -07:00
zpencer 850249f9ca
core: fix lints from import (#4472)
The import surfaced a few linter issues.
2018-05-17 16:21:30 -07:00
ZHANG Dapeng 4d94163bbd
core: Add ChannelTracer
This is a class similar to `CallTracer`, to be used for Channel Tracing.

The constructor arg `maxEvents` is a param supposed to be provided by `ManagedChannelBuilder`.
2018-05-16 10:56:41 -07:00
zpencer e806e387fe
core: put @ExperimentalApi on CallOptions.of() (#4466)
This marks the method as still experimental, to be clear to users that
they are using an experimental api.
2018-05-15 15:56:42 -07:00
ZHANG Dapeng 6a96656764
core: make channel stats' channel state consistent with subchannels
Moved `setState()` inside of `channelExecutor` runnable together with `setSubchannels`, otherwise the state may be inconsistent with subchannels, say channel state is the initial IDLE but subchannels is non-empty with active transports, or channel state is READY but subchannels is empty.
2018-05-15 15:48:57 -07:00
ZHANG Dapeng bf4a00c6de
context/core/netty: Add @CheckReturnValue to Context
By adding inner class annotations without introducing external dependencies.
2018-05-15 13:14:30 -07:00
ZHANG Dapeng 561583be14
core,services: Add ChannelTracing data object
Added `ChannelTrace` as an inner class of `Channelz`.

This is in preparation for the implementation of [Channel Tracing](https://github.com/grpc/proposal/blob/master/A3-channel-tracing.md)
2018-05-15 11:33:44 -07:00
zpencer 04a90bcad2
core: stabilize custom CallOptions API (#4457)
Deprecate `of()` in favor of `create()` and
`createWithDefault()`. Emphasize that the name string is only for
debug purposes.

Fixes #1869
2018-05-15 10:03:46 -07:00
zpencer 21b73bbdc1
core: partially stabilize Attributes API (#4458)
Deprecate static builder method, Keys.of(), add a notice of plans to
remove keys(), emphasize that the name is only a debug label.
The `@ExperimentalAPI` is left on the class because there are still
issues around hashCode/equals.
2018-05-14 11:30:06 -07:00
zpencer 902baa0cde
core: do not use internal accessors in BinaryLogProvider (#4454)
Now that this class is in `io.grpc` the accessor is not needed.
2018-05-09 12:21:05 -07:00
zpencer 247a76ed93
core,netty,okhttp: make toString more consistent for channelz (#4434)
Use MoreObjects.toStringHelper and use only the log id's long value,
because the class name is already present in the toStringHelper.
2018-05-07 11:38:16 -07:00
zpencer 6f29b60dcf
core, services: make BinaryLog an explicit object that is passed into channels/servers (#4431)
remove SPI, io.grpc.BinaryLog is a public API that is passed into builders and must be
explicitly closed.
2018-05-07 07:41:42 -07:00
Carl Mastrangelo 60a0b0c471
all: normalize copyright header 2018-05-03 14:55:21 -07:00
Eric Anderson 894c815832
core: Document Metadata ownership passes to the Call{,Listener}
This was already clearly documented in ServerCall.Listener and
ClientCall.
2018-05-03 07:08:27 -07:00
Grant Oakley 6bf8e0c084 core: add @CheckReturnValue to CallOptions 2018-05-02 15:56:52 -07:00
ZHANG Dapeng e19e8f7d40
core: populate effective deadline to ClientStream
Added `ClientStream.setDeadline(Deadline deadline)` method, which will set the timeout header.

Resolves #4412
2018-05-02 13:31:03 -07:00
ZHANG Dapeng d50c191aca
core/stub: fix lint warnings 2018-05-02 10:37:32 -07:00