Commit Graph

225 Commits

Author SHA1 Message Date
Carl Mastrangelo c6b505229c
all: move LB parsing logic into LB.Factory 2019-03-19 13:33:13 -07:00
Carl Mastrangelo e5e01b5169
core,grpclb: use better generics on service config 2019-03-08 14:11:13 -08:00
Eric Anderson b48b0ac1d4 all: Stop committing generated protobuf messages
This commit swaps to using a Sync task to place generated code in the
src/generated folder instead of the gradle-protobuf-plugin's
generatedFilesBaseDir. This provides much nicer results on failed
builds, and you will no longer see all the generated files deleted.

But at the same time the Sync task makes it easy to only copy the
grpc-generated code. This was not previously done because we were lazy
and using generatedFilesBaseDir, which made it difficult to treat the
services differently from the messages.
2019-03-05 16:28:55 -07:00
Kun Zhang a15a3117de
core: deprecate LoadBalancer.Helper#getNameResolverFactory (#5418)
This was added for the potential use case of needing to resolve target
names (of the same scheme as the top-level channel's target's) in the
LoadBalancer.  Now actual use cases come up in xDS that we need to
resolve fully-qualified target strings with arbitrary schemes.  This
method has never been used and won't fit future uses because it's too
restrictive.
2019-03-04 13:37:49 -08:00
Eric Anderson eaca73473c
Upgrade to protobuf 3.6.1
For Bazel, we upgrade to protobuf 3.6.1.2 and javalite HEAD to fix
incompatibilities in newer Bazel releases.

compiler/Dockerfile is unused, so it was removed instead of being updated.

protoc no longer includes codegen for nano, so we remain on the older protoc
any time nano is used.

Protobuf now requires C++11 when compiling, so windows was swapped to
VC 14.
2019-02-07 13:40:53 -08:00
Carl Mastrangelo 3a39b81cf5
all: remove java6 type args 2019-02-04 10:03:50 -08:00
Tim van der Lippe a31473ef20 all: fix usages of any(<Primitive>.class) matchers
In Mockito 2, if a method expects a primitive type, but an any(<Primitive>.class) matcher is used in its place, it will throw an error. To prepare for this upcoming breakage, change
all existing any(<Primitive>.class) matchers to use the correct any<Primitive>() matcher.
2019-02-04 09:01:00 -08:00
Carl Mastrangelo ed0a9f3f1d
netty: support listening on multiple ports 2019-01-29 10:13:40 -08:00
Carl Mastrangelo 32fc0bcd38
services: set the default serving state of all services to SERVING 2019-01-24 15:19:51 -08:00
Kun Zhang abdae25c8d
services: log an error to java logger when health-check disabled. (#5261)
This is required by the spec:
https://github.com/grpc/proposal/blob/master/A17-client-side-health-checking.md
2019-01-22 15:17:56 -08:00
Eric Anderson e06b941eb7 alts,services: Sync proto sources from grpc-proto
This syncs to grpc/grpc-proto@f9af5e19f7
2019-01-15 16:15:22 -08:00
Eric Anderson 186409f868
buildscripts: Add script to sync .protos from grpc-proto
Currently only protos in alts, grpclb, and services are synced. Once
some java_package options are "upstreamed", benchmarks and
interop-testing could also be synced.
2018-12-20 16:47:16 -07:00
Carl Mastrangelo 1bf8476cd7
core: standardize logid format and add details for channelz 2018-12-17 17:22:11 -08:00
Carl Mastrangelo dbc9a89e26
services: add method to make health checker permanently go unhealthy 2018-12-07 14:39:37 -08:00
Kun Zhang c8b2cdc023
core/services: test/debug friendlier (#5079)
Raise visibility of AutoConfiguredLoadBalancerFactory as internal
tests need to access it from a different package.

Rename HealthCheckingLoadBalancerFactory.LoadBalancerImpl to
*.HealthCheckingLoadBalancer so that its toString() output is more
informative.
2018-11-21 10:45:18 -08:00
zpencer ea9bdabcb2
services: use Durations.toNanos instead of Duration.getNanos (#5059)
getNanos will return the fractional nanos of the duration, which is
not the same as toNanos for durations larger than 1s.
2018-11-14 10:06:04 -08:00
Kun Zhang 5b87e99622
core: move round-robin to util and include it to hard-coded list (#5057)
This is needed for internal issue b/119247688.

A particular test that runs GRPC Android build in a non-Android
environment failed because RoundRobinLoadBalancerProvider was deleted
by ProGuard but the service-loader META-INF file still referred to it,
causing a loading failure.

This could be fixed by adding RoundRobinLoadBalancerProvider to the
hard-coded list, which is recognized by ProGuard then it will keep the
class.

However, we don't expect anyone to use RoundRobinLoadBalancerProvider
on Android, including the class on Android would increase code size,
which Android apps are sensitive to. Hence we move
RoundRobinLoadBalancerProvider to a different package (util), which is
built as a separate artifact internally which Android users usually
don't depend on. (Note that in open-source util is in the same artifact as core,
which is unfortunately).
2018-11-13 17:06:01 -08:00
Eric Anderson 7a89ce2a90 Lint fixes
Remove unused variables and prefer ArrayDeque to LinkedList. The swap to
Queue from Deque was just to make it more obvious what the usage was,
since the original swap to Deque was to avoid the same LinkedList lint
violation (3d51756d).
2018-11-09 17:15:25 -08:00
Kun Zhang 31af0657d0
services: cancel health-check when LoadBalancer.shutdown() is called. (#5051)
The health checking balancer won't receive an update about Subchannel
shutdown via handleSubchannelState(), because no more callback will be
called after LoadBalancer.shutdown() is called.
2018-11-09 14:51:36 -08:00
Kun Zhang 11154074bd
services: HealthCheckingLoadBalancer logs to ChannelLogger (#5042)
Log the event that health check is disabled due to UNIMPLEMENTED as required in the spec:
https://github.com/grpc/proposal/blob/master/A17-client-side-health-checking.md

Also log every Subchannel state change that is affected by health-checking, i.e., the state changes when the raw Subchannel state is READY and health-check is running.

Tracking issue: #4932
2018-11-08 15:16:52 -08:00
Kun Zhang 21bd098d7b
services: annotate HCRRLBP with RunWith (#5039) 2018-11-07 11:37:26 -08:00
Kun Zhang 99f5943520
services: HealthCheckingLoadBalancerUtil and HealthCheckingLoadBalancerProvider (#5026)
HealthCheckingLoadBalancerUtil is the public wrapper utility that helps
turn a LoadBalancerFactory into a health-checking capable one.

HealthCheckingRoundRobinLoadBalancerProvider overrides the
RoundRobinLoadBalancerProvider from grpc-core.
2018-11-06 09:14:56 -08:00
Kun Zhang 65bd38476f
services: define SERVICE_NAME_ALL_SERVICES for the empty service name (#5027) 2018-11-02 17:09:22 -07:00
zpencer 3d51756d61
core, services: fix more import lints (#5021) 2018-11-01 16:14:42 -07:00
Kun Zhang f5d0f40bdf
services: client-side health checking main implementation (#5014)
Spec: https://github.com/grpc/proposal/blob/master/A17-client-side-health-checking.md

This comes in the form of a wrapper LoadBalancerFactory. The public wrapping utility and the wrapping of RoundRobinLoadBalancer will come in follow-up changes.
2018-10-31 09:29:46 -07:00
Carl Mastrangelo e8762c941c
services: include an error message in channelz 2018-10-26 16:27:29 -07:00
Carl Mastrangelo 04d8d2d382
services: propagate thrown status exceptions in Channelz 2018-10-26 12:52:52 -07:00
ZHANG Dapeng 41c8d8020f
all: fix lint 2018-10-23 14:03:46 -07:00
Thomas Broyer 183e1f6735 all: update Error Prone to 2.3.2
This will allow enabling Error Prone on JDK 10+ (after
updating the net.ltgt.errorprone plugin), and is also a
prerequisite to that plugin update.

Also remove net.ltgt.apt plugin, as Gradle has native
support for annotationProcessor.
2018-10-19 13:08:36 -07:00
zpencer a2ef4c0dbf
services: update v1alpha reflection.proto to match grpc-proto repo (#4973)
More info: https://github.com/grpc/grpc-proto/pull/38
2018-10-18 16:27:07 -07:00
zpencer 9b5115d5a7
services: remove v1alpha binlog (#4963)
We should use v1 everywhere.
2018-10-17 14:46:56 -07:00
Kun Zhang 11f9e8d5b4
services: implement Health.Watch (#4930) 2018-10-12 16:02:25 -07:00
zpencer 308887a150
services: fix callbacks not being forwarded in binlog interceptors (#4918) 2018-10-09 10:45:25 -07:00
zpencer 5eb816b3b1
services: fix channelz javadoc lint for reference not found (#4920)
Referring to the public abstract class seems to be OK, but referring
to the package private impl trips the linter.
2018-10-09 09:23:16 -07:00
zpencer f10676c2b4
services: allow config str to be passed into binlog object (#4919)
Do not require binlog str to be defined by env var.

This allows --flag=value styled configuration, which is more common
internally.
2018-10-08 15:44:18 -07:00
Kun Zhang 31328652d4
services: sync health.proto from grpc-proto repo (#4904) 2018-10-03 16:24:09 -07:00
zpencer da87ffb329
core,services: v1 binlog (#4846)
Log using new proto definition

- Remove io.grpc.BinaryLog.CallId because a call ID is now an AtomicLong
- Add the concept of "always included" and "never included" metadata
  keys. This is needed because grpc-status-details-bin is already
  logged in the binlog msg, and we will log grpc-trace-bin for the
  census info.
- unit tests are effectively rewritten
2018-09-27 13:19:24 -07:00
zpencer ed70594294
services: disallow duplicate rules for a binlogging services or methods (#4868)
Let's simplify the impementation to just disallow duplicates. This
makes it easier to maintain.

Background info: https://github.com/grpc/proposal/pull/104
2018-09-21 15:59:37 -07:00
Carl Mastrangelo b0f423295b
all: use Java7 brackets 2018-09-14 13:52:29 -07:00
Carl Mastrangelo 6cf849a7ce
services: fix duration in Binlog test 2018-09-12 17:16:52 -07:00
zpencer 8fc84daa30
services: remove RequireDoubleSubmitCookieInterceptor (#4833)
The CSRF protection has been moved to the proxy implementation. This
is cleaner since the gRPC service should not need to know about
grpc-web.
2018-09-10 18:06:26 -07:00
zpencer 95fd47d747
core, services: remove census from binary logs (#4845)
The exact census span behavior wrt gRPC is not yet defined, so let's
punt on tight integration.

It may be fine to log grpc-trace-bin on server side because it is a
key visible to the application.
2018-09-10 16:35:08 -07:00
zpencer 2fca42feb9
all: prepend internal classes with Internal (#4826)
This is a safer way to hide the classes, because they will not appear
in public targets for some build configurations.
2018-09-05 18:48:42 -07:00
zpencer 4d366ce978
all: move Channelz to io.grpc as InternalChannelz (#4797)
This is an API used to coordinate across packages and must live in
`io.grpc`.

Prepending `Internal` makes it easier to detect and hide this class
from public visibility when using certain build tools.

fixes #4796
2018-09-04 16:52:01 -07:00
Eric Anderson 73b098702c services: Fix binarylog's java_package to include version 2018-08-17 09:38:38 -07:00
Carl Mastrangelo 85448189ee
all: add tracking issues for all experimental APIs and make it required
Additionally, make Status*Exception.getTrailers() non experimental
2018-07-31 15:28:48 -07:00
zpencer f013771bd1
services: binlog status code, msg, details (#4551)
Log these additional bits of info in RPC binary logging
2018-07-17 11:29:45 -07:00
Carl Mastrangelo 146b6006b3
compiler,stub: update RpcMethod docs and usage 2018-07-12 17:01:47 -07:00
zpencer 9925049dce
services: double submit cookie interceptor for channelz (#4628)
Add this form of XSRF protection to channelz. Non web browser clients
calling channelz must also include the cookie header and metadata
key.
2018-07-12 16:45:17 -07:00
zpencer 3181fbcf03
services: add javadocs for binlog constructors (#4637) 2018-07-12 12:57:10 -07:00
jbingham-google ffcb3b964b compiler, stub: Rename inputType and outputType in @RpcMethod 2018-07-10 13:24:50 -07:00
jbingham-google 9229e30287 compiler, stub: Add @RpcMethod annotation
This annotation will enable Java APT to generate code.

Addresses part of #3173.
2018-07-06 17:02:01 -07:00
Eric Gribkoff 8c52e138ee
services: delete unused proto (#4604) 2018-06-29 15:03:08 -07:00
zpencer 5a8b790a3c
services: binlog method name should include leading / char (#4588)
The APIs of the other implementations all begin method names with `/`,
since the binlog is language agnostic we should be aligned.
2018-06-27 11:22:51 -07:00
ZHANG Dapeng 9d26c5c405
all: fix lint warnings 2018-06-11 12:57:27 -07:00
zpencer 2a971e3b05
services: binlog deadline and methodName at start of rpc (#4550)
These are first class concepts and should be pulled to top level of
proto.
2018-06-07 19:16:34 -07:00
zpencer c05d0f40ea
services: add sequence id within call for entries (#4549)
This is a new field added to the binlog proto so that we can detect when
the storage impl reorders or drops recorded entries.
2018-06-07 12:19:36 -07:00
zpencer 181aac425f
services: sync binlog proto (#4542)
https://github.com/grpc/grpc-proto/pull/20
2018-06-07 10:45:55 -07:00
Carl Mastrangelo 4c4fda3e5d
stub: remove static Method descriptors and stabilize method accessors 2018-06-05 11:19:28 -07:00
zpencer 29dba1e89f
services: allow binlog to blacklist methods (#4523)
The spec says users can specify a blacklist a method from binlogs by
saying "-package.service/method".
2018-06-04 14:11:55 -07:00
ZHANG Dapeng a0794d5a68
core: Implement channel tracing for some events
Includes 
- API plumbing for channel tracing `+ManagedChannelBuilder.maxTraceEvents()`
- trace logging for channel/subchannel creation events and name resolving events.

Trace logging for all other events will be added in subsequent PRs.
2018-06-01 07:33:37 -07:00
zpencer 43831bc0e8
services: update the binlog message truncation flag (#4508)
The truncation flag tells us whether the msg or meta was truncated.
2018-05-31 18:13:59 -07:00
Nathan Herring 8db8497ef0 services: use RE2/J
RE2/J ensures linear time matching, and as such is preferred over `java.util.regex` for safety in the mono repo. While currently safe, this conversion future-proofs any `Pattern`s in BinlogHelper, and removes the need to maintain an exception for gRPC wrt. use of regexes.
2018-05-30 12:26:02 -07:00
zpencer ff9aa39bb6
core,services: fix lints (#4504)
- Type parameters must be upper camel case
- constant case field must be final
2018-05-24 11:12:30 -07:00
zpencer 27439876f2
services,core: make BinaryLogSink visible, add setter (#4503)
* make BinaryLogSink public
* add io.grpc.BinaryLog to server/channel builder
2018-05-23 20:49:09 -07:00
zpencer 1f99fcdc82
move io.grpc.BinaryLogProvider to io.grpc.services (#4501)
Add internal accessors for ServerInterceptors and ClientInterceptors because some helpers were pkg private
Fix tests that were once creating BinaryLogProvider instances, they should now only create io.grpc.BinaryLog instances
2018-05-23 17:04:48 -07:00
zpencer c60580c669
services: use Peer.address proto (#4502)
address+ip_port is the new way; peer is deprecated
2018-05-23 16:58:15 -07:00
zpencer 869363cdb8
services: sync binarylog.proto (#4500)
Sync from grpc/grpc-proto and rerun codegen.
2018-05-23 16:17:05 -07:00
zpencer 1aec994475
io.grpc.BinaryLog is now the fully functional API (#4498)
classes in internal now use the io.grpc.BinaryLog API,
io.grpc.BinaryLogProvider is an implementation of the API and will be
moved to io.grpc.services.
2018-05-23 15:58:37 -07:00
Kun Zhang 68a462e447
noop: resolve lint warnings found at import. (#4496) 2018-05-23 11:58:19 -07:00
ZHANG Dapeng f5f560ad36
all: TimeProvider to use nanos rather than millis
This is the same practice as #2833
2018-05-21 12:44:06 -07:00
zpencer 30478d88c7
services: add temp file based binary log sink (#4404)
No need to use service provider for BinaryLogSink, it can just be an
interface that is passed into BinaryLogProviderImpl.

Add a default TempFileSink that uses the protobuf object's
writeDelimited method to write to the output stream.
Warning: TempFileSink blocks.
2018-05-20 17:23:48 -07:00
zpencer faffb09f0a
core: remove io.grpc core dependency on census (#4461)
io.grpc (core) does not strictly require census. Move the usages and
and remove census from BUILD.bazel .
2018-05-18 14:56:45 -07:00
zpencer ad370d807b
services: fix lint on toString (#4481)
BinlogHelper does not override toString so print something else that
is more descriptive.
2018-05-18 12:54:17 -07:00
zpencer 5509fd35a7
services: differentiate between tcpi_retransmits and tcpi_retrans (#4470)
These are subtly differently named but distinct fields.
2018-05-16 17:34:49 -07:00
ZHANG Dapeng 561583be14
core,services: Add ChannelTracing data object
Added `ChannelTrace` as an inner class of `Channelz`.

This is in preparation for the implementation of [Channel Tracing](https://github.com/grpc/proposal/blob/master/A3-channel-tracing.md)
2018-05-15 11:33:44 -07:00
zpencer 73fdb8716d
services: allow binlog env str to be empty or null (#4447)
Binary log objects are explicitly passed into channel and server
builders, but the configuration is something that's from the
environment variables. An unset or empty GRPC_BINARY_LOG_CONFIG
should be allowed to disable logging.

Previously, the SPI just reported itself as not available when the
conf str was not something valid.
2018-05-15 09:57:39 -07:00
zpencer 6f29b60dcf
core, services: make BinaryLog an explicit object that is passed into channels/servers (#4431)
remove SPI, io.grpc.BinaryLog is a public API that is passed into builders and must be
explicitly closed.
2018-05-07 07:41:42 -07:00
ZHANG Dapeng 46079fff8a
services: complete ChannelTrace in channelz.proto 2018-05-04 16:59:33 -07:00
Carl Mastrangelo 60a0b0c471
all: normalize copyright header 2018-05-03 14:55:21 -07:00
zpencer 722acb151a
services: BinaryLog is an overused term, rename to BinlogHelper (#4429)
This class really is an internal helper class for the implementation.
2018-05-02 18:11:21 -07:00
Carl Mastrangelo 9817281dc3
service: include go package in channelz.proto 2018-05-02 17:15:00 -07:00
Carl Mastrangelo aeb61db278
services: update channelz proto package 2018-05-02 15:16:29 -07:00
Eric Anderson 3a11a1f8b2 services: BinaryLogProviderImpl is experimental (#4420)
This class is not stable API.
2018-05-02 08:21:37 -07:00
Carl Mastrangelo d35e77869e
services: move health.proto to correct directory 2018-04-30 18:32:02 -07:00
Carl Mastrangelo 6bd5582175
services: update channelz with docs from upstream 2018-04-30 15:26:36 -07:00
Carl Mastrangelo f4f4302b85
services: remove monitoring file 2018-04-27 21:34:43 -07:00
zpencer acfb3b9851
services,core: simplify CallId generation (#4365)
BinaryLog.java is the class that is responsible for intercepting
client and server calls. It now requires a CallId to be passed
in. The BinaryLogProviderImpl is responsible for generating a
CallId and passing it in.
2018-04-27 18:32:36 -07:00
zpencer 44c79fbe5d
services: fix channelz linter warnings (#4395)
- final classes should have toString()
- fix arg name mismatch
2018-04-25 17:55:42 -07:00
Carl Mastrangelo 2027f6f81c
services: update to remote channelz.proto def 2018-04-25 17:05:58 -07:00
zpencer 1a2d076aed
core,netty,okhttp,services,testing: expose security info to channelz (#4300)
Pull the TLS info from the SSLSession object for TLS, and AltsContext for ALTS.
2018-04-25 14:38:09 -07:00
Carl Mastrangelo c8aa9f70ca
services: move channelz proto into normalized directory structure 2018-04-20 10:53:45 -07:00
zpencer f5e8ec18b7
services: avoid static initialization for BinaryLog.java (#4363)
The code that uses this will create an instance.
2018-04-19 17:51:49 -07:00
zpencer c04278ea07
services: turn channelz null futures into Status.UNIMPLEMENTED (#4346)
Then channelz GUI will take this into account. This is particularly
useful for InProcessTransport, where I have decided we do not need
special support for in channelz. The server and channel stats are
already sufficient.
2018-04-16 09:54:35 -07:00
zpencer 791a29f63b
services: socket options were erroneously ignored if socket has no stats (#4336)
This fixes listen sockets. It is ok to have no data but report socket
options.
2018-04-13 14:59:35 -07:00
zpencer 1d5ed78c8f
services: make channelz constructor static (#4334)
Right now this is inaccessible to services.
2018-04-13 14:10:50 -07:00
zpencer 62cc2b1ae3
core,netty: add NettySocketSupport to populate TcpInfo (#4306)
NettySocketSupport is responsible for making the low level calls to
get and populate the TcpInfo structure.
2018-04-05 10:36:49 -07:00
zpencer 276586a4fb
netty,services: fix param comment lint (#4304)
Fix linter complaint because comment does not match arg name.
2018-04-04 12:06:31 -07:00
zpencer b1d91b9f60
core,services: binary log should use real peer socket and call id (#4266)
The peer socket is read from TRANSPORT_ATTR_REMOTE_ADDR from the
stream attributes. We only log the peer on receive initial metadata.

The call id assumes census is available. The call ID read from the
context via SERVER_CALL_ID_CONTEXT_KEY on server side, and read from
CallOptions via CLIENT_CALL_ID_CALLOPTION_KEY on client side. The
value is copied from CONTEXT_SPAN_KEY which is set by census.

Pass around CallId with two longs, not a byte[].
2018-04-03 13:39:11 -07:00
zpencer 724e32fe57
core,netty,services: add server listen sockets to channelz proto service (#4220)
Server listen sockets differ from normal sockets in that they do not
have a remote address, do not have stats on calls started/failed/etc,
and do not have security info.
2018-04-03 10:05:47 -07:00