Also removed warnings about protoc version matching runtime, since this
is no longer supposed to be a problem (starting with 3.0.0-beta-4) and
all our tests ran fine when using protoc 3.0.2 with protobuf runtime
3.1.0.
Fixes#2316
Highlights
==========
StatsTraceContext
-----------------
The bridge between gRPC library and Census. It keeps track of the total
payload sizes and the elapsed time of a Call. The rest of the gRPC code
doesn't invoke Census directly.
Context propagation
-------------------
StatsTraceContext carries CensusContext (and the upcoming TraceContext)
and is attached to the gRPC Context.
1. StatsTraceContext is created by ManagedChannelImpl, by calling
createClientContext(), which inherits the current CensusContext if available.
2. ManagedChannelImpl passes StatsTraceContext to ClientCallImpl, then
to the stream, then to the framer and deframer explicitly.
3. ClientCallImpl propagates the CensusContext to the headers.
1. ServerImpl creates a StatsTraceContext by implementing a new callback
method StatsTraceContext methodDetermined(MethodDescriptor, Metadata) on
ServerTransportListener.
2. NettyServerHandler calls methodDetermined() before creating the
stream, and passes the StatsTraceContext to the stream.
3. When ServerImpl creates the gRPC Context for the new ServerCall, it
calls the new method statsTraceContext() on ServerStream and puts the
StatsTraceContext in the Context.
Metrics recording
-----------------
1. Client-side start time: when ClientCallImpl is created
2. Server-side start time: when methodDetermined() is called
3. Server-side end time: in ServerStreamListener.closed(), but before
calling onComplete() or onCancel() on ServerCall.Listener.
4. Client-side end time: in ClientStreamListener.closed(), but before
calling onClonse() on ClientCall.Listener
Message sizes are recorded in MessageFramer and MessageDeframer. Both
the uncompressed and wire (possibly compressed) payload sizes are
counted.
TODOs
=====
The CensusContext created from headers on the server side should be
attached to the gRPC Context for the call. It's not done at this moment
because Census lacks the proper API to do it. It only affects tracing
and resource accounting, but doesn't affect stats functionality
protoc no longer builds in 3.0.0 because auto-download of the gmock zip
now fails. 3.0.2 has a fix to autogen:
bba446bbf2
All that was strictly necessary was to update .travis.yml and
buildscripts/, but it helps our sanity to keep the rest of the protobuf
versions in sync. Lite is left on its existing version, because it did
not see a bump of neither the java library nor the protoc plugin.
Protobuf-lite since beta-4 is now more of a fork than a subset of
protobuf-java, which may cause us problems later since lite API is not
stable. Also, lite-generated code may now depend on APIs only in
protobuf-lite, so our users must depend on the protobuf-lite runtime.
Having all our users explicitly override the dependency is bothersome to
them and can easily only expose problems only after we do a release.
So now we are doing the dependency overriding; most users should "just
work" and pick up the correct protobuf artifact. I've confirmed the
exclusion is listed in the grpc-protobuf pom and "gradle dependencies"
and "mvn dependency:tree" do not include protobuf-lite for the examples.
Vanilla protobuf users are most likely to experience any breakage, which
should detect problems more quickly since we use protobuf-java more
frequently than protobuf-lite during development.
protobuf-lite does not include pre-generated code for the well-known
protos, so users will need to generate them themselves for the moment
(google/protobuf#1889).
Note that today changing deps does not noticeably reduce the method code
for our users, since ProGuard already is stripping most classes. The
difference in output is only a reduction of 3 classes and 6 methods for
the android example.
partially resolving #1469
The added option for java_plugin `enable_deprecated` is `true` by default in `java_plugin.cpp`, so the generated code for `TestService.java` (`compiler/build.gradle` not setting this option) has all deprecated interfaces and static bindService method.
`./build.gradle` and `examples/build.gradle` set this option explicitly to `false`, so all the other generated classes do not have deprecated code.
Will set `enable_deprecated` to `false` by default in future PR when we are ready.
This allows us to play with zero-copy and proto3 support for lite.
Unfortunately, it introduced some warnings, so deprecated warnings are
now ignored for benchmarks and interop-testing.
It converts Google Auth Library Credentials to CallCredentials, and
supersedes ClientAuthInterceptor, which is now deprecated.
Also swaps out the ClientAuthInterceptor implementation.
Caveat: This in fact changes ClientAuthInterceptor's behavior. Before
this change, if multiple ClientAuthInterceptors were attached, their
effects would be additive. After this change, only the last executed one
would take effect, and it would also overwrite the CallCredentials set
in CallOptions. We don't think it's an issue, since other languages also
only allow one call credentials to be attached to an RPC.
Bumping tcnative was not required to be done at the same time as Netty.
But the new tcnative should now work correctly on Windows and be smaller
on Linux.
This now catches a few more places we needed -Xlint:-options.
InProcessSocketAddress is technically already in our stable API, so I
maintained its current serialVersionUID.
This improves our documentation for the gradle protobuf plugin, as its
version is dependent on the gradle version.
Gradle now has the --tests flag, performance improvements, and support
for OpenPGP subkeys.
The internal package is very unstable, so any users of it are not
compatible with alternative versions of grpc-core. The same is true for
HTTP/2 support in Netty. This primarily helps Maven users; Gradle
appears to ignore the distinction between '$version' and '[$version]'
and does not trigger a build failure.
Projects like grpc-stub don't depend on internal, so they aren't using
the version pinning. The whitelist is a bit prone to go out-of-date, but
introducing grpc-api would be the long-term solution.
Fixes#1459
When running :grcp-interop-testing:test, Travis has been hanging on OS X
or flaking on Linux with:
> Process 'Gradle Test Executor 4' finished with non-zero exit value 137
Exit code 137 indicates SIGKILL (128 + 9) and is most likely caused by
the JVM being killed by the kernel's OOM killer.
The limit in .travis.yml is 2x what was necessary to do a parallel
build. The main test memory limit in build.gradle is well above 16m
which is necessary for the tests. Interop-testing is well above 64m
which is necessary for interop-testing, but we use 1.5g to help prevent
timeouts on Travis.
Protobuf and protobuf-nano each have one tests that decodes a proto >64M
in size, which prevents them from running with less than 512m and 768m,
respectively.
The jsonp dependency string is no longer shared because it was only used
in one place and someone trying to compile the examples using a new
build.gradle will need to add that dependency. It was a bit complex to
follow how libraries.jsonp worked and it wasn't really adding any value
in this particular case.
Eric Gribkoff
Eric Anderson
Jakob Buchgraber
Kun Zhang
Carl Mastrangelo
Kun Zhang
Naveen Reddy Chedeti
ZHANG Dapeng
Louis Ryan
buchgr
nmittler
Matt Hildebrand