CodedInputStream is risk averse in ways that hurt performance when
parsing large messages. gRPC knows how large the input size is as it
is being read from the wire, and only tries to parse it once the entire
message has been read in. The message is represented as chunks of
memory strung together in a CompositeReadableBuffer, and then wrapped
in a custom BufferInputStream.
When passed to Protobuf, CodedInputStream attempts to read data out
of this InputStream into CIS's internal 4K buffer. For messages that
are much larger, CIS copies from the input in chunks of 4K and saved in
an ArrayList. Once the entire message size is read in, it is re-copied
into one large byte array and passed back up. This only happens for
ByteStrings and ByteBuffers that are read out of CIS. (See
CIS.readRawBytesSlowPath for implementation).
gRPC doesn't need this overhead, since we already have the entire
message in memory, albeit in chunks. This change copies the composite
buffer into a single heap byte buffer, and passes this (via
UnsafeByteOperations) into CodedInputStream. This pays one copy to
build the heap buffer, but avoids the two copes in CIS. This also
ensures that the buffer is considered "immutable" from CIS's point of
view.
Because CIS does not have ByteString aliasing turned on, this large
buffer will not accidentally be kept in memory even if only tiny fields
from the proto are still referenced. Instead, reading ByteStrings out
of CIS will always copy. (This copy, and the problems it avoids, can
be turned off by calling CIS.enableAliasing.)
Benchmark results will come shortly, but initial testing shows
significant speedup in throughput tests. Profiling has shown that
copying memory was a large time consumer for messages of size 1MB.
Protobuf-lite since beta-4 is now more of a fork than a subset of
protobuf-java, which may cause us problems later since lite API is not
stable. Also, lite-generated code may now depend on APIs only in
protobuf-lite, so our users must depend on the protobuf-lite runtime.
Having all our users explicitly override the dependency is bothersome to
them and can easily only expose problems only after we do a release.
So now we are doing the dependency overriding; most users should "just
work" and pick up the correct protobuf artifact. I've confirmed the
exclusion is listed in the grpc-protobuf pom and "gradle dependencies"
and "mvn dependency:tree" do not include protobuf-lite for the examples.
Vanilla protobuf users are most likely to experience any breakage, which
should detect problems more quickly since we use protobuf-java more
frequently than protobuf-lite during development.
protobuf-lite does not include pre-generated code for the well-known
protos, so users will need to generate them themselves for the moment
(google/protobuf#1889).
Note that today changing deps does not noticeably reduce the method code
for our users, since ProGuard already is stripping most classes. The
difference in output is only a reduction of 3 classes and 6 methods for
the android example.
Lite already worked by using the protobuf project, but would bring in
extra dependencies that are not intended to work with lite. Although
protobuf is not yet providing a lite package on Maven Central, we will
be able to swap to it once it is available.
There isn't any new original code in the Java portion, except for a new
overload in ProtoUtils that accepts Message instead of MessageLite.
Depending on Message in ProtoUtils allows us to support extra features
out-of-the-box without any changes to the generated code. For example,
JSON encoding could be supported in this way if Marshaller is enhanced.
However, now codegen must be aware of Lite in order to choose with Util
class to use. That is new code.
Carl Mastrangelo
Eric Anderson