io.grpc has fewer dependencies than io.grpc.internal. Moving it to a
separate artifact lets users use the API without bringing in the deps.
If the library has an optional dependency on grpc, that can be quite
convenient.
We now version-pin both grpc-api and grpc-core, since both contain
internal APIs.
I had to change a few tests in grpc-api to avoid FakeClock. Moving
FakeClock to grpc-api was difficult because it uses
io.grpc.internal.TimeProvider, which can't be moved since it is a
production class. Having grpc-api's tests depend on grpc-core's test
classes would be weird and cause a circular dependincy. Having
grpc-api's tests depend on grpc-core is likely possible, but weird and
fairly unnecessary at this point. So instead I rewrote the tests to
avoid FakeClock.
Fixes#1447
Motivation:
To support TCP_USER_TIMEOUT(proposal). Nio doesn't support TCP_USER_TIMEOUT while Epoll and KQueue supports TCP_USER_TIME. Since most users/servers are using linux based system, adding Epoll is necessary. KQueue maybe supported later, but not in this PR.
To make it backward compatible for cases where channelType and eventLoop is mixed in with default and user provided object(s), we will fallback to Nio (NioSocketChannel, NioEventLoop). This ensures not breaking existing code (same as existing behavior). Users not specified both channelType and EventLoops will be affect by this Epoll change if netty-epoll is available.
In later version (possibly 1.22.0), the backward compatible behavior will be removed and it will start to throw exception since this is error prone.
Resolves#5497
## Motivation
In hierarchical `LoadBalancer`s (e.g., `XdsLoadBalancer`) or wrapped `LoadBalancer`s (e.g., `HealthCheckingLoadBalancerFactory`, the top-level `LoadBalancer` receives `Subchannel` state updates from the Channel impl, and they almost always pass it down to its children `LoadBalancer`s.
Sometimes the children `LoadBalancer`s are not directly created by the parent, thus requires whatever API in the middle to also pass Subchannel state updates, complicating that API. For example, the proposed [`RequestDirector`](https://github.com/grpc/grpc-java/issues/5496#issuecomment-476008051) includes `handleSubchannelState()` solely to plumb state updates to where they are used. We also see this pattern in `HealthCheckingLoadBalancerFactory`, `GrpclbState` and `SubchannelPool`.
Another minor issue is, the parent `LoadBalancer` would need to intercept the `Helper` passed to its children to map Subchannels to the children `LoadBalancer`s, so that it pass updates about relevant Subchannels to the children. Otherwise, a child `LoadBalancer` may be surprised by seeing Subchannel not created by it, and it's not efficient to broadcast Subchannel updates to all children.
## API Proposal
We will pass a `SubchannelStateListener` when creating a `Subchannel` to accept state updates, those updates could be directly passed to where the `Subchannel` is created, skipping the explicit chaining in the middle.
Also define a first-class object `CreateSubchannelArgs` to pass arguments for the reasons below:
1. It may avoid breakages when we add new arguments to `createSubchannel()`. For example, a `LoadBalancer` may wrap `Helper` and intercept `createSubchannel()` in a hierarchical case. It may not be interested in all arguments. Passing a single `CreateSubchannelArgs` will not break the parent `LoadBalancer` if we add new fields later.
2. This also reduces the eventual size of Helper interface, as the convenience `createSubchannel()` that accepts one EAG instead of a List is no longer necessary, since that convenience is moved into `CreateSubchannelArgs`.
```java
interface SubchannelStateListener {
void onSubchannelState(Subchannel subchannel, ConnectivityStateInfo newState);
}
abstract class LoadBalancer.Helper {
abstract Subchannel createSubchannel(CreateSubchannelArgs args);
}
final class CreateSubchannelArgs {
final List<EquivalentAddressGroup> getAddresses();
final Attributes getAttributes();
final SubchannelStateListener getStateListener();
final class Builder () {
...
}
}
```
The new `createSubchannel()` must be called from synchronization context, as a step towards #5015.
## How the new API helps
Most hierarchical `LoadBalancer`s would just let the listener from the child `LoadBalancer`s directly pass through to gRPC core, which is less boilerplate than before.
Without any effort by the parent, each child will only see updates for the Subchannels it has created, which is clearer and more efficient.
If a parent `LoadBalancer` does want to look at or alter the Subchannel state updates for its delegate (like in `HealthCheckingLoadBalancerFactory`), it can still do so in the wrapping `LoadBalancer.Helper` passed to the delegate by intercepting the `SubchannelStateListener`.
## Migration implications
Existing `LoadBalancer` implementations will continue to work, while they will see deprecation warnings when compiled:
- The old `LoadBalancer.Helper#createSubchannel` variants are now deprecated, but will work until they are deleted. They create a `SubchannelStateListener` that delegates to `LoadBalancer#handleSubchannelState`.
- `LoadBalancer#handleSubchannelState` is now deprecated, and will throw if called and the implementation doesn't override it. It will be deleted in a future release.
The migration for most `LoadBalancer` implementation would be moving the logic from `LoadBalancer#handleSubchannelState` into a `SubchannelStateListener`.
This class is used in other places than just NameResolver.Helper. It
should not be an inner class of Helper.
Strictly speaking this is an API-breaking change. However, this is
part of the service config error handling API that hasn't been done
yet. Nobody has a legitimate reason to use it.
New fields are added to the test protos according to grpc/grpc-proto#51
This test needs to be run in an environment where "pick_first" or "grpclb" with child policy "pick_first" is configured.
re2j is a fairly unnecessary dependency. Our usage of Pattern is quite limited
and isn't all that hard to do manually. Our usage would be safe to use normal
java.util.regex, but it's sort of annoying to keep re-explaining to others who
are (rightly) concerned with java.util.regex's poor pathological behavior.
We assumed that we were the only caller. Turns out there are
forwarding NameResolver too. Implementing the old override will give
them time to migrate to the new API.
Resolves#5556
Both the config and the error need to be passed to the managed channel so it can decide to keep or reject the config. Passing only the parsed object means the managed channel cannot implement the error handling gRFC.
This mainly avoids protoc from 3.7.0 which has a dependency on libatomic. Most
of our systems have libatomic, so it mostly works, but the interop docker
container does not, so building fails. Version 3.7.1 was rebuilt to avoid
needing the libatomic shared library.
This has the added benefit that Bazel is now on the same version as Gradle, as
3.7.1 included fixes for Bazel.
* added android example running under strictmode to detect grpc clear text network transport
* renamed andriod strictmode example
* use OkHttpChannelBuilder's api to feed with user's own transport executor
* bump gradle and proto-plugin version to match that in master
* add README with a simple demo image
* Update README to use relative link to helloworld
* okhttp: add verbose logging for OkHttp HTTP/2 frames to include brief frame content.
* fix small details and typo issues
* fix branch condition (it's a typo) for logging the whole buffer
* remove redudent log level checking, always not log entire buffer
* convert ByteString to String to avoid printing 'hex=' prefix
* add test for OkHttpFrameLogger for inbound http frames
* refactor OkHttpFrameLogger's help method
* add Constructor for testing and missing logHeaders
* add http/2 framelogger test for outbound side
* use ArgumentMatchers instead of Matchers
* replace assertTrue with Truth.assertThat
* assert buffer log with exact length match instead of <=
This code is highly experimental, so we can change it at will later. This PR is to go ahead and try it out.
Perfmark task calls are added to the client and server calls public APIs, recording when the calls begin and end. They use separate scope IDs (roughly these are Thread IDs) for the listener and the call due to no synchronization between them. However, they both use the same PerfTags, which allows them to be associated.
In the future, we can plumb the tag down into the stream to include deeper information about whats going on in a call.
This resolves#5523
While bumping `com.android.tools.build:gradle:3.1.2` to `3.3.0`, some other plugins/artifacts/maven repo/buildscripts have to be updated:
- gradle (wrapper) need to upgrade to 4.10.x
- protobuf gradle plugin need to bump a version compatible with gradle version.
- need add `google()` and `jcenter()` repos for android (otherwise `com.android.tools.build:aapt2:3.3.0x` and `trove4j` will not be found resp.)
- need to accept license for Android "build-tools;28.0.3" in kokoro env.
The `ThreadlessExecutor` currently used for blocking calls uses `LinkedBlockingQueue` which is relatively heavy both in terms of allocations and synchronization overhead (e.g. when compared to `ConcurrentLinkedQueue`). It accounts for ~10% of allocations and ~5% of allocated bytes per-call in the `TransportBenchmark` when using in-process transport with [stats and tracing disabled](https://github.com/grpc/grpc-java/issues/5510).
Changing to use a `ConcurrentLinkedQueue` results in a ~5% speedup of that benchmark.
Before:
```
Benchmark (direct) (transport) Mode Cnt Score Error Units
TransportBenchmark.unaryCall1024 true INPROCESS avgt 60 1877.339 ± 46.309 ns/op
TransportBenchmark.unaryCall1024 false INPROCESS avgt 60 12680.525 ± 208.684 ns/op
```
After:
```
Benchmark (direct) (transport) Mode Cnt Score Error Units
TransportBenchmark.unaryCall1024 true INPROCESS avgt 60 1779.188 ± 36.769 ns/op
TransportBenchmark.unaryCall1024 false INPROCESS avgt 60 12532.470 ± 238.271 ns/op
```
Kun Zhang
Carl Mastrangelo
Eric Anderson
Jihun Cho
Chengyuan Zhang
ZHANG Dapeng
Solomon Duskis
Akim239
Nick Hill