Commit Graph

6138 Commits

Author SHA1 Message Date
Eric Anderson 43e06372ec kokoro: Build android-interop-testing and binder separately
This avoids an OOM. We could probably configure things to run them
together, but that generally takes trial-and-error.
2023-11-27 10:15:42 -08:00
Eric Anderson a70dddd3dc kokoro: Use Firebase's arm virtual Android devices
They are a lot faster. Instead of 1-3 minutes of test execution, I now
see 2-22 seconds. There still may be 3 minutes of overhead for the
gcloud command to complete, but the reduction is noticable in the total
execution time. And it seems the tests are actually being run, as there
is some flakiness. The flakiness appears to be at a lower rate.

The script was slightly reorganized to make it easier to copy commands
to run locally.
2023-11-27 08:22:52 -08:00
Eric Anderson e65b4eb5a2 all: Add grpc-inprocess
This should have been added when inprocess was moved out of core. This
includes inprocess in the Javadoc and code coverage reporting.
2023-11-25 08:01:18 -08:00
Eric Anderson 0567ab5773 android-interop: Add testing for API levels 28-30
Note that this uses Pixel2 and Pixel3. Also swap 26-27 from Nexus6P to
Pixel2. We tend to prefer the latest (virtual) device for each API
level.

The current models and their supported API levels are available via:
```
gcloud firebase test android models list --filter=form=virtual
```

Pixel2.arm supports 31-32, but is beta, so I didn't swap to it. It also
supports the preview 33.
2023-11-22 14:49:19 -08:00
Eric Anderson 57dae6e20c inprocess: Add missing anonymous address as supported 2023-11-22 13:55:10 -08:00
Eric Anderson 860b5cb1f0 api: Deprecate LoadBalancer.EMPTY_PICKER
FixedResultPicker is our preferred approach.
2023-11-21 12:57:06 -08:00
Eric Anderson 3b92333890 core: Detect NameResolverProviders passed as Factories
This may help some to move closer to Providers. It especially helps
cases where `NameResolverFactory`s aren't returning `InetSocketAddress`,
as it allows them to override `getProducedSocketAddressTypes()`, which
will now fail starting in 15fc70be.
2023-11-21 10:56:55 -08:00
Eric Anderson c6d448e43f Remove getSubjectDN(), which is deprecated in Java 17 2023-11-20 10:06:19 -08:00
Eric Anderson 0987dc401c
netty: Add option to limit RST_STREAM rate
The behavior purposefully mirrors that of Netty's
AbstractHttp2ConnectionHandlerBuilder.decoderEnforceMaxRstFramesPerWindow().
That API is not available to our code as we extend the
Http2ConnectionHandler, but we want our API to be able to delegate to
Netty's in the future if that ever becomes possible.
2023-11-17 15:10:52 -08:00
Terry Wilson da4618ace1
util: Remove shutdown subchannels from OD tracking (#10683)
An OutlierDetectionLoadBalancer child load balancer might decided to
shut down any subchannel it is tracking. We need to make sure that those
subchannels are removed from the outlier detection tracker map to avoid
a memory leak.
2023-11-17 15:04:45 -08:00
John Cormie e89389a50a
api: Clarify that NameResolver can only be started once (#10681) 2023-11-17 07:44:24 -08:00
John Cormie bdb0c72d45
binder: Beef up hasPermissions security policy Javadoc (#10650)
Try to manage the fact that runtime permissions could be granted externally by the user after a hasPermissions() SecurityPolicy check has already been made on a transport.
2023-11-15 10:05:31 -08:00
Eric Anderson 32d85d8309 Start 1.61.0 development cycle 2023-11-15 07:41:12 -08:00
Benjamin Fedorka 2b65e660c0
netty: disable huffman coding in headers (#10563)
Huffman in the datacenter doesn't add much value in the common cases.  It could be useful to turn on huffman based on the connection latency (say, >10ms means "assume cross-datacenter") but the Netty API doesn't lend it to that. The savings here aren't huge and it is expensive; the table provides the biggest savings.
2023-11-14 11:09:38 -08:00
Patrick Koenig 90e76a1b4a
Implement Http2Headers.isEmpty (#10663) 2023-11-14 10:39:16 -08:00
Eric Anderson 0299788807 util: Make grpc-core an implementation dependency
This prevents grpc-core from being exposed on the classpath when
compiling code using grpc-util.
2023-11-13 16:52:42 -08:00
Eric Anderson 6257c59566
benchmarks: Remove usage of deprecated StreamObservers 2023-11-13 16:29:38 -08:00
yifeizhuang ae62785e0b
xds: fix ring hash childLB acceptResolvedAddress not in syncContext (#10664) 2023-11-13 14:58:30 -08:00
Vindhya Ningegowda d5544bbb02
opentelemetry : Implement metrics (#10593)
Adds a new module grpc-opentelemetry that integrates OpenTelemetry and focuses on metrics.

OpenTelemetry APIs are used for instrumenting metrics collection. Users are expected to provide SDK with implementations.

If no SDK is passed, by default gRPC uses OpenTelemetry.noop().
2023-11-13 10:58:16 -08:00
Eric Anderson 84baad12fc inprocess: Remove InProcessNameResolver
It was introduced in 15fc70be but unused. It could be "used" from
inprocess: targets, but the in-process transport wasn't registered, so
would fail.

We do want an in-process name resolver, but we need to agree no the URI
format cross-language before we introduce it.
2023-11-10 15:48:30 -08:00
Eric Anderson f2fc717749
testing: Add FakeNameResolverProvider.getProducedSocketAddressTypes (#10655)
This fixes BinderChannelSmokeTest, broken by 15fc70be.
2023-11-10 09:21:52 -08:00
Larry Safran dfdd50bc79
xds:Make Ring Hash LB a petiole policy (#10610)
* Update picker logic per A61 that it no longer pays attention to the first 2 elements, but rather takes the first ring element not in TF and uses that.
---------
Pulled in by rebase:
Eric Anderson  (android: Remove unneeded proguard rule 44723b6)
Terry Wilson (stub: Deprecate StreamObservers b5434e8)
2023-11-09 13:46:52 -08:00
Terry Wilson 0346b40e4e
stub: Deprecate StreamObservers (#10654)
This class is of questionable utility and generally not used.
2023-11-08 15:00:32 -08:00
Eric Anderson 3ca3b9975e
android: Remove unneeded proguard rule
The methods other than forTarget() were no longer used after 493af030.
forTarget() was no longer used after cda0e9d9.
2023-11-08 14:33:48 -08:00
Jeff Davidson 45ffb06f00 Bump compileSdkVersion and finalize setAllowActivityStarts API.
Per https://developer.android.com/about/versions/14#beta-3, Android U has reached the platform stability milestone which means that all external APIs are finalized.

We can thus bump the compileSdkVersion to 34 (U) and begin using APIs added there. We leave targetSdkVersion unchanged for now to avoid the broader evaluation of whether deeper changes may be necessary as part of the upgrade; this simply allows compile-time access to newer APIs without changing runtime behavior.

See b/274061424
2023-11-07 08:01:44 -08:00
Eric Anderson 7ba30f679a Bump Netty to 4.1.100.Final 2023-11-06 13:05:22 -08:00
Sergii Tkachenko 8b4b14ac40
core: add description to Status.UNKNOWN in ServerImpl's #internalClose (#10643)
This is currently the only place where we return Status.UNKNOWN with no description, which makes is harder to debug and differentiate from statuses originated from non-grpc sources.

This PR enriches ServerImpl's #internalClose `Status.UNKNOWN` with description `Application error processing RPC`.
2023-11-06 12:31:30 -08:00
sanjaypujare 15fc70be2a
core, netty, okhttp: implement new logic for nameResolverFactory API in channelBuilder (#10590)
* core, netty, okhttp: implement new logic for nameResolverFactory API in channelBuilder
fix ManagedChannelImpl to use NameResolverRegistry instead of NameResolverFactory
fix the ManagedChannelImplBuilder and remove nameResolverFactory

* Integrate target parsing and NameResolverProvider searching

Actually creating the name resolver is now delayed to the end of
ManagedChannelImpl.getNameResolver; we don't want to call into the name
resolver to determine if we should use the name resolver.

Added getDefaultScheme() to NameResolverRegistry to avoid needing
NameResolver.Factory.
---------

Co-authored-by: Eric Anderson <ejona@google.com>
2023-11-03 09:57:59 -07:00
Terry Wilson 9888a54abd
lb: acceptResolvedAddresses() to return Status (#10636)
Instead of a boolean, we now return a Status object. Status.OK
represents accepted addresses and other non-acceptance. This allows the
LB to provide more information about why a set of addresses were not
acceptable.

The status will later be sent to the name resolver as well to allow it
to also better react to to bad addresses.
2023-11-03 09:02:46 -07:00
erm-g 4fbefe9f0f
alts: Add propagation of original exception for AltsHandshakerStub (#10644)
This PR addresses a bug which is a part of #9058. It propagates the
original exception caught at AltsHandshakerStub.Reader.onError().
2023-11-03 07:50:52 -07:00
Sergii Tkachenko a294b27d52
core: Deprecate ForwardingChannelBuilder (#10587)
Deprecate `ForwardingChannelBuilder` in favor of `ForwardingChannelBuilder2`.
2023-11-02 10:58:20 -07:00
Eric Anderson 073ca92e24
alts: Re-enable deprecation warnings (#10640)
google/protobuf#4046 was fixed a long time ago. Recently noticed when
working on ForwardingChannelBuilder(2).
2023-11-02 10:09:09 -07:00
Sergii Tkachenko 302f756449
interop-testing: Improve ChannelAndServerBuilderTest readability (#10641)
Add javadoc and code readability refactoring
2023-11-02 10:06:40 -07:00
Terry Wilson 84ddcb04e0
interop-testing: Fix total calls log statement (#10639)
We don't want local specific formatting of the number, so let's just
concatenate.
2023-10-31 14:20:22 -07:00
Terry Wilson d3729faea6
interop-testing: Fix disabling of stream tracers (#10638)
The client stream tracers need to be disabled before setUp() is called.
2023-10-31 14:17:51 -07:00
Mateus Azis b6947de95a
Introduce the AsyncSecurityPolicy class. (#10622)
This is the async variant of SecurityPolicy, allowing callers to implement security checks based on slow calls that aren't meant to block the gRPC thread.

BinderTransportSecurity.checkAuthorization **STILL** blocks while attempting to resolve the ListenableFuture<Status> it gets from the policy object. That will still be adressed in a follow-up.

Relate issue: #10566
2023-10-26 19:14:39 +02:00
Terry Wilson cd810c5284
protobuf: Stabilize ProtoUtils.metadataMarshaller() (#10628)
This method has been around since 2018 and is now considered stable.
2023-10-25 15:38:35 -07:00
Terry Wilson 7bd2e5b59c
protobuf-lite: ProtoLiteUtils experimental comment (#10627)
Add a comment clarifying that the ProtoLiteUtils class will remain
experimental.
2023-10-25 15:37:49 -07:00
Terry Wilson 18a5f8a90b
interop-testing: Stress test log for total calls (#10626)
Allows stress tests to make assertions based on the total amount of
calls made by the stress test client.
2023-10-25 14:19:32 -07:00
Yannick Epstein 86835ae817
xds: Log ORCA UNIMPLEMENTED error to subchannel logger (#10625)
Logging to the static instance would result in application logs filling
up if the Orca service is not available.
We'd like to have the logging on the subchannelLogger, so we make it
visible on demand.

Also succeed Orca logging test if log message present. Using
contains over containsExactly seems more reasonable.
2023-10-25 10:19:37 -07:00
Terry Wilson 6cf6380812
releasing: add git command for pushing gh-pages (#10621) 2023-10-20 15:16:56 -07:00
Terry Wilson 2bec8cdf1d
Update README etc to reference 1.59.0 (#10620) 2023-10-20 14:29:26 -07:00
Mateus Azis 8966680015
binder: Allow async/slow implementations of authorization checks (#10596)
Allow a security policy to returns a `ListenableFuture<Status>` that
callers can implement to perform slower auth checks (like network
calls, disk I/O etc.) without necessarily blocking the gRPC calling
thread.

Partially addresses: https://github.com/grpc/grpc-java/issues/10566
2023-10-20 13:53:12 -07:00
Terry Wilson ccf9101828
interop-testing: allow disabling of stream tracers (#10609)
This adds the ability to disable the installation of stream tracers in
each test call in AbstractInteropTest. The tracers are stored in a list
and used to make assertions by normal tests, but a long running stress
test will accumulate too many entries in this list and the heap gets
wuickly filled up.
2023-10-17 08:37:15 -07:00
Larry Safran 0d39bf5018
lb:Implement LeastRequestLB as a petiole policy and restore RR and WRR (#10584)
* Change LeastRequest, Round Robin and WeightedRoundRobin into petiole policies
2023-10-16 16:40:20 -07:00
Terry Wilson fc03f2be9d Remove extra parenthesis 2023-10-09 17:17:50 -07:00
Terry Wilson 501246374c Line up usage and add assertions 2023-10-09 17:17:50 -07:00
Terry Wilson 21c287f7c3 interop-testing: Custom credentials in stress test
Adds support for specifying either google default or compute engine
"custom" credentials on the command line. This works like it does in
TestServiceClient.

Another feature from TestServiceClient is also included - the channel
builder is created differently when the server port is 0. This avoids
some ipv6 address parsing shenanigans.
2023-10-09 17:17:50 -07:00
Terry Wilson e488b67d99 gradle: Downgrade mockito to 4.4.0
Starting from version 4.5.0 Mockito uses the Java stream APIs, which are
not available on Android API levels < 24. This has been causing the
Android integration tests for API levels 21, 22 and 23 to fail.
2023-10-09 13:29:12 -07:00
amirhadadi 8c0545564f
Avoid flushing headers when the server returns a single response (#9314)
* core,netty: avoid flushing the headers on the server side when the server has a single response.
2023-10-09 09:43:10 -07:00