Commit Graph

1973 Commits

Author SHA1 Message Date
Eric Anderson 847ea7cfc9 Upgrade Mockito to 3.12.4
MockitoAnnotations.initMocks() is deprecated.
2023-05-08 16:39:42 -07:00
Terry Wilson 3c01bfe77f
core: Optional address shuffle in PickFirstLoadBalancer (#10110)
If provided with the new PickFirstLoadBalancerConfig,
PickFirstLoadBalancer will shuffle the list of addresses it receives
from the name resolver.

PickFirstLoadBalancerProvider will now support the new config
if enabled by an env variable.
2023-04-28 15:50:18 -07:00
Terry Wilson fbc8679f7a
core: Sticky TRANSIENT_FAILURE in PickFirstLoadBalancer (#10106)
When the subchannel is transitioning from TRANSIENT_FAILURE to either
IDLE or CONNECTING we will not update the LB state. Additionally, if
the subchannel becomes idle we request a new connection so that the
subchannel will keep on trying to establish a connection.
2023-04-27 15:43:27 -07:00
Eric Anderson 8ca99f661d core: Pass Locality to toLowerCase() in GrpcUtil
We pretty much never want to use the system locality, across all our
code.
2023-04-24 10:53:12 -07:00
Eric Anderson 27edab007d Start 1.56.0 development cycle 2023-04-21 15:56:00 -07:00
Larry Safran 7255c8dfa8
retries:Remove early commit for transparent retries with none remaining. (#10066)
* retries:Remove early commit for transparent retries when no retries or no hedging remain.

Fixes #10011
2023-04-19 13:42:59 -07:00
Carl Mastrangelo 4e3ee4471e
all: use PerfMark.traceTask (#9950)
* all: use PerfMark.traceTask

* make linter happy
2023-04-18 10:46:54 -07:00
yifeizhuang 14ba959545
census: add delayed name resolution tracing annotation (#10044) 2023-04-14 11:44:50 -07:00
Terry Wilson f2b976266e Revert "core: ForwardingLoadBalancer to forward acceptResolvedAddresses() (#10050)"
This reverts commit 086edf0f20.
2023-04-14 10:12:33 -07:00
Terry Wilson 086edf0f20
core: ForwardingLoadBalancer to forward acceptResolvedAddresses() (#10050) 2023-04-13 18:23:11 -07:00
yifeizhuang fc4410f159
api, census: add new pendingStreamCreated on clientStreamTracer and new tracer annotation (#10014) 2023-04-11 13:49:33 -07:00
Eric Anderson 8d98e5ff7f core: Fix NPE race during hedging
The problem was one hedge was committed before another had drained
start(). This was not testable because HedgingRunnable checks whether
scheduledHedgingRef is cancelled, which is racy, but there's no way to
deterministically trigger either race.

The same problem couldn't be triggered with retries because only one
attempt will be draining at a time. Retries with cancellation also
couldn't trigger it, for the surprising reason that the noop stream used
in cancel() wasn't considered drained.

This commit marks the noop stream as drained with cancel(), which allows
memory to be garbage collected sooner and exposes the race for tests.
That then showed the stream as hanging, because inFlightSubStreams
wasn't being decremented.

Fixes #9185
2023-04-11 13:10:41 -07:00
Terry Wilson 3332199106
Start 1.55.0 development cycle (#9939) 2023-03-07 18:14:31 -08:00
Benjamin Peterson ae6c506f96
all: fix build with errorprone 2.18 (#9886)
errorprone cannot be updated past 2.10 because later versions do not support Java 8.

Fixes https://github.com/grpc/grpc-java/issues/9916.
2023-03-01 13:45:18 -08:00
yifeizhuang 8d12baa447
xds: add weighted round robin LB policy support (#9873) 2023-02-27 10:34:51 -08:00
Sergey Matyukevich 67d6600f71
core: add logger to OutlierDetectionLoadBalancer (#9880) 2023-02-14 13:27:21 -08:00
yifeizhuang 3d4d46d9ff
Revert "core: add subchannel list LB commons utility function, refactor for WRR (#9875)" (#9892)
This reverts commit 5a2adcc7c6.
2023-02-13 17:28:24 -08:00
Terry Wilson d42678a3bf
core: fix outlier detection default ejection time (#9889) 2023-02-13 14:40:31 -08:00
Terry Wilson fcb5c54e4b
Move name resolution retry from managed channel to name resolver (take #2) (#9812)
This change has these main aspects to it:

1. Removal of any name resolution responsibility from ManagedChannelImpl
2. Creation of a new RetryScheduler to own generic retry logic
     - Can also be used outside the name resolution context
3. Creation of a new RetryingNameScheduler that can be used to wrap any
   polling name resolver to add retry capability
4. A new facility in NameResolver to allow implementations to notify
   listeners on the success of name resolution attempts
     - RetryingNameScheduler relies on this
2023-02-03 18:23:32 -08:00
yifeizhuang 5a2adcc7c6
core: add subchannel list LB commons utility function, refactor for WRR (#9875) 2023-02-03 17:53:51 -08:00
Kun Zhang 04afea0fbd
test: new JDK fixed InetSocketAddress toString so we are updating our test (#9876)
The change was introduced in JDK 14 (f8440c33d2) so JDK 17 for us to care about.

Fixes internal issue: b/261217361
2023-02-02 21:35:24 -08:00
yifeizhuang 4876996d84
Start 1.54.0 development cycle (#9857) 2023-01-25 10:56:58 -08:00
Eric Anderson 44847bf4e9
Upgrade JUnit to 4.13.2
ExpectedException is deprecated, so I fixed the new warnings. However,
we are still using ExpectedException many places and had previously
supressed the warning. See
https://github.com/grpc/grpc-java/issues/7467 . I did not fix those
existing instances that had suppressed the warning, since it is
unrelated to the upgrade and we have been free to fix them at any time
since we dropped Java 7.
2023-01-20 09:33:11 -08:00
Eric Anderson 5a2c94bca1 core: Free unused MessageProducer in RetriableStream
This prevents leaking message buffers.

Fixes #9563
2023-01-20 07:48:35 -08:00
pandaapo 0f4b767660
core: Add grpc-previous-rpc-attempts to the initial response metadata (#9686)
Fixes #9641
2023-01-17 15:33:45 -08:00
Eric Anderson 62d924745d core: Remove unused helper field from SubchannelImpl 2023-01-17 13:06:00 -08:00
Eric Anderson 89b823c550 core: Synchronize access to retry's throttle
Use a volatile to publish the value even though it is final. TSAN
ignores the final aspect of a field, which is fair since another thread
may not see the parent's pointer become updated and use a stale value.
The lack of synchronization was clearly against lastServiceConfig's
documentation.

Fixes #9267
2023-01-13 11:19:15 -08:00
yifeizhuang 2b9bd6cdac
core: delay retriable stream master listener close until all sub streams are closed (#9754)
This helps to prevent retryable stream from using calloptions.executor when it shouldn't, e.g. call is already notified closed. It is done by delaying notifying upper layer (through masterListener).
2023-01-12 14:51:13 -08:00
Chris Povirk c0de130ac6 Change ArgumentCaptor.forClass(null) to ArgumentCaptor.forClass(Foo.class)
The former will [soon throw NullPointerException][notnull].

[notnull]: fe1cb2de09 (diff-8d274a9bda2d871524d15bbfcd6272bd893a47e6b1a0b460d82a8845615f26daR31)
2023-01-09 07:15:23 -08:00
Eric Anderson d17a2db4bd Upgrade to Checkstyle 8.28
Trying to upgrade Gradle to 7.6 improved the checkstyle plugin such that
it appears to have been running in new occasions. That in turn exposed
us to https://github.com/checkstyle/checkstyle/issues/5088. That bug was
fixed in 8.28, which also fixed lots of other bugs. So now we have
better checking and some existing volations needed fixing. Since the
code style fixes generated a lot of noise, this is a pre-fix to reduce
the size of a Gradle upgrade.

I did not upgrade past 8.28 because at some point some other bugs were
introduced, in particular with the Indentation module. I chose the
oldest version that had the particular bug impacting me fixed. Upgrading
to this old-but-newer version still makes it easier to upgrade to a
newer version in the future.
2023-01-05 17:07:04 -08:00
Eric Anderson 7c73baa450 Revert "Move name resolution retry from managed channel to name resolver. (#9758)"
This reverts commit 43bc578f20. It breaks
API without stair-stepping and needs to be tweaked architecturally.
2022-12-20 08:35:31 -08:00
Terry Wilson 43bc578f20
Move name resolution retry from managed channel to name resolver. (#9758)
This change has these main aspects to it:

1. Removal of any name resolution responsibility from ManagedChannelImpl
2. Creation of a new RetryScheduler to own generic retry logic
     - Can also be used outside the name resolution context
3. Creation of a new RetryingNameScheduler that can be used to wrap any
   polling name resolver to add retry capability
4. A new facility in NameResolver to allow implementations to notify
   listeners on the success of name resolution attempts
     - RetryingNameScheduler relies on this
2022-12-16 15:30:57 -08:00
Larry Safran ccb5d945c0
Start 1.53.0 development cycle (#9757) 2022-12-14 18:07:57 -08:00
Eric Anderson b51cd9fd99
core: Replace AtomicInteger.updateAndGet with compareAndSet
updateAndGet is only available in API level 24+. It is unclear why
AnimalSniffer didn't detect this.

This was noticed when doing the import, but the Android CI has also been
failing because of it.
2022-12-01 11:09:40 -08:00
Larry Safran c145473756
Improve error message when deadline is exceeded, making it clear which deadline (Context or CallOptions) was exceeded and making the grammar clearer. (#9713) 2022-11-30 13:58:57 -08:00
yifeizhuang b593871801
core: fix RejectedExecutionException in Retriable Stream (#9626)
Add big negative integer to pending stream count when cancelled. The count is used to delay closing master listener until streams fully drained.
Increment pending stream count before creating one. The count is also used to indicate callExecutor is safe to be used. New stream will not be created if big negative number was added, i.e. stream cancelled. New stream is created if not cancelled, callExecutor is safe to be used, because cancel will be delayed.
Create new streams (retry, hedging) is moved to the main thread, before callExecutor calls drain.
Minor refactor the masterListener.close() scenario.
2022-11-22 13:04:06 -08:00
RapperCL 096898a46e
ManagedChannelImpl.SubchannelImpl args check bug (#9651)
Previously it could trigger NPE without the string saying which argument
was null.
2022-11-10 16:15:13 -08:00
Sergii Tkachenko eb1e5a11c6 Start 1.52.0 development cycle 2022-11-07 11:56:25 -08:00
Terry Wilson ba182c3e02
core: round_robin to use acceptResolvedAddresses()
Switching to new API to allow eventual deprecation of
handleResolvedAddresses()
2022-11-03 12:40:05 -07:00
yifeizhuang 7007d78b71
doc: improve java doc ManagedChannelOrphanWrapper (#9636) 2022-10-20 11:16:53 -07:00
Eric Anderson 5b62d22802 Start 1.51.0 development cycle 2022-09-26 16:37:56 -07:00
Terry Wilson 2289956ec7
core: outlier_detection LB to use acceptResolvedAddresses() (#9557)
Switch OutlierDetectionLoadBalancer to implement
acceptResolvedAddresses() to allow for the eventual deprecation of
handleResolvedAddresses().
2022-09-19 20:20:08 -07:00
yifeizhuang a3c1d7711f
okhttp: add okhttpServerBuilder permitKeepAliveTime() and permitKeepAliveWithoutCalls() for server keepAlive enforcement (#9544) 2022-09-15 16:08:55 -07:00
Terry Wilson 15033caf1c
core: pick_first LB to use acceptResolvedAddresses() (#9548) 2022-09-15 14:21:10 -07:00
Terry Wilson 341fea8996
core: Convert inappropriate call cred errors (#9543)
If a CallCredentials implementation returns an error that is not
appropriate to propagate from the control plane to the data plane, we
convert it to an INTERNAL error. This makes the inappropriate control
plane behavior to be discoverable in the logs.

https://github.com/grpc/proposal/blob/master/A54-restrict-control-plane-status-codes.md
2022-09-13 13:36:40 -07:00
Terry Wilson 944cbf84ed
core: Comment on "inappropriate" service config errors (#9542)
https://github.com/grpc/proposal/blob/master/A54-restrict-control-plane-status-codes.md
2022-09-13 09:57:55 -07:00
Kun Zhang 38b338228a
core: add getCallOptions() to CallCredentials.RequestInfo (#9538)
This is needed internally (b/134067789#comment20).
2022-09-13 08:55:05 -07:00
Terry Wilson 9853a0c463
core: Don't delegate inappropriate ConfigSelector errors (#9536)
In case a control plane returns an "inappropriate" response code, it is converted to INTERNAL to highlight the bug in the control plane.

https://github.com/grpc/proposal/blob/master/A54-restrict-control-plane-status-codes.md
2022-09-12 13:17:16 -07:00
Terry Wilson 84d0b0474f
core: Replace inappropriate picker result status codes (#9530)
Certain status codes are inappropriate for a control plane to be
returning and indicate a bug in the control plane. These status codes
will be replaced by INTERNAL, to make it clearer to see that the control
plane is misbehaving.
2022-09-09 15:46:28 -07:00
Chris Povirk 6bafca93a7
core: Use real ByteBuffer instead of mock (#9523)
My motivation for making this change is that [`ByteBuffer` is becoming
`sealed`](https://download.java.net/java/early_access/loom/docs/api/java.base/java/nio/ByteBuffer.html)
in new versions of Java. This makes it impossible for Mockito's
_current_ default mockmaker to mock it.

That said, Mockito will likely [switch its default
mockmaker](https://github.com/mockito/mockito/issues/2589) to an
alternative that _is_ able to mock `sealed` classes. However, there are
downside to that, such as [slower
performance](https://github.com/mockito/mockito/issues/2589#issuecomment-1192725206),
so it's probably better to leave our options open by avoiding mocking at
all.

And in this case, it's equally easy to use real objects.

As a bonus, I think that real objects makes the code a little easier to
follow: Before, we created mocks that the code under test never
interacted with in any way. (The code just passed them through to a
delegate.) When I first read the tests, I was confused, since I assumed
that the mock we were creating was the same mock that we then passed to
`verify` at the end of the method. That turned out not to be the case.
2022-09-07 15:50:31 -07:00
Terry Wilson 2ee65a9c26
core: Don't forward acceptResolvedAddresses() (#9511)
Forwarding acceptResolvedAddresses() to a delegate in ForwardingLoadBalancer can cause problems if an extending class expects its handleResolvedAddresses implementation to be called even when a client calls handleResolvedAddresses(). This would not happen as ForwardingLoadBalancer would directly send the call to the delegate.
2022-09-06 13:06:48 -07:00
Larry Safran c27f2d5a57
[core,api,auth: Choose executor based on need for thread affinity (#9504)
* core,api,auth: Choose the callOptions executor when applying request metadata to credentials during newStream based upon whether AppEngineCredentials are being used as they require a specific thread to do the processing.

Add an interface to differentiate whether the specific thread is needed.

Fixes b/244209681
2022-09-02 21:55:38 -07:00
Terry Wilson 1f33fe6383
core: Assure that context cancellationCause is set (#9501)
core: Assure that context cancellationCause is set

Makes sure that whenever a context is in a cancelled state, we also have
a cancellationCause.
2022-08-31 10:43:22 -07:00
Terry Wilson 4b4cb0bd3b
api,core: Add LoadBalancer.acceptResolvedAddresses() (#9498)
Introduces a new acceptResolvedAddresses() method in LoadBalancer that
is to ultimately replace the existing handleResolvedAddresses(). The new
method allows the LoadBalancer implementation to reject the given
addresses by returning false from the method.

The long deprecated handleResolvedAddressGroups is also removed.
2022-08-31 08:36:50 -07:00
Eric Anderson 8dbff5ba6a core: DelayedClientCall should propagate context
This was already being done for the Listener, but it was missed for the
ClientCall draining itself. That's not too surprising, since very few
things should be looking at the context in that path. We don't care too
much about this client call case, but if the context _does_ end up
mattering it could be painful to debug.

Fixes #9478
2022-08-30 16:42:46 -07:00
Terry Wilson db05295fde
core: Enable outlier detection unit test (#9493)
Implements a fake load balancer with round robin like behavior in order
to test an outlier detection scenario where a subchannel goes from
having multiple associated addresses to one.
2022-08-29 15:23:41 -07:00
yifeizhuang 96b9fc1be4 fix 2022-08-25 12:53:11 -07:00
yifeizhuang 221ee494d9 move max connection idle manager to core 2022-08-25 12:53:11 -07:00
Terry Wilson d3331d953d
core: outlier detection max ejection logic update (#9492)
Stop further ejection if the ejection percentage is lesser than or equal
to the maximum ejection percentage. Previously this was only done when
the current ejection percentage was lesser than the maximum.
2022-08-25 10:48:28 -07:00
Terry Wilson 70bc7470c6
core: outlier detection to min host request volume (#9490)
When doing the failure percentage algorithm, ejections should be skipped
if there are not enough addresses that meet the minimum volume
requirement.
2022-08-24 15:49:01 -07:00
Terry Wilson a74f82ac26
core: Update outlier detection max ejection logic. (#9489)
Instead of strictly enforcing the max ejection percentage setting,
allow one additional ejection past the maximum. This quarantees at least
one ejection and matches Envoy proxy behavior.
2022-08-24 14:44:28 -07:00
Terry Wilson b16e0e34a4
core: [outlier detection] eject/uneject states to delegate listener. (#9469) 2022-08-22 11:34:50 -07:00
Terry Wilson fbefdf5ea7
core: [outlier detection] Get addresses from subchannel args. (#9468) 2022-08-22 11:12:51 -07:00
Terry Wilson f5670b39c4
core: OutlierDetectionLoadBalancer to pass child LB config. (#9467) 2022-08-22 10:29:57 -07:00
Terry Wilson 81abb21e7f
xds: Configure outlier detection. (#9456)
Enables the new OutlierDetectionLoadBalancer when outlier detection is enabled
in the xDS cluster configuration.
2022-08-18 16:06:17 -07:00
Terry Wilson 128688ae4d
Outlier detection load balancer (#9447)
New outlier detection load balancer.

Tracks all RPC results to the addresses it is configured with and periodically attempts
to detect outlier. It wraps a child load balancer from which it hides any addresses that
are deemed outliers.

As specified in gRFC A50: gRPC xDS Outlier Detection Support:
https://github.com/grpc/proposal/blob/master/A50-xds-outlier-detection.md
2022-08-18 08:46:30 -07:00
Larry Safran 01d5bd47cb
Cleanup some of the warnings across the code base (#9445)
No logic changes, just cleans up warnings to make spotting real problems easier.

Remove "public" declarations on interfaces
Remove duplicate semicolons (Java lines ending in ";;")
Remove unneeded import
Change non-javadoc comment to not start with "/**"
Remove unneeded explicit type declarations from generics
Fix broken javadoc links
2022-08-15 11:06:31 -07:00
Eric Anderson 2b50e405b1 core: Remove LB refreshNameResolver check
It's been 17 months since the check was introduced, which is plenty for
the migration. Leaving ignoreRefreshNameResolutionCheck() in-place to
let users delete their call sites. We'll remove the method after a few
releases.

Fixes #9409
2022-08-10 10:26:15 -07:00
Terry Wilson 01aff58178
Start 1.50.0 development cycle (#9433) 2022-08-10 08:17:59 -07:00
Terry Wilson 7665d3850b
Revert "Add LoadBalancer.acceptResolvedAddresses() (#8742)" (#9414)
This reverts commit 70a29fbfe3.
2022-07-28 12:15:26 -07:00
Terry Wilson 70a29fbfe3
Add LoadBalancer.acceptResolvedAddresses() (#8742)
Introduces a new acceptResolvedAddresses() to the LoadBalancer.

This will now be the preferred way to handle addresses from the NameResolver. The existing handleResolvedAddresses() will eventually be deprecated.

The new method returns a boolean based on the LoadBalancers ability to use the provided addresses. If it does not accept them, false is returned. LoadBalancer implementations using the new method should no longer implement the canHandleEmptyAddressListFromNameResolution(), which will eventually be removed, along with handleResolvedAddresses().

Backward compatibility will be maintained so existing load balancers using handleResolvedAddresses() will continue to work.

Additionally the previously deprecated handleResolvedAddressGroups() method is removed.
2022-07-26 09:23:37 -07:00
yifeizhuang 027d36eee7
xds: xdsNameResolver match channel overrideAuthority in virtualHost matching (#9405) 2022-07-22 12:41:16 -07:00
Eric Anderson 0e45e04041
Avoid accidental locale-sensitive String.format()
%s is fairly safe (requires a Formattable to use Locale), so %d is the
main risk item. Places that really didn't need to use String.format()
were converted to plain string concatenation. Logging locations were
generally converted to using the log infrastructure's delayed
formatting, which is generally locale-sensitive but we're okay with
that. That wasn't done in okhttp, however, because Android frequently
doesn't use MessageFormat so we'd lose the parameters. Everywhere else
was explicitly defined to be Locale.US, to be consistent independent of
the default system locale.
2022-07-19 14:41:34 -07:00
Larry Safran 7568f8cce7
core: Disable retry by default for in-process transport's channel (#9361)
See #8712
2022-07-19 12:35:52 -07:00
Eric Anderson 4cb1fbaa9f
core: Workaround retry causing memory leak
Data is getting orphaned sitting in MessageFramer. This hack thus always
flushes data out of the framer so no data can remain sitting there.

See #9340
2022-07-15 15:25:27 -07:00
yifeizhuang f9d5ce7e7a
core: server stream should not deliver halfClose() when call is immediately cancelled(#9362)
Fix a bug where the server stream delivers halfClose() to the call during cancellation. It happens when call has a short deadline. Server sees `INTERNAL, desc: Half-closed without a request` due to the bug.
2022-07-11 17:33:57 -07:00
Eric Anderson e96d04774b okhttp: Add server implementation 2022-07-01 12:38:01 -07:00
Larry Safran 74137b0978
core: Use SyncContext for InProcessTransport listener callbacks to avoid deadlocks
Fixes deadlocks caused by client and server listeners being called in a synchronized block

Also support unary calls returning null values

Fixes #3084
2022-06-30 13:41:36 -07:00
sanjaypujare c9a52eb83f
api,core: change ManagedChannel and Server Builders to use GlobalInterceptors (#9312)
* api,core: change ManagedChannel and Server Builders to use GlobalInterceptors
also added a getter in GlobalInterceptors to expose the set flag
2022-06-30 10:11:58 -07:00
yifeizhuang 377e3ce557
Start 1.49.0 development cycle (#9322) 2022-06-29 11:00:27 -07:00
Larry Safran b361ecfc65
core: Always pass offload executor to CallCredentials. never use the executor from CallOptions (#9313) 2022-06-28 13:29:22 -07:00
amirhadadi 7b80954fd5 core,interop-testing: avoid flushing after writing a message in case the server returns a single response 2022-06-24 07:13:25 -07:00
larry-safran 8e7793652b
core: Use offloadExecutor for CallCredentials (#9263)
Change the construction of CallCredentialsApplyingTransportFactory in
ManagedChannelImpl to use the offloadExecutor as indicated in
https://github.com/grpc/grpc-java/issues/6279#issuecomment-1147507365 .
2022-06-14 18:44:02 -07:00
Eric Anderson d2b05380de core: Clear ConfigSelector in panic mode
If the failure is before the NameResolver has returned the first time,
RPCs would be queued waiting for service config. We don't want to use
the ConfigSelector, as we are trying to circumvent the NameResolver and
LoadBalancer.

Fixes #9257
2022-06-14 17:38:27 -07:00
Eric Anderson 9a09021897
core: Add missed internal comment about idle mode workaround
This should have been included in #8746 but wasn't.
2022-06-06 09:34:33 -07:00
Sergii Tkachenko 9cb3c6e688 Start 1.48.0 development cycle 2022-05-18 13:57:28 -07:00
Ovidiu Tirla a88f19fb4b
Expose SecurityLevel on server-side (#8943) 2022-05-16 17:26:10 -07:00
Eric Anderson ba57a1d893 api: Add connection management APIs to ServerBuilder
These APIs were added to NettyServerBuilder for gRFC A8 and A9. They are
important enough that they shouldn't require using the perma-unstable
transport API to access. This change also allows using these methods
with grpc-netty-shaded.

Fixes #8991
2022-05-16 13:02:09 -07:00
Andrei Kandratovich 2c33e39f5d
binder: Respect requested message limits within a single MessageProducer 2022-05-16 09:23:16 -07:00
Ran e258fc743b
Use `ImmutableMap.Builder.buildOrThrow()` instead of deprecated `build()`. (#9132) 2022-05-02 11:51:42 -07:00
sanjaypujare 538db03d56
api: add support for SocketAddress types in ManagedChannelProvider (#9076)
* api: add support for SocketAddress types in ManagedChannelProvider
also add support for SocketAddress types in NameResolverProvider
Use scheme in target URI to select a NameRseolverProvider and get
that provider's supported SocketAddress types.
implement selection in ManagedChannelRegistry of appropriate
ManagedChannelProvider based on NameResolver's SocketAddress types
2022-04-22 09:10:55 -07:00
markb74 4e9dab9c70
Support includeStatusWithCause from InternalInProcess. (#9080) 2022-04-12 16:59:08 +02:00
Eric Anderson 4a137d6ef0 Start 1.47.0 development cycle 2022-04-11 10:41:33 -07:00
Eric Anderson 1426e2a670 rls: Use FakeClock like rest of grpc tests
No need to create a new (mock-based) ScheduledExecutorService
implementation; it is easy enough to teach FakeClock
scheduleAtFixedRate().
2022-04-06 08:45:53 -07:00
Eric Anderson 004ee10a73
core: Vastly separate types of clock in FakeClock
There was an attempt to use different epochs for the wall clock and the
monotonic clock. However, 123456789 is actually less than a second.
We want the gap between clocks to be at least a day. This issue was
discovered in #8968.

This separation found a bug in an RLS test where it was mixing epochs.
However, it was only a problem in the test. The code under test is
wrongly using wall clock for calculation durations, but that seems to be
a wide-spread problem and will need to be handled separately.
2022-04-04 11:24:55 -07:00
Kurt Alfred Kluever 31ce764723 Use `try/execute/fail/catch` instead of the strongly discouraged `@Test(expected=...)`
cl/437399696
2022-03-28 14:34:33 -07:00
yifeizhuang 86b74d9ecc
core: delayedClientCall returns drainPendingCalls runnable in setCall (#8978)
`setCall()` returns drainPendingCalls runnable only when there are calls to drain, otherwise return null. Preserved the behaviour of `start()` and `cancel()`, as they are protected by `delayOrExecute()`.
2022-03-15 12:57:24 -07:00
John Cormie abd1642371
New streams before transportReady() are not guaranteed to work (#8955)
Update javadoc to mention this previously-unwritten rule.

Update earlyServerClose_serverFailure_withClientCancelOnListenerClosed to obey it.

Update BinderTransport to fail sooner if this rule is broken.
2022-03-03 08:03:07 -08:00
Eric Anderson 4cf3a2b801
core: Decrease transparent retry limit to 1000
The test for 10,000 took 10s to run and would time out when using other
tools like TSAN. "10000" was just a "very large number less than
infinity" and 1000 serves the same purpose and should similarly never
trigger. Using 1000 has the test run in 1s and TSAN completes in 17s.

The limit was originally added in dc6eaccc.
2022-02-23 09:15:04 -08:00
Terry Wilson ec3db2b454
Start 1.46.0 development cycle (#8939) 2022-02-23 08:23:41 -08:00
ZHANG Dapeng dc6eacccbf
core: limit total number of local-only transparent retries
Limit the total number of local-only transparent retries per RPC for the moment to mitigate any potential bug that would trigger infinite loop of transparent retries. If the limit is exceeded, fail the RPC.
2022-02-09 13:20:07 -08:00
ZHANG Dapeng 431fb0255f
core/netty: infinite local-only transparent retry for netty (#8878)
In core, add a new enum element to `RpcProgress` for the case that the stream is closed even before anything leaves the client. `RetriableStream` will do unlimited transparent retry for this type of `RpcProgress` since they are local-only.

In netty, call `tranportReportStatus()` for pending streams on failure.

Also fixes #8394
2022-02-04 22:12:03 -08:00
Sergii Tkachenko c46d2c276a
core: test JsonUtil.getObject with a map containing a null value (#8881)
Verifies the behavior of JsonUtil.getObject when the map contains a null value for a given key.

Note: this may be incorrect behavior. Issue to track the investigation: #8883.
2022-02-04 09:56:26 -08:00
Zhouyihai Ding 46563b75bf
core: include cause when logging Status in InternalSubchannel (#8846)
It would be good to print Cause when the transport is shutdown and has throwable exception messages.

The current log doesn't have this information for debugging:
`SHUTDOWN with UNAVAILABLE(io exception Channel Pipeline: [HttpProxyHandler$HttpClientCodecWrapper#0, HttpProxyHandler#0, TsiHandshakeHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0])`
2022-01-27 11:07:14 -08:00
Penn (Dapeng) Zhang cf4cd65707 Revert "all: clean up code related to android api level less than 19"
This reverts commit 3ad4d9bfb7.
2022-01-18 10:14:50 -08:00
Penn (Dapeng) Zhang 3179bc3be0 Revert "use charset from StandardCharsets instead of 'Charset.forName' (#8779)"
This reverts commit a74a3ad834.
2022-01-18 10:14:50 -08:00
yifeizhuang 26f0d611db
Start 1.45.0 development cycle (#8825) 2022-01-12 12:08:22 -08:00
Eric Anderson 58a7ace6ac
Bump ErrorProne to 2.10.0
Previous versions of error prone were incompatible with Java 17 javac.

In grpc-api, errorprone is now api dependency because it is on a public
API.  I was happy to see that Gradle failed the build without the dep
change, although the error message wasn't super clear as to the cause.

It seems that previously -PerrorProne=false did nothing. I'm guessing
this is due to a behavior change of Gradle at some point. Swapping to
using the project does build without errorProne, although the build
fails with Javac complaining certain classes are unavailable. It's
unclear why. It doesn't seem to be caused by the error-prone plugin.
I've left it failing as a pre-existing issue.

ClientCalls/ServerCalls had Deprecated removed from some methods because
they were only deprecated in the internal class, not the API. And with
Deprecated, InlineMeSuggester complained.

I'm finding InlineMeSuggester to be overzealous, complaining about
package-private methods. In time we may figure out how to use it better,
or we may request changes to the checker in error-prone.
2022-01-12 12:06:27 -08:00
Jintao a74a3ad834
use charset from StandardCharsets instead of 'Charset.forName' (#8779)
Co-authored-by: Penn (Dapeng) Zhang <zdapeng@google.com>
2022-01-07 15:03:14 -08:00
ZHANG Dapeng 3ad4d9bfb7
all: clean up code related to android api level less than 19 2022-01-07 08:13:32 -08:00
ZHANG Dapeng 042f9879d4
all: remove deprecated StreamInfo.transportAttrs (#8768)
APIs such as `StreamInfo.getTransportAttrs()` were [deprecated](860e97d12a (diff-aa4049f54d6d5d462700e9221344184a37d2068b3ba7d715abd417b1df5bf883R114)) since 1.41.0. Removing now.
2021-12-20 09:46:25 -08:00
ZHANG Dapeng 63756ec951
core: remove unused method in AutoConfiguredLoadBalancer (#8769)
The method added in #5821 seems not used now.
2021-12-17 07:36:00 -08:00
ZHANG Dapeng 531db504ca
core: fix race condition in idleTimer & ManagedChannel#enterIdle
PR for Spikhalskiy/grpc-java@6d260b8

Hot fix of #8714
2021-12-14 17:55:48 -08:00
ZHANG Dapeng 65c00cf24e
Start 1.44.0 development cycle (#8729) 2021-11-30 11:53:56 -08:00
ZHANG Dapeng a5f1fb51b8
core: have JsonUtil support parsing String value as number (#8711)
As documented in https://developers.google.com/protocol-buffers/docs/proto3#json,
the canonical proto-to-json converter converts int64 (Java long) values to string values in Json rather than Json numbers (Java Double). Conversely, either Json string value or number value are accepted to be converted to int64 proto value.

To better support service configs defined by protobuf messages, support parsing String values as numbers in `JsonUtil`.
2021-11-19 10:12:39 -08:00
markb74 607362a7d2
Add support for anonymous in-process servers. (#8589)
Support anonymous in-process servers, and InProcessChannelBuilder.forTarget.

Anonymous servers aren't registered statically, meaning they can't be looked up by name.
Only the AnonymousInProcessSocketAddress passed to InProcessServerBuilder.forAddress(),
(or subsequently fetched from Server.getListenSockets()) can be used to connect to the server.

Supporting InProcessChannelBuilder.forTarget is particularly useful for production
Android usage of in-process servers, where process startup latency is crucial.
A custom name resolver can be used to create the server instance on demand
without directly impacting the startup latency of in-process gRPC clients.

Together, these features support a more-standard approach to "OnDeviceServer" referenced in gRFC L73.
https://github.com/grpc/proposal/blob/master/L73-java-binderchannel.md#ondeviceserver
2021-10-25 20:59:48 +02:00
Benjamin Peterson 1fe62dd417
Fix sentences with a missing "be". (#8613) 2021-10-20 15:14:10 -07:00
Sergii Tkachenko 6d41b4283a Start 1.43.0 development cycle 2021-10-20 13:47:03 -04:00
ZhenLian d2b9151e7b
core: remove DSA check in CertificateUtils 2021-10-19 19:50:33 -04:00
ZhenLian e9b0c2e851
Make CertificateUtils to use other key algorithms (#8609) 2021-10-15 14:42:14 -07:00
Sergii Tkachenko 0376de15b8
Fix AbstractManagedChannelImplBuilder#maxInboundMessageSize(int) ABI (#8607)
In refactoring described in #7211, the implementation of #maxInboundMessageSize(int)
(and its corresponding field) were pulled down from internal AbstractManagedChannelImplBuilder
to concrete classes that actually enforce this setting. For the same reason, it wasn't ported
to ManagedChannelImplBuilder (the #delegate()).

Then AbstractManagedChannelImplBuilder was brought back to fix ABI backward compatibility,
and temporarily turned into a ForwardingChannelBuilder, ref PR #7564. Eventually it will
be deleted, after a period with "bridge" ABI solution introduced in #7834.

However, restoring AbstractManagedChannelImplBuilder unintentionally made ABI of
pre-refactoring builds expect it to be a method of AbstractManagedChannelImplBuilder,
and not concrete classes, ref #8313.

The end goal is to keep #maxInboundMessageSize(int) only in concrete classes that enforce it.
To fix method's ABI, we temporary reintroduce it to the original layer it was removed from:
AbstractManagedChannelImplBuilder. This class' only intention is to provide short-term
ABI compatibility. Once we move forward with dropping the ABI, both fixes are no longer
necessary, and both will perish with removing AbstractManagedChannelImplBuilder.
2021-10-14 17:25:06 -07:00
Terry Wilson fc57cad4ec
Revert "Revert "core/auth: Remove CallCredentials2 (#8464)"" (#8572)
This reverts commit a91cc85dfd.
2021-10-05 10:34:44 -07:00
ZHANG Dapeng 28f2647aaf
core: move closed check from Stream.isReady() to Call.isReady() (#8566)
This fixes data race described in #8565.

We are doubtful whether checking closed in isReady() is necessary (#3201 might be a requirement), but it was easier to just maintain the existing behavior than think heavily about it.
2021-09-29 09:42:59 -07:00
Zhouyihai Ding e41df60bea
core: change the mapping from ChannelLogLevel to java.util.logging.Level
Instead of `ChannelLogLevel.{DEBUG,INFO}` mapping to the same java level, `ChannelLogLevel.{WARNING,ERROR}` will shame the same java level. This allows us to be able to independently control the visibility of `ChannelLogLevel.DEBUG` logs which are the most verbose.
2021-09-22 09:43:08 -07:00
ZhenLian 838438cedb
AdvancedTls: add functions to load credentials from static files (#8525)
* AdvancedTls: add functions to load credentials from static files
2021-09-17 09:45:41 -07:00
ZHANG Dapeng 3b237339c7
core: discard outbound content-length header (#8522)
Since netty version v4.1.67, content-lenght header validation will be enforced. So once grpc upgrades netty to that version or above, RPCs with invalid content-length header will fail.

Some libraries such as HTTP to gRPC adapters blindly copy all HTTP headers to gRPC metadata, but the content-length header is one of those that shouldn't be forwarded because gRPC uses different encoding. This mistake has already been in existence for a long time.

Discard outbound content-length headers in gRPC, so that users who encounter invalid content-length issue when upgrading grpc-java version on server/client side would be able to workaround by upgrading grpc-java on client/server side as well without fixing the HTTP adapter.
2021-09-13 17:15:45 -07:00
ZHANG Dapeng 7c6f53ab79
all: add internal API to disable retry stats (#8510)
Resolves b/197648853 for internal performance regression. Reporting retry stats caused significant amount of performance overhead internally.
2021-09-13 09:12:04 -07:00
ZhenLian fb00463001
fix a flaky test in advanced TLS (#8474)
* fix a flaky test in advanced tls
2021-09-08 11:43:23 -07:00
Eric Anderson 1f1396f3f0 Start 1.42.0 development cycle 2021-09-08 09:10:20 -07:00
Sergii Tkachenko a91cc85dfd
Revert "core/auth: Remove CallCredentials2 (#8464)"
This reverts commit 7cde473efa.
2021-09-02 17:20:20 -07:00
Brice Jaglin 62fafe7eda core: clarify exception message
Reformulate message to highlight that SizeEnforcingInputStream is
applied on the message size of the message after decompression.
2021-09-02 15:18:00 -07:00
Anuraag Agrawal 522b37bc3b
Fix drift in MessageFramer comment (#8427) 2021-09-02 08:56:56 -07:00
Terry Wilson 7cde473efa
core/auth: Remove CallCredentials2 (#8464)
- Removes CallCredentials2
- Removes CallCredentials2ApplyingTest
- Adds two tests from CallCredentials2ApplyingTest to CallCredentialsApplyingTest
- Updates GoogleAuthLibraryCallCredentials to extend from CallCredentials instead of CallCredentials2
2021-09-01 09:49:20 -07:00
Terry Wilson df4ac5973c
core: Exit idle mode in enterIdle() if there are pending calls or delayed transport.
This change assures that if there are only calls in real transport the
channel will remain in idle mode. Idle mode will be exited if there
are calls in delayed transport to allow them to be processed.
2021-08-26 14:42:27 -07:00
ZhenLian 3cb0696b1f
advancedtls: change enum to use UPPER_SNAKE_CASE (#8446) 2021-08-25 16:13:09 -07:00
Terry Wilson e45aab085c
core: Don't mark calls as cancelled if they are successfully completed. (#8408)
The semantics around cancel vary slightly between ServerCall and CancellableContext - the context should always be cancelled regardless of the outcome of the call while the ServerCall should only be cancelled on a non-OK status.

This fixes a bug where the ServerCall was always marked cancelled regardless of call status.

Fixes #5882
2021-08-20 14:42:01 -07:00
ZhenLian 2c2ebaebd5
advancedtls: adding AdvancedTlsX509TrustManager and AdvancedTlsX509KeyManager (#8175)
* add advanced TLS classes and tests
2021-08-17 16:13:30 -07:00
ZHANG Dapeng bdf9a96476
core: enable retry by default (#8402)
Stabilize `enableRetry()` and `disableRetry()`.

Disable retry in `ManagedChannelImplTest` because each call attempt will fork the headers to a new instance, and add a ClientStreamTracer.Factory for bufferSizeLimit in CallOptions, which makes verification not straightforward.
2021-08-11 14:44:23 -07:00
ZHANG Dapeng 2142902343
core: fix retry flow control issue (#8401)
There has been an issue about flow control when retry is enabled.

Currently we call `masterListener.onReady()` whenever `substreamListener.onReady()` is called.

The user's `onReady()` implementation might do

```
while(observer.isReady()) {
  // send one more message.
}
```

However, currently if the `RetriableStream` is still draining, `isReady()` is false, and user's `onReady()` exits immediately. And because `substreamListener.onReady()` is already called, it may not be called again after drained.

This PR fixes the issue by

- Use a SerializeExecutor to call all `masterListener` callbacks.
- Once `RetriableStream` is drained, check `isReady()` and if so call `onReady()`.
- Once `substreamListener.onReady()` is called, check `isReady()` and only if so we call `masterListener.onReady()`.
2021-08-11 10:25:57 -07:00
ZHANG Dapeng fd2a58a55e
all: implement retry stats (#8362) 2021-08-11 10:24:37 -07:00
ZHANG Dapeng cbda32a3c1
core: fix RetriableStream edge case bug introduced in #8386 (#8393)
While adding regression tests to #8386, I found a bug in an edge case: while retry attempt is draining the last buffered entry, if it is in the mean time committed and then we cancel the call, the stream will never be cancelled. See the regression test case `commitAndCancelWhileDraining()`.
2021-08-06 18:32:55 -07:00
ZHANG Dapeng 3668f2e52c
core: fix bug RetriableStream cancel() racing with start() (#8386)
There is a bug in the scenario of the following sequence of events:

- `call.start()` 
- received retryable status and about to retry
- The retry attempt Substream is created but not yet `drain()` 
- `call.cancel()`

But `stream.cancel()` cannot be called prior to `stream.start()`, otherwise retry will cause a failure with IllegalStateException. The current RetriableStream code must be fixed to not cancel a stream until `start()` is called in `drain()`.
2021-08-05 18:22:37 -07:00
ZHANG Dapeng c77083f013
core: fix old ClientStreamTracer.Factory creating tracers twice (#8381)
Fix a bug introduced in #8355 : old ClientStreamTracer.Factory implementation creates tracers twice.
2021-08-04 14:32:49 -07:00
Eric Anderson f6ce672369 Revert "core: correcting a minor resource releasing issue"
This reverts commit b9becb5c8e. It is the
caller's responsibility to close their InputStream. The change ended up
closing the passed InputStream.
2021-08-02 11:32:14 -07:00
ZHANG Dapeng 860e97d12a
all: API refactoring in preparation to support retry stats (#8355)
Rebased PR #8343 into the first commit of this PR, then (the 2nd commit) reverted the part for metric recording of retry attempts. The PR as a whole is mechanical refactoring. No behavior change (except that some of the old code path when tracer is created is moved into the new method `streamCreated()`).

The API change is documented in go/grpc-stats-api-change-for-retry-java
2021-07-31 18:33:02 -07:00
yifeizhuang 343eed1c04
Start 1.41.0 development cycle (#8351) 2021-07-28 15:27:55 -07:00
ZHANG Dapeng f3642422b4
xds: support xds retry policy (#8304) 2021-07-22 12:04:06 -07:00
Robert Turner b9becb5c8e
core: correcting a minor resource releasing issue
This change adds a traditional try/finally block around readers and
streams to control the closing of those objects when the method has
completed rather than relying on the GC to deal with them.

This issue was flagged by an analysis tool via binary analysis of the
grpc-core package as part of a dependency from another project.
2021-07-15 17:08:00 -07:00
ZHANG Dapeng 4429ec2b78
core: add perAttemptRecvTimeout to retry policy (#8301) 2021-07-12 14:53:48 -07:00
ZHANG Dapeng 4f09073e0f
all: remove 2-arg ClientStreamListener.closed()
We used to have two ClientStreamListener.closed() methods. One is simply calling the other with default arg. This doubles debugging (e.g. #7921) and sometimes unit testing work. Deleting the 2-arg method to cleanup.

This PR is purely refactoring.
2021-06-29 10:27:03 -07:00
yifeizhuang 7644350ecb
core: fix jumpstream listener not set when closing stream inline (#8290)
* Revert "core: rollback executor supplier, needs investigation (#8289)"

This reverts commit 1b57d48ac1.

* set jumpStream listener before close stream
2021-06-28 08:50:20 -07:00
yifeizhuang 1b57d48ac1
core: rollback executor supplier, needs investigation (#8289)
* Revert "core: fix boq conformance failures (#8281)"

This reverts commit c1ad5de135.

* Revert "core: allow per-service/method executor (#8266)"

This reverts commit c540229d79.
2021-06-25 13:10:45 -07:00
yifeizhuang c1ad5de135
core: fix boq conformance failures (#8281)
Fix false alarm to please boq conformance verifications, ref. go/boq-conformance-violations/BLOCKING#what-should-i-do-instead.
2021-06-24 08:07:06 -07:00
yifeizhuang c540229d79
core: allow per-service/method executor (#8266)
Add ServerCallExecutorSupplier interface in serverBuilder to allow defining which executor to to handle the server call.
Split StreamCreated() contextRunnable into two to support this new feature: one for method lookup, the other for server call handling. methodLookup() runs on default executor, handleServerCall() may run on the executorSupplier executor.
callbacks are queued after methodLookup() and handleServerCall() on serializing executor to ensure stream listener is set when callbacks starts running.
Make executor settable in serializing executor to allow switching executor for the server call handling runnable as a result of the outcome of the method lookup runnable.
2021-06-21 11:17:36 -07:00
ZHANG Dapeng e980ebd118
Start 1.40.0 development cycle (#8263) 2021-06-16 08:04:43 -07:00
Chengyuan Zhang 25122f9e83
api: clean up duplicated method in InternalServerInterceptors (#8261) 2021-06-15 15:11:56 -07:00
cfredri4 c8cd4cb260
netty: Support SocketAddress with ChannelCredentials (#8194)
This adds support for creating a Netty Channel with SocketAddress and ChannelCredentials.

This aligns with NettyServerBuilder.forAddress(SocketAddress address, ServerCredentials creds).
2021-05-24 09:49:20 -07:00
Chengyuan Zhang e5d0e9d9a8
api, core: support zero copy into protobuf (#8102)
Enables a codepath for zero-copy protobuf deserialization. Two new InputStream extension interfaces are added:

- HasByteBuffer: allows access to the underlying buffers containing inbound bytes directly without copying
- Detachable: allows customer marshaller to keep the buffers around until the application code is done with using the protobuf messages

Applications can implement a custom marshaller that takes over the ownership of ByteBuffers and wrap them into ByteStrings with protobuf's UnsafeByteOperations support. Then a RopeByteString, which is a in-place composite of ByteStrings can be created. This enables using the zero-copy codepath (requires immutable ByteBuffer indication) of CodedInputStream for deserialization.
2021-05-14 14:45:03 -07:00
Eric Anderson e08b9db208
Use @DoNotCall for static methods in Builders that throw
Since static methods are pseudo-inherited by Builder implementations but
are trivially accidentally used, we re-define static methods in each
builder to make them behave more like the caller would expect. However,
not all the methods actually work; some just throw because the caller
was certainly not getting what they would expect.

Annotating with `@DoNotCall` can expose the problems at compile time
instead of runtime. While `@Deprecated` would also be an option, it is a
bit harder to figure out the ramifications and whether we want to go
that route.

This change was suggested by a lint tool for XdsServerBuilder and it
seems appropriate so I applied it to the other similar cases I could
find.
2021-05-12 10:12:52 -07:00
Eric Gribkoff c0eca6de25
Start 1.39.0 development cycle (#8147) 2021-05-05 16:30:38 -07:00
Chengyuan Zhang 368c43aec4
core: throw away subchannel references after round_robin is shutdown (#8132)
Triggering balancing state updates after already being shutdown can be confusing for the upstream of round_robin. In cases of the callers not managing round_robin's lifecycle (e.g., not ignoring updates after it shuts down round_robin, which it should), it can make problem very bad, especially with the behavior that round_robin is actually propagating TRANSIENT_FAILURE with a picker that buffers RPCs.

This change only polishes round_robin by always preserving its invariant. Callers/LBs should not rely on this and should still manage the balancing updates from its downstream correctly based on the downstream's lifetime.
2021-04-30 15:42:44 -07:00
ZhenLian 1703d692bc
advancedtls: Add a Utility Class For Loading Certs/Keys (#8023) 2021-04-21 10:07:44 -07:00
Chengyuan Zhang 9614738a7d
core, grpclb, xds: let leaf LB policies explicitly refresh name resolution when subchannel connection is broken (#8048)
Currently each subchannel implicitly refreshes the name resolution when its state changes to IDLE or TRANSIENT_FAILURE. That is, this feature is built into subchannel's internal implementation. Although it eliminates the burden of having LB implementations refreshing the resolver when connections to backends are broken, this is gives LB policies no chance to disable or override this refresh (e.g., in some complex load balancing hierarchy like xDS, LB policies may embed a resolver inside for resolving backends so the refreshing resolution operation should be hooked to the resolver embedded in the LB policy instead of the one in Channel).

In order to make this transition smoothly, we add a check to SubchannelImpl that checks if the LoadBalancer has explicitly called Helper.refreshNameResolution for broken subchannels created by it. If not, it logs a warning and do the refresh.

A temporary LoadBalancer.Helper API ignoreRefreshNameResolution() is added to avoid false-positive warnings for xDS that intentionally does not want a refresh. Once the migration is done, this should be deleted.
2021-04-16 10:49:06 -07:00
ZHANG Dapeng d25ebaf57d
core: fix NPE in ConfigSelectingClientCall
Fix the following bug:

ManagedChannelImpl.ConfigSelectingClientCall may return early in start() leaving delegate null, and fails request() method after start().

Currently the bug can only be triggered when using xDS.
2021-04-14 23:06:37 -07:00
Sergii Tkachenko e4b292aa9b Start 1.38.0 development cycle 2021-03-25 18:49:03 -04:00
yifeizhuang 6a9c9901e4
grpclb: support multiple authorities in lb backends for all SRV records (#7951) 2021-03-11 13:29:41 -08:00
yifeizhuang 528ef63c58
core: Move negotiationLogger from channel attributes to GrpcHttp2ConnectionHandler (#7933) 2021-03-10 16:43:56 -08:00
Chengyuan Zhang b5c0a4a97a Make addServices() a final method on ServerBuilder and delete from its forwarders. 2021-03-05 10:03:22 -08:00
Chengyuan Zhang 37b94b3074
api: delete deprecated NameResolver APIs (#7936)
Deletes deprecated (~2 years) NameResolver APIs that uses NameResolver.Helper/Attributes for passing information from Channel to resolver.
2021-03-04 13:08:18 -08:00
Chengyuan Zhang a598b973a3
api: add ServerBuilder.addServices() API (#7926) 2021-02-28 23:36:27 -08:00
ZHANG Dapeng 132a40a1cf
xds: implement fault injection interceptor in XdsNameResolver 2021-02-18 14:35:45 -08:00
Eric Anderson 2140480736 Start 1.37.0 development cycle 2021-02-11 13:53:10 -08:00
ZHANG Dapeng 92e7fd370b
core: user is responsible to override authority for resolvingOobChannelBuilder
ManagedChannelImpl should not override authority for createResolvingOobChannel(target, creds), because ManagedChannelImpl does not know what target and creds are.
2021-01-29 09:50:59 -08:00
yifeizhuang ef76337f5c
core: add more delayedStream tests (#7843)
Add more delayedStream tests related to #7750, where we changed to call realStream.start() synchronously with setting realStream.
2021-01-29 09:42:10 -08:00
ZHANG Dapeng 9437783838
core: enhance ManagedChannelBuilder.overrideAuthority()
Enhance `ManagedChannelBuilder.overrideAuthority()` to make it impossible to use a different authority to a backend by wrapping ClientTransportFactory.newClientTransport() and setting ClientTransportOptions’ authority. To avoid confusing the LB policy, it would need to keep the original addresses to return during `Subchannel.getAddresses()`

The class `OverrideAuthorityNameResolverFactory` is deleted and its logic is moved into `ManagedChannelImpl`.
2021-01-29 09:29:06 -08:00
ZHANG Dapeng 45a151810c
all: implement Helper.createResolvingOobChannelBuilder(target, creds)
- Add APIs to `ClientTransportFactory`:
```java
public interface ClientTransportFactory {
  /**
   * Swaps to a new ChannelCredentials with all other settings unchanged. Returns null if the
   * ChannelCredentials is not supported by the current ClientTransportFactory settings.
   */
  SwapChannelCredentialsResult swapChannelCredentials(ChannelCredentials channelCreds);

  final class SwapChannelCredentialsResult {
    final ClientTransportFactory transportFactory;
    @Nullable final CallCredentials callCredentials;
  }
}
```

- Add `ChannelCredentials` to constructor args of `ManagedChannelImplBuilder`:
 ```java
public ManagedChannelImplBuilder(
      String target, @Nullable ChannelCredentials channelCreds, @Nullable CallCredentials callCreds, ...)
  ```
2021-01-28 09:49:53 -08:00
yifeizhuang ac2ead70b4
core: delay CallCredentialsApplyingTransport shutdown until metadataApplier finalized (#7813)
Improve the CallCredentialsApplyingTransport shutdown lifecycle management. Right now CallCredentialsApplyingTransport shutdown the delegated real transport too early. It should be waiting for the metadataAppliers to finish because they may execute asynchronously. In addition, there is no shutdown check on CallCredentialsApplyingTransport for newStream(). The degraded lifecycle implementation may cause RejectionExecutionException, or accepting new RPCs after the underlying transport is already closed during channel shutdown.

We added listener on metadataApplier to notify completion, a magic counter to track the pending metadataApplier for delaying shutdown, also added shutdown check for newStream().
2021-01-26 12:01:16 -08:00
Nick Hill 2072df9be6
Random acts of garbage reduction
I noticed some opportunities to reduce allocations on hot paths
2021-01-21 10:45:31 -08:00
Eric Anderson 7b8105e100
core: DelayedStream should start() real stream immediately
DelayedClientTransport needs to avoid becoming terminated while it owns
RPCs. Previously DelayedClientTransport could terminate when some of its
RPCs had their realStream but realStream.start() hadn't yet been called.

To avoid that, we now make sure to call realStream.start()
synchronously with setting realStream. Since start() and the method
calls before start execute quickly, we can run it in-line. But it does
mean we now need to split the Stream methods into "before start" and
"after start" categories for queuing.

Fixes #6283
2021-01-20 15:01:49 -08:00
Chengyuan Zhang ffcc360ba9
core: further clean up leftovers in ManagedChannelImpl's LoadBalancer.Helper and Subchannel implementations (#7806) 2021-01-14 10:41:36 -08:00
Chengyuan Zhang 389c5403e9
core: make subchannel creation timing restriction stricter (#7790)
Throw for subchannel creation if the channel is being shutting down and the delayed transport is terminated (aka, all retry calls has been finished). This enforces load balancer implementations to avoid creating subchannels after being shut down.
2021-01-13 17:01:02 -08:00
Chengyuan Zhang b66d182bb9
api: delete LoadBalancer.Helper APIs that had been deprecated for a long time (#7793) 2021-01-11 15:25:35 -08:00
Yifei Zhuang 53da588dd1
Move multiple-port ServerImpl to NettyServer (#7674)
Change InternalServer to handle multiple addresses and implemented in NettyServer.
It makes ServerImpl to have a single transport server, and this single transport server (NettyServer) will bind to all listening addresses during bootstrap. (#7674)
2021-01-05 13:24:16 -08:00
Eric Gribkoff f2f3bbeb5d
Start 1.36.0 development cycle (#7770) 2020-12-30 17:14:04 -08:00
Eric Anderson 20197d36ce core: Include wait-for-ready in deadline exceeded insights
There was a report from a user in b/176088054 that experienced an RPC
waiting_for_connection that failed after 45 minutes due to deadline.
That would be quite normal for wait-for-ready, but because we didn't
include that detail in the status we have to do code inspection to
determine if wait-for-ready was enabled.
2020-12-23 13:37:21 -08:00
ZHANG Dapeng 90d61178a3
all: ChannelCredentials.withoutBearerTokens() and LoadBalancer.Helper API change (#7748)
API change (See go/grpc-rls-callcreds-to-server):

- Add `ChannelCredentials.withoutBearerTokens()`
- Add `createResolvingOobChannelBuilder(String, ChannelCredentials)`, `getChannelCredentials()` and `getUnsafeChannelCredentials()` for `LoadBalancer.Helper`

This PR does not include the implementation of `createResolvingOobChannelBuilder(String, ChannelCredentials)`.
2020-12-22 22:48:39 -08:00
Yifei Zhuang a67d816f4b
Revert " check pending stream completion at delayed transport lifecycle (#7720)" (#7744)
This reverts commit 90850128a6.
Alternative solution: alternative #7743
2020-12-21 15:15:00 -08:00
Yifei Zhuang 90850128a6
check pending stream completion at delayed transport lifecycle (#7720)
add onTransferComplete() at delayedStream and wait for all pending streams to complete transfer when shutting down delayedClientTransport
2020-12-21 11:56:51 -08:00
Eric Anderson ec70b64610
Bump Gradle and plugin versions
Android plugins weren't touched, since they will need a lot more testing
when doing so.
2020-12-17 15:02:27 -08:00
Eric Anderson 60d3e7c536
core: On unexpected EOS, mention whether the frame was empty
Empty DATA frames with EOS tell a stronger tale as to where the server
may have its bug.
2020-12-10 14:54:33 -08:00
Eric Anderson 4be68f3287
core: Don't leak CallCredentials into OOB channels
The addition of CompositeChannelCredentials allowed CallCredentials to
be passed to the ManagedChannel itself. But the implementation was buggy
and used the call creds for out-of-band channels as well, which is
inappropriate since they have a different authority.

This also fixes a bug where resolving OOB channels would have CallCreds
duplicated; that wasn't noticed or important because we don't use
CallCreds in OOB channels.

Fixes #7643
2020-12-10 11:49:29 -08:00
ZHANG Dapeng af29777686
all: migrate to interceptor-based config selector
This is migration following #7610
2020-12-04 11:11:57 -08:00
ZHANG Dapeng ac2327deb7
api,core: interceptor-based config selector (#7610)
Interceptor-based config selector will be needed for fault injection.

Add `interceptor` field to `InternalConfigSelector.Result`. Keep `callOptions` and `committedCallback` fields for the moment, because it needs a refactoring to migrate the existing xds config selector implementation to the new API.
2020-11-30 09:16:22 -08:00
Chengyuan Zhang f5bbe12c9f
Start 1.35.0 development cycle (#7641) 2020-11-18 13:59:40 -08:00
Chengyuan Zhang 10dc41af74
core: round robin should ignore name resolution error for channel state change when there are READY subchannels (#7595)
Round robin is keeping use of READY subchannels even if there is name resolution error. However, it moves Channel state to TRANSIENT_ERROR.

In hierarchical load balancers, the upstream LB policy may need to aggregate pickers from multiple downstream round_robin LB policy while filtering out non-ready subchannels. It cannot infer if the subchannel can be used just from the SubchannelPicker interface. It relies on the state that the round_robin intends to set channel to.

So the change is to match the readiness of the picker/subchannel with the state that round_robin tries to update. It will completely ignore name resolution error if there are READY subchannels.
2020-11-04 14:36:12 -08:00
Sergii Tkachenko d314c68126
Fix builders ABI backward compatibility broken in v1.33.0
* fix channel builders ABI backward compatibility broken in v1.33.0
* fix server builders ABI backward compatibility broken in v1.33.0
* makes ForwardingServerBuilder package-private
2020-10-29 12:06:37 -04:00
Eric Anderson 735b85fb33 netty: Differentiate GOAWAY closure status descriptions
With this, it will be clear if the RPC failed because the server didn't
use a double-GOAWAY or if it failed because of MAX_CONCURRENT_STREAMS or
if it was due to a local race. It also fixes the status code to be
UNAVAILABLE except for the RPCs included in the GOAWAY error (modulo the
Netty bug).

Fixes #5855
2020-10-29 09:04:37 -07:00
ZHANG Dapeng 654f7c3dc6
core: fix floating-point number formatting Locale (#7473) 2020-10-28 17:23:57 -07:00
Eric Anderson 5733cd481a core: Add ChannelCredentials 2020-10-07 13:58:37 -05:00
Eric Anderson c8a94d1059 api: Add ChannelCredentials 2020-10-07 13:58:37 -05:00