Commit Graph

3524 Commits

Author SHA1 Message Date
Ran 81ba42a1d6
core: expose some of AutoConfiguredLoadBalancer because some internal tests need to access them (#5821)
* core: revert some changes to fix tests

* fix style
2019-06-03 09:34:04 -07:00
Chengyuan Zhang 93551719b9
xds: integrate backend metric API to client load reporting (#5797)
* augmented ClientLoadCounter with backend metrics

* added a listener implementation for receiving backend metrics and aggregate in ClientLoadCounter
2019-05-31 14:28:23 -07:00
Kun Zhang 276b7d8512
interop-testing: create GrpclbLongLivedAffinityTestClient (#5817)
This is a long-running stand-alone test client for a specific customer that uses GRPCLB's pick_first mode.
2019-05-31 12:54:05 -07:00
Manuel Kollus e526891a2b api,protobuf-lite: solve code style issues 2019-05-31 12:49:30 -07:00
Eric Anderson c6c2ee876a core: Remove unnecessary SuppressWarnings from JsonParser 2019-05-31 10:09:39 -07:00
Jihun Cho d37f87abce
core: Migrate InternalSubchannel to use SynchronizedContext (#5555) 2019-05-30 18:40:50 -07:00
ZHANG Dapeng d8aa42723d
xds: fix bug in XdsLoadBalancerProvider.parseLoadBalancingConfigPolicy
Resolves #5804
2019-05-30 16:37:08 -07:00
ZHANG Dapeng f9decbf69d
xds: remove unused variables 2019-05-30 14:38:43 -07:00
Eric Anderson 3c931b40b0 api: Mention similarity of synccontext to a dedicated thread
This is the conceptual model we use. Document it to help aid others'
understanding and make it easier to understand when it is appropriate to
use.
2019-05-30 10:52:22 -07:00
Kun Zhang eff13a9ec8
core: only let ManagedChannelImpl convert empty resolution result to error (#5803)
Previously, AutoConfiguredLoadBalancer was also handling it, but it
doesn't trigger retries.  By returning true for
canHandleEmptyAddressListFromNameResolution(),
AutoConfiguredLoadBalancer effectively by-passed the empty-result
handling logic in ManagedChannelImpl, thus resolution retries were
never triggered.

This change requires AutoConfiguredLoadBalancer to stop being a
LoadBalancer, for its tryHandleResolvedAddresses().  It doesn't
cause any trouble because AutoConfiguredLoadBalancer has become
less and less like a LoadBalancer during the service config changes.
2019-05-30 09:48:30 -07:00
Kun Zhang af2c16d301
api: deprecate Helper.updateSubchannelAddresses() and add equivalent on Subchannel (#5802)
Resolves #5676
2019-05-30 09:16:38 -07:00
Ryan Michela 9b4c958201 Explain why client stub mocking is discouraged (#5796) 2019-05-30 08:52:13 -07:00
Eric Anderson bc2e1764f6 api,stub: Clarify isReady()/onReady() interaction semantics 2019-05-29 17:28:45 -07:00
Tim van der Lippe ad0893737e Migrate org.mockito.Matchers#any* to org.mockito.ArgumentMatchers
The former is deprecated and replaced by the latter in Mockito 2.
However, there is a functional difference: ArgumentMatchers will reject
`null` and check the type if the matcher specified a type (e.g.
`any(Class)` or `anyInt()`). `any()` will remain to accept anything.
2019-05-29 16:53:01 -07:00
Carl Mastrangelo 4e5e19f6fd
Revert "netty: change server to new protocol negotiator model" (#5798)
This reverts commit f94b77c87f.
2019-05-28 21:12:25 -07:00
Eric Anderson 2cbc540cb3 Remove deprecated enableKeepAlive API from transports
As mentioned in 5188[1], the default used with the enableKeepAlive API
conflicted with the default server enforcement. Instead of fixing it,
remove it. These APIs were deprecated in v1.3.0 in April 2017.

1. https://github.com/grpc/grpc-java/issues/5188#issuecomment-482269303
2019-05-28 16:37:02 -07:00
ZHANG Dapeng 77a512551f
xds: handle 100% drop for fallback mode
- Cancel fallback timer and/or exit fallback mode once receiving an EDS response indicating 100% drop.
- Also update balancing state once receiving the first EDS response with drop information when the channel is at the initial IDLE state.
2019-05-24 20:54:37 -07:00
Chengyuan Zhang 7fd5f261b4
xds: implement lb policy backend metric api (#5639)
* implemented utility methods to create ClientStreamTracer.Factory with OrcaReportListener installed for retrieving per-request ORCA data

* added unit tests

* use delegatesTo instead of spy

* implemented OrcaReportingHelper delegating to some original Helper for load balancing policies accessing OOB metric reports

* added unit tests for out-of-band ORCA metric accessing API in a separate test class

* rebase to master, resolve the breaking change of StreamInfo class being final with builder

* trashed hashCode/equal for OrcaReportingConfig

* changed log level and channel trace event level to ERROR as required by design doc

* added OrcaReportingHelperWrapper layer to allow updating report interval at any time

* reverse the naming of parent/child helper, child helper is the outer-most helper in the wrapping structure

* changed orca listener interface to use separate listener interfaces for per-request and out-of-band cases

* added more comprehensive unit tests

* added test case for per-request reporting that parent creates its own stream tracer

* fixed bug of directly assign reporting config, which would cause it be mutated later

* separate test cases for updating reporting config at different time

* fixed lint style error

* polish comments

* minor polish in unit tests

* refactor OrcaUtil class into OrcaOobUtil and OrcaPerRequestUtil and get rid of static methods for easier user testing

* hide BackoffPolicyProvider and Stopwatch supplier in OrcaOobUtil's public API

* add javadoc for getInstance() methods

* ensure the same Subchannel instance created by the helper that has corresponding OrcaOobReportListener registered are passed to the listener callback

* removed costNames foe OrcaReportingConfig

* removed redundant checks

* reformated the OrcaOobUtilTest class to put helper methods in the bottom

* fixed impl with changes made on Subchannel (SubchannelStateListener now ties with Subchannel)

* fixed comments

* added usage examples in javadoc for OrcaUtils

* add method comments for OrcaUtil's listener API threading

* make fields in OrcaReportingConfig final

* fixed OrcaOobUtilTest for calling setOrcaReportingConfig inside syncContext

* added ExperimentalApi annotation for Orca utils
2019-05-24 15:12:22 -07:00
Yang Song 46f34a513f core: Make OpenCensus tags non-propagating. (#5689)
The `method` and `status` shouldn't be propagated in the first place,
but in previous OpenCensus implementation all tags are propagating by
default. Now with the TagMetadata it may make sense to change them to
local tags.

This will be a breaking change to users who depend on the behavior that
these tags propagate through process boundaries.
2019-05-24 14:32:26 -07:00
Chengyuan Zhang d86d3dd363
all: fix lint and revert redundant lint fixes in #5570 (#5787)
* Revert "all: fix lint (#5770)"

This reverts commit 00d4cc29ad.

* all: fix lint and revert redundant lint fix in #5570
2019-05-24 01:02:12 -07:00
ZHANG Dapeng 2180fcd113
xds: fix protobuf fields can not be null 2019-05-23 18:27:09 -07:00
Kun Zhang 881dbeb3a2
core: fix a deterministic race between the old createSubchannel() and Subchannel.requestConnection(). (#5785)
The old createSubchannel() doesn't require being called from
sync-context, thus its implementation schedules start() on
sync-context because start() has such requirement.  However, if a
LoadBalancer call createSubchannel() in sync-context, start() is
queued and only called after the control exits the sync-context.  If
createSubchannel() is immediately followed by other Subchannel methods
that requires start(), such as requestConnection(), they will fail.
This will be a very common issue as most LoadBalancers call
createSubchannel() in the sync-context.

This fix splits out the real work of start() into internalStart() which
is called by the old createSubchannel().
2019-05-23 17:10:27 -07:00
Eric Anderson d7ef437dac RELEASING.md: Update locations for new grpc.io website 2019-05-23 13:31:47 -07:00
Eric Anderson 5a4178a74e Update README to reference 1.21.0 2019-05-23 13:31:25 -07:00
Eric Anderson 58c0ea5961 RELEASING.md: Use search.maven.org instead of mvnrepository.com
search.maven.org is more authoritative than mvnrepository.com
2019-05-23 12:56:29 -07:00
Eric Anderson 9679296573 RELEASING.md: Pull recent version of gh-pages before modifying 2019-05-23 12:55:37 -07:00
Carl Mastrangelo b0167b97e1
alts: don't log in Framer after close 2019-05-23 11:36:00 -07:00
ZHANG Dapeng 08843f8d59
xds: handle drop percentage for SubchannelPicker (#5765)
* xds: handle drop percentage for SubchannelPicker

* XdsCommsTest

* refactor ThreadSafeRandom

* remove hasNonDropBackends

* fix comments
2019-05-23 10:36:59 -07:00
Kun Zhang a1bc92a748
test: cover the change that added Subchannel.getInternalSubchannel() (#5774) 2019-05-23 10:36:03 -07:00
Carl Mastrangelo cdeafcf6e0
core: add discard method for Attributes 2019-05-23 10:34:04 -07:00
Chengyuan Zhang 4dcbe56354
core: create subchannelLogger outside InternalSubchannel and before calling start() (#5775) 2019-05-23 10:30:46 -07:00
Chengyuan Zhang 00d4cc29ad
all: fix lint (#5770)
* fixed lint warnings

* remove unused arg for subchannel
2019-05-23 01:05:22 -07:00
ZHANG Dapeng 54bbd372ef
xds: implement Fallback-at-Startup mode
This is the implementation of the Fallback-at-Startup mode in the design doc.

- The Fallback-After-Startup mode is not implemented.
- Drop related behavior is not implemented.
2019-05-22 17:44:39 -07:00
Carl Mastrangelo 7834a50525
alts: use new ProtocolNegotiator style for ALTS
This change does a few core things, which result in a lot of churn in other parts.  It's not as bad as it seems.

Core things:

1.  AltsProtocolNegotiator is now a shim class, same as ProtocolNegotiators
2.  The protocol negotiators are now in the new style, where there is at most 1 negotiation handler in the pipe at a time.
3.  TsiHandshakeHandler is rewritten with respect to the above.  All errors and buffering are handled by the WBAEH.
4.  TsiFrameHandler is only installed once the negotiation is successful, eliminating the state handling.


The churn in mainly in GoogleDefaultChannel and the GCE channel, which now reuse the *handlers* rather than the negotiators.  This makes it significantly easier to reason about the pipeline state.  The tests are also a source of churn, which no long need to check for most buffering and error conditions.
2019-05-22 16:33:07 -07:00
Kun Zhang f8fffeff12
api: add Subchannel.getInternalSubchannel(). (#5773)
Previously PickResult's Subchannel must be the actual implementation
returned from the Channel's Helper, and Channel would cast it to the
implementation class in order to use it.  This will be broken if
Subchannel is wrapped in the case of hierarchical LoadBalancers.

getInternalSubchannel() is the guaranteed path for the Channel to get
the InternalSubchannel implementation.  It is friendly for wrapping.

Background: #5676
2019-05-22 13:52:28 -07:00
ZHANG Dapeng 36ae0ed165
xds: temporarily use ManagedChannelBuilder.forTarget for creating resolving oob channel 2019-05-22 13:38:00 -07:00
ZHANG Dapeng b3bac95f90
xds: not sending resource_name in EDS request 2019-05-22 09:36:51 -07:00
ZHANG Dapeng 994fd7429a
xds: populate XdsLoadBalancerProvider to ServiceLoader
Also changed `XdsLoadBalancerProvider` to avoid initialization error when using `ServiceLoader`.
2019-05-22 09:35:38 -07:00
Chengyuan Zhang e237ee12a0
examples: use test certs for running example-tls (#5763)
* examples: use test certs for running example-tls

* fixed a typo

* update usage printout for trustCertCollectionFilePath is not optional

* Revert "update usage printout for trustCertCollectionFilePath is not optional"

This reverts commit 2dd6d87f64.

* put back the usage of using system default CA and put notes for it

* fixed cmd-line argument options
2019-05-22 09:05:16 -07:00
Eric Anderson 50b08eee3b
Delete protobuf-nano
Nano was dropped from upstream protobuf. See gRFC L51
2019-05-22 08:49:07 -07:00
Eric Anderson b6720c9fe6 api: Don't yet deprecate NameResolver.Listener
We think NameResolver.Listener2 may still change, so don't yet encourage
people to migrate.

Fixes #5764
2019-05-21 17:49:08 -07:00
Eric Anderson 15a453d9ef api: Fix doc for default value of idleTimeout()
It appears the docs were not updated when this was changed in d74091f5, which
was first released in v1.1.
2019-05-21 13:48:57 -07:00
ZHANG Dapeng c242fc8245
xds: fix XdsLoadStatsStoreTest.recordingDroppedRequests flaky NPE 2019-05-20 16:00:18 -07:00
Kun Zhang a2595d9e7d
api/core/grpclb: add warning about calling Subchannel.requestConnection() outside of sync-context (#5757)
The pick_first policies in core and grpclb previously would call
Subchannel.requestConnection() from data-path.  They now will schedule
that call in the sync-context to avoid the warning.  They will only
call it for the first pick of each picker, to prevent storming the
sync-context.
2019-05-20 11:39:02 -07:00
Kun Zhang 7934594dfe
api: pass Subchannel state updates to SubchannelStateListener rather than LoadBalancer (take 2) (#5722)
This is a revised version of #5503 (62b03fd), which was rolled back in f8d0868. The newer version passes SubchannelStateListener to Subchannel.start() instead of SubchannelCreationArgs, which allows us to remove the Subchannel argument from the listener, which works as a solution for #5676.

LoadBalancers that call the old createSubchannel() will get start() implicitly called with a listener that passes updates to the deprecated LoadBalancer.handleSubchannelState(). Those who call the new createSubchannel() will have to call start() explicitly.

GRPCLB code is still using the old API, because it's a pain to migrate the SubchannelPool to the new API.  Since CachedSubchannelHelper is on the way, it's easier to switch to it when it's ready. Keeping
GRPCLB with the old API would also confirm the backward compatibility.
2019-05-17 16:37:41 -07:00
Carl Mastrangelo f94b77c87f
netty: change server to new protocol negotiator model
Changes:

* PlaintextProtocolNegotiator is the same between client and server
* ServerTlsHandler is rewritten to not handle errors 
  * Also, it now sets the security level attribute, which I don't think it did previously
* NettyServerTransport now uses WBAEH, similar to the client. I don't think the buffer is needed, but it does  correctly handle errors during the startup
2019-05-16 18:15:47 -07:00
Eric Anderson bacb741aaf
examples: Use encouraged APIs and avoid over-specifying configuration
Also fix tcnative for Maven. The version had only been updated for
gradle. The wrong version would cause a crash with NoSuchMethodError.
2019-05-16 16:28:55 -07:00
ZHANG Dapeng 53f74c62ba
all: fix lint 2019-05-16 15:00:20 -07:00
Kun Zhang ae14c8d061
api: temporarily remove the warning about sync-context in requestConnection(). (#5738) (#5752)
Contrary to #5736, we will still keep the sync-context requirement of
requestConnection(), because it prevents API fragmentation.
PickFirstLoadBalancer is the only known violator.  We will fix it on
master, but we don't want to make that change on 1.21.x because the
release is soon.  We simply remove the warning in this release so that
users won't be annoyed.

This supersedes #5736
2019-05-16 13:07:10 -07:00
Kun Zhang cec9ee368d
api: move SubchannelPicker.requestConnection() to LoadBalancer. (#5751)
We will require Subchannel.requestConnection() to be called from
sync-context (#5722), but SubchannelPicker.requestConnection() is
currently calling it with the assumption of thread-safety.  Actually
SubchannelPicker.requestConnection() is called already from
sync-context by ChannelImpl, it makes more sense to move this method
to LoadBalancer where all other methods are sync-context'ed, rather than
making SubchannelPicker.requestConnection() sync-context'ed and fragmenting
the SubchannelPicker API because pickSubchannel() is thread-safe.

C++ also has the requestConnection() equivalent on their LoadBalancer
interface.
2019-05-16 09:50:23 -07:00