grpc
/
grpc-java
mirror of https://github.com/grpc/grpc-java.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

doc: more guidelines for LoadBalancer2 implementations. (#2501)

This commit is contained in:
Kun Zhang 2016-12-16 15:18:59 -08:00 committed by GitHub
parent ce9d152dff
commit a6ecece6f5
1 changed files with 122 additions and 36 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

158
core/src/main/java/io/grpc/LoadBalancer2.java
View File

@ -55,20 +55,21 @@ import javax.annotation.concurrent.ThreadSafe;
* from the Channel Executor. It receives the results from the {@link NameResolver}, updates
* of subchannels' connectivity states, and the channel's request for the LoadBalancer to
* shutdown.</li>
* <li>{@link SubchannelPicker} does the actual load-balancing work. It selects a
* {@link Subchannel} for each new RPC.</li>
* <li>{@link Factory} creates a new {@link LoadBalancer2} instance.
* <li>{@link SubchannelPicker SubchannelPicker} does the actual load-balancing work. It selects
* a {@link Subchannel Subchannel} for each new RPC.</li>
* <li>{@link Factory Factory} creates a new {@link LoadBalancer2} instance.
* </ol>
*
* <p>{@link Helper} is implemented by gRPC library and provided to {@link Factory}. It provides
* functionalities that a {@code LoadBalancer2} implementation would typically need.</li>
* <p>{@link Helper Helper} is implemented by gRPC library and provided to {@link Factory
* Factory}. It provides functionalities that a {@code LoadBalancer2} implementation would typically
* need.</li>
*
* <h3>Channel Executor</h3>
*
* <p>Channel Executor is an internal executor of the channel, which is used to serialize all the
* callback methods on the {@link LoadBalancer2} interface, thus the balancer implementation doesn't
* need to worry about synchronization among them. However, the actual thread of the Channel
* Executor is typically the network thread, thus following rules must be followed to prevent
* Executor is typically the network thread, thus the following rules must be followed to prevent
* blocking or even dead-locking in a network
*
* <ol>
@ -79,13 +80,14 @@ import javax.annotation.concurrent.ThreadSafe;
*
* <li><strong>Avoid calling into other components with lock held</strong>. Channel Executor may
* run callbacks under a lock, e.g., the transport lock of OkHttp. If your LoadBalancer has a
* lock, holds the lock in a callback method (e.g., {@link #handleSubchannelState}) while calling
* into another class that may involve locks, be cautious of deadlock. Generally you wouldn't
* need any locking in the LoadBalancer.</li>
* lock, holds the lock in a callback method (e.g., {@link #handleSubchannelState
* handleSubchannelState()}) while calling into another class that may involve locks, be cautious
* of deadlock. Generally you wouldn't need any locking in the LoadBalancer.</li>
*
* </ol>
*
* <p>{@link Helper#runSerialized} allows you to schedule a task to be run in the Channel Executor.
* <p>{@link Helper#runSerialized Helper.runSerialized()} allows you to schedule a task to be run in
* the Channel Executor.
*
* <h3>The canonical implementation pattern</h3>
*
@ -93,15 +95,16 @@ import javax.annotation.concurrent.ThreadSafe;
* Subchannel(s) and their latest connectivity states. These states are mutated within the Channel
* Executor.
*
* <p>A typical {@link SubchannelPicker} holds a snapshot of these states. It may have its own
* states, e.g., a picker from a round-robin load-balancer may keep a pointer to the next
* Subchannel, which are typically mutated by multiple threads. The picker should only mutate its
* own state, and should not mutate or re-acquire the states of the LoadBalancer. This way
* the picker only needs to synchronize its own states, which is typically trivial.
* <p>A typical {@link SubchannelPicker SubchannelPicker} holds a snapshot of these states. It may
* have its own states, e.g., a picker from a round-robin load-balancer may keep a pointer to the
* next Subchannel, which are typically mutated by multiple threads. The picker should only mutate
* its own state, and should not mutate or re-acquire the states of the LoadBalancer. This way the
* picker only needs to synchronize its own states, which is typically trivial to implement.
*
* <p>When the LoadBalancer states changes, e.g., Subchannels has become or stopped being READY, and
* we want subsequent RPCs to use the latest list of READY Subchannels, LoadBalancer would create
* a new picker, which holds a snapshot of the latest Subchannel list.
* a new picker, which holds a snapshot of the latest Subchannel list. Refer to the javadoc of
* {@link #handleSubchannelState handleSubchannelState()} how to do this properly.
*
* <p>No synchronization should be necessary between LoadBalancer and its pickers if you follow
* the pattern above. It may be possible to implement in a different way, but that would usually
@ -133,6 +136,20 @@ public abstract class LoadBalancer2 {
/**
* Handles a state change on a Subchannel.
*
* <p>The initial state of a Subchannel is IDLE. You won't get a notification for the initial IDLE
* state.
*
* <p>If the new state is not SHUTDOWN, this method should create a new picker and call {@link
* Helper#updatePicker Helper.updatePicker()}. Failing to do so may result in unnecessary delays
* of RPCs. Please refer to {@link PickResult#withSubchannel PickResult.withSubchannel()}'s
* javadoc for more information.
*
* <p>SHUTDOWN can only happen in two cases. One is that LoadBalancer called {@link
* Subchannel#shutdown} earlier, thus it should have already discarded this Subchannel. The other
* is that Channel is doing a {@link ManagedChannel#shutdownNow forced shutdown} or has already
* terminated, thus there won't be further requests to LoadBalancer. Therefore, SHUTDOWN can be
* safely ignored.
*
* @param subchannel the involved Subchannel
* @param stateInfo the new state
*/
@ -162,13 +179,25 @@ public abstract class LoadBalancer2 {
}
/**
* A balancing decision made by {@link SubchannelPicker} for an RPC.
* A balancing decision made by {@link SubchannelPicker SubchannelPicker} for an RPC.
*
* <p>The outcome of the decision will be one of the following:
* <ul>
* <li>Proceed: if a Subchannel is provided via {@link #withSubchannel withSubchannel()}, and is
* in READY state when the RPC tries to start on it, the RPC will proceed on that
* Subchannel.</li>
* <li>Error: if an error is provided via {@link #withError withError()}, and the RPC is not
* wait-for-ready (i.e., {@link CallOptions#withWaitForReady} was not called), the RPC will
* fail immediately with the given error.</li>
* <li>Buffer: in all other cases, the RPC will be buffered in the Channel, until the next
* picker is provided via {@link Helper#updatePicker Helper.updatePicker()}, when the RPC
* will go through the same picking process again.</li>
* </ul>
*/
@Immutable
public static final class PickResult {
private static final PickResult NO_RESULT = new PickResult(null, Status.OK);
// A READY channel, or null
@Nullable private final Subchannel subchannel;
// An error to be propagated to the application if subchannel == null
// Or OK if there is no error.
@ -181,12 +210,64 @@ public abstract class LoadBalancer2 {
}
/**
* A decision to proceed the RPC on a Subchannel. The state of the Subchannel is supposed to be
* {@link ConnectivityState#READY}. However, since such decisions are racy, a non-READY
* Subchannel will not fail the RPC, but will only leave it buffered.
* A decision to proceed the RPC on a Subchannel.
*
* <p>Only Subchannels returned by {@link Helper#createSubchannel} will work. DO NOT try to
* use your own implementations of Subchannels, as they won't work.
* <p>Only Subchannels returned by {@link Helper#createSubchannel Helper.createSubchannel()}
* will work. DO NOT try to use your own implementations of Subchannels, as they won't work.
*
* <p>When the RPC tries to use the return Subchannel, which is briefly after this method
* returns, the state of the Subchannel will decide where the RPC would go:
*
* <ul>
* <li>READY: the RPC will proceed on this Subchannel.</li>
* <li>IDLE: the RPC will be buffered. Subchannel will attempt to create connection.</li>
* <li>All other states: the RPC will be buffered.</li>
* </ul>
*
* <p><strong>All buffered RPCs will stay buffered</strong> until the next call of {@link
* Helper#updatePicker Helper.updatePicker()}, which will trigger a new picking process.
*
* <p>Note that Subchannel's state may change at the same time the picker is making the
* decision, which means the decision may be made with (to-be) outdated information. For
* example, a picker may return a Subchannel known to be READY, but it has become IDLE when is
* about to be used by the RPC, which makes the RPC to be buffered. The LoadBalancer will soon
* learn about the Subchannels' transition from READY to IDLE, create a new picker and allow the
* RPC to use another READY transport if there is any.
*
* <p>You will want to avoid running into a situation where there are READY Subchannels out
* there but some RPCs are still buffered for longer than a brief time.
* <ul>
* <li>This can happen if you return Subchannels with states other than READY and IDLE. For
* example, suppose you round-robin on 2 Subchannels, in READY and CONNECTING states
* respectively. If the picker ignores the state and pick them equally, 50% of RPCs will
* be stuck in buffered state until both Subchannels are READY.</li>
* <li>This can also happen if you don't create a new picker at key state changes of
* Subchannels. Take the above round-robin example again. Suppose you do pick only READY
* and IDLE Subchannels, and initially both Subchannels are READY. Now one becomes IDLE,
* then CONNECTING and stays CONNECTING for a long time. If you don't create a new picker
* in response to the CONNECTING state to exclude that Subchannel, 50% of RPCs will hit it
* and be buffered even though the other Subchannel is READY.</li>
* </ul>
*
* <p>In order to prevent unnecessary delay of RPCs, the rules of thumb are:
* <ol>
* <li>The picker should only pick Subchannels that are known as READY or IDLE. Whether to
* pick IDLE Subchannels depends on whether you want Subchannels to connect on-demand or
* actively:
* <ul>
* <li>If you want connect-on-demand, include IDLE Subchannels in your pick results,
* because when an RPC tries to use an IDLE Subchannel, the Subchannel will try to
* connect.</li>
* <li>If you want Subchannels to be always connected even when there is no RPC, you
* would call {@link Subchannel#requestConnection Subchannel.requestConnection()}
* whenever the Subchannel has transitioned to IDLE, then you don't need to include
* IDLE Subchannels in your pick results.</li>
* </ul></li>
* <li>Always create a new picker and call {@link Helper#updatePicker Helper.updatePicker()}
* whenever {@link #handleSubchannelState handleSubchannelState()} is called, unless the
* new state is SHUTDOWN. See {@code handleSubchannelState}'s javadoc for more
* details.</li>
* </ol>
*/
public static PickResult withSubchannel(Subchannel subchannel) {
return new PickResult(Preconditions.checkNotNull(subchannel, "subchannel"), Status.OK);
@ -212,7 +293,8 @@ public abstract class LoadBalancer2 {
}
/**
* The Subchannel if this result was created by {@link #withSubchannel}, or null otherwise.
* The Subchannel if this result was created by {@link #withSubchannel withSubchannel()}, or
* null otherwise.
*/
@Nullable
public Subchannel getSubchannel() {
@ -220,8 +302,8 @@ public abstract class LoadBalancer2 {
}
/**
* The status associated with this result. Non-{@code OK} if created with {@link #withError},
* or {@code OK} otherwise.
* The status associated with this result. Non-{@code OK} if created with {@link #withError
* withError}, or {@code OK} otherwise.
*/
public Status getStatus() {
return status;
@ -241,7 +323,8 @@ public abstract class LoadBalancer2 {
/**
* Creates a Subchannel, which is a logical connection to the given group of addresses which are
* considered equivalent. The {@code attrs} are custom attributes associated with this
* Subchannel, and can be accessed later through {@link Subchannel#getAttributes}.
* Subchannel, and can be accessed later through {@link Subchannel#getAttributes
* Subchannel.getAttributes()}.
*
* <p>The LoadBalancer is responsible for closing unused Subchannels, and closing all
* Subchannels within {@link #shutdown}.
@ -261,11 +344,12 @@ public abstract class LoadBalancer2 {
/**
* Set a new picker to the channel.
*
* <p>When a new picker is provided via {@link Helper#updatePicker}, the channel will apply the
* picker on all buffered RPCs, by calling {@link SubchannelPicker#pickSubchannel}.
* <p>When a new picker is provided via {@code updatePicker()}, the channel will apply the
* picker on all buffered RPCs, by calling {@link SubchannelPicker#pickSubchannel
* SubchannelPicker.pickSubchannel()}.
*
* <p>The channel will hold the picker and use it for all RPCs, until {@link #updatePicker} is
* called again and a new picker replaces the old one. If {@link #updatePicker} has never been
* <p>The channel will hold the picker and use it for all RPCs, until {@code updatePicker()} is
* called again and a new picker replaces the old one. If {@code updatePicker()} has never been
* called, the channel will buffer all RPCs until a picker is provided.
*/
public abstract void updatePicker(SubchannelPicker picker);
@ -288,7 +372,7 @@ public abstract class LoadBalancer2 {
}
/**
* A logical connection to a server, or a group of equivalent servers represented by an {@link
* A logical connection to a server, or a group of equivalent servers represented by an {@link
* EquivalentAddressGroup}.
*
* <p>It maintains at most one physical connection (aka transport) for sending new RPCs, while
@ -296,12 +380,14 @@ public abstract class LoadBalancer2 {
*
* <p>If there isn't an active transport yet, and an RPC is assigned to the Subchannel, it will
* create a new transport. It won't actively create transports otherwise. {@link
* #requestConnection} can be used to ask Subchannel to create a transport if there isn't any.
* #requestConnection requestConnection()} can be used to ask Subchannel to create a transport if
* there isn't any.
*/
@ThreadSafe
public abstract static class Subchannel {
/**
* Shuts down the Subchannel. No new RPCs will be accepted.
* Shuts down the Subchannel. After this method is called, this Subchannel should no longer
* be returned by the latest {@link SubchannelPicker picker}, and can be safely discarded.
*/
public abstract void shutdown();
@ -316,7 +402,7 @@ public abstract class LoadBalancer2 {
public abstract EquivalentAddressGroup getAddresses();
/**
* The same attributes passed to {@link io.grpc.LoadBalancer2.Helper#createSubchannel}.
* The same attributes passed to {@link Helper#createSubchannel Helper.createSubchannel()}.
* LoadBalancer can use it to attach additional information here, e.g., the shard this
* Subchannel belongs to.
*/
@ -326,7 +412,7 @@ public abstract class LoadBalancer2 {
@ThreadSafe
public abstract static class Factory {
/**
* Creates a {@link LoadBalancer} that will be used inside a channel.
* Creates a {@link LoadBalancer2} that will be used inside a channel.
*/
public abstract LoadBalancer2 newLoadBalancer(Helper helper);
}