diff --git a/config.yaml b/config.yaml index 05c487e..6a9e75b 100644 --- a/config.yaml +++ b/config.yaml @@ -65,6 +65,10 @@ params: url: https://github.com/grpc icon: fab fa-github + mermaid: + enable: true + theme: "neutral" + markup: goldmark: renderer: {unsafe: true} diff --git a/content/en/docs/guides/custom-load-balancing.md b/content/en/docs/guides/custom-load-balancing.md new file mode 100644 index 0000000..88b48fb --- /dev/null +++ b/content/en/docs/guides/custom-load-balancing.md @@ -0,0 +1,88 @@ +--- +title: Custom Load Balancing Policies +description: >- + Explains how custom load balancing policies can help optimize load balancing + under unique circumstances. +--- + +### Overview + +One of the key features of gRPC is load balancing, which allows requests from +clients to be distributed across multiple servers. This helps prevent any +one server from becoming overloaded and allows the system to scale up by adding +more servers. + +A gRPC load balancing policy is given a list of server IP addresses by the name +resolver. The policy is responsible for maintaining connections (subchannels) +to the servers and picking a connection to use when an RPC is sent. + +### Implementing Your Own Policy + +By default the `pick_first` policy will be used. This policy actually does no +load balancing but just tries each address it gets from the name resolver and +uses the first one it can connect to. By updating the gRPC service config you +can also switch to using `round_robin` that connects to every address it gets +and rotates through the connected backends for each RPC. There are also some +other load balancing policies available, but the exact set varies by language. +If the built-in policies do not meet your needs you can also implement you own +custom policy. + +This involves implementing a load balancer interface in the language you are +using. At a high level, you will have to: + +- Register your implementation in the load balancer registry so that it can +be referred to from the service config +- Parse the JSON configuration object of your implementation. This allows your +load balancer to be configured in the service config with any arbitrary JSON +you choose to support +- Manage what backends to maintain a connection with +- Implement a `picker` that will choose which backend to connect to when an +RPC is made. Note that this needs to be a fast operation as it is on the RPC +call path +- To enable your load balancer, configure it in your service config + +The exact steps vary by language, see the language support section for some +concrete examples in your language. + +```mermaid +flowchart TD +NR(Name Resolver) -->|Provides addresses &\nLB config| LB(Load Balancer) +LB --> |Provides a picker| C(Channel) +C -->|Requests\na subchannel| P(Picker) +LB --> |Manages subchannels\nto backends| SC(Subchannel 1..n) +LB -. Creates .-> P +P --> |Picks one| SC +``` + +### Backend Metrics + +What if your load balancing policy needs to know what is going on with the +backend servers in real-time? For this you can rely on backend metrics. You can +have metrics provided to you either in-band, in the backend RPC responses, or +out-of-band as separate RPCs from the backends. Standard metrics like CPU +and memory utilization are provided but you can also implement your own, custom +metrics. + +For more information on this, please see the custom backend metrics guide (TBD) + +### Service Mesh + +If you have a service mesh setup where a central control plane is coordinating +the configuration of your microservices, you cannot configure your custom load +balancer directly via the service config. But support is provided to do this +with the xDS protocol that your control plane uses to communicate with your +gRPC clients. Please refer to your control plane documentation to determine how +custom load balancing configuration is supported. + +For more details, please see gRPC [proposal A52]. + +### Language Support + +| Language | Example | Notes | +|----------|----------------|----------------------------------| +| Java | [Java example] | | +| Go | | Example and xDS support upcoming | +| C++ | | Not yet supported | + +[proposal A52]:https://github.com/grpc/proposal/blob/master/A52-xds-custom-lb-policies.md +[Java example]: https://github.com/grpc/grpc-java/tree/master/examples/src/main/java/io/grpc/examples/customloadbalance