istio.io/_docs/tasks/rate-limiting.md

title	overview	order	layout	type
Enabling Rate Limits	This task shows you how to use Istio to dynamically limit the traffic to a service.	80	docs	markdown

{% include home.html %}

This task shows you how to use Istio to dynamically limit the traffic to a service.

Before you begin

• Setup Istio by following the instructions in the Installation guide.

• Deploy the BookInfo sample application.

• Initialize the application version routing by either first doing the request routing task or by running following commands:

  istioctl create -f samples/apps/bookinfo/route-rule-all-v1.yaml
  istioctl replace -f samples/apps/bookinfo/route-rule-reviews-v2-v3.yaml
  

• Ensure that you can use istioctl mixer by setting up port forwading if needed.

Rate limits

Istio enables users to rate limit traffic to a service.

Consider ratings as an external paid service like Rotten Tomatoes® with 1qps free quota. Using Istio we can ensure that 1qps is not breached.

1. Configure mixer with the rate limit:

   istioctl mixer rule create global ratings.default.svc.cluster.local -f ratelimit.yaml
   

   where ratelimit.yaml is

   rules:
   - aspects:
     - kind: quotas
       params:
         quotas:
         - descriptorName: RequestCount
           maxAmount: 1
           expiration: 1s
   

   istioctl sets configuration for subject=ratings.default.svc.cluster.local

2. Generate load on the productpage with the following command:

   while true; do curl -s -o /dev/null http://$GATEWAY_URL/productpage; done
   

   If you refresh the productpage (http://$GATEWAY_URL/productpage) while the load generator is running (i.e., generating more than 5 req/s), the traffic generated by your browser will be rate limited. When reviews service is unable to access ratings service you stop seeing stars on the UI.

Conditional rate limits

In the previous example we applied a rate limit to the ratings service without regard to any other attributes. It is possible to conditionally apply rate limits based on attributes like the source of the traffic.

The following configuration applies a 1qps rate limit only to version v3 of reviews.

1. Configure mixer with the conditional rate limit:

   istioctl mixer rule create global ratings.default.svc.cluster.local -f ratelimit-conditional.yaml
   

   where ratelimit-conditional.yaml is

   rules:
   - selector: source.labels["app"]=="reviews" && source.labels["version"] == "v3"  
     aspects:
     - kind: quotas
       params:
         quotas:
         - descriptorName: RequestCount
           maxAmount: 1
           expiration: 1s
   

2. Generate load on the productpage with the following command:

   while true; do curl -s -o /dev/null http://$GATEWAY_URL/productpage; done
   

   If you refresh the productpage (http://$GATEWAY_URL/productpage) while the load generator is running (i.e., generating more than 1 req/s), the traffic generated by your browser will be rate limited.

   Since the reviews-v3 service is unable to access the ratings service freely, we should see red stars very infrequently.

Understanding rate limits

In the preceding examples we saw how Mixer applies rate limits to requests that match certain conditions.

Every distinct rate limit configuration represents a counter. If the number of requests in the last expiration duration exceed maxAmount, Mixer returns a RESOURCE_EXHAUSTED message to the proxy. The proxy in turn returns status HTTP 429 to the caller.

Multiple rate limits may apply to the same request.

Mixer MemQuota adapter uses a sliding window of sub second resolution to enforce rate limits.

Consider the following example

descriptorName: RequestCount
maxAmount: 5000
expiration: 5s
labels:
  label1: target.service

This defines a set of counters with a limit of 5000 per every 5 seconds. Individual counters within the set are identified by unique keys. A key is formed on the request path by using all parameters of the configuration. Here we introduce the notion of labels that enable creation of more granular counter keys. When a request arrives at Mixer with target.service=ratings it forms the following counter key.

$aspect_id;RequestCount;maxAmount=5000;expiration=5s;label1=ratings

Using target.service in the counter key enables independent rate limits for every service. In absence of target.service as part of the key, the same counter location is used by all services resulting in combined rate limit of 5000 requests per 5 seconds

Mixer supports an arbitrary number of labels by defining QuotaDescriptors.

name: RequestCount
rate_limit: true
labels:
   label1: 1 # STRING

Here we define RequestCount quota descriptor that takes 1 string label. We recommend using meaningful label names even though label names are arbitrary.

name: RequestCount_byService_byUser
rate_limit: true
labels:
   service: 1 # STRING
   user: 1 # STRING

Mixer expects user,service labels when the RequestCount_byService_byUser descriptor is used and produces the following config validation error if any labels are missing.

* quotas: aspect validation failed: 1 error occurred:
* quotas[RequestCount_byService_byUser].labels: wrong dimensions: descriptor expects 2 labels, found 0 labels

What's next

• Learn more about Mixer and Mixer Config.

• Discover the full Attribute Vocabulary.

• Read the reference guide to Writing Config.