diff --git a/_docs/concepts/traffic-management/load-balancing.md b/_docs/concepts/traffic-management/load-balancing.md index 6c876251a3..3dda57fc18 100644 --- a/_docs/concepts/traffic-management/load-balancing.md +++ b/_docs/concepts/traffic-management/load-balancing.md @@ -8,6 +8,9 @@ layout: docs type: markdown --- +This page describes how Istio load balances traffic across instances of a +service in a service mesh. + **Service registration:** Istio assumes the presence of a service registry to keep track of the pods/VMs of a service in the application. It also assumes that new instances of a service are automatically registered with @@ -27,11 +30,12 @@ dynamically update their load balancing pools accordingly. As illustrated in the figure above, services in the mesh access each other using their DNS names. All HTTP traffic bound to a service is automatically re-routed through Envoy. Envoy distributes the traffic across instances in -the load balancing pool. While Envoy supports several -[sophisticated load balancing algorithms](https://lyft.github.io/envoy/docs/intro/arch_overview/load_balancing.html), -Istio currently allows three load balancing modes: +the load balancing pool. While Envoy supports several +[sophisticated load balancing algorithms](https://lyft.github.io/envoy/docs/intro/arch_overview/load_balancing.html), +Istio currently allows three load balancing modes: round robin, random, and weighted least request. + In addition to load balancing, Envoy periodically checks the health of each instance in the pool. Envoy follows a circuit breaker style pattern to classify instances as unhealthy or healthy based on their failure rates for @@ -39,7 +43,8 @@ the health check API call. In other words, when the number of health check failures for a given instance exceeds a pre-specified threshold, it will be ejected from the load balancing pool. Similarly, when the number of health checks that pass exceed a pre-specified threshold, the instance will -be added back into the load balancing pool. +be added back into the load balancing pool. You can find out more about Envoy's +failure-handling features in [Handling Failures](./handling-failures.html). Services can actively shed load by responding with a HTTP 503 to a health check. In such an event, the service instance will be immediately removed diff --git a/_docs/concepts/traffic-management/manager.md b/_docs/concepts/traffic-management/manager.md index 40bc765ce9..17b69a0a01 100644 --- a/_docs/concepts/traffic-management/manager.md +++ b/_docs/concepts/traffic-management/manager.md @@ -8,25 +8,27 @@ layout: docs type: markdown --- -Istio-Manager service is responsible for the lifecycle of -Envoy instances deployed across the Istio service mesh. It exposes -APIs for [service discovery](https://lyft.github.io/envoy/docs/configuration/cluster_manager/sds_api.html), dynamic updates to [load balancing pools](https://lyft.github.io/envoy/docs/configuration/cluster_manager/cds.html) and -[routing tables](https://lyft.github.io/envoy/docs/configuration/http_conn_man/rds.html). These API decouple Envoy from -platform-specific nuances, simplifying the design and increasing -portability across platforms. +Istio-Manager is responsible for the lifecycle of Envoy instances deployed +across the Istio service mesh.
Istio-Manager's overall architecture.
Istio-Manager Architecture
-As illustrated in the figure above, Istio-Manager maintains a canonical representation of services in the -mesh that is independent of the underlying platform. Platform-specific -adapters in the manager are responsible for populating this canonical model -appropriately. For example, the Kubernetes adapter in the Istio-Manager -implements the necessary controllers to watch the Kubernetes API server for -changes to the pod registration information, ingress resources, and third -party resources that store traffic management rules. This data is -translated into the canonical representation. Envoy-specific configuration -is generated based on the canonical representation. +As illustrated in the figure above, Istio-Manager maintains a canonical +representation of services in the mesh that is independent of the underlying +platform. Platform-specific adapters in the manager are responsible for +populating this canonical model appropriately. For example, the Kubernetes +adapter in the Istio-Manager implements the necessary controllers to watch the +Kubernetes API server for changes to the pod registration information, ingress +resources, and third party resources that store traffic management rules. +This data is translated into the canonical representation. Envoy-specific +configuration is generated based on the canonical representation. + +Istio-Manager exposes APIs for [service discovery](https://lyft.github.io/envoy/docs/configuration/cluster_manager/sds_api.html), +dynamic updates to [load balancing pools](https://lyft.github.io/envoy/docs/configuration/cluster_manager/cds.html) +and [routing tables](https://lyft.github.io/envoy/docs/configuration/http_conn_man/rds.html). +These APIs decouple Envoy from platform-specific nuances, simplifying the +design and increasing portability across platforms. Operators can specify high-level traffic management rules through the [Istio-Manager's Rules API (TBD)](). These rules are translated into low-level diff --git a/_docs/concepts/traffic-management/overview.md b/_docs/concepts/traffic-management/overview.md index 489d9b0db1..4be6b76362 100644 --- a/_docs/concepts/traffic-management/overview.md +++ b/_docs/concepts/traffic-management/overview.md @@ -1,6 +1,6 @@ --- title: Overview -overview: Provides a conceptual overview of traffic management principles in Istio and the kind of features enabled by these principles. +overview: Provides a conceptual overview of traffic management in Istio and the features it enables. order: 0 @@ -8,34 +8,52 @@ layout: docs type: markdown --- -Existing container orchestration platforms like Kubernetes, and Mesos, and -other microservice frameworks allow operators to control when a particular -set of pods/VMs should receive traffic (e.g., by adding/removing specific -labels). Unlike existing techniques, Istio decouples traffic flow and infrastructure -scaling. +This page provides an overview of how traffic management works +in Istio, including the benefits of its traffic management +principles. It assumes that you've already read [What Is Istio?]({{home}}/docs/concepts/what-is-istio/overview.html) +and are familiar with Istio's high-level architecture. You can +find out more about individual traffic management features in the other +guides in this section. + +## Istio-Manager and Envoy + +The core component used for traffic management in Istio is +[Istio-Manager](./manager.html), which manages and configures all the Envoy +proxy instances deployed in a particular Istio service mesh. It lets you +specify what rules you want to use to route traffic between Envoy proxies +and configure failure recovery features such as timeouts, retries, and +circuit breakers. It also maintains a canonical model of all the services +in the mesh and uses this to let Envoys know about the other instances in +the mesh via its discovery service. + +Each Envoy instance maintains [load balancing information](./load-balancing.html) +based on the information it gets from Istio-Manager and periodic health-checks +of other instances in its load-balancing pool, allowing it to intelligently +distribute traffic between destination instances while following its specified +routing rules. + +## Traffic management benefits + +Using Istio's traffic management model essentially decouples traffic flow +and infrastructure scaling, letting operators specify via Istio-Manager what +rules they want traffic to follow rather than which specific pods/VMs should +receive traffic - Istio-Manager and intelligent Envoy proxies look after the +rest. So, for example, you can specify via Istio-Manager that you want 5% +of traffic for a particular service to go to a canary version irrespective +of the size of the canary deployment, or send traffic to a particular version +depending on the content of the request. +
Traffic Management with Istio
Traffic Management with Istio
-As illustrated in the figure above, Istio allows operators to control what -percentage of traffic should be routed to these pods or which requests -(e.g., those containing specific headers) should be routed. These -capabilities are available for both edge and mid-tier services, benefiting -all development teams (not just those working on user-facing services). +Decoupling traffic flow from infrastructure scaling like this allows Istio +to provide a variety of traffic management features that live outside the +application code. As well as dynamic [request routing](request-routing.html) +for A/B testing, gradual rollouts, and canary releases, it also handles +[failure recovery](handling-failures.html) using timeouts, retries, and +circuit breakers, and finally [fault injection](fault-injection.html) to +test the compatibility of failure recovery policies across services. These +capabilities are all realized through the Envoy sidecars/proxies deployed +across the service mesh. -For instance, these capabilities allow operators to control the exact -percentage of traffic that each canary release should receive, irrespective -of the size of a canary deployment. Dark launches can be easily -orchestrated in production while restricting traffic to a specific set of -authenticated test users. Operators can safely test these new versions -before beginning a canary deployment. - -Decoupling traffic flow from infrastructure scaling allows Istio to provide -a variety of traffic management features that reside outside the -application code. Features include dynamic -[request routing](./request-routing.html) for A/B testing, canary releases, -gradual rollouts, etc., [failure recovery](./handling-failures.html) -using timeouts, retries, circuit breakers, and finally -[fault injection](./fault-injection.html) to test compatibility of -failure recovery policies across services. These capabilities are realized -through the Envoy sidecars/proxies deployed across the service mesh. diff --git a/_docs/concepts/traffic-management/request-routing.md b/_docs/concepts/traffic-management/request-routing.md index 75bdb6ae33..224791c43e 100644 --- a/_docs/concepts/traffic-management/request-routing.md +++ b/_docs/concepts/traffic-management/request-routing.md @@ -1,6 +1,6 @@ --- title: Request Routing -overview: Describes how services are modeled within the Istio mesh, the notion of multiple versions of a service, and the communication model between services. +overview: Describes how requests are routed between services in an Istio service mesh. order: 20 @@ -8,21 +8,26 @@ layout: docs type: markdown --- -The Istio model of a service is independent of how it is represented -in the underlying platform (Kubernetes, Mesos, Cloud Foundry, +This page describes how requests are routed between services in an Istio service mesh. + +## Service model and service versions. + +As described in [Istio-Manager](./manager.html), the canonical representation +of services in a particular mesh is maintained by Istio-Manager. The Istio +model of a service is independent of how it is represented in the underlying +platform (Kubernetes, Mesos, Cloud Foundry, etc.). Platform-specific adapters are responsible for populating the -internal model representation with various fields, from the metadata found +internal model representation with various fields from the metadata found in the platform. -## Multiple versions of a service Istio introduces the concept of a service version, which is a finer-grained way to subdivide service instances by versions (`v1`, `v2`) or environment (`staging`, `prod`). These variants are not necessarily different API versions: they could be iterative changes to the same service, deployed in different environments (prod, staging, dev, etc.). Common scenarios where -this occurs include A/B testing, canary rollouts, etc. Istio's [traffic -management rules](./rules-configuration.html) can refer to the service versions, to provide +this is used include A/B testing or canary rollouts. Istio's [traffic +routing rules](./rules-configuration.html) can refer to service versions to provide additional control over traffic between services. ## Communication between services @@ -32,32 +37,36 @@ additional control over traffic between services. As illustrated in the figure above, clients of a service have no knowledge of different versions of the service. They can continue to access the -services using the hostname/IP address of the service. Envoy sidecar/proxy +services using the hostname/IP address of the service. The Envoy sidecar/proxy intercepts and forwards all requests/responses between the client and the service. -The actual choice of the service version is determined dynamically by Envoy -based on the routing rules set forth by the operator. This model enables -the application code to decouple itself from the evolution of its dependent +Envoy determines its actual choice of service version dynamically +based on the routing rules specified by the operator using Istio-Manager. This +model enables the application code to decouple itself from the evolution of its dependent services, while providing other benefits as well (see -[Mixer]({{home}}/docs/concepts/policy-and-control/mixer.html)). Routing rules allow Envoy to select a version based -on criterion such as (headers, url, etc.), tags associated with -source/destination and/or by weights assigned to each version. +[Mixer]({{home}}/docs/concepts/policy-and-control/mixer.html)). Routing +rules allow Envoy to select a version based +on criterion such as headers, tags associated with +source/destination, and/or by weights assigned to each version. -Note that Istio does not provide a DNS. Applications can try to resolve the +Istio also provides load balancing for traffic to multiple instances of +the same service version. You can find out more about this in [Discovery +and Load-Balancing](./load-balancing.html). + +Istio does not provide a DNS. Applications can try to resolve the FQDN using the DNS service present in the underlying platform (kube-dns, mesos-dns, etc.). ## Ingress and Egress Envoys -Istio assumes that all traffic entering and leaving the istio service mesh +Istio assumes that all traffic entering and leaving the service mesh transits through Envoy proxies. By deploying the Envoy proxy in front of services, operators can conduct A/B testing, deploy canary services, etc. for user-facing services. Similarly, by routing traffic to external -web services (e.g., accessing Maps API, or a video service API) via Envoy, +web services (for instance, accessing the Maps API, or a video service API) via Envoy, operators can add failure recovery features such as circuit breakers, -impose rate limits via Mixer, and provide authentication using -Istio-Auth service. +impose rate limits via Mixer, and provide authentication using Istio-Auth.
Ingress and Egress Envoy.
Request Flow
diff --git a/_docs/concepts/what-is-istio/overview.md b/_docs/concepts/what-is-istio/overview.md index bddf3bcfb6..e8630ecbec 100644 --- a/_docs/concepts/what-is-istio/overview.md +++ b/_docs/concepts/what-is-istio/overview.md @@ -69,11 +69,7 @@ Istio uses an extended version of the [Envoy](https://lyft.github.io/envoy/) pro Istio leverages Envoy’s many built-in features such as dynamic service discovery, load balancing, TLS termination, HTTP/2 & gRPC proxying, circuit breakers, health checks, staged rollouts with %-based traffic split, fault injection, and rich metrics. -Envoy is deployed as a **sidecar** to the relevant service in the same Kubernetes pod. This allows Istio to extract a wealth of signals about traffic -behavior as [attributes]({{home}}/docs/concepts/policy-and-control/attributes.html), which in turn it can use in -[Mixer]({{home}}/docs/concepts/policy-and-control/mixer.html) to enforce policy decisions, and be sent to monitoring systems to -provide information about the behavior of the entire mesh. The sidecar proxy model also allows you to add Istio capabilities to -an existing deployment with no need to rearchitect or rewrite code. You can read more about why we chose this approach in our [Design Goals](./goals.html). +Envoy is deployed as a **sidecar** to the relevant service in the same Kubernetes pod. This allows Istio to extract a wealth of signals about traffic behavior as [attributes]({{home}}/docs/concepts/policy-and-control/attributes.html), which in turn it can use in [Mixer]({{home}}/docs/concepts/policy-and-control/mixer.html) to enforce policy decisions, and be sent to monitoring systems to provide information about the behavior of the entire mesh. The sidecar proxy model also allows you to add Istio capabilities to an existing deployment with no need to rearchitect or rewrite code. You can read more about why we chose this approach in our [Design Goals]({{home}}/docs/concepts/goals.html). ### Mixer