mirror of https://github.com/istio/istio.io.git
Updated traffic management concepts docs (#247)
* Updated/restructured overview docs * Implement Martin's edits * Implement Martin's edits * Traffic management doc edits * Minor change * Implementing Martin's edits * Implementing Martin's edits * Implementing rshriram's edits * Implementing Martin's edits * Implementing Martin's edits
This commit is contained in:
parent
936031aebc
commit
85effe2247
|
|
@ -8,6 +8,9 @@ layout: docs
|
||||||
type: markdown
|
type: markdown
|
||||||
---
|
---
|
||||||
|
|
||||||
|
This page describes how Istio load balances traffic across instances of a
|
||||||
|
service in a service mesh.
|
||||||
|
|
||||||
**Service registration:** Istio assumes the presence of a service registry
|
**Service registration:** Istio assumes the presence of a service registry
|
||||||
to keep track of the pods/VMs of a service in the application. It also
|
to keep track of the pods/VMs of a service in the application. It also
|
||||||
assumes that new instances of a service are automatically registered with
|
assumes that new instances of a service are automatically registered with
|
||||||
|
|
@ -32,6 +35,7 @@ the load balancing pool. While Envoy supports several
|
||||||
Istio currently allows three load balancing modes:
|
Istio currently allows three load balancing modes:
|
||||||
round robin, random, and weighted least request.
|
round robin, random, and weighted least request.
|
||||||
|
|
||||||
|
|
||||||
In addition to load balancing, Envoy periodically checks the health of each
|
In addition to load balancing, Envoy periodically checks the health of each
|
||||||
instance in the pool. Envoy follows a circuit breaker style pattern to
|
instance in the pool. Envoy follows a circuit breaker style pattern to
|
||||||
classify instances as unhealthy or healthy based on their failure rates for
|
classify instances as unhealthy or healthy based on their failure rates for
|
||||||
|
|
@ -39,7 +43,8 @@ the health check API call. In other words, when the number of health
|
||||||
check failures for a given instance exceeds a pre-specified threshold, it
|
check failures for a given instance exceeds a pre-specified threshold, it
|
||||||
will be ejected from the load balancing pool. Similarly, when the number of
|
will be ejected from the load balancing pool. Similarly, when the number of
|
||||||
health checks that pass exceed a pre-specified threshold, the instance will
|
health checks that pass exceed a pre-specified threshold, the instance will
|
||||||
be added back into the load balancing pool.
|
be added back into the load balancing pool. You can find out more about Envoy's
|
||||||
|
failure-handling features in [Handling Failures](./handling-failures.html).
|
||||||
|
|
||||||
Services can actively shed load by responding with a HTTP 503 to a health
|
Services can actively shed load by responding with a HTTP 503 to a health
|
||||||
check. In such an event, the service instance will be immediately removed
|
check. In such an event, the service instance will be immediately removed
|
||||||
|
|
|
||||||
|
|
@ -8,25 +8,27 @@ layout: docs
|
||||||
type: markdown
|
type: markdown
|
||||||
---
|
---
|
||||||
|
|
||||||
Istio-Manager service is responsible for the lifecycle of
|
Istio-Manager is responsible for the lifecycle of Envoy instances deployed
|
||||||
Envoy instances deployed across the Istio service mesh. It exposes
|
across the Istio service mesh.
|
||||||
APIs for [service discovery](https://lyft.github.io/envoy/docs/configuration/cluster_manager/sds_api.html), dynamic updates to [load balancing pools](https://lyft.github.io/envoy/docs/configuration/cluster_manager/cds.html) and
|
|
||||||
[routing tables](https://lyft.github.io/envoy/docs/configuration/http_conn_man/rds.html). These API decouple Envoy from
|
|
||||||
platform-specific nuances, simplifying the design and increasing
|
|
||||||
portability across platforms.
|
|
||||||
|
|
||||||
<figure><img src="./img/manager/ManagerAdapters.svg" alt="Istio-Manager's overall architecture." title="Istio-Manager Architecture" />
|
<figure><img src="./img/manager/ManagerAdapters.svg" alt="Istio-Manager's overall architecture." title="Istio-Manager Architecture" />
|
||||||
<figcaption>Istio-Manager Architecture</figcaption></figure>
|
<figcaption>Istio-Manager Architecture</figcaption></figure>
|
||||||
|
|
||||||
As illustrated in the figure above, Istio-Manager maintains a canonical representation of services in the
|
As illustrated in the figure above, Istio-Manager maintains a canonical
|
||||||
mesh that is independent of the underlying platform. Platform-specific
|
representation of services in the mesh that is independent of the underlying
|
||||||
adapters in the manager are responsible for populating this canonical model
|
platform. Platform-specific adapters in the manager are responsible for
|
||||||
appropriately. For example, the Kubernetes adapter in the Istio-Manager
|
populating this canonical model appropriately. For example, the Kubernetes
|
||||||
implements the necessary controllers to watch the Kubernetes API server for
|
adapter in the Istio-Manager implements the necessary controllers to watch the
|
||||||
changes to the pod registration information, ingress resources, and third
|
Kubernetes API server for changes to the pod registration information, ingress
|
||||||
party resources that store traffic management rules. This data is
|
resources, and third party resources that store traffic management rules.
|
||||||
translated into the canonical representation. Envoy-specific configuration
|
This data is translated into the canonical representation. Envoy-specific
|
||||||
is generated based on the canonical representation.
|
configuration is generated based on the canonical representation.
|
||||||
|
|
||||||
|
Istio-Manager exposes APIs for [service discovery](https://lyft.github.io/envoy/docs/configuration/cluster_manager/sds_api.html),
|
||||||
|
dynamic updates to [load balancing pools](https://lyft.github.io/envoy/docs/configuration/cluster_manager/cds.html)
|
||||||
|
and [routing tables](https://lyft.github.io/envoy/docs/configuration/http_conn_man/rds.html).
|
||||||
|
These APIs decouple Envoy from platform-specific nuances, simplifying the
|
||||||
|
design and increasing portability across platforms.
|
||||||
|
|
||||||
Operators can specify high-level traffic management rules through the
|
Operators can specify high-level traffic management rules through the
|
||||||
[Istio-Manager's Rules API (TBD)](). These rules are translated into low-level
|
[Istio-Manager's Rules API (TBD)](). These rules are translated into low-level
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
---
|
---
|
||||||
title: Overview
|
title: Overview
|
||||||
overview: Provides a conceptual overview of traffic management principles in Istio and the kind of features enabled by these principles.
|
overview: Provides a conceptual overview of traffic management in Istio and the features it enables.
|
||||||
|
|
||||||
order: 0
|
order: 0
|
||||||
|
|
||||||
|
|
@ -8,34 +8,52 @@ layout: docs
|
||||||
type: markdown
|
type: markdown
|
||||||
---
|
---
|
||||||
|
|
||||||
Existing container orchestration platforms like Kubernetes, and Mesos, and
|
This page provides an overview of how traffic management works
|
||||||
other microservice frameworks allow operators to control when a particular
|
in Istio, including the benefits of its traffic management
|
||||||
set of pods/VMs should receive traffic (e.g., by adding/removing specific
|
principles. It assumes that you've already read [What Is Istio?]({{home}}/docs/concepts/what-is-istio/overview.html)
|
||||||
labels). Unlike existing techniques, Istio decouples traffic flow and infrastructure
|
and are familiar with Istio's high-level architecture. You can
|
||||||
scaling.
|
find out more about individual traffic management features in the other
|
||||||
|
guides in this section.
|
||||||
|
|
||||||
|
## Istio-Manager and Envoy
|
||||||
|
|
||||||
|
The core component used for traffic management in Istio is
|
||||||
|
[Istio-Manager](./manager.html), which manages and configures all the Envoy
|
||||||
|
proxy instances deployed in a particular Istio service mesh. It lets you
|
||||||
|
specify what rules you want to use to route traffic between Envoy proxies
|
||||||
|
and configure failure recovery features such as timeouts, retries, and
|
||||||
|
circuit breakers. It also maintains a canonical model of all the services
|
||||||
|
in the mesh and uses this to let Envoys know about the other instances in
|
||||||
|
the mesh via its discovery service.
|
||||||
|
|
||||||
|
Each Envoy instance maintains [load balancing information](./load-balancing.html)
|
||||||
|
based on the information it gets from Istio-Manager and periodic health-checks
|
||||||
|
of other instances in its load-balancing pool, allowing it to intelligently
|
||||||
|
distribute traffic between destination instances while following its specified
|
||||||
|
routing rules.
|
||||||
|
|
||||||
|
## Traffic management benefits
|
||||||
|
|
||||||
|
Using Istio's traffic management model essentially decouples traffic flow
|
||||||
|
and infrastructure scaling, letting operators specify via Istio-Manager what
|
||||||
|
rules they want traffic to follow rather than which specific pods/VMs should
|
||||||
|
receive traffic - Istio-Manager and intelligent Envoy proxies look after the
|
||||||
|
rest. So, for example, you can specify via Istio-Manager that you want 5%
|
||||||
|
of traffic for a particular service to go to a canary version irrespective
|
||||||
|
of the size of the canary deployment, or send traffic to a particular version
|
||||||
|
depending on the content of the request.
|
||||||
|
|
||||||
|
|
||||||
<figure><img style="max-width:85%;" src="./img/manager/TrafficManagementOverview.svg" alt="Traffic Management with Istio" title="Traffic Management with Istio" />
|
<figure><img style="max-width:85%;" src="./img/manager/TrafficManagementOverview.svg" alt="Traffic Management with Istio" title="Traffic Management with Istio" />
|
||||||
<figcaption>Traffic Management with Istio</figcaption></figure>
|
<figcaption>Traffic Management with Istio</figcaption></figure>
|
||||||
|
|
||||||
As illustrated in the figure above, Istio allows operators to control what
|
Decoupling traffic flow from infrastructure scaling like this allows Istio
|
||||||
percentage of traffic should be routed to these pods or which requests
|
to provide a variety of traffic management features that live outside the
|
||||||
(e.g., those containing specific headers) should be routed. These
|
application code. As well as dynamic [request routing](request-routing.html)
|
||||||
capabilities are available for both edge and mid-tier services, benefiting
|
for A/B testing, gradual rollouts, and canary releases, it also handles
|
||||||
all development teams (not just those working on user-facing services).
|
[failure recovery](handling-failures.html) using timeouts, retries, and
|
||||||
|
circuit breakers, and finally [fault injection](fault-injection.html) to
|
||||||
|
test the compatibility of failure recovery policies across services. These
|
||||||
|
capabilities are all realized through the Envoy sidecars/proxies deployed
|
||||||
|
across the service mesh.
|
||||||
|
|
||||||
For instance, these capabilities allow operators to control the exact
|
|
||||||
percentage of traffic that each canary release should receive, irrespective
|
|
||||||
of the size of a canary deployment. Dark launches can be easily
|
|
||||||
orchestrated in production while restricting traffic to a specific set of
|
|
||||||
authenticated test users. Operators can safely test these new versions
|
|
||||||
before beginning a canary deployment.
|
|
||||||
|
|
||||||
Decoupling traffic flow from infrastructure scaling allows Istio to provide
|
|
||||||
a variety of traffic management features that reside outside the
|
|
||||||
application code. Features include dynamic
|
|
||||||
[request routing](./request-routing.html) for A/B testing, canary releases,
|
|
||||||
gradual rollouts, etc., [failure recovery](./handling-failures.html)
|
|
||||||
using timeouts, retries, circuit breakers, and finally
|
|
||||||
[fault injection](./fault-injection.html) to test compatibility of
|
|
||||||
failure recovery policies across services. These capabilities are realized
|
|
||||||
through the Envoy sidecars/proxies deployed across the service mesh.
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
---
|
---
|
||||||
title: Request Routing
|
title: Request Routing
|
||||||
overview: Describes how services are modeled within the Istio mesh, the notion of multiple versions of a service, and the communication model between services.
|
overview: Describes how requests are routed between services in an Istio service mesh.
|
||||||
|
|
||||||
order: 20
|
order: 20
|
||||||
|
|
||||||
|
|
@ -8,21 +8,26 @@ layout: docs
|
||||||
type: markdown
|
type: markdown
|
||||||
---
|
---
|
||||||
|
|
||||||
The Istio model of a service is independent of how it is represented
|
This page describes how requests are routed between services in an Istio service mesh.
|
||||||
in the underlying platform (Kubernetes, Mesos, Cloud Foundry,
|
|
||||||
|
## Service model and service versions.
|
||||||
|
|
||||||
|
As described in [Istio-Manager](./manager.html), the canonical representation
|
||||||
|
of services in a particular mesh is maintained by Istio-Manager. The Istio
|
||||||
|
model of a service is independent of how it is represented in the underlying
|
||||||
|
platform (Kubernetes, Mesos, Cloud Foundry,
|
||||||
etc.). Platform-specific adapters are responsible for populating the
|
etc.). Platform-specific adapters are responsible for populating the
|
||||||
internal model representation with various fields, from the metadata found
|
internal model representation with various fields from the metadata found
|
||||||
in the platform.
|
in the platform.
|
||||||
|
|
||||||
## Multiple versions of a service
|
|
||||||
|
|
||||||
Istio introduces the concept of a service version, which is a finer-grained
|
Istio introduces the concept of a service version, which is a finer-grained
|
||||||
way to subdivide service instances by versions (`v1`, `v2`) or environment
|
way to subdivide service instances by versions (`v1`, `v2`) or environment
|
||||||
(`staging`, `prod`). These variants are not necessarily different API
|
(`staging`, `prod`). These variants are not necessarily different API
|
||||||
versions: they could be iterative changes to the same service, deployed in
|
versions: they could be iterative changes to the same service, deployed in
|
||||||
different environments (prod, staging, dev, etc.). Common scenarios where
|
different environments (prod, staging, dev, etc.). Common scenarios where
|
||||||
this occurs include A/B testing, canary rollouts, etc. Istio's [traffic
|
this is used include A/B testing or canary rollouts. Istio's [traffic
|
||||||
management rules](./rules-configuration.html) can refer to the service versions, to provide
|
routing rules](./rules-configuration.html) can refer to service versions to provide
|
||||||
additional control over traffic between services.
|
additional control over traffic between services.
|
||||||
|
|
||||||
## Communication between services
|
## Communication between services
|
||||||
|
|
@ -32,32 +37,36 @@ additional control over traffic between services.
|
||||||
|
|
||||||
As illustrated in the figure above, clients of a service have no knowledge
|
As illustrated in the figure above, clients of a service have no knowledge
|
||||||
of different versions of the service. They can continue to access the
|
of different versions of the service. They can continue to access the
|
||||||
services using the hostname/IP address of the service. Envoy sidecar/proxy
|
services using the hostname/IP address of the service. The Envoy sidecar/proxy
|
||||||
intercepts and forwards all requests/responses between the client and the
|
intercepts and forwards all requests/responses between the client and the
|
||||||
service.
|
service.
|
||||||
|
|
||||||
The actual choice of the service version is determined dynamically by Envoy
|
Envoy determines its actual choice of service version dynamically
|
||||||
based on the routing rules set forth by the operator. This model enables
|
based on the routing rules specified by the operator using Istio-Manager. This
|
||||||
the application code to decouple itself from the evolution of its dependent
|
model enables the application code to decouple itself from the evolution of its dependent
|
||||||
services, while providing other benefits as well (see
|
services, while providing other benefits as well (see
|
||||||
[Mixer]({{home}}/docs/concepts/policy-and-control/mixer.html)). Routing rules allow Envoy to select a version based
|
[Mixer]({{home}}/docs/concepts/policy-and-control/mixer.html)). Routing
|
||||||
on criterion such as (headers, url, etc.), tags associated with
|
rules allow Envoy to select a version based
|
||||||
source/destination and/or by weights assigned to each version.
|
on criterion such as headers, tags associated with
|
||||||
|
source/destination, and/or by weights assigned to each version.
|
||||||
|
|
||||||
Note that Istio does not provide a DNS. Applications can try to resolve the
|
Istio also provides load balancing for traffic to multiple instances of
|
||||||
|
the same service version. You can find out more about this in [Discovery
|
||||||
|
and Load-Balancing](./load-balancing.html).
|
||||||
|
|
||||||
|
Istio does not provide a DNS. Applications can try to resolve the
|
||||||
FQDN using the DNS service present in the underlying platform (kube-dns,
|
FQDN using the DNS service present in the underlying platform (kube-dns,
|
||||||
mesos-dns, etc.).
|
mesos-dns, etc.).
|
||||||
|
|
||||||
## Ingress and Egress Envoys
|
## Ingress and Egress Envoys
|
||||||
|
|
||||||
Istio assumes that all traffic entering and leaving the istio service mesh
|
Istio assumes that all traffic entering and leaving the service mesh
|
||||||
transits through Envoy proxies. By deploying the Envoy proxy in front of
|
transits through Envoy proxies. By deploying the Envoy proxy in front of
|
||||||
services, operators can conduct A/B testing, deploy canary services,
|
services, operators can conduct A/B testing, deploy canary services,
|
||||||
etc. for user-facing services. Similarly, by routing traffic to external
|
etc. for user-facing services. Similarly, by routing traffic to external
|
||||||
web services (e.g., accessing Maps API, or a video service API) via Envoy,
|
web services (for instance, accessing the Maps API, or a video service API) via Envoy,
|
||||||
operators can add failure recovery features such as circuit breakers,
|
operators can add failure recovery features such as circuit breakers,
|
||||||
impose rate limits via Mixer, and provide authentication using
|
impose rate limits via Mixer, and provide authentication using Istio-Auth.
|
||||||
Istio-Auth service.
|
|
||||||
|
|
||||||
<figure><img src="./img/manager/ServiceModel_RequestFlow.svg" alt="Ingress and Egress Envoy." title="Request Flow" />
|
<figure><img src="./img/manager/ServiceModel_RequestFlow.svg" alt="Ingress and Egress Envoy." title="Request Flow" />
|
||||||
<figcaption>Request Flow</figcaption></figure>
|
<figcaption>Request Flow</figcaption></figure>
|
||||||
|
|
|
||||||
|
|
@ -69,11 +69,7 @@ Istio uses an extended version of the [Envoy](https://lyft.github.io/envoy/) pro
|
||||||
Istio leverages Envoy’s many built-in features such as dynamic service discovery, load balancing, TLS termination, HTTP/2 & gRPC proxying, circuit breakers,
|
Istio leverages Envoy’s many built-in features such as dynamic service discovery, load balancing, TLS termination, HTTP/2 & gRPC proxying, circuit breakers,
|
||||||
health checks, staged rollouts with %-based traffic split, fault injection, and rich metrics.
|
health checks, staged rollouts with %-based traffic split, fault injection, and rich metrics.
|
||||||
|
|
||||||
Envoy is deployed as a **sidecar** to the relevant service in the same Kubernetes pod. This allows Istio to extract a wealth of signals about traffic
|
Envoy is deployed as a **sidecar** to the relevant service in the same Kubernetes pod. This allows Istio to extract a wealth of signals about traffic behavior as [attributes]({{home}}/docs/concepts/policy-and-control/attributes.html), which in turn it can use in [Mixer]({{home}}/docs/concepts/policy-and-control/mixer.html) to enforce policy decisions, and be sent to monitoring systems to provide information about the behavior of the entire mesh. The sidecar proxy model also allows you to add Istio capabilities to an existing deployment with no need to rearchitect or rewrite code. You can read more about why we chose this approach in our [Design Goals]({{home}}/docs/concepts/goals.html).
|
||||||
behavior as [attributes]({{home}}/docs/concepts/policy-and-control/attributes.html), which in turn it can use in
|
|
||||||
[Mixer]({{home}}/docs/concepts/policy-and-control/mixer.html) to enforce policy decisions, and be sent to monitoring systems to
|
|
||||||
provide information about the behavior of the entire mesh. The sidecar proxy model also allows you to add Istio capabilities to
|
|
||||||
an existing deployment with no need to rearchitect or rewrite code. You can read more about why we chose this approach in our [Design Goals](./goals.html).
|
|
||||||
|
|
||||||
### Mixer
|
### Mixer
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue