istio.io/content/en/docs/overview/dataplane-modes/index.md

11 KiB
Raw Permalink Blame History

title description weight keywords owner test
Sidecar or ambient? Learn about Istio's two dataplane modes and which you should use. 30
sidecar
ambient
istio/wg-docs-maintainers-english n/a

An Istio service mesh is logically split into a data plane and a control plane.

The {{< gloss >}}data plane{{< /gloss >}} is the set of proxies that mediate and control all network communication between microservices. They also collect and report telemetry on all mesh traffic.

The {{< gloss >}}control plane{{< /gloss >}} manages and configures the proxies in the data plane.

Istio supports two main {{< gloss "data plane mode">}}data plane modes{{< /gloss >}}:

  • sidecar mode, which deploys an Envoy proxy along with each pod that you start in your cluster, or running alongside services running on VMs.
  • ambient mode, which uses a per-node Layer 4 proxy, and optionally a per-namespace Envoy proxy for Layer 7 features.

The two modes can interoperate with one anotherα, and you can opt certain namespaces or workloads into each mode.

Sidecar mode

Istio has been built on the sidecar pattern from its first release in 2017. Sidecar mode is well understood and thoroughly battle-tested, but comes with a resource cost and operational overhead.

  • Each application you deploy has an Envoy proxy {{< gloss "injection" >}}injected{{< /gloss >}} as a sidecar
  • All proxies can process both Layer 4 and Layer 7

Ambient mode

Launched in 2022, ambient mode was built to address the shortcomings reported by users of sidecar mode. As of Istio 1.22, it is production-ready for single cluster use cases.

  • All traffic is proxied through a Layer 4-only node proxy
  • Applications can opt in to routing through an Envoy proxy to get Layer 7 features

Choosing between sidecar and ambient

Users often deploy a mesh to enable a zero-trust security posture as a first-step and then selectively enable L7 capabilities as needed. Ambient mesh allows those users to bypass the cost of L7 processing entirely when its not needed.

Sidecar Ambient
Traffic management Full Istio feature set Full Istio feature set (requires using waypoint)
Security Full Istio feature set Full Istio feature set: encryption and L4 authorization in ambient mode. Requires waypoints for L7 authorization.
Observability Full Istio feature set Full Istio feature set: L4 telemetry in ambient mode; L7 observability when using waypoint
Extensibility Full Istio feature set Full Istio feature set (requires using waypoint) α
Adding workloads to the mesh Label a namespace and restart all pods to have sidecars added Label a namespace - no pod restart required
Incremental deployment Binary: sidecar is injected or it isn't Gradual: L4 is always on, L7 can be added by configuration
Lifecycle management Proxies managed by application developer Platform administrator
Utilization of resources Wasteful; CPU and memory resources must be provisioned for worst case usage of each individual pod Waypoint proxies can be auto-scaled like any other Kubernetes deployment.
A workload with many replicas can use one waypoint, vs. each one having its own sidecar.
Average resource cost Large Small
Average latency (p90/p99) 0.63ms-0.88ms Ambient: 0.16ms-0.20ms
Waypoint: 0.40ms-0.50ms
L7 processing steps 2 (source and destination sidecar) 1 (destination waypoint)
Configuration at scale Requires configuration of the scope of each sidecar to reduce configuration Works without custom configuration
Supports "server-first" protocols Requires configuration Yes
Support for Kubernetes Jobs Complicated by long life of sidecar Transparent
Security model Strongest: each workload has its own keys Strong: each node agent has only the keys for workloads on that node
Compromised application pod
gives access to mesh keys
Yes No
Support Stable, including multi-cluster Beta, single-cluster
Platforms supported Kubernetes (any CNI)
Virtual machines
Kubernetes (any CNI)

Layer 4 vs Layer 7 features

The overhead for processing protocols at Layer 7 is substantially higher than processing network packets at Layer 4. For a given service, if your requirements can be met at L4, service mesh can be delivered at substantially lower cost.

Security

L4 L7
Encryption All traffic between pods is encrypted using {{< gloss "mutual tls authentication" >}}mTLS{{< /gloss >}}. N/A—service identity in Istio is based on TLS.
Service-to-service authentication {{< gloss >}}SPIFFE{{< /gloss >}}, via mTLS certificates. Istio issues a short-lived X.509 certificate that encodes the pod's service account identity. N/A—service identity in Istio is based on TLS.
Service-to-service authorization Network-based authorization, plus identity-based policy, e.g.:
  • A can accept inbound calls from only "10.2.0.0/16";
  • A can call B.
Full policy, e.g.:
  • A can GET /foo on B only with valid end-user credentials containing the READ scope.
End-user authentication N/A—we can't apply per-user settings. Local authentication of JWTs, support for remote authentication via OAuth and OIDC flows.
End-user authorization N/A—see above. Service-to-service policies can be extended to require end-user credentials with specific scopes, issuers, principal, audiences, etc.
Full user-to-resource access can be implemented using external authorization, allowing per-request policy with decisions from an external service, e.g. OPA.

Observability

L4 L7
Logging Basic network information: network 5-tuple, bytes sent/received, etc. See Envoy docs. Full request metadata logging, in addition to basic network information.
Tracing Not today; possible eventually with HBONE. Envoy participates in distributed tracing. See Istio overview on tracing.
Metrics TCP only (bytes sent/received, number of packets, etc.). L7 RED metrics: rate of requests, rate of errors, request duration (latency).

Traffic management

L4 L7
Load balancing Connection level only. See TCP traffic shifting task. Per request, enabling e.g. canary deployments, gRPC traffic, etc. See HTTP traffic shifting task.
Circuit breaking TCP only. HTTP settings in addition to TCP.
Outlier detection On connection establishment/failure. On request success/failure.
Rate limiting Rate limit on L4 connection data only, on connection establishment, with global and local rate limiting options. Rate limit on L7 request metadata, per request.
Timeouts Connection establishment only (connection keep-alive is configured via circuit breaking settings). Per request.
Retries Retry connection establishment Retry per request failure.
Fault injection N/A—fault injection cannot be configured on TCP connections. Full application and connection-level faults (timeouts, delays, specific response codes).
Traffic mirroring N/A—HTTP only Percentage-based mirroring of requests to multiple backends.

Supported features

As of Istio 1.22, the following ambient mode features are implemented but are currently in Alpha status:

  • Interoperability with sidecars
  • Istios classic APIs (VirtualService and DestinationRule)
  • Multi-cluster installations
  • DNS proxying
  • IPv6/Dual stack
  • SOCKS5 support (for outbound)

The following features are not yet implemented:

  • Controlled egress traffic
  • Multi-network support
  • VM support