istio
/
istio.io
mirror of https://github.com/istio/istio.io.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
istio.io/archive/v1.12/blog/feed.xml

12661 lines
1010 KiB
XML
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Istio Blog</title><description>Connect, secure, control, and observe services.</description><link>/v1.12</link><image><url>/v1.12/favicons/android-192x192.png</url><link>/v1.12</link></image><category>Service mesh</category><item><title>An easier way to add virtual machines to Istio service mesh</title><description>
&lt;p>&lt;img src="./traffic-flow.png" alt="Virtual Machine Traffic Flow" />&lt;/p>
&lt;p>Some of the complexities involved with joining a virtual machine are due to the sheer number of features that a service mesh provides. But, what if you only need a subset of those features? For example, secure communication from your virtual machine to services running inside your service mesh. With only a few tradeoffs, you can give your virtual machine service mesh features without all of the overhead.&lt;/p>
&lt;p>What about local development? As more and more micro-services are deployed to Kubernetes and as your dependency graph resembles a spider web, it has become increasingly difficult to do local development. What if a local machine could simply join a service mesh and make calls to mesh applications. This solution could potentially save time and money by not requiring developers to wait for their code to be deployed.&lt;/p>
&lt;h2 id="reduce-complexity">Reduce Complexity&lt;/h2>
&lt;p>&lt;img src="./istio-current-vm-process.png" alt="Virtual Machine Istio Installation" />&lt;/p>
&lt;p>Today, adding a virtual machine to your Istio service mesh involves a lot of &lt;a href="/v1.12/docs/setup/install/virtual-machine/">moving parts&lt;/a>. You must create a Kubernetes service account, Istio workload entry and then generate configuration all before on-boarding a single virtual machine. There are also complexities to automating this, especially for auto scaling VMs. Finally you are required to expose Istiod externally to your cluster.&lt;/p>
&lt;p>The complexity of adding a virtual machine comes from the expectation that the VM should participate 100% within the service mesh. For many this is not a necessity, by looking at the actual requirements of your system you may be able to simplify your virtual machine on-boarding and still get the features you need.&lt;/p>
&lt;p>So what are some use cases that could be met but yet still make virtual machines easier to work with service mesh?&lt;/p>
&lt;h3 id="single-direction-traffic-flow">Single Direction Traffic Flow&lt;/h3>
&lt;p>Sometimes a virtual machine just needs to talk securely to applications within the service mesh. This is often the case when migrating VM based applications to Kubernetes in which other VMs may have depended on those applications. With the described approach below you can still achieve this without all of the operational overhead as shown above.&lt;/p>
&lt;p>&lt;img src="./single-direction-traffic-flow.png" alt="Single Direction Traffic Flow" />&lt;/p>
&lt;h3 id="developer-access-to-service-mesh">Developer Access to Service Mesh&lt;/h3>
&lt;p>Engineers often do not have the resources to run all of the required micro-services for their environment. The approach below explains how you can achieve this in the same way that virtual machines securely communicate with mesh applications.&lt;/p>
&lt;p>&lt;img src="./local-machine.png" alt="Local Machine Service Mesh" />&lt;/p>
&lt;h2 id="decouple-envoy-and-istio">Decouple Envoy and Istio&lt;/h2>
&lt;p>The largest amount of complexity as it relates to virtual machines is the connecting of envoy to istiod to get its configuration. A simpler approach is to just not connect them anymore. Even though Istio will no longer know about the virtual machines that are communicating in the mesh, that communication can still be secure and authenticated. The trick is issuing virtual machines their own workload certificates that are rooted in the same trust chain as the mesh workloads. This also means that the end user will be responsible for configuring envoy manually on the virtual machine. For most this shouldn&amp;rsquo;t be an issue because it is not expected that it will change very often.&lt;/p>
&lt;h2 id="a-simpler-on-boarding-experience">A Simpler On-boarding Experience&lt;/h2>
&lt;p>&lt;img src="./traffic-flow.png" alt="Virtual Machine Traffic Flow" />&lt;/p>
&lt;p>We can achieve a simpler setup by utilizing some built-in Istio features. First we need to expose a secure tunnel for applications outside the mesh to communicate with applications within.&lt;/p>
&lt;p>To do this we simply need to create an Istio east-west gateway and enable &lt;code>AUTO_PASSTHROUGH&lt;/code>. This automatically configures the east-west gateway to pass traffic through to the correct service over mTLS. This gives your virtual machine end to end authenticated encryption with the application its trying to reach.&lt;/p>
&lt;p>&lt;img src="./virtual-machine-on-boarding-steps.png" alt="Virtual Machine On-boarding Steps" />&lt;/p>
&lt;p>Due to the complexity involved in configuring envoy to talk to istiod, it is more practical to directly configure the virtual machine envoy. At first this sounds quite daunting, but due to the reduced complexity we only need to enable a few features to make this work. Envoy will need to be configured to know about each service mesh application that the virtual machine will need to communicate with. We then will configure these as &lt;code>clusters&lt;/code> within envoy and set them up to use mTLS communication passing through the east-west gateway in the service mesh. Secondly a listener will need to be exposed to handle incoming traffic from the virtual machine application. Finally certificates will need to be issued for each virtual machine that share the same root of trust as the service mesh applications. This allows end to end encryption as well as the ability to authorize which applications the virtual machine can communicate with.&lt;/p>
&lt;h3 id="easier-to-automate">Easier to Automate&lt;/h3>
&lt;p>Given that no initialization has to occur on the service mesh cluster when on-boarding a virtual machine, it is much easier to automate. The configuration needed for the virtual machine envoy can be added to your pipeline; the envoy container can either be pulled via docker, or added to your image building infrastructure; the mTLS certificates can also be provisioned and maintained by a third party such as Hashicorp&amp;rsquo;s Vault.&lt;/p>
&lt;h3 id="more-runtime-support">More Runtime Support&lt;/h3>
&lt;p>Due to the fact that this installation method does not require access to the underlying OS networking. You can run this approach in more types of environments including Windows and Docker. The only requirement is that your Envoy include the Istio extensions found &lt;a href="https://github.com/istio/proxy/tree/master/extensions">here&lt;/a>. Using Docker, you can now run the Envoy proxy on your local machine and communicate with the service mesh directly.&lt;/p>
&lt;p>&lt;img src="runtime-support.png" alt="Runtime Support" />&lt;/p>
&lt;h2 id="advanced-use-cases">Advanced Use Cases&lt;/h2>
&lt;h3 id="grpc-to-json">gRPC to JSON&lt;/h3>
&lt;p>This technique can also be leveraged to enable virtual machine applications to communicate with gRPC applications without having to implement the gRPC endpoints. Using envoys gRPC / JSON transformation, the virtual machine application can communicate with its local envoy over REST and envoy will translate that to gRPC.&lt;/p>
&lt;p>&lt;img src="./grpc-json-transcoding.png" alt="gRPC to JSON Transformation" />&lt;/p>
&lt;h3 id="multi-direction">Multi Direction&lt;/h3>
&lt;p>Even though your service mesh may not know about the virtual machines that are communicating with it, you can still add them as external endpoints using Service Entries. That service entry could be an HTTPS Load Balancer endpoint that manages traffic to multiple virtual machines. This setup is still often more feasible than fully on-boarding virtual machines into the virtual mesh.&lt;/p>
&lt;p>&lt;img src="./multi-direction-traffic-flow.png" alt="Multi Direction Traffic Flow" />&lt;/p>
&lt;h3 id="forwarding-proxy">Forwarding Proxy&lt;/h3>
&lt;p>Maybe installing envoy on every virtual machine is still too complex. An alternative is to run envoy (or an autoscaling group) to run on its own virtual machine and act as a forwarding proxy into the mesh. This is a much simpler solution to accessing mesh services as the virtual machines that run the applications are left untouched.&lt;/p>
&lt;p>&lt;img src="./forwarding-proxy.png" alt="Forwarding Proxy" />&lt;/p>
&lt;h2 id="part-2">Part 2…&lt;/h2>
&lt;p>In part 2, I will explain how to configure Istio as well as a virtual machine to communicate within the mesh. If you would like a preview, feel free to reach out to nick.nellis@solo.io&lt;/p>
&lt;h3 id="special-thanks">Special Thanks&lt;/h3>
&lt;p>A special thanks to Dave Ortiz for this virtual machine idea and congrats to Constant Contact &lt;a href="https://github.com/istio/istio.io/pull/10571">a new registered Istio user!&lt;/a>&lt;/p></description><pubDate>Mon, 20 Dec 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/simple-vms/</link><author>Nick Nellis (Solo.io)</author><guid isPermaLink="true">/v1.12/blog/2021/simple-vms/</guid><category>virtualmachine</category><category>Istio</category><category>networking</category><category>envoy</category></item><item><title>Announcing the alpha availability of WebAssembly Plugins</title><description>
&lt;p>&lt;a href="../wasm-progress/">Istio 1.9 introduced&lt;/a> experimental support for WebAssembly (Wasm) module distribution and a Wasm extensions ecosystem repository with canonical examples and use cases for extension development. Over the past 9 months, the Istio, Envoy, and Proxy-Wasm communities have continued our joint effort to make Wasm extensibility stable, reliable, and easy to adopt, and we are pleased to announce Alpha support for Wasm extensibility in Istio 1.12! In the following sections, we&amp;rsquo;ll walk through the updates that have been made to the Wasm support for the 1.12 release.&lt;/p>
&lt;h2 id="new-wasmplugin-api">New WasmPlugin API&lt;/h2>
&lt;p>With the new &lt;code>WasmPlugin&lt;/code> CRD in the &lt;code>extensions.istio.io&lt;/code> namespace, we’re introducing a new high-level API for extending the functionality of the Istio proxy with custom Wasm modules. This effort builds on the excellent work that has gone into the &lt;a href="https://github.com/proxy-wasm">Proxy-Wasm&lt;/a> specification and implementation over the last two years. From now on, you no longer need to use &lt;code>EnvoyFilter&lt;/code> resources to add custom Wasm modules to your proxies. Instead, you can now use a &lt;code>WasmPlugin&lt;/code> resource:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
name: your-filter
spec:
selector:
matchLabels:
app: server
phase: AUTHN
priority: 10
pluginConfig:
someSetting: true
someOtherSetting: false
youNameIt:
- first
- second
url: docker.io/your-org/your-filter:1.0.0
&lt;/code>&lt;/pre>
&lt;p>There are a lot of similarities and a few differences between &lt;code>WasmPlugin&lt;/code> and &lt;code>EnvoyFilter&lt;/code>, so let’s go through the fields one by one.&lt;/p>
&lt;p>The above example deploys a Wasm module to all workloads (including gateway pods) that match the &lt;code>selector&lt;/code> field - this very much works the same as in an &lt;code>EnvoyFilter&lt;/code>.&lt;/p>
&lt;p>The next field below that is the &lt;code>phase&lt;/code>. This determines where in the proxy’s filter chain the Wasm module will be injected. We have defined four distinct phases for injection:&lt;/p>
&lt;ul>
&lt;li>&lt;code>AUTHN&lt;/code>: prior to any Istio authentication and authorization filters.&lt;/li>
&lt;li>&lt;code>AUTHZ&lt;/code>: after the Istio authentication filters and before any first-class authorization filters, i.e., before &lt;code>AuthorizationPolicy&lt;/code> resources have been applied.&lt;/li>
&lt;li>&lt;code>STATS&lt;/code>: after all authorization filters and prior to the Istio stats filter.&lt;/li>
&lt;li>&lt;code>UNSPECIFIED_PHASE&lt;/code>: let the control plane decide where to insert. This will generally be at the end of the filter chain, right before the router. This is the default value for this &lt;code>phase&lt;/code> field.&lt;/li>
&lt;/ul>
&lt;p>The &lt;code>pluginConfig&lt;/code> field is used for configuring your Wasm plugin. Whatever you put into this field will be encoded in JSON and passed on to your filter, where you can access it in the configuration callback of the Proxy-Wasm SDKs. For example, you can retrieve the config with &lt;code>onConfigure&lt;/code> in the &lt;a href="https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/fd0be8405db25de0264bdb78fae3a82668c03782/proxy_wasm_api.h#L329-L331">C++ SDK&lt;/a>, &lt;code>on_configure&lt;/code> in the &lt;a href="https://github.com/proxy-wasm/proxy-wasm-rust-sdk/blob/v0.1.4/src/dispatcher.rs#L255">Rust SDK&lt;/a> or the &lt;code>OnPluginStart&lt;/code> call back in the &lt;a href="https://github.com/tetratelabs/proxy-wasm-go-sdk/blob/v0.15.0/proxywasm/types/context.go#L74">Go SDK&lt;/a>.&lt;/p>
&lt;p>The &lt;code>url&lt;/code> field specifies where to pull the Wasm module. You’ll notice that the &lt;code>url&lt;/code> in this example is a docker URI. Apart from loading Wasm modules via HTTP, HTTPS and the local file system (using file://), we are introducing the OCI image format as the preferred mechanism for distributing Wasm modules.&lt;/p>
&lt;p>One last thing to note is currently the Wasm Plugin API only applies to inbound HTTP filter chains.
Support for network filters and outbound traffic will be added in the future.&lt;/p>
&lt;h2 id="wasm-image-specification">Wasm image specification&lt;/h2>
&lt;p>We believe that containers are the ideal way to store, publish and manage proxy extensions, so we worked with Solo.io to extend their existing Proxy-Wasm container format with a variant that aims to be compatible with all registries and the CLI toolchain. Depending on your processes, you can now build your proxy extension containers using your existing container CLI tooling such as Docker CLI or &lt;a href="https://buildah.io/">buildah&lt;/a>.&lt;/p>
&lt;p>To learn how to build OCI images, please refer to &lt;a href="https://github.com/istio-ecosystem/wasm-extensions/blob/master/doc/how-to-build-oci-images.md">these instructions&lt;/a>.&lt;/p>
&lt;h2 id="image-fetcher-in-istio-agent">Image fetcher in Istio agent&lt;/h2>
&lt;p>Since Istio 1.9, Istio-agent has provided a reliable solution for loading Wasm binaries, fetched from remote HTTP sources configured in the &lt;code>EnvoyFilters&lt;/code>, by leveraging the xDS proxy inside istio-agent and Envoy’s Extension Configuration Discovery Service (ECDS). The same mechanism applies for the new Wasm API implementation in Istio 1.12. You can use HTTP remote resources reliably without concern that Envoy might get stuck with a bad configuration when a remote fetch fails.&lt;/p>
&lt;p>In addition, Istio 1.12 expands this capability to Wasm OCI images. This means the Istio-agent is now able to fetch Wasm images from any OCI registry including Docker Hub, Google Container Registry(GCR), Amazon Elastic Container Registry (Amazon ECR), etc. After fetching images, Istio-agent extracts and caches Wasm binaries from them, and then inserts them into the Envoy filter chains.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:67.90606653620353%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/wasm-api-alpha/istio-agent-architecture.svg" title="Remote Wasm module fetch flow">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/wasm-api-alpha/istio-agent-architecture.svg" alt="Remote Wasm module fetch flow" />
&lt;/a>
&lt;/div>
&lt;figcaption>Remote Wasm module fetch flow&lt;/figcaption>
&lt;/figure>
&lt;h2 id="improvements-in-envoy-wasm-runtime">Improvements in Envoy Wasm runtime&lt;/h2>
&lt;p>The Wasm runtime powered by V8 in Envoy has been shipped since Istio 1.5 and there have been a lot of improvements since then.&lt;/p>
&lt;h3 id="wasi-supports">WASI supports&lt;/h3>
&lt;p>First, some of the WASI (WebAssembly System Interface) system calls are now supported. For example, the &lt;code>clock_time_get&lt;/code> system call can be made from Wasm programs so you can use &lt;code>std::time::SystemTime::now()&lt;/code> in Rust or &lt;code>time.Now().UnixNano()&lt;/code> in Go in your Envoy Wasm extensions, just like any other native platform. Another example is &lt;code>random_get&lt;/code> is now supported by Envoy, so the &amp;ldquo;crypto/rand&amp;rdquo; package is available in Go as a cryptographically secure random number generator. We are also currently looking into file system support as we have seen requests for reading and writing local files from Wasm programs running in Envoy.&lt;/p>
&lt;h3 id="debuggability">Debuggability&lt;/h3>
&lt;p>Next is the improvement in debuggability. The Envoy runtime now emits the stack trace of your program when it causes runtime errors, for example, when null pointer exceptions occur in C++ or the panic function is called in Go or Rust. While Envoy error messages did not previously include anything about the cause, they now show the trace which you can use to debug your program:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >Function: proxy_on_request_headers failed: Uncaught RuntimeError: unreachable
Proxy-Wasm plugin in-VM backtrace:
0: 0xdbd - runtime._panic
1: 0x103ab - main.anotherCalculation
2: 0x10399 - main.someCalculation
3: 0xea57 - main.myHeaderHandler
4: 0xea15 - proxy_on_request_headers
&lt;/code>&lt;/pre>
&lt;p>The above is an example stack trace from a Go SDK based Wasm extension. You might notice that the output does not include file names and line numbers in the trace. This is an important future work item and open issue related to the DWARF format for WebAssembly and the Exception Handling proposal for the WebAssembly specification.&lt;/p>
&lt;h3 id="strace-support-for-wasm-programs">Strace support for Wasm programs&lt;/h3>
&lt;p>You can see &lt;code>strace&lt;/code> equivalent logs emitted by Envoy. With Istio proxy’s component log level &lt;code>wasm:trace&lt;/code>, you can observe all the system calls and Proxy-Wasm ABI calls that go across the boundary between Wasm virtual machines and Envoy. The following is an example of such an &lt;code>strace&lt;/code> log stream:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >[host-&amp;gt;vm] proxy_on_context_create(2, 1)
[host&amp;lt;-vm] proxy_on_context_create return: void
[host-&amp;gt;vm] proxy_on_request_headers(2, 8, 1)
[vm-&amp;gt;host] wasi_snapshot_preview1.random_get(86928, 32)
[vm&amp;lt;-host] wasi_snapshot_preview1.random_get return: 0
[vm-&amp;gt;host] env.proxy_log(2, 87776, 18)
&lt;/code>&lt;/pre>
&lt;p>This is especially useful to debug a Wasm program&amp;rsquo;s execution at runtime, for example, to verify it is not making any malicious system calls.&lt;/p>
&lt;h3 id="arbitrary-prometheus-namespace-for-in-wasm-metrics">Arbitrary Prometheus namespace for in-Wasm metrics&lt;/h3>
&lt;p>The last update is about metrics. Wasm extensions have been able to define their own custom metrics and expose them in Envoy, just like any other metric, but prior to Istio 1.12, all of these custom metrics were prefixed with the &lt;code>envoy_&lt;/code> Prometheus namespace and users were not able to use their own namespaces. Now, you can choose whatever namespace you want, and your metrics will be exposed in Envoy as-is, without being prefixed by &lt;code>envoy_&lt;/code>.&lt;/p>
&lt;p>Note that in order to actually expose these custom metrics, you have to configure &lt;a href="../../../docs/reference/config/istio.mesh.v1alpha1/#ProxyConfig-ProxyStatsMatcher">&lt;code>ProxyConfig.proxyStatsMatcher&lt;/code>&lt;/a> in &lt;code>meshConfig&lt;/code> for global configuration or in &lt;code>proxy.istio.io/config&lt;/code> for per proxy configuration. For detail, please refer to &lt;a href="../../../docs/ops/configuration/telemetry/envoy-stats/">&lt;code>Envoy Statistics&lt;/code>&lt;/a>.&lt;/p>
&lt;h2 id="future-work-and-looking-for-feedback">Future work and looking for feedback&lt;/h2>
&lt;p>Although we have announced the alpha availability of Wasm plugins, there is still a lot of work left to be done. One important work item is &amp;ldquo;Image pull secrets” support in the Wasm API which will allow you to easily consume OCI images in a private repository. Others include first-class support for L4 filters, signature verification of Wasm binaries, runtime improvements in Envoy, Proxy-Wasm SDK improvements, documentation, etc.&lt;/p>
&lt;p>This is just the beginning of our plan to provide 1st-class Wasm support in Istio. We would love to hear your feedback so that we can improve the developer experience using Wasm plugins, in future releases of Istio!&lt;/p></description><pubDate>Thu, 16 Dec 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/wasm-api-alpha/</link><author>Daniel Grimm (Red Hat), Pengyuan Bian (Google), Takeshi Yoneda (Tetrate)</author><guid isPermaLink="true">/v1.12/blog/2021/wasm-api-alpha/</guid><category>wasm</category><category>extensibility</category><category>WebAssembly</category></item><item><title>gRPC Proxyless Service Mesh</title><description>
&lt;p>Istio dynamically configures its Envoy sidecar proxies using a set of discovery APIs, collectively known as the
&lt;a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/operations/dynamic_configuration">xDS APIs&lt;/a>.
These APIs aim to become a &lt;a href="https://blog.envoyproxy.io/the-universal-data-plane-api-d15cec7a?gi=64aa2eea0283">universal data-plane API&lt;/a>.
The gRPC project has significant support for the xDS APIs, which means you can manage gRPC workloads
without having to deploy an Envoy sidecar along with them. You can learn more about the integration in a
&lt;a href="https://www.youtube.com/watch?v=cGJXkZ7jiDk">KubeCon EU 2021 talk from Megan Yahya&lt;/a>. The latest updates on gRPC&amp;rsquo;s
support can be found in their &lt;a href="https://github.com/grpc/proposal/search?q=xds">proposals&lt;/a> along with implementation
status.&lt;/p>
&lt;p>Istio 1.11 adds experimental support for adding gRPC services directly to the mesh. We support basic service
discovery, some VirtualService based traffic policy, and mutual TLS.&lt;/p>
&lt;h2 id="supported-features">Supported Features&lt;/h2>
&lt;p>The current implementation of the xDS APIs within gRPC is limited in some areas compared to Envoy. The following
features should work, although this is not an exhaustive list and other features may have partial functionality:&lt;/p>
&lt;ul>
&lt;li>Basic service discovery. Your gRPC service can reach other pods and virtual machines registered in the mesh.&lt;/li>
&lt;li>&lt;a href="/v1.12/docs/reference/config/networking/destination-rule/">&lt;code>DestinationRule&lt;/code>&lt;/a>:
&lt;ul>
&lt;li>Subsets: Your gRPC service can split traffic based on label selectors to different groups of instances.&lt;/li>
&lt;li>The only Istio &lt;code>loadBalancer&lt;/code> currently supported is &lt;code>ROUND_ROBIN&lt;/code>, &lt;code>consistentHash&lt;/code> will be added in
future versions of Istio (it is supported by gRPC).&lt;/li>
&lt;li>&lt;code>tls&lt;/code> settings are restricted to &lt;code>DISABLE&lt;/code> or &lt;code>ISTIO_MUTUAL&lt;/code>. Other modes will be treated as &lt;code>DISABLE&lt;/code>.&lt;/li>
&lt;/ul>&lt;/li>
&lt;li>&lt;a href="/v1.12/docs/reference/config/networking/virtual-service/">&lt;code>VirtualService&lt;/code>&lt;/a>:
&lt;ul>
&lt;li>Header match and URI match in the format &lt;code>/ServiceName/RPCName&lt;/code>.&lt;/li>
&lt;li>Override destination host and subset.&lt;/li>
&lt;li>Weighted traffic shifting.&lt;/li>
&lt;/ul>&lt;/li>
&lt;li>&lt;a href="/v1.12/docs/reference/config/security/peer_authentication/">&lt;code>PeerAuthentication&lt;/code>&lt;/a>:
&lt;ul>
&lt;li>Only &lt;code>DISABLE&lt;/code> and &lt;code>STRICT&lt;/code> are supported. Other modes will be treated as &lt;code>DISABLE&lt;/code>.&lt;/li>
&lt;li>Support for auto-mTLS may exist in a future release.&lt;/li>
&lt;/ul>&lt;/li>
&lt;/ul>
&lt;p>Other features including faults, retries, timeouts, mirroring and rewrite rules may be supported in a future release.
Some of these features are awaiting implementation in gRPC, and others require work in Istio to support. The status
of xDS features in gRPC can be found &lt;a href="https://github.com/grpc/grpc/blob/master/doc/grpc_xds_features.md">here&lt;/a>. The
status of Istio&amp;rsquo;s support will exist in future official docs.&lt;/p>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">This is feature is &lt;a href="/v1.12/docs/releases/feature-stages/">experimental&lt;/a>. Standard Istio features will become supported
over time along with improvements to the overall design.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;h2 id="architecture-overview">Architecture Overview&lt;/h2>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:44.32692307692307%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/proxyless-grpc/architecture.svg" title="Diagram of how gRPC services communicate with the istiod">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/proxyless-grpc/architecture.svg" alt="Diagram of how gRPC services communicate with the istiod" />
&lt;/a>
&lt;/div>
&lt;figcaption>Diagram of how gRPC services communicate with the istiod&lt;/figcaption>
&lt;/figure>
&lt;p>Although this doesn&amp;rsquo;t use a proxy for data plane communication, it still requires an agent for initialization and
communication with the control-plane. First, the agent generates a &lt;a href="https://github.com/grpc/proposal/blob/master/A27-xds-global-load-balancing.md#xdsclient-and-bootstrap-file">bootstrap file&lt;/a>
at startup the same way it would generate bootstrap for Envoy. This tells the &lt;code>gRPC&lt;/code> library how to connect to &lt;code>istiod&lt;/code>,
where it can find certificates for data plane communication, and what metadata to send to the control plane. Next, the
agent acts as an &lt;code>xDS&lt;/code> proxy, connecting and authenticating with &lt;code>istiod&lt;/code> on the application&amp;rsquo;s behalf. Finally, the
agent fetches and rotates certificates used in data plane traffic.&lt;/p>
&lt;h2 id="changes-to-application-code">Changes to application code&lt;/h2>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">This section covers gRPC’s XDS support in Go. Similar APIs exist for other languages.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>To enable the xDS features in gRPC, there are a handful of required changes your application must make. Your gRPC version should be at least &lt;code>1.39.0&lt;/code>.&lt;/p>
&lt;h3 id="in-the-client">In the client&lt;/h3>
&lt;p>The following side-effect import will register the xDS resolvers and balancers within gRPC. It should be added in your
&lt;code>main&lt;/code> package or in the same package calling &lt;code>grpc.Dial&lt;/code>.&lt;/p>
&lt;pre>&lt;code class='language-go' data-expandlinks='true' data-repo='istio' >import _ &amp;#34;google.golang.org/grpc/xds&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>When creating a gRPC connection the URL must use the &lt;code>xds:///&lt;/code> scheme.&lt;/p>
&lt;pre>&lt;code class='language-go' data-expandlinks='true' data-repo='istio' >conn, err := grpc.DialContext(ctx, &amp;#34;xds:///foo.ns.svc.cluster.local:7070&amp;#34;)
&lt;/code>&lt;/pre>
&lt;p>Additionally, for (m)TLS support, a special &lt;code>TransportCredentials&lt;/code> option has to be passed to &lt;code>DialContext&lt;/code>.
The &lt;code>FallbackCreds&lt;/code> allow us to succeed when istiod doesn’t send security config.&lt;/p>
&lt;pre>&lt;code class='language-go' data-expandlinks='true' data-repo='istio' >import &amp;#34;google.golang.org/grpc/credentials/xds&amp;#34;
...
creds, err := xds.NewClientCredentials(xds.ClientOptions{
FallbackCreds: insecure.NewCredentials()
})
// handle err
conn, err := grpc.DialContext(
ctx,
&amp;#34;xds:///foo.ns.svc.cluster.local:7070&amp;#34;,
grpc.WithTransportCredentials(creds),
)
&lt;/code>&lt;/pre>
&lt;h3 id="on-the-server">On the server&lt;/h3>
&lt;p>To support server-side configurations, such as mTLS, there are a couple of modifications that must be made.&lt;/p>
&lt;p>First, we use a special constructor to create the &lt;code>GRPCServer&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-go' data-expandlinks='true' data-repo='istio' >import &amp;#34;google.golang.org/grpc/xds&amp;#34;
...
server = xds.NewGRPCServer()
RegisterFooServer(server, &amp;amp;fooServerImpl)
&lt;/code>&lt;/pre>
&lt;p>If your &lt;code>protoc&lt;/code> generated Go code is out of date, you may need to regenerate it to be compatible with the xDS server.
Your generated &lt;code>RegisterFooServer&lt;/code> function should look like the following:&lt;/p>
&lt;pre>&lt;code class='language-go' data-expandlinks='true' data-repo='istio' >func RegisterFooServer(s grpc.ServiceRegistrar, srv FooServer) {
s.RegisterService(&amp;amp;FooServer_ServiceDesc, srv)
}
&lt;/code>&lt;/pre>
&lt;p>Finally, as with the client-side changes, we must enable security support:&lt;/p>
&lt;pre>&lt;code class='language-go' data-expandlinks='true' data-repo='istio' >creds, err := xds.NewServerCredentials(xdscreds.ServerOptions{FallbackCreds: insecure.NewCredentials()})
// handle err
server = xds.NewGRPCServer(grpc.Creds(creds))
&lt;/code>&lt;/pre>
&lt;h3 id="in-your-kubernetes-deployment">In your Kubernetes Deployment&lt;/h3>
&lt;p>Assuming your application code is compatible, the Pod simply needs the annotation &lt;code>inject.istio.io/templates: grpc-agent&lt;/code>.
This adds a sidecar container running the agent described above, and some environment variables that gRPC uses to find
the bootstrap file and enable certain features.&lt;/p>
&lt;p>For gRPC servers, your Pod should also be annotated with &lt;code>proxy.istio.io/config: '{&amp;quot;holdApplicationUntilProxyStarts&amp;quot;: true}'&lt;/code>
to make sure the in-agent xDS proxy and bootstrap file are ready before your gRPC server is initialized.&lt;/p>
&lt;h2 id="example">Example&lt;/h2>
&lt;p>In this guide you will deploy &lt;code>echo&lt;/code>, an application that already supports both server-side and client-side
proxyless gRPC. With this app you can try out some supported traffic policies enabling mTLS.&lt;/p>
&lt;h3 id="prerequisites">Prerequisites&lt;/h3>
&lt;p>This guide requires the Istio (1.11+) control plane &lt;a href="/v1.12/docs/setup/install/">to be installed&lt;/a> before proceeding.&lt;/p>
&lt;h3 id="deploy-the-application">Deploy the application&lt;/h3>
&lt;p>Create an injection-enabled namespace &lt;code>echo-grpc&lt;/code>. Next deploy two instances of the &lt;code>echo&lt;/code> app as well as the Service.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create namespace echo-grpc
$ kubectl label namespace echo-grpc istio-injection=enabled
$ kubectl -n echo-grpc apply -f samples/grpc-echo/grpc-echo.yaml
&lt;/code>&lt;/pre>
&lt;p>Make sure the two pods are running:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n echo-grpc get pods
NAME READY STATUS RESTARTS AGE
echo-v1-69d6d96cb7-gpcpd 2/2 Running 0 58s
echo-v2-5c6cbf6dc7-dfhcb 2/2 Running 0 58s
&lt;/code>&lt;/pre>
&lt;h3 id="test-the-grpc-resolver">Test the gRPC resolver&lt;/h3>
&lt;p>First, port-forward &lt;code>17171&lt;/code> to one of the Pods. This port is a non-xDS backed gRPC server that allows making
requests from the port-forwarded Pod.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n echo-grpc port-forward $(kubectl -n echo-grpc get pods -l version=v1 -ojsonpath=&amp;#39;{.items[0].metadata.name}&amp;#39;) 17171 &amp;amp;
&lt;/code>&lt;/pre>
&lt;p>Next, we can fire off a batch of 5 requests:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ grpcurl -plaintext -d &amp;#39;{&amp;#34;url&amp;#34;: &amp;#34;xds:///echo.echo-grpc.svc.cluster.local:7070&amp;#34;, &amp;#34;count&amp;#34;: 5}&amp;#39; :17171 proto.EchoTestService/ForwardEcho | jq -r &amp;#39;.output | join(&amp;#34;&amp;#34;)&amp;#39; | grep Hostname
Handling connection for 17171
[0 body] Hostname=echo-v1-7cf5b76586-bgn6t
[1 body] Hostname=echo-v2-cf97bd94d-qf628
[2 body] Hostname=echo-v1-7cf5b76586-bgn6t
[3 body] Hostname=echo-v2-cf97bd94d-qf628
[4 body] Hostname=echo-v1-7cf5b76586-bgn6t
&lt;/code>&lt;/pre>
&lt;p>You can also use Kubernetes-like name resolution for short names:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ grpcurl -plaintext -d &amp;#39;{&amp;#34;url&amp;#34;: &amp;#34;xds:///echo:7070&amp;#34;}&amp;#39; :17171 proto.EchoTestService/ForwardEcho | jq -r &amp;#39;.output | join
(&amp;#34;&amp;#34;)&amp;#39; | grep Hostname
[0 body] Hostname=echo-v1-7cf5b76586-ltr8q
$ grpcurl -plaintext -d &amp;#39;{&amp;#34;url&amp;#34;: &amp;#34;xds:///echo.echo-grpc:7070&amp;#34;}&amp;#39; :17171 proto.EchoTestService/ForwardEcho | jq -r
&amp;#39;.output | join(&amp;#34;&amp;#34;)&amp;#39; | grep Hostname
[0 body] Hostname=echo-v1-7cf5b76586-ltr8q
$ grpcurl -plaintext -d &amp;#39;{&amp;#34;url&amp;#34;: &amp;#34;xds:///echo.echo-grpc.svc:7070&amp;#34;}&amp;#39; :17171 proto.EchoTestService/ForwardEcho | jq -r
&amp;#39;.output | join(&amp;#34;&amp;#34;)&amp;#39; | grep Hostname
[0 body] Hostname=echo-v2-cf97bd94d-jt5mf
&lt;/code>&lt;/pre>
&lt;h3 id="creating-subsets-with-destination-rule">Creating subsets with destination rule&lt;/h3>
&lt;p>First, create a subset for each version of the workload.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: echo-versions
namespace: echo-grpc
spec:
host: echo.echo-grpc.svc.cluster.local
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
EOF
&lt;/code>&lt;/pre>
&lt;h3 id="traffic-shifting">Traffic shifting&lt;/h3>
&lt;p>Using the subsets defined above, you can send 80 percent of the traffic to a specific version:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: echo-weights
namespace: echo-grpc
spec:
hosts:
- echo.echo-grpc.svc.cluster.local
http:
- route:
- destination:
host: echo.echo-grpc.svc.cluster.local
subset: v1
weight: 20
- destination:
host: echo.echo-grpc.svc.cluster.local
subset: v2
weight: 80
EOF
&lt;/code>&lt;/pre>
&lt;p>Now, send a set of 10 requests:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ grpcurl -plaintext -d &amp;#39;{&amp;#34;url&amp;#34;: &amp;#34;xds:///echo.echo-grpc.svc.cluster.local:7070&amp;#34;, &amp;#34;count&amp;#34;: 10}&amp;#39; :17171 proto.EchoTestService/ForwardEcho | jq -r &amp;#39;.output | join(&amp;#34;&amp;#34;)&amp;#39; | grep ServiceVersion
&lt;/code>&lt;/pre>
&lt;p>The response should contain mostly &lt;code>v2&lt;/code> responses:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >[0 body] ServiceVersion=v2
[1 body] ServiceVersion=v2
[2 body] ServiceVersion=v1
[3 body] ServiceVersion=v2
[4 body] ServiceVersion=v1
[5 body] ServiceVersion=v2
[6 body] ServiceVersion=v2
[7 body] ServiceVersion=v2
[8 body] ServiceVersion=v2
[9 body] ServiceVersion=v2
&lt;/code>&lt;/pre>
&lt;h3 id="enabling-mtls">Enabling mTLS&lt;/h3>
&lt;p>Due to the changes to the application itself required to enable security in gRPC, Istio&amp;rsquo;s traditional method of
automatically detecting mTLS support is unreliable. For this reason, the initial release requires explicitly enabling
mTLS on both the client and server.&lt;/p>
&lt;p>To enable client-side mTLS, apply a &lt;code>DestinationRule&lt;/code> with &lt;code>tls&lt;/code> settings:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: echo-mtls
namespace: echo-grpc
spec:
host: echo.echo-grpc.svc.cluster.local
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
EOF
&lt;/code>&lt;/pre>
&lt;p>Now an attempt to call the server that is not yet configured for mTLS will fail.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ grpcurl -plaintext -d &amp;#39;{&amp;#34;url&amp;#34;: &amp;#34;xds:///echo.echo-grpc.svc.cluster.local:7070&amp;#34;}&amp;#39; :17171 proto.EchoTestService/ForwardEcho | jq -r &amp;#39;.output | join(&amp;#34;&amp;#34;)&amp;#39;
Handling connection for 17171
ERROR:
Code: Unknown
Message: 1/1 requests had errors; first error: rpc error: code = Unavailable desc = all SubConns are in TransientFailure
&lt;/code>&lt;/pre>
&lt;p>To enable server-side mTLS, apply a &lt;code>PeerAuthentication&lt;/code>.&lt;/p>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">The following policy forces STRICT mTLS for the entire namespace.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: echo-mtls
namespace: echo-grpc
spec:
mtls:
mode: STRICT
EOF
&lt;/code>&lt;/pre>
&lt;p>Requests will start to succeed after applying the policy.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ grpcurl -plaintext -d &amp;#39;{&amp;#34;url&amp;#34;: &amp;#34;xds:///echo.echo-grpc.svc.cluster.local:7070&amp;#34;}&amp;#39; :17171 proto.EchoTestService/ForwardEcho | jq -r &amp;#39;.output | join(&amp;#34;&amp;#34;)&amp;#39;
Handling connection for 17171
[0] grpcecho.Echo(&amp;amp;{xds:///echo.echo-grpc.svc.cluster.local:7070 map[] 0 5s false })
[0 body] x-request-id=0
[0 body] Host=echo.echo-grpc.svc.cluster.local:7070
[0 body] content-type=application/grpc
[0 body] user-agent=grpc-go/1.39.1
[0 body] StatusCode=200
[0 body] ServiceVersion=v1
[0 body] ServicePort=17070
[0 body] Cluster=
[0 body] IP=10.68.1.18
[0 body] IstioVersion=
[0 body] Echo=
[0 body] Hostname=echo-v1-7cf5b76586-z5p8l
&lt;/code>&lt;/pre>
&lt;h2 id="limitations">Limitations&lt;/h2>
&lt;p>The initial release comes with several limitations that may be fixed in a future version:&lt;/p>
&lt;ul>
&lt;li>Auto-mTLS isn&amp;rsquo;t supported, and permissive mode isn&amp;rsquo;t supported. Instead we require explicit mTLS configuration with
&lt;code>STRICT&lt;/code> on the server and &lt;code>ISTIO_MUTUAL&lt;/code> on the client. Envoy can be used during the migration to &lt;code>STRICT&lt;/code>.&lt;/li>
&lt;li>&lt;code>grpc.Serve(listener)&lt;/code> or &lt;code>grpc.Dial(&amp;quot;xds:///...&amp;quot;)&lt;/code> called before the bootstrap is written or xDS proxy is ready can
cause a failure. &lt;code>holdApplicationUntilProxyStarts&lt;/code> can be used to work around this, or the application can be more
robust to these failures.&lt;/li>
&lt;li>If the xDS-enabled gRPC server uses mTLS then you will need to make sure your health checks can work around this.
Either a separate port should be used, or your health-checking client needs a way to get the proper client
certificates.&lt;/li>
&lt;li>The implementation of xDS in gRPC does not match Envoys. Certain behaviors may be different, and some features may
be missing. The &lt;a href="https://github.com/grpc/grpc/blob/master/doc/grpc_xds_features.md">feature status for gRPC&lt;/a> provides more detail. Make sure to test that any Istio
configuration actually applies on your proxyless gRPC apps.&lt;/li>
&lt;/ul>
&lt;h2 id="performance">Performance&lt;/h2>
&lt;h3 id="experiment-setup">Experiment Setup&lt;/h3>
&lt;ul>
&lt;li>Using Fortio, a Go-based load testing app
&lt;ul>
&lt;li>Slightly modified, to support gRPC’s XDS features (PR)&lt;/li>
&lt;/ul>&lt;/li>
&lt;li>Resources:
&lt;ul>
&lt;li>GKE 1.20 cluster with 3 &lt;code>e2-standard-16&lt;/code> nodes (16 CPUs + 64 GB memory each)&lt;/li>
&lt;li>Fortio client and server apps: 1.5 vCPU, 1000 MiB memory&lt;/li>
&lt;li>Sidecar (istio-agent and possibly Envoy proxy): 1 vCPU, 512 MiB memory&lt;/li>
&lt;/ul>&lt;/li>
&lt;li>Workload types tested:
&lt;ul>
&lt;li>Baseline: regular gRPC with no Envoy proxy or Proxyless xDS in use&lt;/li>
&lt;li>Envoy: standard istio-agent + Envoy proxy sidecar&lt;/li>
&lt;li>Proxyless: gRPC using the xDS gRPC server implementation and &lt;code>xds:///&lt;/code> resolver on the client&lt;/li>
&lt;li>mTLS enabled/disabled via &lt;code>PeerAuthentication&lt;/code> and &lt;code>DestinationRule&lt;/code>&lt;/li>
&lt;/ul>&lt;/li>
&lt;/ul>
&lt;h3 id="latency">Latency&lt;/h3>
&lt;p>&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/proxyless-grpc/latencies_p50.svg" title="p50 latency comparison chart">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/proxyless-grpc/latencies_p50.svg" alt="p50 latency comparison chart" />
&lt;/a>
&lt;/div>
&lt;figcaption>p50 latency comparison chart&lt;/figcaption>
&lt;/figure>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/proxyless-grpc/latencies_p99.svg" title="p99 latency comparison chart">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/proxyless-grpc/latencies_p99.svg" alt="p99 latency comparison chart" />
&lt;/a>
&lt;/div>
&lt;figcaption>p99 latency comparison chart&lt;/figcaption>
&lt;/figure>&lt;/p>
&lt;p>There is a marginal increase in latency when using the proxyless gRPC resolvers. Compared to Envoy this is a massive
improvement that still allows for advanced traffic management features and mTLS.&lt;/p>
&lt;h3 id="istio-proxy-container-resource-usage">istio-proxy container resource usage&lt;/h3>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>&lt;/th>
&lt;th>Client &lt;code>mCPU&lt;/code>&lt;/th>
&lt;th>Client Memory (&lt;code>MiB&lt;/code>)&lt;/th>
&lt;th>Server &lt;code>mCPU&lt;/code>&lt;/th>
&lt;th>Server Memory (&lt;code>MiB&lt;/code>)&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Envoy Plaintext&lt;/td>
&lt;td>320.44&lt;/td>
&lt;td>66.93&lt;/td>
&lt;td>243.78&lt;/td>
&lt;td>64.91&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Envoy mTLS&lt;/td>
&lt;td>340.87&lt;/td>
&lt;td>66.76&lt;/td>
&lt;td>309.82&lt;/td>
&lt;td>64.82&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Proxyless Plaintext&lt;/td>
&lt;td>0.72&lt;/td>
&lt;td>23.54&lt;/td>
&lt;td>0.84&lt;/td>
&lt;td>24.31&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Proxyless mTLS&lt;/td>
&lt;td>0.73&lt;/td>
&lt;td>25.05&lt;/td>
&lt;td>0.78&lt;/td>
&lt;td>25.43&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>Even though we still require an agent, the agent uses less than 0.1% of a full vCPU, and only 25 MiB of memory,
which is less than half of what running Envoy requires.&lt;/p>
&lt;p>These metrics don’t include additional resource usage by gRPC in the application container,
but serve to demonstrate the resource usage impact of the istio-agent when running in this mode.&lt;/p></description><pubDate>Thu, 28 Oct 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/proxyless-grpc/</link><author>Steven Landow (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/proxyless-grpc/</guid></item><item><title>Aeraki — Manage Any Layer-7 Protocol in Istio Service Mesh</title><description>
&lt;p>Aeraki [Air-rah-ki] is the Greek word for &amp;lsquo;breeze&amp;rsquo;. While Istio connects microservices in a service mesh, Aeraki provides a framework to allow Istio to support more layer-7 protocols other than just HTTP and gRPC. We hope this breeze can help Istio sail a little further.&lt;/p>
&lt;h2 id="lack-of-protocols-support-in-service-mesh">Lack of Protocols Support in Service Mesh&lt;/h2>
&lt;p>We are now facing some challenges with service meshes:&lt;/p>
&lt;ul>
&lt;li>Istio and other popular service mesh implementations have very limited support for layer 7 protocols other than HTTP and gRPC.&lt;/li>
&lt;li>Envoy RDS(Route Discovery Service) is solely designed for HTTP. Other protocols such as Dubbo and Thrift can only use listener in-line routes for traffic management, which breaks existing connections when routes change.&lt;/li>
&lt;li>It takes a lot of effort to introduce a proprietary protocol into a service mesh. You’ll need to write an Envoy filter to handle the traffic in the data plane, and a control plane to manage those Envoys.&lt;/li>
&lt;/ul>
&lt;p>Those obstacles make it very hard, if not impossible, for users to manage the traffic of other widely-used layer-7 protocols in microservices. For example, in a microservices application, we may have the below protocols:&lt;/p>
&lt;ul>
&lt;li>RPC: HTTP, gRPC, Thrift, Dubbo, Proprietary RPC Protocol …&lt;/li>
&lt;li>Messaging: Kafka, RabbitMQ …&lt;/li>
&lt;li>Cache: Redis, Memcached …&lt;/li>
&lt;li>Database: MySQL, PostgreSQL, MongoDB …&lt;/li>
&lt;/ul>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:44.800000000000004%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/aeraki/protocols.png" title="Common Layer-7 Protocols Used in Microservices">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/aeraki/protocols.png" alt="Common Layer-7 Protocols Used in Microservices" />
&lt;/a>
&lt;/div>
&lt;figcaption>Common Layer-7 Protocols Used in Microservices&lt;/figcaption>
&lt;/figure>
&lt;p>If you have already invested a lot of effort in migrating to a service mesh, of course, you want to get the most out of it — managing the traffic of all the protocols in your microservices.&lt;/p>
&lt;h2 id="aeraki-s-approach">Aeraki&amp;rsquo;s approach&lt;/h2>
&lt;p>To address these problems, we create an open-source project, &lt;a href="https://github.com/aeraki-framework">Aeraki&lt;/a>, to provide a non-intrusive, extendable way to manage any layer-7 traffic in an Istio service mesh.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:51.35135135135135%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/aeraki/aeraki-architecture.png" title="Aeraki Architecture">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/aeraki/aeraki-architecture.png" alt="Aeraki Architecture" />
&lt;/a>
&lt;/div>
&lt;figcaption>Aeraki Architecture&lt;/figcaption>
&lt;/figure>
&lt;p>As this diagram shows, Aeraki Framework consists of the following components:&lt;/p>
&lt;ul>
&lt;li>Aeraki: &lt;a href="https://github.com/aeraki-framework/aeraki">Aeraki&lt;/a> provides high-level, user-friendly traffic management rules to operations, translates the rules to envoy filter configurations, and leverages Istio’s &lt;code>EnvoyFilter&lt;/code> API to push the configurations to the sidecar proxies. Aeraki also serves as the RDS server for MetaProtocol proxies in the data plane. Contrary to Envoy RDS, which focuses on HTTP, Aeraki RDS is aimed to provide a general dynamic route capability for all layer-7 protocols.&lt;/li>
&lt;li>MetaProtocol Proxy: &lt;a href="https://github.com/aeraki-framework/meta-protocol-proxy">MetaProtocol Proxy&lt;/a> provides common capabilities for Layer-7 protocols, such as load balancing, circuit breaker, load balancing, routing, rate limiting, fault injection, and auth. Layer-7 protocols can be built on top of MetaProtocol. To add a new protocol into the service mesh, the only thing you need to do is implementing the &lt;a href="https://github.com/aeraki-framework/meta-protocol-proxy/blob/ac788327239bd794e745ce18b382da858ddf3355/src/meta_protocol_proxy/codec/codec.h#L118">codec interface&lt;/a> and a couple of lines of configuration. If you have special requirements which can’t be accommodated by the built-in capabilities, MetaProtocol Proxy also has an application-level filter chain mechanism, allowing users to write their own layer-7 filters to add custom logic into MetaProtocol Proxy.&lt;/li>
&lt;/ul>
&lt;p>&lt;a href="https://github.com/aeraki-framework/meta-protocol-proxy/tree/master/src/application_protocols/dubbo">Dubbo&lt;/a> and &lt;a href="https://github.com/aeraki-framework/meta-protocol-proxy/tree/master/src/application_protocols/thrift">Thrift&lt;/a> have already been implemented based on MetaProtocol. More protocols are on the way. If you&amp;rsquo;re using a close-source, proprietary protocol, you can also manage it in your service mesh simply by writing a MetaProtocol codec for it.&lt;/p>
&lt;p>Most request/response style, stateless protocols can be built on top of the MetaProtocol Proxy. However, some protocols&amp;rsquo; routing policies are too &amp;ldquo;special&amp;rdquo; to be normalized in MetaProtocol. For example, Redis proxy uses a slot number to map a client query to a specific Redis server node, and the slot number is computed by the key in the request. Aeraki can still manage those protocols as long as there&amp;rsquo;s an available Envoy Filter in the Envoy proxy side. Currently, for protocols in this category, &lt;a href="https://github.com/aeraki-framework/aeraki/blob/master/docs/zh/redis.md">Redis&lt;/a> and Kafka are supported in Aeraki.&lt;/p>
&lt;h2 id="deep-dive-into-metaprotocol">Deep Dive Into MetaProtocol&lt;/h2>
&lt;p>Let’s look into how MetaProtocol works. Before MetaProtocol is introduced, if we want to proxy traffic for a specific protocol, we need to write an Envoy filter that understands that protocol and add the code to manipulate the traffic, including routing, header modification, fault injection, traffic mirroring, etc.&lt;/p>
&lt;p>For most request/response style protocols, the code for traffic manipulation is very similar. Therefore, to avoid duplicating these functionalities in different Envoy filters, Aeraki Framework implements most of the common functions of a layer-7 protocol proxy in a single place — the MetaProtocol Proxy filter.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:29.301948051948052%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/aeraki/metaprotocol-proxy.png" title="MetaProtocol Proxy">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/aeraki/metaprotocol-proxy.png" alt="MetaProtocol Proxy" />
&lt;/a>
&lt;/div>
&lt;figcaption>MetaProtocol Proxy&lt;/figcaption>
&lt;/figure>
&lt;p>This approach significantly lowers the barrier to write a new Envoy filter: instead of writing a fully functional filter, now you only need to implement the codec interface. In addition to that, the control plane is already in place — Aeraki works at the control plane to provides MetaProtocol configuration and dynamic routes for all protocols built on top of MetaProtocol.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:44.68571428571428%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/aeraki/metaprotocol-proxy-codec.png" title="Writing an Envoy Filter Before and After MetProtocol">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/aeraki/metaprotocol-proxy-codec.png" alt="Writing an Envoy Filter Before and After MetProtocol" />
&lt;/a>
&lt;/div>
&lt;figcaption>Writing an Envoy Filter Before and After MetProtocol&lt;/figcaption>
&lt;/figure>
&lt;p>There are two important data structures in MetaProtocol Proxy: Metadata and Mutation. Metadata is used for routing, and Mutation is used for header manipulation.&lt;/p>
&lt;p>At the request path, the decoder(the decode method of the codec implementation) populates the Metadata data structure with key-value pairs parsed from the request, then the Metadata will be passed to the MetaProtocol Router. The Router selects an appropriate upstream cluster after matching the route configuration it receives from Aeraki via RDS and the Metadata.&lt;/p>
&lt;p>A custom filter can populate the Mutation data structure with arbitrary key-value pairs if the request needs to be modified: adding a header or changing the value of a header. Then the Mutation data structure will be passed to the encoder(the encode method of the codec implementation). The encoder is responsible for writing the key-value pairs into the wire protocol.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:35.98183881952327%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/aeraki/request-path.png" title="The Request Path">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/aeraki/request-path.png" alt="The Request Path" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Request Path&lt;/figcaption>
&lt;/figure>
&lt;p>The response path is similar to the request path, only in a different direction.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:34.39273552780931%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/aeraki/response-path.png" title="The Response Path">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/aeraki/response-path.png" alt="The Response Path" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Response Path&lt;/figcaption>
&lt;/figure>
&lt;h2 id="an-example">An Example&lt;/h2>
&lt;p>If you need to implement an application protocol based on MetaProtocol, you can follow the below steps(use Thrift as an example):&lt;/p>
&lt;h3 id="data-plane">Data Plane&lt;/h3>
&lt;ul>
&lt;li>&lt;p>Implement the &lt;a href="https://github.com/aeraki-framework/meta-protocol-proxy/blob/ac788327239bd794e745ce18b382da858ddf3355/src/meta_protocol_proxy/codec/codec.h#L118">codec interface&lt;/a> to encode and decode the protocol package. You can refer to &lt;a href="https://github.com/aeraki-framework/meta-protocol-proxy/tree/master/src/application_protocols/dubbo">Dubbo codec&lt;/a> and &lt;a href="https://github.com/aeraki-framework/meta-protocol-proxy/tree/master/src/application_protocols/thrift">Thrift codec&lt;/a> as writing your own implementation.&lt;/p>&lt;/li>
&lt;li>&lt;p>Define the protocol with Aeraki &lt;code>ApplicationProtocol&lt;/code> CRD, as this YAML snippet shows:&lt;/p>&lt;/li>
&lt;/ul>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: metaprotocol.aeraki.io/v1alpha1
kind: ApplicationProtocol
metadata:
name: thrift
namespace: istio-system
spec:
protocol: thrift
codec: aeraki.meta_protocol.codec.thrift
&lt;/code>&lt;/pre>
&lt;h3 id="control-plane">Control Plane&lt;/h3>
&lt;p>You don’t need to implement the control plane. Aeraki watches services and traffic rules, generates the configurations for the sidecar proxies, and sends the configurations to the data plane via &lt;code>EnvoyFilter&lt;/code> and MetaProtocol RDS.&lt;/p>
&lt;h3 id="protocol-selection">Protocol selection&lt;/h3>
&lt;p>Similar to Istio, protocols are identified by service port prefix. Please name service ports with this pattern: tcp-metaprotocol-{application protocol}-xxx. For example, a Thrift service port should be named tcp-metaprotocol-thrift.&lt;/p>
&lt;h3 id="traffic-management">Traffic management&lt;/h3>
&lt;p>You can change the route via &lt;code>MetaRouter&lt;/code> CRD. For example: send 20% of the requests to v1 and 80% to v2:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: metaprotocol.aeraki.io/v1alpha1
kind: MetaRouter
metadata:
name: test-metaprotocol-route
spec:
hosts:
- thrift-sample-server.thrift.svc.cluster.local
routes:
- name: traffic-spilt
route:
- destination:
host: thrift-sample-server.thrift.svc.cluster.local
subset: v1
weight: 20
- destination:
host: thrift-sample-server.thrift.svc.cluster.local
subset: v2
weight: 80
&lt;/code>&lt;/pre>
&lt;p>Hope this helps if you need to manage protocols other than HTTP in a service mesh. Reach out to &lt;a href="https://zhaohuabing.com/">zhaohuabing&lt;/a> if you have any questions.&lt;/p>
&lt;h2 id="reference">Reference&lt;/h2>
&lt;ul>
&lt;li>&lt;a href="https://github.com/aeraki-framework">Aeraki GitHub&lt;/a>&lt;/li>
&lt;li>&lt;a href="http://aeraki.zhaohuabing.com:20001/">Live Demo: Kiali Dashboard&lt;/a>&lt;/li>
&lt;li>&lt;a href="http://aeraki.zhaohuabing.com:3000/d/pgz7wp-Gz/aeraki-demo?orgId=1&amp;amp;refresh=10s&amp;amp;kiosk">Live Demo: Service Metrics: Grafana&lt;/a>&lt;/li>
&lt;li>&lt;a href="http://aeraki.zhaohuabing.com:9090/new/graph?g0.expr=envoy_dubbo_inbound_20880___response_success&amp;amp;g0.tab=0&amp;amp;g0.stacked=1&amp;amp;g0.range_input=1h&amp;amp;g1.expr=envoy_dubbo_outbound_20880__org_apache_dubbo_samples_basic_api_demoservice_request&amp;amp;g1.tab=0&amp;amp;g1.stacked=1&amp;amp;g1.range_input=1h&amp;amp;g2.expr=envoy_thrift_inbound_9090___response&amp;amp;g2.tab=0&amp;amp;g2.stacked=1&amp;amp;g2.range_input=1h&amp;amp;g3.expr=envoy_thrift_outbound_9090__thrift_sample_server_thrift_svc_cluster_local_response_success&amp;amp;g3.tab=0&amp;amp;g3.stacked=1&amp;amp;g3.range_input=1h&amp;amp;g4.expr=envoy_thrift_outbound_9090__thrift_sample_server_thrift_svc_cluster_local_request&amp;amp;g4.tab=0&amp;amp;g4.stacked=1&amp;amp;g4.range_input=1h">Live Demo: Service Metrics: Prometheus&lt;/a>&lt;/li>
&lt;li>Istio meetup China(Chinese): &lt;a href="https://www.youtube.com/watch?v=Bq5T3OR3iTM">Full Stack Service Mesh - Manage Any Layer-7 Traffic in an Istio Service Mesh with Aeraki&lt;/a>&lt;/li>
&lt;li>IstioCon 2021: &lt;a href="https://www.youtube.com/watch?v=sBS4utF68d8">How to Manage Any Layer-7 Traffic in an Istio Service Mesh?&lt;/a>&lt;/li>
&lt;/ul></description><pubDate>Tue, 28 Sep 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/aeraki/</link><author>Huabing Zhao (Tencent)</author><guid isPermaLink="true">/v1.12/blog/2021/aeraki/</guid><category>istio</category><category>envoy</category><category>aeraki</category></item><item><title>Announcing Extended Support for Istio 1.9</title><description>&lt;p>In keeping with our 2021 theme of improving Day 2 Istio operations, the Istio team has been evaluating extending the support window for our releases to give users more time to upgrade. For starters, we are extending the support window of Istio 1.9 by six weeks, to October 5, 2021. We hope that this additional support window will allow the many users who are currently using Istio 1.9 to upgrade, either to Istio 1.10 or directly to Istio 1.11. By overlapping support between 1.9 and 1.11, we intend to create a stable cadence of upgrade windows twice a year for users upgrading directly across two minor versions (i.e. 1.9 to 1.11). Users who prefer upgrading through each minor release to get all the latest and greatest features may continue doing so quarterly.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:55.55555555555556%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/extended-support/extended_support.png" title="Extended Support and Upgrades">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/extended-support/extended_support.png" alt="Extended Support and Upgrades" />
&lt;/a>
&lt;/div>
&lt;figcaption>Extended Support and Upgrades&lt;/figcaption>
&lt;/figure>
&lt;p>During this extended period of support, Istio 1.9 will receive CVE and critical bug fixes only, as our goal is simply to provide users with time to migrate off the release and on to 1.10 or 1.11. And speaking of users, we would love to hear how we’re doing at improving your Day 2 experience of Istio. Is two upgrades per year not the right number? Is a six week upgrade window too short? Please share your thoughts with us on &lt;a href="https://slack.istio.io">slack&lt;/a> (in the user-experience channel), or on &lt;a href="https://twitter.com/istiomesh">twitter&lt;/a>. Thanks!&lt;/p></description><pubDate>Fri, 03 Sep 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/extended-support/</link><author>Mitch Connors (Google), Lin Sun (Solo.io)</author><guid isPermaLink="true">/v1.12/blog/2021/extended-support/</guid><category>upgrade</category><category>Istio</category><category>support</category></item><item><title>Announcing the results of Istio’s first security assessment</title><description>
&lt;p>The Istio service mesh has gained wide production adoption across a wide variety of
industries. The success of the project, and its critical usage for enforcing key
security policies in infrastructure warranted an open and neutral assessment of
the security risks associated with the project.&lt;/p>
&lt;p>To achieve this goal, the Istio community contracted the
&lt;a href="https://www.nccgroup.com/">NCC Group&lt;/a> last year to
conduct a third-party security assessment of the project. The goal of the review
was &amp;ldquo;to identify security issues related to the Istio code base, highlight
high-risk configurations commonly used by administrators, and provide
perspective on whether security features sufficiently address the concerns they
are designed to provide&amp;rdquo;.&lt;/p>
&lt;p>NCC Group carried out the review over a period of five weeks with collaboration
from subject matter experts across the Istio community. In this blog, we will
examine the key findings of the report, actions taken to implement various fixes
and recommendations, and our plan of action for continuous security evaluation
and improvement of the Istio project. You can download and read the
unabridged version of the
&lt;a href="./NCC_Group_Google_GOIST2005_Report_2020-08-06_v1.1.pdf">security assessment report&lt;/a>.&lt;/p>
&lt;h2 id="scope-and-key-findings">Scope and Key Findings&lt;/h2>
&lt;p>The assessment evaluated Istio’s architecture as a whole for security related
issues with focus on key components like istiod (Pilot), Ingress/Egress
gateways, and Istio’s overall Envoy usage as its data plane proxy. Additionally,
Istio documentation, including security guides, were audited for correctness and
clarity. The report was compiled against Istio version 1.6.5, and since then the
Product Security Working Group has issued several security releases as new
vulnerabilities were disclosed, along with fixes to address concerns raised in
the new report.&lt;/p>
&lt;p>An important conclusion from the report is that the auditors found no &amp;ldquo;Critical&amp;rdquo;
issues within the Istio project. This finding validates the continuous and
proactive security review and vulnerability management process implemented by
Istio’s Product Security Working Group (PSWG). For the remaining issues surfaced
by the report, the PSWG went to work on addressing them, and we are glad to
report that all issues marked &amp;ldquo;High&amp;rdquo;, and several marked &amp;ldquo;Medium/Low&amp;rdquo;, have been
resolved in the releases following the report.&lt;/p>
&lt;p>The report also makes strategic recommendations around creating a hardening
guide which is now available in our
&lt;a href="/v1.12/docs/ops/best-practices/security/">Security Best Practices&lt;/a>
guide. This is a comprehensive document which pulls together recommendations
from security experts within the Istio community, and industry leaders running
Istio in production. Work is underway to create an opinionated and hardened
security profile for installing Istio in secure environments, but in the interim
we recommend users follow the Security Best Practices guide and configure Istio
to meet their security requirements. With that, let’s look at the analysis and
resolution for various issues raised in the report.&lt;/p>
&lt;h2 id="resolution-and-learnings">Resolution and learnings&lt;/h2>
&lt;h3 id="inability-to-secure-control-plane-network-communications">Inability to secure control plane network communications&lt;/h3>
&lt;p>The report flags configuration options that were available in older versions of
Istio to control how communication is secured to the control plane. Since 1.7,
Istio by default secures all control plane communication and many configuration
options mentioned in the report to manage control plane encryption are no longer
required.&lt;/p>
&lt;p>The debug endpoint mentioned in the report is enabled by default (as of Istio
1.10) to allow users to debug their Istio service mesh using the &lt;code>istioctl&lt;/code> tool.
It can be disabled by setting the environment variable &lt;code>ENABLE_DEBUG_ON_HTTP&lt;/code> to
false as mentioned in the &lt;a href="/v1.12/docs/ops/best-practices/security/#control-plane">Security Best
Practices&lt;/a>
guide. Additionally, in an upcoming version (1.11), this debug endpoint will
be secured by default and a valid Kubernetes service account token will be
required to gain access.&lt;/p>
&lt;h3 id="lack-of-security-related-documentation">Lack of security related documentation&lt;/h3>
&lt;p>The report points out gaps in the security related documentation published with
Istio 1.6. Since then, we have created a detailed &lt;a href="/v1.12/docs/ops/best-practices/security/">Security Best Practices&lt;/a>
guide with recommendations to ensure users can deploy Istio securely to meet
their requirements. Moving forward, we will continue to augment this
documentation with more hardening recommendations. We advise users to monitor
the guide for updates.&lt;/p>
&lt;h3 id="lack-of-virtualservice-gateway-field-validation-enables-request-hijacking">Lack of VirtualService Gateway field validation enables request hijacking&lt;/h3>
&lt;p>For this issue, the report uses a valid but permissive Gateway configuration
that can cause requests to be routed incorrectly. Similar to the Kubernetes
RBAC, Istio APIs, including Gateways, can be tuned to be permissive or
restrictive depending upon your requirements. However, the report surfaced
missing links in our documentation related to best practices and guiding our
users to secure their environments. To address them, we have added a section to
our Security Best Practices guide with steps for running
&lt;a href="/v1.12/docs/ops/best-practices/security/#gateways">Gateways&lt;/a> securely.
In particular, the section describing &lt;a href="/v1.12/docs/ops/best-practices/security/#avoid-overly-broad-hosts-configurations">using namespace prefixes in hosts
specification&lt;/a>
on Gateway resources is strongly recommended to harden your
configuration and prevent this type of request hijacking.&lt;/p>
&lt;h3 id="ingress-gateway-configuration-generation-enables-request-hijacking">Ingress Gateway configuration generation enables request hijacking&lt;/h3>
&lt;p>The report raises possible request hijacking when using the default mechanism of
selecting gateway workloads by labels across namespaces in a Gateway resource.
This behavior was chosen by default as it allows delegation of managing Gateway
and VirtualService resources to the applications team while allowing operations
teams to centrally manage the ingress gateway workloads for meeting their unique
security requirements like running on dedicated nodes for instance. As
highlighted in the report, if this deployment topology is not a requirement in
your environment it is strongly recommended to co-locate Gateway resources with
your gateway workloads and set the environment variable
&lt;code>PILOT_SCOPE_GATEWAY_TO_NAMESPACE&lt;/code> to true.&lt;/p>
&lt;p>Please refer to the &lt;a href="/v1.12/docs/setup/additional-setup/gateway/#gateway-deployment-topologies">gateway deployment topologies guide&lt;/a>
to understand the various recommended deployment models by the
Istio community. Additionally, as mentioned in the
&lt;a href="/v1.12/docs/ops/best-practices/security/#restrict-gateway-creation-privileges">Security Best Practices&lt;/a>
guide, Gateway resource creation should be access controlled using Kubernetes
RBAC or other policy enforcement mechanisms to ensure only authorized entities
can create them.&lt;/p>
&lt;h3 id="other-medium-and-low-severity-issues">Other Medium and Low Severity Issues&lt;/h3>
&lt;p>There are two medium severity issues reported related to debug information
exposed at various levels within the project which can be used to gain access to
sensitive information or orchestrate Denial of Service (DOS) attacks. While
Istio by default enables these debug interfaces for profiling or enabling tools
like &amp;ldquo;istioctl&amp;rdquo;, they can be disabled by setting the environment variable
&lt;code>ENABLE_DEBUG_ON_HTTP&lt;/code> to false as discussed above.&lt;/p>
&lt;p>The report correctly points out that various utilities like &lt;code>sudo&lt;/code>, &lt;code>tcpdump&lt;/code>, etc.
installed in the default images shipped by Istio can lead to privilege
escalation attacks. These utilities are provided to aid runtime debugging of
packets flowing through the mesh, and users are recommended to use
&lt;a href="/v1.12/docs/ops/configuration/security/harden-docker-images/">hardened versions&lt;/a>
of these images in production.&lt;/p>
&lt;p>The report also surfaces a known architectural limitation with any sidecar
proxy-based service mesh implementation which uses &lt;code>iptables&lt;/code> for intercepting
traffic. This mechanism is susceptible to
&lt;a href="/v1.12/docs/ops/best-practices/security/#understand-traffic-capture-limitations">sidecar proxy bypass&lt;/a>,
which is a valid concern for secure environments. It can be addressed by following the
&lt;a href="/v1.12/docs/ops/best-practices/security/#defense-in-depth-with-networkpolicy">defense-in-depth&lt;/a>
recommendation of the Security Best Practices guide. We are
also investigating more secure options in collaboration with the Kubernetes
community.&lt;/p>
&lt;h2 id="the-tradeoff-between-useful-and-secure">The tradeoff between useful and secure&lt;/h2>
&lt;p>You may have noticed a trend in the findings of the assessment and the
recommendations made to address them. Istio provides various configuration
options to create a more secure installation based on your requirement, and we
have introduced a comprehensive &lt;a href="/v1.12/docs/ops/best-practices/security">Security Best Practices&lt;/a>
guide for our users to follow. As Istio is widely adopted in production, it is
a tradeoff for us between switching to secure defaults and possible migration
issues for our existing users on upgrades. The Istio Product Security Working
Group evaluates each of these issues and creates a plan of action to enable
secure default on a case-by-case basis after giving our users a number of
releases to opt-in the secure configuration and migrate their workloads.&lt;/p>
&lt;p>Lastly, there were several lessons for us during and after undergoing a neutral
security assessment. The primary one was to ensure our security practices are
robust to quickly respond to the findings, and more importantly making security
enhancements while maintaining our standards for upgrades without disruption.&lt;/p>
&lt;p>To continue this endeavor, we are always looking for feedback and participation
in the Istio Product Security Working Group, so
&lt;a href="https://github.com/istio/community/blob/master/WORKING-GROUPS.md">join our public meetings&lt;/a>
to raise issues or learn about what we are doing to keep Istio secure!&lt;/p></description><pubDate>Tue, 13 Jul 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/ncc-security-assessment/</link><author>Neeraj Poddar (Aspen Mesh), on behalf of Istio Product Security Working Group</author><guid isPermaLink="true">/v1.12/blog/2021/ncc-security-assessment/</guid><category>istio</category><category>security</category><category>audit</category><category>ncc</category><category>assessment</category></item><item><title>Join us at the Istio Community Meetup in China</title><description>
&lt;p>With the rapid popularization of cloud native technology in China, Istio has also gained popularity in this corner of the world. Almost all Chinese CSPs have creating and are running service mesh products based on Istio.&lt;/p>
&lt;p>We welcomed thousands of Istio users and developers to the first IstioCon in February 2021, and the attendees expressed an interest in participating in more meetups and helping to grow the community at the local level.&lt;/p>
&lt;p>To this end, the Istio community united six partners — the China Academy of Information and Communications Technology, Alibaba Cloud, Huawei Cloud, Intel, Tencent Cloud, and Tetrate — to co-host the first official Istio Community Meetup China. We have invited a number of industry experts to share comprehensive Istio technical practices with everyone at an in-person meetup. We will serve some refreshments, and seats are limited, so we will operate on a first-come first-served basis. &lt;strong>&lt;a href="https://www.huodongxing.com/event/7604616393700?td=3381727549788">Please register to attend&lt;/a>&lt;/strong>.&lt;/p>
&lt;p>&lt;strong>Time and day:&lt;/strong> 13:30-17:30 (CST), July 10, 2021&lt;/p>
&lt;p>&lt;strong>Venue:&lt;/strong> Industrial Internet Exhibition Hall, 2nd Floor, Research Building, China Academy of Information and Communications Technology, No. 52 Huayuan North Road, Haidian District, Beijing&lt;/p>
&lt;h2 id="agenda">Agenda&lt;/h2>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Session time (CST)&lt;/th>
&lt;th>Title&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>13:30 - 13:50&lt;/td>
&lt;td>Sign in&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>13:50 - 14:00&lt;/td>
&lt;td>&lt;em>Welcome&lt;/em>&lt;br>Craig Box, Istio Steering Committee member, Google Cloud&lt;br>Iris Ding, cloud computing engineer, Intel&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>14:00 - 14:30&lt;/td>
&lt;td>&lt;em>Interpretation of the &amp;ldquo;Service Grid Technical Capability Requirements&amp;rdquo; Standard&lt;/em>&lt;br>Yin Xia Mengxue, Engineer, Cloud Computing Department, Academy of Information and Communications Technology&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>14:30 - 15:00&lt;/td>
&lt;td>&lt;em>Service Mesh Data Plane Hot Upgrade&lt;/em>&lt;br>Shi Zehuan, Alibaba Cloud&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>15:00 - 15:30&lt;/td>
&lt;td>&lt;em>Envoy Principle Introduction and Online Problem Pit&lt;/em>&lt;br>Zhang Wei, Data Plane Technical Expert, Huawei Cloud Service Mesh&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>15:30 - 15:45&lt;/td>
&lt;td>Coffee break&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>15:45 - 16:15&lt;/td>
&lt;td>&lt;em>Use eBPF to accelerate Istio/Envoy networking&lt;/em>&lt;br>Zhong Luyao, Intel&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>16:15 - 16:45&lt;/td>
&lt;td>&lt;em>Full-stack service mesh: how Aeraki helps you manage any Layer 7 traffic in Istio&lt;/em>&lt;br>Huabing Zhao, Senior Engineer, Tencent Cloud&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>16:45 - 17:15&lt;/td>
&lt;td>&lt;em>Securing workload deployment with Istio CNI&lt;/em>&lt;br>Zhang Zhihan, Tetrate&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>We want to thank our community in China who have worked on this event, especially Iris Ding, Wei W Hu, Jimmy Song, Zhonghu Xu, Xining Wang, and Huabing Zhao. We hope you can join!&lt;/p></description><pubDate>Tue, 06 Jul 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/istio-community-meetup-china/</link><author>Iris Ding (Intel), Maria Cruz (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/istio-community-meetup-china/</guid><category>community</category><category>meetup</category><category>China</category></item><item><title>Steering and TOC updates</title><description>
&lt;p>Last year we &lt;a href="/v1.12/blog/2020/steering-changes/">introduced a new Steering Committee charter&lt;/a>, which shares governance responsibilities between Contribution Seats, selected based on contributions to the project, and Community Seats, elected by the project members. We &lt;a href="/v1.12/blog/2020/steering-election-results/">elected four members&lt;/a>, with the committee representing seven different companies.&lt;/p>
&lt;p>It&amp;rsquo;s now time to kick off our 2021 election for Community Seats. Members have two weeks to submit nominations, and voting will run from 12 to 25 July. You can learn all about the election, including how to stand and how to vote, in the &lt;a href="https://github.com/istio/community/tree/master/steering/elections/2021">&lt;code>istio/community&lt;/code> repository on GitHub&lt;/a>.&lt;/p>
&lt;p>Just like last year, any &lt;a href="https://github.com/istio/community/blob/master/ROLES.md#member">project member&lt;/a> can stand for election. All Istio members who have been active in the last 12 months are eligible to vote.&lt;/p>
&lt;h2 id="technical-oversight-committee-updates">Technical Oversight Committee updates&lt;/h2>
&lt;p>We wish to offer our thanks to Dan Berg and Joshua Blatt, both long-time contributors to the Istio project, who have recently taken new jobs outside the service mesh space. That left two vacancies on the &lt;a href="https://github.com/istio/community/blob/master/TECH-OVERSIGHT-COMMITTEE.md">Istio Technical Oversight Committee&lt;/a> (TOC), responsible for cross-cutting product and design decisions.&lt;/p>
&lt;p>TOC members are elected by the Steering Committee from the &lt;a href="https://github.com/istio/community/blob/master/WORKING-GROUPS.md#working-group-leads">working group leads&lt;/a>, and last week we voted for two new members:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>&lt;a href="http://github.com/howardjohn">John Howard&lt;/a>&lt;/strong>, from Google, has become one of the most active contributors to Istio since joining the project in January 2019. He is currently a lead in the Networking working group, and has also served as an Environments working group lead and release manager for version 1.4.&lt;/li>
&lt;li>&lt;strong>&lt;a href="https://github.com/brian-avery">Brian Avery&lt;/a>&lt;/strong>, from Red Hat, has been active in the Istio community for over 3 years. He served as Release Manager for Istio 1.3 and 1.6, and has remained actively involved in the Istio release process, including introducing tooling for release notes, streamlining the feature maturity process, and working on documentation testing. Most recently, Brian was a lead in the Test and Release and Product Security working groups.&lt;/li>
&lt;/ul>
&lt;p>Congratulations to John and Brian!&lt;/p>
&lt;p>As our new TOC members step into their roles, they will be vacating their current positions as working group leads. We are always on the lookout for community members who are interested in joining, or leading, Istio working groups. If you’re interested, please reach out in the working group channels &lt;a href="https://slack.istio.io/">on Slack&lt;/a>, or during the &lt;a href="https://github.com/istio/community/blob/master/WORKING-GROUPS.md#working-group-meetings">public working group meetings&lt;/a> of your interest.&lt;/p></description><pubDate>Tue, 29 Jun 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/steering-election/</link><author>Istio Steering Committee</author><guid isPermaLink="true">/v1.12/blog/2021/steering-election/</guid><category>istio</category><category>steering</category><category>governance</category><category>community</category><category>election</category></item><item><title>Configuring failover for external services</title><description>
&lt;p>Istio’s powerful APIs can be used to solve a variety of service mesh use cases. Many users know about its strong ingress and east-west capabilities but it also offers many features for egress (outgoing) traffic. This is especially useful when your application needs to talk to an external service - such as a database endpoint provided by a cloud provider. There are often multiple endpoints to chose from depending on where your workload is running. For example, Amazon&amp;rsquo;s DynamoDB provides &lt;a href="https://docs.aws.amazon.com/general/latest/gr/ddb.html">several endpoints&lt;/a> across their regions. You typically want to choose the endpoint closest to your workload for latency reasons, but you may need to configure automatic failover to another endpoint in case things are not working as expected.&lt;/p>
&lt;p>Similar to services running inside the service mesh, you can configure Istio to detect outliers and failover to a healthy endpoint, while still being completely transparent to your application. In this example, we’ll use Amazon DynamoDB endpoints and pick a primary region that is the same or close to workloads running in a Google Kubernetes Engine (GKE) cluster. We’ll also configure a failover region.&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Routing&lt;/th>
&lt;th>Endpoint&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Primary&lt;/td>
&lt;td>&lt;a href="http://dynamodb.us-east-1.amazonaws.com">http://dynamodb.us-east-1.amazonaws.com&lt;/a>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Failover&lt;/td>
&lt;td>&lt;a href="http://dynamodb.us-west-1.amazonaws.com">http://dynamodb.us-west-1.amazonaws.com&lt;/a>&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>&lt;img src="./external-locality-failover.png" alt="failover" />&lt;/p>
&lt;h2 id="define-external-endpoints-using-a-serviceentry">Define external endpoints using a ServiceEntry&lt;/h2>
&lt;p>&lt;a href="/v1.12/docs/tasks/traffic-management/locality-load-balancing/">Locality load balancing&lt;/a> works based on &lt;code>region&lt;/code> or &lt;code>zone&lt;/code>, which are usually inferred from labels set on the Kubernetes nodes. First, determine the location of your workloads:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl describe node | grep failure-domain.beta.kubernetes.io/region
failure-domain.beta.kubernetes.io/region=us-east1
failure-domain.beta.kubernetes.io/region=us-east1
&lt;/code>&lt;/pre>
&lt;p>In this example, the GKE cluster nodes are running in &lt;code>us-east1&lt;/code>.&lt;/p>
&lt;p>Next, create a &lt;code>ServiceEntry&lt;/code> which aggregates the endpoints you want to use. In this example, we have selected &lt;code>mydb.com&lt;/code> as the host. This is the address your application should be configured to connect to. Set the &lt;code>locality&lt;/code> of the primary endpoint to the same region as your workload:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: external-svc-dns
spec:
hosts:
- mydb.com
location: MESH_EXTERNAL
ports:
- number: 80
name: http
protocol: HTTP
resolution: DNS
endpoints:
- address: dynamodb.us-east-1.amazonaws.com
locality: us-east1
ports:
http: 80
- address: dynamodb.us-west-1.amazonaws.com
locality: us-west
ports:
http: 80
&lt;/code>&lt;/pre>
&lt;p>Let’s deploy a sleep container to use as a test source for sending requests.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/sleep/sleep.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/sleep/sleep.yaml@
&lt;/code>&lt;/pre>&lt;/div>
&lt;p>From the sleep container try going to &lt;code>http://mydb.com&lt;/code> 5 times:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ for i in {1..5}; do kubectl exec deploy/sleep -c sleep -- curl -sS http://mydb.com; echo; sleep 2; done
healthy: dynamodb.us-east-1.amazonaws.com
healthy: dynamodb.us-west-1.amazonaws.com
healthy: dynamodb.us-west-1.amazonaws.com
healthy: dynamodb.us-east-1.amazonaws.com
healthy: dynamodb.us-east-1.amazonaws.com
&lt;/code>&lt;/pre>
&lt;p>You will see that Istio is sending requests to both endpoints. We only want it to send to the endpoint marked with the same region as our nodes.&lt;/p>
&lt;p>For that, we need to configure a &lt;code>DestinationRule&lt;/code>.&lt;/p>
&lt;h2 id="set-failover-conditions-using-a-destinationrule">Set failover conditions using a &lt;code>DestinationRule&lt;/code>&lt;/h2>
&lt;p>Istio’s &lt;code>DestinationRule&lt;/code> lets you configure load balancing, connection pool, and outlier detection settings. We can specify the conditions used to identify an endpoint as unhealthy and remove it from the load balancing pool.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: mydynamodb
spec:
host: mydb.com
trafficPolicy:
outlierDetection:
consecutive5xxErrors: 1
interval: 15s
baseEjectionTime: 1m
&lt;/code>&lt;/pre>
&lt;p>The above &lt;code>DestinationRule&lt;/code> configures the endpoints to be scanned every 15 seconds, and if any endpoint fails with a 5xx error code, even once, it will be marked unhealthy for one minute. If this circuit breaker is not triggered, the traffic will route to the same region as the pod.&lt;/p>
&lt;p>If we run our curl again, we should see that traffic is always going to the &lt;code>us-east1&lt;/code> endpoint.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ for i in {1..5}; do kubectl exec deploy/sleep -c sleep -- curl -sS http://mydb.com; echo; sleep 2; done
healthy: dynamodb.us-east-1.amazonaws.com
healthy: dynamodb.us-east-1.amazonaws.com
healthy: dynamodb.us-east-1.amazonaws.com
healthy: dynamodb.us-east-1.amazonaws.com
healthy: dynamodb.us-east-1.amazonaws.com
&lt;/code>&lt;/pre>
&lt;h2 id="simulate-a-failure">Simulate a failure&lt;/h2>
&lt;p>Next, let&amp;rsquo;s see what happens if the us-east endpoint goes down. To simulate this, let’s modify the ServiceEntry and set the &lt;code>us-east&lt;/code> endpoint to an invalid port:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: external-svc-dns
spec:
hosts:
- mydb.com
location: MESH_EXTERNAL
ports:
- number: 80
name: http
protocol: HTTP
resolution: DNS
endpoints:
- address: dynamodb.us-east-1.amazonaws.com
locality: us-east1
ports:
http: 81 # INVALID - This is purposefully wrong to trigger failover
- address: dynamodb.us-west-1.amazonaws.com
locality: us-west
ports:
http: 80
&lt;/code>&lt;/pre>
&lt;p>Running our curl again shows that traffic is automatically failed over to our us-west region after failing to connect to the us-east endpoint:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ for i in {1..5}; do kubectl exec deploy/sleep -c sleep -- curl -sS http://mydb.com; echo; sleep 2; done
upstream connect error or disconnect/reset before headers. reset reason: connection failure
healthy: dynamodb.us-west-1.amazonaws.com
healthy: dynamodb.us-west-1.amazonaws.com
healthy: dynamodb.us-west-1.amazonaws.com
healthy: dynamodb.us-west-1.amazonaws.com
&lt;/code>&lt;/pre>
&lt;p>You can check the outlier status of the us-east endpoint by running:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl pc endpoints &amp;lt;sleep-pod&amp;gt; | grep mydb
ENDPOINT STATUS OUTLIER CHECK CLUSTER
52.119.226.80:81 HEALTHY FAILED outbound|80||mydb.com
52.94.12.144:80 HEALTHY OK outbound|80||mydb.com
&lt;/code>&lt;/pre>
&lt;h2 id="failover-for-https">Failover for HTTPS&lt;/h2>
&lt;p>Configuring failover for external HTTPS services is just as easy. Your application can still continue to use plain HTTP, and you can let the Istio proxy perform the TLS origination to the HTTPS endpoint.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: external-svc-dns
spec:
hosts:
- mydb.com
ports:
- number: 80
name: http-port
protocol: HTTP
targetPort: 443
resolution: DNS
endpoints:
- address: dynamodb.us-east-1.amazonaws.com
locality: us-east1
- address: dynamodb.us-west-1.amazonaws.com
locality: us-west
&lt;/code>&lt;/pre>
&lt;p>The above ServiceEntry defines the &lt;code>mydb.com&lt;/code> service on port 80 and redirects traffic to the real DynamoDB endpoints on port 443.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: mydynamodb
spec:
host: mydb.com
trafficPolicy:
tls:
mode: SIMPLE
loadBalancer:
simple: ROUND_ROBIN
localityLbSetting:
enabled: true
failover:
- from: us-east1
to: us-west
outlierDetection:
consecutive5xxErrors: 1
interval: 15s
baseEjectionTime: 1m
&lt;/code>&lt;/pre>
&lt;p>The &lt;code>DestinationRule&lt;/code> now performs TLS origination and configures the outlier detection. The rule also has a &lt;a href="/v1.12/docs/reference/config/networking/destination-rule/#LocalityLoadBalancerSetting">failover&lt;/a> field configured where you can specify exactly what regions are failover targets. This is useful when you have several regions defined.&lt;/p>
&lt;h2 id="wrapping-up">Wrapping Up&lt;/h2>
&lt;p>Istio’s &lt;code>VirtualService&lt;/code> and &lt;code>DestinationRule&lt;/code> API’s provide traffic routing, failure recovery and fault injection features so that you can create resilient applications. The ServiceEntry API extends many of these features to external services that are not part of your service mesh.&lt;/p></description><pubDate>Fri, 04 Jun 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/external-locality-failover/</link><author>Ram Vennam (Solo.io)</author><guid isPermaLink="true">/v1.12/blog/2021/external-locality-failover/</guid><category>locality</category><category>region</category><category>failover</category><category>Istio</category><category>outlier</category><category>external</category></item><item><title>Safely upgrade the Istio control plane with revisions and tags</title><description>
&lt;p>Like all security software, your service mesh should be kept up-to-date. The Istio community &lt;a href="/v1.12/docs/releases/supported-releases/">releases new versions every quarter&lt;/a>, with regular patch releases for bug fixes &lt;a href="/v1.12/blog/2021/patch-tuesdays/">and security vulnerabilities&lt;/a>. The operator of a service mesh will need to upgrade the control plane and data plane components many times. You must take care when upgrading, as a mistake could affect your business traffic. Istio has many mechanisms to make it safe to perform upgrades in a controlled manner, and in Istio 1.10 we further improve this operational experience.&lt;/p>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>&lt;a href="/v1.12/news/releases/1.6.x/announcing-1.6/change-notes/">In Istio 1.6&lt;/a>, we added &lt;a href="/v1.12/blog/2020/multiple-control-planes/">basic support for upgrading the service mesh following a canary pattern using revisions&lt;/a>. Using this approach, you can run multiple control planes side-by-side without impacting an existing deployment and slowly migrate workloads from the old control plane to the new.&lt;/p>
&lt;p>To support this revision-based upgrade, Istio introduced a &lt;code>istio.io/rev&lt;/code> label for namespaces. This indicates which control plane revision should inject sidecar proxies for the workloads in the respective namespace. For example, a label of &lt;code>istio.io/rev=1-9-5&lt;/code> indicates the control plane revision &lt;code>1-9-5&lt;/code> should inject the data plane using proxies for &lt;code>1-9-5&lt;/code> for workloads in that namespace.&lt;/p>
&lt;p>If you wanted to upgrade the data-plane proxies for a particular namespace, you would update the &lt;code>istio.io/rev&lt;/code> label to point to a new version, such as &lt;code>istio.io/rev=1-10-0&lt;/code>. Manually changing (or even trying to orchestrate) changes of labels across a large number of namespaces can be error-prone and lead to unintended downtime.&lt;/p>
&lt;h2 id="introducing-revision-tags">Introducing Revision Tags&lt;/h2>
&lt;p>&lt;a href="/v1.12/news/releases/1.10.x/announcing-1.10/">In Istio 1.10&lt;/a>, we&amp;rsquo;ve improved revision-based upgrades with a new feature called &lt;em>&lt;a href="/v1.12/docs/setup/upgrade/canary/#stable-revision-labels-experimental">revision tags&lt;/a>&lt;/em>. A revision tag reduces the number of changes an operator has to make to use revisions, and safely upgrade an Istio control plane. You use the tag as the label for your namespaces, and assign a revision to that tag. This means you don&amp;rsquo;t have to change the labels on a namespace while upgrading, and minimizes the number of manual steps and configuration changes.&lt;/p>
&lt;p>For example, you can define a tag named &lt;code>prod-stable&lt;/code> and point it to the &lt;code>1-9-5&lt;/code> revision of a control plane. You can also define another tag named &lt;code>prod-canary&lt;/code> which points to the &lt;code>1-10-0&lt;/code> revision. You may have a lot of important namespaces in your cluster, and you can label those namespaces with &lt;code>istio.io/rev=prod-stable&lt;/code>. In other namespaces you may be willing to test the new version of Istio, and you can label that namespace &lt;code>istio.io/rev=prod-canary&lt;/code>. The tag will indirectly associate those namespaces with the &lt;code>1-9-5&lt;/code> revision for &lt;code>prod-stable&lt;/code> and &lt;code>1-10-0&lt;/code> for &lt;code>prod-canary&lt;/code> respectively.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:55.51282051282052%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/revision-tags/tags.png" title="Stable revision tags">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/revision-tags/tags.png" alt="Stable revision tags" />
&lt;/a>
&lt;/div>
&lt;figcaption>Stable revision tags&lt;/figcaption>
&lt;/figure>
&lt;p>Once you&amp;rsquo;ve determined the new control plane is suitable for the rest of the &lt;code>prod-stable&lt;/code> namespaces, you can change the tag to point to the new revision. This enables you to update all the namespaces labeled &lt;code>prod-stable&lt;/code> to the new &lt;code>1-10-0&lt;/code> revision without making any changes to the labels on the namespaces. You will need to restart the workloads in a namespace once you&amp;rsquo;ve changed the tag to point to a different revision.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:55.51282051282052%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/revision-tags/tags-updated.png" title="Updated revision tags">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/revision-tags/tags-updated.png" alt="Updated revision tags" />
&lt;/a>
&lt;/div>
&lt;figcaption>Updated revision tags&lt;/figcaption>
&lt;/figure>
&lt;p>Once you&amp;rsquo;re satisfied with the upgrade to the new control-plane revision, you can remove the old control plane.&lt;/p>
&lt;h2 id="stable-revision-tags-in-action">Stable revision tags in action&lt;/h2>
&lt;p>To create a new &lt;code>prod-stable&lt;/code> tag for a revision &lt;code>1-9-5&lt;/code>, run the following command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag set prod-stable --revision 1-9-5
&lt;/code>&lt;/pre>
&lt;p>You can then label your namespaces with the &lt;code>istio.io/rev=prod-stable&lt;/code> label. Note, if you installed a &lt;code>default&lt;/code> revision (i.e., no revision) of Istio, you will first have to remove the standard injection label:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl label ns istioinaction istio-injection-
$ kubectl label ns istioinaction istio.io/rev=prod-stable
&lt;/code>&lt;/pre>
&lt;p>You can list the tags in your mesh with the following:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag list
TAG REVISION NAMESPACES
prod-stable 1-9-5 istioinaction
&lt;/code>&lt;/pre>
&lt;p>A tag is implemented with a &lt;code>MutatingWebhookConfiguration&lt;/code>. You can verify a corresponding &lt;code>MutatingWebhookConfiguration&lt;/code> has been created:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get MutatingWebhookConfiguration
NAME WEBHOOKS AGE
istio-revision-tag-prod-stable 2 75s
istio-sidecar-injector 1 5m32s
&lt;/code>&lt;/pre>
&lt;p>Let&amp;rsquo;s say you are trying to canary a new revision of the control plane based on 1.10.0. First you would install the new version using a revision:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl install -y --set profile=minimal --revision 1-10-0
&lt;/code>&lt;/pre>
&lt;p>You can create a new tag called &lt;code>prod-canary&lt;/code> and point that to your &lt;code>1-10-0&lt;/code> revision:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag set prod-canary --revision 1-10-0
&lt;/code>&lt;/pre>
&lt;p>Then label your namespaces accordingly:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl label ns istioinaction-canary istio.io/rev=prod-canary
&lt;/code>&lt;/pre>
&lt;p>If you list out the tags in your mesh, you will see two stable tags pointing to two different revisions:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag list
TAG REVISION NAMESPACES
prod-stable 1-9-5 istioinaction
prod-canary 1-10-0 istioinaction-canary
&lt;/code>&lt;/pre>
&lt;p>Any of the namespaces that you have labeled with &lt;code>istio.io/rev=prod-canary&lt;/code> will be injected by the control plane that corresponds to the &lt;code>prod-canary&lt;/code> stable tag name (which in this example points to the &lt;code>1-10-0&lt;/code> revision). When you&amp;rsquo;re ready, you can switch the &lt;code>prod-stable&lt;/code> tag to the new control plane with:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag set prod-stable --revision 1-10-0 --overwrite
&lt;/code>&lt;/pre>
&lt;p>Any time you switch a tag to point to a new revision, you will need to restart the workloads in any respective namespace to pick up the new revision&amp;rsquo;s proxy.&lt;/p>
&lt;p>When both the &lt;code>prod-stable&lt;/code> and &lt;code>prod-canary&lt;/code> no longer point to the old revision, it may be safe to remove the old revision as follows:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x uninstall --revision 1-9-5
&lt;/code>&lt;/pre>
&lt;h2 id="wrapping-up">Wrapping up&lt;/h2>
&lt;p>Using revisions makes it safer to canary changes to an Istio control plane. In large environments with lots of namespaces, you may prefer to use stable tags, as we&amp;rsquo;ve introduced in this blog, to remove the number of moving pieces and simplify any automation you may build around updating an Istio control plane. Please check out the &lt;a href="/v1.12/news/releases/1.10.x/announcing-1.10/">1.10 release&lt;/a> &lt;a href="/v1.12/docs/setup/upgrade/canary/#stable-revision-labels-experimental">and the new tag feature&lt;/a> and give us your feedback!&lt;/p></description><pubDate>Wed, 26 May 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/revision-tags/</link><author>Christian Posta (Solo.io), Lin Sun (Solo.io), Sam Naser (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/revision-tags/</guid><category>upgrades</category><category>revisions</category><category>operations</category><category>canary</category></item><item><title>Happy Birthday, Istio!</title><description>
&lt;h2 id="celebrating-istio-s-4th-birthday">Celebrating Istio’s 4th birthday&lt;/h2>
&lt;p>Four years ago today, the Istio project was born to the open source world. To celebrate this anniversary,
we are hosting a week-long birthday celebration that focuses on contributions to the Istio project that
stem from using Istio in production. Read on to learn how to participate in this celebration and enter a
chance to win some Istio swag.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/istio-4th-birthday/1.png" title="Istio&amp;#39;s 4th Birthday!">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/istio-4th-birthday/1.png" alt="Istio&amp;#39;s 4th Birthday!" />
&lt;/a>
&lt;/div>
&lt;figcaption>Istio&amp;#39;s 4th Birthday!&lt;/figcaption>
&lt;/figure>
&lt;h3 id="a-year-of-important-developments-for-istio">A year of important developments for Istio&lt;/h3>
&lt;p>Over the last 12 months, the Istio project has been very focused on the &lt;a href="https://dzone.com/articles/defining-day-2-operations">day-0
&amp;amp; day-1&lt;/a> experience for
users by actively listening to our users through UX surveys and GitHub issues.&lt;/p>
&lt;ul>
&lt;li>We &lt;a href="/v1.12/blog/2020/istiod/">simplified the control plane architecture&lt;/a> and
made Istio easier to install, configure and upgrade.&lt;/li>
&lt;li>We provided clarity and process to our feature status and promotion of features and APIs.&lt;/li>
&lt;li>We simplified the debugging experience with various istioctl commands.&lt;/li>
&lt;li>We expanded the mesh to services running in &lt;a href="/v1.12/news/releases/1.9.x/announcing-1.9/#virtual-machine-integration-beta">VMs&lt;/a>
and &lt;a href="/v1.12/docs/setup/install/multicluster/">multiple clusters&lt;/a>.&lt;/li>
&lt;li>We made &lt;a href="/v1.12/blog/2021/statefulsets-made-easier/">&lt;code>StatefulSet&lt;/code> easier&lt;/a> to use in Istio 1.10 with zero-configuration.&lt;/li>
&lt;li>We made various performance improvements to the Istio control plane and data plane via &lt;a href="/v1.12/blog/2021/discovery-selectors/">discovery selectors&lt;/a>, sidecar resources etc.&lt;/li>
&lt;li>We &lt;a href="/v1.12/blog/2021/wasm-progress/">introduced WebAssembly&lt;/a> as our extensibility platform which has helped users tailor Istio to their needs.&lt;/li>
&lt;li>We beefed up our CVE management and release processes to meet enterprise needs.&lt;/li>
&lt;/ul>
&lt;p>&lt;a href="/v1.12/blog/2020/tradewinds-2020/">Read more&lt;/a> about improvements to
Istio in 2020 that made this technology easier to use.&lt;/p>
&lt;p>In February 2021, we celebrated the first IstioCon! This community-led event was an opportunity
for users and developers to share &lt;a href="https://www.youtube.com/playlist?list=PL7wB27eZmdffS-g_xh7X-b0echc_XZMKV">many examples&lt;/a>
of how they use Istio in production and lessons
learned from it. IstioCon was a great opportunity to feature more than 25 end-user companies,
like Salesforce, T-Mobile, and Airbnb, among others, to feature maintainers from across the Istio
ecosystem, and to share the &lt;a href="https://www.youtube.com/watch?v=WmjTeN-jtdY">Istio project roadmap&lt;/a>.&lt;/p>
&lt;p>This inaugural community conference was a major success, with more than 4,000 registrants from 80
countries participating. The program was conducted during US and Asia time zones,
and in English and Chinese languages to accommodate big user communities in various continents.
Learn more about the &lt;a href="https://events.istio.io/istiocon-2021/slides/IstioCon2021-Report.pdf">impact&lt;/a>
of IstioCon and find the &lt;a href="https://events.istio.io/istiocon-2021/sessions/">presentations&lt;/a> on
the conference website.&lt;/p>
&lt;h3 id="how-to-participate-in-istio-s-4th-birthday-celebration">How to participate in Istio’s 4th Birthday celebration&lt;/h3>
&lt;p>Contributions are key to the long life of an open source project. This is why, on its 4th birthday,
we want to hear about your contributions to the Istio project. To participate in this campaign,
share on Twitter a contribution you made to the project and why it matters, using the hashtag #IstioTurns4
and #IstioBirthday. You can submit posts from Monday, May 24th at 9 am Pacific, until
Friday, May 28th, at 12 pm Pacific, to enter a chance to win some Istio swag.&lt;/p>
&lt;p>The other way of participating in this campaign is by joining the Istio community meetup, which
will take place on Thursday, May 27th at 10 am Pacific. At this event, we will have Pratima Nambiar
discuss contributions that have stemmed from using Istio in production at Salesforce. Join the event,
and ask a question or make a comment on the demo, and enter a chance to win some Istio swag.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/istio-4th-birthday/2.png" title="Istio Community Meetup!">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/istio-4th-birthday/2.png" alt="Istio Community Meetup!" />
&lt;/a>
&lt;/div>
&lt;figcaption>Istio Community Meetup!&lt;/figcaption>
&lt;/figure></description><pubDate>Mon, 24 May 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/istio-4th-birthday/</link><author>Maria Cruz (Google), Aizhamal Nurmamat kyzy (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/istio-4th-birthday/</guid><category>community</category><category>birthday</category><category>celebration</category></item><item><title>Announcing Support for 1.8 to 1.10 Direct Upgrades</title><description>
&lt;p>As Service Mesh technology moves from cutting edge to stable infrastructure, many users have expressed an interest in upgrading their service mesh less frequently, as qualifying a new minor release can take a lot of time. Upgrading can be especially difficult for users who don’t keep up with new releases, as Istio has not supported upgrades across multiple minor versions. To upgrade from &lt;code>1.6.x&lt;/code> to &lt;code>1.8.x&lt;/code>, users first had to upgrade to &lt;code>1.7.x&lt;/code> and then to &lt;code>1.8.x&lt;/code>.&lt;/p>
&lt;p>With the release of Istio 1.10, we are announcing Alpha level support for upgrading directly from Istio &lt;code>1.8.x&lt;/code> to &lt;code>1.10.x&lt;/code>, without upgrading to &lt;code>1.9.x&lt;/code>. We hope this will reduce the operational burden of running Istio, in keeping with our 2021 theme of improving Day 2 Operations.&lt;/p>
&lt;h2 id="upgrade-from-1-8-to-1-10">Upgrade From 1.8 to 1.10&lt;/h2>
&lt;p>For direct upgrades we recommend using the canary upgrade method so that control plane functionality can be verified before cutting workloads over to the new version. We’ll also be using &lt;a href="/v1.12/blog/2021/revision-tags/">revision tags&lt;/a> in this guide, an improvement to canary upgrades that was introduced in 1.10, so users don’t have to change the labels on a namespace while upgrading.&lt;/p>
&lt;p>First, using a version &lt;code>1.10&lt;/code> or newer &lt;code>istioctl&lt;/code>, create a revision tag &lt;code>stable&lt;/code> pointed to your existing &lt;code>1.8&lt;/code> revision. From now on let’s assume this revision is called &lt;code>1-8-5&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag set stable --revision 1-8-5
&lt;/code>&lt;/pre>
&lt;p>If your 1.8 installation did not have an associated revision, we can create this revision tag with:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag set stable --revision default
&lt;/code>&lt;/pre>
&lt;p>Now, relabel your namespaces that were previously labeled with &lt;code>istio-injection=enabled&lt;/code> or &lt;code>istio.io/rev=&amp;lt;REVISION&amp;gt;&lt;/code> with &lt;code>istio.io/rev=stable&lt;/code>. Download the Istio 1.10.0 release and install the new control plane with a revision:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl install --revision 1-10-0 -y
&lt;/code>&lt;/pre>
&lt;p>Now evaluate that the &lt;code>1.10&lt;/code> revision has come up correctly and is healthy. Once satisfied with the stability of new revision you can set the revision tag to the new revision:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag set stable --revision 1-10-0 --overwrite
&lt;/code>&lt;/pre>
&lt;p>Verify that the revision tag &lt;code>stable&lt;/code> is pointing to the new revision:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag list
TAG REVISION NAMESPACES
stable 1-10-0 ...
&lt;/code>&lt;/pre>
&lt;p>Once prepared to move existing workloads over to the new 1.10 revision, the workloads must be restarted so that the sidecar proxies will use the new control plane. We can go through namespaces one by one and roll the workloads over to the new version:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl rollout restart deployments -n …
&lt;/code>&lt;/pre>
&lt;p>Notice an issue after rolling out workloads to the new Istio version? No problem! Since you’re using canary upgrades, the old control plane is still running and we can just switch back over.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x revision tag set prod --revision 1-8-5
&lt;/code>&lt;/pre>
&lt;p>Then after triggering another rollout, your workloads will be back on the old version.&lt;/p>
&lt;p>We look forward to hearing about your experience with direct upgrades, and look forward to improving and expanding this functionality in the future.&lt;/p></description><pubDate>Mon, 24 May 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/direct-upgrade/</link><author>Mitch Connors (Google), Sam Naser (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/direct-upgrade/</guid><category>upgrade</category><category>Istio</category><category>revision</category></item><item><title>StatefulSets Made Easier With Istio 1.10</title><description>
&lt;p>Kubernetes &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/">&lt;code>StatefulSets&lt;/code>&lt;/a> are commonly used to manage stateful applications. In addition to managing the deployment and scaling of a set of &lt;code>Pods&lt;/code>, &lt;code>StatefulSets&lt;/code> provide guarantees about the ordering and uniqueness of those &lt;code>Pods&lt;/code>. Common applications used with &lt;code>StatefulSets&lt;/code> include ZooKeeper, Cassandra, Elasticsearch, Redis and NiFi.&lt;/p>
&lt;p>The Istio community has been making gradual progress towards zero-configuration support for &lt;code>StatefulSets&lt;/code>; from automatic mTLS, to eliminating the need to create &lt;code>DestinationRule&lt;/code> or &lt;code>ServiceEntry&lt;/code> resources, to the most recent &lt;a href="/v1.12/blog/2021/upcoming-networking-changes/">pod networking changes in Istio 1.10&lt;/a>.&lt;/p>
&lt;p>What is unique about using a &lt;code>StatefulSet&lt;/code> with a service mesh? The &lt;code>StatefulSet&lt;/code> pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling. The kind of apps that run in a &lt;code>StatefulSet&lt;/code> are often those that need to communicate among their pods, and, as they come from a world of hard-coded IP addresses, may listen on the pod IP only, instead of &lt;code>0.0.0.0&lt;/code>.&lt;/p>
&lt;p>ZooKeeper, for example, is configured by default to not listen on all IPs for quorum communication:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >quorumListenOnAllIPs=false
&lt;/code>&lt;/pre>
&lt;p>Over the last few releases, the Istio community has &lt;a href="https://github.com/istio/istio/issues/10659">reported many issues&lt;/a> around support for applications running in &lt;code>StatefulSets&lt;/code>.&lt;/p>
&lt;h2 id="statefulsets-in-action-prior-to-istio-1-10">&lt;code>StatefulSets&lt;/code> in action, prior to Istio 1.10&lt;/h2>
&lt;p>In a GKE cluster running Kubernetes 1.19, we have Istio 1.9.5 installed. We enabled automatic sidecar injection in the &lt;code>default&lt;/code> namespace, then we installed ZooKeeper using the &lt;a href="https://artifacthub.io/packages/helm/bitnami/zookeeper">Helm charts provided by Bitnami&lt;/a>, along with the Istio &lt;code>sleep&lt;/code> pod for interactive debugging:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm install my-release bitnami/zookeeper --set replicaCount=3
$ kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/sleep/sleep.yaml
&lt;/code>&lt;/pre>
&lt;p>After a few minutes, all pods come up nicely with sidecar proxies:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl get pods,svc
NAME READY STATUS RESTARTS AGE
my-release-zookeeper-0 2/2 Running 0 3h4m
my-release-zookeeper-1 2/2 Running 0 3h4m
my-release-zookeeper-2 2/2 Running 0 3h5m
pod/sleep-8f795f47d-qkgh4 2/2 Running 0 3h8m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-release-zookeeper ClusterIP 10.100.1.113 &amp;lt;none&amp;gt; 2181/TCP,2888/TCP,3888/TCP 3h
my-release-zookeeper-headless ClusterIP None &amp;lt;none&amp;gt; 2181/TCP,2888/TCP,3888/TCP 3h
service/sleep ClusterIP 10.100.9.26 &amp;lt;none&amp;gt; 80/TCP 3h
&lt;/code>&lt;/pre>
&lt;p>Are our ZooKeeper services working and is the status &lt;code>Running&lt;/code>? Let’s find out! ZooKeeper listens on 3 ports:&lt;/p>
&lt;ul>
&lt;li>Port 2181 is the TCP port for clients to connect to the ZooKeeper service&lt;/li>
&lt;li>Port 2888 is the TCP port for peers to connect to other peers&lt;/li>
&lt;li>Port 3888 is the dedicated TCP port for leader election&lt;/li>
&lt;/ul>
&lt;p>By default, the ZooKeeper installation configures port 2181 to listen on &lt;code>0.0.0.0&lt;/code> but ports 2888 and 3888 only listen on the pod IP. Let’s check out the network status on each of these ports from one of the ZooKeeper pods:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl exec my-release-zookeeper-1 -c istio-proxy -- netstat -na | grep -E &amp;#39;(2181|2888|3888)&amp;#39;
tcp 0 0 0.0.0.0:2181 0.0.0.0:* LISTEN
tcp 0 0 10.96.7.7:3888 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:2181 127.0.0.1:37412 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37486 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37456 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37498 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37384 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37514 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37402 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37434 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37526 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37374 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37442 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:37464 TIME_WAIT
&lt;/code>&lt;/pre>
&lt;p>There is nothing &lt;code>ESTABLISHED&lt;/code> on port 2888 or 3888. Next, let us get the ZooKeeper server status:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl exec my-release-zookeeper-1 -c zookeeper -- /opt/bitnami/zookeeper/bin/zkServer.sh status
/opt/bitnami/java/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/bitnami/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Error contacting service. It is probably not running.
&lt;/code>&lt;/pre>
&lt;p>From the above output, you can see the ZooKeeper service is not functioning properly. Let us check the cluster configuration for one of the ZooKeeper pods:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ istioctl proxy-config cluster my-release-zookeeper-1 --port 3888 --direction inbound -o json
[
{
&amp;#34;name&amp;#34;: &amp;#34;inbound|3888||&amp;#34;,
&amp;#34;type&amp;#34;: &amp;#34;STATIC&amp;#34;,
&amp;#34;connectTimeout&amp;#34;: &amp;#34;10s&amp;#34;,
&amp;#34;loadAssignment&amp;#34;: {
&amp;#34;clusterName&amp;#34;: &amp;#34;inbound|3888||&amp;#34;,
&amp;#34;endpoints&amp;#34;: [
{
&amp;#34;lbEndpoints&amp;#34;: [
{
&amp;#34;endpoint&amp;#34;: {
&amp;#34;address&amp;#34;: {
&amp;#34;socketAddress&amp;#34;: {
&amp;#34;address&amp;#34;: &amp;#34;127.0.0.1&amp;#34;,
&amp;#34;portValue&amp;#34;: 3888
}
}
}
}
]
}
]
},
...
&lt;/code>&lt;/pre>
&lt;p>What is interesting here is that the inbound on port 3888 has &lt;code>127.0.0.1&lt;/code> as its endpoint. This is because the Envoy proxy, in versions of Istio prior to 1.10, redirects the inbound traffic to the &lt;code>loopback&lt;/code> interface, as described in &lt;a href="/v1.12/blog/2021/upcoming-networking-changes/">our blog post about the change&lt;/a>.&lt;/p>
&lt;h2 id="statefulsets-in-action-with-istio-1-10">&lt;code>StatefulSets&lt;/code> in action with Istio 1.10&lt;/h2>
&lt;p>Now, we have upgraded our cluster to Istio 1.10 and configured the &lt;code>default&lt;/code> namespace to enable 1.10 sidecar injection. Let’s rolling restart the ZooKeeper &lt;code>StatefulSet&lt;/code> to update the pods to use the new version of the sidecar proxy:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl rollout restart statefulset my-release-zookeeper
&lt;/code>&lt;/pre>
&lt;p>Once the ZooKeeper pods reach the running status, let’s check out the network connections for these 3 ports from any of the ZooKeeper pods:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl exec my-release-zookeeper-1 -c istio-proxy -- netstat -na | grep -E &amp;#39;(2181|2888|3888)&amp;#39;
tcp 0 0 0.0.0.0:2181 0.0.0.0:* LISTEN
tcp 0 0 10.96.8.10:2888 0.0.0.0:* LISTEN
tcp 0 0 10.96.8.10:3888 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.6:42571 10.96.8.10:2888 ESTABLISHED
tcp 0 0 10.96.8.10:2888 127.0.0.6:42571 ESTABLISHED
tcp 0 0 127.0.0.6:42655 10.96.8.10:2888 ESTABLISHED
tcp 0 0 10.96.8.10:2888 127.0.0.6:42655 ESTABLISHED
tcp 0 0 10.96.8.10:37876 10.96.6.11:3888 ESTABLISHED
tcp 0 0 10.96.8.10:44872 10.96.7.10:3888 ESTABLISHED
tcp 0 0 10.96.8.10:37878 10.96.6.11:3888 ESTABLISHED
tcp 0 0 10.96.8.10:44870 10.96.7.10:3888 ESTABLISHED
tcp 0 0 127.0.0.1:2181 127.0.0.1:54508 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54616 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54664 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54526 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54532 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54578 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54634 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54588 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54610 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54550 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54560 TIME_WAIT
tcp 0 0 127.0.0.1:2181 127.0.0.1:54644 TIME_WAIT
&lt;/code>&lt;/pre>
&lt;p>There are &lt;code>ESTABLISHED&lt;/code> connections on both port 2888 and 3888! Next, let us check out the ZooKeeper server status:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl exec my-release-zookeeper-1 -c zookeeper -- /opt/bitnami/zookeeper/bin/zkServer.sh status
/opt/bitnami/java/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/bitnami/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower
&lt;/code>&lt;/pre>
&lt;p>The ZooKeeper service is now running!&lt;/p>
&lt;p>We can connect to each of the ZooKeeper pods from the &lt;code>sleep&lt;/code> pod and run the below command to discover the server status of each pod within the &lt;code>StatefulSet&lt;/code>. Note that there is no need to create ServiceEntry resources for any of the ZooKeeper pods and we can call these pods directly using their DNS names (e.g. &lt;code>my-release-zookeeper-0.my-release-zookeeper-headless&lt;/code>) from the &lt;code>sleep&lt;/code> pod.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl exec -it deploy/sleep -c sleep -- sh -c &amp;#39;for x in my-release-zookeeper-0.my-release-zookeeper-headless my-release-zookeeper-1.my-release-zookeeper-headless my-release-zookeeper-2.my-release-zookeeper-headless; do echo $x; echo srvr|nc $x 2181; echo; done&amp;#39;
my-release-zookeeper-0.my-release-zookeeper-headless
Zookeeper version: 3.7.0-e3704b390a6697bfdf4b0bef79e3da7a4f6bac4b, built on 2021-03-17 09:46 UTC
Latency min/avg/max: 1/7.5/20
Received: 3845
Sent: 3844
Connections: 1
Outstanding: 0
Zxid: 0x200000002
Mode: follower
Node count: 6
my-release-zookeeper-1.my-release-zookeeper-headless
Zookeeper version: 3.7.0-e3704b390a6697bfdf4b0bef79e3da7a4f6bac4b, built on 2021-03-17 09:46 UTC
Latency min/avg/max: 0/0.0/0
Received: 3856
Sent: 3855
Connections: 1
Outstanding: 0
Zxid: 0x200000002
Mode: follower
Node count: 6
my-release-zookeeper-2.my-release-zookeeper-headless
Zookeeper version: 3.7.0-e3704b390a6697bfdf4b0bef79e3da7a4f6bac4b, built on 2021-03-17 09:46 UTC
Latency min/avg/max: 0/0.0/0
Received: 3855
Sent: 3854
Connections: 1
Outstanding: 0
Zxid: 0x200000002
Mode: leader
Node count: 6
Proposal sizes last/min/max: 48/48/48
&lt;/code>&lt;/pre>
&lt;p>Now our ZooKeeper service is running, let’s use Istio to secure all communication to our regular and headless services. Apply mutual TLS to the &lt;code>default&lt;/code> namespace:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -n default -f - &amp;lt;&amp;lt;EOF
apiVersion: &amp;#34;security.istio.io/v1beta1&amp;#34;
kind: &amp;#34;PeerAuthentication&amp;#34;
metadata:
name: &amp;#34;default&amp;#34;
spec:
mtls:
mode: STRICT
EOF
&lt;/code>&lt;/pre>
&lt;p>Continue sending some traffic from the &lt;code>sleep&lt;/code> pod and bring up the Kiali dashboard to visualize the services in the &lt;code>default&lt;/code> namespace:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:38.4204909284952%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/statefulsets-made-easier/view-zookeeper-from-kiali.png" title="Visualize the ZooKeeper Services in Kiali">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/statefulsets-made-easier/view-zookeeper-from-kiali.png" alt="Visualize the ZooKeeper Services in Kiali" />
&lt;/a>
&lt;/div>
&lt;figcaption>Visualize the ZooKeeper Services in Kiali&lt;/figcaption>
&lt;/figure>
&lt;p>The padlock icons on the traffic flows indicate that the connections are secure.&lt;/p>
&lt;h2 id="wrapping-up">Wrapping up&lt;/h2>
&lt;p>With the new networking changes in Istio 1.10, a Kubernetes pod with a sidecar has the same networking behavior as a pod without a sidecar. This change enables stateful applications to function properly in Istio as we have shown you in this post. We believe this is a huge step towards Istio’s goal of providing transparent service mesh and zero-configuration Istio.&lt;/p></description><pubDate>Wed, 19 May 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/statefulsets-made-easier/</link><author>Lin Sun (Solo.io), Christian Posta (Solo.io), John Howard (Google), Zhonghu Xu (Huawei)</author><guid isPermaLink="true">/v1.12/blog/2021/statefulsets-made-easier/</guid><category>statefulset</category><category>Istio</category><category>networking</category><category>localhost</category><category>loopback</category><category>eth0</category></item><item><title>Updates to how Istio security releases are handled: Patch Tuesday, embargoes, and 0-days</title><description>
&lt;p>While most of the work in the Istio Product Security Working Group is done behind the scenes, we are listening
to the community in setting expectations for security releases. We understand that it is difficult for mesh
administrators, operators and vendors to be aware of security bulletins and security releases.&lt;/p>
&lt;p>We currently disclose vulnerabilities and security releases via numerous channels:&lt;/p>
&lt;ul>
&lt;li>&lt;a href="https://istio.io">istio.io&lt;/a> via our &lt;a href="/v1.12/news/releases/">Release Announcements&lt;/a> and &lt;a href="/v1.12/news/security/">Security Bulletins&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://discuss.istio.io/c/announcements/5">Discuss&lt;/a>&lt;/li>
&lt;li>announcements channel on &lt;a href="https://istio.slack.com">Slack&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://twitter.com/IstioMesh">Twitter&lt;/a>&lt;/li>
&lt;li>&lt;a href="/v1.12/news/feed.xml">RSS&lt;/a>&lt;/li>
&lt;/ul>
&lt;p>When operating any software, it is preferable to plan for possible downtime when upgrading. Given the work that the Istio
community is doing around Day 2 operations in 2021, the Environments working group has done a good job to streamline many
upgrade issues users have seen. The Product Security Working Group intends to help Day 2 operations by having routine
security release days so that upgrade operations can be planned in advance for our users.&lt;/p>
&lt;h2 id="patch-tuesdays">Patch Tuesdays&lt;/h2>
&lt;p>The Product Security working group is intending to ship a security release the 2nd Tuesday of each month. These security
releases may contain fixes for multiple CVEs. It is the intent of the Product Security working group to have these
security releases not contain any other fixes, although that may not always be possible.&lt;/p>
&lt;p>When the Product Security working group intends to ship an upcoming security patch, an
announcement will be made on &lt;a href="https://discuss.istio.io/c/announcements/5">the Istio discussion
board&lt;/a> 2 weeks prior to release. If you&amp;rsquo;re
running Istio in production, we suggest you watch the Announcements category to be
notified of such a release. If no such announcement is made there will not be a security
release for that month, barring some exceptions listed below.&lt;/p>
&lt;h3 id="first-patch-tuesday">First Patch Tuesday&lt;/h3>
&lt;p>We are pleased to announce that &lt;a href="/v1.12/news/releases/1.9.x/announcing-1.9.5/">Istio 1.9.5&lt;/a>, and the final release of Istio 1.8,
&lt;a href="/v1.12/news/releases/1.8.x/announcing-1.8.6/">1.8.6&lt;/a>, are the first security releases to fit this pattern. As Istio 1.10 will
be shipping soon we are intending to continue this new tradition in June.&lt;/p>
&lt;p>These releases fix 3 CVEs. Please see the release pages for information regarding the specific CVEs fixed.&lt;/p>
&lt;h2 id="unscheduled-security-releases">Unscheduled security releases&lt;/h2>
&lt;h3 id="0-day-vulnerabilities">0-day vulnerabilities&lt;/h3>
&lt;p>Unfortunately, 0-day vulnerabilities cannot be planned. Upon disclosure, the Product Security Working Group will
need to issue an out-of-band security release. The above methods will be used to disclose such issues, so please use
at least one of them to be notified of such disclosures.&lt;/p>
&lt;h3 id="third-party-embargoes">Third party embargoes&lt;/h3>
&lt;p>Similar to 0-day vulnerabilities, security releases can be dictated by third party embargoes, namely Envoy.
When this occurs, Istio will release a same-day patch once the embargo is lifted.&lt;/p>
&lt;h2 id="security-best-practices">Security Best Practices&lt;/h2>
&lt;p>The &lt;a href="/v1.12/docs/ops/best-practices/security/">Istio Security Best Practices&lt;/a> has seen many improvements over the past few
months. We recommend you check it regularly, as many of our recent security bulletins can be mitigated by utilizing
methods discussed in the Security Best Practices page.&lt;/p>
&lt;h2 id="early-disclosure-list">Early Disclosure List&lt;/h2>
&lt;p>If you meet &lt;a href="https://github.com/istio/community/blob/master/EARLY-DISCLOSURE.md#membership-criteria">the criteria&lt;/a> to be
a part of the &lt;a href="https://github.com/istio/community/blob/master/EARLY-DISCLOSURE.md">Istio Early Disclosure&lt;/a> list, please
apply for membership. Patches for upcoming security releases will be made available to the early disclosure list ~2 weeks
prior to Istio&amp;rsquo;s Patch Tuesday.&lt;/p>
&lt;p>There will be times when an upcoming Istio security release will also need patches from Envoy. We cannot redistribute
Envoy patches due to their embargo. &lt;a href="https://github.com/envoyproxy/envoy/security/policy">Please refer to Envoy&amp;rsquo;s guidance&lt;/a>
on how to join their early disclosure list.&lt;/p>
&lt;h2 id="security-feedback">Security Feedback&lt;/h2>
&lt;p>The Product Security Working Group holds bi-weekly meetings on Tuesdays from 9:00-9:30 Pacific. For more information see
the &lt;a href="https://calendar.google.com/calendar/embed?src=4uhe8fi8sf1e3tvmvh6vrq2dog%40group.calendar.google.com&amp;amp;ctz=America%2FLos_Angeles">Istio Working Group Calendar&lt;/a>.&lt;/p>
&lt;p>Our next public meeting will be held on May 25, 2021. Please join us!&lt;/p></description><pubDate>Tue, 11 May 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/patch-tuesdays/</link><author>Jacob Delgado (Aspen Mesh)</author><guid isPermaLink="true">/v1.12/blog/2021/patch-tuesdays/</guid><category>cve</category><category>product security</category></item><item><title>Use discovery selectors to configure namespaces for your Istio service mesh</title><description>
&lt;p>As users move their services to run in the Istio service mesh, they are often surprised that the control plane watches and processes all of the Kubernetes resources, from all namespaces in the cluster, by default. This can be an issue for very large clusters with lots of namespaces and deployments, or even for a moderately sized cluster with rapidly churning resources (for example, Spark jobs).&lt;/p>
&lt;p>Both &lt;a href="https://github.com/istio/istio/issues/26679">in the community&lt;/a> as well as for our large-scale customers at &lt;a href="https://solo.io">Solo.io&lt;/a>, we need a way to dynamically restrict the set of namespaces that are part of the mesh so that the Istio control plane only processes resources in those namespaces. The ability to restrict the namespaces enables Istiod to watch and push fewer resources and associated changes to the sidecars, thus improving the overall performance on the control plane and data plane.&lt;/p>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>By default, Istio watches all Namespaces, Services, Endpoints and Pods in a cluster. For example, in my Kubernetes cluster, I deployed the &lt;code>sleep&lt;/code> service in the default namespace, and the &lt;code>httpbin&lt;/code> service in the &lt;code>ns-x&lt;/code> namespace. I’ve added the &lt;code>sleep&lt;/code> service to the mesh, but I have no plan to add the &lt;code>httpbin&lt;/code> service to the mesh, or have any service in the mesh interact with the &lt;code>httpbin&lt;/code> service.&lt;/p>
&lt;p>Use &lt;code>istioctl proxy-config endpoint&lt;/code> command to display all the endpoints for the &lt;code>sleep&lt;/code> deployment:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:35.128205128205124%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/discovery-selectors/endpoints-default.png" title="Endpoints for Sleep Deployment">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/discovery-selectors/endpoints-default.png" alt="Endpoints for Sleep Deployment" />
&lt;/a>
&lt;/div>
&lt;figcaption>Endpoints for Sleep Deployment&lt;/figcaption>
&lt;/figure>
&lt;p>Note that the &lt;code>httpbin&lt;/code> service endpoint in the &lt;code>ns-x&lt;/code> namespace is in the list of discovered endpoints. This may not be an issue when you only have a few services. However, when you have hundreds of services that don&amp;rsquo;t interact with any of the services running in the Istio service mesh, you probably don&amp;rsquo;t want your Istio control plane to watch these services and send their information to the sidecars of your services in the mesh.&lt;/p>
&lt;h2 id="introducing-discovery-selectors">Introducing Discovery Selectors&lt;/h2>
&lt;p>Starting with Istio 1.10, we are introducing the new &lt;code>discoverySelectors&lt;/code> option to &lt;a href="/v1.12/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig">MeshConfig&lt;/a>, which is an array of Kubernetes &lt;a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#resources-that-support-set-based-requirements">selectors&lt;/a>. The exact type is &lt;code>[]LabelSelector&lt;/code>, as defined &lt;a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#resources-that-support-set-based-requirements">here&lt;/a>, allowing both simple selectors and set-based selectors. These selectors apply to labels on namespaces.&lt;/p>
&lt;p>You can configure each label selector for expressing a variety of use cases, including but not limited to:&lt;/p>
&lt;ul>
&lt;li>Arbitrary label names/values, for example, all namespaces with label &lt;code>istio-discovery=enabled&lt;/code>&lt;/li>
&lt;li>A list of namespace labels using set-based selectors which carries OR semantics, for example, all namespaces with label &lt;code>istio-discovery=enabled&lt;/code> OR &lt;code>region=us-east1&lt;/code>&lt;/li>
&lt;li>Inclusion and/or exclusion of namespaces, for example, all namespaces with label istio-discovery=enabled AND label key &lt;code>app&lt;/code> equal to &lt;code>helloworld&lt;/code>&lt;/li>
&lt;/ul>
&lt;p>Note: &lt;code>discoverySelectors&lt;/code> is not a security boundary. Istiod will continue to have access to all namespaces even when you have configured your &lt;code>discoverySelectors&lt;/code>.&lt;/p>
&lt;h2 id="discovery-selectors-in-action">Discovery Selectors in Action&lt;/h2>
&lt;p>Assuming you know which namespaces to include as part of the service mesh, as a mesh administrator, you can configure &lt;code>discoverySelectors&lt;/code> at installation time or post-installation by adding your desired discovery selectors to Istio’s MeshConfig resource. For example, you can configure Istio to discover only the namespaces that have the label &lt;code>istio-discovery=enabled&lt;/code>.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Using our examples earlier, let’s label the &lt;code>default&lt;/code> namespace with label &lt;code>istio-discovery=enabled&lt;/code>.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl label namespace default istio-discovery=enabled
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Use &lt;code>istioctl&lt;/code> to apply the yaml with &lt;code>discoverySelectors&lt;/code> to update your Istio installation. Note, to avoid any impact to your stable environment, we recommend that you use a different revision for your Istio installation:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl install --skip-confirmation -f - &amp;lt;&amp;lt;EOF
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
namespace: istio-system
spec:
# You may override parts of meshconfig by uncommenting the following lines.
meshConfig:
discoverySelectors:
- matchLabels:
istio-discovery: enabled
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Display the endpoint configuration for the &lt;code>sleep&lt;/code> deployment:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:16.08212147134303%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/discovery-selectors/endpoints-with-discovery-selectors.png" title="Endpoints for Sleep Deployment With Discovery Selectors">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/discovery-selectors/endpoints-with-discovery-selectors.png" alt="Endpoints for Sleep Deployment With Discovery Selectors" />
&lt;/a>
&lt;/div>
&lt;figcaption>Endpoints for Sleep Deployment With Discovery Selectors&lt;/figcaption>
&lt;/figure>
&lt;p>Note this time the &lt;code>httpbin&lt;/code> service in the &lt;code>ns-x&lt;/code> namespace is NOT in the list of discovered endpoints, along with many other services that are not in the default namespace. If you display routes (or cluster or listeners) information for the &lt;code>sleep&lt;/code> deployment, you will also notice much less configuration is returned:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:26.88356164383562%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/discovery-selectors/routes-with-discovery-selectors.png" title="Routes for Sleep Deployment With Discovery Selectors">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/discovery-selectors/routes-with-discovery-selectors.png" alt="Routes for Sleep Deployment With Discovery Selectors" />
&lt;/a>
&lt;/div>
&lt;figcaption>Routes for Sleep Deployment With Discovery Selectors&lt;/figcaption>
&lt;/figure>&lt;/li>
&lt;/ol>
&lt;p>You can use &lt;code>matchLabels&lt;/code> to configure multiple labels with AND semantics or use &lt;code>matchLabels&lt;/code> sets to configure OR semantics among multiple labels. Whether you deploy services or pods to namespaces with different sets of labels or multiple application teams in your organization use different labeling conventions, &lt;code>discoverySelectors&lt;/code> provides the flexibility you need. Furthermore, you could use &lt;code>matchLabels&lt;/code> and &lt;code>matchExpressions&lt;/code> together per our &lt;a href="https://github.com/istio/api/blob/master/mesh/v1alpha1/config.proto#L792">documentation&lt;/a>. Refer to the &lt;a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors">Kubernetes selector docs&lt;/a> for additional detail on selector semantics.&lt;/p>
&lt;h2 id="discovery-selectors-vs-sidecar-resource">Discovery Selectors vs Sidecar Resource&lt;/h2>
&lt;p>The &lt;code>discoverySelectors&lt;/code> configuration enables users to dynamically restrict the set of namespaces that are part of the mesh. A &lt;a href="/v1.12/docs/reference/config/networking/sidecar/">Sidecar&lt;/a> resource also controls the visibility of sidecar configurations and what gets pushed to the sidecar proxy. What are the differences between them?&lt;/p>
&lt;ul>
&lt;li>The &lt;code>discoverySelectors&lt;/code> configuration declares what Istio control plane watches and processes. Without &lt;code>discoverySelectors&lt;/code> configuration, the Istio control plane watches and processes all namespaces/services/endpoints/pods in the cluster regardless of the sidecar resources you have.&lt;/li>
&lt;li>&lt;code>discoverySelectors&lt;/code> is configured globally for the mesh by the mesh administrators. While Sidecar resources can also be configured for the mesh globally by the mesh administrators in the MeshConfig root namespace, they are commonly configured by service owners for their namespaces.&lt;/li>
&lt;/ul>
&lt;p>You can use &lt;code>discoverySelectors&lt;/code> with Sidecar resources. You can use &lt;code>discoverySelectors&lt;/code> to configure at the mesh-wide level what namespaces the Istio control plane should watch and process. For these namespaces in the Istio service mesh, you can create Sidecar resources globally or per namespace to further control what gets pushed to the sidecar proxies. Let us add &lt;code>Bookinfo&lt;/code> services to the &lt;code>ns-y&lt;/code> namespace in the mesh as shown in the diagram below. &lt;code>discoverySelectors&lt;/code> enables us to define the &lt;code>default&lt;/code> and &lt;code>ns-y&lt;/code> namespaces are part of the mesh. How can we configure the &lt;code>sleep&lt;/code> service not to see anything other than the &lt;code>default&lt;/code> namespace? Adding a Sidecar resource for the default namespace, we can effectively configure the &lt;code>sleep&lt;/code> sidecar to only have visibility to the clusters/routes/listeners/endpoints associated with its current namespace plus any other required namespaces.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:83.06010928961749%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/discovery-selectors/discovery-selectors-vs-sidecar.png" title="Discovery Selectors vs Sidecar Resource">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/discovery-selectors/discovery-selectors-vs-sidecar.png" alt="Discovery Selectors vs Sidecar Resource" />
&lt;/a>
&lt;/div>
&lt;figcaption>Discovery Selectors vs Sidecar Resource&lt;/figcaption>
&lt;/figure>
&lt;h2 id="wrapping-up">Wrapping up&lt;/h2>
&lt;p>Discovery selectors are powerful configurations to tune the Istio control plane to only watch and process specific namespaces. If you don&amp;rsquo;t want all namespaces in your Kubernetes cluster to be part of the service mesh or you have multiple Istio service meshes within your Kubernetes cluster, we highly recommend that you explore this configuration and reach out to us for feedback on our &lt;a href="https://istio.slack.com">Istio slack&lt;/a> or GitHub.&lt;/p></description><pubDate>Fri, 30 Apr 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/discovery-selectors/</link><author>Lin Sun (Solo.io), Christian Posta (Solo.io), Harvey Xia (Solo.io)</author><guid isPermaLink="true">/v1.12/blog/2021/discovery-selectors/</guid><category>discoveryselectors</category><category>Istio</category><category>namespaces</category><category>sidecar</category></item><item><title>Upcoming networking changes in Istio 1.10</title><description>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>While Kubernetes networking is customizable, a typical pod&amp;rsquo;s network will look like this:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/upcoming-networking-changes/pod.svg" title="A pod&amp;#39;s network">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/upcoming-networking-changes/pod.svg" alt="A pod&amp;#39;s network" />
&lt;/a>
&lt;/div>
&lt;figcaption>A pod&amp;#39;s network&lt;/figcaption>
&lt;/figure>
&lt;p>An application may choose to bind to either the loopback interface &lt;code>lo&lt;/code> (typically binding to &lt;code>127.0.0.1&lt;/code>), or the pods network interface &lt;code>eth0&lt;/code> (typically to the pod&amp;rsquo;s IP), or both (typically binding to &lt;code>0.0.0.0&lt;/code>).&lt;/p>
&lt;p>Binding to &lt;code>lo&lt;/code> allows calls such as &lt;code>curl localhost&lt;/code> to work from within the pod.
Binding to &lt;code>eth0&lt;/code> allows calls to the pod from other pods.&lt;/p>
&lt;p>Typically, an application will bind to both.
However, applications which have internal logic, such as an admin interface may choose to bind to only &lt;code>lo&lt;/code> to avoid access from other pods.
Additionally, some applications, typically stateful applications, choose to bind only to &lt;code>eth0&lt;/code>.&lt;/p>
&lt;h2 id="current-behavior">Current behavior&lt;/h2>
&lt;p>In Istio prior to release 1.10, the Envoy proxy, running in the same pod as the application, binds to the &lt;code>eth0&lt;/code> interface and redirects all inbound traffic to the &lt;code>lo&lt;/code> interface.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/upcoming-networking-changes/current.svg" title="A pod&amp;#39;s network with Istio today">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/upcoming-networking-changes/current.svg" alt="A pod&amp;#39;s network with Istio today" />
&lt;/a>
&lt;/div>
&lt;figcaption>A pod&amp;#39;s network with Istio today&lt;/figcaption>
&lt;/figure>
&lt;p>This has two important side effects that cause the behavior to differ from standard Kubernetes:&lt;/p>
&lt;ul>
&lt;li>Applications binding only to &lt;code>lo&lt;/code> will receive traffic from other pods, when otherwise this is not allowed.&lt;/li>
&lt;li>Applications binding only to &lt;code>eth0&lt;/code> will not receive traffic.&lt;/li>
&lt;/ul>
&lt;p>Applications that bind to both interfaces (which is typical) will not be impacted.&lt;/p>
&lt;h2 id="future-behavior">Future behavior&lt;/h2>
&lt;p>Starting with Istio 1.10, the networking behavior is changed to align with the standard behavior present in Kubernetes.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/upcoming-networking-changes/planned.svg" title="A pod&amp;#39;s network with Istio in the future">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/upcoming-networking-changes/planned.svg" alt="A pod&amp;#39;s network with Istio in the future" />
&lt;/a>
&lt;/div>
&lt;figcaption>A pod&amp;#39;s network with Istio in the future&lt;/figcaption>
&lt;/figure>
&lt;p>Here we can see that the proxy no longer redirects the traffic to the &lt;code>lo&lt;/code> interface, but instead forwards it to the application on &lt;code>eth0&lt;/code>.
As a result, the standard behavior of Kubernetes is retained, but we still get all the benefits of Istio.
This change allows Istio to get closer to its goal of being a drop-in transparent proxy that works with existing workloads with &lt;a href="/v1.12/blog/2021/zero-config-istio/">zero configuration&lt;/a>.
Additionally, it avoids unintended exposure of applications binding only to &lt;code>lo&lt;/code>.&lt;/p>
&lt;h2 id="am-i-impacted">Am I impacted?&lt;/h2>
&lt;p>For new users, this change should only be an improvement.
However, if you are an existing user, you may have come to depend on the old behavior, intentionally or accidentally.&lt;/p>
&lt;p>To help detect these situations, we have added a check to find pods that will be impacted.
You can run the &lt;code>istioctl experimental precheck&lt;/code> command to get a report of any pods binding to &lt;code>lo&lt;/code> on a port exposed in a &lt;code>Service&lt;/code>.
This command is available in Istio 1.10+.
&lt;strong>Without action, these ports will no longer be accessible upon upgrade.&lt;/strong>&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl experimental precheck
Error [IST0143] (Pod echo-local-849647c5bd-g9wxf.default) Port 443 is exposed in a Service but listens on localhost. It will not be exposed to other pods.
Error [IST0143] (Pod echo-local-849647c5bd-g9wxf.default) Port 7070 is exposed in a Service but listens on localhost. It will not be exposed to other pods.
Error: Issues found when checking the cluster. Istio may not be safe to install or upgrade.
See https://istio.io/latest/docs/reference/config/analysis for more information about causes and resolutions.
&lt;/code>&lt;/pre>
&lt;h3 id="migration">Migration&lt;/h3>
&lt;p>If you are currently binding to &lt;code>lo&lt;/code>, you have a few options:&lt;/p>
&lt;ul>
&lt;li>Switch your application to bind to all interfaces (&lt;code>0.0.0.0&lt;/code> or &lt;code>::&lt;/code>).&lt;/li>
&lt;li>&lt;p>Explicitly configure the port using the &lt;a href="/v1.12/docs/reference/config/networking/sidecar/#IstioIngressListener">&lt;code>Sidecar&lt;/code> ingress configuration&lt;/a> to send to &lt;code>lo&lt;/code>, preserving the old behavior.&lt;/p>
&lt;p>For example, to configure request to be sent to &lt;code>localhost&lt;/code> for the &lt;code>ratings&lt;/code> application:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1beta1
kind: Sidecar
metadata:
name: ratings
spec:
workloadSelector:
labels:
app: ratings
ingress:
- port:
number: 8080
protocol: HTTP
name: http
defaultEndpoint: 127.0.0.1:8080
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Disable the change entirely with the &lt;code>PILOT_ENABLE_INBOUND_PASSTHROUGH=false&lt;/code> environment variable in Istiod, to enable the same behavior as prior to Istio 1.10. This option will be removed in the future.&lt;/p>&lt;/li>
&lt;/ul></description><pubDate>Thu, 15 Apr 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/upcoming-networking-changes/</link><author>John Howard (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/upcoming-networking-changes/</guid></item><item><title>Istio and Envoy WebAssembly Extensibility, One Year On</title><description>
&lt;p>One year ago today, in the 1.5 release, we introduced &lt;a href="/v1.12/blog/2020/wasm-announce/">WebAssembly-based extensibility&lt;/a> to Istio.
Over the course of the year, the Istio, Envoy, and Proxy-Wasm communities have continued our joint efforts to make WebAssembly (Wasm)
extensibility stable, reliable, and easy to adopt. Let&amp;rsquo;s walk through the updates to Wasm support through the Istio 1.9 release,
and our plans for the future.&lt;/p>
&lt;h2 id="webassembly-support-merged-in-upstream-envoy">WebAssembly support merged in upstream Envoy&lt;/h2>
&lt;p>After adding experimental support for Wasm and the WebAssembly for Proxies (Proxy-Wasm) ABI to Istio&amp;rsquo;s fork of Envoy, we collected some great feedback from our community of early adopters. This, combined with the experience gained from developing core Istio Wasm extensions, helped us mature and stabilize the runtime.
These improvements unblocked merging Wasm support directly into Envoy upstream in October 2020, allowing it to become part of all official Envoy releases.
This was a significant milestone, since it indicates that:&lt;/p>
&lt;ul>
&lt;li>The runtime is ready for wider adoption.&lt;/li>
&lt;li>The programming ABI/API, extension configuration API, and runtime behavior, are becoming stable.&lt;/li>
&lt;li>You can expect a larger community of adoption and support moving forward.&lt;/li>
&lt;/ul>
&lt;h2 id="wasm-extensions-ecosystem-repository">&lt;code>wasm-extensions&lt;/code> Ecosystem Repository&lt;/h2>
&lt;p>As an early adopter of the Envoy Wasm runtime, the Istio Extensions and Telemetry working group gained a lot of experience in developing extensions. We built several first-class extensions, including &lt;a href="/v1.12/docs/reference/config/proxy_extensions/metadata_exchange/">metadata exchange&lt;/a>, &lt;a href="/v1.12/docs/reference/config/proxy_extensions/stats/">Prometheus stats&lt;/a>, and &lt;a href="/v1.12/docs/reference/config/proxy_extensions/attributegen/">attribute generation&lt;/a>.
In order to share our learning more broadly, we created a &lt;a href="https://github.com/istio-ecosystem/wasm-extensions">&lt;code>wasm-extensions&lt;/code> repository&lt;/a> in the &lt;code>istio-ecosystem&lt;/code> organization. This repository serves two purposes:&lt;/p>
&lt;ul>
&lt;li>It provides canonical example extensions, covering several highly demanded features (such as &lt;a href="https://github.com/istio-ecosystem/wasm-extensions/tree/master/extensions/basic_auth">basic authentication&lt;/a>).&lt;/li>
&lt;li>It provides a guide for Wasm extension development, testing, and release. The guide is based on the same build tool chains and test frameworks that are used, maintained and tested by the Istio extensibility team.&lt;/li>
&lt;/ul>
&lt;p>The guide currently covers &lt;a href="https://github.com/istio-ecosystem/wasm-extensions/blob/master/doc/write-a-wasm-extension-with-cpp.md">WebAssembly extension development&lt;/a>
and &lt;a href="https://github.com/istio-ecosystem/wasm-extensions/blob/master/doc/write-cpp-unit-test.md">unit testing&lt;/a> with C++,
as well as &lt;a href="https://github.com/istio-ecosystem/wasm-extensions/blob/master/doc/write-integration-test.md">integration testing&lt;/a> with a Go test framework,
which simulates a real runtime by running a Wasm module with the Istio proxy binary.
In the future, we will also add several more canonical extensions, such as an integration with Open Policy Agent, and header manipulation based on JWT tokens.&lt;/p>
&lt;h2 id="wasm-module-distribution-via-the-istio-agent">Wasm module distribution via the Istio Agent&lt;/h2>
&lt;p>Prior to Istio 1.9, &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/base.proto#config-core-v3-remotedatasource">Envoy remote data sources&lt;/a> were needed to distribute remote Wasm modules to the proxy.
&lt;a href="https://gist.github.com/bianpengyuan/8377898190e8052ffa36e88a16911910">In this example&lt;/a>,
you can see two &lt;code>EnvoyFilter&lt;/code> resources are defined: one to add a remote fetch Envoy cluster, and the other one to inject a Wasm filter into the HTTP filter chain.
This method has a drawback: if remote fetch fails, either due to bad configuration or transient error, Envoy will be stuck with the bad configuration.
If a Wasm extension is configured as &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/wasm/v3/wasm.proto#extensions-wasm-v3-pluginconfig">fail closed&lt;/a>, a bad remote fetch will stop Envoy from serving.
To fix this issue, &lt;a href="https://github.com/envoyproxy/envoy/issues/9447">a fundamental change&lt;/a> is needed to the Envoy xDS protocol to make it allow asynchronous xDS responses.&lt;/p>
&lt;p>Istio 1.9 provides a reliable distribution mechanism out of the box by leveraging the xDS proxy inside istio-agent and Envoy&amp;rsquo;s &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/overview/extension">Extension Configuration Discovery Service&lt;/a> (ECDS).&lt;/p>
&lt;p>istio-agent intercepts the extension config resource update from istiod, reads the remote fetch hint from it, downloads the Wasm module, and rewrites the ECDS configuration with the path of the downloaded Wasm module.
If the download fails, istio-agent will reject the ECDS update and prevent a bad configuration reaching Envoy. For more detail, please see &lt;a href="/v1.12/docs/ops/configuration/extensibility/wasm-module-distribution/">our docs on Wasm module distribution&lt;/a>.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/wasm-progress/architecture-istio-agent-downloading-wasm-module.svg" title="Remote Wasm module fetch flow">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/wasm-progress/architecture-istio-agent-downloading-wasm-module.svg" alt="Remote Wasm module fetch flow" />
&lt;/a>
&lt;/div>
&lt;figcaption>Remote Wasm module fetch flow&lt;/figcaption>
&lt;/figure>
&lt;h2 id="istio-wasm-sig-and-future-work">Istio Wasm SIG and Future Work&lt;/h2>
&lt;p>Although we have made a lot of progress on Wasm extensibility, there are still many aspects of the project that remain to be completed. In order to consolidate the efforts from various parties and better tackle the challenges ahead, we have formed an &lt;a href="https://discuss.istio.io/t/introducing-wasm-sig/9930">Istio WebAssembly SIG&lt;/a>, with aim of providing a standard and reliable way for Istio to consume Wasm extensions. Here are some of the things we are working on:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>A first-class extension API&lt;/strong>: Currently Wasm extensions needs to be injected via Istio&amp;rsquo;s &lt;code>EnvoyFilter&lt;/code> API. A first-class extension API will make using Wasm with Istio easier, and we expect this to be introduced in Istio 1.10.&lt;/li>
&lt;li>&lt;strong>Distribution artifacts interoperability&lt;/strong>: Built on top of Solo.io’s &lt;a href="https://www.solo.io/blog/announcing-the-webassembly-wasm-oci-image-spec/">WebAssembly OCI image spec effort&lt;/a>, a standard Wasm artifacts format will make it easy to build, pull, publish, and execute.&lt;/li>
&lt;li>&lt;strong>Container Storage Interface (CSI) based artifacts distribution&lt;/strong>: Using istio-agent to distribute modules is easy for adoption, but may not be efficient as each proxy will keep a copy of the Wasm module. As a more efficient solution, with &lt;a href="https://kubernetes-csi.github.io/docs/ephemeral-local-volumes.html">Ephemeral CSI&lt;/a>, a DaemonSet will be provided which could configure storage for pods. Working similarly to a CNI plugin, a CSI driver would fetch the Wasm module out-of-band from the xDS flow and mount it inside the &lt;code>rootfs&lt;/code> when the pod starts up.&lt;/li>
&lt;/ul>
&lt;p>If you would like to join us, the group will meet every other week Tuesdays at 2PM PT. You can find the meeting on the &lt;a href="https://github.com/istio/community/blob/master/WORKING-GROUPS.md#working-group-meetings">Istio working group calendar&lt;/a>.&lt;/p>
&lt;p>We look forward to seeing how you will use Wasm to extend Istio!&lt;/p></description><pubDate>Fri, 05 Mar 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/wasm-progress/</link><author>Pengyuan Bian (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/wasm-progress/</guid><category>wasm</category><category>extensibility</category><category>WebAssembly</category></item><item><title>Migrate pre-Istio 1.4 Alpha security policy to the current APIs</title><description>
&lt;p>In versions of Istio prior to 1.4, security policy was configured using &lt;code>v1alpha1&lt;/code> APIs (&lt;code>MeshPolicy&lt;/code>, &lt;code>Policy&lt;/code>, &lt;code>ClusterRbacConfig&lt;/code>, &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code>). After consulting with our early adopters, we made &lt;a href="/v1.12/blog/2019/v1beta1-authorization-policy/">major improvements to the policy system&lt;/a> and released &lt;code>v1beta1&lt;/code> APIs along with Istio 1.4. These refreshed APIs (&lt;code>PeerAuthentication&lt;/code>, &lt;code>RequestAuthentication&lt;/code> and &lt;code>AuthorizationPolicy&lt;/code>) helped standardize how we define policy targets in Istio, helped users understand where policies were applied, and cut the number of configuration objects required.&lt;/p>
&lt;p>The old APIs were deprecated in Istio 1.4. Two releases after the &lt;code>v1beta1&lt;/code> APIs were introduced, Istio 1.6 removed support for the &lt;code>v1alpha1&lt;/code> APIs.&lt;/p>
&lt;p>If you are using a version of Istio prior to 1.6 and you want to upgrade, you will have to migrate your alpha security policy objects to the beta API. This tutorial will help you make that move.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">If you adopted Istio after version 1.6, or you&amp;rsquo;re not using &lt;code>v1alpha1&lt;/code> security APIs, you can stop reading.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;h2 id="overview">Overview&lt;/h2>
&lt;p>Your control plane must first be upgraded to a version that supports the &lt;code>v1beta1&lt;/code> security policy.&lt;/p>
&lt;p>It is recommended to first upgrade to Istio 1.5 as a transitive version, because it is the only version that supports both
&lt;code>v1alpha1&lt;/code> and &lt;code>v1beta1&lt;/code> security policies. You will complete the security policy migration in Istio 1.5, remove the
&lt;code>v1alpha1&lt;/code> security policy, and then continue to upgrade to later Istio versions. For a given workload, the &lt;code>v1beta1&lt;/code>
version will take precedence over the &lt;code>v1alpha1&lt;/code> version.&lt;/p>
&lt;p>Alternatively, if you want to do a skip-level upgrade directly from Istio 1.4 to 1.6 or later, you should use the
&lt;a href="/v1.12/docs/setup/upgrade/canary/">canary upgrade&lt;/a> method to install a new Istio version as a separate control plane, and
gradually migrate your workloads to the new control plane completing the security policy migration at the same time.&lt;/p>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Skip-level upgrades are not supported by Istio and there might be other issues in this process. Istio 1.6 does not support
the &lt;code>v1alpha1&lt;/code> security policy, and if you do not migrate your old policies before the upgrade, you are essentially removing
all your security policies.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>In either case, it is recommended to migrate using namespace granularity: for each namespace, find all the
&lt;code>v1alpha1&lt;/code> policies that have an effect on workloads in the namespace and migrate all the policies to &lt;code>v1beta1&lt;/code>
at the same time. This allows a safer migration as you can make sure everything is working as expected,
and then move forward to the next namespace.&lt;/p>
&lt;h2 id="major-differences">Major differences&lt;/h2>
&lt;p>Before starting the migration, read through the &lt;code>v1beta1&lt;/code> &lt;a href="/v1.12/docs/concepts/security/#authentication">authentication&lt;/a>
and &lt;a href="/v1.12/docs/concepts/security/#authorization">authorization&lt;/a> documentation to understand the &lt;code>v1beta1&lt;/code> policy.&lt;/p>
&lt;p>You should examine all of your existing &lt;code>v1alpha1&lt;/code> security policies, find out what fields are used and which policies
need migration, compare the findings with the major differences listed below and confirm there are no blocking issues
(e.g., using an alpha feature that is no longer supported in beta):&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Major Differences&lt;/th>
&lt;th>&lt;code>v1alpha1&lt;/code>&lt;/th>
&lt;th>&lt;code>v1beta1&lt;/code>&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>API stability&lt;/td>
&lt;td>not backward compatible&lt;/td>
&lt;td>backward compatible&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>mTLS&lt;/td>
&lt;td>&lt;code>MeshPolicy&lt;/code> and &lt;code>Policy&lt;/code>&lt;/td>
&lt;td>&lt;code>PeerAuthentication&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>JWT&lt;/td>
&lt;td>&lt;code>MeshPolicy&lt;/code> and &lt;code>Policy&lt;/code>&lt;/td>
&lt;td>&lt;code>RequestAuthentication&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Authorization&lt;/td>
&lt;td>&lt;code>ClusterRbacConfig&lt;/code>, &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code>&lt;/td>
&lt;td>&lt;code>AuthorizationPolicy&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Policy target&lt;/td>
&lt;td>service name based&lt;/td>
&lt;td>workload selector based&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Port number&lt;/td>
&lt;td>service ports&lt;/td>
&lt;td>workload ports&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>Although &lt;code>RequestAuthentication&lt;/code> in &lt;code>v1beta1&lt;/code> security policy is similar to the &lt;code>v1alpha1&lt;/code> JWT policy, there is a notable
semantics change. The &lt;code>v1alpha1&lt;/code> JWT policy needs to be migrated to two &lt;code>v1beta1&lt;/code> resources: &lt;code>RequestAuthentication&lt;/code> and
&lt;code>AuthorizationPolicy&lt;/code>. This will change the JWT deny message due to the use of &lt;code>AuthorizationPolicy&lt;/code>. In the alpha version,
the HTTP code 401 is returned with the body &lt;code>Origin authentication failed&lt;/code>. In the beta version, the HTTP code 403 is
returned with the body &lt;code>RBAC: access denied&lt;/code>.&lt;/p>
&lt;p>The &lt;code>v1alpha1&lt;/code> JWT policy &lt;a href="https://istio.io/v1.4/docs/reference/config/security/istio.authentication.v1alpha1/#Jwt-TriggerRule">&lt;code>triggerRule&lt;/code> field&lt;/a>
is replaced by the &lt;code>AuthorizationPolicy&lt;/code> with the exception that the &lt;a href="https://istio.io/v1.4/docs/reference/config/security/istio.authentication.v1alpha1/#StringMatch">&lt;code>regex&lt;/code> field&lt;/a>
is no longer supported.&lt;/p>
&lt;h2 id="migration-flow">Migration flow&lt;/h2>
&lt;p>This section describes in detail how to migrate a &lt;code>v1alpha1&lt;/code> security policy.&lt;/p>
&lt;h3 id="step-1-find-related-policies">Step 1: Find related policies&lt;/h3>
&lt;p>For each namespace, find all &lt;code>v1alpha1&lt;/code> security policies that have an effect on workloads in the namespace. The result
could include:&lt;/p>
&lt;ul>
&lt;li>a single &lt;code>MeshPolicy&lt;/code> that applies to all services in the mesh;&lt;/li>
&lt;li>a single namespace-level &lt;code>Policy&lt;/code> that applies to all workloads in the namespace;&lt;/li>
&lt;li>multiple service-level &lt;code>Policy&lt;/code> objects that apply to the selected services in the namespace;&lt;/li>
&lt;li>a single &lt;code>ClusterRbacConfig&lt;/code> that enables the RBAC on the whole namespace or some services in the namespace;&lt;/li>
&lt;li>multiple namespace-level &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code> objects that apply to all services in the namespace;&lt;/li>
&lt;li>multiple service-level &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code> objects that apply to the selected services in the namespace;&lt;/li>
&lt;/ul>
&lt;h3 id="step-2-convert-service-name-to-workload-selector">Step 2: Convert service name to workload selector&lt;/h3>
&lt;p>The &lt;code>v1alpha1&lt;/code> policy selects targets using their service name. You should refer to the corresponding service definition to decide
the workload selector that should be used in the &lt;code>v1beta1&lt;/code> policy.&lt;/p>
&lt;p>A single &lt;code>v1alpha1&lt;/code> policy may include multiple services. It will need to be migrated to multiple &lt;code>v1beta1&lt;/code> policies
because the &lt;code>v1beta1&lt;/code> policy currently only supports at most one workload selector per policy.&lt;/p>
&lt;p>Also note the &lt;code>v1alpha1&lt;/code> policy uses service port but the &lt;code>v1beta1&lt;/code> policy uses the workload port. This means the port number might be
different in the migrated &lt;code>v1beta1&lt;/code> policy.&lt;/p>
&lt;h3 id="step-3-migrate-authentication-policy">Step 3: Migrate authentication policy&lt;/h3>
&lt;p>For each &lt;code>v1alpha1&lt;/code> authentication policy, migrate with the following rules:&lt;/p>
&lt;ol>
&lt;li>&lt;p>If the whole namespace is enabled with mTLS or JWT, create the &lt;code>PeerAuthentication&lt;/code>, &lt;code>RequestAuthentication&lt;/code> and
&lt;code>AuthorizationPolicy&lt;/code> without a workload selector for the whole namespace. Fill out the policy based on the
semantics of the corresponding &lt;code>MeshPolicy&lt;/code> or &lt;code>Policy&lt;/code> for the namespace.&lt;/p>&lt;/li>
&lt;li>&lt;p>If a workload is enabled with mTLS or JWT, create the &lt;code>PeerAuthentication&lt;/code>, &lt;code>RequestAuthentication&lt;/code> and
&lt;code>AuthorizationPolicy&lt;/code> with a corresponding workload selector for the workload. Fill out the policy based on the
semantics of the corresponding &lt;code>MeshPolicy&lt;/code> or &lt;code>Policy&lt;/code> for the workload.&lt;/p>&lt;/li>
&lt;li>&lt;p>For mTLS related configuration, use &lt;code>STRICT&lt;/code> mode if the alpha policy is using &lt;code>STRICT&lt;/code>, or use &lt;code>PERMISSIVE&lt;/code> in all other cases.&lt;/p>&lt;/li>
&lt;li>&lt;p>For JWT related configuration, refer to the &lt;a href="/v1.12/docs/tasks/security/authentication/authn-policy/#end-user-authentication">&lt;code>end-user authentication&lt;/code> documentation&lt;/a>
to learn how to migrate to &lt;code>RequestAuthentication&lt;/code> and &lt;code>AuthorizationPolicy&lt;/code>.&lt;/p>&lt;/li>
&lt;/ol>
&lt;p>A &lt;a href="https://github.com/istio-ecosystem/security-policy-migrate">security policy migration tool&lt;/a> is provided to
automatically migrate authentication policy automatically. Please refer to the tool&amp;rsquo;s README for its usage.&lt;/p>
&lt;h3 id="step-4-migrate-rbac-policy">Step 4: Migrate RBAC policy&lt;/h3>
&lt;p>For each &lt;code>v1alpha1&lt;/code> RBAC policy, migrate with the following rules:&lt;/p>
&lt;ol>
&lt;li>&lt;p>If the whole namespace is enabled with RBAC, create an &lt;code>AuthorizationPolicy&lt;/code> without a workload selector for the whole
namespace. Add an empty rule so that it will deny all requests to the namespace by default.&lt;/p>&lt;/li>
&lt;li>&lt;p>If a workload is enabled with RBAC, create an &lt;code>AuthorizationPolicy&lt;/code> with a corresponding workload selector for the workload.
Add rules based on the semantics of the corresponding &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code> for the workload.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h3 id="step-5-verify-migrated-policy">Step 5: Verify migrated policy&lt;/h3>
&lt;ol>
&lt;li>&lt;p>Double check the migrated &lt;code>v1beta1&lt;/code> policies: make sure there are no policies with duplicate names, the namespace
is specified correctly and all &lt;code>v1alpha1&lt;/code> policies for the given namespace are migrated.&lt;/p>&lt;/li>
&lt;li>&lt;p>Dry-run the &lt;code>v1beta1&lt;/code> policy with the command &lt;code>kubectl apply --dry-run=server -f beta-policy.yaml&lt;/code> to make sure it
is valid.&lt;/p>&lt;/li>
&lt;li>&lt;p>Apply the &lt;code>v1beta1&lt;/code> policy to the given namespace and closely monitor the effect. Make sure to test both allow and
deny scenarios if JWT or authorization are used.&lt;/p>&lt;/li>
&lt;li>&lt;p>Migrate the next namespace. Only remove the &lt;code>v1alpha1&lt;/code> policy after completing migration for all namespaces successfully.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h2 id="example">Example&lt;/h2>
&lt;h3 id="v1alpha1-policy">&lt;code>v1alpha1&lt;/code> policy&lt;/h3>
&lt;p>This section gives a full example showing the migration for namespace &lt;code>foo&lt;/code>. Assume the namespace &lt;code>foo&lt;/code> has the following
&lt;code>v1alpha1&lt;/code> policies that affect the workloads in it:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' ># A MeshPolicy that enables mTLS globally, including the whole foo namespace
apiVersion: &amp;#34;authentication.istio.io/v1alpha1&amp;#34;
kind: &amp;#34;MeshPolicy&amp;#34;
metadata:
name: &amp;#34;default&amp;#34;
spec:
peers:
- mtls: {}
---
# A Policy that enables mTLS permissive mode and enables JWT for the httpbin service on port 8000
apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
name: httpbin
namespace: foo
spec:
targets:
- name: httpbin
ports:
- number: 8000
peers:
- mtls:
mode: PERMISSIVE
origins:
- jwt:
issuer: testing@example.com
jwksUri: https://www.example.com/jwks.json
triggerRules:
- includedPaths:
- prefix: /admin/
excludedPaths:
- exact: /admin/status
principalBinding: USE_ORIGIN
---
# A ClusterRbacConfig that enables RBAC globally, including the foo namespace
apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ClusterRbacConfig
metadata:
name: default
spec:
mode: &amp;#39;ON&amp;#39;
---
# A ServiceRole that enables RBAC for the httpbin service
apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRole
metadata:
name: httpbin
namespace: foo
spec:
rules:
- services: [&amp;#34;httpbin.foo.svc.cluster.local&amp;#34;]
methods: [&amp;#34;GET&amp;#34;]
---
# A ServiceRoleBinding for the above ServiceRole
apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRoleBinding
metadata:
name: httpbin
namespace: foo
spec:
subjects:
- user: cluster.local/ns/foo/sa/sleep
roleRef:
kind: ServiceRole
name: httpbin
&lt;/code>&lt;/pre>
&lt;h3 id="httpbin-service">&lt;code>httpbin&lt;/code> service&lt;/h3>
&lt;p>The &lt;code>httpbin&lt;/code> service has the following definition:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Service
metadata:
name: httpbin
namespace: foo
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin
&lt;/code>&lt;/pre>
&lt;p>This means the service name &lt;code>httpbin&lt;/code> should be replaced by the workload selector &lt;code>app: httpbin&lt;/code>, and the service port 8000
should be replaced by the workload port 80.&lt;/p>
&lt;h3 id="v1beta1-authentication-policy">&lt;code>v1beta1&lt;/code> authentication policy&lt;/h3>
&lt;p>The migrated &lt;code>v1beta1&lt;/code> policies for the &lt;code>v1alpha1&lt;/code> authentication policies in &lt;code>foo&lt;/code> namespace are listed below:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' ># A PeerAuthentication that enables mTLS for the foo namespace, migrated from the MeshPolicy
# Alternatively the MeshPolicy could also be migrated to a PeerAuthentication at mesh level
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: foo
spec:
mtls:
mode: STRICT
---
# A PeerAuthentication that enables mTLS for the httpbin workload, migrated from the Policy
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: httpbin
namespace: foo
spec:
selector:
matchLabels:
app: httpbin
# port level mtls set for the workload port 80 corresponding to the service port 8000
portLevelMtls:
80:
mode: PERMISSIVE
--
# A RequestAuthentication that enables JWT for the httpbin workload, migrated from the Policy
apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
name: httpbin
namespace: foo
spec:
selector:
matchLabels:
app: httpbin
jwtRules:
- issuer: testing@example.com
jwksUri: https://www.example.com/jwks.json
---
# An AuthorizationPolicy that enforces to require JWT validation for the httpbin workload, migrated from the Policy
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin-jwt
namespace: foo
spec:
# Use DENY action to explicitly deny requests without JWT token
action: DENY
selector:
matchLabels:
app: httpbin
rules:
- from:
- source:
# This makes sure requests without JWT token will be denied
notRequestPrincipals: [&amp;#34;*&amp;#34;]
to:
- operation:
# This should be the workload port 80, not the service port 8000
ports: [&amp;#34;80&amp;#34;]
# The path and notPath is converted from the trigger rule in the Policy
paths: [&amp;#34;/admin/*&amp;#34;]
notPaths: [&amp;#34;/admin/status&amp;#34;]
&lt;/code>&lt;/pre>
&lt;h3 id="v1beta1-authorization-policy">&lt;code>v1beta1&lt;/code> authorization policy&lt;/h3>
&lt;p>The migrated &lt;code>v1beta1&lt;/code> policies for the &lt;code>v1alpha1&lt;/code> RBAC policies in &lt;code>foo&lt;/code> namespace are listed below:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' ># An AuthorizationPolicy that denies by default, migrated from the ClusterRbacConfig
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: default
namespace: foo
spec:
# An empty rule that allows nothing
{}
---
# An AuthorizationPolicy that enforces to authorization for the httpbin workload, migrated from the ServiceRole and ServiceRoleBinding
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin
namespace: foo
spec:
selector:
matchLabels:
app: httpbin
version: v1
action: ALLOW
rules:
- from:
- source:
principals: [&amp;#34;cluster.local/ns/foo/sa/sleep&amp;#34;]
to:
- operation:
methods: [&amp;#34;GET&amp;#34;]
&lt;/code>&lt;/pre>
&lt;h2 id="finish-the-upgrade">Finish the upgrade&lt;/h2>
&lt;p>Congratulations; having reached this point, you should only have &lt;code>v1beta1&lt;/code> policy objects, and you will be able to continue upgrading Istio to 1.6 and beyond.&lt;/p></description><pubDate>Wed, 03 Mar 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/migrate-alpha-policy/</link><author>Yangmin Zhu (Google), Craig Box (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/migrate-alpha-policy/</guid><category>security</category><category>policy</category><category>migrate</category><category>alpha</category><category>beta</category><category>deprecate</category><category>peer</category><category>jwt</category><category>authorization</category></item><item><title>Zero Configuration Istio</title><description>
&lt;p>When a new user encounters Istio for the first time, they are sometimes overwhelmed by the vast feature
set it exposes. Unfortunately, this can give the impression that Istio is needlessly complex
and not fit for small teams or clusters.&lt;/p>
&lt;p>One great part about Istio, however, is that it aims to bring as much value to users out of the box without any configuration at all.
This enables users to get most of the benefits of Istio with minimal efforts. For some users with simple requirements, custom configurations
may never be required at all. Others will be able to incrementally add Istio configurations once they are more comfortable and as they need them, such as to add
ingress routing, fine-tune networking settings, or lock down security policies.&lt;/p>
&lt;h2 id="getting-started">Getting started&lt;/h2>
&lt;p>To get started, check out our &lt;a href="/v1.12/docs/setup/getting-started/">getting started&lt;/a> documentation, where you will learn how to install Istio.
If you are already familiar, you can simply run &lt;code>istioctl install&lt;/code>.&lt;/p>
&lt;p>Next, we will explore all the benefits Istio provides us, without any configuration or changes to application code.&lt;/p>
&lt;h2 id="security">Security&lt;/h2>
&lt;p>Istio automatically enables &lt;a href="/v1.12/docs/concepts/security/#mutual-tls-authentication">mutual TLS&lt;/a> for traffic between pods in the mesh.
This enables applications to forgo complex TLS configuration and certificate management, and offload all transport layer security to the sidecar.&lt;/p>
&lt;p>Once comfortable with automatic TLS, you may choose to &lt;a href="/v1.12/docs/tasks/security/authentication/mtls-migration/">allow only mTLS traffic&lt;/a>, or configure custom &lt;a href="/v1.12/docs/tasks/security/authorization/">authorization policies&lt;/a> for your needs.&lt;/p>
&lt;h2 id="observability">Observability&lt;/h2>
&lt;p>Istio automatically generates detailed telemetry for all service communications within a mesh.
This telemetry provides observability of service behavior, empowering operators to troubleshoot, maintain, and optimize their applications – without imposing any additional burdens on service developers.
Through Istio, operators gain a thorough understanding of how monitored services are interacting, both with other services and with the Istio components themselves.&lt;/p>
&lt;p>All of this functionality is added by Istio without any configuration. &lt;a href="/v1.12/docs/ops/integrations/">Integrations&lt;/a> with tools such as Prometheus, Grafana, Jaeger, Zipkin, and Kiali are also available.&lt;/p>
&lt;p>For more information about the observability Istio provides, check out the &lt;a href="/v1.12/docs/concepts/observability/">observability overview&lt;/a>.&lt;/p>
&lt;h2 id="traffic-management">Traffic Management&lt;/h2>
&lt;p>While Kubernetes provides a lot of networking functionality, such as service discovery and DNS, this is done at Layer 4, which can have unintended inefficiencies.
For example, in a simple HTTP application sending traffic to a service with 3 replicas, we can see unbalanced load:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl http://echo/{0..5} -s | grep Hostname
Hostname=echo-cb96f8d94-2ssll
Hostname=echo-cb96f8d94-2ssll
Hostname=echo-cb96f8d94-2ssll
Hostname=echo-cb96f8d94-2ssll
Hostname=echo-cb96f8d94-2ssll
Hostname=echo-cb96f8d94-2ssll
$ curl http://echo/{0..5} -s | grep Hostname
Hostname=echo-cb96f8d94-879sn
Hostname=echo-cb96f8d94-879sn
Hostname=echo-cb96f8d94-879sn
Hostname=echo-cb96f8d94-879sn
Hostname=echo-cb96f8d94-879sn
Hostname=echo-cb96f8d94-879sn
&lt;/code>&lt;/pre>
&lt;p>The problem here is Kubernetes will determine the backend to send to when the connection is established, and all future requests on the same connection will be sent to the same backend.
In our example here, our first 5 requests are all sent to &lt;code>echo-cb96f8d94-2ssll&lt;/code>, while our next set (using a new connection) are all sent to &lt;code>echo-cb96f8d94-879sn&lt;/code>.
Our third instance never receives any requests.&lt;/p>
&lt;p>With Istio, HTTP traffic (including HTTP/2 and gRPC) is automatically detected, and our services will automatically be load balanced per &lt;em>request&lt;/em>, rather than per &lt;em>connection&lt;/em>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl http://echo/{0..5} -s | grep Hostname
Hostname=echo-cb96f8d94-wf4xk
Hostname=echo-cb96f8d94-rpfqz
Hostname=echo-cb96f8d94-cgmxr
Hostname=echo-cb96f8d94-wf4xk
Hostname=echo-cb96f8d94-rpfqz
Hostname=echo-cb96f8d94-cgmxr
&lt;/code>&lt;/pre>
&lt;p>Here we can see our requests are &lt;a href="/v1.12/docs/concepts/traffic-management/#load-balancing-options">round-robin&lt;/a> load balanced between all backends.&lt;/p>
&lt;p>In addition to these better defaults, Istio offers customization of a &lt;a href="/v1.12/docs/concepts/traffic-management/">variety of traffic management settings&lt;/a>, including timeouts, retries, and much more.&lt;/p></description><pubDate>Thu, 25 Feb 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/zero-config-istio/</link><author>John Howard (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/zero-config-istio/</guid></item><item><title>IstioCon 2021: Schedule Is Live!</title><description>&lt;p>&lt;a href="https://events.istio.io/istiocon-2021/">IstioCon 2021&lt;/a> is a week-long, community-led, virtual conference starting on February 22.
This event provides an opportunity to hear the lessons learned from companies like Atlassian, Airbnb, FICO, eBay, T-Mobile and
Salesforce running Istio in production, hands-on experiences from the Istio community, and will feature maintainers from across
the Istio ecosystem.&lt;/p>
&lt;p>You can now find the &lt;a href="https://events.istio.io/istiocon-2021/schedule/">full schedule of events&lt;/a> which includes a series of
&lt;a href="https://events.istio.io/istiocon-2021/schedule/english/">English&lt;/a> sessions and
&lt;a href="https://events.istio.io/istiocon-2021/schedule/chinese/">Chinese&lt;/a> sessions.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/istiocon-2021-program/istiocon-program.png" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/istiocon-2021-program/istiocon-program.png" alt="IstioCon logo" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
&lt;p>By attending the conference, you’ll connect with community members from across the globe. Each day you will find keynotes,
technical talks, lightning talks, panel discussions, workshops and roadmap sessions led by diverse speakers representing the
Istio community. You can also connect with other Istio and Open Source ecosystem community members through social hour events
that include activities on the social platform &lt;a href="https://events.istio.io/istiocon-2021/networking/">Gather.town&lt;/a>, a live cartoonist,
virtual swag bags, raffles, live music and games.&lt;/p>
&lt;p>Don’t miss it! &lt;a href="https://events.istio.io/istiocon-2021/">Registration&lt;/a> is free. We look forward to seeing you at the first IstioCon!&lt;/p></description><pubDate>Tue, 16 Feb 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/istiocon-2021-program/</link><author>Istio Steering Committee</author><guid isPermaLink="true">/v1.12/blog/2021/istiocon-2021-program/</guid><category>IstioCon</category><category>Istio</category><category>conference</category></item><item><title>Better External Authorization</title><description>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>Istio&amp;rsquo;s authorization policy provides access control for services in the mesh. It is fast, powerful and a widely used
feature. We have made continuous improvements to make policy more flexible since its first release in Istio 1.4, including
the &lt;a href="/v1.12/docs/tasks/security/authorization/authz-deny/">&lt;code>DENY&lt;/code> action&lt;/a>, &lt;a href="/v1.12/docs/tasks/security/authorization/authz-deny/">exclusion semantics&lt;/a>,
&lt;a href="/v1.12/docs/tasks/security/authorization/authz-ingress/">&lt;code>X-Forwarded-For&lt;/code> header support&lt;/a>, &lt;a href="/v1.12/docs/tasks/security/authorization/authz-jwt/">nested JWT claim support&lt;/a>
and more. These features improve the flexibility of the authorization policy, but there are still many use cases that
cannot be supported with this model, for example:&lt;/p>
&lt;ul>
&lt;li>&lt;p>You have your own in-house authorization system that cannot be easily migrated to, or cannot be easily replaced by, the
authorization policy.&lt;/p>&lt;/li>
&lt;li>&lt;p>You want to integrate with a 3rd-party solution (e.g. &lt;a href="https://www.openpolicyagent.org/docs/latest/envoy-introduction/">Open Policy Agent&lt;/a>
or &lt;a href="https://github.com/oauth2-proxy/oauth2-proxy">&lt;code>oauth2&lt;/code> proxy&lt;/a>) which may require use of the
&lt;a href="/v1.12/docs/reference/config/networking/envoy-filter/">low-level Envoy configuration APIs&lt;/a> in Istio, or may not be possible
at all.&lt;/p>&lt;/li>
&lt;li>&lt;p>Authorization policy lacks necessary semantics for your use case.&lt;/p>&lt;/li>
&lt;/ul>
&lt;h2 id="solution">Solution&lt;/h2>
&lt;p>In Istio 1.9, we have implemented extensibility into authorization policy by introducing a &lt;a href="/v1.12/docs/reference/config/security/authorization-policy/#AuthorizationPolicy-Action">&lt;code>CUSTOM&lt;/code> action&lt;/a>,
which allows you to delegate the access control decision to an external authorization service.&lt;/p>
&lt;p>The &lt;code>CUSTOM&lt;/code> action allows you to integrate Istio with an external authorization system that implements its own custom
authorization logic. The following diagram shows the high level architecture of this integration:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:48.573838213459645%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2021/better-external-authz/external_authz.svg" title="External Authorization Architecture">
&lt;img class="element-to-stretch" src="/v1.12/blog/2021/better-external-authz/external_authz.svg" alt="External Authorization Architecture" />
&lt;/a>
&lt;/div>
&lt;figcaption>External Authorization Architecture&lt;/figcaption>
&lt;/figure>
&lt;p>At configuration time, the mesh admin configures an authorization policy with a &lt;code>CUSTOM&lt;/code> action to enable the
external authorization on a proxy (either gateway or sidecar). The admin should verify the external auth service is up
and running.&lt;/p>
&lt;p>At runtime,&lt;/p>
&lt;ol>
&lt;li>&lt;p>A request is intercepted by the proxy, and the proxy will send check requests to the external auth service, as
configured by the user in the authorization policy.&lt;/p>&lt;/li>
&lt;li>&lt;p>The external auth service will make the decision whether to allow it or not.&lt;/p>&lt;/li>
&lt;li>&lt;p>If allowed, the request will continue and will be enforced by any local authorization defined by &lt;code>ALLOW&lt;/code>/&lt;code>DENY&lt;/code> action.&lt;/p>&lt;/li>
&lt;li>&lt;p>If denied, the request will be rejected immediately.&lt;/p>&lt;/li>
&lt;/ol>
&lt;p>Let&amp;rsquo;s look at an example authorization policy with the &lt;code>CUSTOM&lt;/code> action:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: ext-authz
namespace: istio-system
spec:
# The selector applies to the ingress gateway in the istio-system namespace.
selector:
matchLabels:
app: istio-ingressgateway
# The action &amp;#34;CUSTOM&amp;#34; delegates the access control to an external authorizer, this is different from
# the ALLOW/DENY action that enforces the access control right inside the proxy.
action: CUSTOM
# The provider specifies the name of the external authorizer defined in the meshconfig, which tells where and how to
# talk to the external auth service. We will cover this more later.
provider:
name: &amp;#34;my-ext-authz-service&amp;#34;
# The rule specifies that the access control is triggered only if the request path has the prefix &amp;#34;/admin/&amp;#34;.
# This allows you to easily enable or disable the external authorization based on the requests, avoiding the external
# check request if it is not needed.
rules:
- to:
- operation:
paths: [&amp;#34;/admin/*&amp;#34;]
&lt;/code>&lt;/pre>
&lt;p>It refers to a provider called &lt;code>my-ext-authz-service&lt;/code> which is defined in the mesh config:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >extensionProviders:
# The name &amp;#34;my-ext-authz-service&amp;#34; is referred to by the authorization policy in its provider field.
- name: &amp;#34;my-ext-authz-service&amp;#34;
# The &amp;#34;envoyExtAuthzGrpc&amp;#34; field specifies the type of the external authorization service is implemented by the Envoy
# ext-authz filter gRPC API. The other supported type is the Envoy ext-authz filter HTTP API.
# See more in https://www.envoyproxy.io/docs/envoy/v1.16.2/intro/arch_overview/security/ext_authz_filter.
envoyExtAuthzGrpc:
# The service and port specifies the address of the external auth service, &amp;#34;ext-authz.istio-system.svc.cluster.local&amp;#34;
# means the service is deployed in the mesh. It can also be defined out of the mesh or even inside the pod as a separate
# container.
service: &amp;#34;ext-authz.istio-system.svc.cluster.local&amp;#34;
port: 9000
&lt;/code>&lt;/pre>
&lt;p>The authorization policy of &lt;a href="/v1.12/docs/reference/config/security/authorization-policy/#AuthorizationPolicy-Action">&lt;code>CUSTOM&lt;/code> action&lt;/a>
enables the external authorization in runtime, it could be configured to trigger the external authorization conditionally
based on the request using the same rule that you have already been using with other actions.&lt;/p>
&lt;p>The external authorization service is currently defined in the &lt;a href="/v1.12/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig-ExtensionProvider">&lt;code>meshconfig&lt;/code> API&lt;/a>
and referred to by its name. It could be deployed in the mesh with or without proxy. If with the proxy, you could
further use &lt;code>PeerAuthentication&lt;/code> to enable mTLS between the proxy and your external authorization service.&lt;/p>
&lt;p>The &lt;code>CUSTOM&lt;/code> action is currently in the &lt;strong>experimental stage&lt;/strong>; the API might change in a non-backward compatible way based on user feedback.
The authorization policy rules currently don&amp;rsquo;t support authentication fields (e.g. source principal or JWT claim) when used with the
&lt;code>CUSTOM&lt;/code> action. Only one provider is allowed for a given workload, but you can still use different providers on different workloads.&lt;/p>
&lt;p>For more information, please see the &lt;a href="https://docs.google.com/document/d/1V4mCQCw7mlGp0zSQQXYoBdbKMDnkPOjeyUb85U07iSI/edit#">Better External Authorization design doc&lt;/a>.&lt;/p>
&lt;h2 id="example-with-opa">Example with OPA&lt;/h2>
&lt;p>In this section, we will demonstrate using the &lt;code>CUSTOM&lt;/code> action with the Open Policy Agent as the external authorizer on
the ingress gateway. We will conditionally enable the external authorization on all paths except &lt;code>/ip&lt;/code>.&lt;/p>
&lt;p>You can also refer to the &lt;a href="/v1.12/docs/tasks/security/authorization/authz-custom/">external authorization task&lt;/a> for a more
basic introduction that uses a sample &lt;code>ext-authz&lt;/code> server.&lt;/p>
&lt;h3 id="create-the-example-opa-policy">Create the example OPA policy&lt;/h3>
&lt;p>Run the following command create an OPA policy that allows the request if the prefix of the path is matched with the
claim &amp;ldquo;path&amp;rdquo; (base64 encoded) in the JWT token:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;gt; policy.rego &amp;lt;&amp;lt;EOF
package envoy.authz
import input.attributes.request.http as http_request
default allow = false
token = {&amp;#34;valid&amp;#34;: valid, &amp;#34;payload&amp;#34;: payload} {
[_, encoded] := split(http_request.headers.authorization, &amp;#34; &amp;#34;)
[valid, _, payload] := io.jwt.decode_verify(encoded, {&amp;#34;secret&amp;#34;: &amp;#34;secret&amp;#34;})
}
allow {
is_token_valid
action_allowed
}
is_token_valid {
token.valid
now := time.now_ns() / 1000000000
token.payload.nbf &amp;lt;= now
now &amp;lt; token.payload.exp
}
action_allowed {
startswith(http_request.path, base64url.decode(token.payload.path))
}
EOF
$ kubectl create secret generic opa-policy --from-file policy.rego
&lt;/code>&lt;/pre>
&lt;h3 id="deploy-httpbin-and-opa">Deploy httpbin and OPA&lt;/h3>
&lt;p>Enable the sidecar injection:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl label ns default istio-injection=enabled
&lt;/code>&lt;/pre>
&lt;p>Run the following command to deploy the example application httpbin and OPA. The OPA could be deployed either as a
separate container in the httpbin pod or completely in a separate pod:&lt;/p>
&lt;div id="tabset-blog-2021-better-external-authz-1" role="tablist" class="tabset ">
&lt;div class="tab-strip" data-category-name="opa-deploy" >&lt;button aria-selected="true" data-category-value="opa-same"
aria-controls="tabset-blog-2021-better-external-authz-1-0-panel" id="tabset-blog-2021-better-external-authz-1-0-tab" role="tab">&lt;span>Deploy OPA in the same pod&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="opa-standalone"
aria-controls="tabset-blog-2021-better-external-authz-1-1-panel" id="tabset-blog-2021-better-external-authz-1-1-tab" role="tab">&lt;span>Deploy OPA in a separate pod&lt;/span>
&lt;/button>&lt;/div>
&lt;div class="tab-content">&lt;div id="tabset-blog-2021-better-external-authz-1-0-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2021-better-external-authz-1-0-tab">
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: v1
kind: Service
metadata:
name: httpbin-with-opa
labels:
app: httpbin-with-opa
service: httpbin-with-opa
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin-with-opa
---
# Define the service entry for the local OPA service on port 9191.
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: local-opa-grpc
spec:
hosts:
- &amp;#34;local-opa-grpc.local&amp;#34;
endpoints:
- address: &amp;#34;127.0.0.1&amp;#34;
ports:
- name: grpc
number: 9191
protocol: GRPC
resolution: STATIC
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: httpbin-with-opa
labels:
app: httpbin-with-opa
spec:
replicas: 1
selector:
matchLabels:
app: httpbin-with-opa
template:
metadata:
labels:
app: httpbin-with-opa
spec:
containers:
- image: docker.io/kennethreitz/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 80
- name: opa
image: openpolicyagent/opa:latest-envoy
securityContext:
runAsUser: 1111
volumeMounts:
- readOnly: true
mountPath: /policy
name: opa-policy
args:
- &amp;#34;run&amp;#34;
- &amp;#34;--server&amp;#34;
- &amp;#34;--addr=localhost:8181&amp;#34;
- &amp;#34;--diagnostic-addr=0.0.0.0:8282&amp;#34;
- &amp;#34;--set=plugins.envoy_ext_authz_grpc.addr=:9191&amp;#34;
- &amp;#34;--set=plugins.envoy_ext_authz_grpc.query=data.envoy.authz.allow&amp;#34;
- &amp;#34;--set=decision_logs.console=true&amp;#34;
- &amp;#34;--ignore=.*&amp;#34;
- &amp;#34;/policy/policy.rego&amp;#34;
livenessProbe:
httpGet:
path: /health?plugins
scheme: HTTP
port: 8282
initialDelaySeconds: 5
periodSeconds: 5
readinessProbe:
httpGet:
path: /health?plugins
scheme: HTTP
port: 8282
initialDelaySeconds: 5
periodSeconds: 5
volumes:
- name: proxy-config
configMap:
name: proxy-config
- name: opa-policy
secret:
secretName: opa-policy
EOF
&lt;/code>&lt;/pre>
&lt;/div>&lt;div hidden id="tabset-blog-2021-better-external-authz-1-1-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2021-better-external-authz-1-1-tab">
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: v1
kind: Service
metadata:
name: opa
labels:
app: opa
spec:
ports:
- name: grpc
port: 9191
targetPort: 9191
selector:
app: opa
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: opa
labels:
app: opa
spec:
replicas: 1
selector:
matchLabels:
app: opa
template:
metadata:
labels:
app: opa
spec:
containers:
- name: opa
image: openpolicyagent/opa:latest-envoy
securityContext:
runAsUser: 1111
volumeMounts:
- readOnly: true
mountPath: /policy
name: opa-policy
args:
- &amp;#34;run&amp;#34;
- &amp;#34;--server&amp;#34;
- &amp;#34;--addr=localhost:8181&amp;#34;
- &amp;#34;--diagnostic-addr=0.0.0.0:8282&amp;#34;
- &amp;#34;--set=plugins.envoy_ext_authz_grpc.addr=:9191&amp;#34;
- &amp;#34;--set=plugins.envoy_ext_authz_grpc.query=data.envoy.authz.allow&amp;#34;
- &amp;#34;--set=decision_logs.console=true&amp;#34;
- &amp;#34;--ignore=.*&amp;#34;
- &amp;#34;/policy/policy.rego&amp;#34;
ports:
- containerPort: 9191
livenessProbe:
httpGet:
path: /health?plugins
scheme: HTTP
port: 8282
initialDelaySeconds: 5
periodSeconds: 5
readinessProbe:
httpGet:
path: /health?plugins
scheme: HTTP
port: 8282
initialDelaySeconds: 5
periodSeconds: 5
volumes:
- name: proxy-config
configMap:
name: proxy-config
- name: opa-policy
secret:
secretName: opa-policy
EOF
&lt;/code>&lt;/pre>
&lt;p>Deploy the httpbin as well:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/httpbin/httpbin.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/httpbin/httpbin.yaml@
&lt;/code>&lt;/pre>&lt;/div>
&lt;/div>&lt;/div>
&lt;/div>
&lt;h3 id="define-external-authorizer">Define external authorizer&lt;/h3>
&lt;p>Run the following command to edit the &lt;code>meshconfig&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl edit configmap istio -n istio-system
&lt;/code>&lt;/pre>
&lt;p>Add the following &lt;code>extensionProviders&lt;/code> to the &lt;code>meshconfig&lt;/code>:&lt;/p>
&lt;div id="tabset-blog-2021-better-external-authz-2" role="tablist" class="tabset ">
&lt;div class="tab-strip" data-category-name="opa-deploy" >&lt;button aria-selected="true" data-category-value="opa-same"
aria-controls="tabset-blog-2021-better-external-authz-2-0-panel" id="tabset-blog-2021-better-external-authz-2-0-tab" role="tab">&lt;span>Deploy OPA in the same pod&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="opa-standalone"
aria-controls="tabset-blog-2021-better-external-authz-2-1-panel" id="tabset-blog-2021-better-external-authz-2-1-tab" role="tab">&lt;span>Deploy OPA in a separate pod&lt;/span>
&lt;/button>&lt;/div>
&lt;div class="tab-content">&lt;div id="tabset-blog-2021-better-external-authz-2-0-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2021-better-external-authz-2-0-tab">
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
data:
mesh: |-
# Add the following contents:
extensionProviders:
- name: &amp;#34;opa.local&amp;#34;
envoyExtAuthzGrpc:
service: &amp;#34;local-opa-grpc.local&amp;#34;
port: &amp;#34;9191&amp;#34;
&lt;/code>&lt;/pre>
&lt;/div>&lt;div hidden id="tabset-blog-2021-better-external-authz-2-1-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2021-better-external-authz-2-1-tab">
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
data:
mesh: |-
# Add the following contents:
extensionProviders:
- name: &amp;#34;opa.default&amp;#34;
envoyExtAuthzGrpc:
service: &amp;#34;opa.default.svc.cluster.local&amp;#34;
port: &amp;#34;9191&amp;#34;
&lt;/code>&lt;/pre>
&lt;/div>&lt;/div>
&lt;/div>
&lt;h3 id="create-an-authorizationpolicy-with-a-custom-action">Create an AuthorizationPolicy with a CUSTOM action&lt;/h3>
&lt;p>Run the following command to create the authorization policy that enables the external authorization on all paths
except &lt;code>/ip&lt;/code>:&lt;/p>
&lt;div id="tabset-blog-2021-better-external-authz-3" role="tablist" class="tabset ">
&lt;div class="tab-strip" data-category-name="opa-deploy" >&lt;button aria-selected="true" data-category-value="opa-same"
aria-controls="tabset-blog-2021-better-external-authz-3-0-panel" id="tabset-blog-2021-better-external-authz-3-0-tab" role="tab">&lt;span>Deploy OPA in the same pod&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="opa-standalone"
aria-controls="tabset-blog-2021-better-external-authz-3-1-panel" id="tabset-blog-2021-better-external-authz-3-1-tab" role="tab">&lt;span>Deploy OPA in a separate pod&lt;/span>
&lt;/button>&lt;/div>
&lt;div class="tab-content">&lt;div id="tabset-blog-2021-better-external-authz-3-0-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2021-better-external-authz-3-0-tab">
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin-opa
spec:
selector:
matchLabels:
app: httpbin-with-opa
action: CUSTOM
provider:
name: &amp;#34;opa.local&amp;#34;
rules:
- to:
- operation:
notPaths: [&amp;#34;/ip&amp;#34;]
EOF
&lt;/code>&lt;/pre>
&lt;/div>&lt;div hidden id="tabset-blog-2021-better-external-authz-3-1-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2021-better-external-authz-3-1-tab">
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin-opa
spec:
selector:
matchLabels:
app: httpbin
action: CUSTOM
provider:
name: &amp;#34;opa.default&amp;#34;
rules:
- to:
- operation:
notPaths: [&amp;#34;/ip&amp;#34;]
EOF
&lt;/code>&lt;/pre>
&lt;/div>&lt;/div>
&lt;/div>
&lt;h3 id="test-the-opa-policy">Test the OPA policy&lt;/h3>
&lt;ol>
&lt;li>&lt;p>Create a client pod to send the request:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/sleep/sleep.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/sleep/sleep.yaml@
$ export SLEEP_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Use a test JWT token signed by the OPA:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export TOKEN_PATH_HEADERS=&amp;#34;eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJwYXRoIjoiTDJobFlXUmxjbk09IiwibmJmIjoxNTAwMDAwMDAwLCJleHAiOjE5MDAwMDAwMDB9.9yl8LcZdq-5UpNLm0Hn0nnoBHXXAnK4e8RSl9vn6l98&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>The test JWT token has the following claims:&lt;/p>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;path&amp;#34;: &amp;#34;L2hlYWRlcnM=&amp;#34;,
&amp;#34;nbf&amp;#34;: 1500000000,
&amp;#34;exp&amp;#34;: 1900000000
}
&lt;/code>&lt;/pre>
&lt;p>The &lt;code>path&lt;/code> claim has value &lt;code>L2hlYWRlcnM=&lt;/code> which is the base64 encode of &lt;code>/headers&lt;/code>.&lt;/p>&lt;/li>
&lt;li>&lt;p>Send a request to path &lt;code>/headers&lt;/code> without a token. This should be rejected with 403 because there is no JWT token:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec ${SLEEP_POD} -c sleep -- curl http://httpbin-with-opa:8000/headers -s -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34;
403
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Send a request to path &lt;code>/get&lt;/code> with a valid token. This should be rejected with 403 because the path &lt;code>/get&lt;/code> is not matched with the token &lt;code>/headers&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec ${SLEEP_POD} -c sleep -- curl http://httpbin-with-opa:8000/get -H &amp;#34;Authorization: Bearer $TOKEN_PATH_HEADERS&amp;#34; -s -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34;
403
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Send a request to path &lt;code>/headers&lt;/code> with valid token. This should be allowed with 200 because the path is matched with the token:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec ${SLEEP_POD} -c sleep -- curl http://httpbin-with-opa:8000/headers -H &amp;#34;Authorization: Bearer $TOKEN_PATH_HEADERS&amp;#34; -s -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34;
200
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Send request to path &lt;code>/ip&lt;/code> without token. This should be allowed with 200 because the path &lt;code>/ip&lt;/code> is excluded from
authorization:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec ${SLEEP_POD} -c sleep -- curl http://httpbin-with-opa:8000/ip -s -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34;
200
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Check the proxy and OPA logs to confirm the result.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>In Istio 1.9, the &lt;code>CUSTOM&lt;/code> action in the authorization policy allows you to easily integrate Istio with any external
authorization system with the following benefits:&lt;/p>
&lt;ul>
&lt;li>&lt;p>First-class support in the authorization policy API&lt;/p>&lt;/li>
&lt;li>&lt;p>Ease of usage: define the external authorizer simply with a URL and enable with the authorization policy, no more
hassle with the &lt;code>EnvoyFilter&lt;/code> API&lt;/p>&lt;/li>
&lt;li>&lt;p>Conditional triggering, allowing improved performance&lt;/p>&lt;/li>
&lt;li>&lt;p>Support for various deployment type of the external authorizer:&lt;/p>
&lt;ul>
&lt;li>&lt;p>A normal service and pod with or without proxy&lt;/p>&lt;/li>
&lt;li>&lt;p>Inside the workload pod as a separate container&lt;/p>&lt;/li>
&lt;li>&lt;p>Outside the mesh&lt;/p>&lt;/li>
&lt;/ul>&lt;/li>
&lt;/ul>
&lt;p>We&amp;rsquo;re working to promote this feature to a more stable stage in following versions and welcome your feedback at
&lt;a href="https://discuss.istio.io/c/security/">discuss.istio.io&lt;/a>.&lt;/p>
&lt;h2 id="acknowledgements">Acknowledgements&lt;/h2>
&lt;p>Thanks to &lt;code>Craig Box&lt;/code>, &lt;code>Christian Posta&lt;/code> and &lt;code>Limin Wang&lt;/code> for reviewing drafts of this blog.&lt;/p></description><pubDate>Tue, 09 Feb 2021 00:00:00 +0000</pubDate><link>/v1.12/blog/2021/better-external-authz/</link><author>Yangmin Zhu (Google)</author><guid isPermaLink="true">/v1.12/blog/2021/better-external-authz/</guid><category>authorization</category><category>access control</category><category>opa</category><category>oauth2</category></item><item><title>Proxying legacy services using Istio egress gateways</title><description>
&lt;p>At &lt;a href="https://pan-net.cloud/aboutus">Deutsche Telekom Pan-Net&lt;/a>, we have embraced Istio as the umbrella to cover our services. Unfortunately, there are services which have not yet been migrated to Kubernetes, or cannot be.&lt;/p>
&lt;p>We can set Istio up as a proxy service for these upstream services. This allows us to benefit from capabilities like authorization/authentication, traceability and observability, even while legacy services stand as they are.&lt;/p>
&lt;p>At the end of this article there is a hands-on exercise where you can simulate the scenario. In the exercise, an upstream service hosted at &lt;a href="https://httpbin.org">https://httpbin.org&lt;/a> will be proxied by an Istio egress gateway.&lt;/p>
&lt;p>If you are familiar with Istio, one of the methods offered to connect to upstream services is through an &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/">egress gateway&lt;/a>.&lt;/p>
&lt;p>You can deploy one to control all the upstream traffic or you can deploy multiple in order to have fine-grained control and satisfy the &lt;a href="https://en.wikipedia.org/wiki/Single-responsibility_principle">single-responsibility principle&lt;/a> as this picture shows:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/proxying-legacy-services-using-egress-gateways/proxying-legacy-services-using-egress-gateways-overview.svg" title="Overview multiple Egress Gateways">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/proxying-legacy-services-using-egress-gateways/proxying-legacy-services-using-egress-gateways-overview.svg" alt="Overview multiple Egress Gateways" />
&lt;/a>
&lt;/div>
&lt;figcaption>Overview multiple Egress Gateways&lt;/figcaption>
&lt;/figure>
&lt;p>With this model, one egress gateway is in charge of exactly one upstream service.&lt;/p>
&lt;p>Although the Operator spec allows you to deploy multiple egress gateways, the manifest can become unmanageable:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
[...]
spec:
egressGateways:
- name: egressgateway-1
enabled: true
- name: egressgateway-2
enabled: true
[egressgateway-3, egressgateway-4, ...]
- name: egressgateway-N
enabled: true
[...]
&lt;/code>&lt;/pre>
&lt;p>As a benefit of decoupling egress getaways from the Operator manifest, you have enabled the possibility of setting up custom readiness probes to have both services (Gateway and upstream Service) aligned.&lt;/p>
&lt;p>You can also inject OPA as a sidecar into the pod to perform authorization with complex rules (&lt;a href="https://github.com/open-policy-agent/opa-envoy-plugin">OPA envoy plugin&lt;/a>).&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/proxying-legacy-services-using-egress-gateways/proxying-legacy-services-using-egress-gateways-authz.svg" title="Authorization with OPA and `healthcheck` to external">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/proxying-legacy-services-using-egress-gateways/proxying-legacy-services-using-egress-gateways-authz.svg" alt="Authorization with OPA and `healthcheck` to upstream service" />
&lt;/a>
&lt;/div>
&lt;figcaption>Authorization with OPA and `healthcheck` to external&lt;/figcaption>
&lt;/figure>
&lt;p>As you can see, your possibilities increase and Istio becomes very extensible.&lt;/p>
&lt;p>Let&amp;rsquo;s look at how you can implement this pattern.&lt;/p>
&lt;h2 id="solution">Solution&lt;/h2>
&lt;p>There are several ways to perform this task, but here you will find how to define multiple Operators and deploy the generated resources.&lt;/p>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Yes! &lt;code>Istio 1.8.0&lt;/code> introduced the possibility to have fine-grained control over the objects that Operator deploys. This gives you the opportunity to patch them as you wish. Exactly what you need to proxy legacy services using Istio egress gateways.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>In the following section you will deploy an egress gateway to connect to an upstream service: &lt;code>httpbin&lt;/code> (&lt;a href="https://httpbin.org/">https://httpbin.org/&lt;/a>)&lt;/p>
&lt;p>At the end, you will have:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/proxying-legacy-services-using-egress-gateways/proxying-legacy-services-using-egress-gateways-communication.svg" title="Communication">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/proxying-legacy-services-using-egress-gateways/proxying-legacy-services-using-egress-gateways-communication.svg" alt="Communication" />
&lt;/a>
&lt;/div>
&lt;figcaption>Communication&lt;/figcaption>
&lt;/figure>
&lt;h2 id="hands-on">Hands on&lt;/h2>
&lt;h3 id="prerequisites">Prerequisites&lt;/h3>
&lt;ul>
&lt;li>&lt;a href="https://kind.sigs.k8s.io/docs/user/quick-start/">kind&lt;/a> (Kubernetes-in-Docker - perfect for local development)&lt;/li>
&lt;li>&lt;a href="/v1.12/docs/setup/getting-started/#download">istioctl&lt;/a>&lt;/li>
&lt;/ul>
&lt;h4 id="kind">Kind&lt;/h4>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">If you use &lt;code>kind&lt;/code>, do not forget to set up &lt;code>service-account-issuer&lt;/code> and &lt;code>service-account-signing-key-file&lt;/code> as described below. Otherwise, Istio may not install correctly.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Save this as &lt;code>config.yaml&lt;/code>.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
kubeadmConfigPatches:
- |
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
metadata:
name: config
apiServer:
extraArgs:
&amp;#34;service-account-issuer&amp;#34;: &amp;#34;kubernetes.default.svc&amp;#34;
&amp;#34;service-account-signing-key-file&amp;#34;: &amp;#34;/etc/kubernetes/pki/sa.key&amp;#34;
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kind create cluster --name &amp;lt;my-cluster-name&amp;gt; --config config.yaml
&lt;/code>&lt;/pre>
&lt;p>Where &lt;code>&amp;lt;my-cluster-name&amp;gt;&lt;/code> is the name for the cluster.&lt;/p>
&lt;h4 id="istio-operator-with-istioctl">Istio Operator with Istioctl&lt;/h4>
&lt;p>Install the Operator&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl operator init --watchedNamespaces=istio-operator
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create ns istio-system
&lt;/code>&lt;/pre>
&lt;p>Save this as &lt;code>operator.yaml&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
name: istio-operator
namespace: istio-operator
spec:
profile: default
tag: 1.8.0
meshConfig:
accessLogFile: /dev/stdout
outboundTrafficPolicy:
mode: REGISTRY_ONLY
&lt;/code>&lt;/pre>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">&lt;code>outboundTrafficPolicy.mode: REGISTRY_ONLY&lt;/code> is used to block all external communications which are not specified by a &lt;code>ServiceEntry&lt;/code> resource.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f operator.yaml
&lt;/code>&lt;/pre>
&lt;h3 id="deploy-egress-gateway">Deploy Egress Gateway&lt;/h3>
&lt;p>The steps for this task assume:&lt;/p>
&lt;ul>
&lt;li>The service is installed under the namespace: &lt;code>httpbin&lt;/code>.&lt;/li>
&lt;li>The service name is: &lt;code>http-egress&lt;/code>.&lt;/li>
&lt;/ul>
&lt;p>Istio 1.8 introduced the possibility to apply overlay configuration, to give fine-grain control over the created resources.&lt;/p>
&lt;p>Save this as &lt;code>egress.yaml&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: empty
tag: 1.8.0
namespace: httpbin
components:
egressGateways:
- name: httpbin-egress
enabled: true
label:
app: istio-egressgateway
istio: egressgateway
custom-egress: httpbin-egress
k8s:
overlays:
- kind: Deployment
name: httpbin-egress
patches:
- path: spec.template.spec.containers[0].readinessProbe
value:
failureThreshold: 30
exec:
command:
- /bin/sh
- -c
- curl http://localhost:15021/healthz/ready &amp;amp;&amp;amp; curl https://httpbin.org/status/200
initialDelaySeconds: 1
periodSeconds: 2
successThreshold: 1
timeoutSeconds: 1
values:
gateways:
istio-egressgateway:
runAsRoot: true
&lt;/code>&lt;/pre>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Notice the block under &lt;code>overlays&lt;/code>. You are patching the default &lt;code>egressgateway&lt;/code> to deploy only that component with the new &lt;code>readinessProbe&lt;/code>.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Create the namespace where you will install the egress gateway:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create ns httpbin
&lt;/code>&lt;/pre>
&lt;p>As it is described in the &lt;a href="/v1.12/docs/setup/install/istioctl/#customize-kubernetes-settings">documentation&lt;/a>, you can deploy several Operator resources. However, they have to be pre-parsed and then applied to the cluster.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl manifest generate -f egress.yaml | kubectl apply -f -
&lt;/code>&lt;/pre>
&lt;h3 id="istio-configuration">Istio configuration&lt;/h3>
&lt;p>Now you will configure Istio to allow connections to the upstream service at &lt;a href="https://httpbin.org">https://httpbin.org&lt;/a>.&lt;/p>
&lt;h4 id="certificate-for-tls">Certificate for TLS&lt;/h4>
&lt;p>You need a certificate to make a secure connection from outside the cluster to your egress service.&lt;/p>
&lt;p>How to generate a certificate is explained in the &lt;a href="/v1.12/docs/tasks/traffic-management/ingress/secure-ingress/#generate-client-and-server-certificates-and-keys">Istio ingress documentation&lt;/a>.&lt;/p>
&lt;p>Create and apply one to be used at the end of this article to access the service from outside the cluster (&lt;code>&amp;lt;my-proxied-service-hostname&amp;gt;&lt;/code>):&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create -n istio-system secret tls &amp;lt;my-secret-name&amp;gt; --key=&amp;lt;key&amp;gt; --cert=&amp;lt;cert&amp;gt;
&lt;/code>&lt;/pre>
&lt;p>Where &lt;code>&amp;lt;my-secret-name&amp;gt;&lt;/code> is the name used later for the &lt;code>Gateway&lt;/code> resource. &lt;code>&amp;lt;key&amp;gt;&lt;/code> and &lt;code>&amp;lt;cert&amp;gt;&lt;/code> are the files for the certificate. &lt;code>&amp;lt;cert&amp;gt;&lt;/code>.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">You need to remember &lt;code>&amp;lt;my-proxied-service-hostname&amp;gt;&lt;/code>, &lt;code>&amp;lt;cert&amp;gt;&lt;/code> and &lt;code>&amp;lt;my-secret-name&amp;gt;&lt;/code> because you will use them later in the article.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;h4 id="ingress-gateway">Ingress Gateway&lt;/h4>
&lt;p>Create a &lt;code>Gateway&lt;/code> resource to operate ingress gateway to accept requests.&lt;/p>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Make sure that only one Gateway spec matches the hostname. Istio gets confused when there are multiple Gateway definitions covering the same hostname.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>An example:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: my-ingressgateway
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- &amp;#34;&amp;lt;my-proxied-service-hostname&amp;gt;&amp;#34;
port:
name: http
number: 80
protocol: HTTP
tls:
httpsRedirect: true
- port:
number: 443
name: https
protocol: https
hosts:
- &amp;#34;&amp;lt;my-proxied-service-hostname&amp;gt;&amp;#34;
tls:
mode: SIMPLE
credentialName: &amp;lt;my-secret-name&amp;gt;
&lt;/code>&lt;/pre>
&lt;p>Where &lt;code>&amp;lt;my-proxied-service-hostname&amp;gt;&lt;/code> is the hostname to access the service through the &lt;code>my-ingressgateway&lt;/code> and &lt;code>&amp;lt;my-secret-name&amp;gt;&lt;/code> is the secret which contains the certificate.&lt;/p>
&lt;h4 id="egress-gateway">Egress Gateway&lt;/h4>
&lt;p>Create another Gateway object, but this time to operate the egress gateway you have already installed:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: &amp;#34;httpbin-egress&amp;#34;
namespace: &amp;#34;httpbin&amp;#34;
spec:
selector:
istio: egressgateway
service.istio.io/canonical-name: &amp;#34;httpbin-egress&amp;#34;
servers:
- hosts:
- &amp;#34;&amp;lt;my-proxied-service-hostname&amp;gt;&amp;#34;
port:
number: 80
name: http
protocol: HTTP
&lt;/code>&lt;/pre>
&lt;p>Where &lt;code>&amp;lt;my-proxied-service-hostname&amp;gt;&lt;/code> is the hostname to access through the &lt;code>my-ingressgateway&lt;/code>.&lt;/p>
&lt;h4 id="virtual-service">Virtual Service&lt;/h4>
&lt;p>Create a &lt;code>VirtualService&lt;/code> for three use cases:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>Mesh&lt;/strong> gateway for service-to-service communications within the mesh&lt;/li>
&lt;li>&lt;strong>Ingress Gateway&lt;/strong> for the communication from outside the mesh&lt;/li>
&lt;li>&lt;strong>Egress Gateway&lt;/strong> for the communication to the upstream service&lt;/li>
&lt;/ul>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Mesh and Ingress Gateway will share the same specification. It will redirect the traffic to your egress gateway service.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: &amp;#34;httpbin-egress&amp;#34;
namespace: &amp;#34;httpbin&amp;#34;
spec:
hosts:
- &amp;#34;&amp;lt;my-proxied-service-hostname&amp;gt;&amp;#34;
gateways:
- mesh
- &amp;#34;istio-system/my-ingressgateway&amp;#34;
- &amp;#34;httpbin/httpbin-egress&amp;#34;
http:
- match:
- gateways:
- &amp;#34;istio-system/my-ingressgateway&amp;#34;
- mesh
uri:
prefix: &amp;#34;/&amp;#34;
route:
- destination:
host: &amp;#34;httpbin-egress.httpbin.svc.cluster.local&amp;#34;
port:
number: 80
- match:
- gateways:
- &amp;#34;httpbin/httpbin-egress&amp;#34;
uri:
prefix: &amp;#34;/&amp;#34;
route:
- destination:
host: &amp;#34;httpbin.org&amp;#34;
subset: &amp;#34;http-egress-subset&amp;#34;
port:
number: 443
&lt;/code>&lt;/pre>
&lt;p>Where &lt;code>&amp;lt;my-proxied-service-hostname&amp;gt;&lt;/code> is the hostname to access through the &lt;code>my-ingressgateway&lt;/code>.&lt;/p>
&lt;h4 id="service-entry">Service Entry&lt;/h4>
&lt;p>Create a &lt;code>ServiceEntry&lt;/code> to allow the communication to the upstream service:&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Notice that the port is configured for TLS protocol&lt;/div>
&lt;/aside>
&lt;/div>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: &amp;#34;httpbin-egress&amp;#34;
namespace: &amp;#34;httpbin&amp;#34;
spec:
hosts:
- &amp;#34;httpbin.org&amp;#34;
location: MESH_EXTERNAL
ports:
- number: 443
name: https
protocol: TLS
resolution: DNS
&lt;/code>&lt;/pre>
&lt;h4 id="destination-rule">Destination Rule&lt;/h4>
&lt;p>Create a &lt;code>DestinationRule&lt;/code> to allow TLS origination for egress traffic as explained in the &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-tls-origination/#tls-origination-for-egress-traffic">documentation&lt;/a>&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: &amp;#34;httpbin-egress&amp;#34;
namespace: &amp;#34;httpbin&amp;#34;
spec:
host: &amp;#34;httpbin.org&amp;#34;
subsets:
- name: &amp;#34;http-egress-subset&amp;#34;
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: SIMPLE
&lt;/code>&lt;/pre>
&lt;h4 id="peer-authentication">Peer Authentication&lt;/h4>
&lt;p>To secure the service-to-service, you need to enforce mTLS:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;security.istio.io/v1beta1&amp;#34;
kind: &amp;#34;PeerAuthentication&amp;#34;
metadata:
name: &amp;#34;httpbin-egress&amp;#34;
namespace: &amp;#34;httpbin&amp;#34;
spec:
mtls:
mode: STRICT
&lt;/code>&lt;/pre>
&lt;h3 id="test">Test&lt;/h3>
&lt;p>Verify that your objects were all specified correctly:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl analyze --all-namespaces
&lt;/code>&lt;/pre>
&lt;h4 id="external-access">External access&lt;/h4>
&lt;p>Test the egress gateway from outside the cluster forwarding the &lt;code>ingressgateway&lt;/code> service&amp;rsquo;s port and calling the service&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n istio-system port-forward svc/istio-ingressgateway 15443:443
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl -vvv -k -HHost:&amp;lt;my-proxied-service-hostname&amp;gt; --resolve &amp;#34;&amp;lt;my-proxied-service-hostname&amp;gt;:15443:127.0.0.1&amp;#34; --cacert &amp;lt;cert&amp;gt; &amp;#34;https://&amp;lt;my-proxied-service-hostname&amp;gt;:15443/status/200&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>Where &lt;code>&amp;lt;my-proxied-service-hostname&amp;gt;&lt;/code> is the hostname to access through the &lt;code>my-ingressgateway&lt;/code> and &lt;code>&amp;lt;cert&amp;gt;&lt;/code> is the certificate defined for the &lt;code>ingressgateway&lt;/code> object. This is due to &lt;code>tls.mode: SIMPLE&lt;/code> which &lt;a href="/v1.12/docs/tasks/traffic-management/ingress/secure-ingress/">does not terminate TLS&lt;/a>&lt;/p>
&lt;h4 id="service-to-service-access">Service-to-service access&lt;/h4>
&lt;p>Test the egress gateway from inside the cluster deploying the sleep service. This is useful when you design failover.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl label namespace httpbin istio-injection=enabled --overwrite
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -n httpbin -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/sleep/sleep.yaml
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n httpbin &amp;#34;$(kubectl get pod -n httpbin -l app=sleep -o jsonpath={.items..metadata.name})&amp;#34; -- curl -vvv http://&amp;lt;my-proxied-service-hostname&amp;gt;/status/200
&lt;/code>&lt;/pre>
&lt;p>Where &lt;code>&amp;lt;my-proxied-service-hostname&amp;gt;&lt;/code> is the hostname to access through the &lt;code>my-ingressgateway&lt;/code>.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Notice that &lt;code>http&lt;/code> (and not &lt;code>https&lt;/code>) is the protocol used for service-to-service communication. This is due to Istio handling the &lt;code>TLS&lt;/code> itself. Developers do not care anymore about certificates management. &lt;strong>Fancy!&lt;/strong>&lt;/div>
&lt;/aside>
&lt;/div>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Eat, Sleep, Rave, &lt;strong>REPEAT!&lt;/strong>&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Now it is time to create a second, third and fourth egress gateway pointing to other upstream services.&lt;/p>
&lt;h2 id="final-thoughts">Final thoughts&lt;/h2>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Is the juice worth the squeeze?&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Istio might seem complex to configure. But it is definitely worthwhile, due to the huge set of benefits it brings to your services (with an extra &lt;strong>Olé!&lt;/strong> for Kiali).&lt;/p>
&lt;p>The way Istio is developed allows us, with minimal effort, to satisfy uncommon requirements like the one presented in this article.&lt;/p>
&lt;p>To finish, I just wanted to point out that Istio, as a good cloud native technology, does not require a large team to maintain. For example, our current team is composed of 3 engineers.&lt;/p>
&lt;p>To discuss more about Istio and its possibilities, please contact one of us:&lt;/p>
&lt;ul>
&lt;li>&lt;a href="https://twitter.com/antonio_berben">Antonio Berben&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://www.linkedin.com/in/piotr-ciazynski">Piotr Ciążyński&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://www.linkedin.com/in/patlevic">Kristián Patlevič&lt;/a>&lt;/li>
&lt;/ul></description><pubDate>Wed, 16 Dec 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/proxying-legacy-services-using-egress-gateways/</link><author>Antonio Berben (Deutsche Telekom - PAN-NET)</author><guid isPermaLink="true">/v1.12/blog/2020/proxying-legacy-services-using-egress-gateways/</guid><category>configuration</category><category>egress</category><category>gateway</category><category>external</category><category>service</category></item><item><title>Proxy protocol on AWS NLB and Istio ingress gateway</title><description>
&lt;p>This blog presents my latest experience about how to configure and enable proxy protocol with stack of AWS NLB and Istio Ingress gateway. The &lt;a href="https://www.haproxy.com/blog/haproxy/proxy-protocol/">Proxy Protocol&lt;/a> was designed to chain proxies and reverse-proxies without losing the client information. The proxy protocol prevents the need for infrastructure changes or &lt;code>NATing&lt;/code> firewalls, and offers the benefits of being protocol agnostic and providing good scalability. Additionally, we also enable the &lt;code>X-Forwarded-For&lt;/code> HTTP header in the deployment to make the client IP address easy to read. In this blog, traffic management of Istio ingress is shown with an httpbin service on ports 80 and 443 to demonstrate the use of proxy protocol. Note that both v1 and v2 of the proxy protocol work for the purpose of this example, but because the AWS NLB currently only supports v2, proxy protocol v2 is used in the rest of this blog by default. The following image shows the use of proxy protocol v2 with an AWS NLB.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">&lt;p>A receiver may be configured to support both version 1 and version 2 of the
protocol. Identifying the protocol version is easy:&lt;/p>
&lt;ul>
&lt;li>&lt;p>If the incoming byte count is 16 or more and the first 13 bytes match the protocol signature block &lt;code>\x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A\x02&lt;/code>, the protocol is version 2.&lt;/p>&lt;/li>
&lt;li>&lt;p>Otherwise, if the incoming byte count is 8 or more, and the 5 first characters match the &lt;code>US-ASCII&lt;/code> representation of &amp;ldquo;PROXY&amp;rdquo;(&lt;code>\x50\x52\x4F\x58\x59&lt;/code>), then the protocol must be parsed as version 1.&lt;/p>&lt;/li>
&lt;li>&lt;p>Otherwise the protocol is not covered by this specification and the connection must be dropped.&lt;/p>&lt;/li>
&lt;/ul>
&lt;/div>
&lt;/aside>
&lt;/div>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:80.81149619611158%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/show-source-ip/aws-proxy-protocol.png" title="AWS NLB portal to enable proxy protocol">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/show-source-ip/aws-proxy-protocol.png" alt="AWS NLB portal to enable proxy protocol" />
&lt;/a>
&lt;/div>
&lt;figcaption>AWS NLB portal to enable proxy protocol&lt;/figcaption>
&lt;/figure>
&lt;h2 id="separate-setups-for-80-and-443">Separate setups for 80 and 443&lt;/h2>
&lt;p>Before going through the following steps, an AWS environment that is configured with the proper VPC, IAM, and Kubernetes setup is assumed.&lt;/p>
&lt;h3 id="step-1-install-istio-with-aws-nlb">Step 1: Install Istio with AWS NLB&lt;/h3>
&lt;p>The blog &lt;a href="/v1.12/blog/2018/aws-nlb/">Configuring Istio Ingress with AWS NLB&lt;/a> provides detailed steps to set up AWS IAM roles and enable the usage of AWS NLB by Helm. You can also use other automation tools, such as Terraform, to achieve the same goal. In the following example, more complete configurations are shown in order to enable proxy protocol and &lt;code>X-Forwarded-For&lt;/code> at the same time.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: &amp;#34;*&amp;#34;
service.beta.kubernetes.io/aws-load-balancer-type: &amp;#34;nlb&amp;#34;
proxy.istio.io/config: &amp;#39;{&amp;#34;gatewayTopology&amp;#34; : { &amp;#34;numTrustedProxies&amp;#34;: 2 } }&amp;#39;
labels:
app: istio-ingressgateway
istio: ingressgateway
release: istio
name: istio-ingressgateway
&lt;/code>&lt;/pre>
&lt;h3 id="step-2-create-proxy-protocol-envoy-filter">Step 2: Create proxy-protocol Envoy Filter&lt;/h3>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: proxy-protocol
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: LISTENER
patch:
operation: MERGE
value:
listener_filters:
- name: envoy.filters.listener.proxy_protocol
- name: envoy.filters.listener.tls_inspector
&lt;/code>&lt;/pre>
&lt;h3 id="step-3-enable-x-forwarded-for-header">Step 3: Enable &lt;code>X-Forwarded-For&lt;/code> header&lt;/h3>
&lt;p>This &lt;a href="/v1.12/docs/ops/configuration/traffic-management/network-topologies/">blog&lt;/a> includes several samples of configuring Gateway Network Topology. In the following example, the configurations are tuned to enable &lt;code>X-Forwarded-For&lt;/code> without any middle proxy.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: ingressgateway-settings
namespace: istio-system
spec:
configPatches:
- applyTo: NETWORK_FILTER
match:
listener:
filterChain:
filter:
name: envoy.http_connection_manager
patch:
operation: MERGE
value:
name: envoy.http_connection_manager
typed_config:
&amp;#34;@type&amp;#34;: type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
skip_xff_append: false
use_remote_address: true
xff_num_trusted_hops: 1
&lt;/code>&lt;/pre>
&lt;h3 id="step-4-deploy-ingress-gateway-for-httpbin-on-port-80-and-443">Step 4: Deploy ingress gateway for httpbin on port 80 and 443&lt;/h3>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">When following the &lt;a href="/v1.12/docs/tasks/traffic-management/ingress/secure-ingress/">secure ingress setup&lt;/a>, macOS users must add an additional patch to generate certificates for TLS.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: httpbin-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- &amp;#34;a25fa0b4835b.elb.us-west-2.amazonaws.com&amp;#34;
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- &amp;#34;a25fa0b4835b.elb.us-west-2.amazonaws.com&amp;#34;
gateways:
- httpbin-gateway
http:
- match:
- uri:
prefix: /headers
route:
- destination:
port:
number: 8000
host: httpbin
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: mygateway2
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: httpbin-credential # must be the same as secret
hosts:
- &amp;#34;a25fa0b4835b.elb.us-west-2.amazonaws.com&amp;#34;
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- &amp;#34;a25fa0b4835b.elb.us-west-2.amazonaws.com&amp;#34;
gateways:
- mygateway2
http:
- match:
- uri:
prefix: /headers
route:
- destination:
port:
number: 8000
host: httpbin
&lt;/code>&lt;/pre>
&lt;h3 id="step-5-check-header-output-of-httpbin">Step 5: Check header output of httpbin&lt;/h3>
&lt;p>Check port 443 (80 will be similar) and compare the cases with and without proxy protocol.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >//////with proxy_protocal enabled in the stack
* Trying YY.XXX.141.26...
* TCP_NODELAY set
* Connection failed
* connect to YY.XXX.141.26 port 443 failed: Operation timed out
* Trying YY.XXX.205.117...
* TCP_NODELAY set
* Connected to a25fa0b4835b.elb.us-west-2.amazonaws.com (XX.YYY.205.117) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: new_certificates/example.com.crt
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=a25fa0b4835b.elb.us-west-2.amazonaws.com; O=httpbin organization
* start date: Oct 29 20:39:12 2020 GMT
* expire date: Oct 29 20:39:12 2021 GMT
* common name: a25fa0b4835b.elb.us-west-2.amazonaws.com (matched)
* issuer: O=example Inc.; CN=example.com
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fc6c8810800)
&amp;gt; GET /headers?show_env=1 HTTP/2
&amp;gt; Host: a25fa0b4835b.elb.us-west-2.amazonaws.com
&amp;gt; User-Agent: curl/7.64.1
&amp;gt; Accept: */*
&amp;gt;
* Connection state changed (MAX_CONCURRENT_STREAMS == 2147483647)!
&amp;lt; HTTP/2 200
&amp;lt; server: istio-envoy
&amp;lt; date: Thu, 29 Oct 2020 21:39:46 GMT
&amp;lt; content-type: application/json
&amp;lt; content-length: 629
&amp;lt; access-control-allow-origin: *
&amp;lt; access-control-allow-credentials: true
&amp;lt; x-envoy-upstream-service-time: 2
&amp;lt;
{
&amp;#34;headers&amp;#34;: {
&amp;#34;Accept&amp;#34;: &amp;#34;*/*&amp;#34;,
&amp;#34;Content-Length&amp;#34;: &amp;#34;0&amp;#34;,
&amp;#34;Host&amp;#34;: &amp;#34;a25fa0b4835b.elb.us-west-2.amazonaws.com&amp;#34;,
&amp;#34;User-Agent&amp;#34;: &amp;#34;curl/7.64.1&amp;#34;,
&amp;#34;X-B3-Sampled&amp;#34;: &amp;#34;0&amp;#34;,
&amp;#34;X-B3-Spanid&amp;#34;: &amp;#34;74f99a1c6fc29975&amp;#34;,
&amp;#34;X-B3-Traceid&amp;#34;: &amp;#34;85db86fe6aa322a074f99a1c6fc29975&amp;#34;,
&amp;#34;X-Envoy-Attempt-Count&amp;#34;: &amp;#34;1&amp;#34;,
&amp;#34;X-Envoy-Decorator-Operation&amp;#34;: &amp;#34;httpbin.default.svc.cluster.local:8000/headers*&amp;#34;,
&amp;#34;X-Envoy-External-Address&amp;#34;: &amp;#34;XX.110.54.41&amp;#34;,
&amp;#34;X-Forwarded-For&amp;#34;: &amp;#34;XX.110.54.41&amp;#34;,
&amp;#34;X-Forwarded-Proto&amp;#34;: &amp;#34;https&amp;#34;,
&amp;#34;X-Request-Id&amp;#34;: &amp;#34;5c3bc236-0c49-4401-b2fd-2dbfbce506fc&amp;#34;
}
}
* Connection #0 to host a25fa0b4835b.elb.us-west-2.amazonaws.com left intact
* Closing connection 0
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >//////////without proxy_protocal
* Trying YY.XXX.141.26...
* TCP_NODELAY set
* Connection failed
* connect to YY.XXX.141.26 port 443 failed: Operation timed out
* Trying YY.XXX.205.117...
* TCP_NODELAY set
* Connected to a25fa0b4835b.elb.us-west-2.amazonaws.com (YY.XXX.205.117) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: new_certificates/example.com.crt
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=a25fa0b4835b.elb.us-west-2.amazonaws.com; O=httpbin organization
* start date: Oct 29 20:39:12 2020 GMT
* expire date: Oct 29 20:39:12 2021 GMT
* common name: a25fa0b4835b.elb.us-west-2.amazonaws.com (matched)
* issuer: O=example Inc.; CN=example.com
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fbf8c808200)
&amp;gt; GET /headers?show_env=1 HTTP/2
&amp;gt; Host: a25fa0b4835b.elb.us-west-2.amazonaws.com
&amp;gt; User-Agent: curl/7.64.1
&amp;gt; Accept: */*
&amp;gt;
* Connection state changed (MAX_CONCURRENT_STREAMS == 2147483647)!
&amp;lt; HTTP/2 200
&amp;lt; server: istio-envoy
&amp;lt; date: Thu, 29 Oct 2020 20:44:01 GMT
&amp;lt; content-type: application/json
&amp;lt; content-length: 612
&amp;lt; access-control-allow-origin: *
&amp;lt; access-control-allow-credentials: true
&amp;lt; x-envoy-upstream-service-time: 1
&amp;lt;
{
&amp;#34;headers&amp;#34;: {
&amp;#34;Accept&amp;#34;: &amp;#34;*/*&amp;#34;,
&amp;#34;Content-Length&amp;#34;: &amp;#34;0&amp;#34;,
&amp;#34;Host&amp;#34;: &amp;#34;a25fa0b4835b.elb.us-west-2.amazonaws.com&amp;#34;,
&amp;#34;User-Agent&amp;#34;: &amp;#34;curl/7.64.1&amp;#34;,
&amp;#34;X-B3-Sampled&amp;#34;: &amp;#34;0&amp;#34;,
&amp;#34;X-B3-Spanid&amp;#34;: &amp;#34;69913a6e6e949334&amp;#34;,
&amp;#34;X-B3-Traceid&amp;#34;: &amp;#34;729d5da3618545da69913a6e6e949334&amp;#34;,
&amp;#34;X-Envoy-Attempt-Count&amp;#34;: &amp;#34;1&amp;#34;,
&amp;#34;X-Envoy-Decorator-Operation&amp;#34;: &amp;#34;httpbin.default.svc.cluster.local:8000/headers*&amp;#34;,
&amp;#34;X-Envoy-Internal&amp;#34;: &amp;#34;true&amp;#34;,
&amp;#34;X-Forwarded-For&amp;#34;: &amp;#34;172.16.5.30&amp;#34;,
&amp;#34;X-Forwarded-Proto&amp;#34;: &amp;#34;https&amp;#34;,
&amp;#34;X-Request-Id&amp;#34;: &amp;#34;299c7f8a-5f89-480a-82c9-028c76d45d84&amp;#34;
}
}
* Connection #0 to host a25fa0b4835b.elb.us-west-2.amazonaws.com left intact
* Closing connection 0
&lt;/code>&lt;/pre>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>This blog presents the deployment of a stack that consists of an AWS NLB and Istio ingress gateway that are enabled with proxy-protocol. We hope it is useful to you if you are interested in protocol enabling in an anecdotal, experiential, and more informal way. However, note that the &lt;code>X-Forwarded-For&lt;/code> header should be used only for the convenience of reading in test, as dealing with fake &lt;code>X-Forwarded-For&lt;/code> attacks is not within the scope of this blog.&lt;/p>
&lt;h2 id="references">References&lt;/h2>
&lt;ul>
&lt;li>&lt;p>&lt;a href="https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/">protocol settings&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://www.haproxy.com/blog/haproxy/proxy-protocol/">protocol introduction&lt;/a>&lt;/p>&lt;/li>
&lt;/ul></description><pubDate>Fri, 11 Dec 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/show-source-ip/</link><author>Xinhui Li (Salesforce)</author><guid isPermaLink="true">/v1.12/blog/2020/show-source-ip/</guid><category>trafficManagement</category><category>protocol extending</category></item><item><title>Join us for the first IstioCon in 2021!</title><description>&lt;p>IstioCon 2021 will be the inaugural conference for Istio, the industry&amp;rsquo;s &lt;a href="https://www.cncf.io/wp-content/uploads/2020/11/CNCF_Survey_Report_2020.pdf">most popular service mesh&lt;/a>. In its inaugural year, IstioCon will be 100% virtual, connecting community members across the globe with Istio&amp;rsquo;s ecosystem. This conference will take place at the end of February.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:40.3125%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/istiocon-2021/istioconlogo.jpg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/istiocon-2021/istioconlogo.jpg" alt="IstioCon logo" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
&lt;p>All the information related to IstioCon will be published on the &lt;a href="https://events.istio.io/">conference website&lt;/a>. IstioCon provides an opportunity to showcase the lessons learned from running Istio in production, hands-on experiences from the Istio community, and will feature maintainers from across the Istio ecosystem. At this time, we encourage Istio users, developers, partners, and advocates to &lt;a href="https://sessionize.com/istiocon-2021/">submit a session proposal through the conference&amp;rsquo;s CFP portal&lt;/a>. The conference offers a mix of keynotes, technical talks, lightning talks, workshops, and roadmap sessions. Choose from the following formats to submit a session proposal for IstioCon:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>Presentation:&lt;/strong> 40 minute presentation, maximum of 2 speakers&lt;/li>
&lt;li>&lt;strong>Panel:&lt;/strong> 40 minutes of discussion among 3 to 5 speakers&lt;/li>
&lt;li>&lt;strong>Workshop:&lt;/strong> 160 minute (2h 40m), in-depth, hands-on presentation with 1–4 speakers&lt;/li>
&lt;li>&lt;strong>Lighting Talk:&lt;/strong> 10 minute presentation, limited to 1 speaker&lt;/li>
&lt;/ul>
&lt;p>This community-led event also has in store two social hours to take the load off and mesh with the Istio community, vendors, and maintainers. Participation in the event is free of charge, and will only require participants to register in order to join.&lt;/p>
&lt;p>Stay tuned to hear more about this conference, and we hope you can join us at the first IstioCon in 2021!&lt;/p></description><pubDate>Tue, 08 Dec 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/istiocon-2021/</link><author>Istio Steering Committee</author><guid isPermaLink="true">/v1.12/blog/2020/istiocon-2021/</guid><category>IstioCon</category><category>Istio</category><category>conference</category></item><item><title>Handling Docker Hub rate limiting</title><description>
&lt;p>Since November 20th, 2020, Docker Hub has introduced &lt;a href="https://www.docker.com/increase-rate-limits">rate limits&lt;/a> on image pulls.&lt;/p>
&lt;p>Because Istio uses &lt;a href="https://hub.docker.com/u/istio">Docker Hub&lt;/a> as the default registry, usage on a large cluster may lead
to pods failing to startup due to exceeding rate limits. This can be especially problematic for Istio, as there is typically
the Istio sidecar image alongside most pods in the cluster.&lt;/p>
&lt;h2 id="mitigations">Mitigations&lt;/h2>
&lt;p>Istio allows you to specify a custom docker registry which you can use to make container images be fetched from your private registry. This can be configured by passing &lt;code>--set hub=&amp;lt;some-custom-registry&amp;gt;&lt;/code> at installation time.&lt;/p>
&lt;p>Istio provides official mirrors to &lt;a href="https://gcr.io/istio-release">Google Container Registry&lt;/a>. This can be configured with &lt;code>--set hub=gcr.io/istio-release&lt;/code>. This is available for Istio 1.5+.&lt;/p>
&lt;p>Alternatively, you can copy the official Istio images to your own registry. This is especially useful if your cluster runs in an environment with a registry tailored for your use case (for example, on AWS you may want to mirror images to Amazon ECR) or you have air gapped security requirements where access to public registries is restricted. This can be done with the following script:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ SOURCE_HUB=istio
$ DEST_HUB=my-registry # Replace this with the destination hub
$ IMAGES=( install-cni operator pilot proxyv2 ) # Images to mirror.
$ VERSIONS=( 1.7.5 1.8.0 ) # Versions to copy
$ VARIANTS=( &amp;#34;&amp;#34; &amp;#34;-distroless&amp;#34; ) # Variants to copy
$ for image in $IMAGES; do
$ for version in $VERSIONS; do
$ for variant in $VARIANTS; do
$ name=$image:$version$variant
$ docker pull $SOURCE_HUB/$name
$ docker tag $SOURCE_HUB/$name $DEST_HUB/$name
$ docker push $DEST_HUB/$name
$ docker rmi $SOURCE_HUB/$name
$ docker rmi $DEST_HUB/$name
$ done
$ done
$ done
&lt;/code>&lt;/pre></description><pubDate>Mon, 07 Dec 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/docker-rate-limit/</link><author>John Howard (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/docker-rate-limit/</guid><category>docker</category></item><item><title>Expanding into New Frontiers - Smart DNS Proxying in Istio</title><description>
&lt;p>DNS resolution is a vital component of any application infrastructure
on Kubernetes. When your application code attempts to access another
service in the Kubernetes cluster or even a service on the internet,
it has to first lookup the IP address corresponding to the hostname of
the service, before initiating a connection to the service. This name
lookup process is often referred to as &lt;strong>service discovery&lt;/strong>. In
Kubernetes, the cluster DNS server, be it &lt;code>kube-dns&lt;/code> or CoreDNS,
resolves the service&amp;rsquo;s hostname to a unique non-routable virtual IP (VIP),
if it is a service of type &lt;code>clusterIP&lt;/code>. The &lt;code>kube-proxy&lt;/code> on each node
maps this VIP to a set of pods of the service, and forwards the traffic
to one of them selected at random. When using a service mesh, the
sidecar works similarly to the &lt;code>kube-proxy&lt;/code> as far as traffic forwarding
is concerned.&lt;/p>
&lt;p>The following diagram depicts the role of DNS today:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:57.00636942675159%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/dns-proxy/role-of-dns-today.png" title="Role of DNS in Istio, today">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/dns-proxy/role-of-dns-today.png" alt="Role of DNS in Istio, today" />
&lt;/a>
&lt;/div>
&lt;figcaption>Role of DNS in Istio, today&lt;/figcaption>
&lt;/figure>
&lt;h2 id="problems-posed-by-dns">Problems posed by DNS&lt;/h2>
&lt;p>While the role of DNS within the service mesh may seem insignificant,
it has consistently stood in the way of expanding the mesh to VMs and
enabling seamless multicluster access.&lt;/p>
&lt;h3 id="vm-access-to-kubernetes-services">VM access to Kubernetes services&lt;/h3>
&lt;p>Consider the case of a VM with a sidecar. As shown in the illustration
below, applications on the VM look up the IP addresses of services
inside the Kubernetes cluster as they typically have no access to the
cluster&amp;rsquo;s DNS server.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:42.37837837837838%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/dns-proxy/vm-dns-resolution-issues.png" title="DNS resolution issues on VMs accessing Kubernetes services">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/dns-proxy/vm-dns-resolution-issues.png" alt="DNS resolution issues on VMs accessing Kubernetes services" />
&lt;/a>
&lt;/div>
&lt;figcaption>DNS resolution issues on VMs accessing Kubernetes services&lt;/figcaption>
&lt;/figure>
&lt;p>It is technically possible to use &lt;code>kube-dns&lt;/code> as a name server on the VM if one is
willing to engage in some convoluted workarounds involving &lt;code>dnsmasq&lt;/code> and
external exposure of &lt;code>kube-dns&lt;/code> using &lt;code>NodePort&lt;/code> services: assuming you
manage to convince your cluster administrator to do so. Even so, you are
opening the door to a host of &lt;a href="https://blog.aquasec.com/dns-spoofing-kubernetes-clusters">security
issues&lt;/a>. At
the end of the day, these are point solutions that are typically out
of scope for those with limited organizational capability and domain
expertise.&lt;/p>
&lt;h3 id="external-tcp-services-without-vips">External TCP services without VIPs&lt;/h3>
&lt;p>It is not just the VMs in the mesh that suffer from the DNS issue. For
the sidecar to accurately distinguish traffic between two different
TCP services that are outside the mesh, the services must be on
different ports or they need to have a globally unique VIP, much like
the &lt;code>clusterIP&lt;/code> assigned to Kubernetes services. But what if there is
no VIP? Cloud hosted services like hosted databases, typically do not
have a VIP. Instead, the provider&amp;rsquo;s DNS server returns one of the
instance IPs that can then be directly accessed by the
application. For example, consider the two service entries below,
pointing to two different AWS RDS services:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: db1
namespace: ns1
spec:
hosts:
- mysql-instance1.us-east-1.rds.amazonaws.com
ports:
- name: mysql
number: 3306
protocol: TCP
resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: db2
namespace: ns1
spec:
hosts:
- mysql-instance2.us-east-1.rds.amazonaws.com
ports:
- name: mysql
number: 3306
protocol: TCP
resolution: DNS
&lt;/code>&lt;/pre>
&lt;p>The sidecar has a single listener on &lt;code>0.0.0.0:3306&lt;/code> that looks up the
IP address of &lt;code>mysql-instance1.us-east1.rds.amazonaws.com&lt;/code> from public
DNS servers and forwards traffic to it. It cannot route traffic to
&lt;code>db2&lt;/code> as it has no way of distinguishing whether traffic arriving at
&lt;code>0.0.0.0:3306&lt;/code> is bound for &lt;code>db1&lt;/code> or &lt;code>db2&lt;/code>. The only way to accomplish
this is to set the resolution to &lt;code>NONE&lt;/code> causing the sidecar to
&lt;em>blindly forward any traffic&lt;/em> on port &lt;code>3306&lt;/code> to the original IP
requested by the application. This is akin to punching a hole in the
firewall allowing all traffic to port &lt;code>3306&lt;/code> irrespective of the
destination IP. To get traffic flowing, you are now forced to
compromise on the security posture of your system.&lt;/p>
&lt;h3 id="resolving-dns-for-services-in-remote-clusters">Resolving DNS for services in remote clusters&lt;/h3>
&lt;p>The DNS limitations of a multicluster mesh are well known. Services in
one cluster cannot lookup the IP addresses of services in other
clusters, without clunky workarounds such as creating stub services in
the caller namespace.&lt;/p>
&lt;h2 id="taking-control-of-dns">Taking control of DNS&lt;/h2>
&lt;p>All in all, DNS has been a thorny issue in Istio for a while. It was
time to slay the beast. We (the Istio networking team) decided to
tackle the problem once and for all in a way that is completely
transparent to you, the end user. Our first attempt involved utilizing
Envoy&amp;rsquo;s DNS proxy. It turned out to be very unreliable, and
disappointing overall due to the general lack of sophistication in
the c-ares DNS library used by Envoy. Determined to solve the
problem, we decided to implement the DNS proxy in the Istio sidecar
agent, written in Go. We were able to optimize the implementation to
handle all the scenarios that we wanted to tackle without compromising
on scale and stability. The Go DNS library we use is the same one
used by scalable DNS implementations such as CoreDNS, Consul,
Mesos, etc. It has been battle tested in production for scale and stability.&lt;/p>
&lt;p>Starting with Istio 1.8, the Istio agent on the sidecar will ship with
a caching DNS proxy, programmed dynamically by Istiod. Istiod pushes
the hostname-to-IP-address mappings for all the services that the
application may access based on the Kubernetes services and service
entries in the cluster. DNS lookup queries from the application are
transparently intercepted and served by the Istio agent in the pod or
VM. If the query is for a service within the mesh, &lt;em>irrespective of
the cluster that the service is in&lt;/em>, the agent responds directly to the
application. If not, it forwards the query to the upstream name
servers defined in &lt;code>/etc/resolv.conf&lt;/code>. The following diagram depicts
the interactions that occur when an application tries to access a
service using its hostname.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:41.07929515418502%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/dns-proxy/dns-interception-in-istio.png" title="Smart DNS proxying in Istio sidecar agent">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/dns-proxy/dns-interception-in-istio.png" alt="Smart DNS proxying in Istio sidecar agent" />
&lt;/a>
&lt;/div>
&lt;figcaption>Smart DNS proxying in Istio sidecar agent&lt;/figcaption>
&lt;/figure>
&lt;p>As you will see in the following sections, &lt;em>the DNS proxying feature
has had an enormous impact across many aspects of Istio.&lt;/em>&lt;/p>
&lt;h3 id="reduced-load-on-your-dns-servers-w-faster-resolution">Reduced load on your DNS servers w/ faster resolution&lt;/h3>
&lt;p>The load on your cluster’s Kubernetes DNS server drops drastically as
almost all DNS queries are resolved within the pod by Istio. The
bigger the footprint of mesh on a cluster, the lesser the load on your
DNS servers. Implementing our own DNS proxy in the Istio agent has
allowed us to implement cool optimizations such as &lt;a href="https://coredns.io/plugins/autopath/">CoreDNS
auto-path&lt;/a> without the
correctness issues that CoreDNS currently faces.&lt;/p>
&lt;p>To understand the impact of this optimization, lets take a simple DNS
lookup scenario, in a standard Kubernetes cluster without any custom
DNS setup for pods - i.e., with the default setting of &lt;code>ndots:5&lt;/code> in &lt;code>/etc/resolv.conf&lt;/code>.
When your application starts a DNS lookup for
&lt;code>productpage.ns1.svc.cluster.local&lt;/code>, it appends the DNS search
namespaces in &lt;code>/etc/resolv.conf&lt;/code> (e.g., &lt;code>ns1.svc.cluster.local&lt;/code>) as part
of the DNS query, before querying the host as-is. As a result, the
first DNS query that is actually sent out will look like
&lt;code>productpage.ns1.svc.cluster.local.ns1.svc.cluster.local&lt;/code>, which will
inevitably fail DNS resolution when Istio is not involved. If your
&lt;code>/etc/resolv.conf&lt;/code> has 5 search namespaces, the application will send
two DNS queries for each search namespace, one for the IPv4 &lt;code>A&lt;/code> record
and another for the IPv6 &lt;code>AAAA&lt;/code> record, and then a final pair of
queries with the exact hostname used in the code. &lt;em>Before establishing the
connection, the application performs 12 DNS lookup queries for each host!&lt;/em>&lt;/p>
&lt;p>With Istio&amp;rsquo;s implementation of the CoreDNS style auto-path technique,
the sidecar agent will detect the real hostname being queried within
the first query and return a &lt;code>cname&lt;/code> record to
&lt;code>productpage.ns1.svc.cluster.local&lt;/code> as part of this DNS response, as
well as the &lt;code>A/AAAA&lt;/code> record for
&lt;code>productpage.ns1.svc.cluster.local&lt;/code>. The application receiving this
response can now extract the IP address immediately and proceed to
establishing a TCP connection to that IP. &lt;em>The smart DNS proxy in the
Istio agent dramatically cuts down the number of DNS queries from 12
to just 2!&lt;/em>&lt;/p>
&lt;h3 id="vms-to-kubernetes-integration">VMs to Kubernetes integration&lt;/h3>
&lt;p>Since the Istio agent performs local DNS resolution for services
within the mesh, DNS lookup queries for Kubernetes services from VMs will now
succeed without requiring clunky workarounds for exposing &lt;code>kube-dns&lt;/code>
outside the cluster. The ability to seamlessly resolve internal
services in a cluster will now simplify your monolith to microservice
journey, as the monolith on VMs can now access microservices on
Kubernetes without additional levels of indirection via API gateways.&lt;/p>
&lt;h3 id="automatic-vip-allocation-where-possible">Automatic VIP allocation where possible&lt;/h3>
&lt;p>You may ask, how does this DNS functionality in the agent solve the
problem of distinguishing between multiple external TCP services
without VIPs on the same port?&lt;/p>
&lt;p>Taking inspiration from Kubernetes, Istio will now automatically
allocate non-routable VIPs (from the Class E subnet) to such services
as long as they do not use a wildcard host. The Istio agent on the
sidecar will use the VIPs as responses to the DNS lookup queries from
the application. Envoy can now clearly distinguish traffic bound for
each external TCP service and forward it to the right target. With the
introduction of the DNS proxying, you will no longer need to use
&lt;code>resolution: NONE&lt;/code> for non-wildcard TCP services, improving your
overall security posture. Istio cannot help much with wildcard
external services (e.g., &lt;code>*.us-east1.rds.amazonaws.com&lt;/code>). You will
have to resort to NONE resolution mode to handle such services.&lt;/p>
&lt;h3 id="multicluster-dns-lookup">Multicluster DNS lookup&lt;/h3>
&lt;p>For the adventurous lot, attempting to weave a multicluster mesh where
applications directly call internal services of a namespace in a
remote cluster, the DNS proxy functionality comes in quite handy. Your
applications can &lt;em>resolve Kubernetes services on any cluster in any
namespace&lt;/em>, without the need to create stub Kubernetes services in
every cluster.&lt;/p>
&lt;p>The benefits of the DNS proxy extend beyond the multicluster models
that are currently described in Istio today. At Tetrate, we use this
mechanism extensively in our customers&amp;rsquo; multicluster deployments to
enable sidecars to resolve DNS for hosts exposed at ingress gateways
of all the clusters in a mesh, and access them over mutual TLS.&lt;/p>
&lt;h2 id="concluding-thoughts">Concluding thoughts&lt;/h2>
&lt;p>The problems caused by lack of control over DNS have often been
overlooked and ignored in its entirety when it comes to weaving a mesh
across many clusters, different environments, and integrating external
services. The introduction of a caching DNS proxy in the Istio sidecar
agent solves these issues. Exercising control over the
application’s DNS resolution allows Istio to accurately identify the
target service to which traffic is bound, and enhance the overall
security, routing, and telemetry posture in Istio within and across
clusters.&lt;/p>
&lt;p>Smart DNS proxying is enabled in the &lt;code>preview&lt;/code>
profile in Istio 1.8. Please try it out!&lt;/p></description><pubDate>Thu, 12 Nov 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/dns-proxy/</link><author>Shriram Rajagopalan (Tetrate.io) on behalf of Istio Networking WG</author><guid isPermaLink="true">/v1.12/blog/2020/dns-proxy/</guid><category>dns</category><category>sidecar</category><category>multicluster</category><category>vm</category><category>external services</category></item><item><title>2020 Steering Committee Election Results</title><description>&lt;p>Last month, we &lt;a href="../steering-changes/">announced a revision to our Steering Committee charter&lt;/a>, opening up governance roles to more contributors and community members. The Steering Committee now consists of 9 proportionally-allocated Contribution Seats, and 4 elected Community Seats.&lt;/p>
&lt;p>We have now concluded our &lt;a href="https://github.com/istio/community/tree/master/steering/elections/2020">inaugural election&lt;/a> for the Community Seats, and we&amp;rsquo;re excited to welcome the following new members to the Committee:&lt;/p>
&lt;ul>
&lt;li>&lt;a href="https://github.com/istio/community/blob/master/steering/elections/2020/nrjpoddar.md">Neeraj Poddar&lt;/a> (Aspen Mesh)&lt;/li>
&lt;li>&lt;a href="https://github.com/istio/community/blob/master/steering/elections/2020/zackbutcher.md">Zack Butcher&lt;/a> (Tetrate)&lt;/li>
&lt;li>&lt;a href="https://github.com/istio/community/blob/master/steering/elections/2020/ceposta.md">Christian Posta&lt;/a> (Solo.io)&lt;/li>
&lt;li>&lt;a href="https://github.com/istio/community/blob/master/steering/elections/2020/hzxuzhonghu.md">Zhonghu Xu&lt;/a> (Huawei)&lt;/li>
&lt;/ul>
&lt;p>They join Contribution Seat holders from Google, IBM/Red Hat and Salesforce. We now have representation from 7 organizations on Steering, reflecting the breadth of our contributor ecosystem.&lt;/p>
&lt;p>Thank you to everyone who participated in the election process. The next election will be in July 2021.&lt;/p></description><pubDate>Tue, 29 Sep 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/steering-election-results/</link><author>Istio Steering Committee</author><guid isPermaLink="true">/v1.12/blog/2020/steering-election-results/</guid><category>istio</category><category>steering</category><category>governance</category><category>community</category><category>election</category></item><item><title>Large Scale Security Policy Performance Tests</title><description>
&lt;h2 id="overview">Overview&lt;/h2>
&lt;p>Istio has a wide range of security policies which can be easily configured into systems of services. As the number of applied policies increases, it is important to understand the relationship of latency, memory usage, and CPU usage of the system.&lt;/p>
&lt;p>This blog post goes over common security policies use cases and how the number of security policies or the number of specific rules in a security policy can affect the overall latency of requests.&lt;/p>
&lt;h2 id="setup">Setup&lt;/h2>
&lt;p>There are a wide range of security policies and many more combinations of those policies. We will go over 6 of the most commonly used test cases.&lt;/p>
&lt;p>The following test cases are run in an environment which consists of a &lt;a href="https://fortio.org/">Fortio&lt;/a> client sending requests to a Fortio server, with a baseline of no Envoy sidecars deployed. The following data was gathered by using the &lt;a href="https://github.com/istio/tools/tree/master/perf/benchmark">Istio performance benchmarking tool&lt;/a>.
&lt;figure style="width:55%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/istio_setup.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/istio_setup.svg" alt="Environment setup" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>&lt;/p>
&lt;p>In these test cases, requests either do not match any rules or match only the very last rule in the security policies. This ensures that the RBAC filter is applied to all policy rules, and never matches a policy rule before before viewing all the policies. Even though this is not necessarily what will happen in your own system, this policy setup provides data for the worst possible performance of each test case.&lt;/p>
&lt;h2 id="test-cases">Test cases&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Mutual TLS STRICT vs plaintext.&lt;/p>&lt;/li>
&lt;li>&lt;p>A single authorization policy with a variable number of principal rules as well as a &lt;code>PeerAuthentication&lt;/code> policy. The principal rule is dependent on the &lt;code>PeerAuthentication&lt;/code> policy being applied to the system.&lt;/p>&lt;/li>
&lt;li>&lt;p>A single authorization policy with a variable number of &lt;code>requestPrincipal&lt;/code> rules as well as a &lt;code>RequestAuthentication&lt;/code> policy. The &lt;code>requestPrincipal&lt;/code> is dependent on the &lt;code>RequestAuthentication&lt;/code> policy being applied to the system.&lt;/p>&lt;/li>
&lt;li>&lt;p>A single authorization policy with a variable number of &lt;code>paths&lt;/code> vs &lt;code>sourceIP&lt;/code> rules.&lt;/p>&lt;/li>
&lt;li>&lt;p>A variable number of authorization policies consisting of a single path or &lt;code>sourceIP&lt;/code> rule.&lt;/p>&lt;/li>
&lt;li>&lt;p>A single &lt;code>RequestAuthentication&lt;/code> policy with variable number of &lt;code>JWTRules&lt;/code> rules.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h2 id="data">Data&lt;/h2>
&lt;p>The y-axis of each test is the latency in milliseconds, and the x-axis is the number of concurrent connections. The x-axis of each graph consists of 3 data points that represent a small load (qps=100, conn=8), medium load (qps=500, conn=32), and large load (qps=1000, conn=64).&lt;/p>
&lt;div id="tabset-blog-2020-large-scale-security-policy-performance-tests-1" role="tablist" class="tabset ">
&lt;div class="tab-strip" data-category-name="platform" >&lt;button aria-selected="true" data-category-value="one"
aria-controls="tabset-blog-2020-large-scale-security-policy-performance-tests-1-0-panel" id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-0-tab" role="tab">&lt;span>MTLS vs plainText&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="two"
aria-controls="tabset-blog-2020-large-scale-security-policy-performance-tests-1-1-panel" id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-1-tab" role="tab">&lt;span>AuthZ mTLS SourcePrincipals&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="three"
aria-controls="tabset-blog-2020-large-scale-security-policy-performance-tests-1-2-panel" id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-2-tab" role="tab">&lt;span>AuthZ JWT RequestPrincipal&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="four"
aria-controls="tabset-blog-2020-large-scale-security-policy-performance-tests-1-3-panel" id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-3-tab" role="tab">&lt;span>AuthZ sourceIP&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="five"
aria-controls="tabset-blog-2020-large-scale-security-policy-performance-tests-1-4-panel" id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-4-tab" role="tab">&lt;span>AuthZ paths&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="six"
aria-controls="tabset-blog-2020-large-scale-security-policy-performance-tests-1-5-panel" id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-5-tab" role="tab">&lt;span>RequestAuthN JWT Issuer&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="seven"
aria-controls="tabset-blog-2020-large-scale-security-policy-performance-tests-1-6-panel" id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-6-tab" role="tab">&lt;span>Variable AuthZ&lt;/span>
&lt;/button>&lt;/div>
&lt;div class="tab-content">&lt;div id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-0-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2020-large-scale-security-policy-performance-tests-1-0-tab">
&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/mtls_plaintext.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/mtls_plaintext.svg" alt="MTLS vs plaintext" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
The difference of latency between MTLS mode STRICT and plaintext is very small in lower loads. As the &lt;code>qps&lt;/code> and &lt;code>conn&lt;/code> increase, the latency of requests with MTLS STRICT increases. The additional latency increased in larger loads is minimal compared to that of the increase from having no sidecars to having sidecars in the plaintext.&lt;/div>&lt;div hidden id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-1-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2020-large-scale-security-policy-performance-tests-1-1-tab">
&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_principals.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_principals.svg" alt="Authorization policy variable number of principals" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
&lt;p>For Authorization policies with 10 vs 1000 principal rules, the latency increase of 10 principal rules compared to no policies is greater than the latency increase of 1000 principals compared to 10 principals.&lt;/div>&lt;div hidden id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-2-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2020-large-scale-security-policy-performance-tests-1-2-tab">
&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_requestPrincipals.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_requestPrincipals.svg" alt="Authorization policy with variable principals" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
For Authorization policies with a variable number of &lt;code>requestPrincipal&lt;/code> rules, the latency increase of 10 &lt;code>requestPrincipal&lt;/code> rules compared to no policies is nearly the same as the latency increase of 1000 &lt;code>requestPrincipal&lt;/code> rules compared to 10 &lt;code>requestPrincipal&lt;/code> rules.&lt;/div>&lt;div hidden id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-3-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2020-large-scale-security-policy-performance-tests-1-3-tab">
&lt;p>&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_sourceIP.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_sourceIP.svg" alt="Authorization policy with variable `sourceIP` rules" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
The latency increase of a single &lt;code>AuthZ&lt;/code> policy with 10 &lt;code>sourceIP&lt;/code> rules is not proportional to the latency increase of a single &lt;code>AuthZ&lt;/code> policy with 1000 &lt;code>sourceIP&lt;/code> rules compared to the system with sidecar and no policies.&lt;/p>
&lt;p>&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_paths_vs_sourceIP.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_paths_vs_sourceIP.svg" alt="Authorization policy with both paths and `sourceIP`" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
The latency increase of a variable number of &lt;code>sourceIP&lt;/code> rules is marginally greater than that of path rules.&lt;/p>
&lt;/div>&lt;div hidden id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-4-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2020-large-scale-security-policy-performance-tests-1-4-tab">
&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_paths.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_paths.svg" alt="Authorization policy with variable number of paths" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
The latency increase of a single &lt;code>AuthZ&lt;/code> policy with 10 path rules is not proportional to the latency increase of a single &lt;code>AuthZ&lt;/code> policy with 1000 path rules compared to the system with sidecar and no policies. This trend is similar to that of &lt;code>sourceIP&lt;/code> rules.
&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_paths_vs_sourceIP.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_paths_vs_sourceIP.svg" alt="Authorization policy with both paths and `sourceIP`" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
The latency of a variable number of paths rules is marginally lesser than that of &lt;code>sourceIP&lt;/code> rules.&lt;/div>&lt;div hidden id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-5-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2020-large-scale-security-policy-performance-tests-1-5-tab">
&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/RequestAuthN_jwks.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/RequestAuthN_jwks.svg" alt="Request Authentication with variable number of JWT issuers" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
The latency of a single JWT issuer is comparable to that of no policies, but as the number of JWT issuers increase, the latency increases disproportionately.&lt;/div>&lt;div hidden id="tabset-blog-2020-large-scale-security-policy-performance-tests-1-6-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2020-large-scale-security-policy-performance-tests-1-6-tab">
&lt;p>To test how the number of Authorization policies affect runtime, the tests can be broken into two cases:&lt;/p>
&lt;ol>
&lt;li>&lt;p>Every Authorization policy has a single &lt;code>sourceIP&lt;/code> rule.&lt;/p>&lt;/li>
&lt;li>&lt;p>Every Authorization policy has a single path rule.&lt;/p>&lt;/li>
&lt;/ol>
&lt;p>&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_policies_sourceIP.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_policies_sourceIP.svg" alt="Authorization policy with variable number of policies, with `sourceIP` rule" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
&lt;figure style="width:90%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.34%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_policies_paths.svg" title="">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/large-scale-security-policy-performance-tests/AuthZ_var_policies_paths.svg" alt="Authorization policy with variable number of policies, with path rule" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
The overall trends of both graphs are similar. This is consistent to the paths vs &lt;code>sourceIP&lt;/code> data, which showed that the latency is marginally greater for &lt;code>sourceIP&lt;/code> rules than that of path rules.&lt;/p>
&lt;/div>&lt;/div>
&lt;/div>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;ul>
&lt;li>&lt;p>In general, adding security policies does not add relatively high overhead to the system. The policies that add the most latency include:&lt;/p>
&lt;ol>
&lt;li>&lt;p>Authorization policy with &lt;code>JWTRules&lt;/code> rules.&lt;/p>&lt;/li>
&lt;li>&lt;p>Authorization policy with &lt;code>requestPrincipal&lt;/code> rules.&lt;/p>&lt;/li>
&lt;li>&lt;p>Authorization policy with principals rules.&lt;/p>&lt;/li>
&lt;/ol>&lt;/li>
&lt;li>&lt;p>In lower loads (requests with lower qps and conn) the difference in latency for most policies is minimal.&lt;/p>&lt;/li>
&lt;li>&lt;p>Envoy proxy sidecars increase latency more than most policies, even if the policies are large.&lt;/p>&lt;/li>
&lt;li>&lt;p>The latency increase of extremely large policies is relatively similar to the latency increase of adding Envoy proxy sidecars compared to that of no sidecars.&lt;/p>&lt;/li>
&lt;li>&lt;p>Two different tests determined that the &lt;code>sourceIP&lt;/code> rule is marginally slower than a path rule.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>If you are interested in creating your own large scale security policies and running performance tests with them, see the &lt;a href="https://github.com/istio/tools/tree/master/perf/benchmark/security/generate_policies">performance benchmarking tool README&lt;/a>.&lt;/p>
&lt;p>If you are interested in reading more about the security policies tests, see &lt;a href="https://docs.google.com/document/d/1ZP9eQ_2EJEG12xnfsoo7125FDN38r62iqY1PUn9Dz-0/edit?usp=sharing">our design doc&lt;/a>. If you don&amp;rsquo;t already have access, you can &lt;a href="/v1.12/get-involved/">join the Istio team drive&lt;/a>.&lt;/p></description><pubDate>Tue, 15 Sep 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/large-scale-security-policy-performance-tests/</link><author>Michael Eizaguirre (Google), Yangmin Zhu (Google), Carolyn Hu (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/large-scale-security-policy-performance-tests/</guid><category>test</category><category>security policy</category><category>performance</category></item><item><title>Deploying Istio Control Planes Outside the Mesh</title><description>
&lt;h2 id="overview">Overview&lt;/h2>
&lt;p>From experience working with various service mesh users and vendors, we believe there are 3 key personas for a typical service mesh:&lt;/p>
&lt;ul>
&lt;li>&lt;p>Mesh Operator, who manages the service mesh control plane installation and upgrade.&lt;/p>&lt;/li>
&lt;li>&lt;p>Mesh Admin, often referred as Platform Owner, who owns the service mesh platform and defines the overall strategy and implementation for service owners to adopt service mesh.&lt;/p>&lt;/li>
&lt;li>&lt;p>Mesh User, often referred as Service Owner, who owns one or more services in the mesh.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>Prior to version 1.7, Istio required the control plane to run in one of the &lt;span class="term" data-title="Primary Cluster" data-body="&amp;lt;p&amp;gt;A primary cluster is a &amp;lt;a href=&amp;#34;/docs/reference/glossary/#cluster&amp;#34;&amp;gt;cluster&amp;lt;/a&amp;gt; with a
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#control-plane&amp;#34;&amp;gt;control plane&amp;lt;/a&amp;gt;. A single
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-mesh&amp;#34;&amp;gt;mesh&amp;lt;/a&amp;gt; can have more than
one primary cluster for HA or to reduce latency. Primary clusters can act as the
control plane for &amp;lt;a href=&amp;#34;/docs/reference/glossary/#remote-cluster&amp;#34;&amp;gt;remote clusters&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">primary clusters&lt;/span> in the mesh, leading to a lack of separation between the mesh operator and the mesh admin. Istio 1.7 introduces a new &lt;span class="term" data-title="External Control Plane" data-body="&amp;lt;p&amp;gt;An external control plane is a &amp;lt;a href=&amp;#34;/docs/reference/glossary/#control-plane&amp;#34;&amp;gt;control plane&amp;lt;/a&amp;gt;
that externally manages mesh workloads running in their own &amp;lt;a href=&amp;#34;/docs/reference/glossary/#cluster&amp;#34;&amp;gt;clusters&amp;lt;/a&amp;gt;
or other infrastructure. The control plane may, itself, be deployed in a cluster, although not
in one of the clusters that is part of the mesh it&amp;amp;rsquo;s controlling.
Its purpose is to cleanly separate the control plane from the data plane of a mesh.&amp;lt;/p&amp;gt;
">external control plane&lt;/span> deployment model which enables mesh operators to install and manage mesh control planes on separate external clusters. This deployment model allows a clear separation between mesh operators and mesh admins. Istio mesh operators can now run Istio control planes for mesh admins while mesh admins can still control the configuration of the control plane without worrying about installing or managing the control plane. This model is transparent to mesh users.&lt;/p>
&lt;h2 id="external-control-plane-deployment-model">External control plane deployment model&lt;/h2>
&lt;p>After installing Istio using the &lt;a href="/v1.12/docs/setup/install/istioctl/#install-istio-using-the-default-profile">default installation profile&lt;/a>, you will have an Istiod control plane installed in a single cluster like the diagram below:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:40.78842240615207%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/new-deployment-model/single-cluster.svg" title="Single cluster Istio mesh">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/new-deployment-model/single-cluster.svg" alt="Istio mesh in a single cluster" />
&lt;/a>
&lt;/div>
&lt;figcaption>Istio mesh in a single cluster&lt;/figcaption>
&lt;/figure>
&lt;p>With the new deployment model in Istio 1.7, it&amp;rsquo;s possible to run Istiod on an external cluster, separate from the mesh services as shown in the diagram below. The external control plane cluster is owned by the mesh operator while the mesh admin owns the cluster running services deployed in the mesh. The mesh admin has no access to the external control plane cluster. Mesh operators can follow the &lt;a href="https://github.com/istio/istio/wiki/External-Istiod-single-cluster-steps">external istiod single cluster step by step guide&lt;/a> to explore more on this. (Note: In some internal discussions among Istio maintainers, this model was previously referred to as &amp;ldquo;central istiod&amp;rdquo;.)&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:47.926329500847174%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/new-deployment-model/single-cluster-external-Istiod.svg" title="Single cluster Istio mesh with Istiod outside">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/new-deployment-model/single-cluster-external-Istiod.svg" alt="Istio mesh in a single cluster with Istiod outside" />
&lt;/a>
&lt;/div>
&lt;figcaption>Single cluster Istio mesh with Istiod in an external control plane cluster&lt;/figcaption>
&lt;/figure>
&lt;p>Mesh admins can expand the service mesh to multiple clusters, which are managed by the same Istiod running in the external cluster. None of the mesh clusters are &lt;span class="term" data-title="Primary Cluster" data-body="&amp;lt;p&amp;gt;A primary cluster is a &amp;lt;a href=&amp;#34;/docs/reference/glossary/#cluster&amp;#34;&amp;gt;cluster&amp;lt;/a&amp;gt; with a
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#control-plane&amp;#34;&amp;gt;control plane&amp;lt;/a&amp;gt;. A single
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-mesh&amp;#34;&amp;gt;mesh&amp;lt;/a&amp;gt; can have more than
one primary cluster for HA or to reduce latency. Primary clusters can act as the
control plane for &amp;lt;a href=&amp;#34;/docs/reference/glossary/#remote-cluster&amp;#34;&amp;gt;remote clusters&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">primary clusters&lt;/span>, in this case. They are all &lt;span class="term" data-title="Remote Cluster" data-body="&amp;lt;p&amp;gt;A remote cluster is a &amp;lt;a href=&amp;#34;/docs/reference/glossary/#cluster&amp;#34;&amp;gt;cluster&amp;lt;/a&amp;gt; that
connects to a &amp;lt;a href=&amp;#34;/docs/reference/glossary/#control-plane&amp;#34;&amp;gt;control plane&amp;lt;/a&amp;gt;
residing outside of the cluster. A remote cluster can connect to a control plane
running in a &amp;lt;a href=&amp;#34;/docs/reference/glossary/#primary-cluster&amp;#34;&amp;gt;primary cluster&amp;lt;/a&amp;gt;
or to an &amp;lt;a href=&amp;#34;/docs/reference/glossary/#external-control-plane&amp;#34;&amp;gt;external control plane&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">remote clusters&lt;/span>. However, one of them also serves as the Istio configuration cluster, in addition to running services. The external control plane reads Istio configurations from the &lt;code>config cluster&lt;/code> and Istiod pushes configuration to the data plane running in both the config cluster and other remote clusters as shown in the diagram below.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:44.93126790115233%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/new-deployment-model/multiple-clusters-external-Istiod.svg" title="Multicluster Istio mesh with Istiod outside">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/new-deployment-model/multiple-clusters-external-Istiod.svg" alt="Multicluster Istio mesh with Istiod outside" />
&lt;/a>
&lt;/div>
&lt;figcaption>Multicluster Istio mesh with Istiod in an external control plane cluster&lt;/figcaption>
&lt;/figure>
&lt;p>Mesh operators can further expand this deployment model to manage multiple Istio control planes from an external cluster running multiple Istiod control planes:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.55366354432676%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/new-deployment-model/multiple-external-Istiods.svg" title="Multiple single clusters Istio meshes with Istiod outside">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/new-deployment-model/multiple-external-Istiods.svg" alt="Istio meshes in single clusters with Istiod outside" />
&lt;/a>
&lt;/div>
&lt;figcaption>Multiple single clusters with multiple Istiod control planes in an external control plane cluster&lt;/figcaption>
&lt;/figure>
&lt;p>In this case, each Istiod manages its own remote cluster(s). Mesh operators can even install their own Istio mesh in the external control plane cluster and configure its &lt;code>istio-ingress&lt;/code> gateway to route traffic from remote clusters to their corresponding Istiod control planes. To learn more about this, check out &lt;a href="https://github.com/istio/istio/wiki/External-Istiod-single-cluster-steps#deploy-istio-mesh-on-external-control-plane-cluster-to-manage-traffic-to-istiod-deployments">these steps&lt;/a>.&lt;/p>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>The external control plane deployment model enables the Istio control plane to be run and managed by mesh operators who have operational expertise in Istio, and provides a clean separation between service mesh control and data planes. Mesh operators can run the control plane in their own clusters or other environments, providing the control plane as a service to mesh admins. Mesh operators can run multiple Istiod control planes in a single cluster, deploying their own Istio mesh and using &lt;code>istio-ingress&lt;/code> gateways to control access to these Istiod control planes. Through the examples provided here, mesh operators can explore different implementation choices and choose what works best for them.&lt;/p>
&lt;p>This new model reduces complexity for mesh admins by allowing them to focus on mesh configurations without operating the control plane themselves. Mesh admins can continue to configure mesh-wide settings and Istio resources without any access to external control plane clusters. Mesh users can continue to interact with the service mesh without any changes.&lt;/p></description><pubDate>Thu, 27 Aug 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/new-deployment-model/</link><author>Lin Sun (IBM), Iris Ding (IBM)</author><guid isPermaLink="true">/v1.12/blog/2020/new-deployment-model/</guid><category>istiod</category><category>deployment model</category><category>install</category><category>deploy</category><category>1.7</category></item><item><title>Introducing the new Istio steering committee</title><description>
&lt;p>Today, the Istio project is pleased to announce a new revision to its steering charter, which opens up governance roles to more contributors and community members. This revision solidifies our commitment to open governance, ensuring that the community around the project will always be able to steer its direction, and that no one company has majority voting control over the project.&lt;/p>
&lt;p>The Istio Steering Committee oversees the administrative aspects of the project and sets the marketing direction. From the earliest days of the project, it was bootstrapped with members from Google and IBM, the two founders and largest contributors, with the explicit intention that other seats would be added. We are very happy to deliver on that promise today, with a new charter designed to reward contribution and community.&lt;/p>
&lt;p>The new Steering Committee consists of 13 seats: 9 proportionally allocated &lt;strong>Contribution Seats&lt;/strong>, and 4 elected &lt;strong>Community Seats&lt;/strong>.&lt;/p>
&lt;h2 id="contribution-seats">Contribution Seats&lt;/h2>
&lt;p>The direction of a project is set by the people who contribute to it. We&amp;rsquo;ve designed our committee to reflect that, with 9 seats to be attributed in proportion to contributions made to Istio in the previous 12 months. In Kubernetes, the mantra was &amp;ldquo;chop wood, carry water,&amp;rdquo; and we similarly want to reward companies who are fueling the growth of the project with contributions.&lt;/p>
&lt;p>This year, we&amp;rsquo;ve chosen to use &lt;strong>merged pull requests&lt;/strong> as our &lt;a href="https://github.com/istio/community/blob/master/steering/CONTRIBUTION-FORMULA.md">proxy for proportional contribution&lt;/a>. We know that no measure of contribution is perfect, and as such we will explicitly reconsider the formula every year. (Other measures we considered, including commits, comments, and actions, gave the same results for this period.)&lt;/p>
&lt;p>In order to ensure corporate diversity, there will always be a minimum of three companies represented in Contribution Seats.&lt;/p>
&lt;h2 id="community-seats">Community Seats&lt;/h2>
&lt;p>There are many wonderful contributors to the Istio community, including developers, SREs and mesh admins, working for companies large and small. We wanted to ensure that their voices were included, both in terms of representation and selection.&lt;/p>
&lt;p>We have added 4 seats for representatives from 4 different organizations, who are not represented in the Contribution Seat allocation. These seats will be voted on by the Istio community in an &lt;a href="https://github.com/istio/community/tree/master/steering/elections">annual election&lt;/a>.&lt;/p>
&lt;p>Any &lt;a href="https://github.com/istio/community/blob/master/ROLES.md#member">project member&lt;/a> can stand for election; all Istio members who have been active in the last 12 months are eligible to vote.&lt;/p>
&lt;h2 id="corporate-diversification-is-the-goal">Corporate diversification is the goal&lt;/h2>
&lt;p>Our goal is that the governance of Istio reflects the diverse set of contributors. Both Google and IBM/Red Hat will have fewer seats than previously, and the new model is designed to ensure representation from at least 7 different organizations.&lt;/p>
&lt;p>We also want to make it clear that no single vendor, no matter how large their contribution, has majority voting control over the Istio project. We&amp;rsquo;ve implemented a cap on the number of seats a company can hold, such that they can neither unanimously win a vote, or veto a decision of the rest of the committee.&lt;/p>
&lt;h2 id="the-2020-committee-and-election">The 2020 committee and election&lt;/h2>
&lt;p>According to our &lt;a href="https://docs.google.com/spreadsheets/d/1Dt-h9s8G7Wyt4r16ZVqcmdWXDuCaPC0kPS21BuAfCL8/edit#gid=0">seat allocation process&lt;/a>, this year Google will be allocated 5 seats and IBM/Red Hat will be allocated 3. As the third largest contributor to Istio in the last 12 months, we are pleased to announce that Salesforce has earned a Contribution Seat.&lt;/p>
&lt;p>The first &lt;a href="https://github.com/istio/community/tree/master/steering/elections/2020">election for Community Seats&lt;/a> begins today. Members have two weeks to nominate themselves, and voting will run from 14 to 27 September. You can learn all about the election in the &lt;code>istio/community&lt;/code> repository on GitHub. We&amp;rsquo;re also hosting a special &lt;a href="http://bit.ly/istiocommunitymeet">community meeting&lt;/a> this Thursday at 10:00 Pacific to discuss the changes and the election process. We&amp;rsquo;d love to see you there!&lt;/p></description><pubDate>Mon, 24 Aug 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/steering-changes/</link><author>Istio Steering Committee</author><guid isPermaLink="true">/v1.12/blog/2020/steering-changes/</guid><category>istio</category><category>steering</category><category>governance</category><category>community</category><category>election</category></item><item><title>Using MOSN with Istio: an alternative data plane</title><description>
&lt;p>&lt;a href="https://github.com/mosn/mosn">MOSN&lt;/a> (Modular Open Smart Network) is a network proxy server written in GoLang. It was built at &lt;a href="https://www.antfin.com">Ant Group&lt;/a> as a sidecar/API Gateway/cloud-native Ingress/Layer 4 or Layer 7 load balancer etc. Over time, we&amp;rsquo;ve added extra features, like a multi-protocol framework, multi-process plug-in mechanism, a DSL, and support for the &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol">xDS APIs&lt;/a>. Supporting xDS means we are now able to use MOSN as the network proxy for Istio. This configuration is not supported by the Istio project; for help, please see &lt;a href="#learn-more">Learn More&lt;/a> below.&lt;/p>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>In the service mesh world, using Istio as the control plane has become the mainstream. Because Istio was built on Envoy, it uses Envoy&amp;rsquo;s data plane &lt;a href="https://blog.envoyproxy.io/the-universal-data-plane-api-d15cec7a">APIs&lt;/a> (collectively known as the xDS APIs). These APIs have been standardized separately from Envoy, and so by implementing them in MOSN, we are able to drop in MOSN as a replacement for Envoy. Istio&amp;rsquo;s integration of third-party data planes can be implemented in three steps, as follows.&lt;/p>
&lt;ul>
&lt;li>Implement xDS protocols to fulfill the capabilities for data plane related services.&lt;/li>
&lt;li>Build &lt;code>proxyv2&lt;/code> images using Istio&amp;rsquo;s script and set the relevant &lt;code>SIDECAR&lt;/code> and other parameters.&lt;/li>
&lt;li>Specify a specific data plane via the &lt;code>istioctl&lt;/code> tool and set the proxy-related configuration.&lt;/li>
&lt;/ul>
&lt;h2 id="architecture">Architecture&lt;/h2>
&lt;p>MOSN has a layered architecture with four layers, NET/IO, Protocol, Stream, and Proxy, as shown in the following figure.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:45.77056778679027%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/mosn-proxy/mosn-arch.png" title="The architecture of MOSN">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/mosn-proxy/mosn-arch.png" alt="The architecture of MOSN" />
&lt;/a>
&lt;/div>
&lt;figcaption>The architecture of MOSN&lt;/figcaption>
&lt;/figure>
&lt;ul>
&lt;li>NET/IO acts as the network layer, monitoring connections and incoming packets, and as a mount point for the listener filter and network filter.&lt;/li>
&lt;li>Protocol is the multi-protocol engine layer that examines packets and uses the corresponding protocol for decode/encode processing.&lt;/li>
&lt;li>Stream does a secondary encapsulation of the decode packet into stream, which acts as a mount for the stream filter.&lt;/li>
&lt;li>Proxy acts as a forwarding framework for MOSN, and does proxy processing on the encapsulated streams.&lt;/li>
&lt;/ul>
&lt;h2 id="why-use-mosn">Why use MOSN?&lt;/h2>
&lt;p>Before the service mesh transformation, we have expected that as the next generation of Ant Group&amp;rsquo;s infrastructure, service mesh will inevitably bring revolutionary changes and evolution costs. We have a very ambitious blueprint: ready to integrate the original network and middleware various capabilities have been re-precipitated and polished to create a low-level platform for the next-generation architecture of the future, which will carry the responsibility of various service communications.&lt;/p>
&lt;p>This is a long-term planning project that takes many years to build and meets the needs of the next five or even ten years, and cooperates to build a team that spans business, SRE, middleware, and infrastructure departments. We must have a network proxy forwarding plane with flexible expansion, high performance, and long-term evolution. Nginx and Envoy have a very long-term capacity accumulation and active community in the field of network agents. We have also borrowed from other excellent open source network agents such as Nginx and Envoy. At the same time, we have enhanced research and development efficiency and flexible expansion. Mesh transformation involves a large number of departments and R &amp;amp; D personnel. We must consider the landing cost of cross-team cooperation. Therefore, we have developed a new network proxy MOSN based on GoLang in the cloud-native scenario. For GoLang&amp;rsquo;s performance, we also did a full investigation and test in the early stage to meet the performance requirements of Ant Group&amp;rsquo;s services.&lt;/p>
&lt;p>At the same time, we received a lot of feedback and needs from the end user community. Everyone has the same needs and thoughts. So we combined the actual situation of the community and ourselves to conduct the research and development of MOSN from the perspective of satisfying the community and users. We believe that the open source competition is mainly competition between standards and specifications. We need to make the most suitable implementation choice based on open source standards.&lt;/p>
&lt;h2 id="what-is-the-difference-between-mosn-and-istio-s-default-proxy">What is the difference between MOSN and Istio&amp;rsquo;s default proxy?&lt;/h2>
&lt;h3 id="differences-in-language-stacks">Differences in language stacks&lt;/h3>
&lt;p>MOSN is written in GoLang. GoLang has strong guarantees in terms of production efficiency and memory security. At the same time, GoLang has an extensive library ecosystem in the cloud-native era. The performance is acceptable and usable in the service mesh scenario. Therefore, MOSN has a lower intellectual cost for companies and individuals using languages such as GoLang and Java.&lt;/p>
&lt;h3 id="differentiation-of-core-competence">Differentiation of core competence&lt;/h3>
&lt;ul>
&lt;li>MOSN supports a multi-protocol framework, and users can easily access private protocols with a unified routing framework.&lt;/li>
&lt;li>Multi-process plug-in mechanism, which can easily extend the plug-ins of independent MOSN processes through the plug-in framework, and do some other management, bypass and other functional module extensions.&lt;/li>
&lt;li>Transport layer national secret algorithm support with Chinese encryption compliance, etc.&lt;/li>
&lt;/ul>
&lt;h3 id="what-are-the-drawbacks-of-mosn">What are the drawbacks of MOSN&lt;/h3>
&lt;ul>
&lt;li>Because MOSN is written in GoLang, it doesn&amp;rsquo;t have as good performance as Istio default proxy, but the performance is acceptable and usable in the service mesh scenario.&lt;/li>
&lt;li>Compared with Istio default proxy, some features are not fully supported, such as WASM, HTTP3, Lua, etc. However, these are all in the &lt;a href="https://docs.google.com/document/d/12lgyCW-GmlErr_ihvAO7tMmRe87i70bv2xqe4h2LUz4/edit?usp=sharing">roadmap&lt;/a> of MOSN, and the goal is to be fully compatible with Istio.&lt;/li>
&lt;/ul>
&lt;h2 id="mosn-with-istio">MOSN with Istio&lt;/h2>
&lt;p>The following describes how to set up MOSN as the data plane for Istio.&lt;/p>
&lt;h2 id="setup-istio">Setup Istio&lt;/h2>
&lt;p>You can download a zip file for your operating system from the &lt;a href="https://github.com/istio/istio/releases/tag/1.5.2">Istio release&lt;/a> page. This file contains: the installation file, examples and the &lt;code>istioctl&lt;/code> command line tool.
To download Istio (this example uses Istio 1.5.2) uses the following command.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export ISTIO_VERSION=1.5.2
$ curl -L https://istio.io/downloadIstio | sh -
&lt;/code>&lt;/pre>
&lt;p>The downloaded Istio package is named &lt;code>istio-1.5.2&lt;/code> and contains:
- &lt;code>install/kubernetes&lt;/code>: Contains YAML installation files related to Kubernetes.
- &lt;code>examples/&lt;/code>: Contains example applications.
- &lt;code>bin/&lt;/code>: Contains the istioctl client files.&lt;/p>
&lt;p>Switch to the folder where Istio is located.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cd istio-$ISTIO_VERSION/
&lt;/code>&lt;/pre>
&lt;p>Add the &lt;code>istioctl&lt;/code> client path to &lt;code>$PATH&lt;/code> with the following command.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export PATH=$PATH:$(pwd)/bin
&lt;/code>&lt;/pre>
&lt;h2 id="setting-mosn-as-the-data-plane">Setting MOSN as the Data Plane&lt;/h2>
&lt;p>It is possible to flexibly customize the Istio control plane and data plane configuration parameters using the &lt;code>istioctl&lt;/code> command line tool. MOSN can be specified as the data plane for Istio using the following command.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl manifest apply --set .values.global.proxy.image=&amp;#34;mosnio/proxyv2:1.5.2-mosn&amp;#34; --set meshConfig.defaultConfig.binaryPath=&amp;#34;/usr/local/bin/mosn&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>Check that Istio-related pods and services are deployed successfully.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get svc -n istio-system
&lt;/code>&lt;/pre>
&lt;p>If the service &lt;code>STATUS&lt;/code> is Running, then Istio has been successfully installed using MOSN and you can now deploy the Bookinfo sample.&lt;/p>
&lt;h2 id="bookinfo-examples">Bookinfo Examples&lt;/h2>
&lt;p>You can run the Bookinfo sample by following the &lt;a href="https://katacoda.com/mosn/courses/istio/mosn-with-istio">MOSN with Istio tutorial&lt;/a> where you can find instructions for using MOSN and Istio. You can install MOSN and get to the same point you would have using the default Istio instructions with Envoy.&lt;/p>
&lt;h2 id="moving-forward">Moving forward&lt;/h2>
&lt;p>Next, MOSN will not only be compatible with the features of the latest version of Istio, but also evolve in the following aspects.&lt;/p>
&lt;ul>
&lt;li>&lt;em>As a microservices runtime&lt;/em>, MOSN oriented programming makes services lighter, smaller and faster.&lt;/li>
&lt;li>&lt;em>Programmable&lt;/em>, support WASM.&lt;/li>
&lt;li>&lt;em>More scenario support&lt;/em>, Cache Mesh/Message Mesh/Block-chain Mesh etc.&lt;/li>
&lt;/ul>
&lt;p>MOSN is an open source project that anyone in the community can use, improve, and enjoy. We&amp;rsquo;d love you to join us! &lt;a href="https://github.com/mosn/community">Here&lt;/a> are a few ways to find out what&amp;rsquo;s happening and get involved.&lt;/p>
&lt;h2 id="learn-more">Learn More&lt;/h2>
&lt;ul>
&lt;li>&lt;a href="https://mosn.io/en">MOSN website&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://mosn.io/en/docs/community/">MOSN community&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://katacoda.com/mosn">MOSN tutorials&lt;/a>&lt;/li>
&lt;/ul></description><pubDate>Tue, 28 Jul 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/mosn-proxy/</link><author>Wang Fakang (mosn.io)</author><guid isPermaLink="true">/v1.12/blog/2020/mosn-proxy/</guid><category>mosn</category><category>sidecar</category><category>proxy</category></item><item><title>Open and neutral: transferring our trademarks to the Open Usage Commons</title><description>
&lt;p>Since &lt;a href="/v1.12/news/releases/0.x/announcing-0.1/">day one&lt;/a>, the Istio project has believed in the importance of being contributor-run, open, transparent and available to all. In that spirit, Google is pleased to announce that it will be transferring ownership of the project’s trademarks to the new Open Usage Commons.&lt;/p>
&lt;p>Istio is an open source project, released under the Apache 2.0 license. That means people can copy, modify, distribute, make, use and sell the source code. The only freedom people don&amp;rsquo;t have under the Apache 2.0 license is to use the name Istio, or its logo, in a way that would confuse consumers.&lt;/p>
&lt;p>As one of the founders of the project, Google is the current owner of the Istio trademark. While anyone who is using the software in accordance with the license can use the trademarks, the historic ownership has caused some confusion and uncertainty about who can use the name and how, and at times this confusion has been a barrier to community growth. So today, as part of Istio’s continued commitment to openness, Google is announcing that the Istio trademarks will be transferred to a new organization, the Open Usage Commons, to provide neutral, independent oversight of the marks.&lt;/p>
&lt;h2 id="a-neutral-home-for-istio-s-trademarks">A neutral home for Istio’s trademarks&lt;/h2>
&lt;p>The Open Usage Commons is a new organization that is focused solely on providing management and guidance of open source project trademarks in a way that is aligned with the &lt;a href="https://opensource.org/osd">Open Source Definition&lt;/a>. For projects, particularly projects with robust ecosystems like Istio, ensuring that the trademark is available to anyone who is using the software in accordance with the license is important. The trademark allows maintainers to grow a community and use the name to do so. It also lets ecosystem partners create services on top of the project, and it enables developers to create tooling and integrations that reference the project. Maintainers, ecosystem partners, and developers alike must feel confident in their investments in Istio - for the long term. Google thinks having the Istio trademarks in the Open Usage Commons is the right way to give that clarity and provide that confidence.&lt;/p>
&lt;p>The Open Usage Commons will work with the Istio Steering Committee to generate trademark usage guidelines. There will be no immediate changes to the Istio usage guidelines, and if you are currently using the Istio marks in a way that follows the existing brand guide, you can continue to do so.&lt;/p>
&lt;p>You can learn more about &lt;a href="https://openusage.org/faq">open source project IP and the Open Usage Commons&lt;/a> at &lt;a href="https://openusage.org">openusage.org&lt;/a>.&lt;/p>
&lt;h2 id="a-continued-commitment-to-open">A continued commitment to open&lt;/h2>
&lt;p>The Open Usage Commons is focused on project trademarks; it does not address other facets of an open project, like rules around who gets decision-making votes. Similar to many projects in their early days, Istio’s committees started as small groups that stemmed from the founding companies. But Istio has grown and matured (last year Istio was &lt;a href="https://octoverse.github.com/#fastest-growing-oss-projects-by-contributors">#4 on GitHub&amp;rsquo;s list of fastest growing open source projects!&lt;/a>), and it is time for the next evolution of Istio’s governance.&lt;/p>
&lt;p>Recently, &lt;a href="https://aspenmesh.io/helping-istio-sail/">we were proud to appoint Neeraj Poddar, Co-founder &amp;amp; Chief Architect of Aspen Mesh&lt;/a>, to the Technical Oversight Committee — the group responsible for all technical decision-making in the project. Neeraj is a long-time contributor to the project and served as a Working Group lead. The &lt;a href="https://github.com/istio/community/blob/master/TECH-OVERSIGHT-COMMITTEE.md#committee-members">TOC is now made up of&lt;/a> 7 members from 4 different companies - Tetrate, IBM, Google &amp;amp; now Aspen Mesh.&lt;/p>
&lt;p>Our community is currently discussing how the Steering Committee, which oversees marketing and community activities, should be governed, to reflect the expanding community and ecosystem. If you have ideas for this new governance, visit the &lt;a href="https://github.com/istio/community/pull/361">pull request on GitHub&lt;/a> where an active discussion is taking place.&lt;/p>
&lt;p>In the last 12 months, Istio has had commits from &lt;a href="https://istio.teststats.cncf.io/d/5/companies-table?var-period_name=Last%20year&amp;amp;var-metric=commits">more than 100 organizations&lt;/a> and currently has &lt;a href="http://eng.istio.io/maintainers">70 maintainers from 14 different companies&lt;/a>. This trend is the kind of contributor diversity the project’s founders intended, and nurturing it remains a priority. Google is excited about what the future holds for Istio, and hopes you’ll be a part of it.&lt;/p></description><pubDate>Wed, 08 Jul 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/open-usage/</link><author>Sean Suchter (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/open-usage/</guid><category>trademark</category><category>governance</category><category>steering</category></item><item><title>Reworking our Addon Integrations</title><description>
&lt;p>Starting with Istio 1.6, we are introducing a new method for integration with telemetry addons, such as Grafana, Prometheus, Zipkin, Jaeger, and Kiali.&lt;/p>
&lt;p>In previous releases, these addons were bundled as part of the Istio installation. This allowed users to quickly get started with Istio without any complicated configurations to install and integrate these addons. However, it came with some issues:&lt;/p>
&lt;ul>
&lt;li>The Istio addon installations were not as up to date or feature rich as upstream installation methods. Users were left missing out on some of the great features provided by these applications, such as:
&lt;ul>
&lt;li>Persistent storage&lt;/li>
&lt;li>Features like &lt;code>Alertmanager&lt;/code> for Prometheus&lt;/li>
&lt;li>Advanced security settings&lt;/li>
&lt;/ul>&lt;/li>
&lt;li>Integration with existing deployments that were using these features was more challenging than it should be.&lt;/li>
&lt;/ul>
&lt;h2 id="changes">Changes&lt;/h2>
&lt;p>In order to address these gaps, we have made a number of changes:&lt;/p>
&lt;ul>
&lt;li>&lt;p>Added a new &lt;a href="/v1.12/docs/ops/integrations/">Integrations&lt;/a> documentation section to explain which applications Istio can integrate with, how to use them, and best practices.&lt;/p>&lt;/li>
&lt;li>&lt;p>Reduced the amount of configuration required to set up telemetry addons&lt;/p>
&lt;ul>
&lt;li>&lt;p>Grafana dashboards are now &lt;a href="/v1.12/docs/ops/integrations/grafana/#import-from-grafana-com">published to &lt;code>grafana.com&lt;/code>&lt;/a>.&lt;/p>&lt;/li>
&lt;li>&lt;p>Prometheus can now scrape all Istio pods &lt;a href="/v1.12/docs/ops/integrations/prometheus/#option-2-metrics-merging">using standard &lt;code>prometheus.io&lt;/code> annotations&lt;/a>. This allows most Prometheus deployments to work with Istio without any special configuration.&lt;/p>&lt;/li>
&lt;/ul>&lt;/li>
&lt;li>&lt;p>Removed the bundled addon installations from &lt;code>istioctl&lt;/code> and the operator. Istio does not install components that are not delivered by the Istio project. As a result, Istio will stop shipping installation artifacts related to addons. However, Istio will guarantee version compatibility where necessary. It is the user&amp;rsquo;s responsibility to install these components by using the official &lt;a href="/v1.12/docs/ops/integrations/">Integrations&lt;/a> documentation and artifacts provided by the respective projects. For demos, users can deploy simple YAML files from the &lt;a href="https://github.com/istio/istio/tree/release-1.12/samples/addons">&lt;code>samples/addons/&lt;/code> directory&lt;/a>.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>We hope these changes allow users to make the most of these addons so as to fully experience what Istio can offer.&lt;/p>
&lt;h2 id="timeline">Timeline&lt;/h2>
&lt;ul>
&lt;li>Istio 1.6: The new demo deployments for telemetry addons are available under &lt;code>samples/addons/&lt;/code> directory.&lt;/li>
&lt;li>Istio 1.7: Upstream installation methods or the new samples deployment are the recommended installation methods. Installation by &lt;code>istioctl&lt;/code> is deprecated.&lt;/li>
&lt;li>Istio 1.8: Installation of addons by &lt;code>istioctl&lt;/code> is removed.&lt;/li>
&lt;/ul></description><pubDate>Thu, 04 Jun 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/addon-rework/</link><author>John Howard (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/addon-rework/</guid><category>telemetry</category><category>addons</category><category>integrations</category><category>grafana</category><category>prometheus</category></item><item><title>Introducing Workload Entries</title><description>
&lt;h2 id="introducing-workload-entries-bridging-kubernetes-and-vms">Introducing Workload Entries: Bridging Kubernetes and VMs&lt;/h2>
&lt;p>Historically, Istio has provided great experience to workloads that run on Kubernetes, but it has been less smooth for other types of workloads, such as Virtual Machines (VMs) and bare metal. The gaps included the inability to declaratively specify the properties of a sidecar on a VM, inability to properly respond to the lifecycle changes of the workload (e.g., booting to not ready to ready, or health checks), and cumbersome DNS workarounds as the workloads are migrated into Kubernetes to name a few.&lt;/p>
&lt;p>Istio 1.6 has introduced a few changes in how you manage non-Kubernetes workloads, driven by a desire to make it easier to gain Istio&amp;rsquo;s benefits for use cases beyond containers, such as running traditional databases on a platform outside of Kubernetes, or adopting Istio&amp;rsquo;s features for existing applications without rewriting them.&lt;/p>
&lt;h3 id="background">Background&lt;/h3>
&lt;p>Prior to Istio 1.6, non-containerized workloads were configurable simply as an IP address in a &lt;code>ServiceEntry&lt;/code>, which meant that they only existed as part of a service. Istio lacked a first-class abstraction for these non-containerized workloads, something similar to how Kubernetes treats Pods as the fundamental unit of compute - a named object that serves as the collection point for all things related to a workload - name, labels, security properties, lifecycle status events, etc. Enter &lt;code>WorkloadEntry&lt;/code>.&lt;/p>
&lt;p>Consider the following &lt;code>ServiceEntry&lt;/code> describing a service implemented by a few tens of VMs with IP addresses:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: svc1
spec:
hosts:
- svc1.internal.com
ports:
- number: 80
name: http
protocol: HTTP
resolution: STATIC
endpoints:
- address: 1.1.1.1
- address: 2.2.2.2
....
&lt;/code>&lt;/pre>
&lt;p>If you wanted to migrate this service into Kubernetes in an active-active manner - i.e. launch a bunch of Pods, send a portion of the traffic to the Pods over Istio mutual TLS (mTLS) and send the rest to the VMs without sidecars - how would you do it? You would have needed to use a combination of a Kubernetes service, a virtual service, and a destination rule to achieve the behavior. Now, let&amp;rsquo;s say you decided to add sidecars to these VMs, one by one, such that you want only the traffic to the VMs with sidecars to use Istio mTLS. If any other Service Entry happens to include the same VM in its addresses, things start to get very complicated and error prone.&lt;/p>
&lt;p>The primary source of these complications is that Istio lacked a first-class definition of a non-containerized workload, whose properties can be described independently of the service(s) it is part of.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/workload-entry/workload-entry-first-example.svg" title="The Internal of Service Entries Pointing to Workload Entries">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/workload-entry/workload-entry-first-example.svg" alt="Service Entries Pointing to Workload Entries" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Internal of Service Entries Pointing to Workload Entries&lt;/figcaption>
&lt;/figure>
&lt;h3 id="workload-entry-a-non-kubernetes-endpoint">Workload Entry: A Non-Kubernetes Endpoint&lt;/h3>
&lt;p>&lt;code>WorkloadEntry&lt;/code> was created specifically to solve this problem. &lt;code>WorkloadEntry&lt;/code> allows you to describe non-Pod endpoints that should still be part of the mesh, and treat them the same as a Pod. From here everything becomes easier, like enabling &lt;code>MUTUAL_TLS&lt;/code> between workloads, whether they are containerized or not.&lt;/p>
&lt;p>To create a &lt;a href="/v1.12/docs/reference/config/networking/workload-entry/">&lt;code>WorkloadEntry&lt;/code>&lt;/a> and attach it to a &lt;a href="/v1.12/docs/reference/config/networking/service-entry/">&lt;code>ServiceEntry&lt;/code>&lt;/a> you can do something like this:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >---
apiVersion: networking.istio.io/v1alpha3
kind: WorkloadEntry
metadata:
name: vm1
namespace: ns1
spec:
address: 1.1.1.1
labels:
app: foo
instance-id: vm-78ad2
class: vm
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: svc1
namespace: ns1
spec:
hosts:
- svc1.internal.com
ports:
- number: 80
name: http
protocol: HTTP
resolution: STATIC
workloadSelector:
labels:
app: foo
&lt;/code>&lt;/pre>
&lt;p>This creates a new &lt;code>WorkloadEntry&lt;/code> with a set of labels and an address, and a &lt;code>ServiceEntry&lt;/code> that uses a &lt;code>WorkloadSelector&lt;/code> to select all endpoints with the desired labels, in this case including the &lt;code>WorkloadEntry&lt;/code> that are created for the VM.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/workload-entry/workload-entry-final.svg" title="The Internal of Service Entries Pointing to Workload Entries">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/workload-entry/workload-entry-final.svg" alt="Service Entries Pointing to Workload Entries" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Internal of Service Entries Pointing to Workload Entries&lt;/figcaption>
&lt;/figure>
&lt;p>Notice that the &lt;code>ServiceEntry&lt;/code> can reference both Pods and &lt;code>WorkloadEntries&lt;/code>, using the same selector. VMs and Pods can now be treated identically by Istio, rather than being kept separate.&lt;/p>
&lt;p>If you were to migrate some of your workloads to Kubernetes, and you choose to keep a substantial number of your VMs, the &lt;code>WorkloadSelector&lt;/code> can select both Pods and VMs, and Istio will automatically load balance between them. The 1.6 changes also mean that &lt;code>WorkloadSelector&lt;/code> syncs configurations between the Pods and VMs and removes the manual requirement to target both infrastructures with duplicate policies like mTLS and authorization.
The Istio 1.6 release provides a great starting point for what will be possible for the future of Istio. The ability to describe what exists outside of the mesh the same way you do with a Pod leads to added benefits like improved bootstrapping experience. However, these benefits are merely side effects. The core benefit is you can now have VMs, and Pods co-exist without any configuration needed to bridge the two together.&lt;/p></description><pubDate>Thu, 21 May 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/workload-entry/</link><author>Cynthia Coan (Tetrate), Shriram Rajagopalan (Tetrate), Tia Louden (Tetrate), John Howard (Google), Sven Mawson (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/workload-entry/</guid><category>vm</category><category>workloadentry</category><category>migration</category><category>1.6</category><category>baremetal</category><category>serviceentry</category><category>discovery</category></item><item><title>Safely Upgrade Istio using a Canary Control Plane Deployment</title><description>
&lt;p>Canary deployments are a core feature of Istio. Users rely on Istio&amp;rsquo;s traffic management features to safely control the rollout of new versions of their applications, while making use of Istio&amp;rsquo;s rich telemetry to compare the performance of canaries. However, when it came to upgrading Istio, there was not an easy way to canary the upgrade, and due to the in-place nature of the upgrade, issues or changes found affect the entire mesh at once.&lt;/p>
&lt;p>Istio 1.6 will support a new upgrade model to safely canary-deploy new versions of Istio. In this new model, proxies will associate with a specific control plane that they use. This allows a new version to deploy to the cluster with less risk - no proxies connect to the new version until the user explicitly chooses to. This allows gradually migrating workloads to the new control plane, while monitoring changes using Istio telemetry to investigate any issues, just like using &lt;code>VirtualService&lt;/code> for workloads. Each independent control plane is referred to as a &amp;ldquo;revision&amp;rdquo; and has an &lt;code>istio.io/rev&lt;/code> label.&lt;/p>
&lt;h2 id="understanding-upgrades">Understanding upgrades&lt;/h2>
&lt;p>Upgrading Istio is a complicated process. During the transition period between two versions, which might take a long time for large clusters, there are version differences between proxies and the control plane. In the old model the old and new control planes use the same Service, traffic is randomly distributed between the two, offering no control to the user. However, in the new model, there is not cross-version communication. Look at how the upgrade changes:&lt;/p>
&lt;iframe src="https://docs.google.com/presentation/d/e/2PACX-1vR2R_Nd1XsjriBfwbqmcBc8KtdP4McDqNpp8S5v6woq28FnsW-kATBrKtLEG9k61DuBwTgFKLWyAxuK/embed?start=false&amp;loop=true&amp;delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true">&lt;/iframe>
&lt;h2 id="configuring">Configuring&lt;/h2>
&lt;p>Control plane selection is done based on the sidecar injection webhook. Each control plane is configured to select objects with a matching &lt;code>istio.io/rev&lt;/code> label on the namespace. Then, the upgrade process configures the pods to connect to a control plane specific to that revision. Unlike in the current model, this means that a given proxy connects to the same revision during its lifetime. This avoids subtle issues that might arise when a proxy switches which control plane it is connected to.&lt;/p>
&lt;p>The new &lt;code>istio.io/rev&lt;/code> label will replace the &lt;code>istio-injection=enabled&lt;/code> label when using revisions. For example, if we had a revision named canary, we would label our namespaces that we want to use this revision with istio.io/rev=canary. See the &lt;a href="/v1.12/docs/setup/upgrade">upgrade guide&lt;/a> for more information.&lt;/p></description><pubDate>Tue, 19 May 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/multiple-control-planes/</link><author>John Howard (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/multiple-control-planes/</guid><category>install</category><category>upgrade</category><category>revision</category><category>control plane</category></item><item><title>Direct encrypted traffic from IBM Cloud Kubernetes Service Ingress to Istio Ingress Gateway</title><description>
&lt;p>In this blog post I show how to configure the &lt;a href="https://cloud.ibm.com/docs/containers?topic=containers-ingress-about">Ingress Application Load Balancer (ALB)&lt;/a>
on &lt;a href="https://www.ibm.com/cloud/kubernetes-service/">IBM Cloud Kubernetes Service (IKS)&lt;/a> to direct traffic to the Istio
ingress gateway, while securing the traffic between them using &lt;span class="term" data-title="Mutual TLS Authentication" data-body="&amp;lt;p&amp;gt;Mutual TLS provides strong service-to-service authentication with built-in identity and credential management.
&amp;lt;a href=&amp;#34;/docs/concepts/security/#mutual-tls-authentication&amp;#34;&amp;gt;Learn more about mutual TLS authentication&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">mutual TLS authentication&lt;/span>.&lt;/p>
&lt;p>When you use IKS without Istio, you may control your ingress traffic using the provided ALB. This ingress-traffic
routing is configured using a Kubernetes
&lt;a href="https://kubernetes.io/docs/concepts/services-networking/ingress/">Ingress&lt;/a> resource with
&lt;a href="https://cloud.ibm.com/docs/containers?topic=containers-ingress_annotation">ALB-specific annotations&lt;/a>. IKS provides a
DNS domain name, a TLS certificate that matches the domain, and a private key for the certificate. IKS stores the
certificates and the private key in a &lt;a href="https://kubernetes.io/docs/concepts/configuration/secret/">Kubernetes secret&lt;/a>.&lt;/p>
&lt;p>When you start using Istio in your IKS cluster, the recommended method to send traffic to your Istio enabled workloads
is by using the &lt;a href="/v1.12/docs/tasks/traffic-management/ingress/ingress-control/">Istio Ingress Gateway&lt;/a> instead of using the
&lt;a href="https://kubernetes.io/docs/concepts/services-networking/ingress/">Kubernetes Ingress&lt;/a>. One of the main reasons to use
the Istio ingress gateway is the fact the ALB provided by IKS will not be able to communicate directly with the services
inside the mesh when you enable STRICT mutual TLS. During your transition to having only Istio ingress gateway as your
main entry point, you can continue to use the traditional Ingress for non-Istio services while using the Istio ingress
gateway for services that are part of the mesh.&lt;/p>
&lt;p>IKS provides a convenient way for clients to access Istio ingress gateway by letting you
&lt;a href="https://cloud.ibm.com/docs/containers?topic=containers-loadbalancer_hostname">register a new DNS subdomain&lt;/a> for the
Istio gateway&amp;rsquo;s IP with an IKS command. The domain is in the following
&lt;a href="https://cloud.ibm.com/docs/containers?topic=containers-loadbalancer_hostname#loadbalancer_hostname_format">format&lt;/a>:
&lt;code>&amp;lt;cluster_name&amp;gt;-&amp;lt;globally_unique_account_HASH&amp;gt;-0001.&amp;lt;region&amp;gt;.containers.appdomain.cloud&lt;/code>, for example &lt;code>mycluster-a1b2cdef345678g9hi012j3kl4567890-0001.us-south.containers.appdomain.cloud&lt;/code>. In the same way as for the ALB domain,
IKS provides a certificate and a private key, storing them in another Kubernetes secret.&lt;/p>
&lt;p>This blog describes how you can chain together the IKS Ingress ALB and the Istio ingress gateway to send traffic to your
Istio enabled workloads while being able to continue using the ALB specific features and the ALB subdomain name. You
configure the IKS Ingress ALB to direct traffic to the services inside an Istio service mesh through the Istio ingress
gateway, while using mutual TLS authentication between the ALB and the gateway. For the mutual TLS authentication, you
will configure the ALB and the Istio ingress gateway to use the certificates and keys provided by IKS for the ALB and
NLB subdomains. Using certificates provided by IKS saves you the overhead of managing your own certificates for the
connection between the ALB and the Istio ingress gateway.&lt;/p>
&lt;p>You will use the NLB subdomain certificate as the server certificate for the Istio ingress gateway as intended.
The NLB subdomain certificate represents the identity of the server that serves a particular NLB subdomain, in this
case, the ingress gateway.&lt;/p>
&lt;p>You will use the ALB subdomain certificate as the client certificate in mutual TLS authentication between the ALB and
the Istio Ingress. When ALB acts as a server it presents the ALB certificate to the clients so the clients can
authenticate the ALB. When ALB acts as a client of the Istio ingress gateway, it presents the same certificate to the
Istio ingress gateway, so the Istio ingress gateway could authenticate the ALB.&lt;/p>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Note that the instructions in this blog post only configure the ALB and the Istio ingress gateway to encrypt the traffic
between them and to verify that they receive valid certificates issued by &lt;a href="https://letsencrypt.org">Let&amp;rsquo;s Encrypt&lt;/a>. In
order to specify that only the ALB is allowed to talk to the Istio ingress gateway, an additional Istio security policy
must be defined. In order to verify that the ALB indeed talks to the Istio ingress gateway, additional configuration
must be added to the ALB. The additional configuration of the Istio ingress gateway and the ALB is out of scope for this
blog.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Traffic to the services without an Istio sidecar can continue to flow as before directly from the ALB.&lt;/p>
&lt;p>The diagram below exemplifies the described setting. It shows two services in the cluster, &lt;code>service A&lt;/code> and &lt;code>service B&lt;/code>.
&lt;code>service A&lt;/code> has an Istio sidecar injected and requires mutual TLS. &lt;code>service B&lt;/code> has no Istio sidecar. &lt;code>service B&lt;/code> can
be accessed by clients through the ALB, which directly communicates with &lt;code>service B&lt;/code>. &lt;code>service A&lt;/code> can be also
accessed by clients through the ALB, but in this case the traffic must pass through the Istio ingress gateway. Mutual
TLS authentication between the ALB and the gateway is based on the certificates provided by IKS.
The clients can also access the Istio ingress gateway directly. IKS registers different DNS domains for the ALB and for
the ingress gateway.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:63.32720606343596%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/alb-ingress-gateway-iks/alb-ingress-gateway.svg" title="A cluster with the ALB and the Istio ingress gateway">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/alb-ingress-gateway-iks/alb-ingress-gateway.svg" alt="A cluster with the ALB and the Istio ingress gateway" />
&lt;/a>
&lt;/div>
&lt;figcaption>A cluster with the ALB and the Istio ingress gateway&lt;/figcaption>
&lt;/figure>
&lt;h2 id="initial-setting">Initial setting&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Create the &lt;code>httptools&lt;/code> namespace and enable Istio sidecar injection:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create namespace httptools
$ kubectl label namespace httptools istio-injection=enabled
namespace/httptools created
namespace/httptools labeled
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Deploy the &lt;code>httpbin&lt;/code> sample to &lt;code>httptools&lt;/code>:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/httpbin/httpbin.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/httpbin/httpbin.yaml@ -n httptools
service/httpbin created
deployment.apps/httpbin created
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;/ol>
&lt;h2 id="create-secrets-for-the-alb-and-the-istio-ingress-gateway">Create secrets for the ALB and the Istio ingress gateway&lt;/h2>
&lt;p>IKS generates a TLS certificate and a private key and stores them as a secret in the default namespace when you register
a DNS domain for an external IP by using the &lt;code>ibmcloud ks nlb-dns-create&lt;/code> command. IKS stores the ALB&amp;rsquo;s
certificate and private key also as a secret in the default namespace. You need these credentials to establish the
identities that the ALB and the Istio ingress gateway will present during the mutual TLS authentication between
them. You will configure the ALB and the Istio ingress gateway to exchange these certificates, to trust the certificates
of one another, and to use their private keys to encrypt and sign the traffic.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Store the name of your cluster in the &lt;code>CLUSTER_NAME&lt;/code> environment variable:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export CLUSTER_NAME=&amp;lt;your cluster name&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Store the domain name of your ALB in the &lt;code>ALB_INGRESS_DOMAIN&lt;/code> environment variable:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ ibmcloud ks cluster get --cluster $CLUSTER_NAME | grep Ingress
Ingress Subdomain: &amp;lt;your ALB ingress domain&amp;gt;
Ingress Secret: &amp;lt;your ALB secret&amp;gt;
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export ALB_INGRESS_DOMAIN=&amp;lt;your ALB ingress domain&amp;gt;
$ export ALB_SECRET=&amp;lt;your ALB secret&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Store the external IP of your &lt;code>istio-ingressgateway&lt;/code> service in an environment variable.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export INGRESS_GATEWAY_IP=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath=&amp;#39;{.status.loadBalancer.ingress[0].ip}&amp;#39;)
$ echo INGRESS_GATEWAY_IP = $INGRESS_GATEWAY_IP
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a DNS domain and certificates for the IP of the Istio Ingress Gateway service:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ ibmcloud ks nlb-dns create classic --cluster $CLUSTER_NAME --ip $INGRESS_GATEWAY_IP --secret-namespace istio-system
Host name subdomain is created as &amp;lt;some domain&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Store the domain name from the previous command in an environment variable:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export INGRESS_GATEWAY_DOMAIN=&amp;lt;the domain from the previous command&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>List the registered domain names:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ ibmcloud ks nlb-dnss --cluster $CLUSTER_NAME
Retrieving host names, certificates, IPs, and health check monitors for network load balancer (NLB) pods in cluster &amp;lt;your cluster&amp;gt;...
OK
Hostname IP(s) Health Monitor SSL Cert Status SSL Cert Secret Name Secret Namespace
&amp;lt;your ingress gateway hostname&amp;gt; &amp;lt;your ingress gateway IP&amp;gt; None created &amp;lt;the matching secret name&amp;gt; istio-system
...
&lt;/code>&lt;/pre>
&lt;p>Wait until the status of the certificate (the fourth field) of the new domain name becomes &lt;code>enabled&lt;/code> (initially it is &lt;code>pending&lt;/code>).&lt;/p>&lt;/li>
&lt;li>&lt;p>Store the name of the secret of the new domain name:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export INGRESS_GATEWAY_SECRET=&amp;lt;the secret&amp;#39;s name as shown in the SSL Cert Secret Name column&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Extract the certificate and the key from the secret provided for the ALB:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mkdir alb_certs
$ kubectl get secret $ALB_SECRET --namespace=default -o yaml | grep &amp;#39;tls.key:&amp;#39; | cut -f2 -d: | base64 --decode &amp;gt; alb_certs/client.key
$ kubectl get secret $ALB_SECRET --namespace=default -o yaml | grep &amp;#39;tls.crt:&amp;#39; | cut -f2 -d: | base64 --decode &amp;gt; alb_certs/client.crt
$ ls -al alb_certs
-rw-r--r-- 1 user staff 3738 Sep 11 07:57 client.crt
-rw-r--r-- 1 user staff 1675 Sep 11 07:57 client.key
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Download the issuer certificate of the &lt;a href="https://letsencrypt.org">Let&amp;rsquo;s Encrypt&lt;/a> certificate, which is the
issuer of the certificates provided by IKS. You specify this certificate as the certificate of a certificate
authority to trust, for both the ALB and the Istio ingress gateway.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl https://letsencrypt.org/certs/trustid-x3-root.pem --output trusted.crt
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a Kubernetes secret to be used by the ALB to establish mutual TLS connection.&lt;/p>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">The certificates provided by IKS expire every 90 days and are automatically renewed by
IKS 37 days before they expire.
You will have to recreate the secrets by rerunning the instructions of this section every time the secrets provided
by IKS are updated. You may want to use scripts or operators to automate this and keep the
secrets in sync.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create secret generic alb-certs -n istio-system --from-file=trusted.crt --from-file=alb_certs/client.crt --from-file=alb_certs/client.key
secret &amp;#34;alb-certs&amp;#34; created
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>For mutual TLS, a separate Secret named &lt;code>&amp;lt;tls-cert-secret&amp;gt;-cacert&lt;/code> with a &lt;code>cacert&lt;/code> key is needed for the ingress gateway.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create -n istio-system secret generic $INGRESS_GATEWAY_SECRET-cacert --from-file=ca.crt=trusted.crt
secret/cluster_name-hash-XXXX-cacert created
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h2 id="configure-a-mutual-tls-ingress-gateway">Configure a mutual TLS ingress gateway&lt;/h2>
&lt;p>In this section you configure the Istio ingress gateway to perform mutual TLS between external clients and the gateway.
You use the certificates and the keys provided to you for the ingress gateway and the ALB.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Define a &lt;code>Gateway&lt;/code> to allow access on port 443 only, with mutual TLS:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -n httptools -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: default-ingress-gateway
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: MUTUAL
credentialName: $INGRESS_GATEWAY_SECRET
hosts:
- &amp;#34;$INGRESS_GATEWAY_DOMAIN&amp;#34;
- &amp;#34;httpbin.$ALB_INGRESS_DOMAIN&amp;#34;
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Configure routes for traffic entering via the &lt;code>Gateway&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -n httptools -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: default-ingress
spec:
hosts:
- &amp;#34;$INGRESS_GATEWAY_DOMAIN&amp;#34;
- &amp;#34;httpbin.$ALB_INGRESS_DOMAIN&amp;#34;
gateways:
- default-ingress-gateway
http:
- match:
- uri:
prefix: /status
route:
- destination:
port:
number: 8000
host: httpbin.httptools.svc.cluster.local
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Send a request to &lt;code>httpbin&lt;/code> by &lt;em>curl&lt;/em>, passing as parameters the client certificate
(the &lt;code>--cert&lt;/code> option) and the private key (the &lt;code>--key&lt;/code> option):&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl https://$INGRESS_GATEWAY_DOMAIN/status/418 --cert alb_certs/client.crt --key alb_certs/client.key
-=[ teapot ]=-
_...._
.&amp;#39; _ _ `.
| .&amp;#34;` ^ `&amp;#34;. _,
\_;`&amp;#34;---&amp;#34;`|//
| ;/
\_ _/
`&amp;#34;&amp;#34;&amp;#34;`
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Remove the directories with the ALB and ingress gateway certificates and keys.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ rm -r alb_certs trusted.crt
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h2 id="configure-the-alb">Configure the ALB&lt;/h2>
&lt;p>You need to configure your Ingress resource to direct traffic to the Istio ingress gateway while using the certificate
stored in the &lt;code>alb-certs&lt;/code> secret. Normally, the ALB decrypts HTTPS requests before forwarding traffic to your apps.
You can configure the ALB to re-encrypt the traffic before it is forwarded to the Istio ingress gateway by using the
&lt;code>ssl-services&lt;/code> annotation on the Ingress resource. This annotation also allows you to specify the certificate stored in
the &lt;code>alb-certs&lt;/code> secret, required for mutual TLS.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Configure the &lt;code>Ingress&lt;/code> resource for the ALB. You must create the &lt;code>Ingress&lt;/code> resource in the &lt;code>istio-system&lt;/code> namespace
in order to forward the traffic to the Istio ingress gateway.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: alb-ingress
namespace: istio-system
annotations:
ingress.bluemix.net/ssl-services: &amp;#34;ssl-service=istio-ingressgateway ssl-secret=alb-certs proxy-ssl-name=$INGRESS_GATEWAY_DOMAIN&amp;#34;
spec:
tls:
- hosts:
- httpbin.$ALB_INGRESS_DOMAIN
secretName: $ALB_SECRET
rules:
- host: httpbin.$ALB_INGRESS_DOMAIN
http:
paths:
- path: /status
backend:
serviceName: istio-ingressgateway
servicePort: 443
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Test the ALB ingress:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl https://httpbin.$ALB_INGRESS_DOMAIN/status/418
-=[ teapot ]=-
_...._
.&amp;#39; _ _ `.
| .&amp;#34;` ^ `&amp;#34;. _,
\_;`&amp;#34;---&amp;#34;`|//
| ;/
\_ _/
`&amp;#34;&amp;#34;&amp;#34;`
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;p>Congratulations! You configured the IKS Ingress ALB to send encrypted traffic to the Istio ingress gateway. You
allocated a host name and certificate for your Istio ingress gateway and used that certificate as the server certificate
for Istio ingress gateway. As the client certificate of the ALB you used the certificate provided by IKS for the ALB.
Once you had the certificates deployed as Kubernetes secrets, you directed the ingress traffic from the ALB to the Istio
ingress gateway for some specific paths and used the certificates for mutual TLS authentication between the ALB and the
Istio ingress gateway.&lt;/p>
&lt;h2 id="cleanup">Cleanup&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Delete the &lt;code>Gateway&lt;/code> configuration, the &lt;code>VirtualService&lt;/code>, and the secrets:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete ingress alb-ingress -n istio-system
$ kubectl delete virtualservice default-ingress -n httptools
$ kubectl delete gateway default-ingress-gateway -n httptools
$ kubectl delete secrets alb-certs -n istio-system
$ rm -rf alb_certs trusted.crt
$ unset CLUSTER_NAME ALB_INGRESS_DOMAIN ALB_SECRET INGRESS_GATEWAY_DOMAIN INGRESS_GATEWAY_SECRET
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Shutdown the &lt;code>httpbin&lt;/code> service:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/httpbin/httpbin.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete -f @samples/httpbin/httpbin.yaml@ -n httptools
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Delete the &lt;code>httptools&lt;/code> namespace:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete namespace httptools
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol></description><pubDate>Fri, 15 May 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/alb-ingress-gateway-iks/</link><author>Vadim Eisenberg (IBM)</author><guid isPermaLink="true">/v1.12/blog/2020/alb-ingress-gateway-iks/</guid><category>traffic-management</category><category>ingress</category><category>sds-credentials</category><category>iks</category><category>mutual-tls</category></item><item><title>Provision a certificate and key for an application without sidecars</title><description>&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">The following information describes an experimental feature, which is intended
for evaluation purposes only.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Istio sidecars obtain their certificates using
the secret discovery service.
A service in the service mesh may not need (or want) an Envoy sidecar
to handle its traffic. In this case, the service will need
to obtain a certificate itself if it wants to connect to other TLS or mutual TLS secured services.&lt;/p>
&lt;p>For a service with no need of a sidecar to manage its traffic, a sidecar can nevertheless still be
deployed only to provision the private key and certificates through
the CSR flow from the CA and then share the certificate with the service
through a mounted file in &lt;code>tmpfs&lt;/code>.
We have used Prometheus as our example application for provisioning
a certificate using this mechanism.&lt;/p>
&lt;p>In the example application (i.e., Prometheus), a sidecar is added to the
Prometheus deployment by setting the flag &lt;code>.Values.prometheus.provisionPrometheusCert&lt;/code>
to &lt;code>true&lt;/code> (this flag is set to true by default in an Istio installation).
This deployed sidecar will then request and share a
certificate with Prometheus.&lt;/p>
&lt;p>The key and certificate provisioned for the example application
are mounted in the directory &lt;code>/etc/istio-certs/&lt;/code>.
We can list the key and certificate provisioned for the application by
running the following command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec -it `kubectl get pod -l app=prometheus -n istio-system -o jsonpath=&amp;#39;{.items[0].metadata.name}&amp;#39;` -c prometheus -n istio-system -- ls -la /etc/istio-certs/
&lt;/code>&lt;/pre>
&lt;p>The output from the above command should include non-empty key and certificate files, similar to the following:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >-rwxr-xr-x 1 root root 2209 Feb 25 13:06 cert-chain.pem
-rwxr-xr-x 1 root root 1679 Feb 25 13:06 key.pem
-rwxr-xr-x 1 root root 1054 Feb 25 13:06 root-cert.pem
&lt;/code>&lt;/pre>
&lt;p>If you want to use this mechanism to provision a certificate
for your own application, take a look at our
&lt;a href="https://github.com/istio/istio/blob/release-1.12/manifests/charts/istio-telemetry/prometheus/templates/deployment.yaml">Prometheus example application&lt;/a> and simply follow the same pattern.&lt;/p></description><pubDate>Wed, 25 Mar 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/proxy-cert/</link><author>Lei Tang (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/proxy-cert/</guid><category>certificate</category><category>sidecar</category></item><item><title>Extended and Improved WebAssemblyHub to Bring the Power of WebAssembly to Envoy and Istio</title><description>
&lt;p>&lt;a href="https://www.solo.io/blog/an-extended-and-improved-webassembly-hub-to-helps-bring-the-power-of-webassembly-to-envoy-and-istio/">&lt;em>Originally posted on the Solo.io blog&lt;/em>&lt;/a>&lt;/p>
&lt;p>As organizations adopt Envoy-based infrastructure like Istio to help solve challenges with microservices communication, they inevitably find themselves needing to customize some part of that infrastructure to fit within their organization&amp;rsquo;s constraints. &lt;a href="https://webassembly.org/">WebAssembly (Wasm)&lt;/a> has emerged as a safe, secure, and dynamic environment for platform extension.&lt;/p>
&lt;p>In the recent &lt;a href="/v1.12/blog/2020/wasm-announce/">announcement of Istio 1.5&lt;/a>, the Istio project lays the foundation for bringing WebAssembly to the popular Envoy proxy. &lt;a href="https://solo.io">Solo.io&lt;/a> is collaborating with Google and the Istio community to simplify the overall experience of creating, sharing, and deploying WebAssembly extensions to Envoy and Istio. It wasn&amp;rsquo;t that long ago that Google and others laid the foundation for containers, and Docker built a great user experience to make it consumable. Similarly, this effort makes Wasm consumable by building the best user experience for WebAssembly on Istio.&lt;/p>
&lt;p>Back in December 2019, Solo.io began an effort to provide a great developer experience for WebAssembly with the announcement of WebAssembly Hub. The WebAssembly Hub allows developers to very quickly spin up a new WebAssembly project in C++ (we&amp;rsquo;re expanding this language choice, see below), build it using Bazel in Docker, and push it to an OCI-compliant registry. From there, operators had to pull the module, and configure Envoy proxies themselves to load it from disk. Beta support in &lt;a href="https://docs.solo.io/gloo/latest/">Gloo, an API Gateway built on Envoy&lt;/a> allows you to declaratively and dynamically load the module, the Solo.io team wanted to bring the same effortless and secure experience to other Envoy-based frameworks as well - like Istio.&lt;/p>
&lt;p>There has been a lot of interest in the innovation in this area, and the Solo.io team has been working hard to further the capabilities of WebAssembly Hub and the workflows it supports. In conjunction with Istio 1.5, Solo.io is thrilled to announce new enhancements to WebAssembly Hub that evolve the viability of WebAssembly with Envoy for production, improve the developer experience, and streamline using Wasm with Envoy in Istio.&lt;/p>
&lt;h2 id="evolving-toward-production">Evolving toward production&lt;/h2>
&lt;p>The Envoy community is working hard to bring Wasm support into the upstream project (right now it lives on a working development fork), with Istio declaring Wasm support an Alpha feature. In &lt;a href="https://www.solo.io/blog/announcing-gloo-1-0-a-production-ready-envoy-based-api-gateway/">Gloo 1.0, we also announced&lt;/a> early, non-production support for Wasm. What is Gloo? Gloo is a modern API Gateway and Ingress Controller (built on Envoy Proxy) that supports routing and securing incoming traffic to legacy monoliths, microservices / Kubernetes and serverless functions. Dev and ops teams are able to shape and control traffic patterns from external end users/clients to backend application services. Gloo is a Kubernetes and Istio native ingress gateway.&lt;/p>
&lt;p>Although it&amp;rsquo;s still maturing in each individual project, there are things that we, as a community, can do to improve the foundation for production support.&lt;/p>
&lt;p>The first area is standardizing what a WebAssembly extension for Envoy looks like. Solo.io, Google, and the Istio community have defined an open specification for bundling and distributing WebAssembly modules as OCI images. This specification provides a powerful model for distributing any type of Wasm module including Envoy extensions.&lt;/p>
&lt;p>This is open to the community - &lt;a href="https://github.com/solo-io/wasm-image-spec">Join in the effort&lt;/a>&lt;/p>
&lt;p>The next area is improving the experience of deploying Wasm extensions into an Envoy-based framework running in production. In the Kubernetes ecosystem, it is considered a best practice in production to use declarative CRD-based configuration to manage cluster configuration. The new &lt;a href="https://docs.solo.io/web-assembly-hub/latest/tutorial_code/wasme_operator/">WebAssembly Hub Operator&lt;/a> adds a single, declarative CRD which automatically deploys and configures Wasm filters to Envoy proxies running inside of a Kubernetes cluster. This operator enables GitOps workflows and cluster automation to manage Wasm filters without human intervention or imperative workflows. We will provide more information about the Operator in an upcoming blog post.&lt;/p>
&lt;p>Lastly, the interactions between developers of Wasm extensions and the teams that deploy them need some kind of role-based access, organization management, and facilities to share, discover, and consume these extensions. The WebAssembly Hub adds team management features like permissions, organizations, user management, sharing, and more.&lt;/p>
&lt;h2 id="improving-the-developer-experience">Improving the developer experience&lt;/h2>
&lt;p>As developers want to target more languages and runtimes, the experience must be kept as simple and as productive as possible. Multi-language support and runtime ABI (Application Binary Interface) targets should be handled automatically in tooling.&lt;/p>
&lt;p>One of the benefits of Wasm is the ability to write modules in many languages. The collaboration between Solo.io and Google provides out-of-the-box support for Envoy filters written in C++, Rust, and AssemblyScript. We will continue to add support for more languages.&lt;/p>
&lt;p>Wasm extensions use the Application Binary Interface (ABI) within the Envoy proxy to which they are deployed. The WebAssembly Hub provides strong ABI versioning guarantees between Envoy, Istio, and Gloo to prevent unpredictable behavior and bugs. All you have to worry about is writing your extension code.&lt;/p>
&lt;p>Lastly, like Docker, the WebAssembly Hub stores and distributes Wasm extensions as OCI images. This makes pushing, pulling, and running extensions as easy as Docker containers. Wasm extension images are versioned and cryptographically secure, making it safe to run extensions locally the same way you would in production. This allows you to build and push as well as trust the source when they pull down and deploy images.&lt;/p>
&lt;h2 id="webassembly-hub-with-istio">WebAssembly Hub with Istio&lt;/h2>
&lt;p>The WebAssembly Hub now fully automates the process of deploying Wasm extensions to Istio, (as well as other Envoy-based frameworks like &lt;a href="https://docs.solo.io/gloo/latest/">Gloo API Gateway&lt;/a>) installed in Kubernetes. With this deployment feature, the WebAssembly Hub relieves the operator or end user from having to manually configure the Envoy proxy in their Istio service mesh to use their WebAssembly modules.&lt;/p>
&lt;p>Take a look at the following video to see just how easy it is to get started with WebAssembly and Istio:&lt;/p>
&lt;ul>
&lt;li>&lt;a href="https://www.youtube.com/watch?v=-XPTGXEpUp8">Part 1&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://youtu.be/vuJKRnjh1b8">Part 2&lt;/a>&lt;/li>
&lt;/ul>
&lt;h2 id="get-started">Get Started&lt;/h2>
&lt;p>We hope that the WebAssembly Hub will become a meeting place for the community to share, discover, and distribute Wasm extensions. By providing a great user experience, we hope to make developing, installing, and running Wasm easier and more rewarding. Join us at the &lt;a href="https://webassemblyhub.io">WebAssembly Hub&lt;/a>, share your extensions and &lt;a href="https://slack.solo.io">ideas&lt;/a>, and join an &lt;a href="https://solo.zoom.us/webinar/register/WN_i8MiDTIpRxqX-BjnXbj9Xw">upcoming webinar&lt;/a>.&lt;/p></description><pubDate>Wed, 25 Mar 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/wasmhub-istio/</link><author>Idit Levine (Solo.io)</author><guid isPermaLink="true">/v1.12/blog/2020/wasmhub-istio/</guid><category>wasm</category><category>extensibility</category><category>alpha</category><category>performance</category><category>operator</category></item><item><title>Introducing istiod: simplifying the control plane</title><description>
&lt;p>Microservices are a great pattern when they map services to disparate teams that deliver them, or when the value of independent rollout and the value of independent scale are greater than the cost of orchestration. We regularly talk to customers and teams running Istio in the real world, and they told us that none of these were the case for the Istio control plane. So, in Istio 1.5, we&amp;rsquo;ve changed how Istio is packaged, consolidating the control plane functionality into a single binary called &lt;strong>istiod&lt;/strong>.&lt;/p>
&lt;h2 id="history-of-the-istio-control-plane">History of the Istio control plane&lt;/h2>
&lt;p>Istio implements a pattern that has been in use at both Google and IBM for many years, which later became known as &amp;ldquo;service mesh&amp;rdquo;. By pairing client and server processes with proxy servers, they act as an application-aware &lt;em>data plane&lt;/em> that’s not simply moving packets around hosts, or pulses over wires.&lt;/p>
&lt;p>This pattern helps the world come to terms with &lt;em>microservices&lt;/em>: fine-grained, loosely-coupled services connected via lightweight protocols. The common cross-platform and cross-language standards like HTTP and gRPC that replace proprietary transports, and the widespread presence of the needed libraries, empower different teams to write different parts of an overall architecture in whatever language makes the most sense. Furthermore, each service can scale independently as needed. A desire to implement security, observability and traffic control for such a network powers Istio’s popularity.&lt;/p>
&lt;p>Istio&amp;rsquo;s &lt;em>control plane&lt;/em> is, itself, a modern, cloud-native application. Thus, it was built from the start as a set of microservices. Individual Istio components like service discovery (Pilot), configuration (Galley), certificate generation (Citadel) and extensibility (Mixer) were all written and deployed as separate microservices. The need for these components to communicate securely and be observable, provided opportunities for Istio to eat its own dogfood (or &amp;ldquo;drink its own champagne&amp;rdquo;, to use a more French version of the metaphor!).&lt;/p>
&lt;h2 id="the-cost-of-complexity">The cost of complexity&lt;/h2>
&lt;p>Good teams look back upon their choices and, with the benefit of hindsight, revisit them. Generally, when a team adopts microservices and their inherent complexity, they look for improvements in other areas to justify the tradeoffs. Let&amp;rsquo;s look at the Istio control plane through that lens.&lt;/p>
&lt;ul>
&lt;li>&lt;p>&lt;strong>Microservices empower you to write in different languages.&lt;/strong> The data plane (the Envoy proxy) is written in C++, and this boundary benefits from a clean separation in terms of the xDS APIs. However, all of the Istio control plane components are written in Go. We were able to choose the appropriate language for the appropriate job: highly performant C++ for the proxy, but accessible and speedy-development for everything else.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Microservices empower you to allow different teams to manage services individually.&lt;/strong>. In the vast majority of Istio installations, all the components are installed and operated by a single team or individual. The componentization done within Istio is aligned along the boundaries of the development teams who build it. This would make sense if the Istio components were delivered as a managed service by the people who wrote them, but this is not the case! Making life simpler for the development teams had an outsized impact of the usability for the orders-of-magnitude more users.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Microservices empower you to decouple versions, and release different components at different times.&lt;/strong> All the components of the control plane have always been released at the same version, at the same time. We have never tested or supported running different versions of (for example) Citadel and Pilot.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Microservices empower you to scale components independently.&lt;/strong> In Istio 1.5, control plane costs are dominated by a single feature: serving the Envoy xDS APIs that program the data plane. Every other feature has a marginal cost, which means there is very little value to having those features in separately-scalable microservices.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Microservices empower you to maintain security boundaries.&lt;/strong> Another good reason to separate an application into different microservices is if they have different security roles. Multiple Istio microservices like the sidecar injector, the Envoy bootstrap, Citadel, and Pilot hold nearly equivalent permissions to change the proxy configuration. Therefore, exploiting any of these services would cause near equivalent damage. When you deploy Istio, all the components are installed by default into the same Kubernetes namespace, offering limited security isolation.&lt;/p>&lt;/li>
&lt;/ul>
&lt;h2 id="the-benefit-of-consolidation-introducing-istiod">The benefit of consolidation: introducing istiod&lt;/h2>
&lt;p>Having established that many of the common benefits of microservices didn&amp;rsquo;t apply to the Istio control plane, we decided to unify them into a single binary: &lt;strong>istiod&lt;/strong> (the &amp;rsquo;d&amp;rsquo; is for &lt;a href="https://en.wikipedia.org/wiki/Daemon_%28computing%29">daemon&lt;/a>).&lt;/p>
&lt;p>Let&amp;rsquo;s look at the benefits of the new packaging:&lt;/p>
&lt;ul>
&lt;li>&lt;p>&lt;strong>Installation becomes easier.&lt;/strong> Fewer Kubernetes deployments and associated configurations are required, so the set of configuration options and flags for Istio is reduced significantly. In the simplest case, &lt;strong>&lt;em>you can start the Istio control plane, with all features enabled, by starting a single Pod.&lt;/em>&lt;/strong>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Configuration becomes easier.&lt;/strong> Many of the configuration options that Istio has today are ways to orchestrate the control plane components, and so are no longer needed. You also no longer need to change cluster-wide &lt;code>PodSecurityPolicy&lt;/code> to deploy Istio.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Using VMs becomes easier.&lt;/strong> To add a workload to a mesh, you now just need to install one agent and the generated certificates. That agent connects back to only a single service.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Maintenance becomes easier.&lt;/strong> Installing, upgrading, and removing Istio no longer require a complicated dance of version dependencies and startup orders. For example: To upgrade, you only need to start a new istiod version alongside your existing control plane, canary it, and then move all traffic over to it.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Scalability becomes easier.&lt;/strong> There is now only one component to scale.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Debugging becomes easier.&lt;/strong> Fewer components means less cross-component environmental debugging.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Startup time goes down.&lt;/strong> Components no longer need to wait for each other to start in a defined order.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Resource usage goes down and responsiveness goes up.&lt;/strong> Communication between components becomes guaranteed, and not subject to gRPC size limits. Caches can be shared safely, which decreases the resource footprint as a result.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>istiod unifies functionality that Pilot, Galley, Citadel and the sidecar injector previously performed, into a single binary.&lt;/p>
&lt;p>A separate component, the istio-agent, helps each sidecar connect to the mesh by securely passing configuration and secrets to the Envoy proxies. While the agent, strictly speaking, is still part of the control plane, it runs on a per-pod basis. We’ve further simplified by rolling per-node functionality that used to run as a DaemonSet, into that per-pod agent.&lt;/p>
&lt;h2 id="extra-for-experts">Extra for experts&lt;/h2>
&lt;p>There will still be some cases where you might want to run Istio components independently, or replace certain components.&lt;/p>
&lt;p>Some users might want to use a Certificate Authority (CA) outside the mesh, and we have &lt;a href="/v1.12/docs/tasks/security/cert-management/plugin-ca-cert/">documentation on how to do that&lt;/a>. If you do your certificate provisioning using a different tool, we can use that instead of the built-in CA.&lt;/p>
&lt;h2 id="moving-forward">Moving forward&lt;/h2>
&lt;p>At its heart, istiod is just a packaging and optimization change. It&amp;rsquo;s built on the same code and API contracts as the separate components, and remains covered by our comprehensive test suite. This gives us confidence in making it the default in Istio 1.5. The service is now called &lt;code>istiod&lt;/code> - you’ll see an &lt;code>istio-pilot&lt;/code> for existing proxies as the upgrade process completes.&lt;/p>
&lt;p>While the move to istiod may seem like a big change, and is a huge improvement for the people who &lt;em>administer&lt;/em> and &lt;em>maintain&lt;/em> the mesh, it won’t make the day-to-day life of &lt;em>using&lt;/em> Istio any different. istiod is not changing any of the APIs used to configure your mesh, so your existing processes will all stay the same.&lt;/p>
&lt;p>Does this change imply that microservice are a mistake for &lt;em>all&lt;/em> workloads and architectures? Of course not. They are a tool in a toolbelt, and they work best when they are reflected in your organizational reality. Instead, this change shows a willingness in the project to change based on user feedback, and a continued focus on simplification for all users. Microservices have to be right sized, and we believe we have found the right size for Istio.&lt;/p></description><pubDate>Thu, 19 Mar 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/istiod/</link><author>Craig Box (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/istiod/</guid><category>istiod</category><category>control plane</category><category>operator</category></item><item><title>Declarative WebAssembly deployment for Istio</title><description>
&lt;p>As outlined in the &lt;a href="/v1.12/blog/2020/tradewinds-2020/">Istio 2020 trade winds blog&lt;/a> and more recently &lt;a href="/v1.12/news/releases/1.5.x/announcing-1.5/">announced with Istio 1.5&lt;/a>, WebAssembly (Wasm) is now an (alpha) option for extending the functionality of the Istio service proxy (Envoy proxy). With Wasm, users can build support for new protocols, custom metrics, loggers, and other filters. Working closely with Google, we in the community (&lt;a href="https://solo.io">Solo.io&lt;/a>) have focused on the user experience of building, socializing, and deploying Wasm extensions to Istio. We&amp;rsquo;ve announced &lt;a href="https://webassemblyhub.io">WebAssembly Hub&lt;/a> and &lt;a href="https://docs.solo.io/web-assembly-hub/latest/installation/">associated tooling&lt;/a> to build a &amp;ldquo;docker-like&amp;rdquo; experience for working with Wasm.&lt;/p>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>With the WebAssembly Hub tooling, we can use the &lt;code>wasme&lt;/code> CLI to easily bootstrap a Wasm project for Envoy, push it to a repository, and then pull/deploy it to Istio. For example, to deploy a Wasm extension to Istio with &lt;code>wasme&lt;/code> we can run the following:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ wasme deploy istio webassemblyhub.io/ceposta/demo-add-header:v0.2 \
--id=myfilter \
--namespace=bookinfo \
--config &amp;#39;tomorrow&amp;#39;
&lt;/code>&lt;/pre>
&lt;p>This will add the &lt;code>demo-add-header&lt;/code> extension to all workloads running in the &lt;code>bookinfo&lt;/code> namespace. We can get more fine-grained control over which workloads get the extension by using the &lt;code>--labels&lt;/code> parameter:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ wasme deploy istio webassemblyhub.io/ceposta/demo-add-header:v0.2 \
--id=myfilter \
--namespace=bookinfo \
--config &amp;#39;tomorrow&amp;#39; \
--labels app=details
&lt;/code>&lt;/pre>
&lt;p>This is a much easier experience than manually creating &lt;code>EnvoyFilter&lt;/code> resources and trying to get the Wasm module to each of the pods that are part of the workload you&amp;rsquo;re trying to target. However, this is a very imperative approach to interacting with Istio. Just like users typically don&amp;rsquo;t use &lt;code>kubectl&lt;/code> directly in production and prefer a declarative, resource-based workflow, we want the same for making customizations to our Istio proxies.&lt;/p>
&lt;h2 id="a-declarative-approach">A declarative approach&lt;/h2>
&lt;p>The WebAssembly Hub tooling also includes &lt;a href="https://docs.solo.io/web-assembly-hub/latest/tutorial_code/wasme_operator/">an operator for deploying Wasm extensions to Istio workloads&lt;/a>. The &lt;a href="https://kubernetes.io/docs/concepts/extend-kubernetes/operator/">operator&lt;/a> allows users to define their WebAssembly extensions using a declarative format and leave it to the operator to rectify the deployment. For example, we use a &lt;code>FilterDeployment&lt;/code> resource to define what image and workloads need the extension:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: wasme.io/v1
kind: FilterDeployment
metadata:
name: bookinfo-custom-filter
namespace: bookinfo
spec:
deployment:
istio:
kind: Deployment
labels:
app: details
filter:
config: &amp;#39;world&amp;#39;
image: webassemblyhub.io/ceposta/demo-add-header:v0.2
&lt;/code>&lt;/pre>
&lt;p>We could then take this &lt;code>FilterDeployment&lt;/code> document and version it with the rest of our Istio resources. You may be wondering why we need this Custom Resource to configure Istio&amp;rsquo;s service proxy to use a Wasm extension when Istio already has the &lt;code>EnvoyFilter&lt;/code> resource.&lt;/p>
&lt;p>Let&amp;rsquo;s take a look at exactly how all of this works under the covers.&lt;/p>
&lt;h2 id="how-it-works">How it works&lt;/h2>
&lt;p>Under the covers the operator is doing a few things that aid in deploying and configuring a Wasm extension into the Istio service proxy (Envoy Proxy).&lt;/p>
&lt;ul>
&lt;li>Set up local cache of Wasm extensions&lt;/li>
&lt;li>Pull desired Wasm extension into the local cache&lt;/li>
&lt;li>Mount the &lt;code>wasm-cache&lt;/code> into appropriate workloads&lt;/li>
&lt;li>Configure Envoy with &lt;code>EnvoyFilter&lt;/code> CRD to use the Wasm filter&lt;/li>
&lt;/ul>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:42.663891779396465%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/deploy-wasm-declarative/how-it-works.png" title="Understanding how wasme operator works">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/deploy-wasm-declarative/how-it-works.png" alt="How the wasme operator works" />
&lt;/a>
&lt;/div>
&lt;figcaption>Understanding how wasme operator works&lt;/figcaption>
&lt;/figure>
&lt;p>At the moment, the Wasm image needs to be published into a registry for the operator to correctly cache it. The cache pods run as DaemonSet on each node so that the cache can be mounted into the Envoy container. This is being improved, as it&amp;rsquo;s not the ideal mechanism. Ideally we wouldn&amp;rsquo;t have to deal with mounting anything and could stream the module to the proxy directly over HTTP, so stay tuned for updates (should land within next few days). The mount is established by using the &lt;code>sidecar.istio.io/userVolume&lt;/code> and &lt;code>sidecar.istio.io/userVolumeMount&lt;/code> annotations. See &lt;a href="/v1.12/docs/reference/config/annotations/">the docs on Istio Resource Annotations&lt;/a> for more about how that works.&lt;/p>
&lt;p>Once the Wasm module is cached correctly and mounted into the workload&amp;rsquo;s service proxy, the operator then configures the &lt;code>EnvoyFilter&lt;/code> resources.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: details-v1-myfilter
namespace: bookinfo
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_INBOUND
listener:
filterChain:
filter:
name: envoy.http_connection_manager
subFilter:
name: envoy.router
patch:
operation: INSERT_BEFORE
value:
config:
config:
configuration: tomorrow
name: myfilter
rootId: add_header
vmConfig:
code:
local:
filename: /var/local/lib/wasme-cache/44bf95b368e78fafb663020b43cf099b23fc6032814653f2f47e4d20643e7267
runtime: envoy.wasm.runtime.v8
vmId: myfilter
name: envoy.filters.http.wasm
workloadSelector:
labels:
app: details
version: v1
&lt;/code>&lt;/pre>
&lt;p>You can see the &lt;code>EnvoyFilter&lt;/code> resource configures the proxy to add the &lt;code>envoy.filter.http.wasm&lt;/code> filter and load the Wasm module from the &lt;code>wasme-cache&lt;/code>.&lt;/p>
&lt;p>Once the Wasm extension is loaded into the Istio service proxy, it will extend the capabilities of the proxy with whatever custom code you introduced.&lt;/p>
&lt;h2 id="next-steps">Next Steps&lt;/h2>
&lt;p>In this blog we explored options for installing Wasm extensions into Istio workloads. The easiest way to get started with WebAssembly on Istio is to use the &lt;code>wasme&lt;/code> tool &lt;a href="https://docs.solo.io/web-assembly-hub/latest/tutorial_code/getting_started/">to bootstrap a new Wasm project&lt;/a> with C++, AssemblyScript [or Rust coming really soon!]. For example, to set up a C++ Wasm module, you can run:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ wasme init ./filter --language cpp --platform istio --platform-version 1.5.x
&lt;/code>&lt;/pre>
&lt;p>If we didn&amp;rsquo;t have the extra flags, &lt;code>wasme init&lt;/code> would enter an interactive mode walking you through the correct values to choose.&lt;/p>
&lt;p>Take a look at the &lt;a href="https://docs.solo.io/web-assembly-hub/latest/tutorial_code/getting_started/">WebAssembly Hub wasme tooling&lt;/a> to get started with Wasm on Istio.&lt;/p>
&lt;h2 id="learn-more">Learn more&lt;/h2>
&lt;ul>
&lt;li>&lt;p>Redefine Extensibility &lt;a href="/v1.12/blog/2020/wasm-announce/">with WebAssembly on Envoy and Istio&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>WebAssembly SF talk (video): &lt;a href="https://www.youtube.com/watch?v=OIUPf8m7CGA">Extensions for network proxies&lt;/a>, by John Plevyak&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://www.solo.io/blog/an-extended-and-improved-webassembly-hub-to-helps-bring-the-power-of-webassembly-to-envoy-and-istio/">Solo blog&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://github.com/proxy-wasm/spec">Proxy-Wasm ABI specification&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/master/docs/wasm_filter.md">Proxy-Wasm C++ SDK&lt;/a> and
its &lt;a href="https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/master/docs/wasm_filter.md">developer documentation&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://github.com/proxy-wasm/proxy-wasm-rust-sdk">Proxy-Wasm Rust SDK&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://github.com/solo-io/proxy-runtime">Proxy-Wasm AssemblyScript SDK&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://docs.solo.io/web-assembly-hub/latest/tutorial_code/">Tutorials&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>Videos on the &lt;a href="https://www.youtube.com/channel/UCuketWAG3WqYjjxtQ9Q8ApQ">Solo.io Youtube Channel&lt;/a>&lt;/p>&lt;/li>
&lt;/ul></description><pubDate>Mon, 16 Mar 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/deploy-wasm-declarative/</link><author>Christian Posta (Solo.io)</author><guid isPermaLink="true">/v1.12/blog/2020/deploy-wasm-declarative/</guid><category>wasm</category><category>extensibility</category><category>alpha</category><category>operator</category></item><item><title>Redefining extensibility in proxies - introducing WebAssembly to Envoy and Istio</title><description>
&lt;p>Since adopting &lt;a href="https://www.envoyproxy.io/">Envoy&lt;/a> in 2016, the Istio project has always wanted to
provide a platform on top of which a rich set of extensions could be built, to meet the diverse
needs of our users. There are many reasons to add capability to the data plane of a service
mesh &amp;mdash; to support newer protocols, integrate with proprietary security controls, or enhance
observability with custom metrics, to name a few.&lt;/p>
&lt;p>Over the last year and a half our team here at Google has been working on adding dynamic
extensibility to the Envoy proxy using &lt;a href="https://webassembly.org/">WebAssembly&lt;/a>. We are delighted to
share that work with the world today, as well as
unveiling &lt;a href="https://github.com/proxy-wasm/spec">WebAssembly (Wasm) for Proxies&lt;/a> (Proxy-Wasm): an ABI,
which we intend to standardize; SDKs; and its first major implementation, the new,
lower-latency &lt;a href="/v1.12/docs/reference/config/proxy_extensions/wasm_telemetry/">Istio telemetry system&lt;/a>.&lt;/p>
&lt;p>We have also worked closely with the community to ensure that there is a great developer experience
for users to get started quickly. The Google team has been working closely with the team
at &lt;a href="https://solo.io">Solo.io&lt;/a> who have built the &lt;a href="https://webassemblyhub.io/">WebAssembly Hub,&lt;/a>
a service for building, sharing, discovering and deploying Wasm extensions.
With the WebAssembly Hub, Wasm extensions are as easy to manage, install and and run as containers.&lt;/p>
&lt;p>This work is being released today in Alpha and there is still lots
of &lt;a href="#next-steps">work to be done&lt;/a>, but we are excited to get this into the hands of developers
so they can start experimenting with the tremendous possibilities this opens up.&lt;/p>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>The need for extensibility has been a founding tenet of both the Istio and Envoy projects,
but the two projects took different approaches. Istio project focused on enabling a generic
out-of-process extension model called &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/">Mixer&lt;/a>
with a lightweight developer experience, while Envoy focused on in-proxy &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/extending/extending">extensions&lt;/a>.&lt;/p>
&lt;p>Each approach has its share of pros and cons. The Istio model led to significant resource
inefficiencies that impacted tail latencies and resource utilization. This model was also
intrinsically limited - for example, it was never going to provide support for
implementing &lt;a href="https://blog.envoyproxy.io/how-to-write-envoy-filters-like-a-ninja-part-1-d166e5abec09">custom protocol handling&lt;/a>.&lt;/p>
&lt;p>The Envoy model imposed a monolithic build process, and required extensions to be written in C++,
limiting the developer ecosystem. Rolling out a new extension to the fleet required pushing new
binaries and rolling restarts, which can be difficult to coordinate, and risk downtime. This also
incentivized developers to upstream extensions into Envoy that were used by only a small
percentage of deployments, just to piggyback on its release mechanisms.&lt;/p>
&lt;p>Over time some of the most performance-sensitive features of Istio have been upstreamed
into Envoy - &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/security/rbac_filter">policy checks on traffic&lt;/a>, and
&lt;a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/security/jwt_authn_filter">JWT authentication&lt;/a>, for example.
Still, we have always wanted to converge on a single stack for extensibility that imposes fewer
tradeoffs: something that decouples Envoy releases from its extension ecosystem, enables
developers to work in their languages of choice, and enables Istio to reliably roll out new
capability without downtime risk. Enter WebAssembly.&lt;/p>
&lt;h2 id="what-is-webassembly">What is WebAssembly?&lt;/h2>
&lt;p>&lt;a href="https://webassembly.org/">WebAssembly&lt;/a> (Wasm) is a portable bytecode format for executing code
written in &lt;a href="https://github.com/appcypher/awesome-wasm-langs">multiple languages&lt;/a> at
near-native speed. Its initial &lt;a href="https://webassembly.org/docs/high-level-goals/">design goals&lt;/a> align
well with the challenges outlined above, and it has sizable industry support behind it. Wasm
is the fourth standard language (following HTML, CSS and JavaScript) to run natively in all
the major browsers, having become a &lt;a href="https://www.w3.org/TR/wasm-core-1/">W3C Recommendation&lt;/a> in
December 2019. That gives us confidence in making a strategic bet on it.&lt;/p>
&lt;p>While WebAssembly started life as a client-side technology, there are a number of advantages
to using it on the server. The runtime is memory-safe and sandboxed for security. There is a
large tooling ecosystem for compiling and debugging Wasm in its textual or binary format.
The &lt;a href="https://www.w3.org/">W3C&lt;/a> and &lt;a href="https://bytecodealliance.org/">BytecodeAlliance&lt;/a> have become
active hubs for other server-side efforts. For example, the Wasm community is standardizing
a &lt;a href="https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/">&amp;ldquo;WebAssembly System Interface&amp;rdquo; (WASI)&lt;/a>
at the W3C, with a sample implementation, which provides an OS-like abstraction to Wasm &amp;lsquo;programs&amp;rsquo;.&lt;/p>
&lt;h2 id="bringing-webassembly-to-envoy">Bringing WebAssembly to Envoy&lt;/h2>
&lt;p>&lt;a href="https://github.com/envoyproxy/envoy/issues/4272">Over the past 18 months&lt;/a>, we have been working
with the Envoy community to build Wasm extensibility into Envoy and contribute it upstream.
We&amp;rsquo;re pleased to announce it is available as Alpha in the Envoy build shipped
with &lt;a href="/v1.12/news/releases/1.5.x/announcing-1.5/">Istio 1.5&lt;/a>, with source in
the &lt;a href="https://github.com/envoyproxy/envoy-wasm/">&lt;code>envoy-wasm&lt;/code>&lt;/a> development fork and work ongoing to
merge it into the main Envoy tree. The implementation uses the WebAssembly runtime built into
Google&amp;rsquo;s high performance &lt;a href="https://v8.dev/">V8 engine&lt;/a>.&lt;/p>
&lt;p>In addition to the underlying runtime, we have also built:&lt;/p>
&lt;ul>
&lt;li>&lt;p>A generic Application Binary Interface (ABI) for embedding Wasm in proxies, which means compiled
extensions will work across different versions of Envoy - or even other proxies, should they choose
to implement the ABI&lt;/p>&lt;/li>
&lt;li>&lt;p>SDKs for easy extension development in &lt;a href="https://github.com/proxy-wasm/proxy-wasm-cpp-sdk">C++&lt;/a>,
&lt;a href="https://github.com/proxy-wasm/proxy-wasm-rust-sdk">Rust&lt;/a>
and &lt;a href="https://github.com/solo-io/proxy-runtime">AssemblyScript&lt;/a>, with more to follow&lt;/p>&lt;/li>
&lt;li>&lt;p>Comprehensive &lt;a href="https://docs.solo.io/web-assembly-hub/latest/tutorial_code/">samples and instructions&lt;/a>
on how to deploy in Istio and standalone Envoy&lt;/p>&lt;/li>
&lt;li>&lt;p>Abstractions to allow for other Wasm runtimes to be used, including a &amp;lsquo;null&amp;rsquo; runtime which
simply compiles the extension natively into Envoy &amp;mdash; very useful for testing and debugging&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>Using Wasm for extending Envoy brings us several key benefits:&lt;/p>
&lt;ul>
&lt;li>&lt;p>Agility: Extensions can be delivered and reloaded at runtime using the Istio control plane.
This enables a fast develop → test → release cycle for extensions without
requiring Envoy rollouts.&lt;/p>&lt;/li>
&lt;li>&lt;p>Stock releases: Once merging into the main tree is complete, Istio and others will be able to
use stock releases of Envoy, instead of custom builds. This will also free the Envoy community
to move some of the built-in extensions to this model, thereby reducing their
supported footprint.&lt;/p>&lt;/li>
&lt;li>&lt;p>Reliability and isolation: Extensions are deployed inside a sandbox with resource constraints,
which means they can now crash, or leak memory, without bringing the whole Envoy process down.
CPU and memory usage can also be constrained.&lt;/p>&lt;/li>
&lt;li>&lt;p>Security: The sandbox has a clearly defined API for communicating with Envoy, so extensions
only have access to, and can modify, a limited number of properties of a connection or request.
Furthermore, because Envoy mediates this interaction, it can hide or sanitize sensitive
information from the extension (e.g. &amp;ldquo;Authorization&amp;rdquo; and &amp;ldquo;Cookie&amp;rdquo; HTTP headers, or
the client&amp;rsquo;s IP address).&lt;/p>&lt;/li>
&lt;li>&lt;p>Flexibility: &lt;a href="https://github.com/appcypher/awesome-wasm-langs">over 30 programming languages can be compiled to WebAssembly&lt;/a>,
allowing developers from all backgrounds - C++, Go, Rust, Java, TypeScript, etc. - to write
Envoy extensions in their language of choice.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>&amp;ldquo;I am extremely excited to see WASM support land in Envoy; this is the future of Envoy
extensibility, full stop. Envoy&amp;rsquo;s WASM support coupled with a community driven hub will unlock an
incredible amount of innovation in the networking space across both service mesh and API gateway
use cases. I can&amp;rsquo;t wait to see what the community builds moving forward.&amp;rdquo;
&amp;ndash; Matt Klein, Envoy creator.&lt;/p>
&lt;p>For technical details of the implementation, look out for an upcoming post
to &lt;a href="https://blog.envoyproxy.io/">the Envoy blog&lt;/a>.&lt;/p>
&lt;p>The &lt;a href="https://github.com/proxy-wasm">Proxy-Wasm&lt;/a> interface between host environment and extensions
is deliberately proxy agnostic. We&amp;rsquo;ve built it into Envoy, but it was designed to be adopted by
other proxy vendors. We want to see a world where you can take an extension written for Istio and
Envoy and run it in other infrastructure; you&amp;rsquo;ll hear more about that soon.&lt;/p>
&lt;h2 id="building-on-webassembly-in-istio">Building on WebAssembly in Istio&lt;/h2>
&lt;p>Istio moved several of its extensions into its build of Envoy as part of the 1.5 release, in order
to significantly improve performance. While doing that work we have been testing to ensure those
same extensions can compile and run as Proxy-Wasm modules with no variation in behavior. We&amp;rsquo;re not
quite ready to make this setup the default, given that we consider Wasm support to be Alpha;
however, this has given us a lot of confidence in our general approach and in the host
environment, ABI and SDKs that have been developed.&lt;/p>
&lt;p>We have also been careful to ensure that the Istio control plane and
its &lt;a href="/v1.12/docs/reference/config/networking/envoy-filter/">Envoy configuration APIs&lt;/a> are Wasm-ready.
We have samples to show how several common customizations such as custom header decoding or
programmatic routing can be performed which are common asks from users. As we move this support to
Beta, you will see documentation showing best practices for using Wasm with Istio.&lt;/p>
&lt;p>Finally, we are working with the many vendors who have
written &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/adapters/">Mixer adapters&lt;/a>,
to help them with a migration to Wasm &amp;mdash; if that is the best path forward. Mixer will move to a
community project in a future release, where it will remain available for legacy use cases.&lt;/p>
&lt;h2 id="developer-experience">Developer Experience&lt;/h2>
&lt;p>Powerful tooling is nothing without a great developer experience. Solo.io
&lt;a href="https://www.solo.io/blog/an-extended-and-improved-webassembly-hub-to-helps-bring-the-power-of-webassembly-to-envoy-and-istio/">recently announced&lt;/a>
the release of &lt;a href="https://webassemblyhub.io/">WebAssembly Hub&lt;/a>, a set of tools and repository for
building, deploying, sharing and discovering Envoy Proxy Wasm extensions for Envoy and Istio.&lt;/p>
&lt;p>The WebAssembly Hub fully automates many of the steps required for developing and deploying Wasm
extensions. Using WebAssembly Hub tooling, users can easily compile their code - in any supported
language - into Wasm extensions. The extensions can then be uploaded to the Hub registry, and be
deployed and undeployed to Istio with a single command.&lt;/p>
&lt;p>Behind the scenes the Hub takes care of much of the nitty-gritty, such as pulling in the correct
toolchain, ABI version verification, permission control, and more. The workflow also eliminates
toil with configuration changes across Istio service proxies by automating the deployment of your
extensions. This tooling helps users and operators avoid unexpected behaviors due to
misconfiguration or version mismatches.&lt;/p>
&lt;p>The WebAssembly Hub tools provide a powerful CLI as well as an elegant and easy-to-use graphical
user interface. An important goal of the WebAssembly Hub is to simplify the experience around
building Wasm modules and provide a place of collaboration for developers to share and discover
useful extensions.&lt;/p>
&lt;p>Check out the &lt;a href="https://docs.solo.io/web-assembly-hub/latest/tutorial_code/">getting started guide&lt;/a>
to create your first Proxy-Wasm extension.&lt;/p>
&lt;h2 id="next-steps">Next Steps&lt;/h2>
&lt;p>In addition to working towards a beta release, we are committed to making sure that there is a
durable community around Proxy-Wasm. The ABI needs to be finalized, and turning it into a standard
will be done with broader feedback within the appropriate standards body. Completing upstreaming
support into the Envoy mainline is still in progress. We are also seeking an appropriate
community home for the tooling and the WebAssembly Hub&lt;/p>
&lt;h2 id="learn-more">Learn more&lt;/h2>
&lt;ul>
&lt;li>&lt;p>WebAssembly SF talk (video): &lt;a href="https://www.youtube.com/watch?v=OIUPf8m7CGA">Extensions for network proxies&lt;/a>, by John Plevyak&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://www.solo.io/blog/an-extended-and-improved-webassembly-hub-to-helps-bring-the-power-of-webassembly-to-envoy-and-istio/">Solo blog&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://github.com/proxy-wasm/spec">Proxy-Wasm ABI specification&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/master/docs/wasm_filter.md">Proxy-Wasm C++ SDK&lt;/a> and
its &lt;a href="https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/master/docs/wasm_filter.md">developer documentation&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://github.com/proxy-wasm/proxy-wasm-rust-sdk">Proxy-Wasm Rust SDK&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://github.com/solo-io/proxy-runtime">Proxy-Wasm AssemblyScript SDK&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="https://docs.solo.io/web-assembly-hub/latest/tutorial_code/">Tutorials&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>Videos on the &lt;a href="https://www.youtube.com/channel/UCuketWAG3WqYjjxtQ9Q8ApQ">Solo.io Youtube Channel&lt;/a>&lt;/p>&lt;/li>
&lt;/ul></description><pubDate>Thu, 05 Mar 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/wasm-announce/</link><author>Craig Box, Mandar Jog, John Plevyak, Louis Ryan, Piotr Sikora (Google), Yuval Kohavi, Scott Weiss (Solo.io)</author><guid isPermaLink="true">/v1.12/blog/2020/wasm-announce/</guid><category>wasm</category><category>extensibility</category><category>alpha</category><category>performance</category><category>operator</category></item><item><title>Istio in 2020 - Following the Trade Winds</title><description>
&lt;p>Istio solves real problems that people encounter running microservices. Even
&lt;a href="https://kubernetespodcast.com/episode/016-descartes-labs/">very early pre-release versions&lt;/a>
helped users debug the latency in their architecture, increase the reliability
of services, and transparently secure traffic behind the firewall.&lt;/p>
&lt;p>Last year, the Istio project experienced major growth. After a 9-month gestation
before the 1.1 release in Q1, we set a goal of having a quarterly release
cadence. We knew it was important to deliver value consistently and predictably.
With three releases landing in the successive quarters as planned, we are proud
to have reached that goal.&lt;/p>
&lt;p>During that time, we improved our build and test infrastructure, resulting in
higher quality and easier release cycles. We doubled down on user experience,
adding many commands to make operating and debugging the mesh easier. We also
saw tremendous growth in the number of developers and companies contributing to
the product - culminating in us being
&lt;a href="https://octoverse.github.com/#fastest-growing-oss-projects-by-contributors">#4 on GitHub&amp;rsquo;s top ten list of fastest growing projects&lt;/a>!&lt;/p>
&lt;p>We have ambitious goals for Istio in 2020 and there are many major efforts
underway, but at the same time we strongly believe that good infrastructure
should be &amp;ldquo;boring.&amp;rdquo; Using Istio in production should be a seamless experience;
performance should not be a concern, upgrades should be a non-event and complex
tasks should be automated away. With our investment in a more powerful
extensibility story we think the pace of innovation in the service mesh space
can increase while Istio focuses on being gloriously dull.
More details on our major efforts in 2020 below.&lt;/p>
&lt;h2 id="sleeker-smoother-and-faster">Sleeker, smoother and faster&lt;/h2>
&lt;p>Istio provided for extensibility from day one, implemented by a component called
Mixer. Mixer is a platform that allows custom
&lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/#adapters">adapters&lt;/a>
to act as an intermediary between the data plane and the backends you use for
policy or telemetry. Mixer necessarily added overhead to requests because it
required extensions to be out-of-process. So, we&amp;rsquo;re moving to a model that
enables extension directly in the proxies instead.&lt;/p>
&lt;p>Most of Mixer’s use cases for policy enforcement are already addressed with
Istio&amp;rsquo;s &lt;a href="/v1.12/docs/concepts/security/#authentication-policies">authentication&lt;/a>
and &lt;a href="/v1.12/docs/concepts/security/#authorization">authorization&lt;/a> policies, which
allow you to control workload-to-workload and end-user-to-workload authorization
directly in the proxy. Common monitoring use cases have already moved into the
proxy too - we have
&lt;a href="/v1.12/docs/reference/config/metrics">introduced in-proxy support&lt;/a>
for sending telemetry to Prometheus and Stackdriver.&lt;/p>
&lt;p>Our benchmarking shows that the new telemetry model reduces our latency
dramatically and gives us industry-leading performance, with 50% reductions in
both latency and CPU consumption.&lt;/p>
&lt;h2 id="a-new-model-for-istio-extensibility">A new model for Istio extensibility&lt;/h2>
&lt;p>The model that replaces Mixer uses extensions in Envoy to provide even more
capability. The Istio community is leading the implementation of a
&lt;a href="https://webassembly.org/">WebAssembly&lt;/a> (Wasm) runtime in Envoy, which lets us
implement extensions that are modular, sandboxed, and developed in one of
&lt;a href="https://github.com/appcypher/awesome-wasm-langs">over 20 languages&lt;/a>. Extensions
can be dynamically loaded and reloaded while the proxy continues serving
traffic. Wasm extensions will also be able to extend the platform in ways that
Mixer simply couldn’t. They can act as custom protocol handlers and transform
payloads as they pass through Envoy — in short they can do the same things as
modules built into Envoy.&lt;/p>
&lt;p>We&amp;rsquo;re working with the Envoy community on ways to discover and distribute these
extensions. We want to make WebAssembly extensions as easy to install and run as
containers. Many of our partners have written Mixer adapters, and together we
are getting them ported to Wasm. We are also developing guides and codelabs on
how to write your own extensions for custom integrations.&lt;/p>
&lt;p>By changing the extension model, we were also able to remove dozens of CRDs.
You no longer need a unique CRD for every piece of software you integrate with
Istio.&lt;/p>
&lt;p>Installing Istio 1.5 with the &amp;lsquo;preview&amp;rsquo; configuration profile won&amp;rsquo;t install
Mixer. If you upgrade from a previous release, or install the &amp;lsquo;default&amp;rsquo; profile,
we still keep Mixer around, to be safe. When using Prometheus or Stackdriver for
metrics, we recommend you try out the new mode and see how much your performance
improves.&lt;/p>
&lt;p>You can keep Mixer installed and enabled if you need it. Eventually Mixer will
become a separately released add-on to Istio that is part of the
&lt;a href="https://github.com/istio-ecosystem/">istio-ecosystem&lt;/a>.&lt;/p>
&lt;h2 id="fewer-moving-parts">Fewer moving parts&lt;/h2>
&lt;p>We are also simplifying the deployment of the rest of the control plane. To
that end, we combined several of the control plane components into a single
component: Istiod. This binary includes the features of Pilot, Citadel, Galley,
and the sidecar injector. This approach improves many aspects of installing and
managing Istio &amp;ndash; reducing installation and configuration complexity,
maintenance effort, and issue diagnosis time while increasing responsiveness.
Read more about Istiod in
&lt;a href="https://blog.christianposta.com/microservices/istio-as-an-example-of-when-not-to-do-microservices/">this post from Christian Posta&lt;/a>.&lt;/p>
&lt;p>We are shipping Istiod as the default for all profiles in 1.5.&lt;/p>
&lt;p>To reduce the per-node footprint, we are getting rid of the node-agent, used to
distribute certificates, and moving its functionality to the istio-agent, which
already runs in each Pod. For those of you who like pictures we are moving from
this &amp;hellip;&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/tradewinds-2020/architecture-pre-istiod.svg" title="The Istio architecture today">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/tradewinds-2020/architecture-pre-istiod.svg" alt="Istio architecture with Pilot, Mixer, Citadel, Sidecar injector" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Istio architecture today&lt;/figcaption>
&lt;/figure>
&lt;p>to this&amp;hellip;&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/tradewinds-2020/architecture-post-istiod.svg" title="The Istio architecture in 2020">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/tradewinds-2020/architecture-post-istiod.svg" alt="Istio architecture with Istiod" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Istio architecture in 2020&lt;/figcaption>
&lt;/figure>
&lt;p>In 2020, we will continue to invest in onboarding to achieve our goal of a
&amp;ldquo;zero config&amp;rdquo; default that doesn’t require you to change any of your
application&amp;rsquo;s configuration to take advantage of most Istio features.&lt;/p>
&lt;h2 id="improved-lifecycle-management">Improved lifecycle management&lt;/h2>
&lt;p>To improve Istio’s life-cycle management, we moved to an
&lt;a href="https://kubernetes.io/docs/concepts/extend-kubernetes/operator/">operator&lt;/a>-based
installation. We introduced the
&lt;strong>&lt;a href="/v1.12/docs/setup/install/istioctl/">IstioOperator CRD and two installation modes&lt;/a>&lt;/strong>:&lt;/p>
&lt;ul>
&lt;li>Human-triggered: use istioctl to apply the settings to the cluster.&lt;/li>
&lt;li>Machine-triggered: use a controller that is continually watching for changes
in that CRD and affecting those in real time.&lt;/li>
&lt;/ul>
&lt;p>In 2020 upgrades will getting easier too. We will add support for &amp;ldquo;canarying&amp;rdquo;
new versions of the Istio control plane, which allows you to run a new version
alongside the existing version and gradually switch your data plane over to use
the new one.&lt;/p>
&lt;h2 id="secure-by-default">Secure By Default&lt;/h2>
&lt;p>Istio already provides the fundamentals for strong service security: reliable
workload identity, robust access policies and comprehensive audit logging. We’re
stabilizing APIs for these features; many Alpha APIs are moving to Beta in 1.5,
and we expect them to all be v1 by the end of 2020. To learn more about the
status of our APIs, see our
&lt;a href="/v1.12/docs/releases/feature-stages/#istio-features">features page&lt;/a>.&lt;/p>
&lt;p>Network traffic is also becoming more secure by default. After many users
enabled it in preview,
&lt;a href="/v1.12/docs/tasks/security/authentication/authn-policy/#auto-mutual-tls">automated rollout of mutual TLS&lt;/a>
is becoming the recommended practice in Istio 1.5.&lt;/p>
&lt;p>In addition we will make Istio require fewer privileges and simplify its
dependencies which in turn make it a more robust system. Historically, you had
to mount certificates into Envoy using Kubernetes Secrets, which were mounted as
files into each proxy. By leveraging the
&lt;a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/security/secret">Secret Discovery Service&lt;/a>
we can distribute these certificates securely without concern of them being
intercepted by other workloads on the machine. This mode will become the default
in 1.5.&lt;/p>
&lt;p>Getting rid of the node-agent not only simplifies the deployment, but also
removes the requirement for a cluster-wide &lt;code>PodSecurityPolicy&lt;/code>, further
improving the security posture of your cluster.&lt;/p>
&lt;h2 id="other-features">Other features&lt;/h2>
&lt;p>Here&amp;rsquo;s a snapshot of some more exciting things you can expect from Istio in
2020:&lt;/p>
&lt;ul>
&lt;li>Integration with more hosted Kubernetes environments - service meshes
powered by Istio are currently available from 15 vendors, including Google, IBM,
Red Hat, VMware, Alibaba and Huawei&lt;/li>
&lt;li>More investment in &lt;code>istioctl&lt;/code> and its ability to help diagnose problems&lt;/li>
&lt;li>Better integration of VM-based workloads into meshes&lt;/li>
&lt;li>Continued work towards making multi-cluster and multi-network meshes easier to
configure, maintain, and run&lt;/li>
&lt;li>Integration with more service discovery systems, including
Functions-as-a-Service&lt;/li>
&lt;li>Implementation of the new
&lt;a href="https://kubernetes-sigs.github.io/service-apis/">Kubernetes service APIs&lt;/a>,
which are currently in development&lt;/li>
&lt;li>An &lt;a href="https://github.com/istio/enhancements/">enhancement repository&lt;/a>,
to track feature development&lt;/li>
&lt;li>Making it easier to run Istio without needing Kubernetes!&lt;/li>
&lt;/ul>
&lt;p>From the seas to &lt;a href="https://www.youtube.com/watch?v=YjZ4AZ7hRM0">the skies&lt;/a>,
we&amp;rsquo;re excited to see where you take Istio next.&lt;/p></description><pubDate>Tue, 03 Mar 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/tradewinds-2020/</link><author>Istio Team</author><guid isPermaLink="true">/v1.12/blog/2020/tradewinds-2020/</guid><category>roadmap</category><category>security</category><category>performance</category><category>operator</category></item><item><title>Remove cross-pod unix domain sockets</title><description>&lt;p>In Istio versions before 1.5, during secret discovery service (SDS) execution,
the SDS client and the SDS server communicate through a cross-pod Unix domain
socket (UDS), which needs to be protected by Kubernetes pod security policies.&lt;/p>
&lt;p>With Istio 1.5, Pilot Agent, Envoy, and Citadel Agent will be running in
the same container (the architecture is shown in the following diagram).
To defend against attackers eavesdropping on the cross-pod UDS between Envoy (SDS client)
and Citadel Agent (SDS server), Istio 1.5 merges Pilot Agent and Citadel Agent
into a single Istio Agent and makes the UDS between Envoy and Citadel Agent
private to the Istio Agent container.
The Istio Agent container is deployed as the sidecar of the application service container.&lt;/p>
&lt;figure style="width:70%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:80.39622513132818%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/istio-agent/istio_agent.svg" title="The architecture of Istio Agent">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/istio-agent/istio_agent.svg" alt="The architecture of Istio Agent" />
&lt;/a>
&lt;/div>
&lt;figcaption>The architecture of Istio Agent&lt;/figcaption>
&lt;/figure></description><pubDate>Thu, 20 Feb 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/istio-agent/</link><author>Lei Tang (Google)</author><guid isPermaLink="true">/v1.12/blog/2020/istio-agent/</guid><category>security</category><category>secret discovery service</category><category>unix domain socket</category></item><item><title>Multicluster Istio configuration and service discovery using Admiral</title><description>
&lt;p>At Intuit, we read the blog post &lt;a href="/v1.12/blog/2019/isolated-clusters/">Multi-Mesh Deployments for Isolation and Boundary Protection&lt;/a> and immediately related to some of the problems mentioned.
We realized that even though we wanted to configure a single multi-cluster mesh, instead of a federation of multiple meshes
as described in the blog post, the same non-uniform naming issues also applied in our environment.
This blog post explains how we solved these problems using &lt;a href="https://github.com/istio-ecosystem/admiral">Admiral&lt;/a>, an open source project under &lt;code>istio-ecosystem&lt;/code> in GitHub.&lt;/p>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>Using Istio, we realized the configuration for multi-cluster was complex and challenging to maintain over time. As a result, we chose the model described in &lt;a href="https://istio.io/v1.6/docs/setup/install/multicluster/gateways/#deploy-the-istio-control-plane-in-each-cluster">Multi-Cluster Istio Service Mesh with replicated control planes&lt;/a> for scalability and other operational reasons. Following this model, we had to solve these key requirements before widely adopting an Istio service mesh:&lt;/p>
&lt;ul>
&lt;li>Creation of service DNS entries decoupled from the namespace, as described in &lt;a href="/v1.12/blog/2019/isolated-clusters/#features-of-multi-mesh-deployments">Features of multi-mesh deployments&lt;/a>.&lt;/li>
&lt;li>Service discovery across many clusters.&lt;/li>
&lt;li>Supporting active-active &amp;amp; HA/DR deployments. We also had to support these crucial resiliency patterns with services being deployed in globally unique namespaces across discrete clusters.&lt;/li>
&lt;/ul>
&lt;p>We have over 160 Kubernetes clusters with a globally unique namespace name across all clusters. In this configuration, we can have the same service workload deployed in different regions running in namespaces with different names. As a result, following the routing strategy mentioned in &lt;a href="/v1.12/blog/2019/multicluster-version-routing">Multicluster version routing&lt;/a>, the example name &lt;code>foo.namespace.global&lt;/code> wouldn&amp;rsquo;t work across clusters. We needed a globally unique and discoverable service DNS that resolves service instances in multiple clusters, each instance running/addressable with its own unique Kubernetes FQDN. For example, &lt;code>foo.global&lt;/code> should resolve to both &lt;code>foo.uswest2.svc.cluster.local&lt;/code> &amp;amp; &lt;code>foo.useast2.svc.cluster.local&lt;/code> if &lt;code>foo&lt;/code> is running in two Kubernetes clusters with different names.
Also, our services need additional DNS names with different resolution and global routing properties. For example, &lt;code>foo.global&lt;/code> should resolve locally first, then route to a remote instance using topology routing, while &lt;code>foo-west.global&lt;/code> and &lt;code>foo-east.global&lt;/code> (names used for testing) should always resolve to the respective regions.&lt;/p>
&lt;h2 id="contextual-configuration">Contextual Configuration&lt;/h2>
&lt;p>After further investigation, it was apparent that configuration needed to be contextual: each cluster needs a configuration specifically tailored for its view of the world.&lt;/p>
&lt;p>For example, we have a payments service consumed by orders and reports. The payments service has a HA/DR deployment across &lt;code>us-east&lt;/code> (cluster 3) and &lt;code>us-west&lt;/code> (cluster 2). The payments service is deployed in namespaces with different names in each region. The orders service is deployed in a different cluster as payments in &lt;code>us-west&lt;/code> (cluster 1). The reports service is deployed in the same cluster as payments in &lt;code>us-west&lt;/code> (cluster 2).&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:60.81558869142712%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/multi-cluster-mesh-automation/Istio_mesh_example.svg" title="Cross cluster workload communication with Istio">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/multi-cluster-mesh-automation/Istio_mesh_example.svg" alt="Example of calling a workload in Istio multicluster" />
&lt;/a>
&lt;/div>
&lt;figcaption>Cross cluster workload communication with Istio&lt;/figcaption>
&lt;/figure>
&lt;p>Istio &lt;code>ServiceEntry&lt;/code> yaml for payments service in Cluster 1 and Cluster 2 below illustrates the contextual configuration that other services need to use the payments service:&lt;/p>
&lt;p>Cluster 1 Service Entry&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: payments.global-se
spec:
addresses:
- 240.0.0.10
endpoints:
- address: ef394f...us-east-2.elb.amazonaws.com
locality: us-east-2
ports:
http: 15443
- address: ad38bc...us-west-2.elb.amazonaws.com
locality: us-west-2
ports:
http: 15443
hosts:
- payments.global
location: MESH_INTERNAL
ports:
- name: http
number: 80
protocol: http
resolution: DNS
&lt;/code>&lt;/pre>
&lt;p>Cluster 2 Service Entry&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: payments.global-se
spec:
addresses:
- 240.0.0.10
endpoints:
- address: ef39xf...us-east-2.elb.amazonaws.com
locality: us-east-2
ports:
http: 15443
- address: payments.default.svc.cluster.local
locality: us-west-2
ports:
http: 80
hosts:
- payments.global
location: MESH_INTERNAL
ports:
- name: http
number: 80
protocol: http
resolution: DNS
&lt;/code>&lt;/pre>
&lt;p>The payments &lt;code>ServiceEntry&lt;/code> (Istio CRD) from the point of view of the reports service in Cluster 2, would set the locality &lt;code>us-west&lt;/code> pointing to the local Kubernetes FQDN and locality &lt;code>us-east&lt;/code> pointing to the &lt;code>istio-ingressgateway&lt;/code> (load balancer) for Cluster 3.
The payments &lt;code>ServiceEntry&lt;/code> from the point of view of the orders service in Cluster 1, will set the locality &lt;code>us-west&lt;/code> pointing to Cluster 2 &lt;code>istio-ingressgateway&lt;/code> and locality &lt;code>us-east&lt;/code> pointing to the &lt;code>istio-ingressgateway&lt;/code> for Cluster 3.&lt;/p>
&lt;p>But wait, there&amp;rsquo;s even more complexity: What if the payment services want to move traffic to the &lt;code>us-east&lt;/code> region for a planned maintenance in &lt;code>us-west&lt;/code>? This would require the payments service to change the Istio configuration in all of their clients&amp;rsquo; clusters. This would be nearly impossible to do without automation.&lt;/p>
&lt;h2 id="admiral-to-the-rescue-admiral-is-that-automation">Admiral to the Rescue: Admiral is that Automation&lt;/h2>
&lt;p>&lt;em>Admiral is a controller of Istio control planes.&lt;/em>&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:69.43866943866944%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2020/multi-cluster-mesh-automation/Istio_mesh_example_with_admiral.svg" title="Cross cluster workload communication with Istio and Admiral">
&lt;img class="element-to-stretch" src="/v1.12/blog/2020/multi-cluster-mesh-automation/Istio_mesh_example_with_admiral.svg" alt="Example of calling a workload in Istio multicluster with Admiral" />
&lt;/a>
&lt;/div>
&lt;figcaption>Cross cluster workload communication with Istio and Admiral&lt;/figcaption>
&lt;/figure>
&lt;p>Admiral provides automatic configuration for an Istio mesh spanning multiple clusters to work as a single mesh based on a unique service identifier that associates workloads running on multiple clusters to a service. It also provides automatic provisioning and syncing of Istio configuration across clusters. This removes the burden on developers and mesh operators, which helps scale beyond a few clusters.&lt;/p>
&lt;h2 id="admiral-crds">Admiral CRDs&lt;/h2>
&lt;h3 id="global-traffic-routing">Global Traffic Routing&lt;/h3>
&lt;p>With Admiral’s global traffic policy CRD, the payments service can update regional traffic weights and Admiral updates the Istio configuration in all clusters that consume the payments service.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: admiral.io/v1alpha1
kind: GlobalTrafficPolicy
metadata:
name: payments-gtp
spec:
selector:
identity: payments
policy:
- dns: default.payments.global
lbType: 1
target:
- region: us-west-2/*
weight: 10
- region: us-east-2/*
weight: 90
&lt;/code>&lt;/pre>
&lt;p>In the example above, 90% of the payments service traffic is routed to the &lt;code>us-east&lt;/code> region. This Global Traffic Configuration is automatically converted into Istio configuration and contextually mapped into Kubernetes clusters to enable multi-cluster global routing for the payments service for its clients within the Mesh.&lt;/p>
&lt;p>This Global Traffic Routing feature relies on Istio&amp;rsquo;s locality load-balancing per service available in Istio 1.5 or later.&lt;/p>
&lt;h3 id="dependency">Dependency&lt;/h3>
&lt;p>The Admiral &lt;code>Dependency&lt;/code> CRD allows us to specify a service&amp;rsquo;s dependencies based on a service identifier. This optimizes the delivery of Admiral generated configuration only to the required clusters where the dependent clients of a service are running (instead of writing it to all clusters). Admiral also configures and/or updates the Sidecar Istio CRD in the client&amp;rsquo;s workload namespace to limit the Istio configuration to only its dependencies. We use service-to-service authorization information recorded elsewhere to generate this &lt;code>dependency&lt;/code> records for Admiral to use.&lt;/p>
&lt;p>An example &lt;code>dependency&lt;/code> for the &lt;code>orders&lt;/code> service:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: admiral.io/v1alpha1
kind: Dependency
metadata:
name: dependency
namespace: admiral
spec:
source: orders
identityLabel: identity
destinations:
- payments
&lt;/code>&lt;/pre>
&lt;p>&lt;code>Dependency&lt;/code> is optional and a missing dependency for a service will result in an Istio configuration for that service pushed to all clusters.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>Admiral provides a new Global Traffic Routing and unique service naming functionality to address some challenges posed by the Istio model described in &lt;a href="https://istio.io/v1.6/docs/setup/install/multicluster/gateways/#deploy-the-istio-control-plane-in-each-cluster">multi-cluster deployment with replicated control planes&lt;/a>. It removes the need for manual configuration synchronization between clusters and generates contextual configuration for each cluster. This makes it possible to operate a Service Mesh composed of many Kubernetes clusters.&lt;/p>
&lt;p>We think Istio/Service Mesh community would benefit from this approach, so we &lt;a href="https://github.com/istio-ecosystem/admiral">open sourced Admiral&lt;/a> and would love your feedback and support!&lt;/p></description><pubDate>Sun, 05 Jan 2020 00:00:00 +0000</pubDate><link>/v1.12/blog/2020/multi-cluster-mesh-automation/</link><author>Anil Attuluri (Intuit), Jason Webb (Intuit)</author><guid isPermaLink="true">/v1.12/blog/2020/multi-cluster-mesh-automation/</guid><category>traffic-management</category><category>automation</category><category>configuration</category><category>multicluster</category><category>multi-mesh</category><category>gateway</category><category>federated</category><category>globalidentifer</category></item><item><title>Secure Webhook Management</title><description>&lt;p>Istio has two webhooks: Galley and the sidecar injector.
Galley validates Kubernetes resources and the sidecar injector injects sidecar
containers into Istio.&lt;/p>
&lt;p>By default, Galley and the sidecar injector manage their own webhook configurations.
This can pose a security risk if they are compromised, for example, through buffer overflow attacks.
Configuring a webhook is a highly privileged operation as a webhook may monitor and mutate all
Kubernetes resources.&lt;/p>
&lt;p>In the following example, the attacker compromises
Galley and modifies the webhook configuration of Galley to eavesdrop on all Kubernetes secrets
(the &lt;code>clientConfig&lt;/code> is modified by the attacker to direct the &lt;code>secrets&lt;/code> resources to
a service owned by the attacker).&lt;/p>
&lt;figure style="width:70%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:44.37367303609342%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/webhook/example_attack.png" title="An example attack">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/webhook/example_attack.png" alt="An example attack" />
&lt;/a>
&lt;/div>
&lt;figcaption>An example attack&lt;/figcaption>
&lt;/figure>
&lt;p>To protect against this kind of attack, Istio 1.4 introduces a new feature to securely manage
webhooks using &lt;code>istioctl&lt;/code>:&lt;/p>
&lt;ol>
&lt;li>&lt;p>&lt;code>istioctl&lt;/code>, instead of Galley and the sidecar injector, manage the webhook configurations.
Galley and the sidecar injector are de-privileged so even if they are compromised, they
will not be able to alter the webhook configurations.&lt;/p>&lt;/li>
&lt;li>&lt;p>Before configuring a webhook, &lt;code>istioctl&lt;/code> will verify the webhook server is up
and that the certificate chain used by the webhook server is valid. This reduces the errors
that can occur before a server is ready or if a server has invalid certificates.&lt;/p>&lt;/li>
&lt;/ol>
&lt;p>To try this new feature, refer to the &lt;a href="https://archive.istio.io/v1.4/docs/tasks/security/webhook">Istio webhook management task&lt;/a>.&lt;/p></description><pubDate>Thu, 14 Nov 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/webhook/</link><author>Lei Tang (Google)</author><guid isPermaLink="true">/v1.12/blog/2019/webhook/</guid><category>security</category><category>kubernetes</category><category>webhook</category></item><item><title>Introducing the Istio v1beta1 Authorization Policy</title><description>
&lt;p>Istio 1.4 introduces the
&lt;a href="/v1.12/docs/reference/config/security/authorization-policy/">&lt;code>v1beta1&lt;/code> authorization policy&lt;/a>,
which is a major update to the previous &lt;code>v1alpha1&lt;/code> role-based access control
(RBAC) policy. The new policy provides these improvements:&lt;/p>
&lt;ul>
&lt;li>Aligns with Istio configuration model.&lt;/li>
&lt;li>Improves the user experience by simplifying the API.&lt;/li>
&lt;li>Supports more use cases (e.g. Ingress/Egress gateway support) without
added complexity.&lt;/li>
&lt;/ul>
&lt;p>The &lt;code>v1beta1&lt;/code> policy is not backward compatible and requires a one time
conversion. A tool is provided to automate this process. The previous
configuration resources &lt;code>ClusterRbacConfig&lt;/code>, &lt;code>ServiceRole&lt;/code>, and
&lt;code>ServiceRoleBinding&lt;/code> will not be supported from Istio 1.6 onwards.&lt;/p>
&lt;p>This post describes the new &lt;code>v1beta1&lt;/code> authorization policy model, its
design goals and the migration from &lt;code>v1alpha1&lt;/code> RBAC policies. See the
&lt;a href="/v1.12/docs/concepts/security/#authorization">authorization concept page&lt;/a>
for a detailed in-depth explanation of the &lt;code>v1beta1&lt;/code> authorization policy.&lt;/p>
&lt;p>We welcome your feedback about the &lt;code>v1beta1&lt;/code> authorization policy at
&lt;a href="https://discuss.istio.io/c/security">discuss.istio.io&lt;/a>.&lt;/p>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>To date, Istio provided RBAC policies to enforce access control on
&lt;span class="term" data-title="Service" data-body="&amp;lt;p&amp;gt;A delineated group of related behaviors within a &amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-mesh&amp;#34;&amp;gt;service mesh&amp;lt;/a&amp;gt;. Services are identified using a
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-name&amp;#34;&amp;gt;service name&amp;lt;/a&amp;gt;,
and Istio policies such as load balancing and routing are applied using these names.
A service is typically materialized by one or more &amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-endpoint&amp;#34;&amp;gt;service endpoints&amp;lt;/a&amp;gt;, and may consist of multiple
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-version&amp;#34;&amp;gt;service versions&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">services&lt;/span> using three configuration
resources: &lt;code>ClusterRbacConfig&lt;/code>, &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code>.
With this API, users have been able to enforce control access at mesh-level,
namespace-level and service-level. Like other RBAC policies, Istio RBAC uses
the same concept of role and binding for granting permissions to identities.&lt;/p>
&lt;p>Although Istio RBAC has been working reliably, we&amp;rsquo;ve found that many
improvements were possible.&lt;/p>
&lt;p>For example, users have mistakenly assumed that access control enforcement
happens at service-level because &lt;code>ServiceRole&lt;/code> uses service to specify where
to apply the policy, however, the policy is actually applied on
&lt;span class="term" data-title="Workload" data-body="&amp;lt;p&amp;gt;A binary deployed by &amp;lt;a href=&amp;#34;/docs/reference/glossary/#operator&amp;#34;&amp;gt;operators&amp;lt;/a&amp;gt; to deliver some function of a service mesh application.
Workloads have names, namespaces, and unique ids. These properties are available in policy and telemetry configuration
using the following &amp;lt;a href=&amp;#34;/docs/reference/glossary/#attribute&amp;#34;&amp;gt;attributes&amp;lt;/a&amp;gt;:&amp;lt;/p&amp;gt;
&amp;lt;ul&amp;gt;
&amp;lt;li&amp;gt;&amp;lt;code&amp;gt;source.workload.name&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;source.workload.namespace&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;source.workload.uid&amp;lt;/code&amp;gt;&amp;lt;/li&amp;gt;
&amp;lt;li&amp;gt;&amp;lt;code&amp;gt;destination.workload.name&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;destination.workload.namespace&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;destination.workload.uid&amp;lt;/code&amp;gt;&amp;lt;/li&amp;gt;
&amp;lt;/ul&amp;gt;
&amp;lt;p&amp;gt;In Kubernetes, a workload typically corresponds to a Kubernetes deployment,
while a &amp;lt;a href=&amp;#34;/docs/reference/glossary/#workload-instance&amp;#34;&amp;gt;workload instance&amp;lt;/a&amp;gt; corresponds to an individual &amp;lt;a href=&amp;#34;/docs/reference/glossary/#pod&amp;#34;&amp;gt;pod&amp;lt;/a&amp;gt; managed
by the deployment.&amp;lt;/p&amp;gt;
">workloads&lt;/span>, the service is only used to
find the corresponding workload. This nuance is significant when multiple
services are referring to the same workload. A &lt;code>ServiceRole&lt;/code> for service A
will also affect service B if the two services are referring to the same
workload, which can cause confusion and incorrect configuration.&lt;/p>
&lt;p>An other example is that it&amp;rsquo;s proven difficult for users to maintain and
manage the Istio RBAC configurations because of the need to deeply understand
three related resources.&lt;/p>
&lt;h2 id="design-goals">Design goals&lt;/h2>
&lt;p>The new &lt;code>v1beta1&lt;/code> authorization policy had several design goals:&lt;/p>
&lt;ul>
&lt;li>&lt;p>Align with &lt;a href="https://goo.gl/x3STjD">Istio Configuration Model&lt;/a> for better
clarity on the policy target. The configuration model provides a unified
configuration hierarchy, resolution and target selection.&lt;/p>&lt;/li>
&lt;li>&lt;p>Improve the user experience by simplifying the API. It&amp;rsquo;s easier to manage
one custom resource definition (CRD) that includes all access control
specifications, instead of multiple CRDs.&lt;/p>&lt;/li>
&lt;li>&lt;p>Support more use cases without added complexity. For example, allow the
policy to be applied on Ingress/Egress gateway to enforce access control
for traffic entering/exiting the mesh.&lt;/p>&lt;/li>
&lt;/ul>
&lt;h2 id="authorizationpolicy">&lt;code>AuthorizationPolicy&lt;/code>&lt;/h2>
&lt;p>An &lt;a href="/v1.12/docs/reference/config/security/authorization-policy/">&lt;code>AuthorizationPolicy&lt;/code> custom resource&lt;/a>
enables access control on workloads. This section gives an overview of the
changes in the &lt;code>v1beta1&lt;/code> authorization policy.&lt;/p>
&lt;p>An &lt;code>AuthorizationPolicy&lt;/code> includes a &lt;code>selector&lt;/code> and a list of &lt;code>rule&lt;/code>.
The &lt;code>selector&lt;/code> specifies on which workload to apply the policy and the
list of &lt;code>rule&lt;/code> specifies the detailed access control rule for the workload.&lt;/p>
&lt;p>The &lt;code>rule&lt;/code> is additive, which means a request is allowed if any &lt;code>rule&lt;/code>
allows the request. Each &lt;code>rule&lt;/code> includes a list of &lt;code>from&lt;/code>, &lt;code>to&lt;/code> and
&lt;code>when&lt;/code>, which specifies &lt;strong>who&lt;/strong> is allowed to do &lt;strong>what&lt;/strong> under which
&lt;strong>conditions&lt;/strong>.&lt;/p>
&lt;p>The &lt;code>selector&lt;/code> replaces the functionality provided by &lt;code>ClusterRbacConfig&lt;/code>
and the &lt;code>services&lt;/code> field in &lt;code>ServiceRole&lt;/code>. The &lt;code>rule&lt;/code> replaces the other
fields in the &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code>.&lt;/p>
&lt;h3 id="example">Example&lt;/h3>
&lt;p>The following authorization policy applies to workloads with &lt;code>app: httpbin&lt;/code>
and &lt;code>version: v1&lt;/code> label in the &lt;code>foo&lt;/code> namespace:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin
namespace: foo
spec:
selector:
matchLabels:
app: httpbin
version: v1
rules:
- from:
- source:
principals: [&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;]
to:
- operation:
methods: [&amp;#34;GET&amp;#34;]
when:
- key: request.headers[version]
values: [&amp;#34;v1&amp;#34;, &amp;#34;v2&amp;#34;]
&lt;/code>&lt;/pre>
&lt;p>The policy allows principal &lt;code>cluster.local/ns/default/sa/sleep&lt;/code> to access the
workload using the &lt;code>GET&lt;/code> method when the request includes a &lt;code>version&lt;/code> header
of value &lt;code>v1&lt;/code> or &lt;code>v2&lt;/code>. Any requests not matched with the policy will be denied
by default.&lt;/p>
&lt;p>Assuming the &lt;code>httpbin&lt;/code> service is defined as:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Service
metadata:
name: httpbin
namespace: foo
spec:
selector:
app: httpbin
version: v1
ports:
# omitted
&lt;/code>&lt;/pre>
&lt;p>You would need to configure three resources to achieve the same result in
&lt;code>v1alpha1&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ClusterRbacConfig
metadata:
name: default
spec:
mode: &amp;#39;ON_WITH_INCLUSION&amp;#39;
inclusion:
services: [&amp;#34;httpbin.foo.svc.cluster.local&amp;#34;]
---
apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRole
metadata:
name: httpbin
namespace: foo
spec:
rules:
- services: [&amp;#34;httpbin.foo.svc.cluster.local&amp;#34;]
methods: [&amp;#34;GET&amp;#34;]
constraints:
- key: request.headers[version]
values: [&amp;#34;v1&amp;#34;, &amp;#34;v2&amp;#34;]
---
apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRoleBinding
metadata:
name: httpbin
namespace: foo
spec:
subjects:
- user: &amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;
roleRef:
kind: ServiceRole
name: &amp;#34;httpbin&amp;#34;
&lt;/code>&lt;/pre>
&lt;h3 id="workload-selector">Workload selector&lt;/h3>
&lt;p>A major change in the &lt;code>v1beta1&lt;/code> authorization policy is that it now uses
workload selector to specify where to apply the policy. This is the same
workload selector used in the &lt;code>Gateway&lt;/code>, &lt;code>Sidecar&lt;/code> and &lt;code>EnvoyFilter&lt;/code>
configurations.&lt;/p>
&lt;p>The workload selector makes it clear that the policy is applied and enforced
on workloads instead of services. If a policy applies to a workload that is
used by multiple different services, the same policy will affect the traffic
to all the different services.&lt;/p>
&lt;p>You can simply leave the &lt;code>selector&lt;/code> empty to apply the policy to all
workloads in a namespace. The following policy applies to all workloads in
the namespace &lt;code>bar&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: policy
namespace: bar
spec:
rules:
# omitted
&lt;/code>&lt;/pre>
&lt;h3 id="root-namespace">Root namespace&lt;/h3>
&lt;p>A policy in the root namespace applies to all workloads in the mesh in every
namespaces. The root namespace is configurable in the
&lt;a href="/v1.12/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig">&lt;code>MeshConfig&lt;/code>&lt;/a>
and has the default value of &lt;code>istio-system&lt;/code>.&lt;/p>
&lt;p>For example, you installed Istio in &lt;code>istio-system&lt;/code> namespace and deployed
workloads in &lt;code>default&lt;/code> and &lt;code>bookinfo&lt;/code> namespace. The root namespace is
changed to &lt;code>istio-config&lt;/code> from the default value. The following policy will
apply to workloads in every namespace including &lt;code>default&lt;/code>, &lt;code>bookinfo&lt;/code> and
the &lt;code>istio-system&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: policy
namespace: istio-config
spec:
rules:
# omitted
&lt;/code>&lt;/pre>
&lt;h3 id="ingress-egress-gateway-support">Ingress/Egress Gateway support&lt;/h3>
&lt;p>The &lt;code>v1beta1&lt;/code> authorization policy can also be applied on ingress/egress
gateway to enforce access control on traffic entering/leaving the mesh,
you only need to change the &lt;code>selector&lt;/code> to make select the ingress/egress
workload.&lt;/p>
&lt;p>The following policy applies to workloads with the
&lt;code>app: istio-ingressgateway&lt;/code> label:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: ingress
namespace: istio-system
spec:
selector:
matchLabels:
app: istio-ingressgateway
rules:
# omitted
&lt;/code>&lt;/pre>
&lt;p>Remember the authorization policy only applies to workloads in the same
namespace as the policy, unless the policy is applied in the root namespace:&lt;/p>
&lt;ul>
&lt;li>&lt;p>If you don&amp;rsquo;t change the default root namespace value (i.e. &lt;code>istio-system&lt;/code>),
the above policy will apply to workloads with the &lt;code>app: istio-ingressgateway&lt;/code>
label in &lt;strong>every&lt;/strong> namespace.&lt;/p>&lt;/li>
&lt;li>&lt;p>If you have changed the root namespace to a different value, the above
policy will only apply to workloads with the &lt;code>app: istio-ingressgateway&lt;/code>
label &lt;strong>only&lt;/strong> in the &lt;code>istio-system&lt;/code> namespace.&lt;/p>&lt;/li>
&lt;/ul>
&lt;h3 id="comparison">Comparison&lt;/h3>
&lt;p>The following table highlights the key differences between the old &lt;code>v1alpha1&lt;/code>
RBAC policies and the new &lt;code>v1beta1&lt;/code> authorization policy.&lt;/p>
&lt;h4 id="feature">Feature&lt;/h4>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Feature&lt;/th>
&lt;th>&lt;code>v1alpha1&lt;/code> RBAC policy&lt;/th>
&lt;th>&lt;code>v1beta1&lt;/code> Authorization Policy&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>API stability&lt;/td>
&lt;td>&lt;code>alpha&lt;/code>: &lt;strong>No&lt;/strong> backward compatible&lt;/td>
&lt;td>&lt;code>beta&lt;/code>: backward compatible &lt;strong>guaranteed&lt;/strong>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Number of CRDs&lt;/td>
&lt;td>Three: &lt;code>ClusterRbacConfig&lt;/code>, &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code>&lt;/td>
&lt;td>Only One: &lt;code>AuthorizationPolicy&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Policy target&lt;/td>
&lt;td>&lt;strong>service&lt;/strong>&lt;/td>
&lt;td>&lt;strong>workload&lt;/strong>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Deny-by-default behavior&lt;/td>
&lt;td>Enabled &lt;strong>explicitly&lt;/strong> by configuring &lt;code>ClusterRbacConfig&lt;/code>&lt;/td>
&lt;td>Enabled &lt;strong>implicitly&lt;/strong> with &lt;code>AuthorizationPolicy&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Ingress/Egress gateway support&lt;/td>
&lt;td>Not supported&lt;/td>
&lt;td>Supported&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>The &lt;code>&amp;quot;*&amp;quot;&lt;/code> value in policy&lt;/td>
&lt;td>Match all contents (empty and non-empty)&lt;/td>
&lt;td>Match non-empty contents only&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>The following tables show the relationship between the &lt;code>v1alpha1&lt;/code> and &lt;code>v1beta1&lt;/code> API.&lt;/p>
&lt;h4 id="clusterrbacconfig">&lt;code>ClusterRbacConfig&lt;/code>&lt;/h4>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>&lt;code>ClusterRbacConfig.Mode&lt;/code>&lt;/th>
&lt;th>&lt;code>AuthorizationPolicy&lt;/code>&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>&lt;code>OFF&lt;/code>&lt;/td>
&lt;td>No policy applied&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>ON&lt;/code>&lt;/td>
&lt;td>A deny-all policy applied in root namespace&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>ON_WITH_INCLUSION&lt;/code>&lt;/td>
&lt;td>policies should be applied to namespaces or workloads included by &lt;code>ClusterRbacConfig&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>ON_WITH_EXCLUSION&lt;/code>&lt;/td>
&lt;td>policies should be applied to namespaces or workloads excluded by &lt;code>ClusterRbacConfig&lt;/code>&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;h4 id="servicerole">&lt;code>ServiceRole&lt;/code>&lt;/h4>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>&lt;code>ServiceRole&lt;/code>&lt;/th>
&lt;th>&lt;code>AuthorizationPolicy&lt;/code>&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>&lt;code>services&lt;/code>&lt;/td>
&lt;td>&lt;code>selector&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>paths&lt;/code>&lt;/td>
&lt;td>&lt;code>paths&lt;/code> in &lt;code>to&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>methods&lt;/code>&lt;/td>
&lt;td>&lt;code>methods&lt;/code> in &lt;code>to&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>destination.ip&lt;/code> in constraint&lt;/td>
&lt;td>Not supported&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>destination.port&lt;/code> in constraint&lt;/td>
&lt;td>&lt;code>ports&lt;/code> in &lt;code>to&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>destination.labels&lt;/code> in constraint&lt;/td>
&lt;td>&lt;code>selector&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>destination.namespace&lt;/code> in constraint&lt;/td>
&lt;td>Replaced by the namespace of the policy, i.e. the &lt;code>namespace&lt;/code> in metadata&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>destination.user&lt;/code> in constraint&lt;/td>
&lt;td>Not supported&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>experimental.envoy.filters&lt;/code> in constraint&lt;/td>
&lt;td>&lt;code>experimental.envoy.filters&lt;/code> in &lt;code>when&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>request.headers&lt;/code> in constraint&lt;/td>
&lt;td>&lt;code>request.headers&lt;/code> in &lt;code>when&lt;/code>&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;h4 id="servicerolebinding">&lt;code>ServiceRoleBinding&lt;/code>&lt;/h4>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>&lt;code>ServiceRoleBinding&lt;/code>&lt;/th>
&lt;th>&lt;code>AuthorizationPolicy&lt;/code>&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>&lt;code>user&lt;/code>&lt;/td>
&lt;td>&lt;code>principals&lt;/code> in &lt;code>from&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>group&lt;/code>&lt;/td>
&lt;td>&lt;code>request.auth.claims[group]&lt;/code> in &lt;code>when&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>source.ip&lt;/code> in property&lt;/td>
&lt;td>&lt;code>ipBlocks&lt;/code> in &lt;code>from&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>source.namespace&lt;/code> in property&lt;/td>
&lt;td>&lt;code>namespaces&lt;/code> in &lt;code>from&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>source.principal&lt;/code> in property&lt;/td>
&lt;td>&lt;code>principals&lt;/code> in &lt;code>from&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>request.headers&lt;/code> in property&lt;/td>
&lt;td>&lt;code>request.headers&lt;/code> in &lt;code>when&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>request.auth.principal&lt;/code> in property&lt;/td>
&lt;td>&lt;code>requestPrincipals&lt;/code> in &lt;code>from&lt;/code> or &lt;code>request.auth.principal&lt;/code> in &lt;code>when&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>request.auth.audiences&lt;/code> in property&lt;/td>
&lt;td>&lt;code>request.auth.audiences&lt;/code> in &lt;code>when&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>request.auth.presenter&lt;/code> in property&lt;/td>
&lt;td>&lt;code>request.auth.presenter&lt;/code> in &lt;code>when&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;code>request.auth.claims&lt;/code> in property&lt;/td>
&lt;td>&lt;code>request.auth.claims&lt;/code> in &lt;code>when&lt;/code>&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>Beyond all the differences, the &lt;code>v1beta1&lt;/code> policy is enforced by the same
engine in Envoy and supports the same authenticated identity (mutual TLS or
JWT), condition and other primitives (e.g. IP, port and etc.) as the
&lt;code>v1alpha1&lt;/code> policy.&lt;/p>
&lt;h2 id="future-of-the-v1alpha1-policy">Future of the &lt;code>v1alpha1&lt;/code> policy&lt;/h2>
&lt;p>The &lt;code>v1alpha1&lt;/code> RBAC policy (&lt;code>ClusterRbacConfig&lt;/code>, &lt;code>ServiceRole&lt;/code>, and
&lt;code>ServiceRoleBinding&lt;/code>) is deprecated by the &lt;code>v1beta1&lt;/code> authorization policy.&lt;/p>
&lt;p>Istio 1.4 continues to support the &lt;code>v1alpha1&lt;/code> RBAC policy to give you
enough time to move away from the alpha policies.&lt;/p>
&lt;h2 id="migration-from-the-v1alpha1-policy">Migration from the &lt;code>v1alpha1&lt;/code> policy&lt;/h2>
&lt;p>Istio only supports one of the two versions for a given workload:&lt;/p>
&lt;ul>
&lt;li>If there is only &lt;code>v1beta1&lt;/code> policy for a workload, the &lt;code>v1beta1&lt;/code> policy
will be used.&lt;/li>
&lt;li>If there is only &lt;code>v1alpha1&lt;/code> policy for a workload, the &lt;code>v1alpha1&lt;/code> policy
will be used.&lt;/li>
&lt;li>If there are both &lt;code>v1beta1&lt;/code> and &lt;code>v1alpha1&lt;/code> policies for a workload,
only the &lt;code>v1beta1&lt;/code> policy will be used and the the &lt;code>v1alpha1&lt;/code> policy
will be ignored.&lt;/li>
&lt;/ul>
&lt;h3 id="general-guideline">General Guideline&lt;/h3>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">When migrating to use &lt;code>v1beta1&lt;/code> policy for a given workload, make sure the
new &lt;code>v1beta1&lt;/code> policy covers all the existing &lt;code>v1alpha1&lt;/code> policies applied
for the workload, because the &lt;code>v1alpha1&lt;/code> policies applied for the workload
will be ignored after you applied the &lt;code>v1beta1&lt;/code> policies.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>The typical flow of migrating to &lt;code>v1beta1&lt;/code> policy is to start by checking the
&lt;code>ClusterRbacConfig&lt;/code> to decide which namespace or service is enabled with RBAC.&lt;/p>
&lt;p>For each service enabled with RBAC:&lt;/p>
&lt;ol>
&lt;li>Get the workload selector from the service definition.&lt;/li>
&lt;li>Create a &lt;code>v1beta1&lt;/code> policy with the workload selector.&lt;/li>
&lt;li>Update the &lt;code>v1beta1&lt;/code> policy for each &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code>
applied to the service.&lt;/li>
&lt;li>Apply the &lt;code>v1beta1&lt;/code> policy and monitor the traffic to make sure the
policy is working as expected.&lt;/li>
&lt;li>Repeat the process for the next service enabled with RBAC.&lt;/li>
&lt;/ol>
&lt;p>For each namespace enabled with RBAC:&lt;/p>
&lt;ol>
&lt;li>Apply a &lt;code>v1beta1&lt;/code> policy that denies all traffic to the given namespace.&lt;/li>
&lt;/ol>
&lt;h3 id="migration-example">Migration Example&lt;/h3>
&lt;p>Assume you have the following &lt;code>v1alpha1&lt;/code> policies for the &lt;code>httpbin&lt;/code> service
in the &lt;code>foo&lt;/code> namespace:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ClusterRbacConfig
metadata:
name: default
spec:
mode: &amp;#39;ON_WITH_INCLUSION&amp;#39;
inclusion:
namespaces: [&amp;#34;foo&amp;#34;]
---
apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRole
metadata:
name: httpbin
namespace: foo
spec:
rules:
- services: [&amp;#34;httpbin.foo.svc.cluster.local&amp;#34;]
methods: [&amp;#34;GET&amp;#34;]
---
apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRoleBinding
metadata:
name: httpbin
namespace: foo
spec:
subjects:
- user: &amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;
roleRef:
kind: ServiceRole
name: &amp;#34;httpbin&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>Migrate the above policies to &lt;code>v1beta1&lt;/code> in the following ways:&lt;/p>
&lt;ol>
&lt;li>&lt;p>Assume the &lt;code>httpbin&lt;/code> service has the following workload selector:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >selector:
app: httpbin
version: v1
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a &lt;code>v1beta1&lt;/code> policy with the workload selector:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin
namespace: foo
spec:
selector:
matchLabels:
app: httpbin
version: v1
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Update the &lt;code>v1beta1&lt;/code> policy with each &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code>
applied to the service:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin
namespace: foo
spec:
selector:
matchLabels:
app: httpbin
version: v1
rules:
- from:
- source:
principals: [&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;]
to:
- operation:
methods: [&amp;#34;GET&amp;#34;]
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Apply the &lt;code>v1beta1&lt;/code> policy and monitor the traffic to make sure it works
as expected.&lt;/p>&lt;/li>
&lt;li>&lt;p>Apply the following &lt;code>v1beta1&lt;/code> policy that denies all traffic to the
&lt;code>foo&lt;/code> namespace because the &lt;code>foo&lt;/code> namespace is enabled with RBAC:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: deny-all
namespace: foo
spec:
{}
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;p>Make sure the &lt;code>v1beta1&lt;/code> policy is working as expected and then you can delete
the &lt;code>v1alpha1&lt;/code> policies from the cluster.&lt;/p>
&lt;h3 id="automation-of-the-migration">Automation of the Migration&lt;/h3>
&lt;p>To help ease the migration, the &lt;code>istioctl experimental authz convert&lt;/code>
command is provided to automatically convert the &lt;code>v1alpha1&lt;/code> policies to
the &lt;code>v1beta1&lt;/code> policy.&lt;/p>
&lt;p>You can evaluate the command but it is experimental in Istio 1.4 and doesn&amp;rsquo;t
support the full &lt;code>v1alpha1&lt;/code> semantics as of the date of this blog post.&lt;/p>
&lt;p>The command to support the full &lt;code>v1alpha1&lt;/code> semantics is expected in a patch
release following Istio 1.4.&lt;/p></description><pubDate>Thu, 14 Nov 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/v1beta1-authorization-policy/</link><author>Yangmin Zhu (Google)</author><guid isPermaLink="true">/v1.12/blog/2019/v1beta1-authorization-policy/</guid><category>security</category><category>RBAC</category><category>access control</category><category>authorization</category></item><item><title>Introducing the Istio Operator</title><description>
&lt;p>Kubernetes &lt;a href="https://kubernetes.io/docs/concepts/extend-kubernetes/operator/">operators&lt;/a> provide
a pattern for encoding human operational knowledge in software and are a popular way to simplify
the administration of software infrastructure components. Istio is a natural candidate for an automated
operator as it is challenging to administer.&lt;/p>
&lt;p>Up until now, &lt;a href="https://github.com/helm/helm">Helm&lt;/a> has been the primary tool to install and upgrade Istio.
Istio 1.4 introduces a new method of &lt;a href="/v1.12/docs/setup/install/istioctl/">installation using istioctl&lt;/a>.
This new installation method builds on the strengths of Helm with the addition of the
following:&lt;/p>
&lt;ul>
&lt;li>Users only need to install one tool: &lt;code>istioctl&lt;/code>&lt;/li>
&lt;li>All API fields are validated&lt;/li>
&lt;li>Small customizations not in the API don&amp;rsquo;t require chart or API changes&lt;/li>
&lt;li>Version specific upgrade hooks can be easily and robustly implemented&lt;/li>
&lt;/ul>
&lt;p>The &lt;a href="https://archive.istio.io/1.4/docs/setup/install/helm/">Helm installation&lt;/a> method is in the process of deprecation. Upgrading from Istio
1.4 with a version not initially installed with Helm will also be replaced by a new
&lt;a href="https://archive.istio.io/v1.4/docs/setup/upgrade/istioctl-upgrade/">istioctl upgrade feature&lt;/a>.&lt;/p>
&lt;p>The new &lt;code>istioctl&lt;/code> installation commands use a
&lt;a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/">custom resource&lt;/a>
to configure the installation. The custom resource is part of a new Istio operator
implementation intended to simplify the common administrative tasks of installation, upgrade,
and complex configuration changes for Istio. Validation and checking for installation and upgrade
is tightly integrated with the tools to prevent common errors and simplify troubleshooting.&lt;/p>
&lt;h2 id="the-operator-api">The Operator API&lt;/h2>
&lt;p>Every operator implementation requires a
&lt;a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions">custom resource definition (CRD)&lt;/a>
to define its custom resource, that is, its API. Istio&amp;rsquo;s operator API is defined by the
&lt;a href="https://archive.istio.io/v1.4/docs/reference/config/istio.operator.v1alpha12.pb/">&lt;code>IstioControlPlane&lt;/code> CRD&lt;/a>,
which is generated from an
&lt;a href="https://github.com/istio/operator/blob/release-1.4/pkg/apis/istio/v1alpha2/istiocontrolplane_types.proto">&lt;code>IstioControlPlane&lt;/code> proto&lt;/a>.
The API supports all of Istio&amp;rsquo;s current &lt;a href="/v1.12/docs/setup/additional-setup/config-profiles/">configuration profiles&lt;/a>
using a single field to select the profile. For example, the following &lt;code>IstioControlPlane&lt;/code> resource
configures Istio using the &lt;code>demo&lt;/code> profile:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: install.istio.io/v1alpha2
kind: IstioControlPlane
metadata:
namespace: istio-operator
name: example-istiocontrolplane
spec:
profile: demo
&lt;/code>&lt;/pre>
&lt;p>You can then customize the configuration with additional settings. For example, to disable telemetry:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: install.istio.io/v1alpha2
kind: IstioControlPlane
metadata:
namespace: istio-operator
name: example-istiocontrolplane
spec:
profile: demo
telemetry:
enabled: false
&lt;/code>&lt;/pre>
&lt;h2 id="installing-with-hahahugoshortcode-s4-hbhb">Installing with istioctl&lt;/h2>
&lt;p>The recommended way to use the Istio operator API is through a new set of &lt;code>istioctl&lt;/code> commands.
For example, to install Istio into a cluster:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl manifest apply -f &amp;lt;your-istiocontrolplane-customresource&amp;gt;
&lt;/code>&lt;/pre>
&lt;p>Make changes to the installation configuration by editing the configuration file and executing
&lt;code>istioctl manifest apply&lt;/code> again.&lt;/p>
&lt;p>To upgrade to a new version of Istio:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl x upgrade -f &amp;lt;your-istiocontrolplane-config-changes&amp;gt;
&lt;/code>&lt;/pre>
&lt;p>In addition to specifying the complete configuration in an &lt;code>IstioControlPlane&lt;/code> resource,
the &lt;code>istioctl&lt;/code> commands can also be passed individual settings using a &lt;code>--set&lt;/code> flag:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl manifest apply --set telemetry.enabled=false
&lt;/code>&lt;/pre>
&lt;p>There are also a number of other &lt;code>istioctl&lt;/code> commands that, for example, help you list, display,
and compare configuration profiles and manifests.&lt;/p>
&lt;p>Refer to the Istio &lt;a href="/v1.12/docs/setup/install/istioctl">install instructions&lt;/a> for more details.&lt;/p>
&lt;h2 id="istio-controller-alpha">Istio Controller (alpha)&lt;/h2>
&lt;p>Operator implementations use a Kubernetes controller to continuously monitor their custom resource
and apply the corresponding configuration changes. The Istio controller monitors an &lt;code>IstioControlPlane&lt;/code>
resource and reacts to changes by updating the Istio installation configuration in the corresponding cluster.&lt;/p>
&lt;p>In the 1.4 release, the Istio controller is in the alpha phase of development and not fully
integrated with &lt;code>istioctl&lt;/code>. It is, however,
&lt;a href="/v1.12/docs/setup/install/operator/">available for experimentation&lt;/a> using &lt;code>kubectl&lt;/code> commands.
For example, to install the controller and a default version of Istio into your cluster,
run the following command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f https://&amp;lt;repo URL&amp;gt;/operator.yaml
$ kubectl apply -f https://&amp;lt;repo URL&amp;gt;/default-cr.yaml
&lt;/code>&lt;/pre>
&lt;p>You can then make changes to the Istio installation configuration:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl edit istiocontrolplane example-istiocontrolplane -n istio-system
&lt;/code>&lt;/pre>
&lt;p>As soon as the resource is updated, the controller will detect the changes and respond by updating
the Istio installation correspondingly.&lt;/p>
&lt;p>Both the operator controller and &lt;code>istioctl&lt;/code> commands share the same implementation. The significant
difference is the execution context. In the &lt;code>istioctl&lt;/code> case, the operation runs in the admin user’s
command execution and security context. In the controller case, a pod in the cluster runs the code
in its security context. In both cases, configuration is validated against a schema and the same correctness
checks are performed.&lt;/p>
&lt;h2 id="migration-from-helm">Migration from Helm&lt;/h2>
&lt;p>To help ease the transition from previous configurations using Helm,
&lt;code>istioctl&lt;/code> and the controller support pass-through access for the full Helm installation API.&lt;/p>
&lt;p>You can pass Helm configuration options using &lt;code>istioctl --set&lt;/code> by prepending the string &lt;code>values.&lt;/code> to the option name.
For example, instead of this Helm command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ helm template ... --set global.mtls.enabled=true
&lt;/code>&lt;/pre>
&lt;p>You can use this &lt;code>istioctl&lt;/code> command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl manifest generate ... --set values.global.mtls.enabled=true
&lt;/code>&lt;/pre>
&lt;p>You can also set Helm configuration values in an &lt;code>IstioControlPlane&lt;/code> custom resource.
See &lt;a href="/v1.12/docs/setup/install/istioctl/#customize-istio-settings-using-the-helm-api">Customize Istio settings using Helm&lt;/a>
for details.&lt;/p>
&lt;p>Another feature to help with the transition from Helm is the alpha
&lt;a href="/v1.12/docs/reference/commands/istioctl/#istioctl-manifest-migrate">istioctl manifest migrate&lt;/a> command.
This command can be used to automatically convert a Helm &lt;code>values.yaml&lt;/code> file to a corresponding
&lt;code>IstioControlPlane&lt;/code> configuration.&lt;/p>
&lt;h2 id="implementation">Implementation&lt;/h2>
&lt;p>Several frameworks have been created to help implement operators by generating stubs for some or all of
the components. The Istio operator was created with the help of a combination of
&lt;a href="https://github.com/kubernetes-sigs/kubebuilder">kubebuilder&lt;/a> and
&lt;a href="https://github.com/operator-framework">operator framework&lt;/a>. Istio&amp;rsquo;s installation now uses a proto to
describe the API such that runtime validation can be executed against a schema.&lt;/p>
&lt;p>More information about the implementation can be found in the README and ARCHITECTURE documents
in the &lt;a href="https://github.com/istio/operator">Istio operator repository&lt;/a>.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>Starting in Istio 1.4, Helm installation is being replaced by new &lt;code>istioctl&lt;/code> commands using
a new operator custom resource definition, &lt;code>IstioControlPlane&lt;/code>, for the configuration API.
An alpha controller is also available for early experimentation with the operator.&lt;/p>
&lt;p>The new &lt;code>istioctl&lt;/code> commands and operator controller both validate configuration schemas and perform a range of
checks for installation change or upgrade. These checks are tightly integrated with the tools to prevent
common errors and simplify troubleshooting.&lt;/p>
&lt;p>The Istio maintainers expect that this new approach will improve the user experience during Istio
installation and upgrade, better stabilize the installation API, and help users better manage and
monitor their Istio installations.&lt;/p>
&lt;p>We welcome your feedback about the new installation approach at &lt;a href="https://discuss.istio.io/">discuss.istio.io&lt;/a>.&lt;/p></description><pubDate>Thu, 14 Nov 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/introducing-istio-operator/</link><author>Martin Ostrowski (Google), Frank Budinsky (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/introducing-istio-operator/</guid><category>install</category><category>configuration</category><category>istioctl</category><category>operator</category></item><item><title>Introducing istioctl analyze</title><description>
&lt;p>Istio 1.4 introduces an experimental new tool to help you analyze and debug your clusters running Istio.&lt;/p>
&lt;p>&lt;a href="/v1.12/docs/reference/commands/istioctl/#istioctl-experimental-analyze">&lt;code>istioctl analyze&lt;/code>&lt;/a> is a diagnostic tool that detects potential issues with your
Istio configuration, as well as gives general insights to improve your configuration.
It can run against a live cluster or a set of local configuration files.
It can also run against a combination of the two, allowing you to catch problems before you
apply changes to a cluster.&lt;/p>
&lt;p>To get started with it in just minutes, head over to the &lt;a href="/v1.12/docs/ops/diagnostic-tools/istioctl-analyze/">documentation&lt;/a>.&lt;/p>
&lt;h2 id="designed-to-be-approachable-for-novice-users">Designed to be approachable for novice users&lt;/h2>
&lt;p>One of the key design goals that we followed for this feature is to make it extremely approachable.
This is achieved by making the command useful without having to pass any required complex parameters.&lt;/p>
&lt;p>In practice, here are some of the scenarios that it goes after:&lt;/p>
&lt;ul>
&lt;li>&lt;em>&amp;ldquo;There is some problem with my cluster, but I have no idea where to start&amp;rdquo;&lt;/em>&lt;/li>
&lt;li>&lt;em>&amp;ldquo;Things are generally working, but I&amp;rsquo;m wondering if there is anything I could improve&amp;rdquo;&lt;/em>&lt;/li>
&lt;/ul>
&lt;p>In that sense, it is very different from some of the more advanced diagnostic tools, which go
after scenarios along the lines of (taking &lt;code>istioctl proxy-config&lt;/code> as an example):&lt;/p>
&lt;ul>
&lt;li>&lt;em>&amp;ldquo;Show me the Envoy configuration for this specific pod so I can see if anything looks wrong&amp;rdquo;&lt;/em>&lt;/li>
&lt;/ul>
&lt;p>This can be very useful for advanced debugging, but it requires a lot of expertize before you
can figure out that you need to run this specific command, and which pod to run it on.&lt;/p>
&lt;p>So really, the one-line pitch for &lt;code>analyze&lt;/code> is: just run it! It&amp;rsquo;s completely safe, it takes no thinking,
it might help you, and at worst, you&amp;rsquo;ll have wasted a minute!&lt;/p>
&lt;h2 id="improving-this-tool-over-time">Improving this tool over time&lt;/h2>
&lt;p>In Istio 1.4, &lt;code>analyze&lt;/code> comes with a nice set of analyzers that can detect a number of common issues.
But this is just the beginning, and we are planning to keep growing and fine tuning the analyzers with
each release.&lt;/p>
&lt;p>In fact, we would welcome suggestions from Istio users. Specifically, if you encounter a situation
where you think an issue could be detected via configuration analysis, but is not currently flagged
by &lt;code>analyze&lt;/code>, please do let us know. The best way to do this is to &lt;a href="https://github.com/istio/istio/issues">open an issue on GitHub&lt;/a>.&lt;/p></description><pubDate>Thu, 14 Nov 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/introducing-istioctl-analyze/</link><author>David Ebbo (Google)</author><guid isPermaLink="true">/v1.12/blog/2019/introducing-istioctl-analyze/</guid><category>debugging</category><category>istioctl</category><category>configuration</category></item><item><title>DNS Certificate Management</title><description>&lt;p>By default, Citadel manages the DNS certificates of the Istio control plane.
Citadel is a large component that maintains its own private signing key, and acts as a Certificate Authority (CA).&lt;/p>
&lt;p>New in Istio 1.4, we introduce a feature to securely provision and manage DNS certificates
signed by the Kubernetes CA, which has the following advantages.&lt;/p>
&lt;ul>
&lt;li>&lt;p>Lighter weight DNS certificate management with no dependency on Citadel.&lt;/p>&lt;/li>
&lt;li>&lt;p>Unlike Citadel, this feature doesn&amp;rsquo;t maintain a private signing key, which enhances security.&lt;/p>&lt;/li>
&lt;li>&lt;p>Simplified root certificate distribution to TLS clients.
Clients no longer need to wait for Citadel to generate and distribute its CA certificate.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>The following diagram shows the architecture of provisioning and managing DNS certificates in Istio.
Chiron is the component provisioning and managing DNS certificates in Istio.&lt;/p>
&lt;figure style="width:50%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:82.13367609254499%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/dns-cert/architecture.png" title="The architecture of provisioning and managing DNS certificates in Istio">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/dns-cert/architecture.png" alt="The architecture of provisioning and managing DNS certificates in Istio" />
&lt;/a>
&lt;/div>
&lt;figcaption>The architecture of provisioning and managing DNS certificates in Istio&lt;/figcaption>
&lt;/figure>
&lt;p>To try this new feature, refer to the &lt;a href="/v1.12/docs/tasks/security/cert-management/dns-cert">DNS certificate management task&lt;/a>.&lt;/p></description><pubDate>Thu, 14 Nov 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/dns-cert/</link><author>Lei Tang (Google)</author><guid isPermaLink="true">/v1.12/blog/2019/dns-cert/</guid><category>security</category><category>kubernetes</category><category>certificates</category><category>DNS</category></item><item><title>Announcing Istio client-go</title><description>
&lt;p>We are pleased to announce the initial release of the Istio
&lt;a href="https://github.com/istio/client-go">client go&lt;/a> repository which enables developers
to gain programmatic access to Istio APIs in a Kubernetes environment. The
generated Kubernetes informers and client set in this repository makes it easy
for developers to create controllers and perform Create, Read, Update and Delete
(CRUD) operations for all Istio Custom Resource Definitions (CRDs).&lt;/p>
&lt;p>This was a highly requested functionality by many Istio users, as is evident
from the feature requests on the clients generated by &lt;a href="https://github.com/aspenmesh/istio-client-go">Aspen Mesh&lt;/a>
and the &lt;a href="https://github.com/knative/pkg">Knative project&lt;/a>.
If you&amp;rsquo;re currently using one of the above mentioned clients, you can easily
switch to using &lt;a href="https://github.com/istio/client-go">Istio client go&lt;/a> like
this:&lt;/p>
&lt;pre>&lt;code class='language-go' data-expandlinks='true' data-repo='istio' >import (
...
- versionedclient &amp;#34;github.com/aspenmesh/istio-client-go/pkg/client/clientset/versioned&amp;#34;
+ versionedclient &amp;#34;istio.io/client-go/pkg/clientset/versioned&amp;#34;
)
&lt;/code>&lt;/pre>
&lt;p>As the generated client sets are functionally equivalent, switching the imported
client libraries should be sufficient in order to consume the newly
generated library.&lt;/p>
&lt;h2 id="how-to-use-client-go">How to use client-go&lt;/h2>
&lt;p>The Istio &lt;a href="https://github.com/istio/client-go">client go&lt;/a> repository follows the
same branching strategy as the &lt;a href="https://github.com/istio/api">Istio API&lt;/a>
repository, as the client repository depends on the API definitions. If you want
to use a stable client set, you can use the release branches or tagged versions
in the &lt;a href="https://github.com/istio/client-go">client go&lt;/a> repository.
Using the client set is very similar to using the &lt;a href="https://github.com/kubernetes/client-go">Kubernetes client
go&lt;/a>, here&amp;rsquo;s a quick example of using
the client to list all &lt;a href="/v1.12/docs/reference/config/networking/virtual-service">Istio
virtual services&lt;/a>
in the passed namespace:&lt;/p>
&lt;pre>&lt;code class='language-go' data-expandlinks='true' data-repo='istio' >package main
import (
&amp;#34;log&amp;#34;
&amp;#34;os&amp;#34;
metav1 &amp;#34;k8s.io/apimachinery/pkg/apis/meta/v1&amp;#34;
&amp;#34;k8s.io/client-go/tools/clientcmd&amp;#34;
versionedclient &amp;#34;istio.io/client-go/pkg/clientset/versioned&amp;#34;
)
func main() {
kubeconfig := os.Getenv(&amp;#34;KUBECONFIG&amp;#34;)
namespace := os.Getenv(&amp;#34;NAMESPACE&amp;#34;)
if len(kubeconfig) == 0 || len(namespace) == 0 {
log.Fatalf(&amp;#34;Environment variables KUBECONFIG and NAMESPACE need to be set&amp;#34;)
}
restConfig, err := clientcmd.BuildConfigFromFlags(&amp;#34;&amp;#34;, kubeconfig)
if err != nil {
log.Fatalf(&amp;#34;Failed to create k8s rest client: %s&amp;#34;, err)
}
ic, err := versionedclient.NewForConfig(restConfig)
if err != nil {
log.Fatalf(&amp;#34;Failed to create istio client: %s&amp;#34;, err)
}
// Print all VirtualServices
vsList, err := ic.NetworkingV1alpha3().VirtualServices(namespace).List(metav1.ListOptions{})
if err != nil {
log.Fatalf(&amp;#34;Failed to get VirtualService in %s namespace: %s&amp;#34;, namespace, err)
}
for i := range vsList.Items {
vs := vsList.Items[i]
log.Printf(&amp;#34;Index: %d VirtualService Hosts: %+v\n&amp;#34;, i, vs.Spec.GetHosts())
}
}
&lt;/code>&lt;/pre>
&lt;p>You can find a more in-depth example &lt;a href="https://github.com/istio/client-go/blob/release-1.12/cmd/example/client.go">here&lt;/a>.&lt;/p>
&lt;h2 id="useful-tools-created-for-generating-istio-client-go">Useful tools created for generating Istio client-go&lt;/h2>
&lt;p>If you&amp;rsquo;re wondering why it took so long or why was it difficult to generate
this client set, this section is for you. In Istio, we use
&lt;a href="https://developers.google.com/protocol-buffers">protobuf&lt;/a> specifications to
write APIs which are then converted to Go definitions
using the protobuf tool chain. There are three major challenges which you might
face if you&amp;rsquo;re trying to generate Kubernetes client set from a protobuf-generated API:&lt;/p>
&lt;ul>
&lt;li>&lt;p>&lt;strong>Creating Kubernetes Wrapper Types&lt;/strong> - Kubernetes &lt;a href="https://github.com/kubernetes/code-generator/tree/master/cmd/client-gen">client generation&lt;/a>
library only works for Go objects which follow the Kubernetes object
specification for e.g. &lt;a href="https://github.com/istio/client-go/blob/release-1.12/pkg/apis/authentication/v1alpha1/types.gen.go">Authentication Policy Kubernetes Wrappers&lt;/a>.
This means for every API which needs programmatic access, you need to create
these wrappers. Additionally, there is a fair amount of boilerplate needed for
every &lt;code>CRD&lt;/code> group, version and kind that needs client code generation.
To automate this process, we created a &lt;a href="https://github.com/istio/tools/tree/master/cmd/kubetype-gen">Kubernetes type
generator&lt;/a> tool
which can automatically create the Kubernetes types based on annotations.
The annotations parsed by this tool and the various available options
are explained in the &lt;a href="https://github.com/istio/tools/blob/master/cmd/kubetype-gen/README.md">README&lt;/a>.
Note that if you&amp;rsquo;re using protobuf tools to generate Go types, you would need to
add these annotations as comments in the proto files, so that the comments are
present in the generated Go files which are then used by this tool.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Generating deep copy methods&lt;/strong> - In Kubernetes client machinery, if you want to
mutate any object returned from the client set, you are required to make a copy
of the object to prevent modifying the object in-place in the cache store. The
canonical way to do this is to create a &lt;code>deepcopy&lt;/code> method on all nested types.
We created a tool &lt;a href="https://github.com/istio/tools/tree/master/cmd/protoc-gen-deepcopy">protoc deep copy
generator&lt;/a>
which is a &lt;code>protoc&lt;/code> plugin and can automatically create &lt;code>deepcopy&lt;/code> method
based on annotations using the Proto library utility &lt;a href="https://godoc.org/github.com/golang/protobuf/proto#Clone">Proto
Clone&lt;/a>. Here&amp;rsquo;s an
&lt;a href="https://github.com/istio/api/blob/release-1.12/authentication/v1alpha1/policy_deepcopy.gen.go">example&lt;/a>
of the generated &lt;code>deepcopy&lt;/code> method.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Marshaling and Unmarshaling types to/from JSON&lt;/strong> - For the types generated
from proto definitions, it is often problematic to use the default Go JSON
encoder/decoder as there are various fields like protobuf&amp;rsquo;s &lt;code>oneof&lt;/code> which requires
special handling. Additionally, any Proto fields with underscores in their
name might serialize/deserialize to different field names depending on the
encoder/decoder as the Go struct tag are &lt;a href="https://github.com/istio/istio/issues/17600">generated
differently&lt;/a>.
It is always recommended to use protobuf primitives for
serializing/deserializing to JSON instead of relying on default Go
library. We created a tool &lt;a href="https://github.com/istio/tools/tree/master/cmd/protoc-gen-jsonshim">protoc JSON
shim&lt;/a> which
is a &lt;code>protoc&lt;/code> plugin and can automatically create Marshalers/Unmarshalers for
all Go type generated from Proto definitions. Here&amp;rsquo;s an
&lt;a href="https://github.com/istio/api/blob/release-1.12/authentication/v1alpha1/policy_json.gen.go">example&lt;/a>
of the code generated by this tool.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>I&amp;rsquo;m hoping that the newly released client library enables users to create more
integrations and controllers for the Istio APIs, and the tools mentioned above
can be used by developers to generate Kubernetes client set from Proto APIs.&lt;/p></description><pubDate>Thu, 14 Nov 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/announcing-istio-client-go/</link><author>Neeraj Poddar (Aspen Mesh)</author><guid isPermaLink="true">/v1.12/blog/2019/announcing-istio-client-go/</guid><category>client-go</category><category>tools</category><category>crd</category></item><item><title>Istio as a Proxy for External Services</title><description>
&lt;p>The &lt;a href="/v1.12/docs/tasks/traffic-management/ingress/ingress-control/">Control Ingress Traffic&lt;/a> and the
&lt;a href="/v1.12/docs/tasks/traffic-management/ingress/ingress-sni-passthrough/">Ingress Gateway without TLS Termination&lt;/a> tasks describe
how to configure an ingress gateway to expose services inside the mesh to external traffic. The services can be HTTP or
HTTPS. In the case of HTTPS, the gateway passes the traffic through, without terminating TLS.&lt;/p>
&lt;p>This blog post describes how to use the same ingress gateway mechanism of Istio to enable access to external services and
not to applications inside the mesh. This way Istio as a whole can serve just as a proxy server, with the added value of
observability, traffic management and policy enforcement.&lt;/p>
&lt;p>The blog post shows configuring access to an HTTP and an HTTPS external service, namely &lt;code>httpbin.org&lt;/code> and
&lt;code>edition.cnn.com&lt;/code>.&lt;/p>
&lt;h2 id="configure-an-ingress-gateway">Configure an ingress gateway&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Define an ingress gateway with a &lt;code>servers:&lt;/code> section configuring the &lt;code>80&lt;/code> and &lt;code>443&lt;/code> ports.
Ensure &lt;code>mode:&lt;/code> is set to &lt;code>PASSTHROUGH&lt;/code> for &lt;code>tls:&lt;/code> in the port &lt;code>443&lt;/code>, which instructs the gateway to pass the
ingress traffic AS IS, without terminating TLS.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: proxy
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- httpbin.org
- port:
number: 443
name: tls
protocol: TLS
tls:
mode: PASSTHROUGH
hosts:
- edition.cnn.com
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create service entries for the &lt;code>httpbin.org&lt;/code> and &lt;code>edition.cnn.com&lt;/code> services to make them accessible from the ingress
gateway:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin-ext
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http
protocol: HTTP
resolution: DNS
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: cnn
spec:
hosts:
- edition.cnn.com
ports:
- number: 443
name: tls
protocol: TLS
resolution: DNS
location: MESH_EXTERNAL
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a service entry and configure a destination rule for the &lt;code>localhost&lt;/code> service.
You need this service entry in the next step as a destination for traffic to the external services from
applications inside the mesh to block traffic from inside the mesh. In this example you use Istio as a proxy between
external applications and external services.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: localhost
spec:
hosts:
- localhost.local
location: MESH_EXTERNAL
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: tls
protocol: TLS
resolution: STATIC
endpoints:
- address: 127.0.0.1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: localhost
spec:
host: localhost.local
trafficPolicy:
tls:
mode: DISABLE
sni: localhost.local
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a virtual service for each external service to configure routing to it. Both virtual services include the
&lt;code>proxy&lt;/code> gateway in the &lt;code>gateways:&lt;/code> section and in the &lt;code>match:&lt;/code> section for HTTP and HTTPS traffic accordingly.&lt;/p>
&lt;p>Notice the &lt;code>route:&lt;/code> section for the &lt;code>mesh&lt;/code> gateway, the gateway that represents the applications inside
the mesh. The &lt;code>route:&lt;/code> for the &lt;code>mesh&lt;/code> gateway shows how the traffic is directed to the &lt;code>localhost.local&lt;/code> service,
effectively blocking the traffic.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.org
gateways:
- proxy
- mesh
http:
- match:
- gateways:
- proxy
port: 80
uri:
prefix: /status
route:
- destination:
host: httpbin.org
port:
number: 80
- match:
- gateways:
- mesh
port: 80
route:
- destination:
host: localhost.local
port:
number: 80
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: cnn
spec:
hosts:
- edition.cnn.com
gateways:
- proxy
- mesh
tls:
- match:
- gateways:
- proxy
port: 443
sni_hosts:
- edition.cnn.com
route:
- destination:
host: edition.cnn.com
port:
number: 443
- match:
- gateways:
- mesh
port: 443
sni_hosts:
- edition.cnn.com
route:
- destination:
host: localhost.local
port:
number: 443
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>&lt;a href="/v1.12/docs/tasks/observability/logs/access-log/#enable-envoy-s-access-logging">Enable Envoy&amp;rsquo;s access logging&lt;/a>.&lt;/p>&lt;/li>
&lt;li>&lt;p>Follow the instructions in
&lt;a href="/v1.12/docs/tasks/traffic-management/ingress/ingress-control/#determining-the-ingress-ip-and-ports">Determining the ingress IP and ports&lt;/a>
to define the &lt;code>SECURE_INGRESS_PORT&lt;/code> and &lt;code>INGRESS_HOST&lt;/code> environment variables.&lt;/p>&lt;/li>
&lt;li>&lt;p>Access the &lt;code>httbin.org&lt;/code> service through your ingress IP and port which you stored in the
&lt;code>$INGRESS_HOST&lt;/code> and &lt;code>$INGRESS_PORT&lt;/code> environment variables, respectively, during the previous step.
Access the &lt;code>/status/418&lt;/code> path of the &lt;code>httpbin.org&lt;/code> service that returns the HTTP status
&lt;a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/418">418 I&amp;rsquo;m a teapot&lt;/a>.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl $INGRESS_HOST:$INGRESS_PORT/status/418 -Hhost:httpbin.org
-=[ teapot ]=-
_...._
.&amp;#39; _ _ `.
| .&amp;#34;` ^ `&amp;#34;. _,
\_;`&amp;#34;---&amp;#34;`|//
| ;/
\_ _/
`&amp;#34;&amp;#34;&amp;#34;`
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>If the Istio ingress gateway is deployed in the &lt;code>istio-system&lt;/code> namespace, print the gateway&amp;rsquo;s log with the following command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl logs -l istio=ingressgateway -c istio-proxy -n istio-system | grep &amp;#39;httpbin.org&amp;#39;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Search the log for an entry similar to:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >[2019-01-31T14:40:18.645Z] &amp;#34;GET /status/418 HTTP/1.1&amp;#34; 418 - 0 135 187 186 &amp;#34;10.127.220.75&amp;#34; &amp;#34;curl/7.54.0&amp;#34; &amp;#34;28255618-6ca5-9d91-9634-c562694a3625&amp;#34; &amp;#34;httpbin.org&amp;#34; &amp;#34;34.232.181.106:80&amp;#34; outbound|80||httpbin.org - 172.30.230.33:80 10.127.220.75:52077 -
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Access the &lt;code>edition.cnn.com&lt;/code> service through your ingress gateway:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl -s --resolve edition.cnn.com:$SECURE_INGRESS_PORT:$INGRESS_HOST https://edition.cnn.com:$SECURE_INGRESS_PORT | grep -o &amp;#34;&amp;lt;title&amp;gt;.*&amp;lt;/title&amp;gt;&amp;#34;
&amp;lt;title&amp;gt;CNN International - Breaking News, US News, World News and Video&amp;lt;/title&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>If the Istio ingress gateway is deployed in the &lt;code>istio-system&lt;/code> namespace, print the gateway&amp;rsquo;s log with the following command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl logs -l istio=ingressgateway -c istio-proxy -n istio-system | grep &amp;#39;edition.cnn.com&amp;#39;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Search the log for an entry similar to:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >[2019-01-31T13:40:11.076Z] &amp;#34;- - -&amp;#34; 0 - 589 17798 1644 - &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;172.217.31.132:443&amp;#34; outbound|443||edition.cnn.com 172.30.230.33:54508 172.30.230.33:443 10.127.220.75:49467 edition.cnn.com
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h2 id="cleanup">Cleanup&lt;/h2>
&lt;p>Remove the gateway, the virtual services and the service entries:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete gateway proxy
$ kubectl delete virtualservice cnn httpbin
$ kubectl delete serviceentry cnn httpbin-ext localhost
$ kubectl delete destinationrule localhost
&lt;/code>&lt;/pre></description><pubDate>Tue, 15 Oct 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/proxy/</link><author>Vadim Eisenberg (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/proxy/</guid><category>traffic-management</category><category>ingress</category><category>https</category><category>http</category></item><item><title>Multi-Mesh Deployments for Isolation and Boundary Protection</title><description>
&lt;p>Various compliance standards require protection of sensitive data environments. Some of the important standards and the
types of sensitive data they protect appear in the following table:&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Standard&lt;/th>
&lt;th>Sensitive data&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>&lt;a href="https://www.pcisecuritystandards.org/pci_security">PCI DSS&lt;/a>&lt;/td>
&lt;td>payment card data&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;a href="https://www.fedramp.gov">FedRAMP&lt;/a>&lt;/td>
&lt;td>federal information, data and metadata&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;a href="http://www.gpo.gov/fdsys/search/pagedetails.action?granuleId=CRPT-104hrpt736&amp;amp;packageId=CRPT-104hrpt736">HIPAA&lt;/a>&lt;/td>
&lt;td>personal health data&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;a href="https://gdpr-info.eu">GDPR&lt;/a>&lt;/td>
&lt;td>personal data&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>&lt;a href="https://www.pcisecuritystandards.org/pci_security">PCI DSS&lt;/a>, for example, recommends putting cardholder data
environment on a network, separate from the rest of the system. It also requires using a &lt;a href="https://en.wikipedia.org/wiki/DMZ_(computing)">DMZ&lt;/a>,
and setting firewalls between the public Internet and the DMZ, and between the DMZ and the internal network.&lt;/p>
&lt;p>Isolation of sensitive data environments from other information systems can reduce the scope of the compliance checks
and improve the security of the sensitive data. Reducing the scope reduces the risks of failing a compliance check and
reduces the costs of compliance since there are less components to check and secure, according to compliance
requirements.&lt;/p>
&lt;p>You can achieve isolation of sensitive data by separating the parts of the application that process that data
into a separate service mesh, preferably on a separate network, and then connect the meshes with different
compliance requirements together in a &lt;span class="term" data-title="Multi-Mesh" data-body="&amp;lt;p&amp;gt;Multi-mesh is a deployment model that consists of two or more &amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-mesh&amp;#34;&amp;gt;service meshes&amp;lt;/a&amp;gt;.
Each mesh has independent administration for naming and identities but you can
expose services between meshes through &amp;lt;a href=&amp;#34;/docs/reference/glossary/#mesh-federation&amp;#34;&amp;gt;mesh federation&amp;lt;/a&amp;gt;.
The resulting deployment is a multi-mesh deployment.&amp;lt;/p&amp;gt;
">multi-mesh&lt;/span> deployment.
The process of connecting inter-mesh
applications is called &lt;span class="term" data-title="Mesh Federation" data-body="&amp;lt;p&amp;gt;Mesh federation is the act of exposing services between meshes and enabling
communication across mesh boundaries. Each mesh may expose a subset of its
services to enable one or more other meshes to consume the exposed services. You
can use mesh federation to enable communication between meshes in a
&amp;lt;a href=&amp;#34;/docs/ops/deployment/deployment-models/#multiple-meshes&amp;#34;&amp;gt;multi-mesh deployment&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">mesh federation&lt;/span>.&lt;/p>
&lt;p>Note that using mesh federation to create a multi-mesh deployment is very different than creating a
&lt;span class="term" data-title="Multicluster" data-body="&amp;lt;p&amp;gt;Multicluster is a deployment model that consists of a
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-mesh&amp;#34;&amp;gt;mesh&amp;lt;/a&amp;gt; with multiple
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#cluster&amp;#34;&amp;gt;clusters&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">multicluster&lt;/span> deployment, which defines a single service mesh composed from services spanning more than one cluster. Unlike multi-mesh, a multicluster deployment is not suitable for
applications that require isolation and boundary protection.&lt;/p>
&lt;p>In this blog post I describe the requirements for isolation and boundary protection, and outline the principles of
multi-mesh deployments. Finally, I touch on the current state of mesh-federation support and automation work under way for
Istio.&lt;/p>
&lt;h2 id="isolation-and-boundary-protection">Isolation and boundary protection&lt;/h2>
&lt;p>Isolation and boundary protection mechanisms are explained in the
&lt;a href="http://dx.doi.org/10.6028/NIST.SP.800-53r4">NIST Special Publication 800-53, Revision 4, Security and Privacy Controls for Federal Information Systems and Organizations&lt;/a>,
&lt;em>Appendix F, Security Control Catalog, SC-7 Boundary Protection&lt;/em>.&lt;/p>
&lt;p>In particular, the &lt;em>Boundary protection, isolation of information system components&lt;/em> control enhancement:&lt;/p>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Organizations can isolate information system components performing different missions and/or business functions.
Such isolation limits unauthorized information flows among system components and also provides the opportunity to deploy
greater levels of protection for selected components. Separating system components with boundary protection mechanisms
provides the capability for increased protection of individual components and to more effectively control information
flows between those components. This type of enhanced protection limits the potential harm from cyber attacks and
errors. The degree of separation provided varies depending upon the mechanisms chosen. Boundary protection mechanisms
include, for example, routers, gateways, and firewalls separating system components into physically separate networks or
subnetworks, cross-domain devices separating subnetworks, virtualization techniques, and encrypting information flows
among system components using distinct encryption keys.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Various compliance standards recommend isolating environments that process sensitive data from the rest of the
organization.
The &lt;a href="https://www.pcisecuritystandards.org/pci_security/">Payment Card Industry (PCI) Data Security Standard&lt;/a>
recommends implementing network isolation for &lt;em>cardholder data&lt;/em> environment and requires isolating this environment from
the &lt;a href="https://en.wikipedia.org/wiki/DMZ_(computing)">DMZ&lt;/a>.
&lt;a href="https://www.fedramp.gov/assets/resources/documents/CSP_A_FedRAMP_Authorization_Boundary_Guidance.pdf">FedRAMP Authorization Boundary Guidance&lt;/a>
describes &lt;em>authorization boundary&lt;/em> for federal information and data, while
&lt;a href="https://doi.org/10.6028/NIST.SP.800-37r2">NIST Special Publication 800-37, Revision 2, Risk Management Framework for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy&lt;/a>
recommends protecting of such a boundary in &lt;em>Appendix G, Authorization Boundary Considerations&lt;/em>:&lt;/p>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Dividing a system into subsystems (i.e., divide and conquer) facilitates a targeted application of controls to achieve
adequate security, protection of individual privacy, and a cost-effective risk management process. Dividing complex
systems into subsystems also supports the important security concepts of domain separation and network segmentation,
which can be significant when dealing with high value assets. When systems are divided into subsystems, organizations
may choose to develop individual subsystem security and privacy plans or address the system and subsystems in the same
security and privacy plans.
Information security and privacy architectures play a key part in the process of dividing complex systems into
subsystems. This includes monitoring and controlling communications at internal boundaries among subsystems and
selecting, allocating, and implementing controls that meet or exceed the security and privacy requirements of the
constituent subsystems.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Boundary protection, in particular, means:&lt;/p>
&lt;ul>
&lt;li>put an access control mechanism at the boundary (firewall, gateway, etc.)&lt;/li>
&lt;li>monitor the incoming/outgoing traffic at the boundary&lt;/li>
&lt;li>all the access control mechanisms must be &lt;em>deny-all&lt;/em> by default&lt;/li>
&lt;li>do not expose private IP addresses from the boundary&lt;/li>
&lt;li>do not let components from outside the boundary to impact security inside the boundary&lt;/li>
&lt;/ul>
&lt;p>Multi-mesh deployments facilitate division of a system into subsystems with different
security and compliance requirements, and facilitate the boundary protection.
You put each subsystem into a separate service mesh, preferably on a separate network.
You connect the Istio meshes using gateways. The gateways monitor and control cross-mesh traffic at the boundary of
each mesh.&lt;/p>
&lt;h2 id="features-of-multi-mesh-deployments">Features of multi-mesh deployments&lt;/h2>
&lt;ul>
&lt;li>&lt;strong>non-uniform naming&lt;/strong>. The &lt;code>withdraw&lt;/code> service in the &lt;code>accounts&lt;/code> namespace in one mesh might have
different functionality and API than the &lt;code>withdraw&lt;/code> services in the &lt;code>accounts&lt;/code> namespace in other meshes.
Such situation could happen in an organization where there is no uniform policy on naming of namespaces and services, or
when the meshes belong to different organizations.&lt;/li>
&lt;li>&lt;strong>expose-nothing by default&lt;/strong>. None of the services in a mesh are exposed by default, the mesh owners must
explicitly specify which services are exposed.&lt;/li>
&lt;li>&lt;strong>boundary protection&lt;/strong>. The access control of the traffic must be enforced at the ingress gateway, which stops
forbidden traffic from entering the mesh. This requirement implements
&lt;a href="https://en.wikipedia.org/wiki/Defense_in_depth_(computing)">Defense-in-depth principle&lt;/a> and is part of some compliance
standards, such as the
&lt;a href="https://www.pcisecuritystandards.org/pci_security/">Payment Card Industry (PCI) Data Security Standard&lt;/a>.&lt;/li>
&lt;li>&lt;strong>common trust may not exist&lt;/strong>. The Istio sidecars in one mesh may not trust the Citadel certificates in other
meshes, due to some security requirement or due to the fact that the mesh owners did not initially plan to federate
the meshes.&lt;/li>
&lt;/ul>
&lt;p>While &lt;strong>expose-nothing by default&lt;/strong> and &lt;strong>boundary protection&lt;/strong> are required to facilitate compliance and improve
security, &lt;strong>non-uniform naming&lt;/strong> and &lt;strong>common trust may not exist&lt;/strong> are required when connecting
meshes of different organizations, or of an organization that cannot enforce uniform naming or cannot or may not
establish common trust between the meshes.&lt;/p>
&lt;p>An optional feature that you may want to use is &lt;strong>service location transparency&lt;/strong>: consuming services send requests
to the exposed services in remote meshes using local service names. The consuming services are oblivious to the fact
that some of the destinations are in remote meshes and some are local services. The access is uniform, using the local
service names, for example, in Kubernetes, &lt;code>reviews.default.svc.cluster.local&lt;/code>.
&lt;strong>Service location transparency&lt;/strong> is useful in the cases when you want to be able to change the location of the
consumed services, for example when some service is migrated from private cloud to public cloud, without changing the
code of your applications.&lt;/p>
&lt;h2 id="the-current-mesh-federation-work">The current mesh-federation work&lt;/h2>
&lt;p>While you can perform mesh federation using standard Istio configurations already today,
it requires writing a lot of boilerplate YAML files and is error-prone. There is an effort under way to automate
the mesh federation process. In the meantime, you can look at these
&lt;a href="https://github.com/istio-ecosystem/multi-mesh-examples">multi-mesh deployment examples&lt;/a>
to get an idea of what a generated federation might include.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>In this blog post I described the requirements for isolation and boundary protection of sensitive data environments by
using Istio multi-mesh deployments. I outlined the principles of Istio
multi-mesh deployments and reported the current work on
mesh federation in Istio.&lt;/p>
&lt;p>I will be happy to hear your opinion about &lt;span class="term" data-title="Multi-Mesh" data-body="&amp;lt;p&amp;gt;Multi-mesh is a deployment model that consists of two or more &amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-mesh&amp;#34;&amp;gt;service meshes&amp;lt;/a&amp;gt;.
Each mesh has independent administration for naming and identities but you can
expose services between meshes through &amp;lt;a href=&amp;#34;/docs/reference/glossary/#mesh-federation&amp;#34;&amp;gt;mesh federation&amp;lt;/a&amp;gt;.
The resulting deployment is a multi-mesh deployment.&amp;lt;/p&amp;gt;
">multi-mesh&lt;/span> and
&lt;span class="term" data-title="Multicluster" data-body="&amp;lt;p&amp;gt;Multicluster is a deployment model that consists of a
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-mesh&amp;#34;&amp;gt;mesh&amp;lt;/a&amp;gt; with multiple
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#cluster&amp;#34;&amp;gt;clusters&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">multicluster&lt;/span> at &lt;a href="https://discuss.istio.io">discuss.istio.io&lt;/a>.&lt;/p></description><pubDate>Wed, 02 Oct 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/isolated-clusters/</link><author>Vadim Eisenberg (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/isolated-clusters/</guid><category>traffic-management</category><category>multicluster</category><category>security</category><category>gateway</category><category>tls</category></item><item><title>Monitoring Blocked and Passthrough External Service Traffic</title><description>
&lt;p>Understanding, controlling and securing your external service access is one
of the key benefits that you get from a service mesh like Istio. From a security
and operations point of view, it is critical to monitor what external service traffic
is getting blocked as they might surface possible misconfigurations or a
security vulnerability if an application is attempting to communicate with a
service that it should not be allowed to. Similarly, if you currently have a
policy of allowing any external service access, it is beneficial to monitor
the traffic so you can incrementally add explicit Istio configuration to allow
access and better secure your cluster. In either case, having visibility into this
traffic via telemetry is quite helpful as it enables you to create alerts and
dashboards, and better reason about your security posture. This was a highly
requested feature by production users of Istio and we are excited that the
support for this was added in release 1.3.&lt;/p>
&lt;p>To implement this, the Istio &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/metrics">default
metrics&lt;/a> are augmented with
explicit labels to capture blocked and passthrough external service traffic.
This blog will cover how you can use these augmented metrics to monitor all
external service traffic.&lt;/p>
&lt;p>The Istio control plane configures the sidecar proxy with
predefined clusters called BlackHoleCluster and Passthrough which block or
allow all traffic respectively. To understand these clusters, let&amp;rsquo;s start with
what external and internal services mean in the context of Istio service mesh.&lt;/p>
&lt;h2 id="external-and-internal-services">External and internal services&lt;/h2>
&lt;p>Internal services are defined as services which are part of your platform
and are considered to be in the mesh. For internal services, Istio control
plane provides all the required configuration to the sidecars by default.
For example, in Kubernetes clusters, Istio configures the sidecars for all
Kubernetes services to preserve the default Kubernetes behavior of all
services being able to communicate with other.&lt;/p>
&lt;p>External services are services which are not part of your platform i.e. services
which are outside of the mesh. For external services, Istio provides two
options, first to block all external service access (enabled by setting
&lt;code>global.outboundTrafficPolicy.mode&lt;/code> to &lt;code>REGISTRY_ONLY&lt;/code>) and
second to allow all access to external service (enabled by setting
&lt;code>global.outboundTrafficPolicy.mode&lt;/code> to &lt;code>ALLOW_ANY&lt;/code>). The default option for this
setting (as of Istio 1.3) is to allow all external service access. This
option can be configured via &lt;a href="/v1.12/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig-OutboundTrafficPolicy-Mode">mesh configuration&lt;/a>.&lt;/p>
&lt;p>This is where the BlackHole and Passthrough clusters are used.&lt;/p>
&lt;h2 id="what-are-blackhole-and-passthrough-clusters">What are BlackHole and Passthrough clusters?&lt;/h2>
&lt;ul>
&lt;li>&lt;strong>BlackHoleCluster&lt;/strong> - The BlackHoleCluster is a virtual cluster created
in the Envoy configuration when &lt;code>global.outboundTrafficPolicy.mode&lt;/code> is set to
&lt;code>REGISTRY_ONLY&lt;/code>. In this mode, all traffic to external service is blocked unless
&lt;a href="/v1.12/docs/reference/config/networking/service-entry">service entries&lt;/a>
are explicitly added for each service. To implement this, the default virtual
outbound listener at &lt;code>0.0.0.0:15001&lt;/code> which uses
&lt;a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/service_discovery#original-destination">original destination&lt;/a>
is setup as a TCP Proxy with the BlackHoleCluster as the static cluster.
The configuration for the BlackHoleCluster looks like this:&lt;/li>
&lt;/ul>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;name&amp;#34;: &amp;#34;BlackHoleCluster&amp;#34;,
&amp;#34;type&amp;#34;: &amp;#34;STATIC&amp;#34;,
&amp;#34;connectTimeout&amp;#34;: &amp;#34;10s&amp;#34;
}
&lt;/code>&lt;/pre>
&lt;p>As you can see, this cluster is static with no endpoints so all the traffic
will be dropped. Additionally, Istio creates unique listeners for every
port/protocol combination of platform services which gets hit instead of the
virtual listener if the request is made to an external service on the same port.
In that case, the route configuration of every virtual route in Envoy is augmented to
add the BlackHoleCluster like this:&lt;/p>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;name&amp;#34;: &amp;#34;block_all&amp;#34;,
&amp;#34;domains&amp;#34;: [
&amp;#34;*&amp;#34;
],
&amp;#34;routes&amp;#34;: [
{
&amp;#34;match&amp;#34;: {
&amp;#34;prefix&amp;#34;: &amp;#34;/&amp;#34;
},
&amp;#34;directResponse&amp;#34;: {
&amp;#34;status&amp;#34;: 502
}
}
]
}
&lt;/code>&lt;/pre>
&lt;p>The route is setup as &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route_components.proto#envoy-api-field-route-route-direct-response">direct response&lt;/a>
with &lt;code>502&lt;/code> response code which means if no other routes match the Envoy proxy
will directly return a &lt;code>502&lt;/code> HTTP status code.&lt;/p>
&lt;ul>
&lt;li>&lt;strong>PassthroughCluster&lt;/strong> - The PassthroughCluster is a virtual cluster created
in the Envoy configuration when &lt;code>global.outboundTrafficPolicy.mode&lt;/code> is set to
&lt;code>ALLOW_ANY&lt;/code>. In this mode, all traffic to any external service external is allowed.
To implement this, the default virtual outbound listener at &lt;code>0.0.0.0:15001&lt;/code>
which uses &lt;code>SO_ORIGINAL_DST&lt;/code>, is setup as a TCP Proxy with the PassthroughCluster
as the static cluster.
The configuration for the PassthroughCluster looks like this:&lt;/li>
&lt;/ul>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;name&amp;#34;: &amp;#34;PassthroughCluster&amp;#34;,
&amp;#34;type&amp;#34;: &amp;#34;ORIGINAL_DST&amp;#34;,
&amp;#34;connectTimeout&amp;#34;: &amp;#34;10s&amp;#34;,
&amp;#34;lbPolicy&amp;#34;: &amp;#34;ORIGINAL_DST_LB&amp;#34;,
&amp;#34;circuitBreakers&amp;#34;: {
&amp;#34;thresholds&amp;#34;: [
{
&amp;#34;maxConnections&amp;#34;: 102400,
&amp;#34;maxRetries&amp;#34;: 1024
}
]
}
}
&lt;/code>&lt;/pre>
&lt;p>This cluster uses the &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/service_discovery#original-destination">original destination load balancing&lt;/a>
policy which configures Envoy to send the traffic to the
original destination i.e. passthrough.&lt;/p>
&lt;p>Similar to the BlackHoleCluster, for every port/protocol based listener the
virtual route configuration is augmented to add the PassthroughCluster as the
default route:&lt;/p>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;name&amp;#34;: &amp;#34;allow_any&amp;#34;,
&amp;#34;domains&amp;#34;: [
&amp;#34;*&amp;#34;
],
&amp;#34;routes&amp;#34;: [
{
&amp;#34;match&amp;#34;: {
&amp;#34;prefix&amp;#34;: &amp;#34;/&amp;#34;
},
&amp;#34;route&amp;#34;: {
&amp;#34;cluster&amp;#34;: &amp;#34;PassthroughCluster&amp;#34;
}
}
]
}
&lt;/code>&lt;/pre>
&lt;p>Prior to Istio 1.3, there were no metrics reported or if metrics were reported
there were no explicit labels set when traffic hit these clusters, resulting in
lack of visibility in traffic flowing through the mesh.&lt;/p>
&lt;p>The next section covers how to take advantage of this enhancement as the metrics
and labels emitted are conditional on whether the virtual outbound or explicit port/protocol
listener is being hit.&lt;/p>
&lt;h2 id="using-the-augmented-metrics">Using the augmented metrics&lt;/h2>
&lt;p>To capture all external service traffic in either of the cases (BlackHole or
Passthrough), you will need to monitor &lt;code>istio_requests_total&lt;/code> and
&lt;code>istio_tcp_connections_closed_total&lt;/code> metrics. Depending upon the Envoy listener
type i.e. TCP proxy or HTTP proxy that gets invoked, one of these metrics
will be incremented.&lt;/p>
&lt;p>Additionally, in case of a TCP proxy listener in order to see the IP address of
the external service that is blocked or allowed via BlackHole or Passthrough
cluster, you will need to add the &lt;code>destination_ip&lt;/code> label to the
&lt;code>istio_tcp_connections_closed_total&lt;/code> metric. In this scenario, the host name of
the external service is not captured. This label is not added by default and can
be easily added by augmenting the Istio configuration for attribute generation
and Prometheus handler. You should be careful about cardinality explosion in
time series if you have many services with non-stable IP addresses.&lt;/p>
&lt;h3 id="passthroughcluster-metrics">PassthroughCluster metrics&lt;/h3>
&lt;p>This section explains the metrics and the labels emitted based on the listener
type invoked in Envoy.&lt;/p>
&lt;ul>
&lt;li>HTTP proxy listener: This happens when the port of the external service is
same as one of the service ports defined in the cluster. In this scenario,
when the PassthroughCluster is hit, &lt;code>istio_requests_total&lt;/code> will get increased
like this:&lt;/li>
&lt;/ul>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;metric&amp;#34;: {
&amp;#34;__name__&amp;#34;: &amp;#34;istio_requests_total&amp;#34;,
&amp;#34;connection_security_policy&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_app&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_principal&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_service&amp;#34;: &amp;#34;httpbin.org&amp;#34;,
&amp;#34;destination_service_name&amp;#34;: &amp;#34;PassthroughCluster&amp;#34;,
&amp;#34;destination_service_namespace&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_version&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_workload&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_workload_namespace&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;instance&amp;#34;: &amp;#34;100.96.2.183:42422&amp;#34;,
&amp;#34;job&amp;#34;: &amp;#34;istio-mesh&amp;#34;,
&amp;#34;permissive_response_code&amp;#34;: &amp;#34;none&amp;#34;,
&amp;#34;permissive_response_policyid&amp;#34;: &amp;#34;none&amp;#34;,
&amp;#34;reporter&amp;#34;: &amp;#34;source&amp;#34;,
&amp;#34;request_protocol&amp;#34;: &amp;#34;http&amp;#34;,
&amp;#34;response_code&amp;#34;: &amp;#34;200&amp;#34;,
&amp;#34;response_flags&amp;#34;: &amp;#34;-&amp;#34;,
&amp;#34;source_app&amp;#34;: &amp;#34;sleep&amp;#34;,
&amp;#34;source_principal&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;source_version&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;source_workload&amp;#34;: &amp;#34;sleep&amp;#34;,
&amp;#34;source_workload_namespace&amp;#34;: &amp;#34;default&amp;#34;
},
&amp;#34;value&amp;#34;: [
1567033080.282,
&amp;#34;1&amp;#34;
]
}
&lt;/code>&lt;/pre>
&lt;p>Note that the &lt;code>destination_service_name&lt;/code> label is set to PassthroughCluster to
indicate that this cluster was hit and the &lt;code>destination_service&lt;/code> is set to the
host of the external service.&lt;/p>
&lt;ul>
&lt;li>TCP proxy virtual listener - If the external service port doesn&amp;rsquo;t map to any
HTTP based service ports within the cluster, this listener is invoked and
&lt;code>istio_tcp_connections_closed_total&lt;/code> is the metric that will be increased:&lt;/li>
&lt;/ul>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;status&amp;#34;: &amp;#34;success&amp;#34;,
&amp;#34;data&amp;#34;: {
&amp;#34;resultType&amp;#34;: &amp;#34;vector&amp;#34;,
&amp;#34;result&amp;#34;: [
{
&amp;#34;metric&amp;#34;: {
&amp;#34;__name__&amp;#34;: &amp;#34;istio_tcp_connections_closed_total&amp;#34;,
&amp;#34;connection_security_policy&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_app&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_ip&amp;#34;: &amp;#34;52.22.188.80&amp;#34;,
&amp;#34;destination_principal&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_service&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_service_name&amp;#34;: &amp;#34;PassthroughCluster&amp;#34;,
&amp;#34;destination_service_namespace&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_version&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_workload&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_workload_namespace&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;instance&amp;#34;: &amp;#34;100.96.2.183:42422&amp;#34;,
&amp;#34;job&amp;#34;: &amp;#34;istio-mesh&amp;#34;,
&amp;#34;reporter&amp;#34;: &amp;#34;source&amp;#34;,
&amp;#34;response_flags&amp;#34;: &amp;#34;-&amp;#34;,
&amp;#34;source_app&amp;#34;: &amp;#34;sleep&amp;#34;,
&amp;#34;source_principal&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;source_version&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;source_workload&amp;#34;: &amp;#34;sleep&amp;#34;,
&amp;#34;source_workload_namespace&amp;#34;: &amp;#34;default&amp;#34;
},
&amp;#34;value&amp;#34;: [
1567033761.879,
&amp;#34;1&amp;#34;
]
}
]
}
}
&lt;/code>&lt;/pre>
&lt;p>In this case, &lt;code>destination_service_name&lt;/code> is set to PassthroughCluster and
the &lt;code>destination_ip&lt;/code> is set to the IP address of the external service.
The &lt;code>destination_ip&lt;/code> label can be used to do a reverse DNS lookup and
get the host name of the external service. As this cluster is passthrough,
other TCP related metrics like &lt;code>istio_tcp_connections_opened_total&lt;/code>,
&lt;code>istio_tcp_received_bytes_total&lt;/code> and &lt;code>istio_tcp_sent_bytes_total&lt;/code> are also
updated.&lt;/p>
&lt;h3 id="blackholecluster-metrics">BlackHoleCluster metrics&lt;/h3>
&lt;p>Similar to the PassthroughCluster, this section explains the metrics and the
labels emitted based on the listener type invoked in Envoy.&lt;/p>
&lt;ul>
&lt;li>HTTP proxy listener: This happens when the port of the external service is same
as one of the service ports defined in the cluster.
In this scenario, when the BlackHoleCluster is hit,
&lt;code>istio_requests_total&lt;/code> will get increased like this:&lt;/li>
&lt;/ul>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;metric&amp;#34;: {
&amp;#34;__name__&amp;#34;: &amp;#34;istio_requests_total&amp;#34;,
&amp;#34;connection_security_policy&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_app&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_principal&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_service&amp;#34;: &amp;#34;httpbin.org&amp;#34;,
&amp;#34;destination_service_name&amp;#34;: &amp;#34;BlackHoleCluster&amp;#34;,
&amp;#34;destination_service_namespace&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_version&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_workload&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_workload_namespace&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;instance&amp;#34;: &amp;#34;100.96.2.183:42422&amp;#34;,
&amp;#34;job&amp;#34;: &amp;#34;istio-mesh&amp;#34;,
&amp;#34;permissive_response_code&amp;#34;: &amp;#34;none&amp;#34;,
&amp;#34;permissive_response_policyid&amp;#34;: &amp;#34;none&amp;#34;,
&amp;#34;reporter&amp;#34;: &amp;#34;source&amp;#34;,
&amp;#34;request_protocol&amp;#34;: &amp;#34;http&amp;#34;,
&amp;#34;response_code&amp;#34;: &amp;#34;502&amp;#34;,
&amp;#34;response_flags&amp;#34;: &amp;#34;-&amp;#34;,
&amp;#34;source_app&amp;#34;: &amp;#34;sleep&amp;#34;,
&amp;#34;source_principal&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;source_version&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;source_workload&amp;#34;: &amp;#34;sleep&amp;#34;,
&amp;#34;source_workload_namespace&amp;#34;: &amp;#34;default&amp;#34;
},
&amp;#34;value&amp;#34;: [
1567034251.717,
&amp;#34;1&amp;#34;
]
}
&lt;/code>&lt;/pre>
&lt;p>Note the &lt;code>destination_service_name&lt;/code> label is set to BlackHoleCluster and the
&lt;code>destination_service&lt;/code> to the host name of the external service. The response
code should always be &lt;code>502&lt;/code> in this case.&lt;/p>
&lt;ul>
&lt;li>TCP proxy virtual listener - If the external service port doesn&amp;rsquo;t map to any
HTTP based service ports within the cluster, this listener is invoked and
&lt;code>istio_tcp_connections_closed_total&lt;/code> is the metric that will be increased:&lt;/li>
&lt;/ul>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;metric&amp;#34;: {
&amp;#34;__name__&amp;#34;: &amp;#34;istio_tcp_connections_closed_total&amp;#34;,
&amp;#34;connection_security_policy&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_app&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_ip&amp;#34;: &amp;#34;52.22.188.80&amp;#34;,
&amp;#34;destination_principal&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_service&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_service_name&amp;#34;: &amp;#34;BlackHoleCluster&amp;#34;,
&amp;#34;destination_service_namespace&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_version&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_workload&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;destination_workload_namespace&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;instance&amp;#34;: &amp;#34;100.96.2.183:42422&amp;#34;,
&amp;#34;job&amp;#34;: &amp;#34;istio-mesh&amp;#34;,
&amp;#34;reporter&amp;#34;: &amp;#34;source&amp;#34;,
&amp;#34;response_flags&amp;#34;: &amp;#34;-&amp;#34;,
&amp;#34;source_app&amp;#34;: &amp;#34;sleep&amp;#34;,
&amp;#34;source_principal&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;source_version&amp;#34;: &amp;#34;unknown&amp;#34;,
&amp;#34;source_workload&amp;#34;: &amp;#34;sleep&amp;#34;,
&amp;#34;source_workload_namespace&amp;#34;: &amp;#34;default&amp;#34;
},
&amp;#34;value&amp;#34;: [
1567034481.03,
&amp;#34;1&amp;#34;
]
}
&lt;/code>&lt;/pre>
&lt;p>Note the &lt;code>destination_ip&lt;/code> label represents the IP address of the external
service and the &lt;code>destination_service_name&lt;/code> is set to BlackHoleCluster
to indicate that this traffic was blocked by the mesh. Is is interesting to
note that for the BlackHole cluster case, other TCP related metrics like
&lt;code>istio_tcp_connections_opened_total&lt;/code> are not increased as there&amp;rsquo;s no
connection that is ever established.&lt;/p>
&lt;p>Monitoring these metrics can help operators easily understand all the external
services consumed by the applications in their cluster.&lt;/p></description><pubDate>Sat, 28 Sep 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/monitoring-external-service-traffic/</link><author>Neeraj Poddar (Aspen Mesh)</author><guid isPermaLink="true">/v1.12/blog/2019/monitoring-external-service-traffic/</guid><category>monitoring</category><category>blackhole</category><category>passthrough</category></item><item><title>Mixer Adapter for Knative</title><description>
&lt;p>This post demonstrates how you can use Mixer to push application logic
into Istio. It describes a Mixer adapter which implements the &lt;a href="https://knative.dev/">Knative&lt;/a> scale-from-zero logic
with simple code and similar performance to the original implementation.&lt;/p>
&lt;h2 id="knative-serving">Knative serving&lt;/h2>
&lt;p>&lt;a href="https://knative.dev/docs/serving/">Knative Serving&lt;/a> builds on &lt;a href="https://kubernetes.io/">Kubernetes&lt;/a> to support deploying
and serving of serverless applications. A core capability of serverless platforms is scale-to-zero
functionality which reduces resource usage and cost of inactive workloads.
A new mechanism is required to scale from zero when an idle application receives a new request.&lt;/p>
&lt;p>The following diagram represents the current Knative architecture for scale-from-zero.&lt;/p>
&lt;figure style="width:60%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:76.29350893697084%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/knative-activator-adapter/knative-activator.png" title="Knative scale-from-zero">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/knative-activator-adapter/knative-activator.png" alt="Knative scale-from-zero" />
&lt;/a>
&lt;/div>
&lt;figcaption>Knative scale-from-zero&lt;/figcaption>
&lt;/figure>
&lt;p>The traffic for an idle application is redirected to &lt;strong>Activator&lt;/strong> component by programming Istio with &lt;code>VirtualServices&lt;/code>
and &lt;code>DestinationRules&lt;/code>. When &lt;strong>Activator&lt;/strong> receives a new request, it:&lt;/p>
&lt;ol>
&lt;li>buffers incoming requests&lt;/li>
&lt;li>triggers the &lt;strong>Autoscaler&lt;/strong>&lt;/li>
&lt;li>redirects requests to the application after it has been scaled up, including retries and load-balancing (if needed)&lt;/li>
&lt;/ol>
&lt;p>Once the application is up and running again, Knative restores the routing from &lt;strong>Activator&lt;/strong> to the running application.&lt;/p>
&lt;h2 id="mixer-adapter">Mixer adapter&lt;/h2>
&lt;p>Mixer provides a rich intermediation layer between the Istio components and infrastructure backends.
It is designed as a stand-alone component, separate from &lt;a href="https://www.envoyproxy.io/">Envoy&lt;/a>, and has a simple extensibility model
to enable Istio to interoperate with a wide breadth of backends. Mixer is inherently easier to extend
than Envoy is.&lt;/p>
&lt;p>Mixer is an attribute processing engine that uses operator-supplied configuration to map request attributes from the Istio proxy into calls
to the infrastructure backends systems via a pluggable set of adapters. Adapters enable &lt;strong>Mixer&lt;/strong> to expose a single consistent API, independent of the
infrastructure backends in use. The exact set of adapters used at runtime is determined through operator configuration and can easily
be extended to target new or custom infrastructure backends.&lt;/p>
&lt;p>In order to achieve Knative scale-from-zero, we use a Mixer &lt;a href="https://github.com/istio/istio/wiki/Mixer-Out-Of-Process-Adapter-Dev-Guide">out-of-process adapter&lt;/a>
to call the Autoscaler. Out-of-process adapters for Mixer allow developers to use any
programming language and to build and maintain your extension as a stand-alone program
without the need to build the Istio proxy.&lt;/p>
&lt;p>The following diagram represents the Knative design using the &lt;strong>Mixer&lt;/strong> adapter.&lt;/p>
&lt;figure style="width:60%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:76.29350893697084%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/knative-activator-adapter/knative-mixer-adapter.png" title="Knative scale-from-zero">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/knative-activator-adapter/knative-mixer-adapter.png" alt="Knative scale-from-zero" />
&lt;/a>
&lt;/div>
&lt;figcaption>Knative scale-from-zero&lt;/figcaption>
&lt;/figure>
&lt;p>In this design, there is no need to change the routing from/to &lt;strong>Activator&lt;/strong> for an idle application as in the original Knative setup.
When the Istio proxy represented by the ingress gateway component receives a new request for an idle application, it informs &lt;strong>Mixer&lt;/strong>, including all the
relevant metadata information.
&lt;strong>Mixer&lt;/strong> then calls your adapter which triggers the Knative &lt;strong>Autoscaler&lt;/strong> using the original Knative protocol.&lt;/p>
&lt;div>
&lt;aside class="callout idea">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-idea"/>&lt;/svg>
&lt;/div>
&lt;div class="content">By using this design you do not need to deal with buffering, retries and load-balancing because it is already handled by the Istio proxy.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Istio&amp;rsquo;s use of Mixer adapters makes it possible to replace otherwise complex networking-based application logic with a more
straightforward implementation, as demonstrated in the &lt;a href="https://github.com/zachidan/istio-kactivator">Knative adapter&lt;/a>.&lt;/p>
&lt;p>When the adapter receives a message from &lt;strong>Mixer&lt;/strong>, it sends a &lt;code>StatMessage&lt;/code> directly to &lt;strong>Autoscaler&lt;/strong>
component using the Knative protocol.
The metadata information (&lt;code>namespace&lt;/code> and &lt;code>service name&lt;/code>) required by &lt;strong>Autoscaler&lt;/strong> are transferred by Istio proxy to
&lt;strong>Mixer&lt;/strong> and from there to the adapter.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>I compared the cold-start time of the original Knative reference architecture to the new Istio Mixer adapter reference architecture.
The results show similar cold-start times.
The implementation using the Mixer adapter has greater simplicity. It is not necessary to handle low-level network-based mechanisms as these are handled by Envoy.&lt;/p>
&lt;p>The next step is converting this Mixer adapter into an Envoy-specific filter running inside an ingress gateway.
This will allow to further improve the latency overhead (no more calls to &lt;strong>Mixer&lt;/strong> and the adapter) and
to remove the dependency on the Istio Mixer.&lt;/p></description><pubDate>Wed, 18 Sep 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/knative-activator-adapter/</link><author>Idan Zach (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/knative-activator-adapter/</guid><category>mixer</category><category>adapter</category><category>knative</category><category>scale-from-zero</category></item><item><title>App Identity and Access Adapter</title><description>
&lt;p>If you are running your containerized applications on Kubernetes, you can benefit from using the App Identity and Access Adapter for an abstracted level of security with zero code changes or redeploys.&lt;/p>
&lt;p>Whether your computing environment is based on a single cloud provider, a combination of multiple cloud providers, or following a hybrid cloud approach, having a centralized identity management can help you to preserve existing infrastructure and avoid vendor lock-in.&lt;/p>
&lt;p>With the &lt;a href="https://github.com/ibm-cloud-security/app-identity-and-access-adapter">App Identity and Access Adapter&lt;/a>, you can use any OAuth2/OIDC provider: IBM Cloud App ID, Auth0, Okta, Ping Identity, AWS Cognito, Azure AD B2C and more. Authentication and authorization policies can be applied in a streamlined way in all environments — including frontend and backend applications — all without code changes or redeploys.&lt;/p>
&lt;h2 id="understanding-istio-and-the-adapter">Understanding Istio and the adapter&lt;/h2>
&lt;p>&lt;a href="/v1.12/about/service-mesh/">Istio&lt;/a> is an open source service mesh that
transparently layers onto distributed applications and seamlessly integrates
with Kubernetes. To reduce the complexity of deployments Istio provides
behavioral insights and operational control over the service mesh as a whole.
See the &lt;a href="/v1.12/docs/ops/deployment/architecture/">Istio Architecture&lt;/a> for more details.&lt;/p>
&lt;p>Istio uses &lt;a href="/v1.12/blog/2019/data-plane-setup/">Envoy proxy sidecars&lt;/a> to mediate inbound and outbound traffic for all pods in the service mesh. Istio extracts telemetry from the Envoy sidecars and sends it to Mixer, the Istio component responsible for collecting telemetry and enforcing policy.&lt;/p>
&lt;p>The App Identity and Access adapter extends the Mixer functionality by analyzing the telemetry (attributes) against various access control policies across the service mesh. The access control policies can be linked to a particular Kubernetes services and can be finely tuned to specific service endpoints. For more information about policies and telemetry, see the Istio documentation.&lt;/p>
&lt;p>When &lt;a href="https://github.com/ibm-cloud-security/app-identity-and-access-adapter">App Identity and Access Adapter&lt;/a> is combined with Istio, it provides a scalable, integrated identity and access solution for multicloud architectures that does not require any custom application code changes.&lt;/p>
&lt;h2 id="installation">Installation&lt;/h2>
&lt;p>App Identity and Access adapter can be installed using Helm directly from the &lt;code>github.com&lt;/code> repository&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ helm repo add appidentityandaccessadapter https://raw.githubusercontent.com/ibm-cloud-security/app-identity-and-access-adapter/master/helm/appidentityandaccessadapter
$ helm install --name appidentityandaccessadapter appidentityandaccessadapter/appidentityandaccessadapter
&lt;/code>&lt;/pre>
&lt;p>Alternatively, you can clone the repository and install the Helm chart locally&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ git clone git@github.com:ibm-cloud-security/app-identity-and-access-adapter.git
$ helm install ./helm/appidentityandaccessadapter --name appidentityandaccessadapter.
&lt;/code>&lt;/pre>
&lt;h2 id="protecting-web-applications">Protecting web applications&lt;/h2>
&lt;p>Web applications are most commonly protected by the OpenID Connect (OIDC) workflow called &lt;code>authorization_code&lt;/code>. When an unauthenticated/unauthorized user is detected, they are automatically redirected to the identity service of your choice and presented with the authentication page. When authentication completes, the browser is redirected back to an implicit &lt;code>/oidc/callback&lt;/code> endpoint intercepted by the adapter. At this point, the adapter obtains access and identity tokens from the identity service and then redirects users back to their originally requested URL in the web app.&lt;/p>
&lt;p>Authentication state and tokens are maintained by the adapter. Each request processed by the adapter will include the Authorization header bearing both access and identity tokens in the following format &lt;code>Authorization: Bearer &amp;lt;access_token&amp;gt; &amp;lt;id_token&amp;gt;&lt;/code>&lt;/p>
&lt;p>Developers can read leverage the tokens for application experience adjustments, e.g. displaying user name, adjusting UI based on user role etc.&lt;/p>
&lt;p>In order to terminate the authenticated session and wipe tokens, aka user logout, simply redirect browser to the &lt;code>/oidc/logout&lt;/code> endpoint under the protected service, e.g. if you&amp;rsquo;re serving your app from &lt;code>https://example.com/myapp&lt;/code>, redirect users to &lt;code>https://example.com/myapp/oidc/logout&lt;/code>&lt;/p>
&lt;p>Whenever access token expires, a refresh token is used to automatically acquire new access and identity tokens without your user&amp;rsquo;s needing to re-authenticate. If the configured identity provider returns a refresh token, it is persisted by the adapter and used to retrieve new access and identity tokens when the old ones expire.&lt;/p>
&lt;h3 id="applying-web-application-protection">Applying web application protection&lt;/h3>
&lt;p>Protecting web applications requires creating two types of resources - use &lt;code>OidcConfig&lt;/code> resources to define various OIDC providers, and &lt;code>Policy&lt;/code> resources to define the web app protection policies.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;security.cloud.ibm.com/v1&amp;#34;
kind: OidcConfig
metadata:
name: my-oidc-provider-config
namespace: sample-namespace
spec:
discoveryUrl: &amp;lt;discovery-url-from-oidc-provider&amp;gt;
clientId: &amp;lt;client-id-from-oidc-provider&amp;gt;
clientSecretRef:
name: &amp;lt;kubernetes-secret-name&amp;gt;
key: &amp;lt;kubernetes-secret-key&amp;gt;
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;security.cloud.ibm.com/v1&amp;#34;
kind: Policy
metadata:
name: my-sample-web-policy
namespace: sample-namespace
spec:
targets:
- serviceName: &amp;lt;kubernetes-service-name-to-protect&amp;gt;
paths:
- prefix: /webapp
method: ALL
policies:
- policyType: oidc
config: my-oidc-provider-config
rules: // optional
- claim: iss
match: ALL
source: access_token
values:
- &amp;lt;expected-issuer-id&amp;gt;
- claim: scope
match: ALL
source: access_token
values:
- openid
&lt;/code>&lt;/pre>
&lt;p>&lt;a href="https://github.com/ibm-cloud-security/app-identity-and-access-adapter">Read more about protecting web applications&lt;/a>&lt;/p>
&lt;h2 id="protecting-backend-application-and-apis">Protecting backend application and APIs&lt;/h2>
&lt;p>Backend applications and APIs are protected using the Bearer Token flow, where an incoming token is validated against a particular policy. The Bearer Token authorization flow expects a request to contain the &lt;code>Authorization&lt;/code> header with a valid access token in JWT format. The expected header structure is &lt;code>Authorization: Bearer {access_token}&lt;/code>. In case token is successfully validated request will be forwarded to the requested service. In case token validation fails the HTTP 401 will be returned back to the client with a list of scopes that are required to access the API.&lt;/p>
&lt;h3 id="applying-backend-application-and-apis-protection">Applying backend application and APIs protection&lt;/h3>
&lt;p>Protecting backend applications and APIs requires creating two types of resources - use &lt;code>JwtConfig&lt;/code> resources to define various JWT providers, and &lt;code>Policy&lt;/code> resources to define the backend app protection policies.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;security.cloud.ibm.com/v1&amp;#34;
kind: JwtConfig
metadata:
name: my-jwt-config
namespace: sample-namespace
spec:
jwksUrl: &amp;lt;the-jwks-url&amp;gt;
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;security.cloud.ibm.com/v1&amp;#34;
kind: Policy
metadata:
name: my-sample-backend-policy
namespace: sample-namespace
spec:
targets:
- serviceName: &amp;lt;kubernetes-service-name-to-protect&amp;gt;
paths:
- prefix: /api/files
method: ALL
policies:
- policyType: jwt
config: my-oidc-provider-config
rules: // optional
- claim: iss
match: ALL
source: access_token
values:
- &amp;lt;expected-issuer-id&amp;gt;
- claim: scope
match: ALL
source: access_token
values:
- files.read
- files.write
&lt;/code>&lt;/pre>
&lt;p>&lt;a href="https://github.com/ibm-cloud-security/app-identity-and-access-adapter">Read more about protecting backend applications&lt;/a>&lt;/p>
&lt;h2 id="known-limitations">Known limitations&lt;/h2>
&lt;p>At the time of writing this blog there are two known limitations of the App Identity and Access adapter:&lt;/p>
&lt;ul>
&lt;li>&lt;p>If you use the App Identity and Access adapter for Web Applications you should not create more than a single replica of the adapter. Due to the way Envoy Proxy was handling HTTP headers it was impossible to return multiple &lt;code>Set-Cookie&lt;/code> headers from Mixer back to Envoy. Therefore we couldn&amp;rsquo;t set all the cookies required for handling Web Application scenarios. The issue was recently addressed in Envoy and Mixer and we&amp;rsquo;re planning to address this in future versions of our adapter. &lt;strong>Note that this only affects Web Applications, and doesn&amp;rsquo;t affect Backend Apps and APIs in any way&lt;/strong>.&lt;/p>&lt;/li>
&lt;li>&lt;p>As a general best practice you should always consider using mutual-tls for any in-cluster communications. At the moment the communications channel between Mixer and App Identity and Access adapter currently does not use mutual-tls. In future we plan to address this by implementing an approach described in the &lt;a href="https://github.com/istio/istio/wiki/Mixer-Out-of-Process-Adapter-Walkthrough#step-7-encrypt-connection-between-mixer-and-grpc-adapter">Mixer Adapter developer guide&lt;/a>.&lt;/p>&lt;/li>
&lt;/ul>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>When a multicloud strategy is in place, security can become complicated as the environment grows and diversifies. While cloud providers supply protocols and tools to ensure their offerings are safe, the development teams are still responsible for the application-level security, such as API access control with OAuth2, defending against man-in-the-middle attacks with traffic encryption, and providing mutual TLS for service access control. However, this becomes complex in a multicloud environment since you might need to define those security details for each service separately. With proper security protocols in place, those external and internal threats can be mitigated.&lt;/p>
&lt;p>Development teams have spent time making their services portable to different cloud providers, and in the same regard, the security in place should be flexible and not infrastructure-dependent.&lt;/p>
&lt;p>Istio and App Identity and Access Adapter allow you to secure your Kubernetes apps with absolutely zero code changes or redeployments regardless of which programming language and which frameworks you use. Following this approach ensures maximum portability of your apps, and ability to easily enforce same security policies across multiple environments.&lt;/p>
&lt;p>You can read more about the App Identity and Access Adapter in the &lt;a href="https://www.ibm.com/cloud/blog/using-istio-to-secure-your-multicloud-kubernetes-applications-with-zero-code-change">release blog&lt;/a>.&lt;/p></description><pubDate>Wed, 18 Sep 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/app-identity-and-access-adapter/</link><author>Anton Aleksandrov (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/app-identity-and-access-adapter/</guid><category>security</category><category>oidc</category><category>jwt</category><category>policies</category></item><item><title>Change in Secret Discovery Service in Istio 1.3</title><description>&lt;p>In Istio 1.3, we are taking advantage of improvements in Kubernetes to issue certificates for workload instances more securely.&lt;/p>
&lt;p>When a Citadel Agent sends a certificate signing request to Citadel to get a certificate for a workload instance,
it includes the JWT that the Kubernetes API server issued representing the service account of the workload instance.
If Citadel can authenticate the JWT, it extracts the service account name needed to issue the certificate for the workload instance.&lt;/p>
&lt;p>Before Kubernetes 1.12, the Kubernetes API server issues JWTs with the following issues:&lt;/p>
&lt;ol>
&lt;li>The tokens don&amp;rsquo;t have important fields to limit their scope of usage, such as &lt;code>aud&lt;/code> or &lt;code>exp&lt;/code>. See &lt;a href="https://github.com/kubernetes/community/blob/master/contributors/design-proposals/auth/bound-service-account-tokens.md">Bound Service Tokens&lt;/a> for more info.&lt;/li>
&lt;li>The tokens are mounted onto all the pods without a way to opt-out. See &lt;a href="https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/svcacct-token-volume-source.md">Service Account Token Volumes&lt;/a> for motivation.&lt;/li>
&lt;/ol>
&lt;p>Kubernetes 1.12 introduces &lt;code>trustworthy&lt;/code> JWTs to solve these issues.
However, support for the &lt;code>aud&lt;/code> field to have a different value than the API server audience didn&amp;rsquo;t become available until &lt;a href="https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.13.md">Kubernetes 1.13&lt;/a>.
To better secure the mesh, Istio 1.3 only supports &lt;code>trustworthy&lt;/code> JWTs and requires the value of the &lt;code>aud&lt;/code> field to be &lt;code>istio-ca&lt;/code> when you enable SDS.
Before upgrading your Istio deployment to 1.3 with SDS enabled, verify that you use Kubernetes 1.13 or later.&lt;/p>
&lt;p>Make the following considerations based on your platform of choice:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>GKE:&lt;/strong> Upgrade your cluster version to at least 1.13.&lt;/li>
&lt;li>&lt;strong>On-prem Kubernetes&lt;/strong> and &lt;strong>GKE on-prem:&lt;/strong> Add &lt;a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection">extra configurations&lt;/a> to your Kubernetes. You may
also want to refer to the &lt;a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/">api-server page&lt;/a> for the most up-to-date flag names.&lt;/li>
&lt;li>For other platforms, check with your provider. If your vendor does not support trustworthy JWTs, you will need to fall back to the file-mount approach to propagate the workload keys and certificates in Istio 1.3.&lt;/li>
&lt;/ul></description><pubDate>Tue, 10 Sep 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/trustworthy-jwt-sds/</link><author>Phillip Quy Le (Google)</author><guid isPermaLink="true">/v1.12/blog/2019/trustworthy-jwt-sds/</guid><category>security</category><category>PKI</category><category>certificate</category><category>nodeagent</category><category>sds</category></item><item><title>The Evolution of Istio's APIs</title><description>
&lt;p>One of Istio’s main goals has always been, and continues to be, enabling teams to develop abstractions that work best for their specific organization and workloads. Istio provides robust and powerful building blocks for service-to-service networking. Since &lt;a href="/v1.12/news/releases/0.x/announcing-0.1">Istio 0.1&lt;/a>, the Istio team has been learning from production users about how they map their own architectures, workloads, and constraints to Istio’s capabilities, and we’ve been evolving Istio’s APIs to make them work better for you.&lt;/p>
&lt;h2 id="evolving-istio-s-apis">Evolving Istio’s APIs&lt;/h2>
&lt;p>The next step in Istio’s evolution is to sharpen our focus and align with the roles of Istio’s users. A security admin should be able to interact with an API that logically groups and simplifies security operations within an Istio mesh; the same goes for service operators and traffic management operations.&lt;/p>
&lt;p>Taking it a step further, there’s an opportunity to provide improved experiences for beginning, intermediate, and advanced use cases for each role. There are many common use cases that can be addressed with obvious default settings and a better defined initial experience that requires little to no configuration. For intermediate use cases, the Istio team wants to leverage contextual cues from the environment and provide you with a simpler configuration experience. Finally, for advanced scenarios, our goal is to make &lt;a href="https://www.quora.com/What-is-the-origin-of-the-phrase-make-the-easy-things-easy-and-the-hard-things-possible">easy things easy and hard things possible&lt;/a>.&lt;/p>
&lt;p>To provide these sorts of role-centric abstractions, however, the APIs underneath them must be able to describe all of Istio’s power and capabilities. Historically, Istio’s approach to API design followed paths similar to those of other infrastructure APIs. Istio follows these design principles:&lt;/p>
&lt;ol>
&lt;li>The Istio APIs should seek to:
&lt;ul>
&lt;li>Properly represent the underlying resources to which they are mapped&lt;/li>
&lt;li>Shouldn’t hide any of the underlying resource’s useful capabilities&lt;/li>
&lt;/ul>&lt;/li>
&lt;li>The Istio APIs should also be &lt;a href="https://en.wikipedia.org/wiki/Composability">composable&lt;/a>, so end users can combine infrastructure APIs in a way that makes sense for their own needs.&lt;/li>
&lt;li>The Istio APIs should be flexible: Within an organization, it should be possible to have different representations of the underlying resources and surface the ones that make sense for each individual team.&lt;/li>
&lt;/ol>
&lt;p>Over the course of the next several releases we will share our progress as we strengthen the alignment between Istio’s APIs and the roles of Istio users.&lt;/p>
&lt;h2 id="composability-and-abstractions">Composability and abstractions&lt;/h2>
&lt;p>Istio and Kubernetes often go together, but Istio is much more than an add-on to Kubernetes – it is as much a &lt;em>platform&lt;/em> as Kubernetes is. Istio aims to provide infrastructure, and surface the capabilities you need in a powerful service mesh. For example, there are platform-as-a-service offerings that use Kubernetes as their foundation, and build on Kubernetes’ composability to provide a subset of APIs to application developers.&lt;/p>
&lt;p>The number of objects that must be configured to deploy applications is a concrete example of Kubernetes’ composability. By our count, at least 10 objects need to be configured: &lt;code>Namespace&lt;/code>, &lt;code>Service&lt;/code>, &lt;code>Ingress&lt;/code>, &lt;code>Deployment&lt;/code>, &lt;code>HorizontalPodAutoscaler&lt;/code>, &lt;code>Secret&lt;/code>, &lt;code>ConfigMap&lt;/code>, &lt;code>RBAC&lt;/code>, &lt;code>PodDisruptionBudget&lt;/code>, and &lt;code>NetworkPolicy&lt;/code>.&lt;/p>
&lt;p>It sounds complicated, but not everyone needs to interact with those concepts. Some are the responsibility of different teams like the cluster, network, or security admin teams, and many provide sensible defaults. A great benefit of cloud native platforms and deployment tools is that they can hide that complexity by taking in a small amount of information and configuring those objects for you.&lt;/p>
&lt;p>Another example of composability in the networking space can be found in the &lt;a href="https://cloud.google.com/load-balancing/docs/https/">Google Cloud HTTP(S) Load Balancer&lt;/a> (GCLB). To correctly use an instance of the GCLB, six different infrastructure objects need to be created and configured. This design is the result of our 20 years of experience in operating distributed systems and &lt;a href="https://www.youtube.com/watch?v=J5HJ1y6PeyE">there is a reason why each one is separate from the others&lt;/a>. But the steps are simplified when you’re creating an instance via the Google Cloud console. We provide the more common end-user/role-specific configurations, and you can configure less common settings later. Ultimately, the goals of infrastructure APIs are to offer the most flexibility without sacrificing functionality.&lt;/p>
&lt;p>&lt;a href="http://knative.dev">Knative&lt;/a> is a platform for building, running, and operating serverless workloads that provides a great real-world example of role-centric,
higher-level APIs. &lt;a href="https://knative.dev/docs/serving/">Knative Serving&lt;/a>, a component of Knative that builds on Kubernetes and Istio to support deploying and
serving serverless applications and functions, provides an opinionated workflow for application developers to manage routes and revisions of their services.
Thanks to that opinionated approach, Knative Serving exposes a subset of Istio’s networking APIs that are most relevant to application developers via a simplified
&lt;a href="https://github.com/knative/docs/blob/master/docs/serving/spec/knative-api-specification-1.0.md#route">Routes&lt;/a> object that supports revisions and traffic routing,
abstracting Istio’s &lt;a href="/v1.12/docs/reference/config/networking/virtual-service/">&lt;code>VirtualService&lt;/code>&lt;/a> and &lt;a href="/v1.12/docs/reference/config/networking/destination-rule/">&lt;code>DestinationRule&lt;/code>&lt;/a>
resources.&lt;/p>
&lt;p>As Istio has matured, we’ve also seen production users develop workload- and organization-specific abstractions on top of Istio’s infrastructure APIs.&lt;/p>
&lt;p>AutoTrader UK has one of our favorite examples of a custom platform built on Istio. In &lt;a href="https://kubernetespodcast.com/episode/052-autotrader/">an interview with the Kubernetes Podcast from Google&lt;/a>, Russel Warman and Karl Stoney describe their Kubernetes-based delivery platform, with &lt;a href="https://karlstoney.com/2018/07/07/managing-your-costs-on-kubernetes/">cost dashboards using Prometheus and Grafana&lt;/a>. With minimal effort, they added configuration options to determine what their developers want configured on the network, and it now manages the Istio objects required to make that happen. There are countless other platforms being built in enterprise and cloud-native companies: some designed to replace a web of company-specific custom scripts, and some aimed to be a general-purpose public tool. As more companies start to talk about their tooling publicly, we&amp;rsquo;ll bring their stories to this blog.&lt;/p>
&lt;h2 id="what-s-coming-next">What’s coming next&lt;/h2>
&lt;p>Some areas of improvement that we’re working on for upcoming releases include:&lt;/p>
&lt;ul>
&lt;li>Installation profiles to setup standard patterns for ingress and egress, with the Istio operator&lt;/li>
&lt;li>Automatic inference of container ports and protocols for telemetry&lt;/li>
&lt;li>Support for routing all traffic by default to constrain routing incrementally&lt;/li>
&lt;li>Add a single global flag to enable mutual TLS and encrypt all inter-pod traffic&lt;/li>
&lt;/ul>
&lt;p>Oh, and if for some reason you judge a toolbox by the list of CRDs it installs, in Istio 1.2 we cut the number from 54 down to 23. Why? It turns out that if you have a bunch of features, you need to have a way to configure them all. With the improvements we’ve made to our installer, you can now install Istio using a &lt;a href="/v1.12/docs/setup/additional-setup/config-profiles/">configuration&lt;/a> that works with your adapters.&lt;/p>
&lt;p>All service meshes and, by extension, Istio seeks to automate complex infrastructure operations, like networking and security. That means there will always be complexity in its APIs, but Istio will always aim to solve the needs of operators, while continuing to evolve the API to provide robust building blocks and prioritize flexibility through role-centric abstractions.&lt;/p>
&lt;p>We can&amp;rsquo;t wait for you to join our &lt;a href="/v1.12/get-involved/">community&lt;/a> to see what you build with Istio next!&lt;/p></description><pubDate>Mon, 05 Aug 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/evolving-istios-apis/</link><author>Louis Ryan (Google), Sandeep Parikh (Google)</author><guid isPermaLink="true">/v1.12/blog/2019/evolving-istios-apis/</guid><category>apis</category><category>composability</category><category>evolution</category></item><item><title>Secure Control of Egress Traffic in Istio, part 3</title><description>
&lt;p>Welcome to part 3 in our series about secure control of egress traffic in Istio.
In &lt;a href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-1/">the first part in the series&lt;/a>, I presented the attacks involving
egress traffic and the requirements we collected for a secure control system for egress traffic.
In &lt;a href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-2/">the second part in the series&lt;/a>, I presented the Istio way of
securing egress traffic and showed how you can prevent the attacks using Istio.&lt;/p>
&lt;p>In this installment, I compare secure control of egress traffic in Istio with alternative solutions such as using Kubernetes
network policies and legacy egress proxies and firewalls. Finally, I describe the performance considerations regarding the
secure control of egress traffic in Istio.&lt;/p>
&lt;h2 id="alternative-solutions-for-egress-traffic-control">Alternative solutions for egress traffic control&lt;/h2>
&lt;p>First, let&amp;rsquo;s remember the &lt;a href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-1/#requirements-for-egress-traffic-control">requirements for egress traffic control&lt;/a> we previously collected:&lt;/p>
&lt;ol>
&lt;li>Support of &lt;a href="https://en.wikipedia.org/wiki/Transport_Layer_Security">TLS&lt;/a> with
&lt;a href="https://en.wikipedia.org/wiki/Server_Name_Indication">SNI&lt;/a> or of &lt;a href="/v1.12/docs/reference/glossary/#tls-origination">TLS origination&lt;/a>.&lt;/li>
&lt;li>&lt;strong>Monitor&lt;/strong> SNI and the source workload of every egress access.&lt;/li>
&lt;li>Define and enforce &lt;strong>policies per cluster&lt;/strong>.&lt;/li>
&lt;li>Define and enforce &lt;strong>policies per source&lt;/strong>, &lt;em>Kubernetes-aware&lt;/em>.&lt;/li>
&lt;li>&lt;strong>Prevent tampering&lt;/strong>.&lt;/li>
&lt;li>Traffic control is &lt;strong>transparent&lt;/strong> to the applications.&lt;/li>
&lt;/ol>
&lt;p>Next, I&amp;rsquo;m going to cover two alternative solutions for egress traffic control: the Kubernetes network policies and
egress proxies and firewalls. I show the requirements they satisfy, and, more importantly, the requirements they can&amp;rsquo;t satisfy.&lt;/p>
&lt;p>Kubernetes provides a native solution for traffic control, and in particular, for control of egress traffic, through the &lt;a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/">network policies&lt;/a>.
Using these network policies, cluster operators can configure which pods can access specific external services.
Cluster operators can identify pods by pod labels, namespace labels, or by IP ranges. To specify the external services, cluster operators can use IP ranges, but cannot use domain names like &lt;code>cnn.com&lt;/code>. This is because &lt;strong>Kubernetes network policies are not DNS-aware&lt;/strong>.
Network policies satisfy the first requirement since they can control any TCP traffic.
Network policies only partially satisfy the third and the fourth requirements because cluster operators can specify policies
per cluster or per pod but operators can&amp;rsquo;t identify external services by domain names.
Network policies only satisfy the fifth requirement if the attackers are not able to break from a malicious container into the Kubernetes
node and interfere with the implementation of the policies inside said node.
Lastly, network policies do satisfy the sixth requirement: Operators don&amp;rsquo;t need to change the code or the
container environment. In summary, we can say that Kubernetes Network Policies provide transparent, Kubernetes-aware egress traffic
control, which is not DNS-aware.&lt;/p>
&lt;p>The second alternative predates the Kubernetes network policies. Using a &lt;strong>DNS-aware egress proxy or firewall&lt;/strong> lets you
configure applications to direct the traffic to the proxy and use some proxy protocol, for example,
&lt;a href="https://en.wikipedia.org/wiki/SOCKS">SOCKS&lt;/a>.
Since operators must configure the applications, this solution is not transparent. Moreover, operators can&amp;rsquo;t use
pod labels or pod service accounts to configure the proxies because the egress proxies don&amp;rsquo;t know about them. Therefore, &lt;strong>the egress proxies are not Kubernetes-aware&lt;/strong> and can&amp;rsquo;t fulfill the fourth requirement because
egress proxies cannot enforce policies by source if a Kubernetes artifact specifies the source.
In summary, egress proxies can fulfill the first, second, third and fifth requirements, but can&amp;rsquo;t satisfy the fourth and
the six requirements because they are not transparent and not Kubernetes-aware.&lt;/p>
&lt;h2 id="advantages-of-istio-egress-traffic-control">Advantages of Istio egress traffic control&lt;/h2>
&lt;p>Istio egress traffic control is &lt;strong>DNS-aware&lt;/strong>: you can define policies based on URLs or on wildcard domains like
&lt;code>*.ibm.com&lt;/code>. In this sense, it is better than Kubernetes network policies which are not DNS-aware.&lt;/p>
&lt;p>Istio egress traffic control is &lt;strong>transparent&lt;/strong> with regard to TLS traffic, since Istio is transparent:
you don&amp;rsquo;t need to change the applications or configure their containers.
For HTTP traffic with TLS origination, you must configure the applications in the mesh to use HTTP instead of HTTPS.&lt;/p>
&lt;p>Istio egress traffic control is &lt;strong>Kubernetes-aware&lt;/strong>: the identity of the source of egress traffic is based on
Kubernetes service accounts. Istio egress traffic control is better than the legacy DNS-aware proxies or firewalls which
are not transparent and not Kubernetes-aware.&lt;/p>
&lt;p>Istio egress traffic control is &lt;strong>secure&lt;/strong>: it is based on the strong identity of Istio and, when you
apply
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#additional-security-considerations">additional security measures&lt;/a>,
Istio&amp;rsquo;s traffic control is resilient to tampering.&lt;/p>
&lt;p>Additionally, Istio&amp;rsquo;s egress traffic control provides the following advantages:&lt;/p>
&lt;ul>
&lt;li>Define access policies in the same language for ingress, egress, and in-cluster traffic. You
need to learn a single policy and configuration language for all types of traffic.&lt;/li>
&lt;li>Out-of-the-Box integration of Istio&amp;rsquo;s egress traffic control with Istio&amp;rsquo;s policy and observability adapters.&lt;/li>
&lt;li>Write the adapters to use external monitoring or access control systems with Istio only once and
apply them for all types of traffic: ingress, egress, and in-cluster.&lt;/li>
&lt;li>Use Istio&amp;rsquo;s &lt;a href="/v1.12/docs/concepts/traffic-management/">traffic management features&lt;/a> for egress traffic:
load balancing, passive and active health checking, circuit breaker, timeouts, retries, fault injection, and others.&lt;/li>
&lt;/ul>
&lt;p>We refer to a system with the advantages above as &lt;strong>Istio-aware&lt;/strong>.&lt;/p>
&lt;p>The following table summarizes the egress traffic control features that Istio and the alternative solutions provide:&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>&lt;/th>
&lt;th>Istio Egress Traffic Control&lt;/th>
&lt;th>Kubernetes Network Policies&lt;/th>
&lt;th>Legacy Egress Proxy or Firewall&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>DNS-aware&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#checkmark"/>&lt;/svg>&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#cancel"/>&lt;/svg>&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#checkmark"/>&lt;/svg>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Kubernetes-aware&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#checkmark"/>&lt;/svg>&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#checkmark"/>&lt;/svg>&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#cancel"/>&lt;/svg>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Transparent&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#checkmark"/>&lt;/svg>&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#checkmark"/>&lt;/svg>&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#cancel"/>&lt;/svg>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Istio-aware&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#checkmark"/>&lt;/svg>&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#cancel"/>&lt;/svg>&lt;/td>
&lt;td>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#cancel"/>&lt;/svg>&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;h2 id="performance-considerations">Performance considerations&lt;/h2>
&lt;p>Controlling egress traffic using Istio has a price: increased latency of calls to external services and
increased CPU usage by the cluster&amp;rsquo;s pods.
Traffic passes through two proxies:&lt;/p>
&lt;ul>
&lt;li>The application&amp;rsquo;s sidecar proxy&lt;/li>
&lt;li>The egress gateway&amp;rsquo;s proxy&lt;/li>
&lt;/ul>
&lt;p>If you use &lt;a href="/v1.12/docs/tasks/traffic-management/egress/wildcard-egress-hosts/">TLS egress traffic to wildcard domains&lt;/a>,
you must add
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/wildcard-egress-hosts/#wildcard-configuration-for-arbitrary-domains">an additional proxy&lt;/a>
between the application and the external service. Since the traffic between the egress gateway&amp;rsquo;s proxy and
the proxy needed for the configuration of arbitrary domains using wildcards is on the pod&amp;rsquo;s local
network, that traffic shouldn&amp;rsquo;t have a significant impact on latency.&lt;/p>
&lt;p>See a &lt;a href="/v1.12/blog/2019/egress-performance/">performance evaluation&lt;/a> of different Istio configurations set to control egress
traffic. I would encourage you to carefully measure different configurations with your own applications and your own
external services, before you decide whether you can afford the performance overhead for your use cases. You should weigh the
required level of security versus your performance requirements and compare the performance overhead of all
alternative solutions.&lt;/p>
&lt;p>Let me share my thoughts on the performance overhead that controlling egress traffic using Istio adds:
Accessing external services already could have high latency and the overhead added
because of two or three proxies inside the cluster could likely not be very significant by comparison.
After all, applications with a microservice architecture can have chains of dozens of calls between microservices.
Therefore, an additional hop with one or two proxies in the egress gateway should not have a large impact.&lt;/p>
&lt;p>Moreover, we continue to work towards reducing Istio&amp;rsquo;s performance overhead.
Possible optimizations include:&lt;/p>
&lt;ul>
&lt;li>Extending Envoy to handle wildcard domains: This would eliminate the need for a third proxy between
the application and the external services for that use case.&lt;/li>
&lt;li>Using mutual TLS for authentication only without encrypting the TLS traffic, since the traffic is already
encrypted.&lt;/li>
&lt;/ul>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>I hope that after reading this series you are convinced that controlling egress traffic is very important for the
security of your cluster.
Hopefully, I also managed to convince you that Istio is an effective tool to control egress traffic
securely, and that Istio has multiple advantages over the alternative solutions.
Istio is the only solution I&amp;rsquo;m aware of that lets you:&lt;/p>
&lt;ul>
&lt;li>Control egress traffic in a secure and transparent way&lt;/li>
&lt;li>Specify external services as domain names&lt;/li>
&lt;li>Use Kubernetes artifacts to specify the traffic source&lt;/li>
&lt;/ul>
&lt;p>In my opinion, secure control of egress traffic is a great choice if you are looking for your first Istio use case.
In this case, Istio already provides you some benefits even before you start using all other Istio features:
&lt;a href="/v1.12/docs/tasks/traffic-management/">traffic management&lt;/a>, &lt;a href="/v1.12/docs/tasks/security/">security&lt;/a>,
&lt;a href="https://istio.io/v1.6/docs/tasks/policy-enforcement/">policies&lt;/a> and &lt;a href="/v1.12/docs/tasks/observability/">observability&lt;/a>, applied to traffic between
microservices inside the cluster.&lt;/p>
&lt;p>So, if you haven&amp;rsquo;t had the chance to work with Istio yet, &lt;a href="/v1.12/docs/setup/install/">install Istio&lt;/a> on your cluster
and check our &lt;a href="/v1.12/docs/tasks/traffic-management/egress/">egress traffic control tasks&lt;/a> and the tasks for the other
&lt;a href="/v1.12/docs/tasks/">Istio features&lt;/a>. We also want to hear from you, please join us at &lt;a href="https://discuss.istio.io">discuss.istio.io&lt;/a>.&lt;/p></description><pubDate>Mon, 22 Jul 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/egress-traffic-control-in-istio-part-3/</link><author>Vadim Eisenberg (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/egress-traffic-control-in-istio-part-3/</guid><category>traffic-management</category><category>egress</category><category>security</category><category>gateway</category><category>tls</category></item><item><title>Secure Control of Egress Traffic in Istio, part 2</title><description>
&lt;p>Welcome to part 2 in our new series about secure control of egress traffic in Istio.
In &lt;a href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-1/">the first part in the series&lt;/a>, I presented the attacks involving
egress traffic and the requirements we collected for a secure control system for egress traffic.
In this installment, I describe the Istio way to securely control the egress traffic, and show how Istio can help you
prevent the attacks.&lt;/p>
&lt;h2 id="secure-control-of-egress-traffic-in-istio">Secure control of egress traffic in Istio&lt;/h2>
&lt;p>To implement secure control of egress traffic in Istio, you must
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#egress-gateway-for-https-traffic">direct TLS traffic to external services through an egress gateway&lt;/a>.
Alternatively, you
can &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#egress-gateway-for-http-traffic">direct HTTP traffic through an egress gateway&lt;/a>
and &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/#perform-tls-origination-with-an-egress-gateway">let the egress gateway perform TLS origination&lt;/a>.&lt;/p>
&lt;p>Both alternatives have their pros and cons, you should choose between them according to your circumstances.
The choice mainly depends on whether your application can send unencrypted HTTP requests and whether your
organization&amp;rsquo;s security policies allow sending unencrypted HTTP requests.
For example, if your application uses some client library that encrypts the traffic without a possibility to cancel the
encryption, you cannot use the option of sending unencrypted HTTP traffic.
The same in the case your organization&amp;rsquo;s security policies do not allow sending unencrypted HTTP requests
&lt;strong>inside the pod&lt;/strong> (outside the pod the traffic is encrypted by Istio).&lt;/p>
&lt;p>If the application sends HTTP requests and the egress gateway performs TLS origination, you can monitor HTTP
information like HTTP methods, headers, and URL paths. You can also
&lt;a href="/v1.12/blog/2018/egress-monitoring-access-control">define policies&lt;/a> based on said HTTP information. If the application
performs TLS origination, you can
&lt;a href="https://istio.io/v1.6/docs/tasks/traffic-management/egress/egress_sni_monitoring_and_policies/">monitor SNI and the service account&lt;/a> of the
source pod&amp;rsquo;s TLS traffic, and define policies based on SNI and service accounts.&lt;/p>
&lt;p>You must ensure that traffic from your cluster to the outside cannot bypass the egress gateway. Istio cannot enforce it
for you, so you must apply some
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#additional-security-considerations">additional security mechanisms&lt;/a>,
for example,
the &lt;a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/">Kubernetes network policies&lt;/a> or an L3
firewall. See an example of the
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#apply-kubernetes-network-policies">Kubernetes network policies configuration&lt;/a>.
According to the &lt;a href="https://en.wikipedia.org/wiki/Defense_in_depth_(computing)">Defense in depth&lt;/a> concept, the more
security mechanisms you apply for the same goal, the better.&lt;/p>
&lt;p>You must also ensure that Istio control plane and the egress gateway cannot be compromised. While you may have hundreds
or thousands of application pods in your cluster, there are only a dozen of Istio control plane pods and the gateways.
You can and should focus on protecting the control plane pods and the gateways, since it is easy (there is a small
number of pods to protect) and it is most crucial for the security of your cluster.
If attackers compromise the control plane or the egress gateway, they could violate any policy.&lt;/p>
&lt;p>You might have multiple tools to protect the control plane pods, depending on your environment.
The reasonable security measures are:&lt;/p>
&lt;ul>
&lt;li>Run the control plane pods on nodes separate from the application nodes.&lt;/li>
&lt;li>Run the control plane pods in their own separate namespace.&lt;/li>
&lt;li>Apply the Kubernetes RBAC and network policies to protect the control plane pods.&lt;/li>
&lt;li>Monitor the control plane pods more closely than you do the application pods.&lt;/li>
&lt;/ul>
&lt;p>Once you direct egress traffic through an egress gateway and apply the additional security mechanisms,
you can securely monitor and enforce security policies for the traffic.&lt;/p>
&lt;p>The following diagram shows Istio&amp;rsquo;s security architecture, augmented with an L3 firewall which is part of the
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#additional-security-considerations">additional security mechanisms&lt;/a>
that should be provided outside of Istio.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:54.89557965057057%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-2/SecurityArchitectureWithL3Firewalls.svg" title="Istio Security Architecture with Egress Gateway and L3 Firewall">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-traffic-control-in-istio-part-2/SecurityArchitectureWithL3Firewalls.svg" alt="Istio Security Architecture with Egress Gateway and L3 Firewall" />
&lt;/a>
&lt;/div>
&lt;figcaption>Istio Security Architecture with Egress Gateway and L3 Firewall&lt;/figcaption>
&lt;/figure>
&lt;p>You can configure the L3 firewall trivially to only allow incoming traffic through the Istio ingress gateway and
only allow outgoing traffic through the Istio egress gateway. The Istio proxies of the gateways enforce
policies and report telemetry just as all other proxies in the mesh do.&lt;/p>
&lt;p>Now let&amp;rsquo;s examine possible attacks and let me show you how the secure control of egress traffic in Istio prevents them.&lt;/p>
&lt;h2 id="preventing-possible-attacks">Preventing possible attacks&lt;/h2>
&lt;p>Consider the following security policies for egress traffic:&lt;/p>
&lt;ul>
&lt;li>Application &lt;strong>A&lt;/strong> is allowed to access &lt;code>*.ibm.com&lt;/code>, which includes all the external services with URLs matching
&lt;code>*.ibm.com&lt;/code>.&lt;/li>
&lt;li>Application &lt;strong>B&lt;/strong> is allowed to access &lt;code>mongo1.composedb.com&lt;/code>.&lt;/li>
&lt;li>All egress traffic is monitored.&lt;/li>
&lt;/ul>
&lt;p>Suppose the attackers have the following goals:&lt;/p>
&lt;ul>
&lt;li>Access &lt;code>*.ibm.com&lt;/code> from your cluster.&lt;/li>
&lt;li>Access &lt;code>*.ibm.com&lt;/code> from your cluster, unmonitored. The attackers want their traffic to be unmonitored to prevent a
possibility that you will detect the forbidden access.&lt;/li>
&lt;li>Access &lt;code>mongo1.composedb.com&lt;/code> from your cluster.&lt;/li>
&lt;/ul>
&lt;p>Now suppose that the attackers manage to break into one of the pods of application &lt;strong>A&lt;/strong>, and try to use the compromised
pod to perform the forbidden access. The attackers may try their luck and access the external services in a
straightforward way. You will react to the straightforward attempts as follows:&lt;/p>
&lt;ul>
&lt;li>Initially, there is no way to prevent a compromised application &lt;strong>A&lt;/strong> to access &lt;code>*.ibm.com&lt;/code>, because the compromised
pod is indistinguishable from the original pod.&lt;/li>
&lt;li>Fortunately, you can monitor all access to external services, detect suspicious traffic, and thwart attackers from
gaining unmonitored access to &lt;code>*.ibm.com&lt;/code>. For example, you could apply anomaly detection tools on the
egress traffic logs.&lt;/li>
&lt;li>To stop attackers from accessing &lt;code>mongo1.composedb.com&lt;/code> from your cluster, Istio will correctly detect the source of
the traffic, application &lt;strong>A&lt;/strong> in this case, and verify that it is not allowed to access &lt;code>mongo1.composedb.com&lt;/code>
according to the security policies mentioned above.&lt;/li>
&lt;/ul>
&lt;p>Having failed to achieve their goals in a straightforward way, the malicious actors may resort to advanced attacks:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>Bypass the container&amp;rsquo;s sidecar proxy&lt;/strong> to be able to access any external service directly, without the sidecar&amp;rsquo;s
policy enforcement and reporting. This attack is prevented by a Kubernetes Network Policy or by an L3 firewall that
allow egress traffic to exit the mesh only from the egress gateway.&lt;/li>
&lt;li>&lt;strong>Compromise the egress gateway&lt;/strong> to be able to force it to send fake information to the monitoring system or to
disable enforcement of the security policies. This attack is prevented by applying the special security measures to
the egress gateway pods.&lt;/li>
&lt;li>&lt;strong>Impersonate as application B&lt;/strong> since application &lt;strong>B&lt;/strong> is allowed to access &lt;code>mongo1.composedb.com&lt;/code>. This attack,
fortunately, is prevented by Istio&amp;rsquo;s &lt;a href="/v1.12/docs/concepts/security/#istio-identity">strong identity support&lt;/a>.&lt;/li>
&lt;/ul>
&lt;p>As far as we can see, all the forbidden access is prevented, or at least is monitored and can be prevented later.
If you see other attacks that involve egress traffic or security holes in the current design, we would be happy
&lt;a href="https://discuss.istio.io">to hear about it&lt;/a>.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>Hopefully, I managed to convince you that Istio is an effective tool to prevent attacks involving egress
traffic. In &lt;a href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-3/">the next part of this series&lt;/a>, I compare secure control of egress traffic in Istio with alternative
solutions such as
&lt;a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/">Kubernetes Network Policies&lt;/a> and legacy
egress proxies/firewalls.&lt;/p></description><pubDate>Wed, 10 Jul 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/egress-traffic-control-in-istio-part-2/</link><author>Vadim Eisenberg (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/egress-traffic-control-in-istio-part-2/</guid><category>traffic-management</category><category>egress</category><category>security</category><category>gateway</category><category>tls</category></item><item><title>Best Practices: Benchmarking Service Mesh Performance</title><description>
&lt;p>Service meshes add a lot of functionality to application deployments, including &lt;a href="/v1.12/docs/concepts/traffic-management">traffic policies&lt;/a>, &lt;a href="/v1.12/docs/concepts/observability">observability&lt;/a>, and &lt;a href="/v1.12/docs/concepts/security">secure communication&lt;/a>. But adding a service mesh to your environment comes at a cost, whether that&amp;rsquo;s time (added latency) or resources (CPU cycles). To make an informed decision on whether a service mesh is right for your use case, it&amp;rsquo;s important to evaluate how your application performs when deployed with a service mesh.&lt;/p>
&lt;p>Earlier this year, we published a &lt;a href="/v1.12/blog/2019/istio1.1_perf/">blog post&lt;/a> on Istio&amp;rsquo;s performance improvements in version 1.1. Following the release of &lt;a href="/v1.12/news/releases/1.2.x/announcing-1.2/">Istio 1.2&lt;/a>, we want to provide guidance and tools to help you benchmark Istio&amp;rsquo;s data plane performance in a production-ready Kubernetes environment.&lt;/p>
&lt;p>Overall, we found that Istio&amp;rsquo;s &lt;a href="/v1.12/docs/ops/deployment/architecture/#envoy">sidecar proxy&lt;/a> latency scales with the number of concurrent connections. At 1000 requests per second (RPS), across 16 connections, Istio adds &lt;strong>3 milliseconds&lt;/strong> per request in the 50th percentile, and &lt;strong>10 milliseconds&lt;/strong> in the 99th percentile.&lt;/p>
&lt;p>In the &lt;a href="https://github.com/istio/tools/tree/3ac7ab40db8a0d595b71f47b8ba246763ecd6213/perf/benchmark">Istio Tools repository&lt;/a>, you’ll find scripts and instructions for measuring Istio&amp;rsquo;s data plane performance, with additional instructions on how to run the scripts with &lt;a href="https://linkerd.io">Linkerd&lt;/a>, another service mesh implementation. &lt;a href="https://github.com/istio/tools/tree/3ac7ab40db8a0d595b71f47b8ba246763ecd6213/perf/benchmark#setup">Follow along&lt;/a> as we detail some best practices for each step of the performance test framework.&lt;/p>
&lt;h2 id="1-use-a-production-ready-istio-installation">1. Use a production-ready Istio installation&lt;/h2>
&lt;p>To accurately measure the performance of a service mesh at scale, it&amp;rsquo;s important to use an &lt;a href="https://github.com/istio/tools/tree/3ac7ab40db8a0d595b71f47b8ba246763ecd6213/perf/istio-install#istio-setup">adequately-sized&lt;/a> Kubernetes cluster. We test using three worker nodes, each with at least 4 vCPUs and 15 GB of memory.&lt;/p>
&lt;p>Then, it&amp;rsquo;s important to use a production-ready Istio &lt;strong>installation profile&lt;/strong> on that cluster. This lets us achieve performance-oriented settings such as control plane pod autoscaling, and ensures that resource limits are appropriate for heavy traffic load. The &lt;a href="https://archive.istio.io/1.4/docs/setup/install/helm/#installation-steps">default&lt;/a> Istio installation is suitable for most benchmarking use cases. For extensive performance benchmarking, with thousands of proxy-injected services, we also provide &lt;a href="https://github.com/istio/tools/blob/3ac7ab40db8a0d595b71f47b8ba246763ecd6213/perf/istio-install/values.yaml">a tuned Istio install&lt;/a> that allocates extra memory and CPU to the Istio control plane.&lt;/p>
&lt;p>&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#exclamation-mark"/>&lt;/svg> Istio&amp;rsquo;s &lt;a href="/v1.12/docs/setup/getting-started/">demo installation&lt;/a> is not suitable for performance testing, because it is designed to be deployed on a small trial cluster, and has full tracing and access logs enabled to showcase Istio&amp;rsquo;s features.&lt;/p>
&lt;h2 id="2-focus-on-the-data-plane">2. Focus on the data plane&lt;/h2>
&lt;p>Our benchmarking scripts focus on evaluating the Istio data plane: the &lt;span class="term" data-title="Envoy" data-body="&amp;lt;p&amp;gt;The high-performance proxy that Istio uses to mediate inbound and outbound traffic for all &amp;lt;a href=&amp;#34;/docs/reference/glossary/#service&amp;#34;&amp;gt;services&amp;lt;/a&amp;gt; in the
&amp;lt;a href=&amp;#34;/docs/reference/glossary/#service-mesh&amp;#34;&amp;gt;service mesh&amp;lt;/a&amp;gt;. &amp;lt;a href=&amp;#34;https://envoyproxy.github.io/envoy/&amp;#34;&amp;gt;Learn more about Envoy&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">Envoy&lt;/span> proxies that mediate traffic between application containers. Why focus on the data plane? Because at scale, with lots of application containers, the data plane’s &lt;strong>memory&lt;/strong> and &lt;strong>CPU&lt;/strong> usage quickly eclipses that of the Istio control plane. Let&amp;rsquo;s look at an example of how this happens:&lt;/p>
&lt;p>Say you run 2,000 Envoy-injected pods, each handling 1,000 requests per second. Each proxy is using 50 MB of memory, and to configure all these proxies, Pilot is using 1 vCPU and 1.5 GB of memory. All together, the Istio data plane (the sum of all the Envoy proxies) is using 100 GB of memory, compared to Pilot&amp;rsquo;s 1.5 GB.&lt;/p>
&lt;p>It is also important to focus on data plane performance for &lt;strong>latency&lt;/strong> reasons. This is because most application requests move through the Istio data plane, not the control plane. There are two exceptions:&lt;/p>
&lt;ol>
&lt;li>&lt;strong>Telemetry reporting:&lt;/strong> Each proxy sends raw telemetry data to Mixer, which Mixer processes into metrics, traces, and other telemetry. The raw telemetry data is similar to access logs, and therefore comes at a cost. Access log processing consumes CPU and keeps a worker thread from picking up the next unit of work. At higher throughput, it is more likely that the next unit of work is waiting in the queue to be picked up by the worker. This can lead to long-tail (99th percentile) latency for Envoy.&lt;/li>
&lt;li>&lt;strong>Custom policy checks:&lt;/strong> When using &lt;a href="/v1.12/docs/concepts/observability/">custom Istio policy adapters&lt;/a>, policy checks are on the request path. This means that request headers and metadata on the data path will be sent to the control plane (Mixer), resulting in higher request latency. &lt;strong>Note:&lt;/strong> These policy checks are &lt;a href="https://archive.istio.io/v1.4/docs/reference/config/installation-options/">disabled by default&lt;/a>, as the most common policy use case (&lt;a href="https://archive.istio.io/v1.4/docs/reference/config/security/istio.rbac.v1alpha1">RBAC&lt;/a>) is performed entirely by the Envoy proxies.&lt;/li>
&lt;/ol>
&lt;p>Both of these exceptions will go away in a future Istio release, when &lt;a href="https://docs.google.com/document/d/1QKmtem5jU_2F3Lh5SqLp0IuPb80_70J7aJEYu4_gS-s">Mixer V2&lt;/a> moves all policy and telemetry features directly into the proxies.&lt;/p>
&lt;p>Next, when testing Istio&amp;rsquo;s data plane performance at scale, it&amp;rsquo;s important to test not only at increasing requests per second, but also against an increasing number of &lt;strong>concurrent&lt;/strong> connections. This is because real-world, high-throughput traffic comes from multiple clients. The &lt;a href="https://github.com/istio/tools/tree/3ac7ab40db8a0d595b71f47b8ba246763ecd6213/perf/benchmark#run-performance-tests">provided scripts&lt;/a> allow you to perform the same load test with any number of concurrent connections, at increasing RPS.&lt;/p>
&lt;p>Lastly, our test environment measures requests between two pods, not many. The client pod is &lt;a href="http://fortio.org/">Fortio&lt;/a>, which sends traffic to the server pod.&lt;/p>
&lt;p>Why test with only two pods? Because scaling up throughput (RPS) and connections (threads) has a greater effect on Envoy&amp;rsquo;s performance than increasing the total size of the service registry — or, the total number of pods and services in the Kubernetes cluster. When the size of the service registry grows, Envoy does have to keep track of more endpoints, and lookup time per request does increase, but by a tiny constant. If you have many services, and this constant becomes a latency concern, Istio provides a &lt;a href="/v1.12/docs/reference/config/networking/sidecar/">Sidecar resource&lt;/a>, which allows you to limit which services each Envoy knows about.&lt;/p>
&lt;h2 id="3-measure-with-and-without-proxies">3. Measure with and without proxies&lt;/h2>
&lt;p>While many Istio features, such as &lt;a href="/v1.12/docs/concepts/security/#mutual-tls-authentication">mutual TLS authentication&lt;/a>, rely on an Envoy proxy next to an application pod, you can &lt;a href="https://archive.istio.io/1.4/docs/setup/additional-setup/sidecar-injection/#disabling-or-updating-the-webhook">selectively disable&lt;/a> sidecar proxy injection for some of your mesh services. As you scale up Istio for production, you may want to incrementally add the sidecar proxy to your workloads.&lt;/p>
&lt;p>To that end, the test scripts provide &lt;a href="https://github.com/istio/tools/tree/3ac7ab40db8a0d595b71f47b8ba246763ecd6213/perf/benchmark#run-performance-tests">three different modes&lt;/a>. These modes analyze Istio&amp;rsquo;s performance when a request goes through both the client and server proxies (&lt;code>both&lt;/code>), just the server proxy (&lt;code>serveronly&lt;/code>), and neither proxy (&lt;code>baseline&lt;/code>).&lt;/p>
&lt;p>You can also disable &lt;a href="/v1.12/docs/concepts/observability/">Mixer&lt;/a> to stop Istio&amp;rsquo;s telemetry during the performance tests, which provides results in line with the performance we expect when the Mixer V2 work is completed. Istio also supports &lt;a href="https://github.com/istio/istio/wiki/Envoy-native-telemetry">Envoy native telemetry&lt;/a>, which performs similarly to having Istio&amp;rsquo;s telemetry disabled.&lt;/p>
&lt;h2 id="istio-1-2-performance">Istio 1.2 Performance&lt;/h2>
&lt;p>Let&amp;rsquo;s see how to use this test environment to analyze the data plane performance of Istio 1.2. We also provide instructions to run the &lt;a href="https://github.com/istio/tools/tree/3ac7ab40db8a0d595b71f47b8ba246763ecd6213/perf/benchmark/linkerd">same performance tests for the Linkerd data plane&lt;/a>. Currently, only latency benchmarking is supported for Linkerd.&lt;/p>
&lt;p>For measuring Istio&amp;rsquo;s sidecar proxy latency, we look at the 50th, 90th, and 99th percentiles for an increasing number of concurrent connections,keeping request throughput (RPS) constant.&lt;/p>
&lt;p>We found that with 16 concurrent connections and 1000 RPS, Istio adds &lt;strong>3ms&lt;/strong> over the baseline (P50) when a request travels through both a client and server proxy. (Subtract the pink line, &lt;code>base&lt;/code>, from the green line, &lt;code>both&lt;/code>.) At 64 concurrent connections, Istio adds &lt;strong>12ms&lt;/strong> over the baseline, but with Mixer disabled (&lt;code>nomixer_both&lt;/code>), Istio only adds &lt;strong>7ms&lt;/strong>.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:60%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/performance-best-practices/latency_p50.png" title="Istio sidecar proxy, 50th percentile latency">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/performance-best-practices/latency_p50.png" alt="Istio sidecar proxy, 50th percentile latency" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
&lt;p>In the 90th percentile, with 16 concurrent connections, Istio adds &lt;strong>6ms&lt;/strong>; with 64 connections, Istio adds &lt;strong>20ms&lt;/strong>.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:60%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/performance-best-practices/latency_p90.png" title="Istio sidecar proxy, 90th percentile latency">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/performance-best-practices/latency_p90.png" alt="Istio sidecar proxy, 90th percentile latency" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
&lt;p>Finally, in the 99th percentile, with 16 connections, Istio adds &lt;strong>10ms&lt;/strong> over the baseline. At 64 connections, Istio adds &lt;strong>25ms&lt;/strong> with Mixer, or &lt;strong>10ms&lt;/strong> without Mixer.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:60%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/performance-best-practices/latency_p99.png" title="Istio sidecar proxy, 99th percentile latency">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/performance-best-practices/latency_p99.png" alt="Istio sidecar proxy, 99th percentile latency" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
&lt;p>For CPU usage, we measured with an increasing request throughput (RPS), and a constant number of concurrent connections. We found that Envoy&amp;rsquo;s maximum CPU usage at 3000 RPS, with Mixer enabled, was &lt;strong>1.2 vCPUs&lt;/strong>. At 1000 RPS, one Envoy uses approximately half of a CPU.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:60%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/performance-best-practices/cpu_max.png" title="Istio sidecar proxy, max CPU usage">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/performance-best-practices/cpu_max.png" alt="Istio sidecar proxy, max CPU usage" />
&lt;/a>
&lt;/div>
&lt;figcaption>&lt;/figcaption>
&lt;/figure>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>In the process of benchmarking Istio&amp;rsquo;s performance, we learned several key lessons:&lt;/p>
&lt;ul>
&lt;li>Use an environment that mimics production.&lt;/li>
&lt;li>Focus on data plane traffic.&lt;/li>
&lt;li>Measure against a baseline.&lt;/li>
&lt;li>Increase concurrent connections as well as total throughput.&lt;/li>
&lt;/ul>
&lt;p>For a mesh with 1000 RPS across 16 connections, Istio 1.2 adds just &lt;strong>3 milliseconds&lt;/strong> of latency over the baseline, in the 50th percentile.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Istio&amp;rsquo;s performance depends on your specific setup and traffic load. Because of this variance, make sure your test setup accurately reflects your production workloads. To try out the benchmarking scripts, head over &lt;a href="https://github.com/istio/tools/tree/3ac7ab40db8a0d595b71f47b8ba246763ecd6213/perf/benchmark">to the Istio Tools repository&lt;/a>.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Also check out the &lt;a href="/v1.12/docs/ops/deployment/performance-and-scalability">Istio Performance and Scalability guide&lt;/a> for the most up-to-date performance data.&lt;/p>
&lt;p>Thank you for reading, and happy benchmarking!&lt;/p></description><pubDate>Tue, 09 Jul 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/performance-best-practices/</link><author>Megan O'Keefe (Google), John Howard (Google), Mandar Jog (Google)</author><guid isPermaLink="true">/v1.12/blog/2019/performance-best-practices/</guid><category>performance</category><category>scalability</category><category>scale</category><category>benchmarks</category></item><item><title>Extending Istio Self-Signed Root Certificate Lifetime</title><description>&lt;p>Istio self-signed certificates have historically had a 1 year default lifetime.
If you are using Istio self-signed certificates,
you need to schedule regular root transitions before they expire.
An expiration of a root certificate may lead to an unexpected cluster-wide outage.
The issue affects new clusters created with versions up to 1.0.7 and 1.1.7.&lt;/p>
&lt;p>See &lt;a href="https://istio.io/v1.7/docs/ops/configuration/security/root-transition/">Extending Self-Signed Certificate Lifetime&lt;/a> for
information on how to gauge the age of your certificates and how to perform rotation.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">We strongly recommend you rotate root keys and root certificates annually as a security best practice.
We will send out instructions for root key/cert rotation soon.&lt;/div>
&lt;/aside>
&lt;/div></description><pubDate>Fri, 07 Jun 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/root-transition/</link><author>Oliver Liu</author><guid isPermaLink="true">/v1.12/blog/2019/root-transition/</guid><category>security</category><category>PKI</category><category>certificate</category><category>Citadel</category></item><item><title>Secure Control of Egress Traffic in Istio, part 1</title><description>
&lt;p>This is part 1 in a new series about secure control of egress traffic in Istio that I am going to publish.
In this installment, I explain why you should apply egress traffic control to your cluster, the attacks
involving egress traffic you want to prevent, and the requirements for a system for egress traffic control
to do so.
Once you agree that you should control the egress traffic coming from your cluster, the following questions arise:
What is required from a system for secure control of egress traffic? Which is the best solution to fulfill
these requirements? (spoiler: Istio in my opinion)
Future installments will describe
&lt;a href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-2/">the implementation of the secure control of egress traffic in Istio&lt;/a>
and compare it with other solutions.&lt;/p>
&lt;p>The most important security aspect for a service mesh is probably ingress traffic. You definitely must prevent attackers
from penetrating the cluster through ingress APIs. Having said that, securing
the traffic leaving the mesh is also very important. Once your cluster is compromised, and you must be
prepared for that scenario, you want to reduce the damage as much as possible and prevent the attackers from using the
cluster for further attacks on external services and legacy systems outside of the cluster. To achieve that goal,
you need secure control of egress traffic.&lt;/p>
&lt;p>Compliance requirements are another reason to implement secure control of egress traffic. For example, the &lt;a href="https://www.pcisecuritystandards.org/pci_security/">Payment Card
Industry (PCI) Data Security Standard&lt;/a> requires that inbound
and outbound traffic must be restricted to that which is necessary:&lt;/p>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">&lt;em>1.2.1 Restrict inbound and outbound traffic to that which is necessary for the cardholder data environment, and specifically deny all other traffic.&lt;/em>&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>And specifically regarding outbound traffic:&lt;/p>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">&lt;em>1.3.4 Do not allow unauthorized outbound traffic from the cardholder data environment to the Internet&amp;hellip; All traffic outbound from the cardholder data environment should be evaluated to ensure that it follows established, authorized rules. Connections should be inspected to restrict traffic to only authorized communications (for example by restricting source/destination addresses/ports, and/or blocking of content).&lt;/em>&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Let&amp;rsquo;s start with the attacks that involve egress traffic.&lt;/p>
&lt;h2 id="the-attacks">The attacks&lt;/h2>
&lt;p>An IT organization must assume it will be attacked if it hasn&amp;rsquo;t been attacked already, and that
part of its infrastructure could already be compromised or become compromised in the future.
Once attackers are able to penetrate an application in a cluster, they can proceed to attack external services:
legacy systems, external web services and databases. The attackers may want to steal the data of the application and to
transfer it to their external servers. Attackers&amp;rsquo; malware may require access to attackers&amp;rsquo; servers to download
updates. The attackers may use pods in the cluster to perform DDOS attacks or to break into external systems.
Even though you &lt;a href="https://en.wikipedia.org/wiki/There_are_known_knowns">cannot know&lt;/a> all the possible types of
attacks, you want to reduce possibilities for any attacks, both for known and unknown ones.&lt;/p>
&lt;p>The external attackers gain access to the application’s container from outside the mesh through a
bug in the application but attackers can also be internal, for example, malicious DevOps people inside the
organization.&lt;/p>
&lt;p>To prevent the attacks described above, some form of egress traffic control must be applied. Let me present egress
traffic control in the following section.&lt;/p>
&lt;h2 id="the-solution-secure-control-of-egress-traffic">The solution: secure control of egress traffic&lt;/h2>
&lt;p>Secure control of egress traffic means monitoring the egress traffic and enforcing all the security policies regarding
the egress traffic.
Monitoring the egress traffic, enables you to analyze it, possibly offline, and detect the attacks even if
you were unable to prevent them in real time.
Another good practice to reduce possibilities of attacks is to specify policies that limit access following the
&lt;a href="https://en.wikipedia.org/wiki/Need_to_know#In_computer_technology]">Need to know&lt;/a> principle: only the applications that
need external services should be allowed to access the external services they need.&lt;/p>
&lt;p>Let me now turn to the requirements for egress traffic control we collected.&lt;/p>
&lt;h2 id="requirements-for-egress-traffic-control">Requirements for egress traffic control&lt;/h2>
&lt;p>My colleagues at IBM and I collected requirements for secure control of egress traffic from several customers, and
combined them with the
&lt;a href="https://docs.google.com/document/d/1-Cq_Y-yuyNklvdnaZF9Qngl3xe0NnArT7Xt_Wno9beg">egress traffic control requirements from Kubernetes Network Special Interest Group&lt;/a>.&lt;/p>
&lt;p>Istio 1.1 satisfies all gathered requirements:&lt;/p>
&lt;ol>
&lt;li>&lt;p>Support for &lt;a href="https://en.wikipedia.org/wiki/Transport_Layer_Security">TLS&lt;/a> with
&lt;a href="https://en.wikipedia.org/wiki/Server_Name_Indication">SNI&lt;/a> or for &lt;a href="/v1.12/docs/reference/glossary/#tls-origination">TLS origination&lt;/a> by Istio.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Monitor&lt;/strong> SNI and the source workload of every egress access.&lt;/p>&lt;/li>
&lt;li>&lt;p>Define and enforce &lt;strong>policies per cluster&lt;/strong>, e.g.:&lt;/p>
&lt;ul>
&lt;li>&lt;p>all applications in the cluster may access &lt;code>service1.foo.com&lt;/code> (a specific host)&lt;/p>&lt;/li>
&lt;li>&lt;p>all applications in the cluster may access any host of the form &lt;code>*.bar.com&lt;/code> (a wildcarded domain)&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>All unspecified access must be blocked.&lt;/p>&lt;/li>
&lt;li>&lt;p>Define and enforce &lt;strong>policies per source&lt;/strong>, &lt;em>Kubernetes-aware&lt;/em>:&lt;/p>
&lt;ul>
&lt;li>&lt;p>application &lt;code>A&lt;/code> may access &lt;code>*.foo.com&lt;/code>.&lt;/p>&lt;/li>
&lt;li>&lt;p>application &lt;code>B&lt;/code> may access &lt;code>*.bar.com&lt;/code>.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>All other access must be blocked, in particular access of application &lt;code>A&lt;/code> to &lt;code>service1.bar.com&lt;/code>.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Prevent tampering&lt;/strong>. In case an application pod is compromised, prevent the compromised pod from escaping
monitoring, from sending fake information to the monitoring system, and from breaking the egress policies.&lt;/p>&lt;/li>
&lt;li>&lt;p>Nice to have: traffic control is &lt;strong>transparent&lt;/strong> to the applications.&lt;/p>&lt;/li>
&lt;/ol>
&lt;p>Let me explain each requirement in more detail. The first requirement states that only TLS traffic to the external
services must be supported.
The requirement emerged upon observation that all the traffic that leaves the cluster must be encrypted.
This means that either the applications perform TLS origination or Istio must perform TLS origination
for them.
Note that in the case an application performs TLS origination, the Istio proxies cannot see the original traffic,
only the encrypted one, so the proxies see the TLS protocol only. For the proxies it does not matter if the original
protocol is HTTP or MongoDB, all the Istio proxies can see is TLS traffic.&lt;/p>
&lt;p>The second requirement states that SNI and the source of the traffic must be monitored. Monitoring is the first step to
prevent attacks. Even if attackers would be able to access external services from the cluster, if the access is
monitored, there is a chance to discover the suspicious traffic and take a corrective action.&lt;/p>
&lt;p>Note that in the case of TLS originated by an application, the Istio sidecar proxies can only see TCP traffic and a
TLS handshake that includes SNI.
A label of the source pod could identify the source of the traffic but a service account of the pod or some
other source identifier could be used. We call this property of an egress control system as &lt;em>being Kubernetes-aware&lt;/em>:
the system must understand Kubernetes artifacts like pods and service accounts. If the system is not Kubernetes-aware,
it can only monitor the IP address as the identifier of the source.&lt;/p>
&lt;p>The third requirement states that Istio operators must be able to define policies for egress traffic for the entire
cluster.
The policies state which external services may be accessed by any pod in the cluster. The external services can be
identified either by a &lt;a href="https://en.wikipedia.org/wiki/Fully_qualified_domain_name">Fully qualified domain name&lt;/a> of the
service, e.g. &lt;code>www.ibm.com&lt;/code> or by a wildcarded domain, e.g. &lt;code>*.ibm.com&lt;/code>. Only the specified external services may be
accessed, all other egress traffic is blocked.&lt;/p>
&lt;p>This requirement originates from the need to prevent
attackers from accessing malicious sites, for example for downloading updates/instructions for their malware. You also
want to limit the number of external sites that the attackers can access and attack.
You want to allow access only to the external services that the applications in the cluster need to
access and to block access to all the other services, this way you reduce the
&lt;a href="https://en.wikipedia.org/wiki/Attack_surface">attack surface&lt;/a>. While the external services
can have their own security mechanisms, you want to exercise &lt;a href="https://en.wikipedia.org/wiki/Defense_in_depth_(computing)">Defense in depth&lt;/a> and to have multiple security layers: a security layer in your cluster in addition to
the security layers in the external systems.&lt;/p>
&lt;p>This requirement means that the external services must be identifiable by domain names. We call this property
of an egress control system as &lt;em>being DNS-aware&lt;/em>.
If the system is not DNS-aware, the external services must be specified by IP addresses.
Using IP addresses is not convenient and often is not feasible, since the IP addresses of a service can change. Sometimes
all the IP addresses of a service are not even known, for example in the case of
&lt;a href="https://en.wikipedia.org/wiki/Content_delivery_network">CDNs&lt;/a>.&lt;/p>
&lt;p>The fourth requirement states that the source of the egress traffic must be added to the policies effectively extending
the third requirement.
Policies can specify which source can access which external service and the source must be identified just as in the
second requirement, for example, by a label of the source pod or by service account of the pod.
It means that policy enforcement must also be &lt;em>Kubernetes-aware&lt;/em>.
If policy enforcement is not Kubernetes-aware, the policies must identify the source of traffic by
the IP of the pod, which is not convenient, especially since the pods can come and go so their IPs are not static.&lt;/p>
&lt;p>The fifth requirement states that even if the cluster is compromised and the attackers control some of the pods, they
must not be able to cheat the monitoring or to violate policies of the egress control system. We say that such a
system provides &lt;em>secure&lt;/em> control of egress traffic.&lt;/p>
&lt;p>The sixth requirement states that the traffic control should be provided without changing the application containers, in
particular without changing the code of the applications and without changing the environment of the containers.
We call such a control of egress traffic &lt;em>transparent&lt;/em>.&lt;/p>
&lt;p>In the next posts I will show that Istio can function as an example of an egress traffic control system that satisfies
all of these requirements, in particular it is transparent, DNS-aware, and Kubernetes-aware.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>I hope that you are convinced that controlling egress traffic is important for the security of your cluster. In &lt;a href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-2/">the
part 2 of this series&lt;/a> I describe the Istio way to perform secure
control of egress traffic. In
&lt;a href="/v1.12/blog/2019/egress-traffic-control-in-istio-part-3/">the
part 3 of this series&lt;/a> I compare it with alternative solutions such as
&lt;a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/">Kubernetes Network Policies&lt;/a> and legacy
egress proxies/firewalls.&lt;/p></description><pubDate>Wed, 22 May 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/egress-traffic-control-in-istio-part-1/</link><author>Vadim Eisenberg (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/egress-traffic-control-in-istio-part-1/</guid><category>traffic-management</category><category>egress</category><category>security</category></item><item><title>Architecting Istio 1.1 for Performance</title><description>
&lt;p>Hyper-scale, microservice-based cloud environments have been exciting to build but challenging to manage. Along came Kubernetes (container orchestration) in 2014, followed by Istio (container service management) in 2017. Both open-source projects enable developers to scale container-based applications without spending too much time on administration tasks.&lt;/p>
&lt;p>Now, new enhancements in Istio 1.1 deliver scale-up with improved application performance and service management efficiency.
Simulations using our sample commercial airline reservation application show the following improvements, compared to Istio 1.0.&lt;/p>
&lt;p>We&amp;rsquo;ve seen substantial application performance gains:&lt;/p>
&lt;ul>
&lt;li>up to 30% reduction in application average latency&lt;/li>
&lt;li>up to 40% faster service startup times in a large mesh&lt;/li>
&lt;/ul>
&lt;p>As well as impressive improvements in service management efficiency:&lt;/p>
&lt;ul>
&lt;li>up to 90% reduction in Pilot CPU usage in a large mesh&lt;/li>
&lt;li>up to 50% reduction in Pilot memory usage in a large mesh&lt;/li>
&lt;/ul>
&lt;p>With Istio 1.1, organizations can be more confident in their ability to scale applications with consistency and control &amp;ndash; even in hyper-scale cloud environments.&lt;/p>
&lt;p>Congratulations to the Istio experts around the world who contributed to this release. We could not be more pleased with these results.&lt;/p>
&lt;h2 id="istio-1-1-performance-enhancements">Istio 1.1 performance enhancements&lt;/h2>
&lt;p>As members of the Istio Performance and Scalability workgroup, we have done extensive performance evaluations. We introduced many performance design features for Istio 1.1, in collaboration with other Istio contributors.
Some of the most visible performance enhancements in 1.1 include:&lt;/p>
&lt;ul>
&lt;li>Significant reduction in default collection of Envoy-generated statistics&lt;/li>
&lt;li>Added load-shedding functionality to Mixer workloads&lt;/li>
&lt;li>Improved the protocol between Envoy and Mixer&lt;/li>
&lt;li>Namespace isolation, to reduce operational overhead&lt;/li>
&lt;li>Configurable concurrent worker threads, which can improve overall throughput&lt;/li>
&lt;li>Configurable filters that limit telemetry data&lt;/li>
&lt;li>Removal of synchronization bottlenecks&lt;/li>
&lt;/ul>
&lt;h2 id="continuous-code-quality-and-performance-verification">Continuous code quality and performance verification&lt;/h2>
&lt;p>Regression Patrol drives continuous improvement in Istio performance and quality. Behind the scenes, the Regression Patrol helps Istio developers to identify and fix code issues. Daily builds are checked using a customer-centric benchmark, &lt;a href="https://github.com/blueperf/">BluePerf&lt;/a>. The results are published to the &lt;a href="https://ibmcloud-perf.istio.io/regpatrol/">Istio community web portal&lt;/a>. Various application configurations are evaluated to help provide insights on Istio component performance.&lt;/p>
&lt;p>Another tool that is used to evaluate the performance of Istio’s builds is &lt;a href="https://fortio.org/">Fortio&lt;/a>, which provides a synthetic end to end load testing benchmark.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>Istio 1.1 was designed for performance and scalability. The Istio Performance and Scalability workgroup measured significant performance improvements over 1.0.
Istio 1.1 introduces new features and optimizations to help harden the service mesh for enterprise microservice workloads. The Istio 1.1 Performance and Tuning Guide documents performance simulations, provides sizing and capacity planning guidance, and includes best practices for tuning custom use cases.&lt;/p>
&lt;h2 id="useful-links">Useful links&lt;/h2>
&lt;ul>
&lt;li>&lt;a href="https://www.youtube.com/watch?time_continue=349&amp;amp;v=G4F5aRFEXnU">Istio Service Mesh Performance (34:30)&lt;/a>, by Surya Duggirala, Laurent Demailly and Fawad Khaliq at KubeCon Europe 2018&lt;/li>
&lt;li>&lt;a href="https://discuss.istio.io/c/performance-and-scalability">Istio Performance and Scalability discussion forum&lt;/a>&lt;/li>
&lt;/ul>
&lt;h2 id="disclaimer">Disclaimer&lt;/h2>
&lt;p>The performance data contained herein was obtained in a controlled, isolated environment. Actual results that may be obtained in other operating environments may vary significantly. There is no guarantee that the same or similar results will be obtained elsewhere.&lt;/p></description><pubDate>Tue, 19 Mar 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/istio1.1_perf/</link><author>Surya V Duggirala (IBM), Mandar Jog (Google), Jose Nativio (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/istio1.1_perf/</guid><category>performance</category><category>scalability</category><category>scale</category><category>benchmarks</category></item><item><title>Version Routing in a Multicluster Service Mesh</title><description>
&lt;p>If you&amp;rsquo;ve spent any time looking at Istio, you&amp;rsquo;ve probably noticed that it includes a lot of features that
can be demonstrated with simple &lt;a href="/v1.12/docs/tasks/">tasks&lt;/a> and &lt;a href="/v1.12/docs/examples/">examples&lt;/a>
running on a single Kubernetes cluster.
Because most, if not all, real-world cloud and microservices-based applications are not that simple
and will need to have the services distributed and running in more than one location, you may be
wondering if all these things will be just as simple in your real production environment.&lt;/p>
&lt;p>Fortunately, Istio provides several ways to configure a service mesh so that applications
can, more-or-less transparently, be part of a mesh where the services are running
in more than one cluster, i.e., in a
&lt;a href="/v1.12/docs/ops/deployment/deployment-models/#multiple-clusters">multicluster deployment&lt;/a>.
The simplest way to set up a multicluster mesh, because it has no special networking requirements,
is using a replicated
&lt;a href="/v1.12/docs/ops/deployment/deployment-models/#control-plane-models">control plane model&lt;/a>.
In this configuration, each Kubernetes cluster contributing to the mesh has its own control plane,
but each control plane is synchronized and running under a single administrative control.&lt;/p>
&lt;p>In this article we&amp;rsquo;ll look at how one of the features of Istio,
&lt;a href="/v1.12/docs/concepts/traffic-management/">traffic management&lt;/a>, works in a multicluster mesh with
a dedicated control plane topology.
We&amp;rsquo;ll show how to configure Istio route rules to call remote services in a multicluster service mesh
by deploying the &lt;a href="https://github.com/istio/istio/tree/release-1.12/samples/bookinfo">Bookinfo sample&lt;/a> with version &lt;code>v1&lt;/code> of the &lt;code>reviews&lt;/code> service
running in one cluster, versions &lt;code>v2&lt;/code> and &lt;code>v3&lt;/code> running in a second cluster.&lt;/p>
&lt;h2 id="set-up-clusters">Set up clusters&lt;/h2>
&lt;p>To start, you&amp;rsquo;ll need two Kubernetes clusters, both running a slightly customized configuration of Istio.&lt;/p>
&lt;ul>
&lt;li>&lt;p>Set up a multicluster environment with two Istio clusters by following the
&lt;a href="/v1.12/docs/setup/install/multicluster">replicated control planes&lt;/a> instructions.&lt;/p>&lt;/li>
&lt;li>&lt;p>The &lt;code>kubectl&lt;/code> command is used to access both clusters with the &lt;code>--context&lt;/code> flag.
Use the following command to list your contexts:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* cluster1 cluster1 user@foo.com default
cluster2 cluster2 user@foo.com default
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Export the following environment variables with the context names of your configuration:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export CTX_CLUSTER1=&amp;lt;cluster1 context name&amp;gt;
$ export CTX_CLUSTER2=&amp;lt;cluster2 context name&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ul>
&lt;h2 id="deploy-version-v1-of-the-bookinfo-application-in-cluster1">Deploy version v1 of the &lt;code>bookinfo&lt;/code> application in &lt;code>cluster1&lt;/code>&lt;/h2>
&lt;p>Run the &lt;code>productpage&lt;/code> and &lt;code>details&lt;/code> services and version &lt;code>v1&lt;/code> of the &lt;code>reviews&lt;/code> service in &lt;code>cluster1&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl label --context=$CTX_CLUSTER1 namespace default istio-injection=enabled
$ kubectl apply --context=$CTX_CLUSTER1 -f - &amp;lt;&amp;lt;EOF
apiVersion: v1
kind: Service
metadata:
name: productpage
labels:
app: productpage
spec:
ports:
- port: 9080
name: http
selector:
app: productpage
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: productpage-v1
spec:
replicas: 1
template:
metadata:
labels:
app: productpage
version: v1
spec:
containers:
- name: productpage
image: istio/examples-bookinfo-productpage-v1:1.10.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9080
---
apiVersion: v1
kind: Service
metadata:
name: details
labels:
app: details
spec:
ports:
- port: 9080
name: http
selector:
app: details
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: details-v1
spec:
replicas: 1
template:
metadata:
labels:
app: details
version: v1
spec:
containers:
- name: details
image: istio/examples-bookinfo-details-v1:1.10.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9080
---
apiVersion: v1
kind: Service
metadata:
name: reviews
labels:
app: reviews
spec:
ports:
- port: 9080
name: http
selector:
app: reviews
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: reviews-v1
spec:
replicas: 1
template:
metadata:
labels:
app: reviews
version: v1
spec:
containers:
- name: reviews
image: istio/examples-bookinfo-reviews-v1:1.10.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9080
EOF
&lt;/code>&lt;/pre>
&lt;h2 id="deploy-bookinfo-v2-and-v3-services-in-cluster2">Deploy &lt;code>bookinfo&lt;/code> v2 and v3 services in &lt;code>cluster2&lt;/code>&lt;/h2>
&lt;p>Run the &lt;code>ratings&lt;/code> service and version &lt;code>v2&lt;/code> and &lt;code>v3&lt;/code> of the &lt;code>reviews&lt;/code> service in &lt;code>cluster2&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl label --context=$CTX_CLUSTER2 namespace default istio-injection=enabled
$ kubectl apply --context=$CTX_CLUSTER2 -f - &amp;lt;&amp;lt;EOF
apiVersion: v1
kind: Service
metadata:
name: ratings
labels:
app: ratings
spec:
ports:
- port: 9080
name: http
selector:
app: ratings
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ratings-v1
spec:
replicas: 1
template:
metadata:
labels:
app: ratings
version: v1
spec:
containers:
- name: ratings
image: istio/examples-bookinfo-ratings-v1:1.10.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9080
---
apiVersion: v1
kind: Service
metadata:
name: reviews
labels:
app: reviews
spec:
ports:
- port: 9080
name: http
selector:
app: reviews
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: reviews-v2
spec:
replicas: 1
template:
metadata:
labels:
app: reviews
version: v2
spec:
containers:
- name: reviews
image: istio/examples-bookinfo-reviews-v2:1.10.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9080
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: reviews-v3
spec:
replicas: 1
template:
metadata:
labels:
app: reviews
version: v3
spec:
containers:
- name: reviews
image: istio/examples-bookinfo-reviews-v3:1.10.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9080
EOF
&lt;/code>&lt;/pre>
&lt;h2 id="access-the-bookinfo-application">Access the &lt;code>bookinfo&lt;/code> application&lt;/h2>
&lt;p>Just like any application, we&amp;rsquo;ll use an Istio gateway to access the &lt;code>bookinfo&lt;/code> application.&lt;/p>
&lt;ul>
&lt;li>&lt;p>Create the &lt;code>bookinfo&lt;/code> gateway in &lt;code>cluster1&lt;/code>:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/bookinfo-gateway.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply --context=$CTX_CLUSTER1 -f @samples/bookinfo/networking/bookinfo-gateway.yaml@
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Follow the &lt;a href="/v1.12/docs/examples/bookinfo/#determine-the-ingress-ip-and-port">Bookinfo sample instructions&lt;/a>
to determine the ingress IP and port and then point your browser to &lt;code>http://$GATEWAY_URL/productpage&lt;/code>.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>You should see the &lt;code>productpage&lt;/code> with reviews, but without ratings, because only &lt;code>v1&lt;/code> of the &lt;code>reviews&lt;/code> service
is running on &lt;code>cluster1&lt;/code> and we have not yet configured access to &lt;code>cluster2&lt;/code>.&lt;/p>
&lt;h2 id="create-a-service-entry-and-destination-rule-on-cluster1-for-the-remote-reviews-service">Create a service entry and destination rule on &lt;code>cluster1&lt;/code> for the remote reviews service&lt;/h2>
&lt;p>As described in the &lt;a href="https://istio.io/v1.6/docs/docs/setup/install/multicluster/gateways/#setup-dns">setup instructions&lt;/a>,
remote services are accessed with a &lt;code>.global&lt;/code> DNS name. In our case, it&amp;rsquo;s &lt;code>reviews.default.global&lt;/code>,
so we need to create a service entry and destination rule for that host.
The service entry will use the &lt;code>cluster2&lt;/code> gateway as the endpoint address to access the service.
You can use the gateway&amp;rsquo;s DNS name, if it has one, or its public IP, like this:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export CLUSTER2_GW_ADDR=$(kubectl get --context=$CTX_CLUSTER2 svc --selector=app=istio-ingressgateway \
-n istio-system -o jsonpath=&amp;#34;{.items[0].status.loadBalancer.ingress[0].ip}&amp;#34;)
&lt;/code>&lt;/pre>
&lt;p>Now create the service entry and destination rule using the following command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply --context=$CTX_CLUSTER1 -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: reviews-default
spec:
hosts:
- reviews.default.global
location: MESH_INTERNAL
ports:
- name: http1
number: 9080
protocol: http
resolution: DNS
addresses:
- 240.0.0.3
endpoints:
- address: ${CLUSTER2_GW_ADDR}
labels:
cluster: cluster2
ports:
http1: 15443 # Do not change this port value
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews-global
spec:
host: reviews.default.global
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
subsets:
- name: v2
labels:
cluster: cluster2
- name: v3
labels:
cluster: cluster2
EOF
&lt;/code>&lt;/pre>
&lt;p>The address &lt;code>240.0.0.3&lt;/code> of the service entry can be any arbitrary unallocated IP.
Using an IP from the class E addresses range 240.0.0.0/4 is a good choice.
Check out the
&lt;a href="/v1.12/docs/setup/install/multicluster">gateway-connected multicluster example&lt;/a>
for more details.&lt;/p>
&lt;p>Note that the labels of the subsets in the destination rule map to the service entry
endpoint label (&lt;code>cluster: cluster2&lt;/code>) corresponding to the &lt;code>cluster2&lt;/code> gateway.
Once the request reaches the destination cluster, a local destination rule will be used
to identify the actual pod labels (&lt;code>version: v1&lt;/code> or &lt;code>version: v2&lt;/code>) corresponding to the
requested subset.&lt;/p>
&lt;h2 id="create-a-destination-rule-on-both-clusters-for-the-local-reviews-service">Create a destination rule on both clusters for the local reviews service&lt;/h2>
&lt;p>Technically, we only need to define the subsets of the local service that are being used
in each cluster (i.e., &lt;code>v1&lt;/code> in &lt;code>cluster1&lt;/code>, &lt;code>v2&lt;/code> and &lt;code>v3&lt;/code> in &lt;code>cluster2&lt;/code>), but for simplicity we&amp;rsquo;ll
just define all three subsets in both clusters, since there&amp;rsquo;s nothing wrong with defining subsets
for versions that are not actually deployed.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply --context=$CTX_CLUSTER1 -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews.default.svc.cluster.local
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
- name: v3
labels:
version: v3
EOF
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply --context=$CTX_CLUSTER2 -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews.default.svc.cluster.local
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
- name: v3
labels:
version: v3
EOF
&lt;/code>&lt;/pre>
&lt;h2 id="create-a-virtual-service-to-route-reviews-service-traffic">Create a virtual service to route reviews service traffic&lt;/h2>
&lt;p>At this point, all calls to the &lt;code>reviews&lt;/code> service will go to the local &lt;code>reviews&lt;/code> pods (&lt;code>v1&lt;/code>) because
if you look at the source code you will see that the &lt;code>productpage&lt;/code> implementation is simply making
requests to &lt;code>http://reviews:9080&lt;/code> (which expands to host &lt;code>reviews.default.svc.cluster.local&lt;/code>), the
local version of the service.
The corresponding remote service is named &lt;code>reviews.default.global&lt;/code>, so route rules are needed to
redirect requests to the global host.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Note that if all of the versions of the &lt;code>reviews&lt;/code> service were remote, so there is no local &lt;code>reviews&lt;/code>
service defined, the DNS would resolve &lt;code>reviews&lt;/code> directly to &lt;code>reviews.default.global&lt;/code>. In that case
we could call the remote &lt;code>reviews&lt;/code> service without any route rules.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Apply the following virtual service to direct traffic for user &lt;code>jason&lt;/code> to &lt;code>reviews&lt;/code> versions &lt;code>v2&lt;/code> and &lt;code>v3&lt;/code> (&lt;sup>50&lt;/sup>&amp;frasl;&lt;sub>50&lt;/sub>)
which are running on &lt;code>cluster2&lt;/code>. Traffic for any other user will go to &lt;code>reviews&lt;/code> version &lt;code>v1&lt;/code>.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply --context=$CTX_CLUSTER1 -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews.default.svc.cluster.local
http:
- match:
- headers:
end-user:
exact: jason
route:
- destination:
host: reviews.default.global
subset: v2
weight: 50
- destination:
host: reviews.default.global
subset: v3
weight: 50
- route:
- destination:
host: reviews.default.svc.cluster.local
subset: v1
EOF
&lt;/code>&lt;/pre>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">This &lt;sup>50&lt;/sup>&amp;frasl;&lt;sub>50&lt;/sub> rule isn&amp;rsquo;t a particularly realistic example. It&amp;rsquo;s just a convenient way to demonstrate
accessing multiple subsets of a remote service.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>Return to your browser and login as user &lt;code>jason&lt;/code>. If you refresh the page several times, you should see
the display alternating between black and red ratings stars (&lt;code>v2&lt;/code> and &lt;code>v3&lt;/code>). If you logout, you will
only see reviews without ratings (&lt;code>v1&lt;/code>).&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>In this article, we&amp;rsquo;ve seen how to use Istio route rules to distribute the versions of a service
across clusters in a multicluster service mesh with a replicated control plane model.
In this example, we manually configured the &lt;code>.global&lt;/code> service entry and destination rules needed to provide
connectivity to one remote service, &lt;code>reviews&lt;/code>. In general, however, if we wanted to enable any service
to run either locally or remotely, we would need to create &lt;code>.global&lt;/code> resources for every service.
Fortunately, this process could be automated and likely will be in a future Istio release.&lt;/p></description><pubDate>Thu, 07 Feb 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/multicluster-version-routing/</link><author>Frank Budinsky (IBM)</author><guid isPermaLink="true">/v1.12/blog/2019/multicluster-version-routing/</guid><category>traffic-management</category><category>multicluster</category></item><item><title>Sail the Blog!</title><description>&lt;p>Welcome to the Istio blog!&lt;/p>
&lt;p>To make it easier to publish your content on our website, we
&lt;a href="/v1.12/docs/releases/contribute/add-content/#content-types">updated the content types guide&lt;/a>.&lt;/p>
&lt;p>The goal of the updated guide is to make sharing and finding content easier.&lt;/p>
&lt;p>We want to make sharing timely information on Istio easy and the &lt;a href="/v1.12/blog">Istio blog&lt;/a>
is a good place to start.&lt;/p>
&lt;p>We welcome your posts to the blog if you think your content falls in one of the
following four categories:&lt;/p>
&lt;ul>
&lt;li>Your post details your experience using and configuring Istio. Ideally, your
post shares a novel experience or perspective.&lt;/li>
&lt;li>Your post highlights Istio features.&lt;/li>
&lt;li>Your post details how to accomplish a task or fulfill a specific use case
using Istio.&lt;/li>
&lt;/ul>
&lt;p>Posting your blog is only &lt;a href="/v1.12/docs/releases/contribute/github/">one PR away&lt;/a>
and, if you wish, you can &lt;a href="/v1.12/docs/releases/contribute/review">request a review&lt;/a>.&lt;/p>
&lt;p>We look forward to reading about your Istio experience on the blog soon!&lt;/p></description><pubDate>Tue, 05 Feb 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/sail-the-blog/</link><author>Rigs Caballero, Google</author><guid isPermaLink="true">/v1.12/blog/2019/sail-the-blog/</guid><category>community</category><category>blog</category><category>contribution</category><category>guide</category><category>guideline</category><category>event</category></item><item><title>Egress Gateway Performance Investigation</title><description>
&lt;p>The main objective of this investigation was to determine the impact on performance and resource utilization when an egress gateway is added in the service mesh to access an external service (MongoDB, in this case). The steps to configure an egress gateway for an external MongoDB are described in the blog &lt;a href="/v1.12/blog/2018/egress-mongo/">Consuming External MongoDB Services&lt;/a>.&lt;/p>
&lt;p>The application used for this investigation was the Java version of Acmeair, which simulates an airline reservation system. This application is used in the Performance Regression Patrol of Istio daily builds, but on that setup the microservices have been accessing the external MongoDB directly via their sidecars, without an egress gateway.&lt;/p>
&lt;p>The diagram below illustrates how regression patrol currently runs with Acmeair and Istio:&lt;/p>
&lt;figure style="width:70%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:62.69230769230769%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/acmeair_regpatrol3.png" title="Acmeair benchmark in the Istio performance regression patrol environment">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/acmeair_regpatrol3.png" alt="Acmeair benchmark in the Istio performance regression patrol environment" />
&lt;/a>
&lt;/div>
&lt;figcaption>Acmeair benchmark in the Istio performance regression patrol environment&lt;/figcaption>
&lt;/figure>
&lt;p>Another difference is that the application communicates with the external DB with plain MongoDB protocol. The first change made for this study was to establish a TLS communication between the MongoDB and its clients running within the application, as this is a more realistic scenario.&lt;/p>
&lt;p>Several cases for accessing the external database from the mesh were tested and described next.&lt;/p>
&lt;h2 id="egress-traffic-cases">Egress traffic cases&lt;/h2>
&lt;h3 id="case-1-bypassing-the-sidecar">Case 1: Bypassing the sidecar&lt;/h3>
&lt;p>In this case, the sidecar does not intercept the communication between the application and the external DB. This is accomplished by setting the init container argument -x with the CIDR of the MongoDB, which makes the sidecar ignore messages to/from this IP address. For example:&lt;/p>
&lt;pre>&lt;code> - -x
- &amp;#34;169.47.232.211/32&amp;#34;&lt;/code>&lt;/pre>
&lt;figure style="width:70%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:76.45536869340232%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/case1_sidecar_bypass3.png" title="Traffic to external MongoDB by-passing the sidecar">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/case1_sidecar_bypass3.png" alt="Traffic to external MongoDB by-passing the sidecar" />
&lt;/a>
&lt;/div>
&lt;figcaption>Traffic to external MongoDB by-passing the sidecar&lt;/figcaption>
&lt;/figure>
&lt;h3 id="case-2-through-the-sidecar-with-service-entry">Case 2: Through the sidecar, with service entry&lt;/h3>
&lt;p>This is the default configuration when the sidecar is injected into the application pod. All messages are intercepted by the sidecar and routed to the destination according to the configured rules, including the communication with external services. The MongoDB was defined as a &lt;code>ServiceEntry&lt;/code>.&lt;/p>
&lt;figure style="width:70%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:74.41253263707573%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/case2_sidecar_passthru3.png" title="Sidecar intercepting traffic to external MongoDB">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/case2_sidecar_passthru3.png" alt="Sidecar intercepting traffic to external MongoDB" />
&lt;/a>
&lt;/div>
&lt;figcaption>Sidecar intercepting traffic to external MongoDB&lt;/figcaption>
&lt;/figure>
&lt;h3 id="case-3-egress-gateway">Case 3: Egress gateway&lt;/h3>
&lt;p>The egress gateway and corresponding destination rule and virtual service resources are defined for accessing MongoDB. All traffic to and from the external DB goes through the egress gateway (envoy).&lt;/p>
&lt;figure style="width:70%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:62.309368191721134%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/case3_egressgw3.png" title="Introduction of the egress gateway to access MongoDB">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/case3_egressgw3.png" alt="Introduction of the egress gateway to access MongoDB" />
&lt;/a>
&lt;/div>
&lt;figcaption>Introduction of the egress gateway to access MongoDB&lt;/figcaption>
&lt;/figure>
&lt;h3 id="case-4-mutual-tls-between-sidecars-and-the-egress-gateway">Case 4: Mutual TLS between sidecars and the egress gateway&lt;/h3>
&lt;p>In this case, there is an extra layer of security between the sidecars and the gateway, so some impact in performance is expected.&lt;/p>
&lt;figure style="width:70%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:63.968957871396896%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/case4_egressgw_mtls3.png" title="Enabling mutual TLS between sidecars and the egress gateway">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/case4_egressgw_mtls3.png" alt="Enabling mutual TLS between sidecars and the egress gateway" />
&lt;/a>
&lt;/div>
&lt;figcaption>Enabling mutual TLS between sidecars and the egress gateway&lt;/figcaption>
&lt;/figure>
&lt;h3 id="case-5-egress-gateway-with-sni-proxy">Case 5: Egress gateway with SNI proxy&lt;/h3>
&lt;p>This scenario is used to evaluate the case where another proxy is required to access wildcarded domains. This may be required due current limitations of envoy. An nginx proxy was created as sidecar in the egress gateway pod.&lt;/p>
&lt;figure style="width:70%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:65.2762119503946%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/case5_egressgw_sni_proxy3.png" title="Egress gateway with additional SNI Proxy">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/case5_egressgw_sni_proxy3.png" alt="Egress gateway with additional SNI Proxy" />
&lt;/a>
&lt;/div>
&lt;figcaption>Egress gateway with additional SNI Proxy&lt;/figcaption>
&lt;/figure>
&lt;h2 id="environment">Environment&lt;/h2>
&lt;ul>
&lt;li>Istio version: 1.0.2&lt;/li>
&lt;li>&lt;code>K8s&lt;/code> version: &lt;code>1.10.5_1517&lt;/code>&lt;/li>
&lt;li>Acmeair App: 4 services (1 replica of each), inter-services transactions, external Mongo DB, avg payload: 620 bytes.&lt;/li>
&lt;/ul>
&lt;h2 id="results">Results&lt;/h2>
&lt;p>&lt;code>Jmeter&lt;/code> was used to generate the workload which consisted in a sequence of 5-minute runs, each one using a growing number of clients making http requests. The number of clients used were 1, 5, 10, 20, 30, 40, 50 and 60.&lt;/p>
&lt;h3 id="throughput">Throughput&lt;/h3>
&lt;p>The chart below shows the throughput obtained for the different cases:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:54.29638854296388%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/throughput3.png" title="Throughput obtained for the different cases">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/throughput3.png" alt="Throughput obtained for the different cases" />
&lt;/a>
&lt;/div>
&lt;figcaption>Throughput obtained for the different cases&lt;/figcaption>
&lt;/figure>
&lt;p>As you can see, there is no major impact in having sidecars and the egress gateway between the application and the external MongoDB, but enabling mutual TLS and then adding the SNI proxy caused a degradation in the throughput of about 10% and 24%, respectively.&lt;/p>
&lt;h3 id="response-time">Response time&lt;/h3>
&lt;p>The average response times for the different requests were collected when traffic was being driven with 20 clients. The chart below shows the average, median, 90%, 95% and 99% average values for each case:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:48.76783398184176%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/response_times3.png" title="Response times obtained for the different configurations">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/response_times3.png" alt="Response times obtained for the different configurations" />
&lt;/a>
&lt;/div>
&lt;figcaption>Response times obtained for the different configurations&lt;/figcaption>
&lt;/figure>
&lt;p>Likewise, not much difference in the response times for the 3 first cases, but mutual TLS and the extra proxy adds noticeable latency.&lt;/p>
&lt;h3 id="cpu-utilization">CPU utilization&lt;/h3>
&lt;p>The CPU usage was collected for all Istio components as well as for the sidecars during the runs. For a fair comparison, CPU used by Istio was normalized by the throughput obtained for a given run. The results are shown in the following graph:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:53.96174863387978%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/egress-performance/cpu_usage3.png" title="CPU usage normalized by TPS">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/egress-performance/cpu_usage3.png" alt="CPU usage normalized by TPS" />
&lt;/a>
&lt;/div>
&lt;figcaption>CPU usage normalized by TPS&lt;/figcaption>
&lt;/figure>
&lt;p>In terms of CPU consumption per transaction, Istio has used significantly more CPU only in the egress gateway + SNI proxy case.&lt;/p>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>In this investigation, we tried different options to access an external TLS-enabled MongoDB to compare their performance. The introduction of the Egress Gateway did not have a significant impact on the performance nor meaningful additional CPU consumption. Only when enabling mutual TLS between sidecars and egress gateway or using an additional SNI proxy for wildcarded domains we could observe some degradation.&lt;/p></description><pubDate>Thu, 31 Jan 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/egress-performance/</link><author>Jose Nativio, IBM</author><guid isPermaLink="true">/v1.12/blog/2019/egress-performance/</guid><category>performance</category><category>traffic-management</category><category>egress</category><category>mongo</category></item><item><title>Demystifying Istio's Sidecar Injection Model</title><description>
&lt;p>A simple overview of an Istio service-mesh architecture always starts with describing the control-plane and data-plane.&lt;/p>
&lt;p>&lt;a href="/v1.12/docs/ops/deployment/architecture/">From Istio’s documentation&lt;/a>:&lt;/p>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">&lt;p>An Istio service mesh is logically split into a data plane and a control plane.&lt;/p>
&lt;p>The data plane is composed of a set of intelligent proxies (Envoy) deployed as sidecars. These proxies mediate and control all network communication between microservices along with Mixer, a general-purpose policy and telemetry hub.&lt;/p>
&lt;p>The control plane manages and configures the proxies to route traffic. Additionally, the control plane configures Mixers to enforce policies and collect telemetry.&lt;/p>
&lt;/div>
&lt;/aside>
&lt;/div>
&lt;figure style="width:40%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:80%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2019/data-plane-setup/arch-2.svg" title="Istio Architecture">
&lt;img class="element-to-stretch" src="/v1.12/blog/2019/data-plane-setup/arch-2.svg" alt="The overall architecture of an Istio-based application." />
&lt;/a>
&lt;/div>
&lt;figcaption>Istio Architecture&lt;/figcaption>
&lt;/figure>
&lt;p>It is important to understand that the sidecar injection into the application pods happens automatically, though manual injection is also possible. Traffic is directed from the application services to and from these sidecars without developers needing to worry about it. Once the applications are connected to the Istio service mesh, developers can start using and reaping the benefits of all that the service mesh has to offer. However, how does the data plane plumbing happen and what is really required to make it work seamlessly? In this post, we will deep-dive into the specifics of the sidecar injection models to gain a very clear understanding of how sidecar injection works.&lt;/p>
&lt;h2 id="sidecar-injection">Sidecar injection&lt;/h2>
&lt;p>In simple terms, sidecar injection is adding the configuration of additional containers to the pod template. The added containers needed for the Istio service mesh are:&lt;/p>
&lt;p>&lt;code>istio-init&lt;/code>
This &lt;a href="https://kubernetes.io/docs/concepts/workloads/pods/init-containers/">init container&lt;/a> is used to setup the &lt;code>iptables&lt;/code> rules so that inbound/outbound traffic will go through the sidecar proxy. An init container is different than an app container in following ways:&lt;/p>
&lt;ul>
&lt;li>It runs before an app container is started and it always runs to completion.&lt;/li>
&lt;li>If there are many init containers, each should complete with success before the next container is started.&lt;/li>
&lt;/ul>
&lt;p>So, you can see how this type of container is perfect for a set-up or initialization job which does not need to be a part of the actual application container. In this case, &lt;code>istio-init&lt;/code> does just that and sets up the &lt;code>iptables&lt;/code> rules.&lt;/p>
&lt;p>&lt;code>istio-proxy&lt;/code>
This is the actual sidecar proxy (based on Envoy).&lt;/p>
&lt;h3 id="manual-injection">Manual injection&lt;/h3>
&lt;p>In the manual injection method, you can use &lt;a href="/v1.12/docs/reference/commands/istioctl">&lt;code>istioctl&lt;/code>&lt;/a> to modify the pod template and add the configuration of the two containers previously mentioned. For both manual as well as automatic injection, Istio takes the configuration from the &lt;code>istio-sidecar-injector&lt;/code> configuration map (configmap) and the mesh&amp;rsquo;s &lt;code>istio&lt;/code> configmap.&lt;/p>
&lt;p>Let’s look at the configuration of the &lt;code>istio-sidecar-injector&lt;/code> configmap, to get an idea of what actually is going on.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath=&amp;#39;{.data.config}&amp;#39;
SNIPPET from the output:
policy: enabled
template: |-
initContainers:
- name: istio-init
image: docker.io/istio/proxy_init:1.0.2
args:
- &amp;#34;-p&amp;#34;
- [[ .MeshConfig.ProxyListenPort ]]
- &amp;#34;-u&amp;#34;
- 1337
.....
imagePullPolicy: IfNotPresent
securityContext:
capabilities:
add:
- NET_ADMIN
restartPolicy: Always
containers:
- name: istio-proxy
image: [[ if (isset .ObjectMeta.Annotations &amp;#34;sidecar.istio.io/proxyImage&amp;#34;) -]]
&amp;#34;[[ index .ObjectMeta.Annotations &amp;#34;sidecar.istio.io/proxyImage&amp;#34; ]]&amp;#34;
[[ else -]]
docker.io/istio/proxyv2:1.0.2
[[ end -]]
args:
- proxy
- sidecar
.....
env:
.....
- name: ISTIO_META_INTERCEPTION_MODE
value: [[ or (index .ObjectMeta.Annotations &amp;#34;sidecar.istio.io/interceptionMode&amp;#34;) .ProxyConfig.InterceptionMode.String ]]
imagePullPolicy: IfNotPresent
securityContext:
readOnlyRootFilesystem: true
[[ if eq (or (index .ObjectMeta.Annotations &amp;#34;sidecar.istio.io/interceptionMode&amp;#34;) .ProxyConfig.InterceptionMode.String) &amp;#34;TPROXY&amp;#34; -]]
capabilities:
add:
- NET_ADMIN
restartPolicy: Always
.....
&lt;/code>&lt;/pre>
&lt;p>As you can see, the configmap contains the configuration for both, the &lt;code>istio-init&lt;/code> init container and the &lt;code>istio-proxy&lt;/code> proxy container. The configuration includes the name of the container image and arguments like interception mode, capabilities, etc.&lt;/p>
&lt;p>From a security point of view, it is important to note that &lt;code>istio-init&lt;/code> requires &lt;code>NET_ADMIN&lt;/code> capabilities to modify &lt;code>iptables&lt;/code> within the pod&amp;rsquo;s namespace and so does &lt;code>istio-proxy&lt;/code> if configured in &lt;code>TPROXY&lt;/code> mode. As this is restricted to a pod&amp;rsquo;s namespace, there should be no problem. However, I have noticed that recent open-shift versions may have some issues with it and a workaround is needed. One such option is mentioned at the end of this post.&lt;/p>
&lt;p>To modify the current pod template for sidecar injection, you can:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl kube-inject -f demo-red.yaml | kubectl apply -f -
&lt;/code>&lt;/pre>
&lt;p>OR&lt;/p>
&lt;p>To use modified configmaps or local configmaps:&lt;/p>
&lt;ul>
&lt;li>&lt;p>Create &lt;code>inject-config.yaml&lt;/code> and &lt;code>mesh-config.yaml&lt;/code> from the configmaps&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath=&amp;#39;{.data.config}&amp;#39; &amp;gt; inject-config.yaml
$ kubectl -n istio-system get configmap istio -o=jsonpath=&amp;#39;{.data.mesh}&amp;#39; &amp;gt; mesh-config.yaml
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Modify the existing pod template, in my case, &lt;code>demo-red.yaml&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ istioctl kube-inject --injectConfigFile inject-config.yaml --meshConfigFile mesh-config.yaml --filename demo-red.yaml --output demo-red-injected.yaml
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Apply the &lt;code>demo-red-injected.yaml&lt;/code>&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f demo-red-injected.yaml
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ul>
&lt;p>As seen above, we create a new template using the &lt;code>sidecar-injector&lt;/code> and the mesh configuration to then apply that new template using &lt;code>kubectl&lt;/code>. If we look at the injected YAML file, it has the configuration of the Istio-specific containers, as we discussed above. Once we apply the injected YAML file, we see two containers running. One of them is the actual application container, and the other is the &lt;code>istio-proxy&lt;/code> sidecar.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods | grep demo-red
demo-red-pod-8b5df99cc-pgnl7 2/2 Running 0 3d
&lt;/code>&lt;/pre>
&lt;p>The count is not 3 because the &lt;code>istio-init&lt;/code> container is an init type container that exits after doing what it supposed to do, which is setting up the &lt;code>iptable&lt;/code> rules within the pod. To confirm the init container exit, let’s look at the output of &lt;code>kubectl describe&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl describe pod demo-red-pod-8b5df99cc-pgnl7
SNIPPET from the output:
Name: demo-red-pod-8b5df99cc-pgnl7
Namespace: default
.....
Labels: app=demo-red
pod-template-hash=8b5df99cc
version=version-red
Annotations: sidecar.istio.io/status={&amp;#34;version&amp;#34;:&amp;#34;3c0b8d11844e85232bc77ad85365487638ee3134c91edda28def191c086dc23e&amp;#34;,&amp;#34;initContainers&amp;#34;:[&amp;#34;istio-init&amp;#34;],&amp;#34;containers&amp;#34;:[&amp;#34;istio-proxy&amp;#34;],&amp;#34;volumes&amp;#34;:[&amp;#34;istio-envoy&amp;#34;,&amp;#34;istio-certs...
Status: Running
IP: 10.32.0.6
Controlled By: ReplicaSet/demo-red-pod-8b5df99cc
Init Containers:
istio-init:
Container ID: docker://bef731eae1eb3b6c9d926cacb497bb39a7d9796db49cd14a63014fc1a177d95b
Image: docker.io/istio/proxy_init:1.0.2
Image ID: docker-pullable://docker.io/istio/proxy_init@sha256:e16a0746f46cd45a9f63c27b9e09daff5432e33a2d80c8cc0956d7d63e2f9185
.....
State: Terminated
Reason: Completed
.....
Ready: True
Containers:
demo-red:
Container ID: docker://8cd9957955ff7e534376eb6f28b56462099af6dfb8b9bc37aaf06e516175495e
Image: chugtum/blue-green-image:v3
Image ID: docker-pullable://docker.io/chugtum/blue-green-image@sha256:274756dbc215a6b2bd089c10de24fcece296f4c940067ac1a9b4aea67cf815db
State: Running
Started: Sun, 09 Dec 2018 18:12:31 -0800
Ready: True
istio-proxy:
Container ID: docker://ca5d690be8cd6557419cc19ec4e76163c14aed2336eaad7ebf17dd46ca188b4a
Image: docker.io/istio/proxyv2:1.0.2
Image ID: docker-pullable://docker.io/istio/proxyv2@sha256:54e206530ba6ca9b3820254454e01b7592e9f986d27a5640b6c03704b3b68332
Args:
proxy
sidecar
.....
State: Running
Started: Sun, 09 Dec 2018 18:12:31 -0800
Ready: True
.....
&lt;/code>&lt;/pre>
&lt;p>As seen in the output, the &lt;code>State&lt;/code> of the &lt;code>istio-init&lt;/code> container is &lt;code>Terminated&lt;/code> with the &lt;code>Reason&lt;/code> being &lt;code>Completed&lt;/code>. The only two containers running are the main application &lt;code>demo-red&lt;/code> container and the &lt;code>istio-proxy&lt;/code> container.&lt;/p>
&lt;h3 id="automatic-injection">Automatic injection&lt;/h3>
&lt;p>Most of the times, you don’t want to manually inject a sidecar every time you deploy an application, using the &lt;a href="/v1.12/docs/reference/commands/istioctl">&lt;code>istioctl&lt;/code>&lt;/a> command, but would prefer that Istio automatically inject the sidecar to your pod. This is the recommended approach and for it to work, all you need to do is to label the namespace where you are deploying the app with &lt;code>istio-injection=enabled&lt;/code>.&lt;/p>
&lt;p>Once labeled, Istio injects the sidecar automatically for any pod you deploy in that namespace. In the following example, the sidecar gets automatically injected in the deployed pods in the &lt;code>istio-dev&lt;/code> namespace.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get namespaces --show-labels
NAME STATUS AGE LABELS
default Active 40d &amp;lt;none&amp;gt;
istio-dev Active 19d istio-injection=enabled
istio-system Active 24d &amp;lt;none&amp;gt;
kube-public Active 40d &amp;lt;none&amp;gt;
kube-system Active 40d &amp;lt;none&amp;gt;
&lt;/code>&lt;/pre>
&lt;p>But how does this work? To get to the bottom of this, we need to understand Kubernetes admission controllers.&lt;/p>
&lt;p>&lt;a href="https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/">From Kubernetes documentation:&lt;/a>&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized. You can define two types of admission webhooks, validating admission Webhook and mutating admission webhook. With validating admission Webhooks, you may reject requests to enforce custom admission policies. With mutating admission Webhooks, you may change requests to enforce custom defaults.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>For automatic sidecar injection, Istio relies on &lt;code>Mutating Admission Webhook&lt;/code>. Let’s look at the details of the &lt;code>istio-sidecar-injector&lt;/code> mutating webhook configuration.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl get mutatingwebhookconfiguration istio-sidecar-injector -o yaml
SNIPPET from the output:
apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{&amp;#34;apiVersion&amp;#34;:&amp;#34;admissionregistration.k8s.io/v1beta1&amp;#34;,&amp;#34;kind&amp;#34;:&amp;#34;MutatingWebhookConfiguration&amp;#34;,&amp;#34;metadata&amp;#34;:{&amp;#34;annotations&amp;#34;:{},&amp;#34;labels&amp;#34;:{&amp;#34;app&amp;#34;:&amp;#34;istio-sidecar-injector&amp;#34;,&amp;#34;chart&amp;#34;:&amp;#34;sidecarInjectorWebhook-1.0.1&amp;#34;,&amp;#34;heritage&amp;#34;:&amp;#34;Tiller&amp;#34;,&amp;#34;release&amp;#34;:&amp;#34;istio-remote&amp;#34;},&amp;#34;name&amp;#34;:&amp;#34;istio-sidecar-injector&amp;#34;,&amp;#34;namespace&amp;#34;:&amp;#34;&amp;#34;},&amp;#34;webhooks&amp;#34;:[{&amp;#34;clientConfig&amp;#34;:{&amp;#34;caBundle&amp;#34;:&amp;#34;&amp;#34;,&amp;#34;service&amp;#34;:{&amp;#34;name&amp;#34;:&amp;#34;istio-sidecar-injector&amp;#34;,&amp;#34;namespace&amp;#34;:&amp;#34;istio-system&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/inject&amp;#34;}},&amp;#34;failurePolicy&amp;#34;:&amp;#34;Fail&amp;#34;,&amp;#34;name&amp;#34;:&amp;#34;sidecar-injector.istio.io&amp;#34;,&amp;#34;namespaceSelector&amp;#34;:{&amp;#34;matchLabels&amp;#34;:{&amp;#34;istio-injection&amp;#34;:&amp;#34;enabled&amp;#34;}},&amp;#34;rules&amp;#34;:[{&amp;#34;apiGroups&amp;#34;:[&amp;#34;&amp;#34;],&amp;#34;apiVersions&amp;#34;:[&amp;#34;v1&amp;#34;],&amp;#34;operations&amp;#34;:[&amp;#34;CREATE&amp;#34;],&amp;#34;resources&amp;#34;:[&amp;#34;pods&amp;#34;]}]}]}
creationTimestamp: 2018-12-10T08:40:15Z
generation: 2
labels:
app: istio-sidecar-injector
chart: sidecarInjectorWebhook-1.0.1
heritage: Tiller
release: istio-remote
name: istio-sidecar-injector
.....
webhooks:
- clientConfig:
service:
name: istio-sidecar-injector
namespace: istio-system
path: /inject
name: sidecar-injector.istio.io
namespaceSelector:
matchLabels:
istio-injection: enabled
rules:
- apiGroups:
- &amp;#34;&amp;#34;
apiVersions:
- v1
operations:
- CREATE
resources:
- pods
&lt;/code>&lt;/pre>
&lt;p>This is where you can see the webhook &lt;code>namespaceSelector&lt;/code> label that is matched for sidecar injection with the label &lt;code>istio-injection: enabled&lt;/code>. In this case, you also see the operations and resources for which this is done when the pods are created. When an &lt;code>apiserver&lt;/code> receives a request that matches one of the rules, the &lt;code>apiserver&lt;/code> sends an admission review request to the webhook service as specified in the &lt;code>clientConfig:&lt;/code>configuration with the &lt;code>name: istio-sidecar-injector&lt;/code> key-value pair. We should be able to see that this service is running in the &lt;code>istio-system&lt;/code> namespace.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get svc --namespace=istio-system | grep sidecar-injector
istio-sidecar-injector ClusterIP 10.102.70.184 &amp;lt;none&amp;gt; 443/TCP 24d
&lt;/code>&lt;/pre>
&lt;p>This configuration ultimately does pretty much the same as we saw in manual injection. Just that it is done automatically during pod creation, so you won’t see the change in the deployment. You need to use &lt;code>kubectl describe&lt;/code> to see the sidecar proxy and the init proxy.&lt;/p>
&lt;p>The automatic sidecar injection not only depends on the &lt;code>namespaceSelector&lt;/code> mechanism of the webhook, but also on the default injection policy and the per-pod override annotation.&lt;/p>
&lt;p>If you look at the &lt;code>istio-sidecar-injector&lt;/code> ConfigMap again, it has the default injection policy defined. In our case, it is enabled by default.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-outputis='yaml' data-repo='istio' >$ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath=&amp;#39;{.data.config}&amp;#39;
SNIPPET from the output:
policy: enabled
template: |-
initContainers:
- name: istio-init
image: &amp;#34;gcr.io/istio-release/proxy_init:1.0.2&amp;#34;
args:
- &amp;#34;-p&amp;#34;
- [[ .MeshConfig.ProxyListenPort ]]
&lt;/code>&lt;/pre>
&lt;p>You can also use the annotation &lt;code>sidecar.istio.io/inject&lt;/code> in the pod template to override the default policy. The following example disables the automatic injection of the sidecar for the pods in a &lt;code>Deployment&lt;/code>.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ignored
spec:
template:
metadata:
annotations:
sidecar.istio.io/inject: &amp;#34;false&amp;#34;
spec:
containers:
- name: ignored
image: tutum/curl
command: [&amp;#34;/bin/sleep&amp;#34;,&amp;#34;infinity&amp;#34;]
&lt;/code>&lt;/pre>
&lt;p>This example shows there are many variables, based on whether the automatic sidecar injection is controlled in your namespace, ConfigMap, or pod and they are:&lt;/p>
&lt;ul>
&lt;li>webhooks &lt;code>namespaceSelector&lt;/code> (&lt;code>istio-injection: enabled&lt;/code>)&lt;/li>
&lt;li>default policy (Configured in the ConfigMap &lt;code>istio-sidecar-injector&lt;/code>)&lt;/li>
&lt;li>per-pod override annotation (&lt;code>sidecar.istio.io/inject&lt;/code>)&lt;/li>
&lt;/ul>
&lt;p>The &lt;a href="/v1.12/docs/ops/common-problems/injection/">injection status table&lt;/a> shows a clear picture of the final injection status based on the value of the above variables.&lt;/p>
&lt;h2 id="traffic-flow-from-application-container-to-sidecar-proxy">Traffic flow from application container to sidecar proxy&lt;/h2>
&lt;p>Now that we are clear about how a sidecar container and an init container are injected into an application manifest, how does the sidecar proxy grab the inbound and outbound traffic to and from the container? We did briefly mention that it is done by setting up the &lt;code>iptable&lt;/code> rules within the pod namespace, which in turn is done by the &lt;code>istio-init&lt;/code> container. Now, it is time to verify what actually gets updated within the namespace.&lt;/p>
&lt;p>Let’s get into the application pod namespace we deployed in the previous section and look at the configured iptables. I am going to show an example using &lt;code>nsenter&lt;/code>. Alternatively, you can enter the container in a privileged mode to see the same information. For folks without access to the nodes, using &lt;code>exec&lt;/code> to get into the sidecar and running &lt;code>iptables&lt;/code> is more practical.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ docker inspect b8de099d3510 --format &amp;#39;{{ .State.Pid }}&amp;#39;
4125
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ nsenter -t 4215 -n iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N ISTIO_INBOUND
-N ISTIO_IN_REDIRECT
-N ISTIO_OUTPUT
-N ISTIO_REDIRECT
-A PREROUTING -p tcp -j ISTIO_INBOUND
-A OUTPUT -p tcp -j ISTIO_OUTPUT
-A ISTIO_INBOUND -p tcp -m tcp --dport 80 -j ISTIO_IN_REDIRECT
-A ISTIO_IN_REDIRECT -p tcp -j REDIRECT --to-ports 15001
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -j ISTIO_REDIRECT
-A ISTIO_OUTPUT -m owner --uid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -m owner --gid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -d 127.0.0.1/32 -j RETURN
-A ISTIO_OUTPUT -j ISTIO_REDIRECT
-A ISTIO_REDIRECT -p tcp -j REDIRECT --to-ports 15001
&lt;/code>&lt;/pre>
&lt;p>The output above clearly shows that all the incoming traffic to port 80, which is the port our &lt;code>red-demo&lt;/code> application is listening, is now &lt;code>REDIRECTED&lt;/code> to port &lt;code>15001&lt;/code>, which is the port that the &lt;code>istio-proxy&lt;/code>, an Envoy proxy, is listening. The same holds true for the outgoing traffic.&lt;/p>
&lt;p>This brings us to the end of this post. I hope it helped to de-mystify how Istio manages to inject the sidecar proxies into an existing deployment and how Istio routes the traffic to the proxy.&lt;/p>
&lt;div>
&lt;aside class="callout idea">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-idea"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Update: In place of &lt;code>istio-init&lt;/code>, there now seems to be an option of using the new CNI, which removes the need for the init container and associated privileges. This &lt;a href="https://github.com/istio/cni">&lt;code>istio-cni&lt;/code>&lt;/a> plugin sets up the pods&amp;rsquo; networking to fulfill this requirement in place of the current Istio injected pod &lt;code>istio-init&lt;/code> approach.&lt;/div>
&lt;/aside>
&lt;/div></description><pubDate>Thu, 31 Jan 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/data-plane-setup/</link><author>Manish Chugtu</author><guid isPermaLink="true">/v1.12/blog/2019/data-plane-setup/</guid><category>kubernetes</category><category>sidecar-injection</category><category>traffic-management</category></item><item><title>Sidestepping Dependency Ordering with AppSwitch</title><description>
&lt;p>We are going through an interesting cycle of application decomposition and recomposition. While the microservice paradigm is driving monolithic applications to be broken into separate individual services, the service mesh approach is helping them to be connected back together into well-structured applications. As such, microservices are logically separate but not independent. They are usually closely interdependent and taking them apart introduces many new concerns such as need for mutual authentication between services. Istio directly addresses most of those issues.&lt;/p>
&lt;h2 id="dependency-ordering-problem">Dependency ordering problem&lt;/h2>
&lt;p>An issue that arises due to application decomposition and one that Istio doesn’t address is dependency ordering &amp;ndash; bringing up individual services of an application in an order that guarantees that the application as a whole comes up quickly and correctly. In a monolithic application, with all its components built-in, dependency ordering between the components is enforced by internal locking mechanisms. But with individual services potentially scattered across the cluster in a service mesh, starting a service first requires checking that the services it depends on are up and available.&lt;/p>
&lt;p>Dependency ordering is deceptively nuanced with a host of interrelated problems. Ordering individual services requires having the dependency graph of the services so that they can be brought up starting from leaf nodes back to the root nodes. It is not easy to construct such a graph and keep it updated over time as interdependencies evolve with the behavior of the application. Even if the dependency graph is somehow provided, enforcing the ordering itself is not easy. Simply starting the services in the specified order obviously won’t do. A service may have started but not be ready to accept connections yet. This is the problem with docker-compose&amp;rsquo;s &lt;code>depends-on&lt;/code> tag, for example.&lt;/p>
&lt;p>Apart from introducing sufficiently long sleeps between service startups, a common pattern that is often used is to check for readiness of dependencies before starting a service. In Kubernetes, this could be done with a wait script as part of the init container of the pod. However that means that the entire application would be held up until all its dependencies come alive. Sometimes applications spend several minutes initializing themselves on startup before making their first outbound connection. Not allowing a service to start at all adds substantial overhead to overall startup time of the application. Also, the strategy of waiting on the init container won&amp;rsquo;t work for the case of multiple interdependent services within the same pod.&lt;/p>
&lt;h3 id="example-scenario-ibm-websphere-nd">Example scenario: IBM WebSphere ND&lt;/h3>
&lt;p>Let us consider IBM WebSphere ND &amp;ndash; a widely deployed application middleware &amp;ndash; to grok these problems more closely. It is a fairly complex framework in itself and consists of a central component called deployment manager (&lt;code>dmgr&lt;/code>) that manages a set of node instances. It uses UDP to negotiate cluster membership among the nodes and requires that deployment manager is up and operational before any of the node instances can come up and join the cluster.&lt;/p>
&lt;p>Why are we talking about a traditional application in the modern cloud-native context? It turns out that there are significant gains to be had by enabling them to run on the Kubernetes and Istio platforms. Essentially it&amp;rsquo;s a part of the modernization journey that allows running traditional apps alongside green-field apps on the same modern platform to facilitate interoperation between the two. In fact, WebSphere ND is a demanding application. It expects a consistent network environment with specific network interface attributes etc. AppSwitch is equipped to take care of those requirements. For the purpose of this blog however, I&amp;rsquo;ll focus on the dependency ordering requirement and how AppSwitch addresses it.&lt;/p>
&lt;p>Simply deploying &lt;code>dmgr&lt;/code> and node instances as pods on a Kubernetes cluster does not work. &lt;code>dmgr&lt;/code> and the node instances happen to have a lengthy initialization process that can take several minutes. If they are all co-scheduled, the application typically ends up in a funny state. When a node instance comes up and finds that &lt;code>dmgr&lt;/code> is missing, it would take an alternate startup path. Instead, if it had exited immediately, Kubernetes crash-loop would have taken over and perhaps the application would have come up. But even in that case, it turns out that a timely startup is not guaranteed.&lt;/p>
&lt;p>One &lt;code>dmgr&lt;/code> along with its node instances is a basic deployment configuration for WebSphere ND. Applications like IBM Business Process Manager that are built on top of WebSphere ND running in production environments include several other services. In those configurations, there could be a chain of interdependencies. Depending on the applications hosted by the node instances, there may be an ordering requirement among them as well. With long service initialization times and crash-loop restarts, there is little chance for the application to start in any reasonable length of time.&lt;/p>
&lt;h3 id="sidecar-dependency-in-istio">Sidecar dependency in Istio&lt;/h3>
&lt;p>Istio itself is affected by a version of the dependency ordering problem. Since connections into and out of a service running under Istio are redirected through its sidecar proxy, an implicit dependency is created between the application service and its sidecar. Unless the sidecar is fully operational, all requests from and to the service get dropped.&lt;/p>
&lt;h2 id="dependency-ordering-with-appswitch">Dependency ordering with AppSwitch&lt;/h2>
&lt;p>So how do we go about addressing these issues? One way is to defer it to the applications and say that they are supposed to be &amp;ldquo;well behaved&amp;rdquo; and implement appropriate logic to make themselves immune to startup order issues. However, many applications (especially traditional ones) either timeout or deadlock if misordered. Even for new applications, implementing one off logic for each service is substantial additional burden that is best avoided. Service mesh needs to provide adequate support around these problems. After all, factoring out common patterns into an underlying framework is really the point of service mesh.&lt;/p>
&lt;p>&lt;a href="http://appswitch.io">AppSwitch&lt;/a> explicitly addresses dependency ordering. It sits on the control path of the application’s network interactions between clients and services in a cluster and knows precisely when a service becomes a client by making the &lt;code>connect&lt;/code> call and when a particular service becomes ready to accept connections by making the &lt;code>listen&lt;/code> call. It&amp;rsquo;s &lt;em>service router&lt;/em> component disseminates information about these events across the cluster and arbitrates interactions among clients and servers. That is how AppSwitch implements functionality such as load balancing and isolation in a simple and efficient manner. Leveraging the same strategic location of the application&amp;rsquo;s network control path, it is conceivable that the &lt;code>connect&lt;/code> and &lt;code>listen&lt;/code> calls made by those services can be lined up at a finer granularity rather than coarsely sequencing entire services as per a dependency graph. That would effectively solve the multilevel dependency problem and speedup application startup.&lt;/p>
&lt;p>But that still requires a dependency graph. A number of products and tools exist to help with discovering service dependencies. But they are typically based on passive monitoring of network traffic and cannot provide the information beforehand for any arbitrary application. Network level obfuscation due to encryption and tunneling also makes them unreliable. The burden of discovering and specifying the dependencies ultimately falls to the developer or the operator of the application. As it is, even consistency checking a dependency specification is itself quite complex and any way to avoid requiring a dependency graph would be most desirable.&lt;/p>
&lt;p>The point of a dependency graph is to know which clients depend on a particular service so that those clients can then be made to wait for the respective service to become live. But does it really matter which specific clients? Ultimately one tautology that always holds is that all clients of a service have an implicit dependency on the service. That’s what AppSwitch leverages to get around the requirement. In fact, that sidesteps dependency ordering altogether. All services of the application can be co-scheduled without regard to any startup order. Interdependencies among them automatically work themselves out at the granularity of individual requests and responses, resulting in quick and correct application startups.&lt;/p>
&lt;h3 id="appswitch-model-and-constructs">AppSwitch model and constructs&lt;/h3>
&lt;p>Now that we have a conceptual understanding of AppSwitch’s high-level approach, let’s look at the constructs involved. But first a quick summary of the usage model is in order. Even though it is written for a different context, reviewing my earlier &lt;a href="/v1.12/blog/2018/delayering-istio/">blog&lt;/a> on this topic would be useful as well. For completeness, let me also note AppSwitch doesn’t bother with non-network dependencies. For example it may be possible for two services to interact using IPC mechanisms or through the shared file system. Processes with deep ties like that are typically part of the same service anyway and don’t require framework’s intervention for ordering.&lt;/p>
&lt;p>At its core, AppSwitch is built on a mechanism that allows instrumenting the BSD socket API and other related calls like &lt;code>fcntl&lt;/code> and &lt;code>ioctl&lt;/code> that deal with sockets. As interesting as the details of its implementation are, it’s going to distract us from the main topic, so I’d just summarize the key properties that distinguish it from other implementations. (1) It’s fast. It uses a combination of &lt;code>seccomp&lt;/code> filtering and binary instrumentation to aggressively limit intervening with application’s normal execution. AppSwitch is particularly suited for service mesh and application networking use cases given that it implements those features without ever having to actually touch the data. In contrast, network level approaches incur per-packet cost. Take a look at this &lt;a href="/v1.12/blog/2018/delayering-istio/">blog&lt;/a> for some of the performance measurements. (2) It doesn’t require any kernel support, kernel module or a patch and works on standard distro kernels (3) It can run as regular user (no root). In fact, the mechanism can even make it possible to run &lt;a href="https://linuxpiter.com/en/materials/2478">Docker daemon without root&lt;/a> by removing root requirement to network containers (4) It doesn’t require any changes to the applications whatsoever and works for any type of application &amp;ndash; from WebSphere ND and SAP to custom C apps to statically linked Go apps. Only requirement at this point is Linux/x86.&lt;/p>
&lt;h3 id="decoupling-services-from-their-references">Decoupling services from their references&lt;/h3>
&lt;p>AppSwitch is built on the fundamental premise that applications should be decoupled from their references. The identity of applications is traditionally derived from the identity of the host on which they run. However, applications and hosts are very different objects that need to be referenced independently. Detailed discussion around this topic along with a conceptual foundation of AppSwitch is presented in this &lt;a href="https://arxiv.org/abs/1711.02294">research paper&lt;/a>.&lt;/p>
&lt;p>The central AppSwitch construct that achieves the decoupling between services objects and their identities is &lt;em>service reference&lt;/em> (&lt;em>reference&lt;/em>, for short). AppSwitch implements service references based on the API instrumentation mechanism outlined above. A service reference consists of an IP:port pair (and optionally a DNS name) and a label-selector that selects the service represented by the reference and the clients to which this reference applies. A reference supports a few key properties. (1) It can be named independently of the name of the object it refers to. That is, a service may be listening on an IP and port but a reference allows that service to be reached on any other IP and port chosen by the user. This is what allows AppSwitch to run traditional applications captured from their source environments with static IP configurations to run on Kubernetes by providing them with necessary IP addresses and ports regardless of the target network environment. (2) It remains unchanged even if the location of the target service changes. A reference automatically redirects itself as its label-selector now resolves to the new instance of the service (3) Most important for this discussion, a reference remains valid even as the target service is coming up.&lt;/p>
&lt;p>To facilitate discovering services that can be accessed through service references, AppSwitch provides an &lt;em>auto-curated service registry&lt;/em>. The registry is automatically kept up to date as services come and go across the cluster based on the network API that AppSwitch tracks. Each entry in the registry consists of the IP and port where the respective service is bound. Along with that, it includes a set of labels indicating the application to which this service belongs, the IP and port that the application passed through the socket API when creating the service, the IP and port where AppSwitch actually bound the service on the underlying host on behalf of the application etc. In addition, applications created under AppSwitch carry a set of labels passed by the user that describe the application together with a few default system labels indicating the user that created the application and the host where the application is running etc. These labels are all available to be expressed in the label-selector carried by a service reference. A service in the registry can be made accessible to clients by creating a service reference. A client would then be able to reach the service at the reference’s name (IP:port). Now let’s look at how AppSwitch guarantees that the reference remains valid even when the target service has not yet come up.&lt;/p>
&lt;h3 id="non-blocking-requests">Non-blocking requests&lt;/h3>
&lt;p>AppSwitch leverages the semantics of the BSD socket API to ensure that service references appear valid from the perspective of clients as corresponding services come up. When a client makes a blocking connect call to another service that has not yet come up, AppSwitch blocks the call for a certain time waiting for the target service to become live. Since it is known that the target service is a part of the application and is expected to come up shortly, making the client block rather than returning an error such as &lt;code>ECONNREFUSED&lt;/code> prevents the application from failing to start. If the service doesn’t come up within time, an error is returned to the application so that framework-level mechanisms like Kubernetes crash-loop can kick in.&lt;/p>
&lt;p>If the client request is marked as non-blocking, AppSwitch handles that by returning &lt;code>EAGAIN&lt;/code> to inform the application to retry rather than give up. Once again, that is in-line with the semantics of socket API and prevents failures due to startup races. AppSwitch essentially enables the retry logic already built into applications in support of the BSD socket API to be transparently repurposed for dependency ordering.&lt;/p>
&lt;h3 id="application-timeouts">Application timeouts&lt;/h3>
&lt;p>What if the application times out based on its own internal timer? Truth be told, AppSwitch can also fake application’s perception of time if wanted but that would be overstepping and actually unnecessary. Application decides and knows best how long it should wait and it’s not appropriate for AppSwitch to mess with that. Application timeouts are conservatively long and if the target service still hasn’t come up in time, it is unlikely to be a dependency ordering issue. There must be something else going on that should not be masked.&lt;/p>
&lt;h3 id="wildcard-service-references-for-sidecar-dependency">Wildcard service references for sidecar dependency&lt;/h3>
&lt;p>Service references can be used to address the Istio sidecar dependency issue mentioned earlier. AppSwitch allows the IP:port specified as part of a service reference to be a wildcard. That is, the service reference IP address can be a netmask indicating the IP address range to be captured. If the label selector of the service reference points to the sidecar service, then all outgoing connections of any application for which this service reference is applied, will be transparently redirected to the sidecar. And of course, the service reference remains valid while sidecar is still coming up and the race is removed.&lt;/p>
&lt;p>Using service references for sidecar dependency ordering also implicitly redirects application’s connections to the sidecar without requiring iptables and attendant privilege issues. Essentially it works as if the application is directly making connections to the sidecar rather than the target destination, leaving the sidecar in charge of what to do. AppSwitch would interject metadata about the original destination etc. into the data stream of the connection using the proxy protocol that the sidecar could decode before passing the connection through to the application. Some of these details were discussed &lt;a href="/v1.12/blog/2018/delayering-istio/">here&lt;/a>. That takes care of outbound connections but what about incoming connections? With all services and their sidecars running under AppSwitch, any incoming connections that would have come from remote nodes would be redirected to their respective remote sidecars. So nothing special to do about incoming connections.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>Dependency ordering is a pesky problem. This is mostly due to lack of access to fine-grain application-level events around inter-service interactions. Addressing this problem would have normally required applications to implement their own internal logic. But AppSwitch makes those internal application events to be instrumented without requiring application changes. AppSwitch then leverages the ubiquitous support for the BSD socket API to sidestep the requirement of ordering dependencies.&lt;/p>
&lt;h2 id="acknowledgements">Acknowledgements&lt;/h2>
&lt;p>Thanks to Eric Herness and team for their insights and support with IBM WebSphere and BPM products as we modernized them onto the Kubernetes platform and to Mandar Jog, Martin Taillefer and Shriram Rajagopalan for reviewing early drafts of this blog.&lt;/p></description><pubDate>Mon, 14 Jan 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/appswitch/</link><author>Dinesh Subhraveti (AppOrbit and Columbia University)</author><guid isPermaLink="true">/v1.12/blog/2019/appswitch/</guid><category>appswitch</category><category>performance</category></item><item><title>Deploy a Custom Ingress Gateway Using Cert-Manager</title><description>
&lt;p>This post provides instructions to manually create a custom ingress &lt;a href="/v1.12/docs/reference/config/networking/gateway/">gateway&lt;/a> with automatic provisioning of certificates based on cert-manager.&lt;/p>
&lt;p>The creation of custom ingress gateway could be used in order to have different &lt;code>loadbalancer&lt;/code> in order to isolate traffic.&lt;/p>
&lt;h2 id="before-you-begin">Before you begin&lt;/h2>
&lt;ul>
&lt;li>Set up Istio by following the instructions in the
&lt;a href="/v1.12/docs/setup/">Installation guide&lt;/a>.&lt;/li>
&lt;li>Set up &lt;code>cert-manager&lt;/code> with helm &lt;a href="https://github.com/helm/charts/tree/master/stable/cert-manager#installing-the-chart">chart&lt;/a>&lt;/li>
&lt;li>We will use &lt;code>demo.mydemo.com&lt;/code> for our example,
it must be resolved with your DNS&lt;/li>
&lt;/ul>
&lt;h2 id="configuring-the-custom-ingress-gateway">Configuring the custom ingress gateway&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Check if &lt;a href="https://github.com/helm/charts/tree/master/stable/cert-manager">cert-manager&lt;/a> was installed using Helm with the following command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ helm ls
&lt;/code>&lt;/pre>
&lt;p>The output should be similar to the example below and show cert-manager with a &lt;code>STATUS&lt;/code> of &lt;code>DEPLOYED&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
istio 1 Thu Oct 11 13:34:24 2018 DEPLOYED istio-1.0.X 1.0.X istio-system
cert 1 Wed Oct 24 14:08:36 2018 DEPLOYED cert-manager-v0.6.0-dev.2 v0.6.0-dev.2 istio-system
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>To create the cluster&amp;rsquo;s issuer, apply the following configuration:&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">Change the cluster&amp;rsquo;s &lt;a href="https://cert-manager.readthedocs.io/en/latest/reference/issuers.html">issuer&lt;/a> provider with your own configuration values. The example uses the values under &lt;code>route53&lt;/code>.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: letsencrypt-demo
namespace: kube-system
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: &amp;lt;REDACTED&amp;gt;
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-demo
dns01:
# Here we define a list of DNS-01 providers that can solve DNS challenges
providers:
- name: your-dns
route53:
accessKeyID: &amp;lt;REDACTED&amp;gt;
region: eu-central-1
secretAccessKeySecretRef:
name: prod-route53-credentials-secret
key: secret-access-key
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>If you use the &lt;code>route53&lt;/code> &lt;a href="https://cert-manager.readthedocs.io/en/latest/tasks/acme/configuring-dns01/route53.html">provider&lt;/a>, you must provide a secret to perform DNS ACME Validation. To create the secret, apply the following configuration file:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Secret
metadata:
name: prod-route53-credentials-secret
type: Opaque
data:
secret-access-key: &amp;lt;REDACTED BASE64&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create your own certificate:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: demo-certificate
namespace: istio-system
spec:
acme:
config:
- dns01:
provider: your-dns
domains:
- &amp;#39;*.mydemo.com&amp;#39;
commonName: &amp;#39;*.mydemo.com&amp;#39;
dnsNames:
- &amp;#39;*.mydemo.com&amp;#39;
issuerRef:
kind: ClusterIssuer
name: letsencrypt-demo
secretName: istio-customingressgateway-certs
&lt;/code>&lt;/pre>
&lt;p>Make a note of the value of &lt;code>secretName&lt;/code> since a future step requires it.&lt;/p>&lt;/li>
&lt;li>&lt;p>To scale automatically, declare a new horizontal pod autoscaler with the following configuration:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: my-ingressgateway
namespace: istio-system
spec:
maxReplicas: 5
minReplicas: 1
scaleTargetRef:
apiVersion: apps/v1beta1
kind: Deployment
name: my-ingressgateway
targetCPUUtilizationPercentage: 80
status:
currentCPUUtilizationPercentage: 0
currentReplicas: 1
desiredReplicas: 1
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Apply your deployment with declaration provided in the &lt;a href="/v1.12/blog/2019/custom-ingress-gateway/deployment-custom-ingress.yaml">yaml definition&lt;/a>&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">The annotations used, for example &lt;code>aws-load-balancer-type&lt;/code>, only apply for AWS.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;/li>
&lt;li>&lt;p>Create your service:&lt;/p>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">The &lt;code>NodePort&lt;/code> used needs to be an available port.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Service
metadata:
name: my-ingressgateway
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: nlb
labels:
app: my-ingressgateway
istio: my-ingressgateway
spec:
type: LoadBalancer
selector:
app: my-ingressgateway
istio: my-ingressgateway
ports:
-
name: http2
nodePort: 32380
port: 80
targetPort: 80
-
name: https
nodePort: 32390
port: 443
-
name: tcp
nodePort: 32400
port: 31400
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create your Istio custom gateway configuration object:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
annotations:
name: istio-custom-gateway
namespace: default
spec:
selector:
istio: my-ingressgateway
servers:
- hosts:
- &amp;#39;*.mydemo.com&amp;#39;
port:
name: http
number: 80
protocol: HTTP
tls:
httpsRedirect: true
- hosts:
- &amp;#39;*.mydemo.com&amp;#39;
port:
name: https
number: 443
protocol: HTTPS
tls:
mode: SIMPLE
privateKey: /etc/istio/ingressgateway-certs/tls.key
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Link your &lt;code>istio-custom-gateway&lt;/code> with your &lt;code>VirtualService&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: my-virtualservice
spec:
hosts:
- &amp;#34;demo.mydemo.com&amp;#34;
gateways:
- istio-custom-gateway
http:
- route:
- destination:
host: my-demoapp
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Correct certificate is returned by the server and it is successfully verified (&lt;em>SSL certificate verify ok&lt;/em> is printed):&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl -v `https://demo.mydemo.com`
Server certificate:
SSL certificate verify ok.
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;p>&lt;strong>Congratulations!&lt;/strong> You can now use your custom &lt;code>istio-custom-gateway&lt;/code> &lt;a href="/v1.12/docs/reference/config/networking/gateway/">gateway&lt;/a> configuration object.&lt;/p></description><pubDate>Thu, 10 Jan 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/custom-ingress-gateway/</link><author>Julien Senon</author><guid isPermaLink="true">/v1.12/blog/2019/custom-ingress-gateway/</guid><category>ingress</category><category>traffic-management</category></item><item><title>Announcing discuss.istio.io</title><description>&lt;p>We in the Istio community have been working to find the right medium for users to engage with other members of the community &amp;ndash; to ask questions,
to get help from other users, and to engage with developers working on the project.&lt;/p>
&lt;p>We’ve tried several different avenues, but each has had some downsides. RocketChat was our most recent endeavor, but the lack of certain
features (for example, threading) meant it wasn’t ideal for any longer discussions around a single issue. It also led to a dilemma for
some users &amp;ndash; when should I email istio-users@googlegroups.com and when should I use RocketChat?&lt;/p>
&lt;p>We think we’ve found the right balance of features in a single platform, and we’re happy to announce
&lt;a href="https://discuss.istio.io">discuss.istio.io&lt;/a>. It’s a full-featured forum where we will have discussions about Istio from here on out.
It will allow you to ask a question and get threaded replies! As a real bonus, you can use your GitHub identity.&lt;/p>
&lt;p>If you prefer emails, you can configure it to send emails just like Google groups did.&lt;/p>
&lt;p>We will be marking our Google groups &amp;ldquo;read only&amp;rdquo; so that the content remains, but we ask you to send further questions over to
&lt;a href="https://discuss.istio.io">discuss.istio.io&lt;/a>. If you have any outstanding questions or discussions in the groups, please move the conversation over.&lt;/p>
&lt;p>Happy meshing!&lt;/p></description><pubDate>Thu, 10 Jan 2019 00:00:00 +0000</pubDate><link>/v1.12/blog/2019/announcing-discuss.istio.io/</link><author/><guid isPermaLink="true">/v1.12/blog/2019/announcing-discuss.istio.io/</guid></item><item><title>Incremental Istio Part 1, Traffic Management</title><description>
&lt;p>Traffic management is one of the critical benefits provided by Istio. At the heart of Istio’s traffic management is the ability to decouple traffic flow and infrastructure scaling. This lets you control your traffic in ways that aren’t possible without a service mesh like Istio.&lt;/p>
&lt;p>For example, let’s say you want to execute a &lt;a href="https://martinfowler.com/bliki/CanaryRelease.html">canary deployment&lt;/a>. With Istio, you can specify that &lt;strong>v1&lt;/strong> of a service receives 90% of incoming traffic, while &lt;strong>v2&lt;/strong> of that service only receives 10%. With standard Kubernetes deployments, the only way to achieve this is to manually control the number of available Pods for each version, for example 9 Pods running v1 and 1 Pod running v2. This type of manual control is hard to implement, and over time may have trouble scaling. For more information, check out &lt;a href="/v1.12/blog/2017/0.1-canary/">Canary Deployments using Istio&lt;/a>.&lt;/p>
&lt;p>The same issue exists when deploying updates to existing services. While you can update deployments with Kubernetes, it requires replacing v1 Pods with v2 Pods. Using Istio, you can deploy v2 of your service and use built-in traffic management mechanisms to shift traffic to your updated services at a network level, then remove the v1 Pods.&lt;/p>
&lt;p>In addition to canary deployments and general traffic shifting, Istio also gives you the ability to implement dynamic request routing (based on HTTP headers), failure recovery, retries, circuit breakers, and fault injection. For more information, check out the &lt;a href="/v1.12/docs/concepts/traffic-management/">Traffic Management documentation&lt;/a>.&lt;/p>
&lt;p>This post walks through a technique that highlights a particularly useful way that you can implement Istio incrementally &amp;ndash; in this case, only the traffic management features &amp;ndash; without having to individually update each of your Pods.&lt;/p>
&lt;h2 id="setup-why-implement-istio-traffic-management-features">Setup: why implement Istio traffic management features?&lt;/h2>
&lt;p>Of course, the first question is: Why would you want to do this?&lt;/p>
&lt;p>If you’re part of one of the many organizations out there that have a large cluster with lots of teams deploying, the answer is pretty clear. Let’s say Team A is getting started with Istio and wants to start some canary deployments on Service A, but Team B hasn’t started using Istio, so they don’t have sidecars deployed.&lt;/p>
&lt;p>With Istio, Team A can still implement their canaries by having Service B call Service A through Istio’s ingress gateway.&lt;/p>
&lt;h2 id="background-traffic-routing-in-an-istio-mesh">Background: traffic routing in an Istio mesh&lt;/h2>
&lt;p>But how can you use Istio’s traffic management capabilities without updating each of your applications’ Pods to include the Istio sidecar? Before answering that question, let’s take a quick high-level look at how traffic enters an Istio mesh and how it’s routed.&lt;/p>
&lt;p>Pods that are part of the Istio mesh contain a sidecar proxy that is responsible for mediating all inbound and outbound traffic to the Pod. Within an Istio mesh, Pilot is responsible for converting high-level routing rules into configurations and propagating them to the sidecar proxies. That means when services communicate with one another, their routing decisions are determined from the client side.&lt;/p>
&lt;p>Let’s say you have two services that are part of the Istio mesh, Service A and Service B. When A wants to communicate with B, the sidecar proxy of Pod A is responsible for directing traffic to Service B. For example, if you wanted to split traffic &lt;sup>50&lt;/sup>&amp;frasl;&lt;sub>50&lt;/sub> across Service B v1 and v2, the traffic would flow as follows:&lt;/p>
&lt;figure style="width:60%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:42.66666666666667%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/incremental-traffic-management/fifty-fifty.png" title="50/50 Traffic Split">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/incremental-traffic-management/fifty-fifty.png" alt="50/50 Traffic Split" />
&lt;/a>
&lt;/div>
&lt;figcaption>50/50 Traffic Split&lt;/figcaption>
&lt;/figure>
&lt;p>If Services A and B are not part of the Istio mesh, there is no sidecar proxy that knows how to route traffic to different versions of Service B. In that case you need to use another approach to get traffic from Service A to Service B, following the &lt;sup>50&lt;/sup>&amp;frasl;&lt;sub>50&lt;/sub> rules you’ve setup.&lt;/p>
&lt;p>Fortunately, a standard Istio deployment already includes a &lt;a href="/v1.12/docs/concepts/traffic-management/#gateways">Gateway&lt;/a> that specifically deals with ingress traffic outside of the Istio mesh. This Gateway is used to allow ingress traffic from outside the cluster via an external load balancer, or to allow ingress traffic from within the Kubernetes cluster but outside the service mesh. It can be configured to proxy incoming ingress traffic to the appropriate Pods, even if they don’t have a sidecar proxy. While this approach allows you to leverage Istio’s traffic management features, it does mean that traffic going through the ingress gateway will incur an extra hop.&lt;/p>
&lt;figure style="width:60%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:54.83870967741935%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/incremental-traffic-management/fifty-fifty-ingress-gateway.png" title="50/50 Traffic Split using Ingress Gateway">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/incremental-traffic-management/fifty-fifty-ingress-gateway.png" alt="50/50 Traffic Split using Ingress Gateway" />
&lt;/a>
&lt;/div>
&lt;figcaption>50/50 Traffic Split using Ingress Gateway&lt;/figcaption>
&lt;/figure>
&lt;h2 id="in-action-traffic-routing-with-istio">In action: traffic routing with Istio&lt;/h2>
&lt;p>A simple way to see this type of approach in action is to first setup your Kubernetes environment using the &lt;a href="/v1.12/docs/setup/platform-setup/">Platform Setup&lt;/a> instructions, and then install the &lt;strong>minimal&lt;/strong> Istio profile using &lt;a href="https://archive.istio.io/1.4/docs/setup/install/helm/">Helm&lt;/a>, including only the traffic management components (ingress gateway, egress gateway, Pilot). The following example uses &lt;a href="https://cloud.google.com/gke">Google Kubernetes Engine&lt;/a>.&lt;/p>
&lt;p>First, setup and configure &lt;a href="/v1.12/docs/setup/platform-setup/gke/">GKE&lt;/a>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ gcloud container clusters create istio-inc --zone us-central1-f
$ gcloud container clusters get-credentials istio-inc
$ kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole=cluster-admin \
--user=$(gcloud config get-value core/account)
&lt;/code>&lt;/pre>
&lt;p>Next, &lt;a href="https://helm.sh/docs/intro/install/">install Helm&lt;/a> and &lt;a href="https://archive.istio.io/1.4/docs/setup/install/helm/">generate a minimal Istio install&lt;/a> &amp;ndash; only traffic management components:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ helm template install/kubernetes/helm/istio \
--name istio \
--namespace istio-system \
--set security.enabled=false \
--set galley.enabled=false \
--set sidecarInjectorWebhook.enabled=false \
--set mixer.enabled=false \
--set prometheus.enabled=false \
--set pilot.sidecar=false &amp;gt; istio-minimal.yaml
&lt;/code>&lt;/pre>
&lt;p>Then create the &lt;code>istio-system&lt;/code> namespace and deploy Istio:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create namespace istio-system
$ kubectl apply -f istio-minimal.yaml
&lt;/code>&lt;/pre>
&lt;p>Next, deploy the Bookinfo sample without the Istio sidecar containers:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo.yaml@
&lt;/code>&lt;/pre>&lt;/div>
&lt;p>Now, configure a new Gateway that allows access to the reviews service from outside the Istio mesh, a new &lt;code>VirtualService&lt;/code> that splits traffic evenly between v1 and v2 of the reviews service, and a set of new &lt;code>DestinationRule&lt;/code> resources that match destination subsets to service versions:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: reviews-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- &amp;#34;*&amp;#34;
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- &amp;#34;*&amp;#34;
gateways:
- reviews-gateway
http:
- match:
- uri:
prefix: /reviews
route:
- destination:
host: reviews
subset: v1
weight: 50
- destination:
host: reviews
subset: v2
weight: 50
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
- name: v3
labels:
version: v3
EOF
&lt;/code>&lt;/pre>
&lt;p>Finally, deploy a pod that you can use for testing with &lt;code>curl&lt;/code> (and without the Istio sidecar container):&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/sleep/sleep.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/sleep/sleep.yaml@
&lt;/code>&lt;/pre>&lt;/div>
&lt;h2 id="testing-your-deployment">Testing your deployment&lt;/h2>
&lt;p>Now, you can test different behaviors using the &lt;code>curl&lt;/code> commands via the sleep Pod.&lt;/p>
&lt;p>The first example is to issue requests to the reviews service using standard Kubernetes service DNS behavior (&lt;strong>note&lt;/strong>: &lt;a href="https://stedolan.github.io/jq/">&lt;code>jq&lt;/code>&lt;/a> is used in the examples below to filter the output from &lt;code>curl&lt;/code>):&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export SLEEP_POD=$(kubectl get pod -l app=sleep \
-o jsonpath={.items..metadata.name})
$ for i in `seq 3`; do \
kubectl exec -it $SLEEP_POD curl http://reviews:9080/reviews/0 | \
jq &amp;#39;.reviews|.[]|.rating?&amp;#39;; \
done
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;stars&amp;#34;: 5,
&amp;#34;color&amp;#34;: &amp;#34;black&amp;#34;
}
{
&amp;#34;stars&amp;#34;: 4,
&amp;#34;color&amp;#34;: &amp;#34;black&amp;#34;
}
null
null
{
&amp;#34;stars&amp;#34;: 5,
&amp;#34;color&amp;#34;: &amp;#34;red&amp;#34;
}
{
&amp;#34;stars&amp;#34;: 4,
&amp;#34;color&amp;#34;: &amp;#34;red&amp;#34;
}
&lt;/code>&lt;/pre>
&lt;p>Notice how we’re getting responses from all three versions of the reviews service (&lt;code>null&lt;/code> is from reviews v1 which doesn’t have ratings) and not getting the even split across v1 and v2. This is expected behavior because the &lt;code>curl&lt;/code> command is using Kubernetes service load balancing across all three versions of the reviews service. In order to access the reviews &lt;sup>50&lt;/sup>&amp;frasl;&lt;sub>50&lt;/sub> split we need to access the service via the ingress Gateway:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ for i in `seq 4`; do \
kubectl exec -it $SLEEP_POD curl http://istio-ingressgateway.istio-system/reviews/0 | \
jq &amp;#39;.reviews|.[]|.rating?&amp;#39;; \
done
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;stars&amp;#34;: 5,
&amp;#34;color&amp;#34;: &amp;#34;black&amp;#34;
}
{
&amp;#34;stars&amp;#34;: 4,
&amp;#34;color&amp;#34;: &amp;#34;black&amp;#34;
}
null
null
{
&amp;#34;stars&amp;#34;: 5,
&amp;#34;color&amp;#34;: &amp;#34;black&amp;#34;
}
{
&amp;#34;stars&amp;#34;: 4,
&amp;#34;color&amp;#34;: &amp;#34;black&amp;#34;
}
null
null
&lt;/code>&lt;/pre>
&lt;p>Mission accomplished! This post showed how to deploy a minimal installation of Istio that only contains the traffic management components (Pilot, ingress Gateway), and then use those components to direct traffic to specific versions of the reviews service. And it wasn&amp;rsquo;t necessary to deploy the Istio sidecar proxy to gain these capabilities, so there was little to no interruption of existing workloads or applications.&lt;/p>
&lt;p>Using the built-in ingress gateway (along with some &lt;code>VirtualService&lt;/code> and &lt;code>DestinationRule&lt;/code> resources) this post showed how you can easily leverage Istio’s traffic management for cluster-external ingress traffic and cluster-internal service-to-service traffic. This technique is a great example of an incremental approach to adopting Istio, and can be especially useful in real-world cases where Pods are owned by different teams or deployed to different namespaces.&lt;/p></description><pubDate>Wed, 21 Nov 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/incremental-traffic-management/</link><author>Sandeep Parikh</author><guid isPermaLink="true">/v1.12/blog/2018/incremental-traffic-management/</guid><category>traffic-management</category><category>gateway</category></item><item><title>Consuming External MongoDB Services</title><description>
&lt;p>In the &lt;a href="/v1.12/blog/2018/egress-tcp/">Consuming External TCP Services&lt;/a> blog post, I described how external services
can be consumed by in-mesh Istio applications via TCP. In this post, I demonstrate consuming external MongoDB services.
You use the &lt;a href="/v1.12/docs/examples/bookinfo/">Istio Bookinfo sample application&lt;/a>, the version in which the book
ratings data is persisted in a MongoDB database. You deploy this database outside the cluster and configure the
&lt;em>ratings&lt;/em> microservice to use it. You will learn multiple options of controlling traffic to external MongoDB services and their
pros and cons.&lt;/p>
&lt;h2 id="bookinfo-with-external-ratings-database">Bookinfo with external ratings database&lt;/h2>
&lt;p>First, you set up a MongoDB database instance to hold book ratings data outside of your Kubernetes cluster. Then you
modify the &lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo sample application&lt;/a> to use your database.&lt;/p>
&lt;h3 id="setting-up-the-ratings-database">Setting up the ratings database&lt;/h3>
&lt;p>For this task you set up an instance of &lt;a href="https://www.mongodb.com">MongoDB&lt;/a>. You can use any MongoDB instance; I used
&lt;a href="https://www.ibm.com/cloud/compose/mongodb">Compose for MongoDB&lt;/a>.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Set an environment variable for the password of your &lt;code>admin&lt;/code> user. To prevent the password from being preserved in
the Bash history, remove the command from the history immediately after running the command, using
&lt;a href="https://www.gnu.org/software/bash/manual/html_node/Bash-History-Builtins.html#Bash-History-Builtins">history -d&lt;/a>.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export MONGO_ADMIN_PASSWORD=&amp;lt;your MongoDB admin password&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Set an environment variable for the password of the new user you will create, namely &lt;code>bookinfo&lt;/code>.
Remove the command from the history using
&lt;a href="https://www.gnu.org/software/bash/manual/html_node/Bash-History-Builtins.html#Bash-History-Builtins">history -d&lt;/a>.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export BOOKINFO_PASSWORD=&amp;lt;password&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Set environment variables for your MongoDB service, &lt;code>MONGODB_HOST&lt;/code> and &lt;code>MONGODB_PORT&lt;/code>.&lt;/p>&lt;/li>
&lt;li>&lt;p>Create the &lt;code>bookinfo&lt;/code> user:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u admin -p $MONGO_ADMIN_PASSWORD --authenticationDatabase admin
use test
db.createUser(
{
user: &amp;#34;bookinfo&amp;#34;,
pwd: &amp;#34;$BOOKINFO_PASSWORD&amp;#34;,
roles: [ &amp;#34;read&amp;#34;]
}
);
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a &lt;em>collection&lt;/em> to hold ratings. The following command sets both ratings to be equal &lt;code>1&lt;/code> to provide a visual
clue when your database is used by the Bookinfo &lt;em>ratings&lt;/em> service (the default Bookinfo &lt;em>ratings&lt;/em> are &lt;code>4&lt;/code> and &lt;code>5&lt;/code>).&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u admin -p $MONGO_ADMIN_PASSWORD --authenticationDatabase admin
use test
db.createCollection(&amp;#34;ratings&amp;#34;);
db.ratings.insert(
[{rating: 1},
{rating: 1}]
);
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Check that &lt;code>bookinfo&lt;/code> user can get ratings:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u bookinfo -p $BOOKINFO_PASSWORD --authenticationDatabase test
use test
db.ratings.find({});
EOF
&lt;/code>&lt;/pre>
&lt;p>The output should be similar to:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >MongoDB server version: 3.4.10
switched to db test
{ &amp;#34;_id&amp;#34; : ObjectId(&amp;#34;5b7c29efd7596e65b6ed2572&amp;#34;), &amp;#34;rating&amp;#34; : 1 }
{ &amp;#34;_id&amp;#34; : ObjectId(&amp;#34;5b7c29efd7596e65b6ed2573&amp;#34;), &amp;#34;rating&amp;#34; : 1 }
bye
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h3 id="initial-setting-of-bookinfo-application">Initial setting of Bookinfo application&lt;/h3>
&lt;p>To demonstrate the scenario of using an external database, you start with a Kubernetes cluster with &lt;a href="/v1.12/docs/setup/getting-started/">Istio installed&lt;/a>. Then you deploy the
&lt;a href="/v1.12/docs/examples/bookinfo/">Istio Bookinfo sample application&lt;/a>, &lt;a href="/v1.12/docs/examples/bookinfo/#apply-default-destination-rules">apply the default destination rules&lt;/a>, and
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-control/#change-to-the-blocking-by-default-policy">change Istio to the blocking-egress-by-default policy&lt;/a>.&lt;/p>
&lt;p>This application uses the &lt;code>ratings&lt;/code> microservice to fetch book ratings, a number between 1 and 5. The ratings are
displayed as stars for each review. There are several versions of the &lt;code>ratings&lt;/code> microservice. You will deploy the
version that uses &lt;a href="https://www.mongodb.com">MongoDB&lt;/a> as the ratings database in the next subsection.&lt;/p>
&lt;p>The example commands in this blog post work with Istio 1.0.&lt;/p>
&lt;p>As a reminder, here is the end-to-end architecture of the application from the
&lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo sample application&lt;/a>.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.086918235567985%">
&lt;a data-skipendnotes="true" href="/v1.12/docs/examples/bookinfo/withistio.svg" title="The original Bookinfo application">
&lt;img class="element-to-stretch" src="/v1.12/docs/examples/bookinfo/withistio.svg" alt="The original Bookinfo application" />
&lt;/a>
&lt;/div>
&lt;figcaption>The original Bookinfo application&lt;/figcaption>
&lt;/figure>
&lt;h3 id="use-the-external-database-in-bookinfo-application">Use the external database in Bookinfo application&lt;/h3>
&lt;ol>
&lt;li>&lt;p>Deploy the spec of the &lt;em>ratings&lt;/em> microservice that uses a MongoDB database (&lt;em>ratings v2&lt;/em>):&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml@
serviceaccount &amp;#34;bookinfo-ratings-v2&amp;#34; created
deployment &amp;#34;ratings-v2&amp;#34; created
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Update the &lt;code>MONGO_DB_URL&lt;/code> environment variable to the value of your MongoDB:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl set env deployment/ratings-v2 &amp;#34;MONGO_DB_URL=mongodb://bookinfo:$BOOKINFO_PASSWORD@$MONGODB_HOST:$MONGODB_PORT/test?authSource=test&amp;amp;ssl=true&amp;#34;
deployment.extensions/ratings-v2 env updated
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Route all the traffic destined to the &lt;em>reviews&lt;/em> service to its &lt;em>v3&lt;/em> version. You do this to ensure that the
&lt;em>reviews&lt;/em> service always calls the &lt;em>ratings&lt;/em> service. In addition, route all the traffic destined to the &lt;em>ratings&lt;/em>
service to &lt;em>ratings v2&lt;/em> that uses your database.&lt;/p>
&lt;p>Specify the routing for both services above by adding two
&lt;a href="/v1.12/docs/reference/config/networking/virtual-service/">virtual services&lt;/a>. These virtual services are
specified in &lt;code>samples/bookinfo/networking/virtual-service-ratings-mongodb.yaml&lt;/code> of an Istio release archive.
&lt;strong>&lt;em>Important:&lt;/em>&lt;/strong> make sure you
&lt;a href="/v1.12/docs/examples/bookinfo/#apply-default-destination-rules">applied the default destination rules&lt;/a> before running the
following command.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-ratings-db.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/networking/virtual-service-ratings-db.yaml@
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;/ol>
&lt;p>The updated architecture appears below. Note that the blue arrows inside the mesh mark the traffic configured according
to the virtual services we added. According to the virtual services, the traffic is sent to &lt;em>reviews v3&lt;/em> and
&lt;em>ratings v2&lt;/em>.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.314858206480224%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-mongo/bookinfo-ratings-v2-mongodb-external.svg" title="The Bookinfo application with ratings v2 and an external MongoDB database">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-mongo/bookinfo-ratings-v2-mongodb-external.svg" alt="The Bookinfo application with ratings v2 and an external MongoDB database" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Bookinfo application with ratings v2 and an external MongoDB database&lt;/figcaption>
&lt;/figure>
&lt;p>Note that the MongoDB database is outside the Istio service mesh, or more precisely outside the Kubernetes cluster. The
boundary of the service mesh is marked by a dashed line.&lt;/p>
&lt;h3 id="access-the-webpage">Access the webpage&lt;/h3>
&lt;p>Access the webpage of the application, after
&lt;a href="/v1.12/docs/examples/bookinfo/#determine-the-ingress-ip-and-port">determining the ingress IP and port&lt;/a>.&lt;/p>
&lt;p>Since you did not configure the egress traffic control yet, the access to the MongoDB service is blocked by Istio.
This is why instead of the rating stars, the message &lt;em>&amp;ldquo;Ratings service is currently unavailable&amp;rdquo;&lt;/em> is currently
displayed below each review:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.18705035971223%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-mongo/errorFetchingBookRating.png" title="The Ratings service error messages">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-mongo/errorFetchingBookRating.png" alt="The Ratings service error messages" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Ratings service error messages&lt;/figcaption>
&lt;/figure>
&lt;p>In the following sections you will configure egress access to the external MongoDB service, using different options for
egress control in Istio.&lt;/p>
&lt;h2 id="egress-control-for-tcp">Egress control for TCP&lt;/h2>
&lt;p>Since &lt;a href="https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/">MongoDB Wire Protocol&lt;/a> runs on top of TCP, you
can control the egress traffic to your MongoDB as traffic to any other &lt;a href="/v1.12/blog/2018/egress-tcp/">external TCP service&lt;/a>. To
control TCP traffic, a block of IPs in the &lt;a href="https://tools.ietf.org/html/rfc2317">CIDR&lt;/a> notation that includes the IP
address of your MongoDB host must be specified. The caveat here is that sometimes the IP of the MongoDB host is not
stable or known in advance.&lt;/p>
&lt;p>In the cases when the IP of the MongoDB host is not stable, the egress traffic can either be
&lt;a href="#egress-control-for-tls">controlled as TLS traffic&lt;/a>, or the traffic can be routed
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-control/#direct-access-to-external-services">directly&lt;/a>, bypassing the Istio sidecar
proxies.&lt;/p>
&lt;p>Get the IP address of your MongoDB database instance. As an option, you can use the
&lt;a href="https://linux.die.net/man/1/host">host&lt;/a> command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export MONGODB_IP=$(host $MONGODB_HOST | grep &amp;#34; has address &amp;#34; | cut -d&amp;#34; &amp;#34; -f4)
&lt;/code>&lt;/pre>
&lt;h3 id="control-tcp-egress-traffic-without-a-gateway">Control TCP egress traffic without a gateway&lt;/h3>
&lt;p>In case you do not need to direct the traffic through an
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#use-case">egress gateway&lt;/a>, for example if you do not have a
requirement that all the traffic that exists your mesh must exit through the gateway, follow the
instructions in this section. Alternatively, if you do want to direct your traffic through an egress gateway, proceed to
&lt;a href="#direct-tcp-egress-traffic-through-an-egress-gateway">Direct TCP egress traffic through an egress gateway&lt;/a>.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Define a TCP mesh-external service entry:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mongo
spec:
hosts:
- my-mongo.tcp.svc
addresses:
- $MONGODB_IP/32
ports:
- number: $MONGODB_PORT
name: tcp
protocol: TCP
location: MESH_EXTERNAL
resolution: STATIC
endpoints:
- address: $MONGODB_IP
EOF
&lt;/code>&lt;/pre>
&lt;p>Note that the protocol &lt;code>TCP&lt;/code> is specified instead of &lt;code>MONGO&lt;/code> due to the fact that the traffic can be encrypted in
case &lt;a href="https://docs.mongodb.com/manual/tutorial/configure-ssl/">the MongoDB protocol runs on top of TLS&lt;/a>.
If the traffic is encrypted, the encrypted MongoDB protocol cannot be parsed by the Istio proxy.&lt;/p>
&lt;p>If you know that the plain MongoDB protocol is used, without encryption, you can specify the protocol as &lt;code>MONGO&lt;/code> and
let the Istio proxy produce
&lt;a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/mongo_proxy_filter#statistics">MongoDB related statistics&lt;/a>.
Also note that when the protocol &lt;code>TCP&lt;/code> is specified, the configuration is not specific for MongoDB, but is the same
for any other database with the protocol on top of TCP.&lt;/p>
&lt;p>Note that the host of your MongoDB is not used in TCP routing, so you can use any host, for example &lt;code>my-mongo.tcp.svc&lt;/code>. Notice the &lt;code>STATIC&lt;/code> resolution and the endpoint with the IP of your MongoDB service. Once you define such an endpoint, you can access MongoDB services that do not have a domain name.&lt;/p>&lt;/li>
&lt;li>&lt;p>Refresh the web page of the application. Now the application should display the ratings without error:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.69064748201439%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-mongo/externalDBRatings.png" title="Book Ratings Displayed Correctly">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-mongo/externalDBRatings.png" alt="Book Ratings Displayed Correctly" />
&lt;/a>
&lt;/div>
&lt;figcaption>Book Ratings Displayed Correctly&lt;/figcaption>
&lt;/figure>
&lt;p>Note that you see a one-star rating for both displayed reviews, as expected. You set the ratings to be one star to
provide yourself with a visual clue that your external database is indeed being used.&lt;/p>&lt;/li>
&lt;li>&lt;p>If you want to direct the traffic through an egress gateway, proceed to the next section. Otherwise, perform
&lt;a href="#cleanup-of-tcp-egress-traffic-control">cleanup&lt;/a>.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h3 id="direct-tcp-egress-traffic-through-an-egress-gateway">Direct TCP Egress traffic through an egress gateway&lt;/h3>
&lt;p>In this section you handle the case when you need to direct the traffic through an
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#use-case">egress gateway&lt;/a>. The sidecar proxy routes TCP
connections from the MongoDB client to the egress gateway, by matching the IP of the MongoDB host (a CIDR block of
length 32). The egress gateway forwards the traffic to the MongoDB host, by its hostname.&lt;/p>
&lt;ol>
&lt;li>&lt;p>&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#deploy-istio-egress-gateway">Deploy Istio egress gateway&lt;/a>.&lt;/p>&lt;/li>
&lt;li>&lt;p>If you did not perform the steps in &lt;a href="#control-tcp-egress-traffic-without-a-gateway">the previous section&lt;/a>, perform them now.&lt;/p>&lt;/li>
&lt;li>&lt;p>You may want to enable &lt;span class="term" data-title="Mutual TLS Authentication" data-body="&amp;lt;p&amp;gt;Mutual TLS provides strong service-to-service authentication with built-in identity and credential management.
&amp;lt;a href=&amp;#34;/docs/concepts/security/#mutual-tls-authentication&amp;#34;&amp;gt;Learn more about mutual TLS authentication&amp;lt;/a&amp;gt;.&amp;lt;/p&amp;gt;
">mutual TLS Authentication&lt;/span> between the sidecar proxies of
your MongoDB clients and the egress gateway to let the egress gateway monitor the identity of the source pods and to
enable Mixer policy enforcement based on that identity. By enabling mutual TLS you also encrypt the traffic.
If you do not want to enable mutual TLS, proceed to the &lt;a href="/v1.12/blog/2018/egress-mongo/#mutual-tls-between-the-sidecar-proxies-and-the-egress-gateway">Mutual TLS between the sidecar proxies and the egress gateway&lt;/a> section.
Otherwise, proceed to the following section.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h4 id="configure-tcp-traffic-from-sidecars-to-the-egress-gateway">Configure TCP traffic from sidecars to the egress gateway&lt;/h4>
&lt;ol>
&lt;li>&lt;p>Define the &lt;code>EGRESS_GATEWAY_MONGODB_PORT&lt;/code> environment variable to hold some port for directing traffic through
the egress gateway, e.g. &lt;code>7777&lt;/code>. You must select a port that is not used for any other service in the mesh.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export EGRESS_GATEWAY_MONGODB_PORT=7777
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Add the selected port to the &lt;code>istio-egressgateway&lt;/code> service. You should use the same values you used for installing
Istio, in particular you have to specify all the ports of the &lt;code>istio-egressgateway&lt;/code> service that you previously
configured.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ helm template install/kubernetes/helm/istio/ --name istio-egressgateway --namespace istio-system -x charts/gateways/templates/deployment.yaml -x charts/gateways/templates/service.yaml --set gateways.istio-ingressgateway.enabled=false --set gateways.istio-egressgateway.enabled=true --set gateways.istio-egressgateway.ports[0].port=80 --set gateways.istio-egressgateway.ports[0].name=http --set gateways.istio-egressgateway.ports[1].port=443 --set gateways.istio-egressgateway.ports[1].name=https --set gateways.istio-egressgateway.ports[2].port=$EGRESS_GATEWAY_MONGODB_PORT --set gateways.istio-egressgateway.ports[2].name=mongo | kubectl apply -f -
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Check that the &lt;code>istio-egressgateway&lt;/code> service indeed has the selected port:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get svc istio-egressgateway -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-egressgateway ClusterIP 172.21.202.204 &amp;lt;none&amp;gt; 80/TCP,443/TCP,7777/TCP 34d
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Disable mutual TLS authentication for the &lt;code>istio-egressgateway&lt;/code> service:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
name: istio-egressgateway
namespace: istio-system
spec:
targets:
- name: istio-egressgateway
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create an egress &lt;code>Gateway&lt;/code> for your MongoDB service, and destination rules and a virtual service to direct the
traffic through the egress gateway and from the egress gateway to the external service.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: $EGRESS_GATEWAY_MONGODB_PORT
name: tcp
protocol: TCP
hosts:
- my-mongo.tcp.svc
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-mongo
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: mongo
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: mongo
spec:
host: my-mongo.tcp.svc
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-mongo-through-egress-gateway
spec:
hosts:
- my-mongo.tcp.svc
gateways:
- mesh
- istio-egressgateway
tcp:
- match:
- gateways:
- mesh
destinationSubnets:
- $MONGODB_IP/32
port: $MONGODB_PORT
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: mongo
port:
number: $EGRESS_GATEWAY_MONGODB_PORT
- match:
- gateways:
- istio-egressgateway
port: $EGRESS_GATEWAY_MONGODB_PORT
route:
- destination:
host: my-mongo.tcp.svc
port:
number: $MONGODB_PORT
weight: 100
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>&lt;a href="#verify-that-egress-traffic-is-directed-through-the-egress-gateway">Verify that egress traffic is directed through the egress gateway&lt;/a>.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h4 id="mutual-tls-between-the-sidecar-proxies-and-the-egress-gateway">Mutual TLS between the sidecar proxies and the egress gateway&lt;/h4>
&lt;ol>
&lt;li>&lt;p>Delete the previous configuration:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete gateway istio-egressgateway --ignore-not-found=true
$ kubectl delete virtualservice direct-mongo-through-egress-gateway --ignore-not-found=true
$ kubectl delete destinationrule egressgateway-for-mongo mongo --ignore-not-found=true
$ kubectl delete policy istio-egressgateway -n istio-system --ignore-not-found=true
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Enforce mutual TLS authentication for the &lt;code>istio-egressgateway&lt;/code> service:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
name: istio-egressgateway
namespace: istio-system
spec:
targets:
- name: istio-egressgateway
peers:
- mtls: {}
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create an egress &lt;code>Gateway&lt;/code> for your MongoDB service, and destination rules and a virtual service
to direct the traffic through the egress gateway and from the egress gateway to the external service.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 443
name: tls
protocol: TLS
hosts:
- my-mongo.tcp.svc
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-mongo
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: mongo
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: ISTIO_MUTUAL
sni: my-mongo.tcp.svc
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: mongo
spec:
host: my-mongo.tcp.svc
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-mongo-through-egress-gateway
spec:
hosts:
- my-mongo.tcp.svc
gateways:
- mesh
- istio-egressgateway
tcp:
- match:
- gateways:
- mesh
destinationSubnets:
- $MONGODB_IP/32
port: $MONGODB_PORT
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: mongo
port:
number: 443
- match:
- gateways:
- istio-egressgateway
port: 443
route:
- destination:
host: my-mongo.tcp.svc
port:
number: $MONGODB_PORT
weight: 100
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Proceed to the next section.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h4 id="verify-that-egress-traffic-is-directed-through-the-egress-gateway">Verify that egress traffic is directed through the egress gateway&lt;/h4>
&lt;ol>
&lt;li>&lt;p>Refresh the web page of the application again and verify that the ratings are still displayed correctly.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="/v1.12/docs/tasks/observability/logs/access-log/#enable-envoy-s-access-logging">Enable Envoy’s access logging&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>Check the log of the egress gateway&amp;rsquo;s Envoy and see a line that corresponds to your
requests to the MongoDB service. If Istio is deployed in the &lt;code>istio-system&lt;/code> namespace, the command to print the
log is:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl logs -l istio=egressgateway -n istio-system
[2019-04-14T06:12:07.636Z] &amp;#34;- - -&amp;#34; 0 - &amp;#34;-&amp;#34; 1591 4393 94 - &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;&amp;lt;Your MongoDB IP&amp;gt;:&amp;lt;your MongoDB port&amp;gt;&amp;#34; outbound|&amp;lt;your MongoDB port&amp;gt;||my-mongo.tcp.svc 172.30.146.119:59924 172.30.146.119:443 172.30.230.1:59206 -
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h3 id="cleanup-of-tcp-egress-traffic-control">Cleanup of TCP egress traffic control&lt;/h3>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry mongo
$ kubectl delete gateway istio-egressgateway --ignore-not-found=true
$ kubectl delete virtualservice direct-mongo-through-egress-gateway --ignore-not-found=true
$ kubectl delete destinationrule egressgateway-for-mongo mongo --ignore-not-found=true
$ kubectl delete policy istio-egressgateway -n istio-system --ignore-not-found=true
&lt;/code>&lt;/pre>
&lt;h2 id="egress-control-for-tls">Egress control for TLS&lt;/h2>
&lt;p>In the real life, most of the communication to the external services must be encrypted and
&lt;a href="https://docs.mongodb.com/manual/tutorial/configure-ssl/">the MongoDB protocol runs on top of TLS&lt;/a>.
Also, the TLS clients usually send
&lt;a href="https://en.wikipedia.org/wiki/Server_Name_Indication">Server Name Indication&lt;/a>, SNI, as part of their handshake. If your
MongoDB server runs TLS and your MongoDB client sends SNI as part of the handshake, you can control your MongoDB egress
traffic as any other TLS-with-SNI traffic. With TLS and SNI, you do not need to specify the IP addresses of your MongoDB
servers. You specify their host names instead, which is more convenient since you do not have to rely on the stability of
the IP addresses. You can also specify wildcards as a prefix of the host names, for example allowing access to any
server from the &lt;code>*.com&lt;/code> domain.&lt;/p>
&lt;p>To check if your MongoDB server supports TLS, run:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ openssl s_client -connect $MONGODB_HOST:$MONGODB_PORT -servername $MONGODB_HOST
&lt;/code>&lt;/pre>
&lt;p>If the command above prints a certificate returned by the server, the server supports TLS. If not, you have to control
your MongoDB egress traffic on the TCP level, as described in the previous sections.&lt;/p>
&lt;h3 id="control-tls-egress-traffic-without-a-gateway">Control TLS egress traffic without a gateway&lt;/h3>
&lt;p>In case you &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#use-case">do not need an egress gateway&lt;/a>, follow the
instructions in this section. If you want to direct your traffic through an egress gateway, proceed to
&lt;a href="#direct-tcp-egress-traffic-through-an-egress-gateway">Direct TCP Egress traffic through an egress gateway&lt;/a>.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Create a &lt;code>ServiceEntry&lt;/code> for the MongoDB service:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mongo
spec:
hosts:
- $MONGODB_HOST
ports:
- number: $MONGODB_PORT
name: tls
protocol: TLS
resolution: DNS
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Refresh the web page of the application. The application should display the ratings without error.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h4 id="cleanup-of-the-egress-configuration-for-tls">Cleanup of the egress configuration for TLS&lt;/h4>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry mongo
&lt;/code>&lt;/pre>
&lt;h3 id="direct-tls-egress-traffic-through-an-egress-gateway">Direct TLS Egress traffic through an egress gateway&lt;/h3>
&lt;p>In this section you handle the case when you need to direct the traffic through an
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#use-case">egress gateway&lt;/a>. The sidecar proxy routes TLS
connections from the MongoDB client to the egress gateway, by matching the SNI of the MongoDB host.
The egress gateway forwards the traffic to the MongoDB host. Note that the sidecar proxy rewrites the destination port
to be 443. The egress gateway accepts the MongoDB traffic on the port 443, matches the MongoDB host by SNI, and rewrites
the port again to be the port of the MongoDB server.&lt;/p>
&lt;ol>
&lt;li>&lt;p>&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#deploy-istio-egress-gateway">Deploy Istio egress gateway&lt;/a>.&lt;/p>&lt;/li>
&lt;li>&lt;p>Create a &lt;code>ServiceEntry&lt;/code> for the MongoDB service:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mongo
spec:
hosts:
- $MONGODB_HOST
ports:
- number: $MONGODB_PORT
name: tls
protocol: TLS
- number: 443
name: tls-port-for-egress-gateway
protocol: TLS
resolution: DNS
location: MESH_EXTERNAL
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Refresh the web page of the application and verify that the ratings are displayed correctly.&lt;/p>&lt;/li>
&lt;li>&lt;p>Create an egress &lt;code>Gateway&lt;/code> for your MongoDB service, and destination rules and virtual services
to direct the traffic through the egress gateway and from the egress gateway to the external service.&lt;/p>
&lt;p>If you want to enable &lt;a href="/v1.12/docs/tasks/security/authentication/authn-policy/">mutual TLS Authentication&lt;/a> between the sidecar proxies of
your application pods and the egress gateway, use the following command. (You may want to enable mutual TLS to let
the egress gateway monitor the identity of the source pods and to enable Mixer policy enforcement based on that
identity.)&lt;/p>
&lt;div id="tabset-blog-2018-egress-mongo-1" role="tablist" class="tabset ">
&lt;div class="tab-strip" data-category-name="mtls" >&lt;button aria-selected="true" data-category-value="enabled"
aria-controls="tabset-blog-2018-egress-mongo-1-0-panel" id="tabset-blog-2018-egress-mongo-1-0-tab" role="tab">&lt;span>mutual TLS enabled&lt;/span>
&lt;/button>&lt;button tabindex="-1" data-category-value="disabled"
aria-controls="tabset-blog-2018-egress-mongo-1-1-panel" id="tabset-blog-2018-egress-mongo-1-1-tab" role="tab">&lt;span>mutual TLS disabled&lt;/span>
&lt;/button>&lt;/div>
&lt;div class="tab-content">&lt;div id="tabset-blog-2018-egress-mongo-1-0-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2018-egress-mongo-1-0-tab">
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 443
name: tls
protocol: TLS
hosts:
- $MONGODB_HOST
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-mongo
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: mongo
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: ISTIO_MUTUAL
sni: $MONGODB_HOST
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-mongo-through-egress-gateway
spec:
hosts:
- $MONGODB_HOST
gateways:
- mesh
- istio-egressgateway
tls:
- match:
- gateways:
- mesh
port: $MONGODB_PORT
sni_hosts:
- $MONGODB_HOST
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: mongo
port:
number: 443
tcp:
- match:
- gateways:
- istio-egressgateway
port: 443
route:
- destination:
host: $MONGODB_HOST
port:
number: $MONGODB_PORT
weight: 100
EOF
&lt;/code>&lt;/pre>
&lt;/div>&lt;div hidden id="tabset-blog-2018-egress-mongo-1-1-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2018-egress-mongo-1-1-tab">
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 443
name: tls
protocol: TLS
hosts:
- $MONGODB_HOST
tls:
mode: PASSTHROUGH
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-mongo
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: mongo
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-mongo-through-egress-gateway
spec:
hosts:
- $MONGODB_HOST
gateways:
- mesh
- istio-egressgateway
tls:
- match:
- gateways:
- mesh
port: $MONGODB_PORT
sni_hosts:
- $MONGODB_HOST
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: mongo
port:
number: 443
- match:
- gateways:
- istio-egressgateway
port: 443
sni_hosts:
- $MONGODB_HOST
route:
- destination:
host: $MONGODB_HOST
port:
number: $MONGODB_PORT
weight: 100
EOF
&lt;/code>&lt;/pre>
&lt;/div>&lt;/div>
&lt;/div>
&lt;/li>
&lt;li>&lt;p>&lt;a href="#verify-that-egress-traffic-is-directed-through-the-egress-gateway">Verify that the traffic is directed though the egress gateway&lt;/a>&lt;/p>&lt;/li>
&lt;/ol>
&lt;h4 id="cleanup-directing-tls-egress-traffic-through-an-egress-gateway">Cleanup directing TLS egress traffic through an egress gateway&lt;/h4>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry mongo
$ kubectl delete gateway istio-egressgateway
$ kubectl delete virtualservice direct-mongo-through-egress-gateway
$ kubectl delete destinationrule egressgateway-for-mongo
&lt;/code>&lt;/pre>
&lt;h3 id="enable-mongodb-tls-egress-traffic-to-arbitrary-wildcarded-domains">Enable MongoDB TLS egress traffic to arbitrary wildcarded domains&lt;/h3>
&lt;p>Sometimes you want to configure egress traffic to multiple hostnames from the same domain, for example traffic to all
MongoDB services from &lt;code>*.&amp;lt;your company domain&amp;gt;.com&lt;/code>. You do not want to create multiple configuration items, one for
each and every MongoDB service in your company. To configure access to all the external services from the same domain by
a single configuration, you use &lt;em>wildcarded&lt;/em> hosts.&lt;/p>
&lt;p>In this section you configure egress traffic for a wildcarded domain. I used a MongoDB instance at &lt;code>composedb.com&lt;/code>
domain, so configuring egress traffic for &lt;code>*.com&lt;/code> worked for me (I could have used &lt;code>*.composedb.com&lt;/code> as well).
You can pick a wildcarded domain according to your MongoDB host.&lt;/p>
&lt;p>To configure egress gateway traffic for a wildcarded domain, you will first need to deploy a custom egress
gateway with
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/wildcard-egress-hosts/#wildcard-configuration-for-arbitrary-domains">an additional SNI proxy&lt;/a>.
This is needed due to current limitations of Envoy, the proxy used by the standard Istio egress gateway.&lt;/p>
&lt;h4 id="prepare-a-new-egress-gateway-with-an-sni-proxy">Prepare a new egress gateway with an SNI proxy&lt;/h4>
&lt;p>In this subsection you deploy an egress gateway with an SNI proxy, in addition to the standard Istio Envoy proxy. You
can use any SNI proxy that is capable of routing traffic according to arbitrary, not-preconfigured SNI values; we used
&lt;a href="http://nginx.org">Nginx&lt;/a> to achieve this functionality.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Create a configuration file for the Nginx SNI proxy. You may want to edit the file to specify additional Nginx
settings, if required.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF &amp;gt; ./sni-proxy.conf
user www-data;
events {
}
stream {
log_format log_stream &amp;#39;\$remote_addr [\$time_local] \$protocol [\$ssl_preread_server_name]&amp;#39;
&amp;#39;\$status \$bytes_sent \$bytes_received \$session_time&amp;#39;;
access_log /var/log/nginx/access.log log_stream;
error_log /var/log/nginx/error.log;
# tcp forward proxy by SNI
server {
resolver 8.8.8.8 ipv6=off;
listen 127.0.0.1:$MONGODB_PORT;
proxy_pass \$ssl_preread_server_name:$MONGODB_PORT;
ssl_preread on;
}
}
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a Kubernetes &lt;a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/">ConfigMap&lt;/a>
to hold the configuration of the Nginx SNI proxy:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl create configmap egress-sni-proxy-configmap -n istio-system --from-file=nginx.conf=./sni-proxy.conf
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>The following command will generate &lt;code>istio-egressgateway-with-sni-proxy.yaml&lt;/code> to edit and deploy.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | helm template install/kubernetes/helm/istio/ --name istio-egressgateway-with-sni-proxy --namespace istio-system -x charts/gateways/templates/deployment.yaml -x charts/gateways/templates/service.yaml -x charts/gateways/templates/serviceaccount.yaml -x charts/gateways/templates/autoscale.yaml -x charts/gateways/templates/role.yaml -x charts/gateways/templates/rolebindings.yaml --set global.mtls.enabled=true --set global.istioNamespace=istio-system -f - &amp;gt; ./istio-egressgateway-with-sni-proxy.yaml
gateways:
enabled: true
istio-ingressgateway:
enabled: false
istio-egressgateway:
enabled: false
istio-egressgateway-with-sni-proxy:
enabled: true
labels:
app: istio-egressgateway-with-sni-proxy
istio: egressgateway-with-sni-proxy
replicaCount: 1
autoscaleMin: 1
autoscaleMax: 5
cpu:
targetAverageUtilization: 80
serviceAnnotations: {}
type: ClusterIP
ports:
- port: 443
name: https
secretVolumes:
- name: egressgateway-certs
secretName: istio-egressgateway-certs
mountPath: /etc/istio/egressgateway-certs
- name: egressgateway-ca-certs
secretName: istio-egressgateway-ca-certs
mountPath: /etc/istio/egressgateway-ca-certs
configVolumes:
- name: sni-proxy-config
configMapName: egress-sni-proxy-configmap
additionalContainers:
- name: sni-proxy
image: nginx
volumeMounts:
- name: sni-proxy-config
mountPath: /etc/nginx
readOnly: true
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Deploy the new egress gateway:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f ./istio-egressgateway-with-sni-proxy.yaml
serviceaccount &amp;#34;istio-egressgateway-with-sni-proxy-service-account&amp;#34; created
role &amp;#34;istio-egressgateway-with-sni-proxy-istio-system&amp;#34; created
rolebinding &amp;#34;istio-egressgateway-with-sni-proxy-istio-system&amp;#34; created
service &amp;#34;istio-egressgateway-with-sni-proxy&amp;#34; created
deployment &amp;#34;istio-egressgateway-with-sni-proxy&amp;#34; created
horizontalpodautoscaler &amp;#34;istio-egressgateway-with-sni-proxy&amp;#34; created
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Verify that the new egress gateway is running. Note that the pod has two containers (one is the Envoy proxy and the
second one is the SNI proxy).&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pod -l istio=egressgateway-with-sni-proxy -n istio-system
NAME READY STATUS RESTARTS AGE
istio-egressgateway-with-sni-proxy-79f6744569-pf9t2 2/2 Running 0 17s
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a service entry with a static address equal to 127.0.0.1 (&lt;code>localhost&lt;/code>), and disable mutual TLS on the traffic directed to the new
service entry:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: sni-proxy
spec:
hosts:
- sni-proxy.local
location: MESH_EXTERNAL
ports:
- number: $MONGODB_PORT
name: tcp
protocol: TCP
resolution: STATIC
endpoints:
- address: 127.0.0.1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: disable-mtls-for-sni-proxy
spec:
host: sni-proxy.local
trafficPolicy:
tls:
mode: DISABLE
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h4 id="configure-access-to-com-using-the-new-egress-gateway">Configure access to &lt;code>*.com&lt;/code> using the new egress gateway&lt;/h4>
&lt;ol>
&lt;li>&lt;p>Define a &lt;code>ServiceEntry&lt;/code> for &lt;code>*.com&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl create -f -
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mongo
spec:
hosts:
- &amp;#34;*.com&amp;#34;
ports:
- number: 443
name: tls
protocol: TLS
- number: $MONGODB_PORT
name: tls-mongodb
protocol: TLS
location: MESH_EXTERNAL
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create an egress &lt;code>Gateway&lt;/code> for &lt;em>*.com&lt;/em>, port 443, protocol TLS, a destination rule to set the
&lt;a href="https://en.wikipedia.org/wiki/Server_Name_Indication">SNI&lt;/a> for the gateway, and Envoy filters to prevent tampering
with SNI by a malicious application (the filters verify that the SNI issued by the application is the SNI reported
to Mixer).&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway-with-sni-proxy
spec:
selector:
istio: egressgateway-with-sni-proxy
servers:
- port:
number: 443
name: tls
protocol: TLS
hosts:
- &amp;#34;*.com&amp;#34;
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: mtls-for-egress-gateway
spec:
host: istio-egressgateway-with-sni-proxy.istio-system.svc.cluster.local
subsets:
- name: mongo
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: ISTIO_MUTUAL
---
# The following filter is used to forward the original SNI (sent by the application) as the SNI of the mutual TLS
# connection.
# The forwarded SNI will be reported to Mixer so that policies will be enforced based on the original SNI value.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: forward-downstream-sni
spec:
filters:
- listenerMatch:
portNumber: $MONGODB_PORT
listenerType: SIDECAR_OUTBOUND
filterName: forward_downstream_sni
filterType: NETWORK
filterConfig: {}
---
# The following filter verifies that the SNI of the mutual TLS connection (the SNI reported to Mixer) is
# identical to the original SNI issued by the application (the SNI used for routing by the SNI proxy).
# The filter prevents Mixer from being deceived by a malicious application: routing to one SNI while
# reporting some other value of SNI. If the original SNI does not match the SNI of the mutual TLS connection, the
# filter will block the connection to the external service.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: egress-gateway-sni-verifier
spec:
workloadLabels:
app: istio-egressgateway-with-sni-proxy
filters:
- listenerMatch:
portNumber: 443
listenerType: GATEWAY
filterName: sni_verifier
filterType: NETWORK
filterConfig: {}
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Route the traffic destined for &lt;em>*.com&lt;/em> to the egress gateway and from the egress gateway to the SNI proxy.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-mongo-through-egress-gateway
spec:
hosts:
- &amp;#34;*.com&amp;#34;
gateways:
- mesh
- istio-egressgateway-with-sni-proxy
tls:
- match:
- gateways:
- mesh
port: $MONGODB_PORT
sni_hosts:
- &amp;#34;*.com&amp;#34;
route:
- destination:
host: istio-egressgateway-with-sni-proxy.istio-system.svc.cluster.local
subset: mongo
port:
number: 443
weight: 100
tcp:
- match:
- gateways:
- istio-egressgateway-with-sni-proxy
port: 443
route:
- destination:
host: sni-proxy.local
port:
number: $MONGODB_PORT
weight: 100
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Refresh the web page of the application again and verify that the ratings are still displayed correctly.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="/v1.12/docs/tasks/observability/logs/access-log/#enable-envoy-s-access-logging">Enable Envoy’s access logging&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>Check the log of the egress gateway&amp;rsquo;s Envoy proxy. If Istio is deployed in the &lt;code>istio-system&lt;/code> namespace, the command
to print the log is:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl logs -l istio=egressgateway-with-sni-proxy -c istio-proxy -n istio-system
&lt;/code>&lt;/pre>
&lt;p>You should see lines similar to the following:&lt;/p>
&lt;pre>&lt;code class='language-plain' data-expandlinks='true' data-repo='istio' >[2019-01-02T17:22:04.602Z] &amp;#34;- - -&amp;#34; 0 - 768 1863 88 - &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;127.0.0.1:28543&amp;#34; outbound|28543||sni-proxy.local 127.0.0.1:49976 172.30.146.115:443 172.30.146.118:58510 &amp;lt;your MongoDB host&amp;gt;
[2019-01-02T17:22:04.713Z] &amp;#34;- - -&amp;#34; 0 - 1534 2590 85 - &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;-&amp;#34; &amp;#34;127.0.0.1:28543&amp;#34; outbound|28543||sni-proxy.local 127.0.0.1:49988 172.30.146.115:443 172.30.146.118:58522 &amp;lt;your MongoDB host&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Check the logs of the SNI proxy. If Istio is deployed in the &lt;code>istio-system&lt;/code> namespace, the command to print the
log is:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl logs -l istio=egressgateway-with-sni-proxy -n istio-system -c sni-proxy
127.0.0.1 [23/Aug/2018:03:28:18 +0000] TCP [&amp;lt;your MongoDB host&amp;gt;]200 1863 482 0.089
127.0.0.1 [23/Aug/2018:03:28:18 +0000] TCP [&amp;lt;your MongoDB host&amp;gt;]200 2590 1248 0.095
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h4 id="understanding-what-happened">Understanding what happened&lt;/h4>
&lt;p>In this section you configured egress traffic to your MongoDB host using a wildcarded domain. While for a single MongoDB
host there is no gain in using wildcarded domains (an exact hostname can be specified), it could be beneficial for
cases when the applications in the cluster access multiple MongoDB hosts that match some wildcarded domain. For example,
if the applications access &lt;code>mongodb1.composedb.com&lt;/code>, &lt;code>mongodb2.composedb.com&lt;/code> and &lt;code>mongodb3.composedb.com&lt;/code>, the egress
traffic can be configured by a single configuration for the wildcarded domain &lt;code>*.composedb.com&lt;/code>.&lt;/p>
&lt;p>I will leave it as an exercise for the reader to verify that no additional Istio configuration is required when you
configure an app to use another instance of MongoDB with a hostname that matches the wildcarded domain used in this
section.&lt;/p>
&lt;h4 id="cleanup-of-configuration-for-mongodb-tls-egress-traffic-to-arbitrary-wildcarded-domains">Cleanup of configuration for MongoDB TLS egress traffic to arbitrary wildcarded domains&lt;/h4>
&lt;ol>
&lt;li>&lt;p>Delete the configuration items for &lt;em>*.com&lt;/em>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry mongo
$ kubectl delete gateway istio-egressgateway-with-sni-proxy
$ kubectl delete virtualservice direct-mongo-through-egress-gateway
$ kubectl delete destinationrule mtls-for-egress-gateway
$ kubectl delete envoyfilter forward-downstream-sni egress-gateway-sni-verifier
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Delete the configuration items for the &lt;code>egressgateway-with-sni-proxy&lt;/code> deployment:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry sni-proxy
$ kubectl delete destinationrule disable-mtls-for-sni-proxy
$ kubectl delete -f ./istio-egressgateway-with-sni-proxy.yaml
$ kubectl delete configmap egress-sni-proxy-configmap -n istio-system
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Remove the configuration files you created:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ rm ./istio-egressgateway-with-sni-proxy.yaml
$ rm ./nginx-sni-proxy.conf
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h2 id="cleanup">Cleanup&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Drop the &lt;code>bookinfo&lt;/code> user:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u admin -p $MONGO_ADMIN_PASSWORD --authenticationDatabase admin
use test
db.dropUser(&amp;#34;bookinfo&amp;#34;);
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Drop the &lt;em>ratings&lt;/em> collection:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u admin -p $MONGO_ADMIN_PASSWORD --authenticationDatabase admin
use test
db.ratings.drop();
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Unset the environment variables you used:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ unset MONGO_ADMIN_PASSWORD BOOKINFO_PASSWORD MONGODB_HOST MONGODB_PORT MONGODB_IP
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Remove the virtual services:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-ratings-db.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete -f @samples/bookinfo/networking/virtual-service-ratings-db.yaml@
Deleted config: virtual-service/default/reviews
Deleted config: virtual-service/default/ratings
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Undeploy &lt;em>ratings v2-mongodb&lt;/em>:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete -f @samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml@
deployment &amp;#34;ratings-v2&amp;#34; deleted
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;/ol>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>In this blog post I demonstrated various options for MongoDB egress traffic control. You can control the MongoDB egress
traffic on a TCP or TLS level where applicable. In both TCP and TLS cases, you can direct the traffic from the sidecar
proxies directly to the external MongoDB host, or direct the traffic through an egress gateway, according to your
organization&amp;rsquo;s security requirements. In the latter case, you can also decide to apply or disable mutual TLS
authentication between the sidecar proxies and the egress gateway. If you want to control MongoDB egress traffic on the
TLS level by specifying wildcarded domains like &lt;code>*.com&lt;/code> and you need to direct the traffic through the egress gateway,
you must deploy a custom egress gateway with an SNI proxy.&lt;/p>
&lt;p>Note that the configuration and considerations described in this blog post for MongoDB are rather the same for other
non-HTTP protocols on top of TCP/TLS.&lt;/p></description><pubDate>Fri, 16 Nov 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/egress-mongo/</link><author>Vadim Eisenberg</author><guid isPermaLink="true">/v1.12/blog/2018/egress-mongo/</guid><category>traffic-management</category><category>egress</category><category>tcp</category><category>mongo</category></item><item><title>All Day Istio Twitch Stream</title><description>
&lt;p>To celebrate the 1.0 release and to promote the software to a wider audience, the Istio community is hosting an all day live stream on Twitch on August 17th.&lt;/p>
&lt;h2 id="what-is-twitch">What is Twitch?&lt;/h2>
&lt;p>&lt;a href="https://twitch.tv/">Twitch&lt;/a> is a popular video gaming live streaming platform and recently has seen a lot of coding content showing up. The IBM Advocates have been doing live coding and presentations there and it&amp;rsquo;s been fun. While mostly used for gaming content, there is a &lt;a href="https://www.twitch.tv/communities/programming">growing community&lt;/a> sharing and watching programming content on the site.&lt;/p>
&lt;h2 id="what-does-this-have-to-do-with-istio">What does this have to do with Istio?&lt;/h2>
&lt;p>The stream is going to be a full day of Istio content. Hopefully we&amp;rsquo;ll have a good mix of deep technical content, beginner content and line-of-business content for our audience. We&amp;rsquo;ll have developers, users, and evangelists on throughout the day to share their demos and stories. Expect live coding, q and a, and some surprises. We have stellar guests lined up from IBM, Google, Datadog, Pivotal, and more!&lt;/p>
&lt;h2 id="recordings">Recordings&lt;/h2>
&lt;p>Recordings are available &lt;a href="https://www.youtube.com/playlist?list=PLzpeuWUENMK0V3dwpx5gPJun-SLG0USqU">here&lt;/a>.&lt;/p>
&lt;h2 id="schedule">Schedule&lt;/h2>
&lt;p>All times are &lt;code>PDT&lt;/code>.&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Time&lt;/th>
&lt;th>Speaker&lt;/th>
&lt;th>Affiliation&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>10:00 - 10:30&lt;/td>
&lt;td>&lt;code>Spencer Krum + Lisa-Marie Namphy&lt;/code>&lt;/td>
&lt;td>&lt;code>IBM / Portworx&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>10:30 - 11:00&lt;/td>
&lt;td>&lt;code>Lin Sun / Spencer Krum / Sven Mawson&lt;/code>&lt;/td>
&lt;td>&lt;code>IBM / Google&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>11:00 - 11:10&lt;/td>
&lt;td>&lt;code>Lin Sun / Spencer Krum&lt;/code>&lt;/td>
&lt;td>&lt;code>IBM&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>11:10 - 11:30&lt;/td>
&lt;td>&lt;code>Jason Yee / Ilan Rabinovich&lt;/code>&lt;/td>
&lt;td>&lt;code>Datadog&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>11:30 - 11:50&lt;/td>
&lt;td>&lt;code>April Nassl&lt;/code>&lt;/td>
&lt;td>&lt;code>Google&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>11:50 - 12:10&lt;/td>
&lt;td>&lt;code>Spike Curtis&lt;/code>&lt;/td>
&lt;td>&lt;code>Tigera&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>12:10 - 12:30&lt;/td>
&lt;td>&lt;code>Shannon Coen&lt;/code>&lt;/td>
&lt;td>&lt;code>Pivotal&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>12:30 - 1:00&lt;/td>
&lt;td>&lt;code>Matt Klein&lt;/code>&lt;/td>
&lt;td>&lt;code>Lyft&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>1:00 - 1:20&lt;/td>
&lt;td>&lt;code>Zach Jory&lt;/code>&lt;/td>
&lt;td>&lt;code>F5/Aspen Mesh&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>1:20 - 1:40&lt;/td>
&lt;td>&lt;code>Dan Ciruli&lt;/code>&lt;/td>
&lt;td>&lt;code>Google&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>1:40 - 2:00&lt;/td>
&lt;td>&lt;code>Isaiah Snell-Feikema&lt;/code> / &lt;code>Greg Hanson&lt;/code>&lt;/td>
&lt;td>&lt;code>IBM&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2:00 - 2:20&lt;/td>
&lt;td>&lt;code>Zach Butcher&lt;/code>&lt;/td>
&lt;td>&lt;code>Tetrate&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2:20 - 2:40&lt;/td>
&lt;td>&lt;code>Ray Hudaihed&lt;/code>&lt;/td>
&lt;td>&lt;code>American Airlines&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>2:40 - 3:00&lt;/td>
&lt;td>&lt;code>Christian Posta&lt;/code>&lt;/td>
&lt;td>&lt;code>Red Hat&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>3:00 - 3:20&lt;/td>
&lt;td>&lt;code>Google/IBM China&lt;/code>&lt;/td>
&lt;td>&lt;code>Google / IBM&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>3:20 - 3:40&lt;/td>
&lt;td>&lt;code>Colby Dyess&lt;/code>&lt;/td>
&lt;td>&lt;code>Tuffin&lt;/code>&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>3:40 - 4:00&lt;/td>
&lt;td>&lt;code>Rohit Agarwalla&lt;/code>&lt;/td>
&lt;td>&lt;code>Cisco&lt;/code>&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table></description><pubDate>Fri, 03 Aug 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/istio-twitch-stream/</link><author>Spencer Krum, IBM</author><guid isPermaLink="true">/v1.12/blog/2018/istio-twitch-stream/</guid></item><item><title>Istio a Game Changer for HP's FitStation Platform</title><description>&lt;p>The FitStation team at HP strongly believes in the future of Kubernetes, BPF and service-mesh as the next standards in cloud infrastructure. We are also very happy to see Istio coming to its official Istio 1.0 release &amp;ndash; thanks to the joint collaboration that started at Google, IBM and Lyft beginning in May 2017.&lt;/p>
&lt;p>Throughout the development of FitStation’s large scale and progressive cloud platform, Istio, Cilium and Kubernetes technologies have delivered a multitude of opportunities to make our systems more robust and scalable. Istio was a game changer in creating reliable and dynamic network communication.&lt;/p>
&lt;p>&lt;a href="http://www.fitstation.com">FitStation powered by HP&lt;/a> is a technology platform that captures 3D biometric data to design personalized footwear to perfectly fit individual foot size and shape as well as gait profile. It uses 3D scanning, pressure sensing, 3D printing and variable density injection molding to create unique footwear. Footwear brands such as Brooks, Steitz Secura or Superfeet are connecting to FitStation to build their next generation of high performance sports, professional and medical shoes.&lt;/p>
&lt;p>FitStation is built on the promise of ultimate security and privacy for users&amp;rsquo; biometric data. ISTIO is the cornerstone to make that possible for data-at-flight within our cloud. By managing these aspects at the infrastructure level, we focused on solving business problems instead of spending time on individual implementations of secure service communication. Using Istio allowed us to dramatically reduce the complexity of maintaining a multitude of libraries and services to provide secure service communication.&lt;/p>
&lt;p>As a bonus benefit of Istio 1.0, we gained network visibility, metrics and tracing out of the box. This radically improved decision-making and response quality for our development
and devops teams. The team got in-depth insight in the network communication across the entire platform, both for new as well as legacy applications. The integration of Cilium
with Envoy delivered a remarkable performance benefit on Istio service mesh communication, combined with a fine-grained kernel driven L7 network security layer. This was due to the powers of BPF brought to Istio by Cilium. We believe this will drive the future of Linux kernel security.&lt;/p>
&lt;p>It has been very exciting to follow Istio’s growth. We have been able to see clear improvements of performance and stability over the different development versions. The improvements between version 0.7 and 0.8 made our teams feel comfortable with version 1.0, we can state that Istio is now ready for real production usage.&lt;/p>
&lt;p>We are looking forward to the promising roadmaps of Istio, Envoy, Cilium and CNCF.&lt;/p></description><pubDate>Tue, 31 Jul 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/hp/</link><author>Steven Ceuppens, Chief Software Architect @ HP FitStation, Open Source Advocate &amp; Contributor</author><guid isPermaLink="true">/v1.12/blog/2018/hp/</guid></item><item><title>Delayering Istio with AppSwitch</title><description>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">All problems in computer science can be solved with another layer, except of course the problem of too many layers. &amp;ndash; David Wheeler&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>The sidecar proxy approach enables a lot of awesomeness. Squarely in the datapath between microservices, the sidecar can precisely tell what the application is trying to do. It can monitor and instrument protocol traffic, not in the bowels of the networking layers but at the application level, to enable deep visibility, access controls and traffic management.&lt;/p>
&lt;p>If we look closely however, there are many intermediate layers that the data has to pass through before the high-value analysis of application-traffic can be performed. Most of those layers are part of the base plumbing infrastructure that are there just to push the data along. In doing so, they add latency to communication and complexity to the overall system.&lt;/p>
&lt;p>Over the years, there has been much collective effort in implementing aggressive fine-grained optimizations within the layers of the network datapath. Each iteration may shave another few microseconds. But then the true necessity of those layers itself has not been questioned.&lt;/p>
&lt;h2 id="don-t-optimize-layers-remove-them">Don’t optimize layers, remove them&lt;/h2>
&lt;p>In my belief, optimizing something is a poor fallback to removing its requirement altogether. That was the goal of my initial work (broken link: &lt;code>https://apporbit.com/a-brief-history-of-containers-from-reality-to-hype/&lt;/code>) on OS-level virtualization that led to Linux containers which effectively &lt;a href="https://www.oreilly.com/ideas/the-unwelcome-guest-why-vms-arent-the-solution-for-next-gen-applications">removed virtual machines&lt;/a> by running applications directly on the host operating system without requiring an intermediate guest. For a long time the industry was fighting the wrong battle distracted by optimizing VMs rather than removing the additional layer altogether.&lt;/p>
&lt;p>I see the same pattern repeat itself with the connectivity of microservices, and networking in general. The network has been going through the changes that physical servers have gone through a decade earlier. New set of layers and constructs are being introduced. They are being baked deep into the protocol stack and even silicon without adequately considering low-touch alternatives. Perhaps there is a way to remove those additional layers altogether.&lt;/p>
&lt;p>I have been thinking about these problems for some time and believe that an approach similar in concept to containers can be applied to the network stack that would fundamentally simplify how application endpoints are connected across the complexity of many intermediate layers. I have reapplied the same principles from the original work on containers to create &lt;a href="http://appswitch.io">AppSwitch&lt;/a>. Similar to the way containers provide an interface that applications can directly consume, AppSwitch plugs directly into well-defined and ubiquitous network API that applications currently use and directly connects application clients to appropriate servers, skipping all intermediate layers. In the end, that&amp;rsquo;s what networking is all about.&lt;/p>
&lt;p>Before going into the details of how AppSwitch promises to remove unnecessary layers from the Istio stack, let me give a very brief introduction to its architecture. Further details are available at the &lt;a href="https://appswitch.readthedocs.io/en/latest/">documentation&lt;/a> page.&lt;/p>
&lt;h2 id="appswitch">AppSwitch&lt;/h2>
&lt;p>Not unlike the container runtime, AppSwitch consists of a client and a daemon that speak over HTTP via a REST API. Both the client and the daemon are built as one self-contained binary, &lt;code>ax&lt;/code>. The client transparently plugs into the application and tracks its system calls related to network connectivity and notifies the daemon about their occurrences. As an example, let’s say an application makes the &lt;code>connect(2)&lt;/code> system call to the service IP of a Kubernetes service. The AppSwitch client intercepts the connect call, nullifies it and notifies the daemon about its occurrence along with some context that includes the system call arguments. The daemon would then handle the system call, potentially by directly connecting to the Pod IP of the upstream server on behalf of the application.&lt;/p>
&lt;p>It is important to note that no data is forwarded between AppSwitch client and daemon. They are designed to exchange file descriptors (FDs) over a Unix domain socket to avoid having to copy data. Note also that client is not a separate process. Rather it directly runs in the context of the application itself. There is no data copy between the application and AppSwitch client either.&lt;/p>
&lt;h2 id="delayering-the-stack">Delayering the stack&lt;/h2>
&lt;p>Now that we have an idea about what AppSwitch does, let’s look at the layers that it optimizes away from a standard service mesh.&lt;/p>
&lt;h3 id="network-devirtualization">Network devirtualization&lt;/h3>
&lt;p>Kubernetes offers simple and well-defined network constructs to the microservice applications it runs. In order to support them however, it imposes specific &lt;a href="https://kubernetes.io/docs/concepts/cluster-administration/networking/">requirements&lt;/a> on the underlying network. Meeting those requirements is often not easy. The go-to solution of adding another layer is typically adopted to satisfy the requirements. In most cases the additional layer consists of a network overlay that sits between Kubernetes and underlying network. Traffic produced by the applications is encapsulated at the source and decapsulated at the target, which not only costs network resources but also takes up compute cores.&lt;/p>
&lt;p>Because AppSwitch arbitrates what the application sees through its touchpoints with the platform, it projects a consistent virtual view of the underlying network to the application similar to an overlay but without introducing an additional layer of processing along the datapath. Just to draw a parallel to containers, the inside of a container looks and feels like a VM. However the underlying implementation does not intervene along the high-incidence control paths of low-level interrupts etc.&lt;/p>
&lt;p>AppSwitch can be injected into a standard Kubernetes manifest (similar to Istio injection) such that the application’s network is directly handled by AppSwitch bypassing any network overlay underneath. More details to follow in just a bit.&lt;/p>
&lt;h3 id="artifacts-of-container-networking">Artifacts of container networking&lt;/h3>
&lt;p>Extending network connectivity from host into the container has been a &lt;a href="https://kubernetes.io/blog/2016/01/why-kubernetes-doesnt-use-libnetwork/">major challenge&lt;/a>. New layers of network plumbing were invented explicitly for that purpose. As such, an application running in a container is simply a process on the host. However due to a &lt;a href="http://appswitch.io/blog/kubernetes_istio_and_network_function_devirtualization_with_appswitch/">fundamental misalignment&lt;/a> between the network abstraction expected by the application and the abstraction exposed by container network namespace, the process cannot directly access the host network. Applications think of networking in terms of sockets or sessions whereas network namespaces expose a device abstraction. Once placed in a network namespace, the process suddenly loses all connectivity. The notion of veth-pair and corresponding tooling were invented just to close that gap. The data would now have to go from a host interface into a virtual switch and then through a veth-pair to the virtual network interface of the container network namespace.&lt;/p>
&lt;p>AppSwitch can effectively remove both the virtual switch and veth-pair layers on both ends of the connection. Since the connections are established by the daemon running on the host using the network that’s already available on the host, there is no need for additional plumbing to bridge host network into the container. The socket FDs created on the host are passed to the application running within the pod’s network namespace. By the time the application receives the FD, all control path work (security checks, connection establishment) is already done and the FD is ready for actual IO.&lt;/p>
&lt;h3 id="skip-tcp-ip-for-colocated-endpoints">Skip TCP/IP for colocated endpoints&lt;/h3>
&lt;p>TCP/IP is the universal protocol medium over which pretty much all communication occurs. But if application endpoints happen to be on the same host, is TCP/IP really required? After all, it does do quite a bit of work and it is quite complex. Unix sockets are explicitly designed for intrahost communication and AppSwitch can transparently switch the communication to occur over a Unix socket for colocated endpoints.&lt;/p>
&lt;p>For each listening socket of an application, AppSwitch maintains two listening sockets, one each for TCP and Unix. When a client tries to connect to a server that happens to be colocated, AppSwitch daemon would choose to connect to the Unix listening socket of the server. The resulting Unix sockets on each end are passed into respective applications. Once a fully connected FD is returned, the application would simply treat it as a bit pipe. The protocol doesn’t really matter. The application may occasionally make protocol specific calls such as &lt;code>getsockname(2)&lt;/code> and AppSwitch would handle them in kind. It would present consistent responses such that the application would continue to run on.&lt;/p>
&lt;h3 id="data-pushing-proxy">Data pushing proxy&lt;/h3>
&lt;p>As we continue to look for layers to remove, let us also reconsider the requirement of the proxy layer itself. There are times when the role of the proxy may degenerate into a plain data pusher:&lt;/p>
&lt;ul>
&lt;li>There may not be a need for any protocol decoding&lt;/li>
&lt;li>The protocol may not be recognized by the proxy&lt;/li>
&lt;li>The communication may be encrypted and the proxy cannot access relevant headers&lt;/li>
&lt;li>The application (redis, memcached etc.) may be too latency-sensitive and cannot afford the cost of an intermediate proxy&lt;/li>
&lt;/ul>
&lt;p>In all these cases, the proxy is not different from any low-level plumbing layer. In fact, the latency introduced can be far higher because the same level of optimizations won’t be available to a proxy.&lt;/p>
&lt;p>To illustrate this with an example, consider the application shown below. It consists of a Python app and a set of memcached servers behind it. An upstream memcached server is selected based on connection time routing. Speed is the primary concern here.&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:38.63965267727931%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/delayering-istio/memcached.png" title="Latency-sensitive application scenario">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/delayering-istio/memcached.png" alt="Proxyless datapath" />
&lt;/a>
&lt;/div>
&lt;figcaption>Latency-sensitive application scenario&lt;/figcaption>
&lt;/figure>
&lt;p>If we look at the data flow in this setup, the Python app makes a connection to the service IP of memcached. It is redirected to the client-side sidecar. The sidecar routes the connection to one of the memcached servers and copies the data between the two sockets &amp;ndash; one connected to the app and another connected to memcached. And the same also occurs on the server side between the server-side sidecar and memcached. The role of proxy at that point is just boring shoveling of bits between the two sockets. However, it ends up adding substantial latency to the end-to-end connection.&lt;/p>
&lt;p>Now let us imagine that the app is somehow made to connect directly to memcached, then the two intermediate proxies could be skipped. The data would flow directly between the app and memcached without any intermediate hops. AppSwitch can arrange for that by transparently tweaking the target address passed by the Python app when it makes the &lt;code>connect(2)&lt;/code> system call.&lt;/p>
&lt;h3 id="proxyless-protocol-decoding">Proxyless protocol decoding&lt;/h3>
&lt;p>Things are going to get a bit strange here. We have seen that the proxy can be bypassed for cases that don’t involve looking into application traffic. But is there anything we can do even for those other cases? It turns out, yes.&lt;/p>
&lt;p>In a typical communication between microservices, much of the interesting information is exchanged in the initial headers. Headers are followed by body or payload which typically represents bulk of the communication. And once again the proxy degenerates into a data pusher for this part of communication. AppSwitch provides a nifty mechanism to skip proxy for these cases.&lt;/p>
&lt;p>Even though AppSwitch is not a proxy, it &lt;em>does&lt;/em> arbitrate connections between application endpoints and it &lt;em>does&lt;/em> have access to corresponding socket FDs. Normally, AppSwitch simply passes those FDs to the application. But it can also peek into the initial message received on the connection using the &lt;code>MSG_PEEK&lt;/code> option of the &lt;code>recvfrom(2)&lt;/code> system call on the socket. It allows AppSwitch to examine application traffic without actually removing it from the socket buffers. When AppSwitch returns the FD to the application and steps out of the datapath, the application would do an actual read on the connection. AppSwitch uses this technique to perform deeper analysis of application-level traffic and implement sophisticated network functions as discussed in the next section, all without getting into the datapath.&lt;/p>
&lt;h3 id="zero-cost-load-balancer-firewall-and-network-analyzer">Zero-cost load balancer, firewall and network analyzer&lt;/h3>
&lt;p>Typical implementations of network functions such as load balancers and firewalls require an intermediate layer that needs to tap into data/packet stream. Kubernetes&amp;rsquo; implementation of load balancer (&lt;code>kube-proxy&lt;/code>) for example introduces a probe into the packet stream through iptables and Istio implements the same at the proxy layer. But if all that is required is to redirect or drop connections based on policy, it is not really necessary to stay in the datapath during the entire course of the connection. AppSwitch can take care of that much more efficiently by simply manipulating the control path at the API level. Given its intimate proximity to the application, AppSwitch also has easy access to various pieces of application level metrics such as dynamics of stack and heap usage, precisely when a service comes alive, attributes of active connections etc., all of which could potentially form a rich signal for monitoring and analytics.&lt;/p>
&lt;p>To go a step further, AppSwitch can also perform L7 load balancing and firewall functions based on the protocol data that it obtains from the socket buffers. It can synthesize the protocol data and various other signals with the policy information acquired from Pilot to implement a highly efficient form of routing and access control enforcement. It can essentially &amp;ldquo;influence&amp;rdquo; the application to connect to the right backend server without requiring any changes to the application or its configuration. It is as if the application itself is infused with policy and traffic-management intelligence. Except in this case, the application can&amp;rsquo;t escape the influence.&lt;/p>
&lt;p>There is some more black-magic possible that would actually allow modifying the application data stream without getting into the datapath but I am going to save that for a later post. Current implementation of AppSwitch uses a proxy if the use case requires application protocol traffic to be modified. For those cases, AppSwitch provides a highly optimal mechanism to attract traffic to the proxy as discussed in the next section.&lt;/p>
&lt;h3 id="traffic-redirection">Traffic redirection&lt;/h3>
&lt;p>Before the sidecar proxy can look into application protocol traffic, it needs to first receive the connections. Redirection of connections coming into and going out of the application is currently done by a layer of packet filtering that rewrites packets such that they go to respective sidecars. Creating potentially large number of rules required to represent the redirection policy is tedious. And the process of applying the rules and updating them, as the target subnets to be captured by the sidecar change, is expensive.&lt;/p>
&lt;p>While some of the performance concerns are being addressed by the Linux community, there is another concern related to privilege: iptables rules need to be updated whenever the policy changes. Given the current architecture, all privileged operations are performed in an init container that runs just once at the very beginning before privileges are dropped for the actual application. Since updating iptables rules requires root privileges, there is no way to do that without restarting the application.&lt;/p>
&lt;p>AppSwitch provides a way to redirect application connections without root privilege. As such, an unprivileged application is already able to connect to any host (modulo firewall rules etc.) and the owner of the application should be allowed to change the host address passed by its application via &lt;code>connect(2)&lt;/code> without requiring additional privilege.&lt;/p>
&lt;h4 id="socket-delegation">Socket delegation&lt;/h4>
&lt;p>Let&amp;rsquo;s see how AppSwitch could help redirect connections without using iptables. Imagine that the application somehow voluntarily passes the socket FDs that it uses for its communication to the sidecar, then there would be no need for iptables. AppSwitch provides a feature called &lt;em>socket delegation&lt;/em> that does exactly that. It allows the sidecar to transparently gain access to copies of socket FDs that the application uses for its communication without any changes to the application itself.&lt;/p>
&lt;p>Here are the sequence of steps that would achieve this in the context of the Python application example.&lt;/p>
&lt;ol>
&lt;li>The application initiates a connection request to the service IP of memcached service.&lt;/li>
&lt;li>The connection request from client is forwarded to the daemon.&lt;/li>
&lt;li>The daemon creates a pair of pre-connected Unix sockets (using &lt;code>socketpair(2)&lt;/code> system call).&lt;/li>
&lt;li>It passes one end of the socket pair into the application such that the application would use that socket FD for read/write. It also ensures that the application consistently sees it as a legitimate TCP socket as it expects by interposing all calls that query connection properties.&lt;/li>
&lt;li>The other end is passed to sidecar over a different Unix socket where the daemon exposes its API. Information such as the original destination that the application was connecting to is also conveyed over the same interface.&lt;/li>
&lt;/ol>
&lt;figure style="width:50%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:22.442748091603054%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/delayering-istio/socket-delegation.png" title="Socket delegation based connection redirection">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/delayering-istio/socket-delegation.png" alt="Socket delegation protocol" />
&lt;/a>
&lt;/div>
&lt;figcaption>Socket delegation based connection redirection&lt;/figcaption>
&lt;/figure>
&lt;p>Once the application and sidecar are connected, the rest happens as usual. Sidecar would initiate a connection to upstream server and proxy data between the socket received from the daemon and the socket connected to upstream server. The main difference here is that sidecar would get the connection, not through the &lt;code>accept(2)&lt;/code> system call as it is in the normal case, but from the daemon over the Unix socket. In addition to listening for connections from applications through the normal &lt;code>accept(2)&lt;/code> channel, the sidecar proxy would connect to the AppSwitch daemon’s REST endpoint and receive sockets that way.&lt;/p>
&lt;p>For completeness, here are the sequence of steps that would occur on the server side:&lt;/p>
&lt;ol>
&lt;li>The application receives a connection&lt;/li>
&lt;li>AppSwitch daemon accepts the connection on behalf of the application&lt;/li>
&lt;li>It creates a pair of pre-connected Unix sockets using &lt;code>socketpair(2)&lt;/code> system call&lt;/li>
&lt;li>One end of the socket pair is returned to the application through the &lt;code>accept(2)&lt;/code> system call&lt;/li>
&lt;li>The other end of the socket pair along with the socket originally accepted by the daemon on behalf of the application is sent to sidecar&lt;/li>
&lt;li>Sidecar would extract the two socket FDs &amp;ndash; a Unix socket FD connected to the application and a TCP socket FD connected to the remote client&lt;/li>
&lt;li>Sidecar would read the metadata supplied by the daemon about the remote client and perform its usual operations&lt;/li>
&lt;/ol>
&lt;h4 id="sidecar-aware-applications">&amp;ldquo;Sidecar-aware&amp;rdquo; applications&lt;/h4>
&lt;p>Socket delegation feature can be very useful for applications that are explicitly aware of the sidecar and wish to take advantage of its features. They can voluntarily delegate their network interactions by passing their sockets to the sidecar using the same feature. In a way, AppSwitch transparently turns every application into a sidecar-aware application.&lt;/p>
&lt;h2 id="how-does-it-all-come-together">How does it all come together?&lt;/h2>
&lt;p>Just to step back, Istio offloads common connectivity concerns from applications to a sidecar proxy that performs those functions on behalf of the application. And AppSwitch simplifies and optimizes the service mesh by sidestepping intermediate layers and invoking the proxy only for cases where it is truly necessary.&lt;/p>
&lt;p>In the rest of this section, I outline how AppSwitch may be integrated with Istio based on a very cursory initial implementation. This is not intended to be anything like a design doc &amp;ndash; not every possible way of integration is explored and not every detail is worked out. The intent is to discuss high-level aspects of the implementation to present a rough idea of how the two systems may come together. The key is that AppSwitch would act as a cushion between Istio and a real proxy. It would serve as the &amp;ldquo;fast-path&amp;rdquo; for cases that can be performed more efficiently without invoking the sidecar proxy. And for the cases where the proxy is used, it would shorten the datapath by cutting through unnecessary layers. Look at this &lt;a href="http://appswitch.io/blog/kubernetes_istio_and_network_function_devirtualization_with_appswitch/">blog&lt;/a> for a more detailed walk through of the integration.&lt;/p>
&lt;h3 id="appswitch-client-injection">AppSwitch client injection&lt;/h3>
&lt;p>Similar to Istio sidecar-injector, a simple tool called &lt;code>ax-injector&lt;/code> injects AppSwitch client into a standard Kubernetes manifest. Injected client transparently monitors the application and intimates AppSwitch daemon of the control path network API events that the application produces.&lt;/p>
&lt;p>It is possible to not require the injection and work with standard Kubernetes manifests if AppSwitch CNI plugin is used. In that case, the CNI plugin would perform necessary injection when it gets the initialization callback. Using injector does have some advantages, however: (1) It works in tightly-controlled environments like GKE (2) It can be easily extended to support other frameworks such as Mesos (3) Same cluster would be able to run standard applications alongside &amp;ldquo;AppSwitch-enabled&amp;rdquo; applications.&lt;/p>
&lt;h3 id="appswitch-daemonset">AppSwitch &lt;code>DaemonSet&lt;/code>&lt;/h3>
&lt;p>AppSwitch daemon can be configured to run as a &lt;code>DaemonSet&lt;/code> or as an extension to the application that is directly injected into application manifest. In either case it handles network events coming in from the applications that it supports.&lt;/p>
&lt;h3 id="agent-for-policy-acquisition">Agent for policy acquisition&lt;/h3>
&lt;p>This is the component that conveys policy and configuration dictated by Istio to AppSwitch. It implements xDS API to listen from Pilot and calls appropriate AppSwitch APIs to program the daemon. For example, it allows the load balancing strategy, as specified by &lt;code>istioctl&lt;/code>, to be translated into equivalent AppSwitch capability.&lt;/p>
&lt;h3 id="platform-adapter-for-appswitch-auto-curated-service-registry">Platform adapter for AppSwitch &amp;ldquo;Auto-Curated&amp;rdquo; service registry&lt;/h3>
&lt;p>Given that AppSwitch is in the control path of applications’ network APIs, it has ready access to the topology of services across the cluster. AppSwitch exposes that information in the form of a service registry that is automatically and (almost) synchronously updated as applications and their services come and go. A new platform adapter for AppSwitch alongside Kubernetes, Eureka etc. would provide the details of upstream services to Istio. This is not strictly necessary but it does make it easier to correlate service endpoints received from Pilot by AppSwitch agent above.&lt;/p>
&lt;h3 id="proxy-integration-and-chaining">Proxy integration and chaining&lt;/h3>
&lt;p>Connections that do require deep scanning and mutation of application traffic are handed off to an external proxy through the socket delegation mechanism discussed earlier. It uses an extended version of &lt;a href="https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt">proxy protocol&lt;/a>. In addition to the simple parameters supported by the proxy protocol, a variety of other metadata (including the initial protocol headers obtained from the socket buffers) and live socket FDs (representing application connections) are forwarded to the proxy.&lt;/p>
&lt;p>The proxy can look at the metadata and decide how to proceed. It could respond by accepting the connection to do the proxying or by directing AppSwitch to allow the connection and use the fast-path or to just drop the connection.&lt;/p>
&lt;p>One of the interesting aspects of the mechanism is that, when the proxy accepts a socket from AppSwitch, it can in turn delegate the socket to another proxy. In fact that is how AppSwitch currently works. It uses a simple built-in proxy to examine the metadata and decide whether to handle the connection internally or to hand it off to an external proxy (Envoy). The same mechanism can be potentially extended to allow for a chain of plugins, each looking for a specific signature, with the last one in the chain doing the real proxy work.&lt;/p>
&lt;h2 id="it-s-not-just-about-performance">It&amp;rsquo;s not just about performance&lt;/h2>
&lt;p>Removing intermediate layers along the datapath is not just about improving performance. Performance is a great side effect, but it &lt;em>is&lt;/em> a side effect. There are a number of important advantages to an API level approach.&lt;/p>
&lt;h3 id="automatic-application-onboarding-and-policy-authoring">Automatic application onboarding and policy authoring&lt;/h3>
&lt;p>Before microservices and service mesh, traffic management was done by load balancers and access controls were enforced by firewalls. Applications were identified by IP addresses and DNS names which were relatively static. In fact, that&amp;rsquo;s still the status quo in most environments. Such environments stand to benefit immensely from service mesh. However a practical and scalable bridge to the new world needs to be provided. The difficulty in transformation is not as much due to lack of features and functionality but the investment required to rethink and reimplement the entire application infrastructure. Currently most of the policy and configuration exists in the form of load balancer and firewall rules. Somehow that existing context needs to be leveraged in providing a scalable path to adopting the service mesh model.&lt;/p>
&lt;p>AppSwitch can substantially ease the onboarding process. It can project the same network environment to the application at the target as its current source environment. Not having any assistance here is typically a non-starter in case of traditional applications which have complex configuration files with static IP addresses or specific DNS names hard-coded in them. AppSwitch could help capture those applications along with their existing configuration and connect them over a service mesh without requiring any changes.&lt;/p>
&lt;h3 id="broader-application-and-protocol-support">Broader application and protocol support&lt;/h3>
&lt;p>HTTP clearly dominates the modern application landscapes but once we talk about traditional applications and environments, we&amp;rsquo;d encounter all kinds of protocols and transports. Particularly, support for UDP becomes unavoidable. Traditional application servers such as IBM WebSphere rely extensively on UDP. Most multimedia applications use UDP media streams. Of course DNS is probably the most widely used UDP &amp;ldquo;application&amp;rdquo;. AppSwitch supports UDP at the API level much the same way as TCP and when it detects a UDP connection, it can transparently handle it in its &amp;ldquo;fast-path&amp;rdquo; rather than delegating it to the proxy.&lt;/p>
&lt;h3 id="client-ip-preservation-and-end-to-end-principle">Client IP preservation and end-to-end principle&lt;/h3>
&lt;p>The same mechanism that preserves the source network environment can also preserve client IP addresses as seen by the servers. With a sidecar proxy in place, connection requests come from the proxy rather than the client. As a result, the peer address (IP:port) of the connection as seen by the server would be that of the proxy rather than the client. AppSwitch ensures that the server sees correct address of the client, logs it correctly and any decisions made based on the client address remain valid. More generally, AppSwitch preserves the &lt;a href="https://en.wikipedia.org/wiki/End-to-end_principle">end-to-end principle&lt;/a> which is otherwise broken by intermediate layers that obfuscate the true underlying context.&lt;/p>
&lt;h3 id="enhanced-application-signal-with-access-to-encrypted-headers">Enhanced application signal with access to encrypted headers&lt;/h3>
&lt;p>Encrypted traffic completely undermines the ability of the service mesh to analyze application traffic. API level interposition could potentially offer a way around it. Current implementation of AppSwitch gains access to application&amp;rsquo;s network API at the system call level. However it is possible in principle to influence the application at an API boundary, higher in the stack where application data is not yet encrypted or already decrypted. Ultimately the data is always produced in the clear by the application and then encrypted at some point before it goes out. Since AppSwitch directly runs within the memory context of the application, it is possible to tap into the data higher on the stack where it is still held in clear. Only requirement for this to work is that the API used for encryption should be well-defined and amenable for interposition. Particularly, it requires access to the symbol table of the application binaries. Just to be clear, AppSwitch doesn&amp;rsquo;t implement this today.&lt;/p>
&lt;h2 id="so-what-s-the-net">So what’s the net?&lt;/h2>
&lt;p>AppSwitch removes a number of layers and processing from the standard service mesh stack. What does all that translate to in terms of performance?&lt;/p>
&lt;p>We ran some initial experiments to characterize the extent of the opportunity for optimization based on the initial integration of AppSwitch discussed earlier. The experiments were run on GKE using &lt;code>fortio-0.11.0&lt;/code>, &lt;code>istio-0.8.0&lt;/code> and &lt;code>appswitch-0.4.0-2&lt;/code>. In case of the proxyless test, AppSwitch daemon was run as a &lt;code>DaemonSet&lt;/code> on the Kubernetes cluster and the Fortio pod spec was modified to inject AppSwitch client. These were the only two changes made to the setup. The test was configured to measure the latency of GRPC requests across 100 concurrent connections.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:54.66034755134282%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/delayering-istio/perf.png" title="Latency with and without AppSwitch">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/delayering-istio/perf.png" alt="Performance comparison" />
&lt;/a>
&lt;/div>
&lt;figcaption>Latency with and without AppSwitch&lt;/figcaption>
&lt;/figure>
&lt;p>Initial results indicate a difference of over 18x in p50 latency with and without AppSwitch (3.99ms vs 72.96ms). The difference was around 8x when mixer and access logs were disabled. Clearly the difference was due to sidestepping all those intermediate layers along the datapath. Unix socket optimization wasn&amp;rsquo;t triggered in case of AppSwitch because client and server pods were scheduled to separate hosts. End-to-end latency of AppSwitch case would have been even lower if the client and server happened to be colocated. Essentially the client and server running in their respective pods of the Kubernetes cluster are directly connected over a TCP socket going over the GKE network &amp;ndash; no tunneling, bridge or proxies.&lt;/p>
&lt;h2 id="net-net">Net Net&lt;/h2>
&lt;p>I started out with David Wheeler&amp;rsquo;s seemingly reasonable quote that says adding another layer is not a solution for the problem of too many layers. And I argued through most of the blog that current network stack already has too many layers and that they should be removed. But isn&amp;rsquo;t AppSwitch itself a layer?&lt;/p>
&lt;p>Yes, AppSwitch is clearly another layer. However it is one that can remove multiple other layers. In doing so, it seamlessly glues the new service mesh layer with existing layers of traditional network environments. It offsets the cost of sidecar proxy and as Istio graduates to 1.0, it provides a bridge for existing applications and their network environments to transition to the new world of service mesh.&lt;/p>
&lt;p>Perhaps Wheeler’s quote should read:&lt;/p>
&lt;div>
&lt;aside class="callout quote">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-quote"/>&lt;/svg>
&lt;/div>
&lt;div class="content">All problems in computer science can be solved with another layer, &lt;strong>even&lt;/strong> the problem of too many layers!&lt;/div>
&lt;/aside>
&lt;/div>
&lt;h2 id="acknowledgements">Acknowledgements&lt;/h2>
&lt;p>Thanks to Mandar Jog (Google) for several discussions about the value of AppSwitch for Istio and to the following individuals (in alphabetical order) for their review of early drafts of this blog.&lt;/p>
&lt;ul>
&lt;li>Frank Budinsky (IBM)&lt;/li>
&lt;li>Lin Sun (IBM)&lt;/li>
&lt;li>Shriram Rajagopalan (VMware)&lt;/li>
&lt;/ul></description><pubDate>Mon, 30 Jul 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/delayering-istio/</link><author>Dinesh Subhraveti (AppOrbit and Columbia University)</author><guid isPermaLink="true">/v1.12/blog/2018/delayering-istio/</guid><category>appswitch</category><category>performance</category></item><item><title>Micro-Segmentation with Istio Authorization</title><description>
&lt;p>Micro-segmentation is a security technique that creates secure zones in cloud deployments and allows organizations to
isolate workloads from one another and secure them individually.
&lt;a href="/v1.12/docs/concepts/security/#authorization">Istio&amp;rsquo;s authorization feature&lt;/a>, also known as Istio Role Based Access Control,
provides micro-segmentation for services in an Istio mesh. It features:&lt;/p>
&lt;ul>
&lt;li>Authorization at different levels of granularity, including namespace level, service level, and method level.&lt;/li>
&lt;li>Service-to-service and end-user-to-service authorization.&lt;/li>
&lt;li>High performance, as it is enforced natively on Envoy.&lt;/li>
&lt;li>Role-based semantics, which makes it easy to use.&lt;/li>
&lt;li>High flexibility as it allows users to define conditions using
&lt;a href="/v1.12/docs/reference/config/security/conditions/">combinations of attributes&lt;/a>.&lt;/li>
&lt;/ul>
&lt;p>In this blog post, you&amp;rsquo;ll learn about the main authorization features and how to use them in different situations.&lt;/p>
&lt;h2 id="characteristics">Characteristics&lt;/h2>
&lt;h3 id="rpc-level-authorization">RPC level authorization&lt;/h3>
&lt;p>Authorization is performed at the level of individual RPCs. Specifically, it controls &amp;ldquo;who can access my &lt;code>bookstore&lt;/code> service”,
or &amp;ldquo;who can access method &lt;code>getBook&lt;/code> in my &lt;code>bookstore&lt;/code> service”. It is not designed to control access to application-specific
resource instances, like access to &amp;ldquo;storage bucket X” or access to &amp;ldquo;3rd book on 2nd shelf”. Today this kind of application
specific access control logic needs to be handled by the application itself.&lt;/p>
&lt;h3 id="role-based-access-control-with-conditions">Role-based access control with conditions&lt;/h3>
&lt;p>Authorization is a &lt;a href="https://en.wikipedia.org/wiki/Role-based_access_control">role-based access control (RBAC)&lt;/a> system,
contrast this to an &lt;a href="https://en.wikipedia.org/wiki/Attribute-based_access_control">attribute-based access control (ABAC)&lt;/a>
system. Compared to ABAC, RBAC has the following advantages:&lt;/p>
&lt;ul>
&lt;li>&lt;p>&lt;strong>Roles allow grouping of attributes.&lt;/strong> Roles are groups of permissions, which specifies the actions you are allowed
to perform on a system. Users are grouped based on the roles within an organization. You can define the roles and reuse
them for different cases.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>It is easier to understand and reason about who has access.&lt;/strong> The RBAC concepts map naturally to business concepts.
For example, a DB admin may have all access to DB backend services, while a web client may only be able to view the
frontend service.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>It reduces unintentional errors.&lt;/strong> RBAC policies make otherwise complex security changes easier. You won&amp;rsquo;t have
duplicate configurations in multiple places and later forget to update some of them when you need to make changes.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>On the other hand, Istio&amp;rsquo;s authorization system is not a traditional RBAC system. It also allows users to define &lt;strong>conditions&lt;/strong> using
&lt;a href="/v1.12/docs/reference/config/security/conditions/">combinations of attributes&lt;/a>. This gives Istio
flexibility to express complex access control policies. In fact, &lt;strong>the &amp;ldquo;RBAC + conditions” model
that Istio authorization adopts, has all the benefits an RBAC system has, and supports the level of flexibility that
normally an ABAC system provides.&lt;/strong> You&amp;rsquo;ll see some &lt;a href="#examples">examples&lt;/a> below.&lt;/p>
&lt;h3 id="high-performance">High performance&lt;/h3>
&lt;p>Because of its simple semantics, Istio authorization is enforced on Envoy as a native authorization support. At runtime, the
authorization decision is completely done locally inside an Envoy filter, without dependency to any external module.
This allows Istio authorization to achieve high performance and availability.&lt;/p>
&lt;h3 id="work-with-without-primary-identities">Work with/without primary identities&lt;/h3>
&lt;p>Like any other RBAC system, Istio authorization is identity aware. In Istio authorization policy, there is a primary
identity called &lt;code>user&lt;/code>, which represents the principal of the client.&lt;/p>
&lt;p>In addition to the primary identity, you can also specify any conditions that define the identities. For example,
you can specify the client identity as &amp;ldquo;user Alice calling from Bookstore frontend service”, in which case,
you have a combined identity of the calling service (&lt;code>Bookstore frontend&lt;/code>) and the end user (&lt;code>Alice&lt;/code>).&lt;/p>
&lt;p>To improve security, you should enable &lt;a href="/v1.12/docs/concepts/security/#authentication">authentication features&lt;/a>,
and use authenticated identities in authorization policies. However, strongly authenticated identity is not required
for using authorization. Istio authorization works with or without identities. If you are working with a legacy system,
you may not have mutual TLS or JWT authentication setup for your mesh. In this case, the only way to identify the client is, for example,
through IP. You can still use Istio authorization to control which IP addresses or IP ranges are allowed to access your service.&lt;/p>
&lt;h2 id="examples">Examples&lt;/h2>
&lt;p>The &lt;a href="/v1.12/docs/tasks/security/authorization/authz-http/">authorization task&lt;/a> shows you how to
use Istio&amp;rsquo;s authorization feature to control namespace level and service level access using the
&lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo application&lt;/a>. In this section, you&amp;rsquo;ll see more examples on how to achieve
micro-segmentation with Istio authorization.&lt;/p>
&lt;h3 id="namespace-level-segmentation-via-rbac-conditions">Namespace level segmentation via RBAC + conditions&lt;/h3>
&lt;p>Suppose you have services in the &lt;code>frontend&lt;/code> and &lt;code>backend&lt;/code> namespaces. You would like to allow all your services
in the &lt;code>frontend&lt;/code> namespace to access all services that are marked &lt;code>external&lt;/code> in the &lt;code>backend&lt;/code> namespace.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRole
metadata:
name: external-api-caller
namespace: backend
spec:
rules:
- services: [&amp;#34;*&amp;#34;]
methods: [&amp;#34;*”]
constraints:
- key: &amp;#34;destination.labels[visibility]”
values: [&amp;#34;external&amp;#34;]
---
apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRoleBinding
metadata:
name: external-api-caller
namespace: backend
spec:
subjects:
- properties:
source.namespace: &amp;#34;frontend”
roleRef:
kind: ServiceRole
name: &amp;#34;external-api-caller&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>The &lt;code>ServiceRole&lt;/code> and &lt;code>ServiceRoleBinding&lt;/code> above expressed &amp;ldquo;&lt;em>who&lt;/em> is allowed to do &lt;em>what&lt;/em> under *which conditions*”
(RBAC + conditions). Specifically:&lt;/p>
&lt;ul>
&lt;li>&lt;strong>&amp;ldquo;who”&lt;/strong> are the services in the &lt;code>frontend&lt;/code> namespace.&lt;/li>
&lt;li>&lt;strong>&amp;ldquo;what”&lt;/strong> is to call services in &lt;code>backend&lt;/code> namespace.&lt;/li>
&lt;li>&lt;strong>&amp;ldquo;conditions”&lt;/strong> is the &lt;code>visibility&lt;/code> label of the destination service having the value &lt;code>external&lt;/code>.&lt;/li>
&lt;/ul>
&lt;h3 id="service-method-level-isolation-with-without-primary-identities">Service/method level isolation with/without primary identities&lt;/h3>
&lt;p>Here is another example that demonstrates finer grained access control at service/method level. The first step
is to define a &lt;code>book-reader&lt;/code> service role that allows READ access to &lt;code>/books/*&lt;/code> resource in &lt;code>bookstore&lt;/code> service.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRole
metadata:
name: book-reader
namespace: default
spec:
rules:
- services: [&amp;#34;bookstore.default.svc.cluster.local&amp;#34;]
paths: [&amp;#34;/books/*”]
methods: [&amp;#34;GET”]
&lt;/code>&lt;/pre>
&lt;h4 id="using-authenticated-client-identities">Using authenticated client identities&lt;/h4>
&lt;p>Suppose you want to grant this &lt;code>book-reader&lt;/code> role to your &lt;code>bookstore-frontend&lt;/code> service. If you have enabled
&lt;a href="/v1.12/docs/concepts/security/#mutual-tls-authentication">mutual TLS authentication&lt;/a> for your mesh, you can use a
service account to identify your &lt;code>bookstore-frontend&lt;/code> service. Granting the &lt;code>book-reader&lt;/code> role to the &lt;code>bookstore-frontend&lt;/code>
service can be done by creating a &lt;code>ServiceRoleBinding&lt;/code> as shown below:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRoleBinding
metadata:
name: book-reader
namespace: default
spec:
subjects:
- user: &amp;#34;cluster.local/ns/default/sa/bookstore-frontend”
roleRef:
kind: ServiceRole
name: &amp;#34;book-reader&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>You may want to restrict this further by adding a condition that &amp;ldquo;only users who belong to the &lt;code>qualified-reviewer&lt;/code> group are
allowed to read books”. The &lt;code>qualified-reviewer&lt;/code> group is the end user identity that is authenticated by
&lt;a href="/v1.12/docs/concepts/security/#authentication">JWT authentication&lt;/a>. In this case, the combination of the client service identity
(&lt;code>bookstore-frontend&lt;/code>) and the end user identity (&lt;code>qualified-reviewer&lt;/code>) is used in the authorization policy.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRoleBinding
metadata:
name: book-reader
namespace: default
spec:
subjects:
- user: &amp;#34;cluster.local/ns/default/sa/bookstore-frontend&amp;#34;
properties:
request.auth.claims[group]: &amp;#34;qualified-reviewer&amp;#34;
roleRef:
kind: ServiceRole
name: &amp;#34;book-reader&amp;#34;
&lt;/code>&lt;/pre>
&lt;h4 id="client-does-not-have-identity">Client does not have identity&lt;/h4>
&lt;p>Using authenticated identities in authorization policies is strongly recommended for security. However, if you have a
legacy system that does not support authentication, you may not have authenticated identities for your services.
You can still use Istio authorization to protect your services even without authenticated identities. The example below
shows that you can specify allowed source IP range in your authorization policy.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;rbac.istio.io/v1alpha1&amp;#34;
kind: ServiceRoleBinding
metadata:
name: book-reader
namespace: default
spec:
subjects:
- properties:
source.ip: 10.20.0.0/9
roleRef:
kind: ServiceRole
name: &amp;#34;book-reader&amp;#34;
&lt;/code>&lt;/pre>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>Istio’s authorization feature provides authorization at namespace-level, service-level, and method-level granularity.
It adopts &amp;ldquo;RBAC + conditions” model, which makes it easy to use and understand as an RBAC system, while providing the level of
flexibility that an ABAC system normally provides. Istio authorization achieves high performance as it is enforced
natively on Envoy. While it provides the best security by working together with
&lt;a href="/v1.12/docs/concepts/security/#authentication">Istio authentication features&lt;/a>, Istio authorization can also be used to
provide access control for legacy systems that do not have authentication.&lt;/p></description><pubDate>Fri, 20 Jul 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/istio-authorization/</link><author>Limin Wang</author><guid isPermaLink="true">/v1.12/blog/2018/istio-authorization/</guid><category>authorization</category><category>rbac</category><category>security</category></item><item><title>Exporting Logs to BigQuery, GCS, Pub/Sub through Stackdriver</title><description>
&lt;p>This post shows how to direct Istio logs to &lt;a href="https://cloud.google.com/stackdriver/">Stackdriver&lt;/a>
and export those logs to various configured sinks such as such as
&lt;a href="https://cloud.google.com/bigquery/">BigQuery&lt;/a>, &lt;a href="https://cloud.google.com/storage/">Google Cloud Storage&lt;/a>
or &lt;a href="https://cloud.google.com/pubsub/">Cloud Pub/Sub&lt;/a>. At the end of this post you can perform
analytics on Istio data from your favorite places such as BigQuery, GCS or Cloud Pub/Sub.&lt;/p>
&lt;p>The &lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo&lt;/a> sample application is used as the example
application throughout this task.&lt;/p>
&lt;h2 id="before-you-begin">Before you begin&lt;/h2>
&lt;p>&lt;a href="/v1.12/docs/setup/">Install Istio&lt;/a> in your cluster and deploy an application.&lt;/p>
&lt;h2 id="configuring-istio-to-export-logs">Configuring Istio to export logs&lt;/h2>
&lt;p>Istio exports logs using the &lt;code>logentry&lt;/code> &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/templates/logentry">template&lt;/a>.
This specifies all the variables that are available for analysis. It
contains information like source service, destination service, auth
metrics (coming..) among others. Following is a diagram of the pipeline:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/export-logs-through-stackdriver/istio-analytics-using-stackdriver.png" title="Exporting logs from Istio to Stackdriver for analysis">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/export-logs-through-stackdriver/istio-analytics-using-stackdriver.png" alt="Exporting logs from Istio to Stackdriver for analysis" />
&lt;/a>
&lt;/div>
&lt;figcaption>Exporting logs from Istio to Stackdriver for analysis&lt;/figcaption>
&lt;/figure>
&lt;p>Istio supports exporting logs to Stackdriver which can in turn be configured to export
logs to your favorite sink like BigQuery, Pub/Sub or GCS. Please follow the steps
below to setup your favorite sink for exporting logs first and then Stackdriver
in Istio.&lt;/p>
&lt;h3 id="setting-up-various-log-sinks">Setting up various log sinks&lt;/h3>
&lt;p>Common setup for all sinks:&lt;/p>
&lt;ol>
&lt;li>Enable &lt;a href="https://cloud.google.com/monitoring/api/enable-api">Stackdriver Monitoring API&lt;/a> for the project.&lt;/li>
&lt;li>Make sure &lt;code>principalEmail&lt;/code> that would be setting up the sink has write access to the project and Logging Admin role permissions.&lt;/li>
&lt;li>Make sure the &lt;code>GOOGLE_APPLICATION_CREDENTIALS&lt;/code> environment variable is set. Please follow instructions &lt;a href="https://cloud.google.com/docs/authentication/getting-started">here&lt;/a> to set it up.&lt;/li>
&lt;/ol>
&lt;h4 id="bigquery">BigQuery&lt;/h4>
&lt;ol>
&lt;li>&lt;a href="https://cloud.google.com/bigquery/docs/datasets">Create a BigQuery dataset&lt;/a> as a destination for the logs export.&lt;/li>
&lt;li>Record the ID of the dataset. It will be needed to configure the Stackdriver handler.
It would be of the form &lt;code>bigquery.googleapis.com/projects/[PROJECT_ID]/datasets/[DATASET_ID]&lt;/code>&lt;/li>
&lt;li>Give &lt;a href="https://cloud.google.com/logging/docs/api/tasks/exporting-logs#writing_to_the_destination">sink’s writer identity&lt;/a>: &lt;code>cloud-logs@system.gserviceaccount.com&lt;/code> BigQuery Data Editor role in IAM.&lt;/li>
&lt;li>If using &lt;a href="/v1.12/docs/setup/platform-setup/gke/">Google Kubernetes Engine&lt;/a>, make sure &lt;code>bigquery&lt;/code> &lt;a href="https://cloud.google.com/sdk/gcloud/reference/container/clusters/create">Scope&lt;/a> is enabled on the cluster.&lt;/li>
&lt;/ol>
&lt;h4 id="google-cloud-storage-gcs">Google Cloud Storage (GCS)&lt;/h4>
&lt;ol>
&lt;li>&lt;a href="https://cloud.google.com/storage/docs/creating-buckets">Create a GCS bucket&lt;/a> where you would like logs to get exported in GCS.&lt;/li>
&lt;li>Recode the ID of the bucket. It will be needed to configure Stackdriver.
It would be of the form &lt;code>storage.googleapis.com/[BUCKET_ID]&lt;/code>&lt;/li>
&lt;li>Give &lt;a href="https://cloud.google.com/logging/docs/api/tasks/exporting-logs#writing_to_the_destination">sink’s writer identity&lt;/a>: &lt;code>cloud-logs@system.gserviceaccount.com&lt;/code> Storage Object Creator role in IAM.&lt;/li>
&lt;/ol>
&lt;h4 id="google-cloud-pub-sub">Google Cloud Pub/Sub&lt;/h4>
&lt;ol>
&lt;li>&lt;a href="https://cloud.google.com/pubsub/docs/admin">Create a topic&lt;/a> where you would like logs to get exported in Google Cloud Pub/Sub.&lt;/li>
&lt;li>Recode the ID of the topic. It will be needed to configure Stackdriver.
It would be of the form &lt;code>pubsub.googleapis.com/projects/[PROJECT_ID]/topics/[TOPIC_ID]&lt;/code>&lt;/li>
&lt;li>Give &lt;a href="https://cloud.google.com/logging/docs/api/tasks/exporting-logs#writing_to_the_destination">sink’s writer identity&lt;/a>: &lt;code>cloud-logs@system.gserviceaccount.com&lt;/code> Pub/Sub Publisher role in IAM.&lt;/li>
&lt;li>If using &lt;a href="/v1.12/docs/setup/platform-setup/gke/">Google Kubernetes Engine&lt;/a>, make sure &lt;code>pubsub&lt;/code> &lt;a href="https://cloud.google.com/sdk/gcloud/reference/container/clusters/create">Scope&lt;/a> is enabled on the cluster.&lt;/li>
&lt;/ol>
&lt;h3 id="setting-up-stackdriver">Setting up Stackdriver&lt;/h3>
&lt;p>A Stackdriver handler must be created to export data to Stackdriver. The configuration for
a Stackdriver handler is described &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/adapters/stackdriver/">here&lt;/a>.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Save the following yaml file as &lt;code>stackdriver.yaml&lt;/code>. Replace &lt;code>&amp;lt;project_id&amp;gt;,
&amp;lt;sink_id&amp;gt;, &amp;lt;sink_destination&amp;gt;, &amp;lt;log_filter&amp;gt;&lt;/code> with their specific values.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: stackdriver
metadata:
name: handler
namespace: istio-system
spec:
# We&amp;#39;ll use the default value from the adapter, once per minute, so we don&amp;#39;t need to supply a value.
# pushInterval: 1m
# Must be supplied for the Stackdriver adapter to work
project_id: &amp;#34;&amp;lt;project_id&amp;gt;&amp;#34;
# One of the following must be set; the preferred method is `appCredentials`, which corresponds to
# Google Application Default Credentials.
# If none is provided we default to app credentials.
# appCredentials:
# apiKey:
# serviceAccountPath:
# Describes how to map Istio logs into Stackdriver.
logInfo:
accesslog.logentry.istio-system:
payloadTemplate: &amp;#39;{{or (.sourceIp) &amp;#34;-&amp;#34;}} - {{or (.sourceUser) &amp;#34;-&amp;#34;}} [{{or (.timestamp.Format &amp;#34;02/Jan/2006:15:04:05 -0700&amp;#34;) &amp;#34;-&amp;#34;}}] &amp;#34;{{or (.method) &amp;#34;-&amp;#34;}} {{or (.url) &amp;#34;-&amp;#34;}} {{or (.protocol) &amp;#34;-&amp;#34;}}&amp;#34; {{or (.responseCode) &amp;#34;-&amp;#34;}} {{or (.responseSize) &amp;#34;-&amp;#34;}}&amp;#39;
httpMapping:
url: url
status: responseCode
requestSize: requestSize
responseSize: responseSize
latency: latency
localIp: sourceIp
remoteIp: destinationIp
method: method
userAgent: userAgent
referer: referer
labelNames:
- sourceIp
- destinationIp
- sourceService
- sourceUser
- sourceNamespace
- destinationIp
- destinationService
- destinationNamespace
- apiClaims
- apiKey
- protocol
- method
- url
- responseCode
- responseSize
- requestSize
- latency
- connectionMtls
- userAgent
- responseTimestamp
- receivedBytes
- sentBytes
- referer
sinkInfo:
id: &amp;#39;&amp;lt;sink_id&amp;gt;&amp;#39;
destination: &amp;#39;&amp;lt;sink_destination&amp;gt;&amp;#39;
filter: &amp;#39;&amp;lt;log_filter&amp;gt;&amp;#39;
---
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: rule
metadata:
name: stackdriver
namespace: istio-system
spec:
match: &amp;#34;true&amp;#34; # If omitted match is true.
actions:
- handler: handler.stackdriver
instances:
- accesslog.logentry
---
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Push the configuration&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f stackdriver.yaml
stackdriver &amp;#34;handler&amp;#34; created
rule &amp;#34;stackdriver&amp;#34; created
logentry &amp;#34;stackdriverglobalmr&amp;#34; created
metric &amp;#34;stackdriverrequestcount&amp;#34; created
metric &amp;#34;stackdriverrequestduration&amp;#34; created
metric &amp;#34;stackdriverrequestsize&amp;#34; created
metric &amp;#34;stackdriverresponsesize&amp;#34; created
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Send traffic to the sample application.&lt;/p>
&lt;p>For the Bookinfo sample, visit &lt;code>http://$GATEWAY_URL/productpage&lt;/code> in your web
browser or issue the following command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl http://$GATEWAY_URL/productpage
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Verify that logs are flowing through Stackdriver to the configured sink.&lt;/p>
&lt;ul>
&lt;li>Stackdriver: Navigate to the &lt;a href="https://pantheon.corp.google.com/logs/viewer">Stackdriver Logs
Viewer&lt;/a> for your project
and look under &amp;ldquo;GKE Container&amp;rdquo; -&amp;gt; &amp;ldquo;Cluster Name&amp;rdquo; -&amp;gt; &amp;ldquo;Namespace Id&amp;rdquo; for
Istio Access logs.&lt;/li>
&lt;li>BigQuery: Navigate to the &lt;a href="https://bigquery.cloud.google.com/">BigQuery
Interface&lt;/a> for your project and you
should find a table with prefix &lt;code>accesslog_logentry_istio&lt;/code> in your sink
dataset.&lt;/li>
&lt;li>GCS: Navigate to the &lt;a href="https://pantheon.corp.google.com/storage/browser/">Storage
Browser&lt;/a> for your
project and you should find a bucket named
&lt;code>accesslog.logentry.istio-system&lt;/code> in your sink bucket.&lt;/li>
&lt;li>Pub/Sub: Navigate to the &lt;a href="https://pantheon.corp.google.com/cloudpubsub/topicList">Pub/Sub
Topic List&lt;/a> for
your project and you should find a topic for &lt;code>accesslog&lt;/code> in your sink
topic.&lt;/li>
&lt;/ul>&lt;/li>
&lt;/ol>
&lt;h2 id="understanding-what-happened">Understanding what happened&lt;/h2>
&lt;p>&lt;code>Stackdriver.yaml&lt;/code> file above configured Istio to send access logs to
Stackdriver and then added a sink configuration where these logs could be
exported. In detail as follows:&lt;/p>
&lt;ol>
&lt;li>&lt;p>Added a handler of kind &lt;code>stackdriver&lt;/code>&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: stackdriver
metadata:
name: handler
namespace: &amp;lt;your defined namespace&amp;gt;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Added &lt;code>logInfo&lt;/code> in spec&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >spec:
logInfo: accesslog.logentry.istio-system:
labelNames:
- sourceIp
- destinationIp
...
...
sinkInfo:
id: &amp;#39;&amp;lt;sink_id&amp;gt;&amp;#39;
destination: &amp;#39;&amp;lt;sink_destination&amp;gt;&amp;#39;
filter: &amp;#39;&amp;lt;log_filter&amp;gt;&amp;#39;
&lt;/code>&lt;/pre>
&lt;p>In the above configuration sinkInfo contains information about the sink where you want
the logs to get exported to. For more information on how this gets filled for different sinks please refer
&lt;a href="https://cloud.google.com/logging/docs/export/#sink-terms">here&lt;/a>.&lt;/p>&lt;/li>
&lt;li>&lt;p>Added a rule for Stackdriver&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: rule
metadata:
name: stackdriver
namespace: istio-system spec:
match: &amp;#34;true&amp;#34; # If omitted match is true
actions:
- handler: handler.stackdriver
instances:
- accesslog.logentry
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h2 id="cleanup">Cleanup&lt;/h2>
&lt;ul>
&lt;li>&lt;p>Remove the new Stackdriver configuration:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete -f stackdriver.yaml
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>If you are not planning to explore any follow-on tasks, refer to the
&lt;a href="/v1.12/docs/examples/bookinfo/#cleanup">Bookinfo cleanup&lt;/a> instructions to shutdown
the application.&lt;/p>&lt;/li>
&lt;/ul>
&lt;h2 id="availability-of-logs-in-export-sinks">Availability of logs in export sinks&lt;/h2>
&lt;p>Export to BigQuery is within minutes (we see it to be almost instant), GCS can
have a delay of 2 to 12 hours and Pub/Sub is almost immediately.&lt;/p></description><pubDate>Mon, 09 Jul 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/export-logs-through-stackdriver/</link><author>Nupur Garg and Douglas Reid</author><guid isPermaLink="true">/v1.12/blog/2018/export-logs-through-stackdriver/</guid></item><item><title>Monitoring and Access Policies for HTTP Egress Traffic</title><description>
&lt;p>While Istio&amp;rsquo;s main focus is management of traffic between microservices inside a service mesh, Istio can also manage
ingress (from outside into the mesh) and egress (from the mesh outwards) traffic. Istio can uniformly enforce access
policies and aggregate telemetry data for mesh-internal, ingress and egress traffic.&lt;/p>
&lt;p>In this blog post, we show how to apply monitoring and access policies to HTTP egress traffic with Istio.&lt;/p>
&lt;h2 id="use-case">Use case&lt;/h2>
&lt;p>Consider an organization that runs applications that process content from &lt;em>cnn.com&lt;/em>. The applications are decomposed
into microservices deployed in an Istio service mesh. The applications access pages of various topics from &lt;em>cnn.com&lt;/em>: &lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a>, &lt;a href="https://edition.cnn.com/sport">edition.cnn.com/sport&lt;/a> and &lt;a href="https://edition.cnn.com/health">edition.cnn.com/health&lt;/a>. The organization &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/">configures Istio to allow access to edition.cnn.com&lt;/a> and everything works fine. However, at some
point in time, the organization decides to banish politics. Practically, it means blocking access to
&lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> and allowing access to
&lt;a href="https://edition.cnn.com/sport">edition.cnn.com/sport&lt;/a> and &lt;a href="https://edition.cnn.com/health">edition.cnn.com/health&lt;/a>
only. The organization will grant permissions to individual applications and to particular users to access &lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a>, on a case-by-case basis.&lt;/p>
&lt;p>To achieve that goal, the organization&amp;rsquo;s operations people monitor access to the external services and
analyze Istio logs to verify that no unauthorized request was sent to
&lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a>. They also configure Istio to prevent access to
&lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> automatically.&lt;/p>
&lt;p>The organization is resolved to prevent any tampering with the new policy. It decides to put mechanisms in place that
will prevent any possibility for a malicious application to access the forbidden topic.&lt;/p>
&lt;h2 id="related-tasks-and-examples">Related tasks and examples&lt;/h2>
&lt;ul>
&lt;li>The &lt;a href="/v1.12/docs/tasks/traffic-management/egress/">Control Egress Traffic&lt;/a> task demonstrates how external (outside the
Kubernetes cluster) HTTP and HTTPS services can be accessed by applications inside the mesh.&lt;/li>
&lt;li>The &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/">Configure an Egress Gateway&lt;/a> example describes how to configure
Istio to direct egress traffic through a dedicated gateway service called &lt;em>egress gateway&lt;/em>.&lt;/li>
&lt;li>The &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/">Egress Gateway with TLS Origination&lt;/a> example
demonstrates how to allow applications to send HTTP requests to external servers that require HTTPS, while directing
traffic through egress gateway.&lt;/li>
&lt;li>The &lt;a href="/v1.12/docs/tasks/observability/metrics/using-istio-dashboard/">Visualizing Metrics with Grafana&lt;/a>
describes the Istio Dashboard to monitor mesh traffic.&lt;/li>
&lt;li>The &lt;a href="https://istio.io/v1.6/docs/tasks/policy-enforcement/denial-and-list/">Basic Access Control&lt;/a> task shows how to control access to
in-mesh services.&lt;/li>
&lt;li>The &lt;a href="https://istio.io/v1.6/docs/tasks/policy-enforcement/denial-and-list/">Denials and White/Black Listing&lt;/a> task shows how to configure
access policies using black or white list checkers.&lt;/li>
&lt;/ul>
&lt;p>As opposed to the observability and security tasks above, this blog post describes Istio&amp;rsquo;s monitoring and access policies
applied exclusively to the egress traffic.&lt;/p>
&lt;h2 id="before-you-begin">Before you begin&lt;/h2>
&lt;p>Follow the steps in the &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/">Egress Gateway with TLS Origination&lt;/a> example, &lt;strong>with mutual TLS authentication enabled&lt;/strong>, without
the &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway-tls-origination//#cleanup">Cleanup&lt;/a> step.
After completing that example, you can access &lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> from an in-mesh container with &lt;code>curl&lt;/code> installed. This blog post assumes that the &lt;code>SOURCE_POD&lt;/code> environment variable contains the source pod&amp;rsquo;s name and that the container&amp;rsquo;s name is &lt;code>sleep&lt;/code>.&lt;/p>
&lt;h2 id="configure-monitoring-and-access-policies">Configure monitoring and access policies&lt;/h2>
&lt;p>Since you want to accomplish your tasks in a &lt;em>secure way&lt;/em>, you should direct egress traffic through
&lt;em>egress gateway&lt;/em>, as described in the &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/">Egress Gateway with TLS Origination&lt;/a>
task. The &lt;em>secure way&lt;/em> here means that you want to prevent malicious applications from bypassing Istio monitoring and
policy enforcement.&lt;/p>
&lt;p>According to our scenario, the organization performed the instructions in the
&lt;a href="#before-you-begin">Before you begin&lt;/a> section, enabled HTTP traffic to &lt;em>edition.cnn.com&lt;/em>, and configured that traffic
to pass through the egress gateway. The egress gateway performs TLS origination to &lt;em>edition.cnn.com&lt;/em>, so the traffic
leaves the mesh encrypted. At this point, the organization is ready to configure Istio to monitor and apply access policies for
the traffic to &lt;em>edition.cnn.com&lt;/em>.&lt;/p>
&lt;h3 id="logging">Logging&lt;/h3>
&lt;p>Configure Istio to log access to &lt;em>*.cnn.com&lt;/em>. You create a &lt;code>logentry&lt;/code> and two
&lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/adapters/stdio/">stdio&lt;/a> &lt;code>handlers&lt;/code>, one for logging forbidden access
(&lt;em>error&lt;/em> log level) and another one for logging all access to &lt;em>*.cnn.com&lt;/em> (&lt;em>info&lt;/em> log level). Then you create &lt;code>rules&lt;/code> to
direct your &lt;code>logentry&lt;/code> instances to your &lt;code>handlers&lt;/code>. One rule directs access to &lt;em>*.cnn.com/politics&lt;/em> to the handler for
logging forbidden access, another rule directs log entries to the handler that outputs each access to &lt;em>*.cnn.com&lt;/em> as an
&lt;em>info&lt;/em> log entry. To understand the Istio &lt;code>logentries&lt;/code>, &lt;code>rules&lt;/code>, and &lt;code>handlers&lt;/code>, see
&lt;a href="/v1.12/blog/2017/adapter-model/">Istio Adapter Model&lt;/a>. A diagram with the involved entities and dependencies between them
appears below:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:46.46700562636976%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-monitoring-access-control/egress-adapters-monitoring.svg" title="Instances, rules and handlers for egress monitoring">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-monitoring-access-control/egress-adapters-monitoring.svg" alt="Instances, rules and handlers for egress monitoring" />
&lt;/a>
&lt;/div>
&lt;figcaption>Instances, rules and handlers for egress monitoring&lt;/figcaption>
&lt;/figure>
&lt;ol>
&lt;li>&lt;p>Create the &lt;code>logentry&lt;/code>, &lt;code>rules&lt;/code> and &lt;code>handlers&lt;/code>. Note that you specify &lt;code>context.reporter.uid&lt;/code> as
&lt;code>kubernetes://istio-egressgateway&lt;/code> in the rules to get logs from the egress gateway only.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
# Log entry for egress access
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: logentry
metadata:
name: egress-access
namespace: istio-system
spec:
severity: &amp;#39;&amp;#34;info&amp;#34;&amp;#39;
timestamp: request.time
variables:
destination: request.host | &amp;#34;unknown&amp;#34;
path: request.path | &amp;#34;unknown&amp;#34;
responseCode: response.code | 0
responseSize: response.size | 0
reporterUID: context.reporter.uid | &amp;#34;unknown&amp;#34;
sourcePrincipal: source.principal | &amp;#34;unknown&amp;#34;
monitored_resource_type: &amp;#39;&amp;#34;UNSPECIFIED&amp;#34;&amp;#39;
---
# Handler for error egress access entries
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: stdio
metadata:
name: egress-error-logger
namespace: istio-system
spec:
severity_levels:
info: 2 # output log level as error
outputAsJson: true
---
# Rule to handle access to *.cnn.com/politics
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: rule
metadata:
name: handle-politics
namespace: istio-system
spec:
match: request.host.endsWith(&amp;#34;cnn.com&amp;#34;) &amp;amp;&amp;amp; request.path.startsWith(&amp;#34;/politics&amp;#34;) &amp;amp;&amp;amp; context.reporter.uid.startsWith(&amp;#34;kubernetes://istio-egressgateway&amp;#34;)
actions:
- handler: egress-error-logger.stdio
instances:
- egress-access.logentry
---
# Handler for info egress access entries
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: stdio
metadata:
name: egress-access-logger
namespace: istio-system
spec:
severity_levels:
info: 0 # output log level as info
outputAsJson: true
---
# Rule to handle access to *.cnn.com
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: rule
metadata:
name: handle-cnn-access
namespace: istio-system
spec:
match: request.host.endsWith(&amp;#34;.cnn.com&amp;#34;) &amp;amp;&amp;amp; context.reporter.uid.startsWith(&amp;#34;kubernetes://istio-egressgateway&amp;#34;)
actions:
- handler: egress-access-logger.stdio
instances:
- egress-access.logentry
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Send three HTTP requests to &lt;em>cnn.com&lt;/em>, to &lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a>, &lt;a href="https://edition.cnn.com/sport">edition.cnn.com/sport&lt;/a> and &lt;a href="https://edition.cnn.com/health">edition.cnn.com/health&lt;/a>.
All three should return &lt;em>200 OK&lt;/em>.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &amp;#39;curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/health&amp;#39;
200
200
200
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Query the Mixer log and see that the information about the requests appears in the log:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep egress-access | grep cnn | tail -4
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T07:43:24.611462Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/politics&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:200,&amp;#34;responseSize&amp;#34;:1883355,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T07:43:24.886316Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/sport&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:200,&amp;#34;responseSize&amp;#34;:2094561,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T07:43:25.369663Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/health&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:200,&amp;#34;responseSize&amp;#34;:2157009,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;error&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T07:43:24.611462Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/politics&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:200,&amp;#34;responseSize&amp;#34;:1883355,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;}
&lt;/code>&lt;/pre>
&lt;p>You see four log entries related to your three requests. Three &lt;em>info&lt;/em> entries about the access to &lt;em>edition.cnn.com&lt;/em>
and one &lt;em>error&lt;/em> entry about the access to &lt;em>edition.cnn.com/politics&lt;/em>. The service mesh operators can see all the
access instances, and can also search the log for &lt;em>error&lt;/em> log entries that represent forbidden accesses. This is the
first security measure the organization can apply before blocking the forbidden accesses automatically, namely
logging all the forbidden access instances as errors. In some settings this can be a sufficient security measure.&lt;/p>
&lt;p>Note the attributes:&lt;/p>
&lt;ul>
&lt;li>&lt;code>destination&lt;/code>, &lt;code>path&lt;/code>, &lt;code>responseCode&lt;/code>, &lt;code>responseSize&lt;/code> are related to HTTP parameters of the requests&lt;/li>
&lt;li>&lt;code>sourcePrincipal&lt;/code>:&lt;code>cluster.local/ns/default/sa/sleep&lt;/code> - a string that represents the &lt;code>sleep&lt;/code> service account in
the &lt;code>default&lt;/code> namespace&lt;/li>
&lt;li>&lt;code>reporterUID&lt;/code>: &lt;code>kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&lt;/code> - a UID of the reporting pod, in
this case &lt;code>istio-egressgateway-747b6764b8-44rrh&lt;/code> in the &lt;code>istio-system&lt;/code> namespace&lt;/li>
&lt;/ul>&lt;/li>
&lt;/ol>
&lt;h3 id="access-control-by-routing">Access control by routing&lt;/h3>
&lt;p>After enabling logging of access to &lt;em>edition.cnn.com&lt;/em>, automatically enforce an access policy, namely allow
accessing &lt;em>/health&lt;/em> and &lt;em>/sport&lt;/em> URL paths only. Such a simple policy control can be implemented with Istio routing.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Redefine your &lt;code>VirtualService&lt;/code> for &lt;em>edition.cnn.com&lt;/em>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-cnn-through-egress-gateway
spec:
hosts:
- edition.cnn.com
gateways:
- istio-egressgateway
- mesh
http:
- match:
- gateways:
- mesh
port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: cnn
port:
number: 443
weight: 100
- match:
- gateways:
- istio-egressgateway
port: 443
uri:
regex: &amp;#34;/health|/sport&amp;#34;
route:
- destination:
host: edition.cnn.com
port:
number: 443
weight: 100
EOF
&lt;/code>&lt;/pre>
&lt;p>Note that you added a &lt;code>match&lt;/code> by &lt;code>uri&lt;/code> condition that checks that the URL path is
either &lt;em>/health&lt;/em> or &lt;em>/sport&lt;/em>. Also note that this condition is added to the &lt;code>istio-egressgateway&lt;/code>
section of the &lt;code>VirtualService&lt;/code>, since the egress gateway is a hardened component in terms of security (see
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/#additional-security-considerations">egress gateway security considerations&lt;/a>). You don&amp;rsquo;t want any tampering
with your policies.&lt;/p>&lt;/li>
&lt;li>&lt;p>Send the previous three HTTP requests to &lt;em>cnn.com&lt;/em>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &amp;#39;curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/health&amp;#39;
404
200
200
&lt;/code>&lt;/pre>
&lt;p>The request to &lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> returned &lt;em>404 Not Found&lt;/em>, while requests
to &lt;a href="https://edition.cnn.com/sport">edition.cnn.com/sport&lt;/a> and
&lt;a href="https://edition.cnn.com/health">edition.cnn.com/health&lt;/a> returned &lt;em>200 OK&lt;/em>, as expected.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">You may need to wait several seconds for the update of the &lt;code>VirtualService&lt;/code> to propagate to the egress
gateway.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;/li>
&lt;li>&lt;p>Query the Mixer log and see that the information about the requests appears again in the log:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep egress-access | grep cnn | tail -4
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T07:55:59.686082Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/politics&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:404,&amp;#34;responseSize&amp;#34;:0,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T07:55:59.697565Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/sport&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:200,&amp;#34;responseSize&amp;#34;:2094561,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T07:56:00.264498Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/health&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:200,&amp;#34;responseSize&amp;#34;:2157009,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;error&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T07:55:59.686082Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/politics&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:404,&amp;#34;responseSize&amp;#34;:0,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/sleep&amp;#34;}
&lt;/code>&lt;/pre>
&lt;p>You still get info and error messages regarding accesses to
&lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a>, however this time the &lt;code>responseCode&lt;/code> is &lt;code>404&lt;/code>, as
expected.&lt;/p>&lt;/li>
&lt;/ol>
&lt;p>While implementing access control using Istio routing worked for us in this simple case, it would not suffice for more
complex cases. For example, the organization may want to allow access to
&lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> under certain conditions, so more complex policy logic than
just filtering by URL paths will be required. You may want to apply &lt;a href="/v1.12/blog/2017/adapter-model/">Istio Mixer Adapters&lt;/a>,
for example
&lt;a href="https://istio.io/v1.6/docs/tasks/policy-enforcement/denial-and-list/#attribute-based-whitelists-or-blacklists">white lists or black lists&lt;/a>
of allowed/forbidden URL paths, respectively.
&lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/istio.policy.v1beta1/">Policy Rules&lt;/a> allow specifying complex conditions,
specified in a &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/expression-language/">rich expression language&lt;/a>, which
includes AND and OR logical operators. The rules can be reused for both logging and policy checks. More advanced users
may want to apply &lt;a href="/v1.12/docs/concepts/security/#authorization">Istio Role-Based Access Control&lt;/a>.&lt;/p>
&lt;p>An additional aspect is integration with remote access policy systems. If the organization in our use case operates some
&lt;a href="https://en.wikipedia.org/wiki/Identity_management">Identity and Access Management&lt;/a> system, you may want to configure
Istio to use access policy information from such a system. You implement this integration by applying
&lt;a href="/v1.12/blog/2017/adapter-model/">Istio Mixer Adapters&lt;/a>.&lt;/p>
&lt;p>Cancel the access control by routing you used in this section and implement access control by Mixer policy checks
in the next section.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Replace the &lt;code>VirtualService&lt;/code> for &lt;em>edition.cnn.com&lt;/em> with your previous version from the &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/#perform-tls-origination-with-an-egress-gateway">Configure an Egress Gateway&lt;/a> example:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-cnn-through-egress-gateway
spec:
hosts:
- edition.cnn.com
gateways:
- istio-egressgateway
- mesh
http:
- match:
- gateways:
- mesh
port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: cnn
port:
number: 443
weight: 100
- match:
- gateways:
- istio-egressgateway
port: 443
route:
- destination:
host: edition.cnn.com
port:
number: 443
weight: 100
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Send the previous three HTTP requests to &lt;em>cnn.com&lt;/em>, this time you should get three &lt;em>200 OK&lt;/em> responses as
previously:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &amp;#39;curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/health&amp;#39;
200
200
200
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">You may need to wait several seconds for the update of the &lt;code>VirtualService&lt;/code> to propagate to the egress
gateway.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;h3 id="access-control-by-mixer-policy-checks">Access control by Mixer policy checks&lt;/h3>
&lt;p>In this step you use a Mixer
&lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/adapters/list/">&lt;code>Listchecker&lt;/code> adapter&lt;/a>, its whitelist
variety. You define a &lt;code>listentry&lt;/code> with the URL path of the request and a &lt;code>listchecker&lt;/code> to check the &lt;code>listentry&lt;/code> using a
static list of allowed URL paths, specified by the &lt;code>overrides&lt;/code> field. For an external &lt;a href="https://en.wikipedia.org/wiki/Identity_management">Identity and Access Management&lt;/a> system, use the &lt;code>providerurl&lt;/code> field instead. The updated
diagram of the instances, rules and handlers appears below. Note that you reuse the same policy rule, &lt;code>handle-cnn-access&lt;/code>
both for logging and for access policy checks.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:52.79420593027812%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-monitoring-access-control/egress-adapters-monitoring-policy.svg" title="Instances, rules and handlers for egress monitoring and access policies">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-monitoring-access-control/egress-adapters-monitoring-policy.svg" alt="Instances, rules and handlers for egress monitoring and access policies" />
&lt;/a>
&lt;/div>
&lt;figcaption>Instances, rules and handlers for egress monitoring and access policies&lt;/figcaption>
&lt;/figure>
&lt;ol>
&lt;li>&lt;p>Define &lt;code>path-checker&lt;/code> and &lt;code>request-path&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl create -f -
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: listchecker
metadata:
name: path-checker
namespace: istio-system
spec:
overrides: [&amp;#34;/health&amp;#34;, &amp;#34;/sport&amp;#34;] # overrides provide a static list
blacklist: false
---
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: listentry
metadata:
name: request-path
namespace: istio-system
spec:
value: request.path
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Modify the &lt;code>handle-cnn-access&lt;/code> policy rule to send &lt;code>request-path&lt;/code> instances to the &lt;code>path-checker&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
# Rule handle egress access to cnn.com
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: rule
metadata:
name: handle-cnn-access
namespace: istio-system
spec:
match: request.host.endsWith(&amp;#34;.cnn.com&amp;#34;) &amp;amp;&amp;amp; context.reporter.uid.startsWith(&amp;#34;kubernetes://istio-egressgateway&amp;#34;)
actions:
- handler: egress-access-logger.stdio
instances:
- egress-access.logentry
- handler: path-checker.listchecker
instances:
- request-path.listentry
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Perform your usual test by sending HTTP requests to
&lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a>, &lt;a href="https://edition.cnn.com/sport">edition.cnn.com/sport&lt;/a>
and &lt;a href="https://edition.cnn.com/health">edition.cnn.com/health&lt;/a>. As expected, the request to
&lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> returns &lt;em>403&lt;/em> (Forbidden).&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &amp;#39;curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/health&amp;#39;
403
200
200
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h3 id="access-control-by-mixer-policy-checks-part-2">Access control by Mixer policy checks, part 2&lt;/h3>
&lt;p>After the organization in our use case managed to configure logging and access control, it decided to extend its access
policy by allowing the applications with a special
&lt;a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/">Service Account&lt;/a> to access any topic of &lt;em>cnn.com&lt;/em>, without being monitored. You&amp;rsquo;ll see how this requirement can be configured in Istio.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Start the &lt;a href="https://github.com/istio/istio/tree/release-1.12/samples/sleep">sleep&lt;/a> sample with the &lt;code>politics&lt;/code> service account.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/sleep/sleep.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ sed &amp;#39;s/: sleep/: politics/g&amp;#39; @samples/sleep/sleep.yaml@ | kubectl create -f -
serviceaccount &amp;#34;politics&amp;#34; created
service &amp;#34;politics&amp;#34; created
deployment &amp;#34;politics&amp;#34; created
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Define the &lt;code>SOURCE_POD_POLITICS&lt;/code> shell variable to hold the name of the source pod with the &lt;code>politics&lt;/code> service
account, for sending requests to external services.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export SOURCE_POD_POLITICS=$(kubectl get pod -l app=politics -o jsonpath={.items..metadata.name})
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Perform your usual test of sending three HTTP requests this time from &lt;code>SOURCE_POD_POLITICS&lt;/code>.
The request to &lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> returns &lt;em>403&lt;/em>, since you did not configure
the exception for the &lt;em>politics&lt;/em> namespace.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec -it $SOURCE_POD_POLITICS -c politics -- sh -c &amp;#39;curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/health&amp;#39;
403
200
200
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Query the Mixer log and see that the information about the requests from the &lt;em>politics&lt;/em> namespace appears in
the log:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep egress-access | grep cnn | tail -4
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T08:04:42.559812Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/politics&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:403,&amp;#34;responseSize&amp;#34;:84,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/politics&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T08:04:42.568424Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/sport&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:200,&amp;#34;responseSize&amp;#34;:2094561,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/politics&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;error&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T08:04:42.559812Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/politics&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:403,&amp;#34;responseSize&amp;#34;:84,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/politics&amp;#34;}
{&amp;#34;level&amp;#34;:&amp;#34;info&amp;#34;,&amp;#34;time&amp;#34;:&amp;#34;2019-01-29T08:04:42.615641Z&amp;#34;,&amp;#34;instance&amp;#34;:&amp;#34;egress-access.logentry.istio-system&amp;#34;,&amp;#34;destination&amp;#34;:&amp;#34;edition.cnn.com&amp;#34;,&amp;#34;path&amp;#34;:&amp;#34;/health&amp;#34;,&amp;#34;reporterUID&amp;#34;:&amp;#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&amp;#34;,&amp;#34;responseCode&amp;#34;:200,&amp;#34;responseSize&amp;#34;:2157009,&amp;#34;sourcePrincipal&amp;#34;:&amp;#34;cluster.local/ns/default/sa/politics&amp;#34;}
&lt;/code>&lt;/pre>
&lt;p>Note that &lt;code>sourcePrincipal&lt;/code> is &lt;code>cluster.local/ns/default/sa/politics&lt;/code> which represents the &lt;code>politics&lt;/code> service
account in the &lt;code>default&lt;/code> namespace.&lt;/p>&lt;/li>
&lt;li>&lt;p>Redefine &lt;code>handle-cnn-access&lt;/code> and &lt;code>handle-politics&lt;/code> policy rules, to make the applications in the &lt;em>politics&lt;/em>
namespace exempt from monitoring and policy enforcement.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat &amp;lt;&amp;lt;EOF | kubectl apply -f -
# Rule to handle access to *.cnn.com/politics
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: rule
metadata:
name: handle-politics
namespace: istio-system
spec:
match: request.host.endsWith(&amp;#34;cnn.com&amp;#34;) &amp;amp;&amp;amp; context.reporter.uid.startsWith(&amp;#34;kubernetes://istio-egressgateway&amp;#34;) &amp;amp;&amp;amp; request.path.startsWith(&amp;#34;/politics&amp;#34;) &amp;amp;&amp;amp; source.principal != &amp;#34;cluster.local/ns/default/sa/politics&amp;#34;
actions:
- handler: egress-error-logger.stdio
instances:
- egress-access.logentry
---
# Rule handle egress access to cnn.com
apiVersion: &amp;#34;config.istio.io/v1alpha2&amp;#34;
kind: rule
metadata:
name: handle-cnn-access
namespace: istio-system
spec:
match: request.host.endsWith(&amp;#34;.cnn.com&amp;#34;) &amp;amp;&amp;amp; context.reporter.uid.startsWith(&amp;#34;kubernetes://istio-egressgateway&amp;#34;) &amp;amp;&amp;amp; source.principal != &amp;#34;cluster.local/ns/default/sa/politics&amp;#34;
actions:
- handler: egress-access-logger.stdio
instances:
- egress-access.logentry
- handler: path-checker.listchecker
instances:
- request-path.listentry
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Perform your usual test from &lt;code>SOURCE_POD&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &amp;#39;curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/health&amp;#39;
403
200
200
&lt;/code>&lt;/pre>
&lt;p>Since &lt;code>SOURCE_POD&lt;/code> does not have &lt;code>politics&lt;/code> service account, access to
&lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> is forbidden, as previously.&lt;/p>&lt;/li>
&lt;li>&lt;p>Perform the previous test from &lt;code>SOURCE_POD_POLITICS&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl exec -it $SOURCE_POD_POLITICS -c politics -- sh -c &amp;#39;curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &amp;#34;%{http_code}\n&amp;#34; http://edition.cnn.com/health&amp;#39;
200
200
200
&lt;/code>&lt;/pre>
&lt;p>Access to all the topics of &lt;em>edition.cnn.com&lt;/em> is allowed.&lt;/p>&lt;/li>
&lt;li>&lt;p>Examine the Mixer log and see that no more requests with &lt;code>sourcePrincipal&lt;/code> equal
&lt;code>cluster.local/ns/default/sa/politics&lt;/code> appear in the log.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep egress-access | grep cnn | tail -4
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h2 id="comparison-with-https-egress-traffic-control">Comparison with HTTPS egress traffic control&lt;/h2>
&lt;p>In this use case the applications use HTTP and Istio Egress Gateway performs TLS origination for them. Alternatively,
the applications could originate TLS themselves by issuing HTTPS requests to &lt;em>edition.cnn.com&lt;/em>. In this section we
describe both approaches and their pros and cons.&lt;/p>
&lt;p>In the HTTP approach, the requests are sent unencrypted on the local host, intercepted by the Istio sidecar proxy and
forwarded to the egress gateway. Since you configure Istio to use mutual TLS between the sidecar proxy and the egress
gateway, the traffic leaves the pod encrypted. The egress gateway decrypts the traffic, inspects the URL path, the
HTTP method and headers, reports telemetry and performs policy checks. If the request is not blocked by some policy
check, the egress gateway performs TLS origination to the external destination (&lt;em>cnn.com&lt;/em> in our case), so the request
is encrypted again and sent encrypted to the external destination. The diagram below demonstrates the network flow of
this approach. The HTTP protocol inside the gateway designates the protocol as seen by the gateway after decryption.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:64.81718469808756%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-monitoring-access-control/http-to-gateway.svg" title="HTTP egress traffic through an egress gateway">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-monitoring-access-control/http-to-gateway.svg" alt="HTTP egress traffic through an egress gateway" />
&lt;/a>
&lt;/div>
&lt;figcaption>HTTP egress traffic through an egress gateway&lt;/figcaption>
&lt;/figure>
&lt;p>The drawback of this approach is that the requests are sent unencrypted inside the pod, which may be against security
policies in some organizations. Also some SDKs have external service URLs hard-coded, including the protocol, so
sending HTTP requests could be impossible. The advantage of this approach is the ability to inspect HTTP methods,
headers and URL paths, and to apply policies based on them.&lt;/p>
&lt;p>In the HTTPS approach, the requests are encrypted end-to-end, from the application to the external destination. The
diagram below demonstrates the network flow of this approach. The HTTPS protocol inside the gateway designates the
protocol as seen by the gateway.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:64.81718469808756%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-monitoring-access-control/https-to-gateway.svg" title="HTTPS egress traffic through an egress gateway">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-monitoring-access-control/https-to-gateway.svg" alt="HTTPS egress traffic through an egress gateway" />
&lt;/a>
&lt;/div>
&lt;figcaption>HTTPS egress traffic through an egress gateway&lt;/figcaption>
&lt;/figure>
&lt;p>The end-to-end HTTPS is considered a better approach from the security point of view. However, since the traffic is
encrypted the Istio proxies and the egress gateway can only see the source and destination IPs and the &lt;a href="https://en.wikipedia.org/wiki/Server_Name_Indication">SNI&lt;/a> of the destination. Since you configure Istio to use mutual TLS between the sidecar proxy
and the egress gateway, the &lt;a href="/v1.12/docs/concepts/security/#istio-identity">identity of the source&lt;/a> is also known.
The gateway is unable to inspect the URL path, the HTTP method and the headers of the requests, so no monitoring and
policies based on the HTTP information can be possible.
In our use case, the organization would be able to allow access to &lt;em>edition.cnn.com&lt;/em> and to specify which applications
are allowed to access &lt;em>edition.cnn.com&lt;/em>.
However, it will not be possible to allow or block access to specific URL paths of &lt;em>edition.cnn.com&lt;/em>.
Neither blocking access to &lt;a href="https://edition.cnn.com/politics">edition.cnn.com/politics&lt;/a> nor monitoring such access are
possible with the HTTPS approach.&lt;/p>
&lt;p>We guess that each organization will consider the pros and cons of the two approaches and choose the one most
appropriate to its needs.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>In this blog post we showed how different monitoring and policy mechanisms of Istio can be applied to HTTP egress
traffic. Monitoring can be implemented by configuring a logging adapter. Access
policies can be implemented by configuring &lt;code>VirtualServices&lt;/code> or by configuring various policy check adapters. We
demonstrated a simple policy that allowed certain URL paths only. We also showed a more complex policy that extended the
simple policy by making an exemption to the applications with a certain service account. Finally, we compared
HTTP-with-TLS-origination egress traffic with HTTPS egress traffic, in terms of control possibilities by Istio.&lt;/p>
&lt;h2 id="cleanup">Cleanup&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Perform the instructions in &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway//#cleanup">Cleanup&lt;/a> section of the
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-gateway/">Configure an Egress Gateway&lt;/a> example.&lt;/p>&lt;/li>
&lt;li>&lt;p>Delete the logging and policy checks configuration:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete logentry egress-access -n istio-system
$ kubectl delete stdio egress-error-logger -n istio-system
$ kubectl delete stdio egress-access-logger -n istio-system
$ kubectl delete rule handle-politics -n istio-system
$ kubectl delete rule handle-cnn-access -n istio-system
$ kubectl delete -n istio-system listchecker path-checker
$ kubectl delete -n istio-system listentry request-path
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Delete the &lt;em>politics&lt;/em> source pod:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/sleep/sleep.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ sed &amp;#39;s/: sleep/: politics/g&amp;#39; @samples/sleep/sleep.yaml@ | kubectl delete -f -
serviceaccount &amp;#34;politics&amp;#34; deleted
service &amp;#34;politics&amp;#34; deleted
deployment &amp;#34;politics&amp;#34; deleted
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;/ol></description><pubDate>Fri, 22 Jun 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/egress-monitoring-access-control/</link><author>Vadim Eisenberg and Ronen Schaffer (IBM)</author><guid isPermaLink="true">/v1.12/blog/2018/egress-monitoring-access-control/</guid><category>egress</category><category>traffic-management</category><category>access-control</category><category>monitoring</category></item><item><title>Introducing the Istio v1alpha3 routing API</title><description>
&lt;p>Up until now, Istio has provided a simple API for traffic management using four configuration resources:
&lt;code>RouteRule&lt;/code>, &lt;code>DestinationPolicy&lt;/code>, &lt;code>EgressRule&lt;/code>, and (Kubernetes) &lt;code>Ingress&lt;/code>.
With this API, users have been able to easily manage the flow of traffic in an Istio service mesh.
The API has allowed users to route requests to specific versions of services, inject delays and failures for resilience
testing, add timeouts and circuit breakers, and more, all without changing the application code itself.&lt;/p>
&lt;p>While this functionality has proven to be a very compelling part of Istio, user feedback has also shown that this API does
have some shortcomings, specifically when using it to manage very large applications containing thousands of services, and
when working with protocols other than HTTP. Furthermore, the use of Kubernetes &lt;code>Ingress&lt;/code> resources to configure external
traffic has proven to be woefully insufficient for our needs.&lt;/p>
&lt;p>To address these, and other concerns, a new traffic management API, a.k.a. &lt;code>v1alpha3&lt;/code>, is being introduced, which will
completely replace the previous API going forward. Although the &lt;code>v1alpha3&lt;/code> model is fundamentally the same, it is not
backward compatible and will require manual conversion from the old API.&lt;/p>
&lt;p>To justify this disruption, the &lt;code>v1alpha3&lt;/code> API has gone through a long and painstaking community
review process that has hopefully resulted in a greatly improved API that will stand the test of time. In this article,
we will introduce the new configuration model and attempt to explain some of the motivation and design principles that
influenced it.&lt;/p>
&lt;h2 id="design-principles">Design principles&lt;/h2>
&lt;p>A few key design principles played a role in the routing model redesign:&lt;/p>
&lt;ul>
&lt;li>Explicitly model infrastructure as well as intent. For example, in addition to configuring an ingress gateway, the
component (controller) implementing it can also be specified.&lt;/li>
&lt;li>The authoring model should be &amp;ldquo;producer oriented&amp;rdquo; and &amp;ldquo;host centric&amp;rdquo; as opposed to compositional. For example, all
rules associated with a particular host are configured together, instead of individually.&lt;/li>
&lt;li>Clear separation of routing from post-routing behaviors.&lt;/li>
&lt;/ul>
&lt;h2 id="configuration-resources-in-v1alpha3">Configuration resources in v1alpha3&lt;/h2>
&lt;p>A typical mesh will have one or more load balancers (we call them gateways)
that terminate TLS from external networks and allow traffic into the mesh.
Traffic then flows through internal services via sidecar gateways.
It is also common for applications to consume external
services (e.g., Google Maps API). These may be called directly or, in certain deployments, all traffic
exiting the mesh may be forced through dedicated egress gateways. The following diagram depicts
this mental model.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:35.204472660409245%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/v1alpha3-routing/gateways.svg" title="Gateways in an Istio service mesh">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/v1alpha3-routing/gateways.svg" alt="Role of gateways in the mesh" />
&lt;/a>
&lt;/div>
&lt;figcaption>Gateways in an Istio service mesh&lt;/figcaption>
&lt;/figure>
&lt;p>With the above setup in mind, &lt;code>v1alpha3&lt;/code> introduces the following new
configuration resources to control traffic routing into, within, and out of the mesh.&lt;/p>
&lt;ol>
&lt;li>&lt;code>Gateway&lt;/code>&lt;/li>
&lt;li>&lt;code>VirtualService&lt;/code>&lt;/li>
&lt;li>&lt;code>DestinationRule&lt;/code>&lt;/li>
&lt;li>&lt;code>ServiceEntry&lt;/code>&lt;/li>
&lt;/ol>
&lt;p>&lt;code>VirtualService&lt;/code>, &lt;code>DestinationRule&lt;/code>, and &lt;code>ServiceEntry&lt;/code> replace &lt;code>RouteRule&lt;/code>,
&lt;code>DestinationPolicy&lt;/code>, and &lt;code>EgressRule&lt;/code> respectively. The &lt;code>Gateway&lt;/code> is a
platform independent abstraction to model the traffic flowing into
dedicated middleboxes.&lt;/p>
&lt;p>The figure below depicts the flow of control across configuration
resources.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:41.164966727369595%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/v1alpha3-routing/virtualservices-destrules.svg" title="Relationship between different v1alpha3 elements">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/v1alpha3-routing/virtualservices-destrules.svg" alt="Relationship between different v1alpha3 elements" />
&lt;/a>
&lt;/div>
&lt;figcaption>Relationship between different v1alpha3 elements&lt;/figcaption>
&lt;/figure>
&lt;h3 id="gateway">&lt;code>Gateway&lt;/code>&lt;/h3>
&lt;p>A &lt;a href="/v1.12/docs/reference/config/networking/gateway/">&lt;code>Gateway&lt;/code>&lt;/a>
configures a load balancer for HTTP/TCP traffic, regardless of
where it will be running. Any number of gateways can exist within the mesh
and multiple different gateway implementations can co-exist. In fact, a
gateway configuration can be bound to a particular workload by specifying
the set of workload (pod) labels as part of the configuration, allowing
users to reuse off the shelf network appliances by writing a simple gateway
controller.&lt;/p>
&lt;p>For ingress traffic management, you might ask: &lt;em>Why not reuse Kubernetes Ingress APIs&lt;/em>?
The Ingress APIs proved to be incapable of expressing Istio&amp;rsquo;s routing needs.
By trying to draw a common denominator across different HTTP proxies, the
Ingress is only able to support the most basic HTTP routing and ends up
pushing every other feature of modern proxies into non-portable
annotations.&lt;/p>
&lt;p>Istio &lt;code>Gateway&lt;/code> overcomes the &lt;code>Ingress&lt;/code> shortcomings by separating the
L4-L6 spec from L7. It only configures the L4-L6 functions (e.g., ports to
expose, TLS configuration) that are uniformly implemented by all good L7
proxies. Users can then use standard Istio rules to control HTTP
requests as well as TCP traffic entering a &lt;code>Gateway&lt;/code> by binding a
&lt;code>VirtualService&lt;/code> to it.&lt;/p>
&lt;p>For example, the following simple &lt;code>Gateway&lt;/code> configures a load balancer
to allow external https traffic for host &lt;code>bookinfo.com&lt;/code> into the mesh:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: bookinfo-gateway
spec:
servers:
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- bookinfo.com
tls:
mode: SIMPLE
serverCertificate: /tmp/tls.crt
privateKey: /tmp/tls.key
&lt;/code>&lt;/pre>
&lt;p>To configure the corresponding routes, a &lt;code>VirtualService&lt;/code> (described in the &lt;a href="#virtualservice">following section&lt;/a>)
must be defined for the same host and bound to the &lt;code>Gateway&lt;/code> using
the &lt;code>gateways&lt;/code> field in the configuration:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: bookinfo
spec:
hosts:
- bookinfo.com
gateways:
- bookinfo-gateway # &amp;lt;---- bind to gateway
http:
- match:
- uri:
prefix: /reviews
route:
...
&lt;/code>&lt;/pre>
&lt;p>The &lt;code>Gateway&lt;/code> can be used to model an edge-proxy or a purely internal proxy
as shown in the first figure. Irrespective of the location, all gateways
can be configured and controlled in the same way.&lt;/p>
&lt;h3 id="virtualservice">&lt;code>VirtualService&lt;/code>&lt;/h3>
&lt;p>Replacing route rules with something called &amp;ldquo;virtual services” might seem peculiar at first, but in reality it’s
fundamentally a much better name for what is being configured, especially after redesigning the API to address the
scalability issues with the previous model.&lt;/p>
&lt;p>In effect, what has changed is that instead of configuring routing using a set of individual configuration resources
(rules) for a particular destination service, each containing a precedence field to control the order of evaluation, we
now configure the (virtual) destination itself, with all of its rules in an ordered list within a corresponding
&lt;a href="/v1.12/docs/reference/config/networking/virtual-service/">&lt;code>VirtualService&lt;/code>&lt;/a> resource.
For example, where previously we had two &lt;code>RouteRule&lt;/code> resources for the
&lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo&lt;/a> application’s &lt;code>reviews&lt;/code> service, like this:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: config.istio.io/v1alpha2
kind: RouteRule
metadata:
name: reviews-default
spec:
destination:
name: reviews
precedence: 1
route:
- labels:
version: v1
---
apiVersion: config.istio.io/v1alpha2
kind: RouteRule
metadata:
name: reviews-test-v2
spec:
destination:
name: reviews
precedence: 2
match:
request:
headers:
cookie:
regex: &amp;#34;^(.*?;)?(user=jason)(;.*)?$&amp;#34;
route:
- labels:
version: v2
&lt;/code>&lt;/pre>
&lt;p>In &lt;code>v1alpha3&lt;/code>, we provide the same configuration in a single &lt;code>VirtualService&lt;/code> resource:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- match:
- headers:
cookie:
regex: &amp;#34;^(.*?;)?(user=jason)(;.*)?$&amp;#34;
route:
- destination:
host: reviews
subset: v2
- route:
- destination:
host: reviews
subset: v1
&lt;/code>&lt;/pre>
&lt;p>As you can see, both of the rules for the &lt;code>reviews&lt;/code> service are consolidated in one place, which at first may or may not
seem preferable. However, if you look closer at this new model, you’ll see there are fundamental differences that make
&lt;code>v1alpha3&lt;/code> vastly more functional.&lt;/p>
&lt;p>First of all, notice that the destination service for the &lt;code>VirtualService&lt;/code> is specified using a &lt;code>hosts&lt;/code> field (repeated field, in fact) and is then again specified in a &lt;code>destination&lt;/code> field of each of the route specifications. This is a
very important difference from the previous model.&lt;/p>
&lt;p>A &lt;code>VirtualService&lt;/code> describes the mapping between one or more user-addressable destinations to the actual destination workloads inside the mesh. In our example, they are the same, however, the user-addressed hosts can be any DNS
names with optional wildcard prefix or CIDR prefix that will be used to address the service. This can be particularly
useful in facilitating turning monoliths into a composite service built out of distinct microservices without requiring the
consumers of the service to adapt to the transition.&lt;/p>
&lt;p>For example, the following rule allows users to address both the &lt;code>reviews&lt;/code> and &lt;code>ratings&lt;/code> services of the Bookinfo application
as if they are parts of a bigger (virtual) service at &lt;code>http://bookinfo.com/&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: bookinfo
spec:
hosts:
- bookinfo.com
http:
- match:
- uri:
prefix: /reviews
route:
- destination:
host: reviews
- match:
- uri:
prefix: /ratings
route:
- destination:
host: ratings
...
&lt;/code>&lt;/pre>
&lt;p>The hosts of a &lt;code>VirtualService&lt;/code> do not actually have to be part of the service registry, they are simply virtual
destinations. This allows users to model traffic for virtual hosts that do not have routable entries inside the mesh.
These hosts can be exposed outside the mesh by binding the &lt;code>VirtualService&lt;/code> to a &lt;code>Gateway&lt;/code> configuration for the same host
(as described in the &lt;a href="#gateway">previous section&lt;/a>).&lt;/p>
&lt;p>In addition to this fundamental restructuring, &lt;code>VirtualService&lt;/code> includes several other important changes:&lt;/p>
&lt;ol>
&lt;li>&lt;p>Multiple match conditions can be expressed inside the &lt;code>VirtualService&lt;/code> configuration, reducing the need for redundant
rules.&lt;/p>&lt;/li>
&lt;li>&lt;p>Each service version has a name (called a service subset). The set of pods/VMs belonging to a subset is defined in a
&lt;code>DestinationRule&lt;/code>, described in the following section.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;code>VirtualService&lt;/code> hosts can be specified using wildcard DNS prefixes to create a single rule for all matching services.
For example, in Kubernetes, to apply the same rewrite rule for all services in the &lt;code>foo&lt;/code> namespace, the &lt;code>VirtualService&lt;/code>
would use &lt;code>*.foo.svc.cluster.local&lt;/code> as the host.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h3 id="destinationrule">&lt;code>DestinationRule&lt;/code>&lt;/h3>
&lt;p>A &lt;a href="/v1.12/docs/reference/config/networking/destination-rule/">&lt;code>DestinationRule&lt;/code>&lt;/a>
configures the set of policies to be applied while forwarding traffic to a service. They are
intended to be authored by service owners, describing the circuit breakers, load balancer settings, TLS settings, etc..
&lt;code>DestinationRule&lt;/code> is more or less the same as its predecessor, &lt;code>DestinationPolicy&lt;/code>, with the following exceptions:&lt;/p>
&lt;ol>
&lt;li>The &lt;code>host&lt;/code> of a &lt;code>DestinationRule&lt;/code> can include wildcard prefixes, allowing a single rule to be specified for many actual
services.&lt;/li>
&lt;li>A &lt;code>DestinationRule&lt;/code> defines addressable &lt;code>subsets&lt;/code> (i.e., named versions) of the corresponding destination host. These
subsets are used in &lt;code>VirtualService&lt;/code> route specifications when sending traffic to specific versions of the service.
Naming versions this way allows us to cleanly refer to them across different virtual services, simplify the stats that
Istio proxies emit, and to encode subsets in SNI headers.&lt;/li>
&lt;/ol>
&lt;p>A &lt;code>DestinationRule&lt;/code> that configures policies and subsets for the reviews service might look something like this:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
trafficPolicy:
loadBalancer:
simple: RANDOM
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
- name: v3
labels:
version: v3
&lt;/code>&lt;/pre>
&lt;p>Notice that, unlike &lt;code>DestinationPolicy&lt;/code>, multiple policies (e.g., default and v2-specific) are specified in a single
&lt;code>DestinationRule&lt;/code> configuration.&lt;/p>
&lt;h3 id="serviceentry">&lt;code>ServiceEntry&lt;/code>&lt;/h3>
&lt;p>&lt;a href="/v1.12/docs/reference/config/networking/service-entry/">&lt;code>ServiceEntry&lt;/code>&lt;/a>
is used to add additional entries into the service registry that Istio maintains internally.
It is most commonly used to allow one to model traffic to external dependencies of the mesh
such as APIs consumed from the web or traffic to services in legacy infrastructure.&lt;/p>
&lt;p>Everything you could previously configure using an &lt;code>EgressRule&lt;/code> can just as easily be done with a &lt;code>ServiceEntry&lt;/code>.
For example, access to a simple external service from inside the mesh can be enabled using a configuration
something like this:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: foo-ext
spec:
hosts:
- foo.com
ports:
- number: 80
name: http
protocol: HTTP
&lt;/code>&lt;/pre>
&lt;p>That said, &lt;code>ServiceEntry&lt;/code> has significantly more functionality than its predecessor.
First of all, a &lt;code>ServiceEntry&lt;/code> is not limited to external service configuration,
it can be of two types: mesh-internal or mesh-external.
Mesh-internal entries are like all other internal services but are used to explicitly add services
to the mesh. They can be used to add services as part of expanding the service mesh to include unmanaged infrastructure
(e.g., VMs added to a Kubernetes-based service mesh).
Mesh-external entries represent services external to the mesh.
For them, mutual TLS authentication is disabled and policy enforcement is performed on the client-side,
instead of on the usual server-side for internal service requests.&lt;/p>
&lt;p>Because a &lt;code>ServiceEntry&lt;/code> configuration simply adds a destination to the internal service registry, it can be
used in conjunction with a &lt;code>VirtualService&lt;/code> and/or &lt;code>DestinationRule&lt;/code>, just like any other service in the registry.
The following &lt;code>DestinationRule&lt;/code>, for example, can be used to initiate mutual TLS connections for an external service:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: foo-ext
spec:
host: foo.com
trafficPolicy:
tls:
mode: MUTUAL
clientCertificate: /etc/certs/myclientcert.pem
privateKey: /etc/certs/client_private_key.pem
caCertificates: /etc/certs/rootcacerts.pem
&lt;/code>&lt;/pre>
&lt;p>In addition to its expanded generality, &lt;code>ServiceEntry&lt;/code> provides several other improvements over &lt;code>EgressRule&lt;/code>
including the following:&lt;/p>
&lt;ol>
&lt;li>A single &lt;code>ServiceEntry&lt;/code> can configure multiple service endpoints, which previously would have required multiple
&lt;code>EgressRules&lt;/code>.&lt;/li>
&lt;li>The resolution mode for the endpoints is now configurable (&lt;code>NONE&lt;/code>, &lt;code>STATIC&lt;/code>, or &lt;code>DNS&lt;/code>).&lt;/li>
&lt;li>Additionally, we are working on addressing another pain point: the need to access secure external services over plain
text ports (e.g., &lt;code>http://google.com:443&lt;/code>). This should be fixed in the coming weeks, allowing you to directly access
&lt;code>https://google.com&lt;/code> from your application. Stay tuned for an Istio patch release (0.8.x) that addresses this limitation.&lt;/li>
&lt;/ol>
&lt;h2 id="creating-and-deleting-v1alpha3-route-rules">Creating and deleting v1alpha3 route rules&lt;/h2>
&lt;p>Because all route rules for a given destination are now stored together as an ordered
list in a single &lt;code>VirtualService&lt;/code> resource, adding a second and subsequent rules for a particular destination
is no longer done by creating a new (&lt;code>RouteRule&lt;/code>) resource, but instead by updating the one-and-only &lt;code>VirtualService&lt;/code>
resource for the destination.&lt;/p>
&lt;p>old routing rules:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f my-second-rule-for-destination-abc.yaml
&lt;/code>&lt;/pre>
&lt;p>&lt;code>v1alpha3&lt;/code> routing rules:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f my-updated-rules-for-destination-abc.yaml
&lt;/code>&lt;/pre>
&lt;p>Deleting route rules other than the last one for a particular destination is also done by updating
the existing resource using &lt;code>kubectl apply&lt;/code>.&lt;/p>
&lt;p>When adding or removing routes that refer to service versions, the &lt;code>subsets&lt;/code> will need to be updated in
the service&amp;rsquo;s corresponding &lt;code>DestinationRule&lt;/code>.
As you might have guessed, this is also done using &lt;code>kubectl apply&lt;/code>.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>The Istio &lt;code>v1alpha3&lt;/code> routing API has significantly more functionality than
its predecessor, but unfortunately is not backwards compatible, requiring a
one time manual conversion. The previous configuration resources,
&lt;code>RouteRule&lt;/code>, &lt;code>DesintationPolicy&lt;/code>, and &lt;code>EgressRule&lt;/code>, will not be supported
from Istio 0.9 onwards. Kubernetes users can continue to use &lt;code>Ingress&lt;/code> to
configure their edge load balancers for basic routing. However, advanced
routing features (e.g., traffic split across two versions) will require use
of &lt;code>Gateway&lt;/code>, a significantly more functional and highly
recommended &lt;code>Ingress&lt;/code> replacement.&lt;/p>
&lt;h2 id="acknowledgments">Acknowledgments&lt;/h2>
&lt;p>Credit for the routing model redesign and implementation work goes to the
following people (in alphabetical order):&lt;/p>
&lt;ul>
&lt;li>Frank Budinsky (IBM)&lt;/li>
&lt;li>Zack Butcher (Google)&lt;/li>
&lt;li>Greg Hanson (IBM)&lt;/li>
&lt;li>Costin Manolache (Google)&lt;/li>
&lt;li>Martin Ostrowski (Google)&lt;/li>
&lt;li>Shriram Rajagopalan (VMware)&lt;/li>
&lt;li>Louis Ryan (Google)&lt;/li>
&lt;li>Isaiah Snell-Feikema (IBM)&lt;/li>
&lt;li>Kuat Yessenov (Google)&lt;/li>
&lt;/ul></description><pubDate>Wed, 25 Apr 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/v1alpha3-routing/</link><author>Frank Budinsky (IBM) and Shriram Rajagopalan (VMware)</author><guid isPermaLink="true">/v1.12/blog/2018/v1alpha3-routing/</guid><category>traffic-management</category></item><item><title>Configuring Istio Ingress with AWS NLB</title><description>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">This post was updated on January 16, 2019 to include some usage warnings.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>This post provides instructions to use and configure ingress Istio with &lt;a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html">AWS Network Load Balancer&lt;/a>.&lt;/p>
&lt;p>Network load balancer (NLB) could be used instead of classical load balancer. You can see the &lt;a href="https://aws.amazon.com/elasticloadbalancing/details/#Product_comparisons">comparison&lt;/a> between different AWS &lt;code>loadbalancer&lt;/code> for more explanation.&lt;/p>
&lt;h2 id="prerequisites">Prerequisites&lt;/h2>
&lt;p>The following instructions require a Kubernetes &lt;strong>1.9.0 or newer&lt;/strong> cluster.&lt;/p>
&lt;div>
&lt;aside class="callout warning">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-warning"/>&lt;/svg>
&lt;/div>
&lt;div class="content">&lt;p>Usage of AWS &lt;code>nlb&lt;/code> on Kubernetes is an Alpha feature and not recommended for production clusters.&lt;/p>
&lt;p>Usage of AWS &lt;code>nlb&lt;/code> does not support the creation of two or more Kubernetes clusters running Istio in the same zone as a result of &lt;a href="https://github.com/kubernetes/kubernetes/issues/69264">Kubernetes Bug #69264&lt;/a>.&lt;/p>
&lt;/div>
&lt;/aside>
&lt;/div>
&lt;h2 id="iam-policy">IAM policy&lt;/h2>
&lt;p>You need to apply policy on the master role in order to be able to provision network load balancer.&lt;/p>
&lt;ol>
&lt;li>&lt;p>In AWS &lt;code>iam&lt;/code> console click on policies and click on create a new one:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:52.430278884462155%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/aws-nlb/createpolicystart.png" title="Create a new policy">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/aws-nlb/createpolicystart.png" alt="Create a new policy" />
&lt;/a>
&lt;/div>
&lt;figcaption>Create a new policy&lt;/figcaption>
&lt;/figure>&lt;/li>
&lt;li>&lt;p>Select &lt;code>json&lt;/code>:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:50.63492063492063%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/aws-nlb/createpolicyjson.png" title="Select json">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/aws-nlb/createpolicyjson.png" alt="Select json" />
&lt;/a>
&lt;/div>
&lt;figcaption>Select json&lt;/figcaption>
&lt;/figure>&lt;/li>
&lt;li>&lt;p>Copy/paste text below:&lt;/p>
&lt;pre>&lt;code class='language-json' data-expandlinks='true' data-repo='istio' >{
&amp;#34;Version&amp;#34;: &amp;#34;2012-10-17&amp;#34;,
&amp;#34;Statement&amp;#34;: [
{
&amp;#34;Sid&amp;#34;: &amp;#34;kopsK8sNLBMasterPermsRestrictive&amp;#34;,
&amp;#34;Effect&amp;#34;: &amp;#34;Allow&amp;#34;,
&amp;#34;Action&amp;#34;: [
&amp;#34;ec2:DescribeVpcs&amp;#34;,
&amp;#34;elasticloadbalancing:AddTags&amp;#34;,
&amp;#34;elasticloadbalancing:CreateListener&amp;#34;,
&amp;#34;elasticloadbalancing:CreateTargetGroup&amp;#34;,
&amp;#34;elasticloadbalancing:DeleteListener&amp;#34;,
&amp;#34;elasticloadbalancing:DeleteTargetGroup&amp;#34;,
&amp;#34;elasticloadbalancing:DescribeListeners&amp;#34;,
&amp;#34;elasticloadbalancing:DescribeLoadBalancerPolicies&amp;#34;,
&amp;#34;elasticloadbalancing:DescribeTargetGroups&amp;#34;,
&amp;#34;elasticloadbalancing:DescribeTargetHealth&amp;#34;,
&amp;#34;elasticloadbalancing:ModifyListener&amp;#34;,
&amp;#34;elasticloadbalancing:ModifyTargetGroup&amp;#34;,
&amp;#34;elasticloadbalancing:RegisterTargets&amp;#34;,
&amp;#34;elasticloadbalancing:SetLoadBalancerPoliciesOfListener&amp;#34;
],
&amp;#34;Resource&amp;#34;: [
&amp;#34;*&amp;#34;
]
},
{
&amp;#34;Effect&amp;#34;: &amp;#34;Allow&amp;#34;,
&amp;#34;Action&amp;#34;: [
&amp;#34;ec2:DescribeVpcs&amp;#34;,
&amp;#34;ec2:DescribeRegions&amp;#34;
],
&amp;#34;Resource&amp;#34;: &amp;#34;*&amp;#34;
}
]
}
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Click review policy, fill all fields and click create policy:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:60.08097165991902%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/aws-nlb/create_policy.png" title="Validate policy">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/aws-nlb/create_policy.png" alt="Validate policy" />
&lt;/a>
&lt;/div>
&lt;figcaption>Validate policy&lt;/figcaption>
&lt;/figure>&lt;/li>
&lt;li>&lt;p>Click on roles, select you master role nodes, and click attach policy:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:30.328324986087924%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/aws-nlb/roles_summary.png" title="Attach policy">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/aws-nlb/roles_summary.png" alt="Attach policy" />
&lt;/a>
&lt;/div>
&lt;figcaption>Attach policy&lt;/figcaption>
&lt;/figure>&lt;/li>
&lt;li>&lt;p>Your policy is now attach to your master node.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h2 id="generate-the-istio-manifest">Generate the Istio manifest&lt;/h2>
&lt;p>To use an AWS &lt;code>nlb&lt;/code> load balancer, it is necessary to add an AWS specific
annotation to the Istio installation. These instructions explain how to
add the annotation.&lt;/p>
&lt;p>Save this as the file &lt;code>override.yaml&lt;/code>:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >gateways:
istio-ingressgateway:
serviceAnnotations:
service.beta.kubernetes.io/aws-load-balancer-type: &amp;#34;nlb&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>Generate a manifest with Helm:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ helm template install/kubernetes/helm/istio --namespace istio -f override.yaml &amp;gt; $HOME/istio.yaml
&lt;/code>&lt;/pre></description><pubDate>Fri, 20 Apr 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/aws-nlb/</link><author>Julien SENON</author><guid isPermaLink="true">/v1.12/blog/2018/aws-nlb/</guid><category>ingress</category><category>traffic-management</category><category>aws</category></item><item><title>Istio Soft Multi-Tenancy Support</title><description>
&lt;p>Multi-tenancy is commonly used in many environments across many different applications,
but the implementation details and functionality provided on a per tenant basis does not
follow one model in all environments. The &lt;a href="https://github.com/kubernetes/community/blob/master/wg-multitenancy/README.md">Kubernetes multi-tenancy working group&lt;/a>
is working to define the multi-tenant use cases and functionality that should be available
within Kubernetes. However, from their work so far it is clear that only &amp;ldquo;soft multi-tenancy&amp;rdquo;
is possible due to the inability to fully protect against malicious containers or workloads
gaining access to other tenant&amp;rsquo;s pods or kernel resources.&lt;/p>
&lt;h2 id="soft-multi-tenancy">Soft multi-tenancy&lt;/h2>
&lt;p>For this blog, &amp;ldquo;soft multi-tenancy&amp;rdquo; is defined as having a single Kubernetes control plane
with multiple Istio control planes and multiple meshes, one control plane and one mesh
per tenant. The cluster administrator gets control and visibility across all the Istio
control planes, while the tenant administrator only gets control of a specific Istio
instance. Separation between the tenants is provided by Kubernetes namespaces and RBAC.&lt;/p>
&lt;p>One use case for this deployment model is a shared corporate infrastructure where malicious
actions are not expected, but a clean separation of the tenants is still required.&lt;/p>
&lt;p>Potential future Istio multi-tenant deployment models are described at the bottom of this
blog.&lt;/p>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">This blog is a high-level description of how to deploy Istio in a
limited multi-tenancy environment. The &lt;a href="/v1.12/docs/">docs&lt;/a> section will be updated
when official multi-tenancy support is provided.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;h2 id="deployment">Deployment&lt;/h2>
&lt;h3 id="multiple-istio-control-planes">Multiple Istio control planes&lt;/h3>
&lt;p>Deploying multiple Istio control planes starts by replacing all &lt;code>namespace&lt;/code> references
in a manifest file with the desired namespace. Using &lt;code>istio.yaml&lt;/code> as an example, if two tenant
level Istio control planes are required; the first can use the &lt;code>istio.yaml&lt;/code> default name of
&lt;code>istio-system&lt;/code> and a second control plane can be created by generating a new yaml file with
a different namespace. As an example, the following command creates a yaml file with
the Istio namespace of &lt;code>istio-system1&lt;/code>.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ cat istio.yaml | sed s/istio-system/istio-system1/g &amp;gt; istio-system1.yaml
&lt;/code>&lt;/pre>
&lt;p>The &lt;code>istio.yaml&lt;/code> file contains the details of the Istio control plane deployment, including the
pods that make up the control plane (Mixer, Pilot, Ingress, Galley, CA). Deploying the two Istio
control plane yaml files:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f install/kubernetes/istio.yaml
$ kubectl apply -f install/kubernetes/istio-system1.yaml
&lt;/code>&lt;/pre>
&lt;p>Results in two Istio control planes running in two namespaces.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
istio-system istio-ca-ffbb75c6f-98w6x 1/1 Running 0 15d
istio-system istio-ingress-68d65fc5c6-dnvfl 1/1 Running 0 15d
istio-system istio-mixer-5b9f8dffb5-8875r 3/3 Running 0 15d
istio-system istio-pilot-678fc976c8-b8tv6 2/2 Running 0 15d
istio-system1 istio-ca-5f496fdbcd-lqhlk 1/1 Running 0 15d
istio-system1 istio-ingress-68d65fc5c6-2vldg 1/1 Running 0 15d
istio-system1 istio-mixer-7d4f7b9968-66z44 3/3 Running 0 15d
istio-system1 istio-pilot-5bb6b7669c-779vb 2/2 Running 0 15d
&lt;/code>&lt;/pre>
&lt;p>The Istio &lt;a href="/v1.12/docs/setup/additional-setup/sidecar-injection/">sidecar&lt;/a>
and &lt;a href="/v1.12/docs/tasks/observability/">addons&lt;/a>, if required, manifests must also be
deployed to match the configured &lt;code>namespace&lt;/code> in use by the tenant&amp;rsquo;s Istio
control plane.&lt;/p>
&lt;p>The execution of these two yaml files is the responsibility of the cluster
administrator, not the tenant level administrator. Additional RBAC restrictions will also
need to be configured and applied by the cluster administrator, limiting the tenant
administrator to only the assigned namespace.&lt;/p>
&lt;h3 id="split-common-and-namespace-specific-resources">Split common and namespace specific resources&lt;/h3>
&lt;p>The manifest files in the Istio repositories create both common resources that would
be used by all Istio control planes as well as resources that are replicated per control
plane. Although it is a simple matter to deploy multiple control planes by replacing the
&lt;code>istio-system&lt;/code> namespace references as described above, a better approach is to split the
manifests into a common part that is deployed once for all tenants and a tenant
specific part. For the &lt;a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions">Custom Resource Definitions&lt;/a>, the roles and the role
bindings should be separated out from the provided Istio manifests. Additionally, the
roles and role bindings in the provided Istio manifests are probably unsuitable for a
multi-tenant environment and should be modified or augmented as described in the next
section.&lt;/p>
&lt;h3 id="kubernetes-rbac-for-istio-control-plane-resources">Kubernetes RBAC for Istio control plane resources&lt;/h3>
&lt;p>To restrict a tenant administrator to a single Istio namespace, the cluster
administrator would create a manifest containing, at a minimum, a &lt;code>Role&lt;/code> and &lt;code>RoleBinding&lt;/code>
similar to the one below. In this example, a tenant administrator named &lt;em>sales-admin&lt;/em>
is limited to the namespace &lt;code>istio-system1&lt;/code>. A completed manifest would contain many
more &lt;code>apiGroups&lt;/code> under the &lt;code>Role&lt;/code> providing resource access to the tenant administrator.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: istio-system1
name: ns-access-for-sales-admin-istio-system1
rules:
- apiGroups: [&amp;#34;&amp;#34;] # &amp;#34;&amp;#34; indicates the core API group
resources: [&amp;#34;*&amp;#34;]
verbs: [&amp;#34;*&amp;#34;]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: access-all-istio-system1
namespace: istio-system1
subjects:
- kind: User
name: sales-admin
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: ns-access-for-sales-admin-istio-system1
apiGroup: rbac.authorization.k8s.io
&lt;/code>&lt;/pre>
&lt;h3 id="watching-specific-namespaces-for-service-discovery">Watching specific namespaces for service discovery&lt;/h3>
&lt;p>In addition to creating RBAC rules limiting the tenant administrator&amp;rsquo;s access to a specific
Istio control plane, the Istio manifest must be updated to specify the application namespace
that Pilot should watch for creation of its xDS cache. This is done by starting the Pilot
component with the additional command line arguments &lt;code>--appNamespace, ns-1&lt;/code>. Where &lt;em>ns-1&lt;/em>
is the namespace that the tenant’s application will be deployed in. An example snippet from
the &lt;code>istio-system1.yaml&lt;/code> file is shown below.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: istio-pilot
namespace: istio-system1
annotations:
sidecar.istio.io/inject: &amp;#34;false&amp;#34;
spec:
replicas: 1
template:
metadata:
labels:
istio: pilot
spec:
serviceAccountName: istio-pilot-service-account
containers:
- name: discovery
image: docker.io/&amp;lt;user ID&amp;gt;/pilot:&amp;lt;tag&amp;gt;
imagePullPolicy: IfNotPresent
args: [&amp;#34;discovery&amp;#34;, &amp;#34;-v&amp;#34;, &amp;#34;2&amp;#34;, &amp;#34;--admission-service&amp;#34;, &amp;#34;istio-pilot&amp;#34;, &amp;#34;--appNamespace&amp;#34;, &amp;#34;ns-1&amp;#34;]
ports:
- containerPort: 8080
- containerPort: 443
&lt;/code>&lt;/pre>
&lt;h3 id="deploying-the-tenant-application-in-a-namespace">Deploying the tenant application in a namespace&lt;/h3>
&lt;p>Now that the cluster administrator has created the tenant&amp;rsquo;s namespace (ex. &lt;code>istio-system1&lt;/code>) and
Pilot&amp;rsquo;s service discovery has been configured to watch for a specific application
namespace (ex. &lt;em>ns-1&lt;/em>), create the application manifests to deploy in that tenant&amp;rsquo;s specific
namespace. For example:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Namespace
metadata:
name: ns-1
&lt;/code>&lt;/pre>
&lt;p>And add the namespace reference to each resource type included in the application&amp;rsquo;s manifest
file. For example:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Service
metadata:
name: details
labels:
app: details
namespace: ns-1
&lt;/code>&lt;/pre>
&lt;p>Although not shown, the application namespaces will also have RBAC settings limiting access
to certain resources. These RBAC settings could be set by the cluster administrator and/or
the tenant administrator.&lt;/p>
&lt;h3 id="using-kubectl-in-a-multi-tenant-environment">Using &lt;code>kubectl&lt;/code> in a multi-tenant environment&lt;/h3>
&lt;p>When defining &lt;a href="https://archive.istio.io/v0.7/docs/reference/config/istio.routing.v1alpha1/#RouteRule">route rules&lt;/a>
or &lt;a href="https://archive.istio.io/v0.7/docs/reference/config/istio.routing.v1alpha1/#DestinationPolicy">destination policies&lt;/a>,
it is necessary to ensure that the &lt;code>kubectl&lt;/code> command is scoped to
the namespace the Istio control plane is running in to ensure the resource is created
in the proper namespace. Additionally, the rule itself must be scoped to the tenant&amp;rsquo;s namespace
so that it will be applied properly to that tenant&amp;rsquo;s mesh. The &lt;em>-i&lt;/em> option is used to create
(or get or describe) the rule in the namespace that the Istio control plane is deployed in.
The &lt;em>-n&lt;/em> option will scope the rule to the tenant&amp;rsquo;s mesh and should be set to the namespace that
the tenant&amp;rsquo;s app is deployed in. Note that the &lt;em>-n&lt;/em> option can be skipped on the command line if
the .yaml file for the resource scopes it properly instead.&lt;/p>
&lt;p>For example, the following command would be required to add a route rule to the &lt;code>istio-system1&lt;/code>
namespace:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl –i istio-system1 apply -n ns-1 -f route_rule_v2.yaml
&lt;/code>&lt;/pre>
&lt;p>And can be displayed using the command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl -i istio-system1 -n ns-1 get routerule
NAME KIND NAMESPACE
details-Default RouteRule.v1alpha2.config.istio.io ns-1
productpage-default RouteRule.v1alpha2.config.istio.io ns-1
ratings-default RouteRule.v1alpha2.config.istio.io ns-1
reviews-default RouteRule.v1alpha2.config.istio.io ns-1
&lt;/code>&lt;/pre>
&lt;p>See the &lt;a href="/v1.12/blog/2018/soft-multitenancy/#multiple-istio-control-planes">Multiple Istio control planes&lt;/a> section of this document for more details on &lt;code>namespace&lt;/code> requirements in a
multi-tenant environment.&lt;/p>
&lt;h3 id="test-results">Test results&lt;/h3>
&lt;p>Following the instructions above, a cluster administrator can create an environment limiting,
via RBAC and namespaces, what a tenant administrator can deploy.&lt;/p>
&lt;p>After deployment, accessing the Istio control plane pods assigned to a specific tenant
administrator is permitted:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods -n istio-system
NAME READY STATUS RESTARTS AGE
grafana-78d649479f-8pqk9 1/1 Running 0 1d
istio-ca-ffbb75c6f-98w6x 1/1 Running 0 1d
istio-ingress-68d65fc5c6-dnvfl 1/1 Running 0 1d
istio-mixer-5b9f8dffb5-8875r 3/3 Running 0 1d
istio-pilot-678fc976c8-b8tv6 2/2 Running 0 1d
istio-sidecar-injector-7587bd559d-5tgk6 1/1 Running 0 1d
prometheus-cf8456855-hdcq7 1/1 Running 0 1d
&lt;/code>&lt;/pre>
&lt;p>However, accessing all the cluster&amp;rsquo;s pods is not permitted:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods --all-namespaces
Error from server (Forbidden): pods is forbidden: User &amp;#34;dev-admin&amp;#34; cannot list pods at the cluster scope
&lt;/code>&lt;/pre>
&lt;p>And neither is accessing another tenant&amp;rsquo;s namespace:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods -n istio-system1
Error from server (Forbidden): pods is forbidden: User &amp;#34;dev-admin&amp;#34; cannot list pods in the namespace &amp;#34;istio-system1&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>The tenant administrator can deploy applications in the application namespace configured for
that tenant. As an example, updating the &lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo&lt;/a>
manifests and then deploying under the tenant&amp;rsquo;s application namespace of &lt;em>ns-0&lt;/em>, listing the
pods in use by this tenant&amp;rsquo;s namespace is permitted:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods -n ns-0
NAME READY STATUS RESTARTS AGE
details-v1-64b86cd49-b7rkr 2/2 Running 0 1d
productpage-v1-84f77f8747-rf2mt 2/2 Running 0 1d
ratings-v1-5f46655b57-5b4c5 2/2 Running 0 1d
reviews-v1-ff6bdb95b-pm5lb 2/2 Running 0 1d
reviews-v2-5799558d68-b989t 2/2 Running 0 1d
reviews-v3-58ff7d665b-lw5j9 2/2 Running 0 1d
&lt;/code>&lt;/pre>
&lt;p>But accessing another tenant&amp;rsquo;s application namespace is not:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods -n ns-1
Error from server (Forbidden): pods is forbidden: User &amp;#34;dev-admin&amp;#34; cannot list pods in the namespace &amp;#34;ns-1&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>If the &lt;a href="/v1.12/docs/tasks/observability/">add-on tools&lt;/a>, example
&lt;a href="/v1.12/docs/tasks/observability/metrics/querying-metrics/">Prometheus&lt;/a>, are deployed
(also limited by an Istio &lt;code>namespace&lt;/code>) the statistical results returned would represent only
that traffic seen from that tenant&amp;rsquo;s application namespace.&lt;/p>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>The evaluation performed indicates Istio has sufficient capabilities and security to meet a
small number of multi-tenant use cases. It also shows that Istio and Kubernetes &lt;strong>cannot&lt;/strong>
provide sufficient capabilities and security for other use cases, especially those use
cases that require complete security and isolation between untrusted tenants. The improvements
required to reach a more secure model of security and isolation require work in container
technology, ex. Kubernetes, rather than improvements in Istio capabilities.&lt;/p>
&lt;h2 id="issues">Issues&lt;/h2>
&lt;ul>
&lt;li>The CA (Certificate Authority) and Mixer pod logs from one tenant&amp;rsquo;s Istio control
plane (e.g. &lt;code>istio-system&lt;/code> namespace) contained &amp;lsquo;info&amp;rsquo; messages from a second tenant&amp;rsquo;s
Istio control plane (e.g. &lt;code>istio-system1&lt;/code> namespace).&lt;/li>
&lt;/ul>
&lt;h2 id="challenges-with-other-multi-tenancy-models">Challenges with other multi-tenancy models&lt;/h2>
&lt;p>Other multi-tenancy deployment models were considered:&lt;/p>
&lt;ol>
&lt;li>&lt;p>A single mesh with multiple applications, one for each tenant on the mesh. The cluster
administrator gets control and visibility mesh wide and across all applications, while the
tenant administrator only gets control of a specific application.&lt;/p>&lt;/li>
&lt;li>&lt;p>A single Istio control plane with multiple meshes, one mesh per tenant. The cluster
administrator gets control and visibility across the entire Istio control plane and all
meshes, while the tenant administrator only gets control of a specific mesh.&lt;/p>&lt;/li>
&lt;li>&lt;p>A single cloud environment (cluster controlled), but multiple Kubernetes control planes
(tenant controlled).&lt;/p>&lt;/li>
&lt;/ol>
&lt;p>These options either can&amp;rsquo;t be properly supported without code changes or don&amp;rsquo;t fully
address the use cases.&lt;/p>
&lt;p>Current Istio capabilities are poorly suited to support the first model as it lacks
sufficient RBAC capabilities to support cluster versus tenant operations. Additionally,
having multiple tenants under one mesh is too insecure with the current mesh model and the
way Istio drives configuration to the Envoy proxies.&lt;/p>
&lt;p>Regarding the second option, the current Istio paradigm assumes a single mesh per Istio control
plane. The needed changes to support this model are substantial. They would require
finer grained scoping of resources and security domains based on namespaces, as well as,
additional Istio RBAC changes. This model will likely be addressed by future work, but not
currently possible.&lt;/p>
&lt;p>The third model doesn’t satisfy most use cases, as most cluster administrators prefer
a common Kubernetes control plane which they provide as a
&lt;a href="https://en.wikipedia.org/wiki/Platform_as_a_service">PaaS&lt;/a> to their tenants.&lt;/p>
&lt;h2 id="future-work">Future work&lt;/h2>
&lt;p>Allowing a single Istio control plane to control multiple meshes would be an obvious next
feature. An additional improvement is to provide a single mesh that can host different
tenants with some level of isolation and security between the tenants. This could be done
by partitioning within a single control plane using the same logical notion of namespace as
Kubernetes. A &lt;a href="https://docs.google.com/document/d/14Hb07gSrfVt5KX9qNi7FzzGwB_6WBpAnDpPG6QEEd9Q">document&lt;/a>
has been started within the Istio community to define additional use cases and the
Istio functionality required to support those use cases.&lt;/p>
&lt;h2 id="references">References&lt;/h2>
&lt;ul>
&lt;li>Video on Kubernetes multi-tenancy support, &lt;a href="https://www.youtube.com/watch?v=ahwCkJGItkU">Multi-Tenancy Support &amp;amp; Security Modeling with RBAC and Namespaces&lt;/a>, and the &lt;a href="https://schd.ws/hosted_files/kccncna17/21/Multi-tenancy%20Support%20%26%20Security%20Modeling%20with%20RBAC%20and%20Namespaces.pdf">supporting slide deck&lt;/a>.&lt;/li>
&lt;li>KubeCon talk on security that discusses Kubernetes support for &amp;ldquo;Cooperative soft multi-tenancy&amp;rdquo;, &lt;a href="https://www.youtube.com/watch?v=YRR-kZub0cA">Building for Trust: How to Secure Your Kubernetes&lt;/a>.&lt;/li>
&lt;li>Kubernetes documentation on &lt;a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">RBAC&lt;/a> and &lt;a href="https://kubernetes.io/docs/tasks/administer-cluster/namespaces-walkthrough/">namespaces&lt;/a>.&lt;/li>
&lt;li>KubeCon slide deck on &lt;a href="https://schd.ws/hosted_files/kccncna17/a9/kubecon-multitenancy.pdf">Multi-tenancy Deep Dive&lt;/a>.&lt;/li>
&lt;li>Google document on &lt;a href="https://docs.google.com/document/d/15w1_fesSUZHv-vwjiYa9vN_uyc--PySRoLKTuDhimjc">Multi-tenancy models for Kubernetes&lt;/a>. (Requires permission)&lt;/li>
&lt;li>Cloud Foundry WIP document, &lt;a href="https://docs.google.com/document/d/14Hb07gSrfVt5KX9qNi7FzzGwB_6WBpAnDpPG6QEEd9Q">Multi-cloud and Multi-tenancy&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://docs.google.com/document/d/12F183NIRAwj2hprx-a-51ByLeNqbJxK16X06vwH5OWE">Istio Auto Multi-Tenancy 101&lt;/a>&lt;/li>
&lt;/ul></description><pubDate>Thu, 19 Apr 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/soft-multitenancy/</link><author>John Joyce and Rich Curran</author><guid isPermaLink="true">/v1.12/blog/2018/soft-multitenancy/</guid><category>tenancy</category></item><item><title>Traffic Mirroring with Istio for Testing in Production</title><description>&lt;p>Trying to enumerate all the possible combinations of test cases for testing services in non-production/test environments can be daunting. In some cases, you&amp;rsquo;ll find that all of the effort that goes into cataloging these use cases doesn&amp;rsquo;t match up to real production use cases. Ideally, we could use live production use cases and traffic to help illuminate all of the feature areas of the service under test that we might miss in more contrived testing environments.&lt;/p>
&lt;p>Istio can help here. With the release of &lt;a href="/v1.12/news/releases/0.x/announcing-0.5">Istio 0.5&lt;/a>, Istio can mirror traffic to help test your services. You can write route rules similar to the following to enable traffic mirroring:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: config.istio.io/v1alpha2
kind: RouteRule
metadata:
name: mirror-traffic-to-httbin-v2
spec:
destination:
name: httpbin
precedence: 11
route:
- labels:
version: v1
weight: 100
- labels:
version: v2
weight: 0
mirror:
name: httpbin
labels:
version: v2
&lt;/code>&lt;/pre>
&lt;p>A few things to note here:&lt;/p>
&lt;ul>
&lt;li>When traffic gets mirrored to a different service, that happens outside the critical path of the request&lt;/li>
&lt;li>Responses to any mirrored traffic is ignored; traffic is mirrored as &amp;ldquo;fire-and-forget&amp;rdquo;&lt;/li>
&lt;li>You&amp;rsquo;ll need to have the 0-weighted route to hint to Istio to create the proper Envoy cluster under the covers; &lt;a href="https://github.com/istio/istio/issues/3270">this should be ironed out in future releases&lt;/a>.&lt;/li>
&lt;/ul>
&lt;p>Learn more about mirroring by visiting the &lt;a href="/v1.12/docs/tasks/traffic-management/mirroring/">Mirroring Task&lt;/a> and see a more
&lt;a href="https://dzone.com/articles/traffic-shadowing-with-istio-reducing-the-risk-of">comprehensive treatment of this scenario on my blog&lt;/a>.&lt;/p></description><pubDate>Thu, 08 Feb 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/traffic-mirroring/</link><author>Christian Posta</author><guid isPermaLink="true">/v1.12/blog/2018/traffic-mirroring/</guid><category>traffic-management</category><category>mirroring</category></item><item><title>Consuming External TCP Services</title><description>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">This blog post was updated on July 23, 2018 to use the new
&lt;a href="/v1.12/blog/2018/v1alpha3-routing/">v1alpha3 traffic management API&lt;/a>. If you need to use the old version, follow these &lt;a href="https://archive.istio.io/v0.7/blog/2018/egress-tcp.html">docs&lt;/a>.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>In my previous blog post, &lt;a href="/v1.12/blog/2018/egress-https/">Consuming External Web Services&lt;/a>, I described how external services
can be consumed by in-mesh Istio applications via HTTPS. In this post, I demonstrate consuming external services
over TCP. You will use the &lt;a href="/v1.12/docs/examples/bookinfo/">Istio Bookinfo sample application&lt;/a>, the version in which the book
ratings data is persisted in a MySQL database. You deploy this database outside the cluster and configure the
&lt;em>ratings&lt;/em> microservice to use it. You define a
&lt;a href="/v1.12/docs/reference/config/networking/service-entry/">Service Entry&lt;/a> to allow the in-mesh applications to
access the external database.&lt;/p>
&lt;h2 id="bookinfo-sample-application-with-external-ratings-database">Bookinfo sample application with external ratings database&lt;/h2>
&lt;p>First, you set up a MySQL database instance to hold book ratings data outside of your Kubernetes cluster. Then you
modify the &lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo sample application&lt;/a> to use your database.&lt;/p>
&lt;h3 id="setting-up-the-database-for-ratings-data">Setting up the database for ratings data&lt;/h3>
&lt;p>For this task you set up an instance of &lt;a href="https://www.mysql.com">MySQL&lt;/a>. You can use any MySQL instance; I used
&lt;a href="https://www.ibm.com/cloud/compose/mysql">Compose for MySQL&lt;/a>. I used &lt;code>mysqlsh&lt;/code>
(&lt;a href="https://dev.mysql.com/doc/mysql-shell/en/">MySQL Shell&lt;/a>) as a MySQL client to feed the ratings data.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Set the &lt;code>MYSQL_DB_HOST&lt;/code> and &lt;code>MYSQL_DB_PORT&lt;/code> environment variables:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export MYSQL_DB_HOST=&amp;lt;your MySQL database host&amp;gt;
$ export MYSQL_DB_PORT=&amp;lt;your MySQL database port&amp;gt;
&lt;/code>&lt;/pre>
&lt;p>In case of a local MySQL database with the default port, the values are &lt;code>localhost&lt;/code> and &lt;code>3306&lt;/code>, respectively.&lt;/p>&lt;/li>
&lt;li>&lt;p>To initialize the database, run the following command entering the password when prompted. The command is
performed with the credentials of the &lt;code>admin&lt;/code> user, created by default by
&lt;a href="https://www.ibm.com/cloud/compose/mysql">Compose for MySQL&lt;/a>.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl -s https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/src/mysql/mysqldb-init.sql | mysqlsh --sql --ssl-mode=REQUIRED -u admin -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT
&lt;/code>&lt;/pre>
&lt;p>&lt;em>&lt;strong>OR&lt;/strong>&lt;/em>&lt;/p>
&lt;p>When using the &lt;code>mysql&lt;/code> client and a local MySQL database, run:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ curl -s https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/src/mysql/mysqldb-init.sql | mysql -u root -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Create a user with the name &lt;code>bookinfo&lt;/code> and grant it &lt;em>SELECT&lt;/em> privilege on the &lt;code>test.ratings&lt;/code> table:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mysqlsh --sql --ssl-mode=REQUIRED -u admin -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &amp;#34;CREATE USER &amp;#39;bookinfo&amp;#39; IDENTIFIED BY &amp;#39;&amp;lt;password you choose&amp;gt;&amp;#39;; GRANT SELECT ON test.ratings to &amp;#39;bookinfo&amp;#39;;&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>&lt;em>&lt;strong>OR&lt;/strong>&lt;/em>&lt;/p>
&lt;p>For &lt;code>mysql&lt;/code> and the local database, the command is:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mysql -u root -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &amp;#34;CREATE USER &amp;#39;bookinfo&amp;#39; IDENTIFIED BY &amp;#39;&amp;lt;password you choose&amp;gt;&amp;#39;; GRANT SELECT ON test.ratings to &amp;#39;bookinfo&amp;#39;;&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>Here you apply the &lt;a href="https://en.wikipedia.org/wiki/Principle_of_least_privilege">principle of least privilege&lt;/a>. This
means that you do not use your &lt;code>admin&lt;/code> user in the Bookinfo application. Instead, you create a special user for the
Bookinfo application , &lt;code>bookinfo&lt;/code>, with minimal privileges. In this case, the &lt;em>bookinfo&lt;/em> user only has the &lt;code>SELECT&lt;/code>
privilege on a single table.&lt;/p>
&lt;p>After running the command to create the user, you may want to clean your bash history by checking the number of the last
command and running &lt;code>history -d &amp;lt;the number of the command that created the user&amp;gt;&lt;/code>. You don&amp;rsquo;t want the password of the
new user to be stored in the bash history. If you&amp;rsquo;re using &lt;code>mysql&lt;/code>, remove the last command from
&lt;code>~/.mysql_history&lt;/code> file as well. Read more about password protection of the newly created user in &lt;a href="https://dev.mysql.com/doc/refman/5.5/en/create-user.html">MySQL documentation&lt;/a>.&lt;/p>&lt;/li>
&lt;li>&lt;p>Inspect the created ratings to see that everything worked as expected:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mysqlsh --sql --ssl-mode=REQUIRED -u bookinfo -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &amp;#34;select * from test.ratings;&amp;#34;
Enter password:
+----------+--------+
| ReviewID | Rating |
+----------+--------+
| 1 | 5 |
| 2 | 4 |
+----------+--------+
&lt;/code>&lt;/pre>
&lt;p>&lt;em>&lt;strong>OR&lt;/strong>&lt;/em>&lt;/p>
&lt;p>For &lt;code>mysql&lt;/code> and the local database:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mysql -u bookinfo -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &amp;#34;select * from test.ratings;&amp;#34;
Enter password:
+----------+--------+
| ReviewID | Rating |
+----------+--------+
| 1 | 5 |
| 2 | 4 |
+----------+--------+
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Set the ratings temporarily to &lt;code>1&lt;/code> to provide a visual clue when our database is used by the Bookinfo &lt;em>ratings&lt;/em>
service:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mysqlsh --sql --ssl-mode=REQUIRED -u admin -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &amp;#34;update test.ratings set rating=1; select * from test.ratings;&amp;#34;
Enter password:
Rows matched: 2 Changed: 2 Warnings: 0
+----------+--------+
| ReviewID | Rating |
+----------+--------+
| 1 | 1 |
| 2 | 1 |
+----------+--------+
&lt;/code>&lt;/pre>
&lt;p>&lt;em>&lt;strong>OR&lt;/strong>&lt;/em>&lt;/p>
&lt;p>For &lt;code>mysql&lt;/code> and the local database:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mysql -u root -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &amp;#34;update test.ratings set rating=1; select * from test.ratings;&amp;#34;
Enter password:
+----------+--------+
| ReviewID | Rating |
+----------+--------+
| 1 | 1 |
| 2 | 1 |
+----------+--------+
&lt;/code>&lt;/pre>
&lt;p>You used the &lt;code>admin&lt;/code> user (and &lt;code>root&lt;/code> for the local database) in the last command since the &lt;code>bookinfo&lt;/code> user does not
have the &lt;code>UPDATE&lt;/code> privilege on the &lt;code>test.ratings&lt;/code> table.&lt;/p>&lt;/li>
&lt;/ol>
&lt;p>Now you are ready to deploy a version of the Bookinfo application that will use your database.&lt;/p>
&lt;h3 id="initial-setting-of-bookinfo-application">Initial setting of Bookinfo application&lt;/h3>
&lt;p>To demonstrate the scenario of using an external database, you start with a Kubernetes cluster with &lt;a href="/v1.12/docs/setup/getting-started/">Istio installed&lt;/a>. Then you deploy the
&lt;a href="/v1.12/docs/examples/bookinfo/">Istio Bookinfo sample application&lt;/a>, &lt;a href="/v1.12/docs/examples/bookinfo/#apply-default-destination-rules">apply the default destination rules&lt;/a>, and &lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-control/#change-to-the-blocking-by-default-policy">change Istio to the blocking-egress-by-default policy&lt;/a>.&lt;/p>
&lt;p>This application uses the &lt;code>ratings&lt;/code> microservice to fetch
book ratings, a number between 1 and 5. The ratings are displayed as stars for each review. There are several versions
of the &lt;code>ratings&lt;/code> microservice. Some use &lt;a href="https://www.mongodb.com">MongoDB&lt;/a>, others use &lt;a href="https://www.mysql.com">MySQL&lt;/a>
as their database.&lt;/p>
&lt;p>The example commands in this blog post work with Istio 0.8+, with or without
&lt;a href="/v1.12/docs/concepts/security/#mutual-tls-authentication">mutual TLS&lt;/a> enabled.&lt;/p>
&lt;p>As a reminder, here is the end-to-end architecture of the application from the
&lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo sample application&lt;/a>.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.086918235567985%">
&lt;a data-skipendnotes="true" href="/v1.12/docs/examples/bookinfo/withistio.svg" title="The original Bookinfo application">
&lt;img class="element-to-stretch" src="/v1.12/docs/examples/bookinfo/withistio.svg" alt="The original Bookinfo application" />
&lt;/a>
&lt;/div>
&lt;figcaption>The original Bookinfo application&lt;/figcaption>
&lt;/figure>
&lt;h3 id="use-the-database-for-ratings-data-in-bookinfo-application">Use the database for ratings data in Bookinfo application&lt;/h3>
&lt;ol>
&lt;li>&lt;p>Modify the deployment spec of a version of the &lt;em>ratings&lt;/em> microservice that uses a MySQL database, to use your
database instance. The spec is in &lt;a href="https://github.com/istio/istio/blob/release-1.12/samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml">&lt;code>samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml&lt;/code>&lt;/a>
of an Istio release archive. Edit the following lines:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >- name: MYSQL_DB_HOST
value: mysqldb
- name: MYSQL_DB_PORT
value: &amp;#34;3306&amp;#34;
- name: MYSQL_DB_USER
value: root
- name: MYSQL_DB_PASSWORD
value: password
&lt;/code>&lt;/pre>
&lt;p>Replace the values in the snippet above, specifying the database host, port, user, and password. Note that the
correct way to work with passwords in container&amp;rsquo;s environment variables in Kubernetes is &lt;a href="https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables">to use secrets&lt;/a>. For this
example task only, you may want to write the password directly in the deployment spec. &lt;strong>Do not do it&lt;/strong> in a real
environment! I also assume everyone realizes that &lt;code>&amp;quot;password&amp;quot;&lt;/code> should not be used as a password&amp;hellip;&lt;/p>&lt;/li>
&lt;li>&lt;p>Apply the modified spec to deploy the version of the &lt;em>ratings&lt;/em> microservice, &lt;em>v2-mysql&lt;/em>, that will use your
database.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml@
deployment &amp;#34;ratings-v2-mysql&amp;#34; created
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Route all the traffic destined to the &lt;em>reviews&lt;/em> service to its &lt;em>v3&lt;/em> version. You do this to ensure that the
&lt;em>reviews&lt;/em> service always calls the &lt;em>ratings&lt;/em> service. In addition, route all the traffic destined to the &lt;em>ratings&lt;/em>
service to &lt;em>ratings v2-mysql&lt;/em> that uses your database.&lt;/p>
&lt;p>Specify the routing for both services above by adding two
&lt;a href="/v1.12/docs/reference/config/networking/virtual-service/">virtual services&lt;/a>. These virtual services are
specified in &lt;code>samples/bookinfo/networking/virtual-service-ratings-mysql.yaml&lt;/code> of an Istio release archive.
&lt;strong>&lt;em>Important:&lt;/em>&lt;/strong> make sure you
&lt;a href="/v1.12/docs/examples/bookinfo/#apply-default-destination-rules">applied the default destination rules&lt;/a> before running the
following command.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-ratings-mysql.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/networking/virtual-service-ratings-mysql.yaml@
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;/ol>
&lt;p>The updated architecture appears below. Note that the blue arrows inside the mesh mark the traffic configured according
to the virtual services we added. According to the virtual services, the traffic is sent to &lt;em>reviews v3&lt;/em> and
&lt;em>ratings v2-mysql&lt;/em>.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.314858206480224%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-tcp/bookinfo-ratings-v2-mysql-external.svg" title="The Bookinfo application with ratings v2-mysql and an external MySQL database">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-tcp/bookinfo-ratings-v2-mysql-external.svg" alt="The Bookinfo application with ratings v2-mysql and an external MySQL database" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Bookinfo application with ratings v2-mysql and an external MySQL database&lt;/figcaption>
&lt;/figure>
&lt;p>Note that the MySQL database is outside the Istio service mesh, or more precisely outside the Kubernetes cluster. The
boundary of the service mesh is marked by a dashed line.&lt;/p>
&lt;h3 id="access-the-webpage">Access the webpage&lt;/h3>
&lt;p>Access the webpage of the application, after
&lt;a href="/v1.12/docs/examples/bookinfo/#determine-the-ingress-ip-and-port">determining the ingress IP and port&lt;/a>.&lt;/p>
&lt;p>You have a problem&amp;hellip; Instead of the rating stars, the message &lt;em>&amp;ldquo;Ratings service is currently unavailable&amp;rdquo;&lt;/em> is currently
displayed below each review:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.18705035971223%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-tcp/errorFetchingBookRating.png" title="The Ratings service error messages">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-tcp/errorFetchingBookRating.png" alt="The Ratings service error messages" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Ratings service error messages&lt;/figcaption>
&lt;/figure>
&lt;p>As in &lt;a href="/v1.12/blog/2018/egress-https/">Consuming External Web Services&lt;/a>, you experience &lt;strong>graceful service degradation&lt;/strong>,
which is good. The application did not crash due to the error in the &lt;em>ratings&lt;/em> microservice. The webpage of the
application correctly displayed the book information, the details, and the reviews, just without the rating stars.&lt;/p>
&lt;p>You have the same problem as in &lt;a href="/v1.12/blog/2018/egress-https/">Consuming External Web Services&lt;/a>, namely all the traffic
outside the Kubernetes cluster, both TCP and HTTP, is blocked by default by the sidecar proxies. To enable such traffic
for TCP, a mesh-external service entry for TCP must be defined.&lt;/p>
&lt;h3 id="mesh-external-service-entry-for-an-external-mysql-instance">Mesh-external service entry for an external MySQL instance&lt;/h3>
&lt;p>TCP mesh-external service entries come to our rescue.&lt;/p>
&lt;ol>
&lt;li>&lt;p>Get the IP address of your MySQL database instance. As an option, you can use the
&lt;a href="https://linux.die.net/man/1/host">host&lt;/a> command:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ export MYSQL_DB_IP=$(host $MYSQL_DB_HOST | grep &amp;#34; has address &amp;#34; | cut -d&amp;#34; &amp;#34; -f4)
&lt;/code>&lt;/pre>
&lt;p>For a local database, set &lt;code>MYSQL_DB_IP&lt;/code> to contain the IP of your machine, accessible from your cluster.&lt;/p>&lt;/li>
&lt;li>&lt;p>Define a TCP mesh-external service entry:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mysql-external
spec:
hosts:
- $MYSQL_DB_HOST
addresses:
- $MYSQL_DB_IP/32
ports:
- name: tcp
number: $MYSQL_DB_PORT
protocol: tcp
location: MESH_EXTERNAL
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Review the service entry you just created and check that it contains the correct values:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get serviceentry mysql-external -o yaml
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
...
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;p>Note that for a TCP service entry, you specify &lt;code>tcp&lt;/code> as the protocol of a port of the entry. Also note that you have to
specify the IP of the external service in the list of addresses, as a &lt;a href="https://tools.ietf.org/html/rfc2317">CIDR&lt;/a> block
with suffix &lt;code>32&lt;/code>.&lt;/p>
&lt;p>I will talk more about TCP service entries
&lt;a href="#service-entries-for-tcp-traffic">below&lt;/a>. For now, verify that the service entry we added fixed the problem. Access the
webpage and see if the stars are back.&lt;/p>
&lt;p>It worked! Accessing the web page of the application displays the ratings without error:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.69064748201439%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-tcp/externalMySQLRatings.png" title="Book Ratings Displayed Correctly">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-tcp/externalMySQLRatings.png" alt="Book Ratings Displayed Correctly" />
&lt;/a>
&lt;/div>
&lt;figcaption>Book Ratings Displayed Correctly&lt;/figcaption>
&lt;/figure>
&lt;p>Note that you see a one-star rating for both displayed reviews, as expected. You changed the ratings to be one star to
provide us with a visual clue that our external database is indeed being used.&lt;/p>
&lt;p>As with service entries for HTTP/HTTPS, you can delete and create service entries for TCP using &lt;code>kubectl&lt;/code>, dynamically.&lt;/p>
&lt;h2 id="motivation-for-egress-tcp-traffic-control">Motivation for egress TCP traffic control&lt;/h2>
&lt;p>Some in-mesh Istio applications must access external services, for example legacy systems. In many cases, the access is
not performed over HTTP or HTTPS protocols. Other TCP protocols are used, such as database-specific protocols like
&lt;a href="https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/">MongoDB Wire Protocol&lt;/a> and &lt;a href="https://dev.mysql.com/doc/internals/en/client-server-protocol.html">MySQL Client/Server Protocol&lt;/a> to communicate with external databases.&lt;/p>
&lt;p>Next let me provide more details about the service entries for TCP traffic.&lt;/p>
&lt;h2 id="service-entries-for-tcp-traffic">Service entries for TCP traffic&lt;/h2>
&lt;p>The service entries for enabling TCP traffic to a specific port must specify &lt;code>TCP&lt;/code> as the protocol of the port.
Additionally, for the &lt;a href="https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/">MongoDB Wire Protocol&lt;/a>, the
protocol can be specified as &lt;code>MONGO&lt;/code>, instead of &lt;code>TCP&lt;/code>.&lt;/p>
&lt;p>For the &lt;code>addresses&lt;/code> field of the entry, a block of IPs in &lt;a href="https://tools.ietf.org/html/rfc2317">CIDR&lt;/a>
notation must be used. Note that the &lt;code>hosts&lt;/code> field is ignored for TCP service entries.&lt;/p>
&lt;p>To enable TCP traffic to an external service by its hostname, all the IPs of the hostname must be specified. Each IP
must be specified by a CIDR block.&lt;/p>
&lt;p>Note that all the IPs of an external service are not always known. To enable egress TCP traffic, only the IPs that are
used by the applications must be specified.&lt;/p>
&lt;p>Also note that the IPs of an external service are not always static, for example in the case of
&lt;a href="https://en.wikipedia.org/wiki/Content_delivery_network">CDNs&lt;/a>. Sometimes the IPs are static most of the time, but can
be changed from time to time, for example due to infrastructure changes. In these cases, if the range of the possible
IPs is known, you should specify the range by CIDR blocks. If the range of the possible IPs is not known, service
entries for TCP cannot be used and
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-control/#direct-access-to-external-services">the external services must be called directly&lt;/a>,
bypassing the sidecar proxies.&lt;/p>
&lt;h2 id="relation-to-virtual-machines-support">Relation to virtual machines support&lt;/h2>
&lt;p>Note that the scenario described in this post is different from the
&lt;a href="/v1.12/docs/examples/virtual-machines/">Bookinfo with Virtual Machines&lt;/a> example. In that scenario, a MySQL instance runs on an
external
(outside the cluster) machine (a bare metal or a VM), integrated with the Istio service mesh. The MySQL service becomes
a first-class citizen of the mesh with all the beneficial features of Istio applicable. Among other things, the service
becomes addressable by a local cluster domain name, for example by &lt;code>mysqldb.vm.svc.cluster.local&lt;/code>, and the communication
to it can be secured by
&lt;a href="/v1.12/docs/concepts/security/#mutual-tls-authentication">mutual TLS authentication&lt;/a>. There is no need to create a service
entry to access this service; however, the service must be registered with Istio. To enable such integration, Istio
components (&lt;em>Envoy proxy&lt;/em>, &lt;em>node-agent&lt;/em>, &lt;code>_istio-agent_&lt;/code>) must be installed on the machine and the Istio control plane
(&lt;em>Pilot&lt;/em>, &lt;em>Mixer&lt;/em>, &lt;em>Citadel&lt;/em>) must be accessible from it. See the
&lt;a href="/v1.12/docs/examples/virtual-machines/">Istio VM-related&lt;/a> tasks for more details.&lt;/p>
&lt;p>In our case, the MySQL instance can run on any machine or can be provisioned as a service by a cloud provider. There is
no requirement to integrate the machine with Istio. The Istio control plane does not have to be accessible from the
machine. In the case of MySQL as a service, the machine which MySQL runs on may be not accessible and installing on it
the required components may be impossible. In our case, the MySQL instance is addressable by its global domain name,
which could be beneficial if the consuming applications expect to use that domain name. This is especially relevant when
that expected domain name cannot be changed in the deployment configuration of the consuming applications.&lt;/p>
&lt;h2 id="cleanup">Cleanup&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Drop the &lt;code>test&lt;/code> database and the &lt;code>bookinfo&lt;/code> user:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mysqlsh --sql --ssl-mode=REQUIRED -u admin -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &amp;#34;drop database test; drop user bookinfo;&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>&lt;em>&lt;strong>OR&lt;/strong>&lt;/em>&lt;/p>
&lt;p>For &lt;code>mysql&lt;/code> and the local database:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ mysql -u root -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &amp;#34;drop database test; drop user bookinfo;&amp;#34;
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Remove the virtual services:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-ratings-mysql.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete -f @samples/bookinfo/networking/virtual-service-ratings-mysql.yaml@
Deleted config: virtual-service/default/reviews
Deleted config: virtual-service/default/ratings
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Undeploy &lt;em>ratings v2-mysql&lt;/em>:&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete -f @samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml@
deployment &amp;#34;ratings-v2-mysql&amp;#34; deleted
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Delete the service entry:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry mysql-external -n default
Deleted config: serviceentry mysql-external
&lt;/code>&lt;/pre>&lt;/li>
&lt;/ol>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>In this blog post, I demonstrated how the microservices in an Istio service mesh can consume external services via TCP.
By default, Istio blocks all the traffic, TCP and HTTP, to the hosts outside the cluster. To enable such traffic for
TCP, TCP mesh-external service entries must be created for the service mesh.&lt;/p></description><pubDate>Tue, 06 Feb 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/egress-tcp/</link><author>Vadim Eisenberg</author><guid isPermaLink="true">/v1.12/blog/2018/egress-tcp/</guid><category>traffic-management</category><category>egress</category><category>tcp</category></item><item><title>Consuming External Web Services</title><description>
&lt;p>In many cases, not all the parts of a microservices-based application reside in a &lt;em>service mesh&lt;/em>. Sometimes, the
microservices-based applications use functionality provided by legacy systems that reside outside the mesh. You may want
to migrate these systems to the service mesh gradually. Until these systems are migrated, they must be accessed by the
applications inside the mesh. In other cases, the applications use web services provided by third parties.&lt;/p>
&lt;p>In this blog post, I modify the &lt;a href="/v1.12/docs/examples/bookinfo/">Istio Bookinfo Sample Application&lt;/a> to fetch book details from
an external web service (&lt;a href="https://developers.google.com/books/docs/v1/getting_started">Google Books APIs&lt;/a>). I show how
to enable egress HTTPS traffic in Istio by using &lt;em>mesh-external service entries&lt;/em>. I provide two options for egress
HTTPS traffic and describe the pros and cons of each of the options.&lt;/p>
&lt;h2 id="initial-setting">Initial setting&lt;/h2>
&lt;p>To demonstrate the scenario of consuming an external web service, I start with a Kubernetes cluster with &lt;a href="/v1.12/docs/setup/getting-started/">Istio installed&lt;/a>. Then I deploy
&lt;a href="/v1.12/docs/examples/bookinfo/">Istio Bookinfo Sample Application&lt;/a>. This application uses the &lt;em>details&lt;/em> microservice to fetch
book details, such as the number of pages and the publisher. The original &lt;em>details&lt;/em> microservice provides the book
details without consulting any external service.&lt;/p>
&lt;p>The example commands in this blog post work with Istio 1.0+, with or without
&lt;a href="/v1.12/docs/concepts/security/#mutual-tls-authentication">mutual TLS&lt;/a> enabled. The Bookinfo configuration files reside in the
&lt;code>samples/bookinfo&lt;/code> directory of the Istio release archive.&lt;/p>
&lt;p>Here is a copy of the end-to-end architecture of the application from the original
&lt;a href="/v1.12/docs/examples/bookinfo/">Bookinfo sample application&lt;/a>.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.086918235567985%">
&lt;a data-skipendnotes="true" href="/v1.12/docs/examples/bookinfo/withistio.svg" title="The Original Bookinfo Application">
&lt;img class="element-to-stretch" src="/v1.12/docs/examples/bookinfo/withistio.svg" alt="The Original Bookinfo Application" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Original Bookinfo Application&lt;/figcaption>
&lt;/figure>
&lt;p>Perform the steps in the
&lt;a href="/v1.12/docs/examples/bookinfo/#deploying-the-application">Deploying the application&lt;/a>,
&lt;a href="/v1.12/docs/examples/bookinfo/#confirm-the-app-is-accessible-from-outside-the-cluster">Confirm the app is running&lt;/a>,
&lt;a href="/v1.12/docs/examples/bookinfo/#apply-default-destination-rules">Apply default destination rules&lt;/a>
sections, and
&lt;a href="/v1.12/docs/tasks/traffic-management/egress/egress-control/#change-to-the-blocking-by-default-policy">change Istio to the blocking-egress-by-default policy&lt;/a>.&lt;/p>
&lt;h2 id="bookinfo-with-https-access-to-a-google-books-web-service">Bookinfo with HTTPS access to a Google Books web service&lt;/h2>
&lt;p>Deploy a new version of the &lt;em>details&lt;/em> microservice, &lt;em>v2&lt;/em>, that fetches the book details from &lt;a href="https://developers.google.com/books/docs/v1/getting_started">Google Books APIs&lt;/a>. Run the following command; it sets the
&lt;code>DO_NOT_ENCRYPT&lt;/code> environment variable of the service&amp;rsquo;s container to &lt;code>false&lt;/code>. This setting will instruct the deployed
service to use HTTPS (instead of HTTP) to access to the external service.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo-details-v2.yaml@ --dry-run -o yaml | kubectl set env --local -f - &amp;#39;DO_NOT_ENCRYPT=false&amp;#39; -o yaml | kubectl apply -f -
&lt;/code>&lt;/pre>&lt;/div>
&lt;p>The updated architecture of the application now looks as follows:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:65.1654485092242%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-https/bookinfo-details-v2.svg" title="The Bookinfo Application with details V2">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-https/bookinfo-details-v2.svg" alt="The Bookinfo Application with details V2" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Bookinfo Application with details V2&lt;/figcaption>
&lt;/figure>
&lt;p>Note that the Google Books web service is outside the Istio service mesh, the boundary of which is marked by a dashed
line.&lt;/p>
&lt;p>Now direct all the traffic destined to the &lt;em>details&lt;/em> microservice, to &lt;em>details version v2&lt;/em>.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-details-v2.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/networking/virtual-service-details-v2.yaml@
&lt;/code>&lt;/pre>&lt;/div>
&lt;p>Note that the virtual service relies on a destination rule that you created in the &lt;a href="/v1.12/docs/examples/bookinfo/#apply-default-destination-rules">Apply default destination rules&lt;/a> section.&lt;/p>
&lt;p>Access the web page of the application, after
&lt;a href="/v1.12/docs/examples/bookinfo/#determine-the-ingress-ip-and-port">determining the ingress IP and port&lt;/a>.&lt;/p>
&lt;p>Oops&amp;hellip; Instead of the book details you have the &lt;em>Error fetching product details&lt;/em> message displayed:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.18649965205289%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-https/errorFetchingBookDetails.png" title="The Error Fetching Product Details Message">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-https/errorFetchingBookDetails.png" alt="The Error Fetching Product Details Message" />
&lt;/a>
&lt;/div>
&lt;figcaption>The Error Fetching Product Details Message&lt;/figcaption>
&lt;/figure>
&lt;p>The good news is that your application did not crash. With a good microservice design, you do not have &lt;strong>failure
propagation&lt;/strong>. In your case, the failing &lt;em>details&lt;/em> microservice does not cause the &lt;code>productpage&lt;/code> microservice to fail.
Most of the functionality of the application is still provided, despite the failure in the &lt;em>details&lt;/em> microservice. You
have &lt;strong>graceful service degradation&lt;/strong>: as you can see, the reviews and the ratings are displayed correctly, and the
application is still useful.&lt;/p>
&lt;p>So what might have gone wrong? Ah&amp;hellip; The answer is that I forgot to tell you to enable traffic from inside the mesh to
an external service, in this case to the Google Books web service. By default, the Istio sidecar proxies
(&lt;a href="https://www.envoyproxy.io">Envoy proxies&lt;/a>) &lt;strong>block all the traffic to destinations outside the cluster&lt;/strong>. To enable
such traffic, you must define a
&lt;a href="/v1.12/docs/reference/config/networking/service-entry/">mesh-external service entry&lt;/a>.&lt;/p>
&lt;h3 id="enable-https-access-to-a-google-books-web-service">Enable HTTPS access to a Google Books web service&lt;/h3>
&lt;p>No worries, define a &lt;strong>mesh-external service entry&lt;/strong> and fix your application. You must also define a &lt;em>virtual
service&lt;/em> to perform routing by &lt;a href="https://en.wikipedia.org/wiki/Server_Name_Indication">SNI&lt;/a> to the external service.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: googleapis
spec:
hosts:
- www.googleapis.com
ports:
- number: 443
name: https
protocol: HTTPS
location: MESH_EXTERNAL
resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: googleapis
spec:
hosts:
- www.googleapis.com
tls:
- match:
- port: 443
sni_hosts:
- www.googleapis.com
route:
- destination:
host: www.googleapis.com
port:
number: 443
weight: 100
EOF
&lt;/code>&lt;/pre>
&lt;p>Now accessing the web page of the application displays the book details without error:&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:34.82831114225648%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-https/externalBookDetails.png" title="Book Details Displayed Correctly">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-https/externalBookDetails.png" alt="Book Details Displayed Correctly" />
&lt;/a>
&lt;/div>
&lt;figcaption>Book Details Displayed Correctly&lt;/figcaption>
&lt;/figure>
&lt;p>You can query your service entries:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get serviceentries
NAME AGE
googleapis 8m
&lt;/code>&lt;/pre>
&lt;p>You can delete your service entry:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry googleapis
serviceentry &amp;#34;googleapis&amp;#34; deleted
&lt;/code>&lt;/pre>
&lt;p>and see in the output that the service entry is deleted.&lt;/p>
&lt;p>Accessing the web page after deleting the service entry produces the same error that you experienced before, namely
&lt;em>Error fetching product details&lt;/em>. As you can see, the service entries are defined &lt;strong>dynamically&lt;/strong>, as are many other
Istio configuration artifacts. The Istio operators can decide dynamically which domains they allow the microservices to
access. They can enable and disable traffic to the external domains on the fly, without redeploying the microservices.&lt;/p>
&lt;h3 id="cleanup-of-https-access-to-a-google-books-web-service">Cleanup of HTTPS access to a Google Books web service&lt;/h3>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml'>Zip&lt;/a>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-details-v2.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry googleapis
$ kubectl delete virtualservice googleapis
$ kubectl delete -f @samples/bookinfo/networking/virtual-service-details-v2.yaml@
$ kubectl delete -f @samples/bookinfo/platform/kube/bookinfo-details-v2.yaml@
&lt;/code>&lt;/pre>&lt;/div>
&lt;h2 id="tls-origination-by-istio">TLS origination by Istio&lt;/h2>
&lt;p>There is a caveat to this story. Suppose you want to monitor which specific set of
&lt;a href="https://developers.google.com/apis-explorer/">Google APIs&lt;/a> your microservices use
(&lt;a href="https://developers.google.com/books/docs/v1/getting_started">Books&lt;/a>,
&lt;a href="https://developers.google.com/calendar/">Calendar&lt;/a>, &lt;a href="https://developers.google.com/tasks/">Tasks&lt;/a> etc.)
Suppose you want to enforce a policy that using only
&lt;a href="https://developers.google.com/books/docs/v1/getting_started">Books APIs&lt;/a> is allowed. Suppose you want to monitor the
book identifiers that your microservices access. For these monitoring and policy tasks you need to know the URL path.
Consider for example the URL
&lt;a href="https://www.googleapis.com/books/v1/volumes?q=isbn:0486424618">&lt;code>www.googleapis.com/books/v1/volumes?q=isbn:0486424618&lt;/code>&lt;/a>.
In that URL, &lt;a href="https://developers.google.com/books/docs/v1/getting_started">Books APIs&lt;/a> is specified by the path segment
&lt;code>/books&lt;/code>, and the &lt;a href="https://en.wikipedia.org/wiki/International_Standard_Book_Number">ISBN&lt;/a> number by the path segment
&lt;code>/volumes?q=isbn:0486424618&lt;/code>. However, in HTTPS, all the HTTP details (hostname, path, headers etc.) are encrypted and
such monitoring and policy enforcement by the sidecar proxies is not possible. Istio can only know the server name of
the encrypted requests by the &lt;a href="https://tools.ietf.org/html/rfc3546#section-3.1">SNI&lt;/a> (&lt;em>Server Name Indication&lt;/em>) field,
in this case &lt;code>www.googleapis.com&lt;/code>.&lt;/p>
&lt;p>To allow Istio to perform monitoring and policy enforcement of egress requests based on HTTP details, the microservices
must issue HTTP requests. Istio then opens an HTTPS connection to the destination (performs TLS origination). The code
of the microservices must be written differently or configured differently, according to whether the microservice runs
inside or outside an Istio service mesh. This contradicts the Istio design goal of &lt;a href="/v1.12/docs/ops/deployment/architecture/#design-goals">maximizing transparency&lt;/a>. Sometimes you need to compromise&amp;hellip;&lt;/p>
&lt;p>The diagram below shows two options for sending HTTPS traffic to external services. On the top, a microservice sends
regular HTTPS requests, encrypted end-to-end. On the bottom, the same microservice sends unencrypted HTTP requests
inside a pod, which are intercepted by the sidecar Envoy proxy. The sidecar proxy performs TLS origination, so the
traffic between the pod and the external service is encrypted.&lt;/p>
&lt;figure style="width:60%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:95.1355088590701%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2018/egress-https/https_from_the_app.svg" title="HTTPS traffic to external services, with TLS originated by the microservice vs. by the sidecar proxy">
&lt;img class="element-to-stretch" src="/v1.12/blog/2018/egress-https/https_from_the_app.svg" alt="HTTPS traffic to external services, with TLS originated by the microservice vs. by the sidecar proxy" />
&lt;/a>
&lt;/div>
&lt;figcaption>HTTPS traffic to external services, with TLS originated by the microservice vs. by the sidecar proxy&lt;/figcaption>
&lt;/figure>
&lt;p>Here is how both patterns are supported in the
&lt;a href="https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/src/details/details.rb">Bookinfo details microservice code&lt;/a>, using the Ruby
&lt;a href="https://docs.ruby-lang.org/en/2.0.0/Net/HTTP.html">net/http module&lt;/a>:&lt;/p>
&lt;pre>&lt;code class='language-ruby' data-expandlinks='true' data-repo='istio' >uri = URI.parse(&amp;#39;https://www.googleapis.com/books/v1/volumes?q=isbn:&amp;#39; + isbn)
http = Net::HTTP.new(uri.host, ENV[&amp;#39;DO_NOT_ENCRYPT&amp;#39;] === &amp;#39;true&amp;#39; ? 80:443)
...
unless ENV[&amp;#39;DO_NOT_ENCRYPT&amp;#39;] === &amp;#39;true&amp;#39; then
http.use_ssl = true
end
&lt;/code>&lt;/pre>
&lt;p>When the &lt;code>DO_NOT_ENCRYPT&lt;/code> environment variable is defined, the request is performed without SSL (plain HTTP) to port 80.&lt;/p>
&lt;p>You can set the &lt;code>DO_NOT_ENCRYPT&lt;/code> environment variable to &lt;em>&amp;ldquo;true&amp;rdquo;&lt;/em> in the
&lt;a href="https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml">Kubernetes deployment spec of details v2&lt;/a>,
the &lt;code>container&lt;/code> section:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >env:
- name: DO_NOT_ENCRYPT
value: &amp;#34;true&amp;#34;
&lt;/code>&lt;/pre>
&lt;p>In the next section you will configure TLS origination for accessing an external web service.&lt;/p>
&lt;h2 id="bookinfo-with-tls-origination-to-a-google-books-web-service">Bookinfo with TLS origination to a Google Books web service&lt;/h2>
&lt;ol>
&lt;li>&lt;p>Deploy a version of &lt;em>details v2&lt;/em> that sends an HTTP request to
&lt;a href="https://developers.google.com/books/docs/v1/getting_started">Google Books APIs&lt;/a>. The &lt;code>DO_NOT_ENCRYPT&lt;/code> variable
is set to true in
&lt;a href="https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml">&lt;code>bookinfo-details-v2.yaml&lt;/code>&lt;/a>.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo-details-v2.yaml@
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Direct the traffic destined to the &lt;em>details&lt;/em> microservice, to &lt;em>details version v2&lt;/em>.&lt;/p>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-details-v2.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f @samples/bookinfo/networking/virtual-service-details-v2.yaml@
&lt;/code>&lt;/pre>&lt;/div>&lt;/li>
&lt;li>&lt;p>Create a mesh-external service entry for &lt;code>www.google.apis&lt;/code> , a virtual service to rewrite the destination port from
80 to 443, and a destination rule to perform TLS origination.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: googleapis
spec:
hosts:
- www.googleapis.com
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS
resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: rewrite-port-for-googleapis
spec:
hosts:
- www.googleapis.com
http:
- match:
- port: 80
route:
- destination:
host: www.googleapis.com
port:
number: 443
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: originate-tls-for-googleapis
spec:
host: www.googleapis.com
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: SIMPLE # initiates HTTPS when accessing www.googleapis.com
EOF
&lt;/code>&lt;/pre>&lt;/li>
&lt;li>&lt;p>Access the web page of the application and verify that the book details are displayed without errors.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;a href="/v1.12/docs/tasks/observability/logs/access-log/#enable-envoy-s-access-logging">Enable Envoy’s access logging&lt;/a>&lt;/p>&lt;/li>
&lt;li>&lt;p>Check the log of of the sidecar proxy of &lt;em>details v2&lt;/em> and see the HTTP request.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl logs $(kubectl get pods -l app=details -l version=v2 -o jsonpath=&amp;#39;{.items[0].metadata.name}&amp;#39;) istio-proxy | grep googleapis
[2018-08-09T11:32:58.171Z] &amp;#34;GET /books/v1/volumes?q=isbn:0486424618 HTTP/1.1&amp;#34; 200 - 0 1050 264 264 &amp;#34;-&amp;#34; &amp;#34;Ruby&amp;#34; &amp;#34;b993bae7-4288-9241-81a5-4cde93b2e3a6&amp;#34; &amp;#34;www.googleapis.com:80&amp;#34; &amp;#34;172.217.20.74:80&amp;#34;
EOF
&lt;/code>&lt;/pre>
&lt;p>Note the URL path in the log, the path can be monitored and access policies can be applied based on it. To read more
about monitoring and access policies for HTTP egress traffic, check out &lt;a href="https://archive.istio.io/v0.8/blog/2018/egress-monitoring-access-control/#logging">this blog post&lt;/a>.&lt;/p>&lt;/li>
&lt;/ol>
&lt;h3 id="cleanup-of-tls-origination-to-a-google-books-web-service">Cleanup of TLS origination to a Google Books web service&lt;/h3>
&lt;div>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml'>Zip&lt;/a>&lt;a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-details-v2.yaml'>Zip&lt;/a>&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl delete serviceentry googleapis
$ kubectl delete virtualservice rewrite-port-for-googleapis
$ kubectl delete destinationrule originate-tls-for-googleapis
$ kubectl delete -f @samples/bookinfo/networking/virtual-service-details-v2.yaml@
$ kubectl delete -f @samples/bookinfo/platform/kube/bookinfo-details-v2.yaml@
&lt;/code>&lt;/pre>&lt;/div>
&lt;h3 id="relation-to-istio-mutual-tls">Relation to Istio mutual TLS&lt;/h3>
&lt;p>Note that the TLS origination in this case is unrelated to
&lt;a href="/v1.12/docs/concepts/security/#mutual-tls-authentication">the mutual TLS&lt;/a> applied by Istio. The TLS origination for the
external services will work, whether the Istio mutual TLS is enabled or not. The &lt;strong>mutual&lt;/strong> TLS secures
service-to-service communication &lt;strong>inside&lt;/strong> the service mesh and provides each service with a strong identity. The
&lt;strong>external services&lt;/strong> in this blog post were accessed using &lt;strong>one-way TLS&lt;/strong>, the same mechanism used to secure communication between a
web browser and a web server. TLS is applied to the communication with external services to verify the identity of the
external server and to encrypt the traffic.&lt;/p>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>In this blog post I demonstrated how microservices in an Istio service mesh can consume external web services by
HTTPS. By default, Istio blocks all the traffic to the hosts outside the cluster. To enable such traffic, mesh-external
service entries must be created for the service mesh. It is possible to access the external sites either by
issuing HTTPS requests, or by issuing HTTP requests with Istio performing TLS origination. When the microservices issue
HTTPS requests, the traffic is encrypted end-to-end, however Istio cannot monitor HTTP details like the URL paths of the
requests. When the microservices issue HTTP requests, Istio can monitor the HTTP details of the requests and enforce
HTTP-based access policies. However, in that case the traffic between microservice and the sidecar proxy is unencrypted.
Having part of the traffic unencrypted can be forbidden in organizations with very strict security requirements.&lt;/p></description><pubDate>Wed, 31 Jan 2018 00:00:00 +0000</pubDate><link>/v1.12/blog/2018/egress-https/</link><author>Vadim Eisenberg</author><guid isPermaLink="true">/v1.12/blog/2018/egress-https/</guid><category>traffic-management</category><category>egress</category><category>https</category></item><item><title>Mixer and the SPOF Myth</title><description>
&lt;p>As &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/">Mixer&lt;/a> is in the request path, it is natural to question how it impacts
overall system availability and latency. A common refrain we hear when people first glance at Istio architecture diagrams is
&amp;ldquo;Isn&amp;rsquo;t this just introducing a single point of failure?&amp;rdquo;&lt;/p>
&lt;p>In this post, we’ll dig deeper and cover the design principles that underpin Mixer and the surprising fact Mixer actually
increases overall mesh availability and reduces average request latency.&lt;/p>
&lt;p>Istio&amp;rsquo;s use of Mixer has two main benefits in terms of overall system availability and latency:&lt;/p>
&lt;ul>
&lt;li>&lt;p>&lt;strong>Increased SLO&lt;/strong>. Mixer insulates proxies and services from infrastructure backend failures, enabling higher effective mesh availability. The mesh as a whole tends to experience a lower rate of failure when interacting with the infrastructure backends than if Mixer were not present.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Reduced Latency&lt;/strong>. Through aggressive use of shared multi-level caches and sharding, Mixer reduces average observed latencies across the mesh.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>We&amp;rsquo;ll explain this in more detail below.&lt;/p>
&lt;h2 id="how-we-got-here">How we got here&lt;/h2>
&lt;p>For many years at Google, we’ve been using an internal API &amp;amp; service management system to handle the many APIs exposed by Google. This system has been fronting the world’s biggest services (Google Maps, YouTube, Gmail, etc) and sustains a peak rate of hundreds of millions of QPS. Although this system has served us well, it had problems keeping up with Google’s rapid growth, and it became clear that a new architecture was needed in order to tamp down ballooning operational costs.&lt;/p>
&lt;p>In 2014, we started an initiative to create a replacement architecture that would scale better. The result has proven extremely successful and has been gradually deployed throughout Google, saving in the process millions of dollars a month in ops costs.&lt;/p>
&lt;p>The older system was built around a centralized fleet of fairly heavy proxies into which all incoming traffic would flow, before being forwarded to the services where the real work was done. The newer architecture jettisons the shared proxy design and instead consists of a very lean and efficient distributed sidecar proxy sitting next to service instances, along with a shared fleet of sharded control plane intermediaries:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:74.79295535770372%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2017/mixer-spof-myth/mixer-spof-myth-1.svg" title="Google System Topology">
&lt;img class="element-to-stretch" src="/v1.12/blog/2017/mixer-spof-myth/mixer-spof-myth-1.svg" alt="Google System Topology" />
&lt;/a>
&lt;/div>
&lt;figcaption>Google&amp;#39;s API &amp;amp; Service Management System&lt;/figcaption>
&lt;/figure>
&lt;p>Look familiar? Of course: it’s just like Istio! Istio was conceived as a second generation of this distributed proxy architecture. We took the core lessons from this internal system, generalized many of the concepts by working with our partners, and created Istio.&lt;/p>
&lt;h2 id="architecture-recap">Architecture recap&lt;/h2>
&lt;p>As shown in the diagram below, Mixer sits between the mesh and the infrastructure backends that support it:&lt;/p>
&lt;figure style="width:75%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:65.89948886170049%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2017/mixer-spof-myth/mixer-spof-myth-2.svg" title="Istio Topology">
&lt;img class="element-to-stretch" src="/v1.12/blog/2017/mixer-spof-myth/mixer-spof-myth-2.svg" alt="Istio Topology" />
&lt;/a>
&lt;/div>
&lt;figcaption>Istio Topology&lt;/figcaption>
&lt;/figure>
&lt;p>The Envoy sidecar logically calls Mixer before each request to perform precondition checks, and after each request to report telemetry.
The sidecar has local caching such that a relatively large percentage of precondition checks can be performed from cache. Additionally, the
sidecar buffers outgoing telemetry such that it only actually needs to call Mixer once for every several thousands requests. Whereas precondition
checks are synchronous to request processing, telemetry reports are done asynchronously with a fire-and-forget pattern.&lt;/p>
&lt;p>At a high level, Mixer provides:&lt;/p>
&lt;ul>
&lt;li>&lt;p>&lt;strong>Backend Abstraction&lt;/strong>. Mixer insulates the Istio components and services within the mesh from the implementation details of individual infrastructure backends.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Intermediation&lt;/strong>. Mixer allows operators to have fine-grained control over all interactions between the mesh and the infrastructure backends.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>However, even beyond these purely functional aspects, Mixer has other characteristics that provide the system with additional benefits.&lt;/p>
&lt;h2 id="mixer-slo-booster">Mixer: SLO booster&lt;/h2>
&lt;p>Contrary to the claim that Mixer is a SPOF and can therefore lead to mesh outages, we believe it in fact improves the effective availability of a mesh. How can that be? There are three basic characteristics at play:&lt;/p>
&lt;ul>
&lt;li>&lt;p>&lt;strong>Statelessness&lt;/strong>. Mixer is stateless in that it doesn’t manage any persistent storage of its own.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Hardening&lt;/strong>. Mixer proper is designed to be a highly reliable component. The design intent is to achieve &amp;gt; 99.999% uptime for any individual Mixer instance.&lt;/p>&lt;/li>
&lt;li>&lt;p>&lt;strong>Caching and Buffering&lt;/strong>. Mixer is designed to accumulate a large amount of transient ephemeral state.&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>The sidecar proxies that sit next to each service instance in the mesh must necessarily be frugal in terms of memory consumption, which constrains the possible amount of local caching and buffering. Mixer, however, lives independently and can use considerably larger caches and output buffers. Mixer thus acts as a highly-scaled and highly-available second-level cache for the sidecars.&lt;/p>
&lt;p>Mixer’s expected availability is considerably higher than most infrastructure backends (those often have availability of perhaps 99.9%). Its local caches and buffers help mask infrastructure backend failures by being able to continue operating even when a backend has become unresponsive.&lt;/p>
&lt;h2 id="mixer-latency-slasher">Mixer: Latency slasher&lt;/h2>
&lt;p>As we explained above, the Istio sidecars generally have fairly effective first-level caching. They can serve the majority of their traffic from cache. Mixer provides a much greater shared pool of second-level cache, which helps Mixer contribute to a lower average per-request latency.&lt;/p>
&lt;p>While it’s busy cutting down latency, Mixer is also inherently cutting down the number of calls your mesh makes to infrastructure backends. Depending on how you’re paying for these backends, this might end up saving you some cash by cutting down the effective QPS to the backends.&lt;/p>
&lt;h2 id="work-ahead">Work ahead&lt;/h2>
&lt;p>We have opportunities ahead to continue improving the system in many ways.&lt;/p>
&lt;h3 id="configuration-canaries">Configuration canaries&lt;/h3>
&lt;p>Mixer is highly scaled so it is generally resistant to individual instance failures. However, Mixer is still susceptible to cascading
failures in the case when a poison configuration is deployed which causes all Mixer instances to crash basically at the same time
(yeah, that would be a bad day). To prevent this from happening, configuration changes can be canaried to a small set of Mixer instances,
and then more broadly rolled out.&lt;/p>
&lt;p>Mixer doesn’t yet do canarying of configuration changes, but we expect this to come online as part of Istio’s ongoing work on reliable
configuration distribution.&lt;/p>
&lt;h3 id="cache-tuning">Cache tuning&lt;/h3>
&lt;p>We have yet to fine-tune the sizes of the sidecar and Mixer caches. This work will focus on achieving the highest performance possible using the least amount of resources.&lt;/p>
&lt;h3 id="cache-sharing">Cache sharing&lt;/h3>
&lt;p>At the moment, each Mixer instance operates independently of all other instances. A request handled by one Mixer instance will not leverage data cached in a different instance. We will eventually experiment with a distributed cache such as memcached or Redis in order to provide a much larger mesh-wide shared cache, and further reduce the number of calls to infrastructure backends.&lt;/p>
&lt;h3 id="sharding">Sharding&lt;/h3>
&lt;p>In very large meshes, the load on Mixer can be great. There can be a large number of Mixer instances, each straining to keep caches primed to
satisfy incoming traffic. We expect to eventually introduce intelligent sharding such that Mixer instances become slightly specialized in
handling particular data streams in order to increase the likelihood of cache hits. In other words, sharding helps improve cache
efficiency by routing related traffic to the same Mixer instance over time, rather than randomly dispatching to
any available Mixer instance.&lt;/p>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>Practical experience at Google showed that the model of a slim sidecar proxy and a large shared caching control plane intermediary hits a sweet
spot, delivering excellent perceived availability and latency. We’ve taken the lessons learned there and applied them to create more sophisticated and
effective caching, prefetching, and buffering strategies in Istio. We’ve also optimized the communication protocols to reduce overhead when a cache miss does occur.&lt;/p>
&lt;p>Mixer is still young. As of Istio 0.3, we haven’t really done significant performance work within Mixer itself. This means when a request misses the sidecar
cache, we spend more time in Mixer to respond to requests than we should. We’re doing a lot of work to improve this in coming months to reduce the overhead
that Mixer imparts in the synchronous precondition check case.&lt;/p>
&lt;p>We hope this post makes you appreciate the inherent benefits that Mixer brings to Istio.
Don’t hesitate to post comments or questions to &lt;a href="https://groups.google.com/forum/#!forum/istio-policies-and-telemetry">istio-policies-and-telemetry@&lt;/a>.&lt;/p></description><pubDate>Thu, 07 Dec 2017 00:00:00 +0000</pubDate><link>/v1.12/blog/2017/mixer-spof-myth/</link><author>Martin Taillefer</author><guid isPermaLink="true">/v1.12/blog/2017/mixer-spof-myth/</guid><category>adapters</category><category>mixer</category><category>policies</category><category>telemetry</category><category>availability</category><category>latency</category></item><item><title>Mixer Adapter Model</title><description>
&lt;p>Istio 0.2 introduced a new Mixer adapter model which is intended to increase Mixer’s flexibility to address a varied set of infrastructure backends. This post intends to put the adapter model in context and explain how it works.&lt;/p>
&lt;h2 id="why-adapters">Why adapters?&lt;/h2>
&lt;p>Infrastructure backends provide support functionality used to build services. They include such things as access control systems, telemetry capturing systems, quota enforcement systems, billing systems, and so forth. Services traditionally directly integrate with these backend systems, creating a hard coupling and baking-in specific semantics and usage options.&lt;/p>
&lt;p>Mixer serves as an abstraction layer between Istio and an open-ended set of infrastructure backends. The Istio components and services that run within the mesh can interact with these backends, while not being coupled to the backends’ specific interfaces.&lt;/p>
&lt;p>In addition to insulating application-level code from the details of infrastructure backends, Mixer provides an intermediation model that allows operators to inject and control policies between application code and backends. Operators can control which data is reported to which backend, which backend to consult for authorization, and much more.&lt;/p>
&lt;p>Given that individual infrastructure backends each have different interfaces and operational models, Mixer needs custom
code to deal with each and we call these custom bundles of code &lt;a href="https://github.com/istio/istio/wiki/Mixer-Compiled-In-Adapter-Dev-Guide">&lt;em>adapters&lt;/em>&lt;/a>.&lt;/p>
&lt;p>Adapters are Go packages that are directly linked into the Mixer binary. It’s fairly simple to create custom Mixer binaries linked with specialized sets of adapters, in case the default set of adapters is not sufficient for specific use cases.&lt;/p>
&lt;h2 id="philosophy">Philosophy&lt;/h2>
&lt;p>Mixer is essentially an attribute processing and routing machine. The proxy sends it &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/#attributes">attributes&lt;/a> as part of doing precondition checks and telemetry reports, which it turns into a series of calls into adapters. The operator supplies configuration which describes how to map incoming attributes to inputs for the adapters.&lt;/p>
&lt;figure style="width:60%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:42.60859894197215%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2017/adapter-model/machine.svg" title="Attribute Machine">
&lt;img class="element-to-stretch" src="/v1.12/blog/2017/adapter-model/machine.svg" alt="Attribute Machine" />
&lt;/a>
&lt;/div>
&lt;figcaption>Attribute Machine&lt;/figcaption>
&lt;/figure>
&lt;p>Configuration is a complex task. In fact, evidence shows that the overwhelming majority of service outages are caused by configuration errors. To help combat this, Mixer’s configuration model enforces a number of constraints designed to avoid errors. For example, the configuration model uses strong typing to ensure that only meaningful attributes or attribute expressions are used in any given context.&lt;/p>
&lt;h2 id="handlers-configuring-adapters">Handlers: configuring adapters&lt;/h2>
&lt;p>Each adapter that Mixer uses requires some configuration to operate. Typically, adapters need things like the URL to their backend, credentials, caching options, and so forth. Each adapter defines the exact configuration data it needs via a &lt;a href="https://developers.google.com/protocol-buffers/">protobuf&lt;/a> message.&lt;/p>
&lt;p>You configure each adapter by creating &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/#handlers">&lt;em>handlers&lt;/em>&lt;/a> for them. A handler is a
configuration resource which represents a fully configured adapter ready for use. There can be any number of handlers for a single adapter, making it possible to reuse an adapter in different scenarios.&lt;/p>
&lt;h2 id="templates-adapter-input-schema">Templates: adapter input schema&lt;/h2>
&lt;p>Mixer is typically invoked twice for every incoming request to a mesh service, once for precondition checks and once for telemetry reporting. For every such call, Mixer invokes one or more adapters. Different adapters need different pieces of data as input in order to do their work. A logging adapter needs a log entry, a metric adapter needs a metric, an authorization adapter needs credentials, etc.
Mixer &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/templates/">&lt;em>templates&lt;/em>&lt;/a> are used to describe the exact data that an adapter consumes at request time.&lt;/p>
&lt;p>Each template is specified as a &lt;a href="https://developers.google.com/protocol-buffers/">protobuf&lt;/a> message. A single template describes a bundle of data that is delivered to one or more adapters at runtime. Any given adapter can be designed to support any number of templates, the specific templates the adapter supports is determined by the adapter developer.&lt;/p>
&lt;p>&lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/templates/metric/">&lt;code>metric&lt;/code>&lt;/a> and &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/templates/logentry/">&lt;code>logentry&lt;/code>&lt;/a> are two of the most essential templates used within Istio. They represent respectively the payload to report a single metric and a single log entry to appropriate backends.&lt;/p>
&lt;h2 id="instances-attribute-mapping">Instances: attribute mapping&lt;/h2>
&lt;p>You control which data is delivered to individual adapters by creating
&lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/#instances">&lt;em>instances&lt;/em>&lt;/a>.
Instances control how Mixer uses the &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/#attributes">attributes&lt;/a> delivered
by the proxy into individual bundles of data that can be routed to different adapters.&lt;/p>
&lt;p>Creating instances generally requires using &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/expression-language/">attribute expressions&lt;/a>. The point of these expressions is to use any attribute or literal value in order to produce a result that can be assigned to an instance’s field.&lt;/p>
&lt;p>Every instance field has a type, as defined in the template, every attribute has a
&lt;a href="https://github.com/istio/api/blob/release-1.12/policy/v1beta1/value_type.proto">type&lt;/a>, and every attribute expression has a type.
You can only assign type-compatible expressions to any given instance fields. For example, you can’t assign an integer expression
to a string field. This kind of strong typing is designed to minimize the risk of creating bogus configurations.&lt;/p>
&lt;h2 id="rules-delivering-data-to-adapters">Rules: delivering data to adapters&lt;/h2>
&lt;p>The last piece to the puzzle is telling Mixer which instances to send to which handler and when. This is done by
creating &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/#rules">&lt;em>rules&lt;/em>&lt;/a>. Each rule identifies a specific handler and the set of
instances to send to that handler. Whenever Mixer processes an incoming call, it invokes the indicated handler and gives it the specific set of instances for processing.&lt;/p>
&lt;p>Rules contain matching predicates. A predicate is an attribute expression which returns a true/false value. A rule only takes effect if its predicate expression returns true. Otherwise, it’s like the rule didn’t exist and the indicated handler isn’t invoked.&lt;/p>
&lt;h2 id="future">Future&lt;/h2>
&lt;p>We are working to improve the end to end experience of using and developing adapters. For example, several new features are planned to make templates more expressive. Additionally, the expression language is being substantially enhanced to be more powerful and well-rounded.&lt;/p>
&lt;p>Longer term, we are evaluating ways to support adapters which aren’t directly linked into the main Mixer binary. This would simplify deployment and composition.&lt;/p>
&lt;h2 id="conclusion">Conclusion&lt;/h2>
&lt;p>The refreshed Mixer adapter model is designed to provide a flexible framework to support an open-ended set of infrastructure backends.&lt;/p>
&lt;p>Handlers provide configuration data for individual adapters, templates determine exactly what kind of data different adapters want to consume at runtime, instances let operators prepare this data, rules direct the data to one or more handlers.&lt;/p>
&lt;p>You can learn more about Mixer&amp;rsquo;s overall architecture &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/">here&lt;/a>, and learn the specifics of templates, handlers,
and rules &lt;a href="https://istio.io/v1.6/docs/reference/config/policy-and-telemetry">here&lt;/a>. You can find many examples of Mixer configuration resources in the Bookinfo sample
&lt;a href="https://github.com/istio/istio/tree/release-1.12/samples/bookinfo">here&lt;/a>.&lt;/p></description><pubDate>Fri, 03 Nov 2017 00:00:00 +0000</pubDate><link>/v1.12/blog/2017/adapter-model/</link><author>Martin Taillefer</author><guid isPermaLink="true">/v1.12/blog/2017/adapter-model/</guid><category>adapters</category><category>mixer</category><category>policies</category><category>telemetry</category></item><item><title>Using Network Policy with Istio</title><description>
&lt;p>The use of Network Policy to secure applications running on Kubernetes is a now a widely accepted industry best practice. Given that Istio also supports policy, we want to spend some time explaining how Istio policy and Kubernetes Network Policy interact and support each other to deliver your application securely.&lt;/p>
&lt;p>Let’s start with the basics: why might you want to use both Istio and Kubernetes Network Policy? The short answer is that they are good at different things. Consider the main differences between Istio and Network Policy (we will describe &amp;ldquo;typical” implementations, e.g. Calico, but implementation details can vary with different network providers):&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>&lt;/th>
&lt;th>Istio Policy&lt;/th>
&lt;th>Network Policy&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>&lt;strong>Layer&lt;/strong>&lt;/td>
&lt;td>&amp;ldquo;Service&amp;rdquo; &amp;mdash; L7&lt;/td>
&lt;td>&amp;ldquo;Network&amp;rdquo; &amp;mdash; L3-4&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;strong>Implementation&lt;/strong>&lt;/td>
&lt;td>User space&lt;/td>
&lt;td>Kernel&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>&lt;strong>Enforcement Point&lt;/strong>&lt;/td>
&lt;td>Pod&lt;/td>
&lt;td>Node&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;h2 id="layer">Layer&lt;/h2>
&lt;p>Istio policy operates at the &amp;ldquo;service” layer of your network application. This is Layer 7 (Application) from the perspective of the OSI model, but the de facto model of cloud native applications is that Layer 7 actually consists of at least two layers: a service layer and a content layer. The service layer is typically HTTP, which encapsulates the actual application data (the content layer). It is at this service layer of HTTP that the Istio’s Envoy proxy operates. In contrast, Network Policy operates at Layers 3 (Network) and 4 (Transport) in the OSI model.&lt;/p>
&lt;p>Operating at the service layer gives the Envoy proxy a rich set of attributes to base policy decisions on, for protocols it understands, which at present includes HTTP/1.1 &amp;amp; HTTP/2 (gRPC operates over HTTP/2). So, you can apply policy based on virtual host, URL, or other HTTP headers. In the future, Istio will support a wide range of Layer 7 protocols, as well as generic TCP and UDP transport.&lt;/p>
&lt;p>In contrast, operating at the network layer has the advantage of being universal, since all network applications use IP. At the network layer you can apply policy regardless of the layer 7 protocol: DNS, SQL databases, real-time streaming, and a plethora of other services that do not use HTTP can be secured. Network Policy isn’t limited to a classic firewall’s tuple of IP addresses, proto, and ports. Both Istio and Network Policy are aware of rich Kubernetes labels to describe pod endpoints.&lt;/p>
&lt;h2 id="implementation">Implementation&lt;/h2>
&lt;p>The Istio’s proxy is based on &lt;a href="https://envoyproxy.github.io/envoy/">Envoy&lt;/a>, which is implemented as a user space daemon in the data plane that
interacts with the network layer using standard sockets. This gives it a large amount of flexibility in processing, and allows it to be
distributed (and upgraded!) in a container.&lt;/p>
&lt;p>Network Policy data plane is typically implemented in kernel space (e.g. using iptables, eBPF filters, or even custom kernel modules). Being in kernel space
allows them to be extremely fast, but not as flexible as the Envoy proxy.&lt;/p>
&lt;h2 id="enforcement-point">Enforcement point&lt;/h2>
&lt;p>Policy enforcement using the Envoy proxy is implemented inside the pod, as a sidecar container in the same network namespace. This allows a simple deployment model. Some containers are given permission to reconfigure the networking inside their pod (&lt;code>CAP_NET_ADMIN&lt;/code>). If such a service instance is compromised, or misbehaves (as in a malicious tenant) the proxy can be bypassed.&lt;/p>
&lt;p>While this won’t let an attacker access other Istio-enabled pods, so long as they are correctly configured, it opens several attack vectors:&lt;/p>
&lt;ul>
&lt;li>Attacking unprotected pods&lt;/li>
&lt;li>Attempting to deny service to protected pods by sending lots of traffic&lt;/li>
&lt;li>Exfiltrating data collected in the pod&lt;/li>
&lt;li>Attacking the cluster infrastructure (servers or Kubernetes services)&lt;/li>
&lt;li>Attacking services outside the mesh, like databases, storage arrays, or legacy systems.&lt;/li>
&lt;/ul>
&lt;p>Network Policy is typically enforced at the host node, outside the network namespace of the guest pods. This means that compromised or misbehaving pods must break into the root namespace to avoid enforcement. With the addition of egress policy due in Kubernetes 1.8, this difference makes Network Policy a key part of protecting your infrastructure from compromised workloads.&lt;/p>
&lt;h2 id="examples">Examples&lt;/h2>
&lt;p>Let’s walk through a few examples of what you might want to do with Kubernetes Network Policy for an Istio-enabled application. Consider the Bookinfo sample application. We’re going to cover the following use cases for Network Policy:&lt;/p>
&lt;ul>
&lt;li>Reduce attack surface of the application ingress&lt;/li>
&lt;li>Enforce fine-grained isolation within the application&lt;/li>
&lt;/ul>
&lt;h3 id="reduce-attack-surface-of-the-application-ingress">Reduce attack surface of the application ingress&lt;/h3>
&lt;p>Our application ingress controller is the main entry-point to our application from the outside world. A quick peek at &lt;code>istio.yaml&lt;/code> (used to install Istio) defines the Istio ingress like this:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Service
metadata:
name: istio-ingress
labels:
istio: ingress
spec:
type: LoadBalancer
ports:
- port: 80
name: http
- port: 443
name: https
selector:
istio: ingress
&lt;/code>&lt;/pre>
&lt;p>The &lt;code>istio-ingress&lt;/code> exposes ports 80 and 443. Let’s limit incoming traffic to just these two ports. Envoy has a &lt;a href="https://www.envoyproxy.io/docs/envoy/latest/operations/admin.html#operations-admin-interface">built-in administrative interface&lt;/a>, and we don’t want a misconfigured &lt;code>istio-ingress&lt;/code> image to accidentally expose our admin interface to the outside world. This is an example of defense in depth: a properly configured image should not expose the interface, and a properly configured Network Policy will prevent anyone from connecting to it. Either can fail or be misconfigured and we are still protected.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: istio-ingress-lockdown
namespace: default
spec:
podSelector:
matchLabels:
istio: ingress
ingress:
- ports:
- protocol: TCP
port: 80
- protocol: TCP
port: 443
&lt;/code>&lt;/pre>
&lt;h3 id="enforce-fine-grained-isolation-within-the-application">Enforce fine-grained isolation within the application&lt;/h3>
&lt;p>Here is the service graph for the Bookinfo application.&lt;/p>
&lt;figure style="width:80%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.086918235567985%">
&lt;a data-skipendnotes="true" href="/v1.12/docs/examples/bookinfo/withistio.svg" title="Bookinfo Service Graph">
&lt;img class="element-to-stretch" src="/v1.12/docs/examples/bookinfo/withistio.svg" alt="Bookinfo Service Graph" />
&lt;/a>
&lt;/div>
&lt;figcaption>Bookinfo Service Graph&lt;/figcaption>
&lt;/figure>
&lt;p>This graph shows every connection that a correctly functioning application should be allowed to make. All other connections, say from the Istio Ingress directly to the Rating service, are not part of the application. Let’s lock out those extraneous connections so they cannot be used by an attacker. Imagine, for example, that the Ingress pod is compromised by an exploit that allows an attacker to run arbitrary code. If we only allow connections to the Product Page pods using Network Policy, the attacker has gained no more access to my application backends &lt;em>even though they have compromised a member of the service mesh&lt;/em>.&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: product-page-ingress
namespace: default
spec:
podSelector:
matchLabels:
app: productpage
ingress:
- ports:
- protocol: TCP
port: 9080
from:
- podSelector:
matchLabels:
istio: ingress
&lt;/code>&lt;/pre>
&lt;p>You can and should write a similar policy for each service to enforce which other pods are allowed to access each.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>Our take is that Istio and Network Policy have different strengths in applying policy. Istio is application-protocol aware and highly flexible, making it ideal for applying policy in support of operational goals, like service routing, retries, circuit-breaking, etc, and for security that operates at the application layer, such as token validation. Network Policy is universal, highly efficient, and isolated from the pods, making it ideal for applying policy in support of network security goals. Furthermore, having policy that operates at different layers of the network stack is a really good thing as it gives each layer specific context without commingling of state and allows separation of responsibility.&lt;/p>
&lt;p>This post is based on the three part blog series by Spike Curtis, one of the Istio team members at Tigera. The full series can be found here: &lt;a href="https://www.projectcalico.org/using-network-policy-in-concert-with-istio/">https://www.projectcalico.org/using-network-policy-in-concert-with-istio/&lt;/a>&lt;/p></description><pubDate>Thu, 10 Aug 2017 00:00:00 +0000</pubDate><link>/v1.12/blog/2017/0.1-using-network-policy/</link><author>Spike Curtis</author><guid isPermaLink="true">/v1.12/blog/2017/0.1-using-network-policy/</guid></item><item><title>Canary Deployments using Istio</title><description>
&lt;div>
&lt;aside class="callout tip">
&lt;div class="type">
&lt;svg class="large-icon">&lt;use xlink:href="/v1.12/img/icons.svg#callout-tip"/>&lt;/svg>
&lt;/div>
&lt;div class="content">This post was updated on May 16, 2018 to use the latest version of the traffic management model.&lt;/div>
&lt;/aside>
&lt;/div>
&lt;p>One of the benefits of the &lt;a href="/v1.12/">Istio&lt;/a> project is that it provides the control needed to deploy canary services. The idea behind
canary deployment (or rollout) is to introduce a new version of a service by first testing it using a small percentage of user
traffic, and then if all goes well, increase, possibly gradually in increments, the percentage while simultaneously phasing out
the old version. If anything goes wrong along the way, we abort and roll back to the previous version. In its simplest form,
the traffic sent to the canary version is a randomly selected percentage of requests, but in more sophisticated schemes it
can be based on the region, user, or other properties of the request.&lt;/p>
&lt;p>Depending on your level of expertise in this area, you may wonder why Istio&amp;rsquo;s support for canary deployment is even needed, given that platforms like Kubernetes already provide a way to do &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment">version rollout&lt;/a> and &lt;a href="https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments">canary deployment&lt;/a>. Problem solved, right? Well, not exactly. Although doing a rollout this way works in simple cases, it’s very limited, especially in large scale cloud environments receiving lots of (and especially varying amounts of) traffic, where autoscaling is needed.&lt;/p>
&lt;h2 id="canary-deployment-in-kubernetes">Canary deployment in Kubernetes&lt;/h2>
&lt;p>As an example, let&amp;rsquo;s say we have a deployed service, &lt;strong>helloworld&lt;/strong> version &lt;strong>v1&lt;/strong>, for which we would like to test (or simply roll out) a new version, &lt;strong>v2&lt;/strong>. Using Kubernetes, you can roll out a new version of the &lt;strong>helloworld&lt;/strong> service by simply updating the image in the service’s corresponding &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployment&lt;/a> and letting the &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment">rollout&lt;/a> happen automatically. If we take particular care to ensure that there are enough &lt;strong>v1&lt;/strong> replicas running when we start and &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#pausing-and-resuming-a-deployment">pause&lt;/a> the rollout after only one or two &lt;strong>v2&lt;/strong> replicas have been started, we can keep the canary’s effect on the system very small. We can then observe the effect before deciding to proceed or, if necessary, &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-back-a-deployment">roll back&lt;/a>. Best of all, we can even attach a &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment">horizontal pod autoscaler&lt;/a> to the Deployment and it will keep the replica ratios consistent if, during the rollout process, it also needs to scale replicas up or down to handle traffic load.&lt;/p>
&lt;p>Although fine for what it does, this approach is only useful when we have a properly tested version that we want to deploy, i.e., more of a blue/green, a.k.a. red/black, kind of upgrade than a &amp;ldquo;dip your feet in the water&amp;rdquo; kind of canary deployment. In fact, for the latter (for example, testing a canary version that may not even be ready or intended for wider exposure), the canary deployment in Kubernetes would be done using two Deployments with &lt;a href="https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#using-labels-effectively">common pod labels&lt;/a>. In this case, we can’t use autoscaling anymore because it’s now being done by two independent autoscalers, one for each Deployment, so the replica ratios (percentages) may vary from the desired ratio, depending purely on load.&lt;/p>
&lt;p>Whether we use one deployment or two, canary management using deployment features of container orchestration platforms like Docker, Mesos/Marathon, or Kubernetes has a fundamental problem: the use of instance scaling to manage the traffic; traffic version distribution and replica deployment are not independent in these systems. All replica pods, regardless of version, are treated the same in the &lt;code>kube-proxy&lt;/code> round-robin pool, so the only way to manage the amount of traffic that a particular version receives is by controlling the replica ratio. Maintaining canary traffic at small percentages requires many replicas (e.g., 1% would require a minimum of 100 replicas). Even if we ignore this problem, the deployment approach is still very limited in that it only supports the simple (random percentage) canary approach. If, instead, we wanted to limit the visibility of the canary to requests based on some specific criteria, we still need another solution.&lt;/p>
&lt;h2 id="enter-istio">Enter Istio&lt;/h2>
&lt;p>With Istio, traffic routing and replica deployment are two completely independent functions. The number of pods implementing services are free to scale up and down based on traffic load, completely orthogonal to the control of version traffic routing. This makes managing a canary version in the presence of autoscaling a much simpler problem. Autoscalers may, in fact, respond to load variations resulting from traffic routing changes, but they are nevertheless functioning independently and no differently than when loads change for other reasons.&lt;/p>
&lt;p>Istio’s &lt;a href="/v1.12/docs/concepts/traffic-management/#routing-rules">routing rules&lt;/a> also provide other important advantages; you can easily control
fine-grained traffic percentages (e.g., route 1% of traffic without requiring 100 pods) and you can control traffic using other criteria (e.g., route traffic for specific users to the canary version). To illustrate, let’s look at deploying the &lt;strong>helloworld&lt;/strong> service and see how simple the problem becomes.&lt;/p>
&lt;p>We begin by defining the &lt;strong>helloworld&lt;/strong> Service, just like any other Kubernetes service, something like this:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: v1
kind: Service
metadata:
name: helloworld
labels:
app: helloworld
spec:
selector:
app: helloworld
...
&lt;/code>&lt;/pre>
&lt;p>We then add 2 Deployments, one for each version (&lt;strong>v1&lt;/strong> and &lt;strong>v2&lt;/strong>), both of which include the service selector’s &lt;code>app: helloworld&lt;/code> label:&lt;/p>
&lt;pre>&lt;code class='language-yaml' data-expandlinks='true' data-repo='istio' >apiVersion: apps/v1
kind: Deployment
metadata:
name: helloworld-v1
spec:
replicas: 1
template:
metadata:
labels:
app: helloworld
version: v1
spec:
containers:
- image: helloworld-v1
...
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: helloworld-v2
spec:
replicas: 1
template:
metadata:
labels:
app: helloworld
version: v2
spec:
containers:
- image: helloworld-v2
...
&lt;/code>&lt;/pre>
&lt;p>Note that this is exactly the same way we would do a &lt;a href="https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments">canary deployment&lt;/a> using plain Kubernetes, but in that case we would need to adjust the number of replicas of each Deployment to control the distribution of traffic. For example, to send 10% of the traffic to the canary version (&lt;strong>v2&lt;/strong>), the replicas for &lt;strong>v1&lt;/strong> and &lt;strong>v2&lt;/strong> could be set to 9 and 1, respectively.&lt;/p>
&lt;p>However, since we are going to deploy the service in an &lt;a href="/v1.12/docs/setup/">Istio enabled&lt;/a> cluster, all we need to do is set a routing
rule to control the traffic distribution. For example if we want to send 10% of the traffic to the canary, we could use &lt;code>kubectl&lt;/code>
to set a routing rule something like this:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: helloworld
spec:
hosts:
- helloworld
http:
- route:
- destination:
host: helloworld
subset: v1
weight: 90
- destination:
host: helloworld
subset: v2
weight: 10
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: helloworld
spec:
host: helloworld
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
EOF
&lt;/code>&lt;/pre>
&lt;p>After setting this rule, Istio will ensure that only one tenth of the requests will be sent to the canary version, regardless of how many replicas of each version are running.&lt;/p>
&lt;h2 id="autoscaling-the-deployments">Autoscaling the deployments&lt;/h2>
&lt;p>Because we don’t need to maintain replica ratios anymore, we can safely add Kubernetes &lt;a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/">horizontal pod autoscalers&lt;/a> to manage the replicas for both version Deployments:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl autoscale deployment helloworld-v1 --cpu-percent=50 --min=1 --max=10
deployment &amp;#34;helloworld-v1&amp;#34; autoscaled
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl autoscale deployment helloworld-v2 --cpu-percent=50 --min=1 --max=10
deployment &amp;#34;helloworld-v2&amp;#34; autoscaled
&lt;/code>&lt;/pre>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get hpa
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
Helloworld-v1 Deployment/helloworld-v1 50% 47% 1 10 17s
Helloworld-v2 Deployment/helloworld-v2 50% 40% 1 10 15s
&lt;/code>&lt;/pre>
&lt;p>If we now generate some load on the &lt;strong>helloworld&lt;/strong> service, we would notice that when scaling begins, the &lt;strong>v1&lt;/strong> autoscaler will scale up its replicas significantly higher than the &lt;strong>v2&lt;/strong> autoscaler will for its replicas because &lt;strong>v1&lt;/strong> pods are handling 90% of the load.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods | grep helloworld
helloworld-v1-3523621687-3q5wh 0/2 Pending 0 15m
helloworld-v1-3523621687-73642 2/2 Running 0 11m
helloworld-v1-3523621687-7hs31 2/2 Running 0 19m
helloworld-v1-3523621687-dt7n7 2/2 Running 0 50m
helloworld-v1-3523621687-gdhq9 2/2 Running 0 11m
helloworld-v1-3523621687-jxs4t 0/2 Pending 0 15m
helloworld-v1-3523621687-l8rjn 2/2 Running 0 19m
helloworld-v1-3523621687-wwddw 2/2 Running 0 15m
helloworld-v1-3523621687-xlt26 0/2 Pending 0 19m
helloworld-v2-4095161145-963wt 2/2 Running 0 50m
&lt;/code>&lt;/pre>
&lt;p>If we then change the routing rule to send 50% of the traffic to &lt;strong>v2&lt;/strong>, we should, after a short delay, notice that the &lt;strong>v1&lt;/strong> autoscaler will scale down the replicas of &lt;strong>v1&lt;/strong> while the &lt;strong>v2&lt;/strong> autoscaler will perform a corresponding scale up.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods | grep helloworld
helloworld-v1-3523621687-73642 2/2 Running 0 35m
helloworld-v1-3523621687-7hs31 2/2 Running 0 43m
helloworld-v1-3523621687-dt7n7 2/2 Running 0 1h
helloworld-v1-3523621687-gdhq9 2/2 Running 0 35m
helloworld-v1-3523621687-l8rjn 2/2 Running 0 43m
helloworld-v2-4095161145-57537 0/2 Pending 0 21m
helloworld-v2-4095161145-9322m 2/2 Running 0 21m
helloworld-v2-4095161145-963wt 2/2 Running 0 1h
helloworld-v2-4095161145-c3dpj 0/2 Pending 0 21m
helloworld-v2-4095161145-t2ccm 0/2 Pending 0 17m
helloworld-v2-4095161145-v3v9n 0/2 Pending 0 13m
&lt;/code>&lt;/pre>
&lt;p>The end result is very similar to the simple Kubernetes Deployment rollout, only now the whole process is not being orchestrated and managed in one place. Instead, we’re seeing several components doing their jobs independently, albeit in a cause and effect manner.
What&amp;rsquo;s different, however, is that if we now stop generating load, the replicas of both versions will eventually scale down to their minimum (1), regardless of what routing rule we set.&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl get pods | grep helloworld
helloworld-v1-3523621687-dt7n7 2/2 Running 0 1h
helloworld-v2-4095161145-963wt 2/2 Running 0 1h
&lt;/code>&lt;/pre>
&lt;h2 id="focused-canary-testing">Focused canary testing&lt;/h2>
&lt;p>As mentioned above, the Istio routing rules can be used to route traffic based on specific criteria, allowing more sophisticated canary deployment scenarios. Say, for example, instead of exposing the canary to an arbitrary percentage of users, we want to try it out on internal users, maybe even just a percentage of them. The following command could be used to send 50% of traffic from users at &lt;em>some-company-name.com&lt;/em> to the canary version, leaving all other users unaffected:&lt;/p>
&lt;pre>&lt;code class='language-bash' data-expandlinks='true' data-repo='istio' >$ kubectl apply -f - &amp;lt;&amp;lt;EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: helloworld
spec:
hosts:
- helloworld
http:
- match:
- headers:
cookie:
regex: &amp;#34;^(.*?;)?(email=[^;]*@some-company-name.com)(;.*)?$&amp;#34;
route:
- destination:
host: helloworld
subset: v1
weight: 50
- destination:
host: helloworld
subset: v2
weight: 50
- route:
- destination:
host: helloworld
subset: v1
EOF
&lt;/code>&lt;/pre>
&lt;p>As before, the autoscalers bound to the 2 version Deployments will automatically scale the replicas accordingly, but that will have no affect on the traffic distribution.&lt;/p>
&lt;h2 id="summary">Summary&lt;/h2>
&lt;p>In this article we’ve seen how Istio supports general scalable canary deployments, and how this differs from the basic deployment support in Kubernetes. Istio’s service mesh provides the control necessary to manage traffic distribution with complete independence from deployment scaling. This allows for a simpler, yet significantly more functional, way to do canary test and rollout.&lt;/p>
&lt;p>Intelligent routing in support of canary deployment is just one of the many features of Istio that will make the production deployment of large-scale microservices-based applications much simpler. Check out &lt;a href="/v1.12/">istio.io&lt;/a> for more information and to try it out.
The sample code used in this article can be found &lt;a href="https://github.com/istio/istio/tree/release-1.12/samples/helloworld">here&lt;/a>.&lt;/p></description><pubDate>Wed, 14 Jun 2017 00:00:00 +0000</pubDate><link>/v1.12/blog/2017/0.1-canary/</link><author>Frank Budinsky</author><guid isPermaLink="true">/v1.12/blog/2017/0.1-canary/</guid><category>traffic-management</category><category>canary</category></item><item><title>Using Istio to Improve End-to-End Security</title><description>
&lt;p>Conventional network security approaches fail to address security threats to distributed applications deployed in dynamic production environments. Today, we describe how Istio authentication enables enterprises to transform their security posture from just protecting the edge to consistently securing all inter-service communications deep within their applications. With Istio authentication, developers and operators can protect services with sensitive data against unauthorized insider access and they can achieve this without any changes to the application code!&lt;/p>
&lt;p>Istio authentication is the security component of the broader Istio platform. It incorporates the learnings of securing millions of microservice
endpoints in Google’s production environment.&lt;/p>
&lt;h2 id="background">Background&lt;/h2>
&lt;p>Modern application architectures are increasingly based on shared services that are deployed and scaled dynamically on cloud platforms. Traditional network edge security (e.g. firewall) is too coarse-grained and allows access from unintended clients. An example of a security risk is stolen authentication tokens that can be replayed from another client. This is a major risk for companies with sensitive data that are concerned about insider threats. Other network security approaches like IP whitelists have to be statically defined, are hard to manage at scale, and are unsuitable for dynamic production environments.&lt;/p>
&lt;p>Thus, security administrators need a tool that enables them to consistently, and by default, secure all communication between services across diverse production environments.&lt;/p>
&lt;h2 id="solution-strong-service-identity-and-authentication">Solution: strong service identity and authentication&lt;/h2>
&lt;p>Google has, over the years, developed architecture and technology to uniformly secure millions of microservice endpoints in its production environment against
external
attacks and insider threats. Key security principles include trusting the endpoints and not the network, strong mutual authentication based on service identity and service level authorization. Istio authentication is based on the same principles.&lt;/p>
&lt;p>The version 0.1 release of Istio authentication runs on Kubernetes and provides the following features:&lt;/p>
&lt;ul>
&lt;li>&lt;p>Strong identity assertion between services&lt;/p>&lt;/li>
&lt;li>&lt;p>Access control to limit the identities that can access a service (and its data)&lt;/p>&lt;/li>
&lt;li>&lt;p>Automatic encryption of data in transit&lt;/p>&lt;/li>
&lt;li>&lt;p>Management of keys and certificates at scale&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>Istio authentication is based on industry standards like mutual TLS and X.509. Furthermore, Google is actively contributing to an open, community-driven service security framework called &lt;a href="https://spiffe.io/">SPIFFE&lt;/a>. As the &lt;a href="https://spiffe.io/">SPIFFE&lt;/a> specifications mature, we intend for Istio authentication to become a reference implementation of the same.&lt;/p>
&lt;p>The diagram below provides an overview of the Istio&amp;rsquo;s service authentication architecture on Kubernetes.&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2017/0.1-auth/istio_auth_overview.svg" title="Istio Authentication Overview">
&lt;img class="element-to-stretch" src="/v1.12/blog/2017/0.1-auth/istio_auth_overview.svg" alt="Istio Authentication Overview" />
&lt;/a>
&lt;/div>
&lt;figcaption>Istio Authentication Overview&lt;/figcaption>
&lt;/figure>
&lt;p>The above diagram illustrates three key security features:&lt;/p>
&lt;h3 id="strong-identity">Strong identity&lt;/h3>
&lt;p>Istio authentication uses &lt;a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/">Kubernetes service accounts&lt;/a> to identify who the service runs as. The identity is used to establish trust and define service level access policies. The identity is assigned at service deployment time and encoded in the SAN (Subject Alternative Name) field of an X.509 certificate. Using a service account as the identity has the following advantages:&lt;/p>
&lt;ul>
&lt;li>&lt;p>Administrators can configure who has access to a Service Account by using the &lt;a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">RBAC&lt;/a> feature introduced in Kubernetes 1.6&lt;/p>&lt;/li>
&lt;li>&lt;p>Flexibility to identify a human user, a service, or a group of services&lt;/p>&lt;/li>
&lt;li>&lt;p>Stability of the service identity for dynamically placed and auto-scaled workloads&lt;/p>&lt;/li>
&lt;/ul>
&lt;h3 id="communication-security">Communication security&lt;/h3>
&lt;p>Service-to-service communication is tunneled through high performance client side and server side &lt;a href="https://envoyproxy.github.io/envoy/">Envoy&lt;/a> proxies. The communication between the proxies is secured using mutual TLS. The benefit of using mutual TLS is that the service identity is not expressed as a bearer token that can be stolen or replayed from another source. Istio authentication also introduces the concept of Secure Naming to protect from a server spoofing attacks - the client side proxy verifies that the authenticated server&amp;rsquo;s service account is allowed to run the named service.&lt;/p>
&lt;h3 id="key-management-and-distribution">Key management and distribution&lt;/h3>
&lt;p>Istio authentication provides a per-cluster CA (Certificate Authority) and automated key &amp;amp; certificate management. In this context, Istio authentication:&lt;/p>
&lt;ul>
&lt;li>&lt;p>Generates a key and certificate pair for each service account.&lt;/p>&lt;/li>
&lt;li>&lt;p>Distributes keys and certificates to the appropriate pods using &lt;a href="https://kubernetes.io/docs/concepts/configuration/secret/">Kubernetes Secrets&lt;/a>.&lt;/p>&lt;/li>
&lt;li>&lt;p>Rotates keys and certificates periodically.&lt;/p>&lt;/li>
&lt;li>&lt;p>Revokes a specific key and certificate pair when necessary (future).&lt;/p>&lt;/li>
&lt;/ul>
&lt;p>The following diagram explains the end to end Istio authentication workflow on Kubernetes:&lt;/p>
&lt;figure style="width:100%">
&lt;div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
&lt;a data-skipendnotes="true" href="/v1.12/blog/2017/0.1-auth/istio_auth_workflow.svg" title="Istio Authentication Workflow">
&lt;img class="element-to-stretch" src="/v1.12/blog/2017/0.1-auth/istio_auth_workflow.svg" alt="Istio Authentication Workflow" />
&lt;/a>
&lt;/div>
&lt;figcaption>Istio Authentication Workflow&lt;/figcaption>
&lt;/figure>
&lt;p>Istio authentication is part of the broader security story for containers. Red Hat, a partner on the development of Kubernetes, has identified &lt;a href="https://www.redhat.com/en/resources/container-security-openshift-cloud-devops-whitepaper">10 Layers&lt;/a> of container security. Istio addresses two of these layers: &amp;ldquo;Network Isolation&amp;rdquo; and &amp;ldquo;API and Service Endpoint Management&amp;rdquo;. As cluster federation evolves on Kubernetes and other platforms, our intent is for Istio to secure communications across services spanning multiple federated clusters.&lt;/p>
&lt;h2 id="benefits-of-istio-authentication">Benefits of Istio authentication&lt;/h2>
&lt;p>&lt;strong>Defense in depth&lt;/strong>: When used in conjunction with Kubernetes (or infrastructure) network policies, users achieve higher levels of confidence, knowing that pod-to-pod or service-to-service communication is secured both at network and application layers.&lt;/p>
&lt;p>&lt;strong>Secure by default&lt;/strong>: When used with Istio’s proxy and centralized policy engine, Istio authentication can be configured during deployment with minimal or no application change. Administrators and operators can thus ensure that service communications are secured by default and that they can enforce these policies consistently across diverse protocols and runtimes.&lt;/p>
&lt;p>&lt;strong>Strong service authentication&lt;/strong>: Istio authentication secures service communication using mutual TLS to ensure that the service identity is not expressed as a bearer token that can be stolen or replayed from another source. This ensures that services with sensitive data can only be accessed from strongly authenticated and authorized clients.&lt;/p>
&lt;h2 id="join-us-in-this-journey">Join us in this journey&lt;/h2>
&lt;p>Istio authentication is the first step towards providing a full stack of capabilities to protect services with sensitive data from external attacks and insider
threats. While the initial version runs on Kubernetes, our goal is to enable Istio authentication to secure services across diverse production environments. We encourage the
community to &lt;a href="https://github.com/istio/istio/tree/release-1.12/security">join us&lt;/a> in making robust service security easy and ubiquitous across different application
stacks and runtime platforms.&lt;/p></description><pubDate>Thu, 25 May 2017 00:00:00 +0000</pubDate><link>/v1.12/blog/2017/0.1-auth/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.12/blog/2017/0.1-auth/</guid></item></channel></rss>
Reference in New Issue View Git Blame Copy Permalink