mirror of https://github.com/istio/istio.io.git
6195 lines
541 KiB
XML
6195 lines
541 KiB
XML
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Istio Blog</title><description>Connect, secure, control, and observe services.</description><link>/v1.1</link><image><url>/v1.1/favicons/android-192x192.png</url><link>/v1.1</link></image><category>Service mesh</category><item><title>Announcing Istio 1.1.9</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.1.9. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<a class="btn" href="https://github.com/istio/istio/releases/tag/1.1.9">DOWNLOAD 1.1.9</a>
|
||
<a class="btn" href="https://istio.io/docs">1.1.9 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.8...1.1.9">CHANGES IN 1.1.9</a>
|
||
</div>
|
||
<h2 id="bug-fixes">Bug fixes</h2>
|
||
<ul>
|
||
<li>Prevent overly large strings from being sent to Prometheus (<a href="https://github.com/istio/istio/issues/14642">Issue 14642</a>).</li>
|
||
<li>Reuse previously cached JWT public keys if transport errors are encountered during renewal (<a href="https://github.com/istio/istio/issues/14638">Issue 14638</a>).</li>
|
||
<li>Bypass JWT authentication for HTTP OPTIONS methods to support CORS requests (<a href="https://github.com/istio/proxy/issues/2160">Issue 2160</a>).</li>
|
||
<li>Fix Envoy crash caused by the Mixer filter (<a href="https://github.com/istio/istio/issues/14707">Issue 14707</a>).</li>
|
||
</ul>
|
||
<h2 id="small-enhancements">Small enhancements</h2>
|
||
<ul>
|
||
<li>Expose cryptographic signature verification functions to <code>Lua</code> Envoy filters (<a href="https://github.com/envoyproxy/envoy/issues/7009">Envoy Issue 7009</a>).</li>
|
||
</ul></description><pubDate>Mon, 17 Jun 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.9/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.9/</guid></item><item><title>Extending Istio Self-Signed Root Certificate Lifetime</title><description><p>Istio self-signed certificates have historically had a 1 year default lifetime.
|
||
If you are using Istio self-signed certificates,
|
||
you need to schedule regular root transitions before they expire.
|
||
An expiration of a root certificate may lead to an unexpected cluster-wide outage.
|
||
The issue affects all versions up to 1.0.7 and 1.1.7.</p>
|
||
<p>See <a href="/v1.1/help/ops/security/root-transition/">Extending Self-Signed Certificate Lifetime</a> for
|
||
information on how to gauge the age of your certificates and how to perform rotation.</p>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">We strongly recommend you rotate root keys and root certificates annually as a security best practice.
|
||
We will send out instructions for root key/cert rotation as a follow-up.</div>
|
||
</aside>
|
||
</div></description><pubDate>Fri, 07 Jun 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/root-transition/</link><author>Oliver Liu</author><guid isPermaLink="true">/v1.1/blog/2019/root-transition/</guid><category>security</category><category>PKI</category><category>certificate</category><category>Citadel</category></item><item><title>Announcing Istio 1.0.8</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.0.8. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<a class="btn" href="https://github.com/istio/istio/releases/tag/1.0.8">DOWNLOAD 1.0.8</a>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0.8 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.0.7...1.0.8">CHANGES IN 1.0.8</a>
|
||
</div>
|
||
<h2 id="bug-fixes">Bug fixes</h2>
|
||
<ul>
|
||
<li>Fix issue where Citadel could generate a new root CA if it cannot contact the Kubernetes API server, causing mutual TLS verification to incorrectly fail (<a href="https://github.com/istio/istio/issues/14512">Issue 14512</a>).</li>
|
||
</ul>
|
||
<h2 id="small-enhancements">Small enhancements</h2>
|
||
<ul>
|
||
<li>Update Citadel&rsquo;s default root CA certificate TTL from 1 year to 10 years.</li>
|
||
</ul></description><pubDate>Fri, 07 Jun 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.0.8/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.0.8/</guid></item><item><title>Announcing Istio 1.1.8</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.1.8. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.8"
|
||
data-updateadvice='Before you download 1.1.8, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1.8
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1.8 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.7...1.1.8">CHANGES IN 1.1.8</a>
|
||
</div>
|
||
<h2 id="bug-fixes">Bug fixes</h2>
|
||
<ul>
|
||
<li>Fix <code>PASSTHROUGH</code> <code>DestinationRules</code> for CDS clusters (<a href="https://github.com/istio/istio/issues/13744">Issue 13744</a>).</li>
|
||
<li>Make the <code>appVersion</code> and <code>version</code> fields in the Helm charts display the correct Istio version (<a href="https://github.com/istio/istio/issues/14290">Issue 14290</a>).</li>
|
||
<li>Fix Mixer crash affecting both policy and telemetry servers (<a href="https://github.com/istio/istio/issues/14235">Issue 14235</a>).</li>
|
||
<li>Fix multicluster issue where two pods in different clusters could not share the same IP address (<a href="https://github.com/istio/istio/issues/14066">Issue 14066</a>).</li>
|
||
<li>Fix issue where Citadel could generate a new root CA if it cannot contact the Kubernetes API server, causing mutual TLS verification to incorrectly fail (<a href="https://github.com/istio/istio/issues/14512">Issue 14512</a>).</li>
|
||
<li>Improve Pilot validation to reject different <code>VirtualServices</code> with the same domain since Envoy will not accept them (<a href="https://github.com/istio/istio/issues/13267">Issue 13267</a>).</li>
|
||
<li>Fix locality load balancing issue where only one replica in a locality would receive traffic (<a href="https://github.com/istio/istio/issues/13994">13994</a>).</li>
|
||
<li>Fix issue where Pilot Agent might not notice a TLS certificate rotation (<a href="https://github.com/istio/istio/issues/14539">Issue 14539</a>).</li>
|
||
<li>Fix a <code>LuaJIT</code> panic in Envoy (<a href="https://github.com/envoyproxy/envoy/pull/6994">Envoy Issue 6994</a>).</li>
|
||
<li>Fix a race condition where Envoy might reuse a HTTP/1.1 connection after the downstream peer had already closed the TCP connection, causing 503 errors and retries (<a href="https://github.com/istio/istio/issues/14037">Issue 14037</a>).</li>
|
||
<li>Fix a tracing issue in Mixer&rsquo;s Zipkin adapter causing missing spans (<a href="https://github.com/istio/istio/issues/13391">Issue 13391</a>).</li>
|
||
</ul>
|
||
<h2 id="small-enhancements">Small enhancements</h2>
|
||
<ul>
|
||
<li>Reduce Pilot log spam by logging the <code>the endpoints within network ... will be ignored for no network configured</code> message at <code>DEBUG</code>.</li>
|
||
<li>Make it easier to rollback by making pilot-agent ignore unknown flags.</li>
|
||
<li>Update Citadel&rsquo;s default root CA certificate TTL from 1 year to 10 years.</li>
|
||
</ul></description><pubDate>Thu, 06 Jun 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.8/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.8/</guid></item><item><title>Security Update - CVE-2019-12243</title><description>
|
||
<p>During review of the <a href="/v1.1/about/notes/1.1.7">Istio 1.1.7</a> release notes, we realized that <a href="https://github.com/istio/istio/issues/13868">issue 13868</a>,
|
||
which is fixed in the release, actually represents a security vulnerability.</p>
|
||
<p>Initially we thought the bug was impacting the <a href="/v1.1/about/feature-stages/#security-and-policy-enforcement">TCP Authorization</a> feature advertised
|
||
as alpha stability, which would not have required invoking this security advisory process, but we later realized that the
|
||
<a href="/v1.1/docs/reference/config/policy-and-telemetry/adapters/denier/">Deny Checker</a> and
|
||
<a href="/v1.1/docs/reference/config/policy-and-telemetry/adapters/list/">List Checker</a> feature were affected and those are considered stable features.
|
||
We are revisiting our processes to flag vulnerabilities that are initially reported as bugs instead of through the
|
||
<a href="/v1.1/about/security-vulnerabilities/">private disclosure process</a>.</p>
|
||
<p>We tracked the bug to a code change introduced in Istio 1.1 and affecting all versions up to 1.1.6.</p>
|
||
<p>This vulnerability is referred to as <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12243">CVE 2019-12243</a></p>
|
||
<h2 id="affected-istio-releases">Affected Istio releases</h2>
|
||
<p>The following Istio releases are vulnerable:</p>
|
||
<ul>
|
||
<li>1.1, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6</li>
|
||
</ul>
|
||
<h2 id="impact-score">Impact score</h2>
|
||
<p>Overall CVSS score: 8.9 <a href="https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:A/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:N/E:H/RL:O/RC:C">AV:A/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:N/E:H/RL:O/RC:C</a></p>
|
||
<h2 id="vulnerability-impact-and-detection">Vulnerability impact and Detection</h2>
|
||
<p>Since Istio 1.1, In the default Istio installation profile, policy enforcement is disabled by default.</p>
|
||
<p>You can check the status of policy enforcement for your mesh with the following command:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl -n istio-system get cm istio -o jsonpath=&#34;{@.data.mesh}&#34; | grep disablePolicyChecks
|
||
disablePolicyChecks: true
|
||
</code></pre>
|
||
<p>You are not impacted by this vulnerability if <code>disablePolicyChecks</code> is set to true.</p>
|
||
<p>You are impacted by the vulnerability issue if the following conditions are all true:</p>
|
||
<ul>
|
||
<li>You are running one of the affected Istio releases.</li>
|
||
<li><code>disablePolicyChecks</code> is set to false (follow the steps mentioned above to check)</li>
|
||
<li>Your workload is NOT using HTTP, HTTP/2, or gRPC protocols</li>
|
||
<li>A mixer adapter (e.g., Deny Checker, List Checker) is used to provide authorization for your backend TCP service.</li>
|
||
</ul>
|
||
<h2 id="mitigation">Mitigation</h2>
|
||
<ul>
|
||
<li>Users of Istio 1.0.x are not affected</li>
|
||
<li>For Istio 1.1.x deployments: update to a minimum version of <a href="/v1.1/about/notes/1.1.7">Istio 1.1.7</a></li>
|
||
</ul>
|
||
<h2 id="credit">Credit</h2>
|
||
<p>The Istio team would like to thank <code>Haim Helman</code> for the original bug report.</p></description><pubDate>Tue, 28 May 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/cve-2019-12243/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/cve-2019-12243/</guid></item><item><title>Support for Istio 1.0 ends on June 19th, 2019</title><description><p>According to Istio&rsquo;s <a href="/v1.1/about/release-cadence/">support policy</a>, LTS releases like 1.0 are supported for three months after the next LTS release. Since <a href="/v1.1/about/notes/1.1/">1.1 was released on March 19th</a>, support for 1.0 will end on June 19th, 2019.</p>
|
||
<p>At that point we will stop back-porting fixes for security issues and critical bugs to 1.0, so we encourage you to upgrade to the latest version of Istio (1.1.9). If you don&rsquo;t do this you may put yourself in the position of having to do a major upgrade on a short timeframe to pick up a critical fix.</p>
|
||
<p>We care about you and your clusters, so please be kind to yourself and upgrade.</p></description><pubDate>Thu, 23 May 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.0-eol/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.0-eol/</guid></item><item><title>Announcing Istio 1.1.7</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.1.7. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.7"
|
||
data-updateadvice='Before you download 1.1.7, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1.7
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1.7 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.6...1.1.7">CHANGES IN 1.1.7</a>
|
||
</div>
|
||
<h2 id="security-update">Security update</h2>
|
||
<p>This release fixes <a href="/v1.1/blog/2019/cve-2019-12243">CVE 2019-12243</a>.</p>
|
||
<h2 id="bug-fixes">Bug fixes</h2>
|
||
<ul>
|
||
<li>Fix issue where two gateways with overlapping hosts, created at the same second, can cause Pilot to fail to generate routes correctly and lead to Envoy listeners stuck indefinitely at startup in a warming state.</li>
|
||
<li>Improve the robustness of the SDS node agent: if Envoy sends a SDS request with an empty <code>ResourceNames</code>, ignore it and wait for the next request instead of closing the connection (<a href="https://github.com/istio/istio/issues/13853">Issue 13853</a>).</li>
|
||
<li>In prior releases Pilot automatically injected the experimental <code>envoy.filters.network.mysql_proxy</code> filter into the outbound filter chain if the service port name is <code>mysql</code>. This was surprising and caused issues for some operators, so Pilot will now automatically inject the <code>envoy.filters.network.mysql_proxy</code> filter only if the <code>PILOT_ENABLE_MYSQL_FILTER</code> environment variable is set to <code>1</code> (<a href="https://github.com/istio/istio/issues/13998">Issue 13998</a>).</li>
|
||
<li>Fix issue where Mixer policy checks were incorrectly disabled for TCP (<a href="https://github.com/istio/istio/issues/13868">Issue 13868</a>).</li>
|
||
</ul>
|
||
<h2 id="small-enhancements">Small enhancements</h2>
|
||
<ul>
|
||
<li>Add <code>--applicationPorts</code> option to the <code>ingressgateway</code> Helm charts. When set to a comma-delimited list of ports, readiness checks will fail until all the ports become active. When configured, traffic will not be sent to Envoys stuck in the warming state.</li>
|
||
<li>Increase memory limit in the <code>ingressgateway</code> Helm chart to 1GB and add resource <code>request</code> and <code>limits</code> to the SDS node agent container to support HPA autoscaling.</li>
|
||
</ul></description><pubDate>Fri, 17 May 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.7/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.7/</guid></item><item><title>Announcing Istio 1.1.6</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.1.6. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.6"
|
||
data-updateadvice='Before you download 1.1.6, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1.6
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1.6 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.5...1.1.6">CHANGES IN 1.1.6</a>
|
||
</div>
|
||
<h2 id="bug-fixes">Bug fixes</h2>
|
||
<ul>
|
||
<li>Fix Galley Helm charts so that the <code>validatingwebhookconfiguration</code> object can now deployed to a namespace other than <code>istio-system</code> (<a href="https://github.com/istio/istio/issues/13625">Issue 13625</a>).</li>
|
||
<li>Additional Helm chart fixes for anti-affinity support: fix <code>gatewaypodAntiAffinityRequiredDuringScheduling</code> and <code>podAntiAffinityLabelSelector</code> match expressions and fix the default value for <code>podAntiAffinityLabelSelector</code> (<a href="https://github.com/istio/istio/issues/13892">Issue 13892</a>).</li>
|
||
<li>Make Pilot handle a condition where Envoy continues to request routes for a deleted gateway while listeners are still draining (<a href="https://github.com/istio/istio/issues/13739">Issue 13739</a>).</li>
|
||
</ul>
|
||
<h2 id="small-enhancements">Small enhancements</h2>
|
||
<ul>
|
||
<li>If access logs are enabled, <code>passthrough</code> listener requests will be logged.</li>
|
||
<li>Make Pilot tolerate unknown JSON fields to make it easier to rollback to older versions during upgrade.</li>
|
||
<li>Add support for fallback secrets to <code>SDS</code> which Envoy can use instead of waiting indefinitely for late or non-existent secrets during startup (<a href="https://github.com/istio/istio/issues/13853">Issue 13853</a>).</li>
|
||
</ul></description><pubDate>Sat, 11 May 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.6/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.6/</guid></item><item><title>Announcing Istio 1.1.5</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.1.5. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.5"
|
||
data-updateadvice='Before you download 1.1.5, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1.5
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1.5 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.4...1.1.5">CHANGES IN 1.1.5</a>
|
||
</div>
|
||
<h2 id="bug-fixes">Bug fixes</h2>
|
||
<ul>
|
||
<li>Add additional validation to Pilot to reject gateway configuration with overlapping hosts matches (<a href="https://github.com/istio/istio/issues/13717">Issue 13717</a>).</li>
|
||
<li>Build against the latest stable version of <code>istio-cni</code> instead of the latest daily build (<a href="https://github.com/istio/istio/issues/13171">Issue 13171</a>).</li>
|
||
</ul>
|
||
<h2 id="small-enhancements">Small enhancements</h2>
|
||
<ul>
|
||
<li>Add additional logging to help diagnose hostname resolution failures (<a href="https://github.com/istio/istio/issues/13581">Issue 13581</a>).</li>
|
||
<li>Improve ease of installing <code>prometheus</code> by removing unnecessary use of <code>busybox</code> image (<a href="https://github.com/istio/istio/issues/13501">Issue 13501</a>).</li>
|
||
<li>Make Pilot Agent&rsquo;s certificate paths configurable (<a href="https://github.com/istio/istio/issues/11984">Issue 11984</a>).</li>
|
||
</ul></description><pubDate>Fri, 03 May 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.5/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.5/</guid></item><item><title>Announcing Istio 1.1.4</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.1.4. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.4"
|
||
data-updateadvice='Before you download 1.1.4, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1.4
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1.4 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.3...1.1.4">CHANGES IN 1.1.4</a>
|
||
</div>
|
||
<h2 id="behavior-change">Behavior change</h2>
|
||
<ul>
|
||
<li>Changed the default behavior for Pilot to allow traffic to outside the mesh, even if it is on the same port as an internal service.
|
||
This behavior can be controlled by the <code>PILOT_ENABLE_FALLTHROUGH_ROUTE</code> environment variable.</li>
|
||
</ul>
|
||
<h2 id="bug-fixes">Bug fixes</h2>
|
||
<ul>
|
||
<li><p>Fixed egress route generation for services of type <code>ExternalName</code>.</p></li>
|
||
<li><p>Added support for configuring Envoy&rsquo;s idle connection timeout, which prevents running out of
|
||
memory or IP ports over time (<a href="https://github.com/istio/istio/issues/13355">Issue 13355</a>).</p></li>
|
||
<li><p>Fixed a crashing bug in Pilot in failover handling of locality-based load balancing.</p></li>
|
||
<li><p>Fixed a crashing bug in Pilot when it was given custom certificate paths.</p></li>
|
||
<li><p>Fixed a bug in Pilot where it was ignoring short names used as service entry hosts (<a href="https://github.com/istio/istio/issues/13436">Issue 13436</a>).</p></li>
|
||
<li><p>Added missing <code>https_protocol_options</code> to the envoy-metrics-service cluster configuration.</p></li>
|
||
<li><p>Fixed a bug in Pilot where it didn&rsquo;t handle https traffic correctly in the fall through route case (<a href="https://github.com/istio/istio/issues/13386">Issue 13386</a>).</p></li>
|
||
<li><p>Fixed a bug where Pilot didn&rsquo;t remove endpoints from Envoy after they were removed from Kubernetes (<a href="https://github.com/istio/istio/issues/13402">Issue 13402</a>).</p></li>
|
||
<li><p>Fixed a crashing bug in the node agent (<a href="https://github.com/istio/istio/issues/13325">Issue 13325</a>).</p></li>
|
||
<li><p>Added missing validation to prevent gateway names from containing dots (<a href="https://github.com/istio/istio/issues/13211">Issue 13211</a>).</p></li>
|
||
<li><p>Fixed bug where <a href="/v1.1/docs/reference/config/networking/v1alpha3/destination-rule/#LoadBalancerSettings-ConsistentHashLB"><code>ConsistentHashLB.minimumRingSize</code></a>
|
||
was defaulting to 0 instead of the documented 1024 (<a href="https://github.com/istio/istio/issues/13261">Issue 13261</a>).</p></li>
|
||
</ul>
|
||
<h2 id="small-enhancements">Small enhancements</h2>
|
||
<ul>
|
||
<li><p>Updated to the latest version of the <a href="https://www.kiali.io">Kiali</a> add-on.</p></li>
|
||
<li><p>Updated to the latest version of <a href="https://grafana.com">Grafana</a>.</p></li>
|
||
<li><p>Added validation to ensure Citadel is only deployed with a single replica (<a href="https://github.com/istio/istio/issues/13383">Issue 13383</a>).</p></li>
|
||
<li><p>Added support to configure the logging level of the proxy and Istio control plane ((<a href="https://github.com/istio/istio/issues/11847">Issue 11847</a>).</p></li>
|
||
<li><p>Allow sidecars to bind to any loopback address and not just 127.0.0.1 (<a href="https://github.com/istio/istio/issues/13201">Issue 13201</a>).</p></li>
|
||
</ul></description><pubDate>Fri, 26 Apr 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.4/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.4/</guid></item><item><title>Announcing Istio 1.1.3</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.1.3. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.3"
|
||
data-updateadvice='Before you download 1.1.3, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1.3
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1.3 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.2...1.1.3">CHANGES IN 1.1.3</a>
|
||
</div>
|
||
<h2 id="known-issues-with-1-1-3">Known issues with 1.1.3</h2>
|
||
<ul>
|
||
<li>A <a href="https://github.com/istio/istio/issues/13325">panic in the Node Agent</a> was discovered late in the 1.1.3 qualification process. The panic only occurs in clusters with the alpha-quality SDS certificate rotation feature enabled. Since this is the first time we have included SDS certificate rotation in our long-running release tests, we don&rsquo;t know whether this is a latent bug or a new regression. Considering SDS certificate rotation is in alpha, we have decided to release 1.1.3 with this issue and target a fix for the 1.1.4 release.</li>
|
||
</ul>
|
||
<h2 id="bug-fixes">Bug fixes</h2>
|
||
<ul>
|
||
<li><p>Istio-specific back-ports of Envoy patches for <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9900"><code>CVE-2019-9900</code></a> and
|
||
<a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9901"><code>CVE-2019-9901</code></a> included in Istio 1.1.2 have been dropped in favor of an
|
||
Envoy update which contains the final version of the patches.</p></li>
|
||
<li><p>Fix load balancer weight setting for split horizon <code>EDS</code>.</p></li>
|
||
<li><p>Fix typo in the default Envoy <code>JSON</code> log format (<a href="https://github.com/istio/istio/issues/12232">Issue 12232</a>).</p></li>
|
||
<li><p>Correctly reload out-of-process adapter address upon configuration change (<a href="https://github.com/istio/istio/issues/12488">Issue 12488</a>).</p></li>
|
||
<li><p>Restore Kiali settings that were accidentally deleted (<a href="https://github.com/istio/istio/issues/3660">Issue 3660</a>).</p></li>
|
||
<li><p>Prevent services with same target port resulting in duplicate inbound listeners (<a href="https://github.com/istio/istio/issues/9504">Issue 9504</a>).</p></li>
|
||
<li><p>Fix issue with configuring <code>Sidecar</code> <code>egress</code> ports for namespaces other than <code>istio-system</code> resulting in a <code>envoy.tcp_proxy</code> filter of <code>BlackHoleCluster</code> by auto binding
|
||
to services for <code>Sidecar</code> listeners (<a href="https://github.com/istio/istio/issues/12536">Issue 12536</a>).</p></li>
|
||
<li><p>Fix gateway <code>vhost</code> configuration generation issue by favoring more specific host matches (<a href="https://github.com/istio/istio/issues/12655">Issue 12655</a>).</p></li>
|
||
<li><p>Fix <code>ALLOW_ANY</code> so it now allows external traffic if there is already an http service present on a port.</p></li>
|
||
<li><p>Fix validation logic so that <code>port.name</code> is no longer a valid <code>PortSelection</code>.</p></li>
|
||
<li><p>Fix <code>istioctl proxy-config clusters</code> cluster type column rendering (<a href="https://github.com/istio/istio/issues/12455">Issue 12455</a>).</p></li>
|
||
<li><p>Fix SDS secret mount configuration.</p></li>
|
||
<li><p>Fix incorrect Istio version in the Helm charts.</p></li>
|
||
<li><p>Fix partial DNS failures in the presence of overlapping ports (<a href="https://github.com/istio/istio/issues/11658">Issue 11658</a>).</p></li>
|
||
<li><p>Fix Helm <code>podAntiAffinity</code> template error (<a href="https://github.com/istio/istio/issues/12790">Issue 12790</a>).</p></li>
|
||
<li><p>Fix bug with the original destination service discovery not using the original destination load balancer.</p></li>
|
||
<li><p>Fix SDS memory leak in the presence of invalid or missing keying materials (<a href="https://github.com/istio/istio/issues/13197">Issue 13197</a>).</p></li>
|
||
</ul>
|
||
<h2 id="small-enhancements">Small enhancements</h2>
|
||
<ul>
|
||
<li><p>Hide <code>ServiceAccounts</code> from <code>PushContext</code> log to reduce log volume.</p></li>
|
||
<li><p>Configure <code>localityLbSetting</code> in <code>values.yaml</code> by passing it through to the mesh configuration.</p></li>
|
||
<li><p>Remove the soon-to-be deprecated <code>critical-pod</code> annotation from Helm charts (<a href="https://github.com/istio/istio/issues/12650">Issue 12650</a>).</p></li>
|
||
<li><p>Support pod anti-affinity annotations to improve control plane availability (<a href="https://github.com/istio/istio/issues/11333">Issue 11333</a>).</p></li>
|
||
<li><p>Pretty print <code>IP</code> addresses in access logs.</p></li>
|
||
<li><p>Remove redundant write header to further reduce log volume.</p></li>
|
||
<li><p>Improve destination host validation in Pilot.</p></li>
|
||
<li><p>Explicitly configure <code>istio-init</code> to run as root so use of pod-level <code>securityContext.runAsUser</code> doesn&rsquo;t break it (<a href="https://github.com/istio/istio/issues/5453">Issue 5453</a>).</p></li>
|
||
<li><p>Add configuration samples for Vault integration.</p></li>
|
||
<li><p>Respect locality load balancing weight settings from <code>ServiceEntry</code>.</p></li>
|
||
<li><p>Make the TLS certificate location watched by Pilot Agent configurable (<a href="https://github.com/istio/istio/issues/11984">Issue 11984</a>).</p></li>
|
||
<li><p>Add support for Datadog tracing.</p></li>
|
||
<li><p>Add alias to <code>istioctl</code> so &lsquo;x&rsquo; can be used instead of &lsquo;experimental&rsquo;.</p></li>
|
||
<li><p>Provide improved distribution of sidecar certificate by adding jitter to their CSR requests.</p></li>
|
||
<li><p>Allow weighted load balancing registry locality to be configured.</p></li>
|
||
<li><p>Add support for standard CRDs for compiled-in Mixer adapters.</p></li>
|
||
<li><p>Reduce Pilot resource requirements for demo configuration.</p></li>
|
||
<li><p>Fully populate Galley dashboard by adding data source (<a href="https://github.com/istio/istio/issues/13040">Issue 13040</a>).</p></li>
|
||
<li><p>Propagate 1.1.0 <code>sidecar</code> performance tuning to the <code>istio-gateway</code>.</p></li>
|
||
<li><p>Improve destination host validation by rejecting <code>*</code> hosts (<a href="https://github.com/istio/istio/issues/12794">Issue 12794</a>).</p></li>
|
||
<li><p>Expose upstream <code>idle_timeout</code> in cluster definition so dead connections can sometimes be removed from connection pools before they are used
|
||
(<a href="https://github.com/istio/istio/issues/9113">Issue 9113</a>).</p></li>
|
||
<li><p>When registering a <code>Sidecar</code> resource to restrict what a pod can see, the restrictions are now applied if the spec contains a
|
||
<code>workloadSelector</code> (<a href="https://github.com/istio/istio/issues/11818">Issue 11818</a>).</p></li>
|
||
<li><p>Update the Bookinfo example to use port 80 for TLS origination.</p></li>
|
||
<li><p>Add liveness probe for Citadel.</p></li>
|
||
<li><p>Improve AWS ELB interoperability by making 15020 the first port listed in the <code>ingressgateway</code> service (<a href="https://github.com/istio/istio/issues/12503">Issue 12502</a>).</p></li>
|
||
<li><p>Use outlier detection for failover mode but not for distribute mode for locality weighted load balancing (<a href="https://github.com/istio/istio/issues/12961">Issues 12965</a>).</p></li>
|
||
<li><p>Replace generation of Envoy&rsquo;s deprecated <code>enabled</code> field in <code>CorsPolicy</code> with the replacement <code>filter_enabled</code> field for 1.1.0+ sidecars only.</p></li>
|
||
<li><p>Standardize labels on Mixer&rsquo;s Helm charts.</p></li>
|
||
</ul></description><pubDate>Mon, 15 Apr 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.3/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.3/</guid></item><item><title>Announcing Istio 1.1.2 with Important Security Update</title><description><p>We&rsquo;re announcing immediate availability of Istio 1.1.2 which contains some important security updates. Please see below for details.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.2"
|
||
data-updateadvice='Before you download 1.1.2, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1.2
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1.2 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.1...1.1.2">CHANGES IN 1.1.2</a>
|
||
</div>
|
||
<h2 id="security-update">Security update</h2>
|
||
<p>Two security vulnerabilities have recently been identified in the Envoy proxy
|
||
(<a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9900">CVE 2019-9900</a> and <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9901">CVE 2019-9901</a>). The
|
||
vulnerabilities have now been patched in Envoy version 1.9.1, and correspondingly in the Envoy builds
|
||
embedded in Istio 1.1.2 and Istio 1.0.7. Since Envoy is an integral part of Istio, users are advised to update Istio
|
||
immediately to mitigate security risks arising from these vulnerabilities.</p>
|
||
<p>The vulnerabilities are centered on the fact that Envoy did not normalize HTTP URI paths and did not fully validate HTTP/1.1 header values. These
|
||
vulnerabilities impact Istio features that rely on Envoy to enforce any of authorization, routing, or rate limiting.</p>
|
||
<h2 id="affected-istio-releases">Affected Istio releases</h2>
|
||
<p>The following Istio releases are vulnerable:</p>
|
||
<ul>
|
||
<li><p>1.1, 1.1.1</p>
|
||
<ul>
|
||
<li>These releases can be patched to Istio 1.1.2.</li>
|
||
<li>1.1.2 is built from the same source as 1.1.1 with the addition of Envoy patches minimally sufficient to address the CVEs.</li>
|
||
</ul></li>
|
||
<li><p>1.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6</p>
|
||
<ul>
|
||
<li>These releases can be patched to Istio 1.0.7</li>
|
||
<li>1.0.7 is built from the same source as 1.0.6 with the addition of Envoy patches minimally sufficient to address the CVEs.</li>
|
||
</ul></li>
|
||
<li><p>0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8</p>
|
||
<ul>
|
||
<li>These releases are no longer supported and will not be patched. Please upgrade to a supported release with the necessary fixes.</li>
|
||
</ul></li>
|
||
</ul>
|
||
<h2 id="vulnerability-impact">Vulnerability impact</h2>
|
||
<p><a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9900">CVE 2019-9900</a> and <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9901">CVE 2019-9901</a>
|
||
allow remote attackers access to unauthorized resources by using specially crafted request URI paths (9901) and NUL bytes in
|
||
HTTP/1.1 headers (9900), potentially circumventing DoS prevention systems such as rate limiting, or routing to a unexposed upstream system. Refer to
|
||
<a href="https://github.com/envoyproxy/envoy/issues/6434">issue 6434</a>
|
||
and <a href="https://github.com/envoyproxy/envoy/issues/6435">issue 6435</a> for more information.</p>
|
||
<p>As Istio is based on Envoy, Istio customers can be affected by these vulnerabilities based on whether paths and request headers are used within Istio
|
||
policies or routing rules and how the backend HTTP implementation resolves them. If prefix path matching rules are used by Mixer or by Istio authorization
|
||
policies or the routing rules, an attacker could exploit these vulnerabilities to gain access to unauthorized paths on certain HTTP backends.</p>
|
||
<h2 id="mitigation">Mitigation</h2>
|
||
<p>Eliminating the vulnerabilities requires updating to a corrected version of Envoy. We’ve incorporated the necessary updates in the latest Istio patch releases.</p>
|
||
<p>For Istio 1.1.x deployments: update to a minimum of <a href="/v1.1/about/notes/1.1.2">Istio 1.1.2</a></p>
|
||
<p>For Istio 1.0.x deployments: update to a minimum of <a href="/v1.1/about/notes/1.0.7">Istio 1.0.7</a></p>
|
||
<p>While Envoy 1.9.1 requires opting in to path normalization to address CVE 2019-9901, the version of Envoy embedded in Istio 1.1.2 and 1.0.7 enables path
|
||
normalization by default.</p>
|
||
<h2 id="detection-of-nul-header-exploit">Detection of NUL header exploit</h2>
|
||
<p>Based on current information, this only affects HTTP/1.1 traffic. If this is not structurally possible in your network or configuration, then it is unlikely
|
||
that this vulnerability applies.</p>
|
||
<p>File-based access logging uses the <code>c_str()</code> representation for header values, as does gRPC access logging, so there will be no trivial detection via
|
||
Envoy’s access logs by scanning for NUL. Instead, operators might look for inconsistencies in logs between the routing that Envoy performs and the logic
|
||
intended in the <code>RouteConfiguration</code>.</p>
|
||
<p>External authorization and rate limit services can check for NULs in headers. Backend servers might have sufficient logging to detect NULs or unintended
|
||
access; it’s likely that many will simply reject NULs in this scenario via 400 Bad Request, as per RFC 7230.</p>
|
||
<h2 id="detection-of-path-traversal-exploit">Detection of path traversal exploit</h2>
|
||
<p>Envoy’s access logs (whether file-based or gRPC) will contain the unnormalized path, so it is possible to examine these logs to detect suspicious patterns and
|
||
requests that are incongruous with the intended operator configuration intent. In addition, unnormalized paths are available at <code>ext_authz</code>, rate limiting
|
||
and backend servers for log inspection.</p></description><pubDate>Fri, 05 Apr 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.2/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.2/</guid></item><item><title>Announcing Istio 1.0.7 with Important Security Update</title><description><p>We&rsquo;re announcing immediate availability of Istio 1.0.7 which contains some important security updates. Please see below for details.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.0.7"
|
||
data-updateadvice='Before you download 1.0.7, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.0.8'
|
||
data-updatehref="/v1.1/about/notes/1.0.8">
|
||
DOWNLOAD 1.0.7
|
||
</button>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0.7 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.0.6...1.0.7">CHANGES IN 1.0.7</a>
|
||
</div>
|
||
<h2 id="security-update">Security update</h2>
|
||
<p>Two security vulnerabilities have recently been identified in the Envoy proxy
|
||
(<a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9900">CVE 2019-9900</a> and <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9901">CVE 2019-9901</a>). The
|
||
vulnerabilities have now been patched in Envoy version 1.9.1, and correspondingly in the Envoy builds
|
||
embedded in Istio 1.1.2 and Istio 1.0.7. Since Envoy is an integral part of Istio, users are advised to update Istio
|
||
immediately to mitigate security risks arising from these vulnerabilities.</p>
|
||
<p>The vulnerabilities are centered on the fact that Envoy did not normalize HTTP URI paths and did not fully validate HTTP/1.1 header values. These
|
||
vulnerabilities impact Istio features that rely on Envoy to enforce any of authorization, routing, or rate limiting.</p>
|
||
<h2 id="affected-istio-releases">Affected Istio releases</h2>
|
||
<p>The following Istio releases are vulnerable:</p>
|
||
<ul>
|
||
<li><p>1.1, 1.1.1</p>
|
||
<ul>
|
||
<li>These releases can be patched to Istio 1.1.2.</li>
|
||
<li>1.1.2 is built from the same source as 1.1.1 with the addition of Envoy patches minimally sufficient to address the CVEs.</li>
|
||
</ul></li>
|
||
<li><p>1.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6</p>
|
||
<ul>
|
||
<li>These releases can be patched to Istio 1.0.7</li>
|
||
<li>1.0.7 is built from the same source as 1.0.6 with the addition of Envoy patches minimally sufficient to address the CVEs.</li>
|
||
</ul></li>
|
||
<li><p>0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8</p>
|
||
<ul>
|
||
<li>These releases are no longer supported and will not be patched. Please upgrade to a supported release with the necessary fixes.</li>
|
||
</ul></li>
|
||
</ul>
|
||
<h2 id="vulnerability-impact">Vulnerability impact</h2>
|
||
<p><a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9900">CVE 2019-9900</a> and <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9901">CVE 2019-9901</a>
|
||
allow remote attackers access to unauthorized resources by using specially crafted request URI paths (9901) and NUL bytes in
|
||
HTTP/1.1 headers (9900), potentially circumventing DoS prevention systems such as rate limiting, or routing to a unexposed upstream system. Refer to
|
||
<a href="https://github.com/envoyproxy/envoy/issues/6434">issue 6434</a>
|
||
and <a href="https://github.com/envoyproxy/envoy/issues/6435">issue 6435</a> for more information.</p>
|
||
<p>As Istio is based on Envoy, Istio customers can be affected by these vulnerabilities based on whether paths and request headers are used within Istio
|
||
policies or routing rules and how the backend HTTP implementation resolves them. If prefix path matching rules are used by Mixer or by Istio authorization
|
||
policies or the routing rules, an attacker could exploit these vulnerabilities to gain access to unauthorized paths on certain HTTP backends.</p>
|
||
<h2 id="mitigation">Mitigation</h2>
|
||
<p>Eliminating the vulnerabilities requires updating to a corrected version of Envoy. We’ve incorporated the necessary updates in the latest Istio patch releases.</p>
|
||
<p>For Istio 1.1.x deployments: update to a minimum of <a href="/v1.1/about/notes/1.1.2">Istio 1.1.2</a></p>
|
||
<p>For Istio 1.0.x deployments: update to a minimum of <a href="/v1.1/about/notes/1.0.7">Istio 1.0.7</a></p>
|
||
<p>While Envoy 1.9.1 requires opting in to path normalization to address CVE 2019-9901, the version of Envoy embedded in Istio 1.1.2 and 1.0.7 enables path
|
||
normalization by default.</p>
|
||
<h2 id="detection-of-nul-header-exploit">Detection of NUL header exploit</h2>
|
||
<p>Based on current information, this only affects HTTP/1.1 traffic. If this is not structurally possible in your network or configuration, then it is unlikely
|
||
that this vulnerability applies.</p>
|
||
<p>File-based access logging uses the <code>c_str()</code> representation for header values, as does gRPC access logging, so there will be no trivial detection via
|
||
Envoy’s access logs by scanning for NUL. Instead, operators might look for inconsistencies in logs between the routing that Envoy performs and the logic
|
||
intended in the <code>RouteConfiguration</code>.</p>
|
||
<p>External authorization and rate limit services can check for NULs in headers. Backend servers might have sufficient logging to detect NULs or unintended
|
||
access; it’s likely that many will simply reject NULs in this scenario via 400 Bad Request, as per RFC 7230.</p>
|
||
<h2 id="detection-of-path-traversal-exploit">Detection of path traversal exploit</h2>
|
||
<p>Envoy’s access logs (whether file-based or gRPC) will contain the unnormalized path, so it is possible to examine these logs to detect suspicious patterns and
|
||
requests that are incongruous with the intended operator configuration intent. In addition, unnormalized paths are available at <code>ext_authz</code>, rate limiting
|
||
and backend servers for log inspection.</p></description><pubDate>Fri, 05 Apr 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.0.7/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.0.7/</guid></item><item><title>Announcing Istio 1.1.1</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.1.1. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.1"
|
||
data-updateadvice='Before you download 1.1.1, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1.1
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1.1 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.1.0...1.1.1">CHANGES IN 1.1.1</a>
|
||
</div>
|
||
<h2 id="bug-fixes-and-minor-enhancements">Bug fixes and minor enhancements</h2>
|
||
<ul>
|
||
<li>Configure Prometheus to monitor Citadel (<a href="https://github.com/istio/istio/pull/12175">Issue 12175</a>)</li>
|
||
<li>Improve output of <a href="/v1.1/docs/reference/commands/istioctl/#istioctl-experimental-verify-install"><code>istioctl experimental verify-install</code></a> command (<a href="https://github.com/istio/istio/pull/12174">Issue 12174</a>)</li>
|
||
<li>Reduce log level for missing service account messages for a SPIFFE URI (<a href="https://github.com/istio/istio/issues/12108">Issue 12108</a>)</li>
|
||
<li>Fix broken path on the opt-in SDS feature&rsquo;s Unix domain socket (<a href="https://github.com/istio/istio/pull/12688">Issue 12688</a>)</li>
|
||
<li>Fix Envoy tracing that was preventing a child span from being created if the parent span was propagated with an empty string (<a href="https://github.com/envoyproxy/envoy/pull/6263">Envoy Issue 6263</a>)</li>
|
||
<li>Add namespace scoping to the Gateway &lsquo;port&rsquo; names. This fixes two issues:
|
||
<ul>
|
||
<li><code>IngressGateway</code> only respects first port 443 Gateway definition (<a href="https://github.com/istio/istio/issues/11509">Issue 11509</a>)</li>
|
||
<li>Istio <code>IngressGateway</code> routing broken with two different gateways with same port name (SDS) (<a href="https://github.com/istio/istio/issues/12500">Issue 12500</a>)</li>
|
||
</ul></li>
|
||
<li>Five bug fixes for locality weighted load balancing:
|
||
<ul>
|
||
<li>Fix bug causing empty endpoints per locality (<a href="https://github.com/istio/istio/issues/12610">Issue 12610</a>)</li>
|
||
<li>Apply locality weighted load balancing configuration correctly (<a href="https://github.com/istio/istio/issues/12587">Issue 12587</a>)</li>
|
||
<li>Locality label <code>istio-locality</code> in Kubernetes should not contain <code>/</code>, use <code>.</code> (<a href="https://github.com/istio/istio/issues/12582">Issue 12582</a>)</li>
|
||
<li>Fix crash in locality load balancing (<a href="https://github.com/istio/istio/pull/12649">Issue 12649</a>)</li>
|
||
<li>Fix bug in locality load balancing normalization (<a href="https://github.com/istio/istio/pull/12579">Issue 12579</a>)</li>
|
||
</ul></li>
|
||
<li>Propagate Envoy Metrics Service configuration (<a href="https://github.com/istio/istio/issues/12569">Issue 12569</a>)</li>
|
||
<li>Do not apply <code>VirtualService</code> rule to the wrong gateway (<a href="https://github.com/istio/istio/issues/10313">Issue 10313</a>)</li>
|
||
</ul></description><pubDate>Mon, 25 Mar 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1.1/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1.1/</guid></item><item><title>Architecting Istio 1.1 for Performance</title><description>
|
||
<p>Hyper-scale, microservice-based cloud environments have been exciting to build but challenging to manage. Along came Kubernetes (container orchestration) in 2014, followed by Istio (container service management) in 2017. Both open-source projects enable developers to scale container-based applications without spending too much time on administration tasks.</p>
|
||
<p>Now, new enhancements in Istio 1.1 deliver scale-up with improved application performance and service management efficiency.
|
||
Simulations using our sample commercial airline reservation application show the following improvements, compared to Istio 1.0.</p>
|
||
<p>We&rsquo;ve seen substantial application performance gains:</p>
|
||
<ul>
|
||
<li>up to 30% reduction in application average latency</li>
|
||
<li>up to 40% faster service startup times in a large mesh</li>
|
||
</ul>
|
||
<p>As well as impressive improvements in service management efficiency:</p>
|
||
<ul>
|
||
<li>up to 90% reduction in Pilot CPU usage in a large mesh</li>
|
||
<li>up to 50% reduction in Pilot memory usage in a large mesh</li>
|
||
</ul>
|
||
<p>With Istio 1.1, organizations can be more confident in their ability to scale applications with consistency and control &ndash; even in hyper-scale cloud environments.</p>
|
||
<p>Congratulations to the Istio experts around the world who contributed to this release. We could not be more pleased with these results.</p>
|
||
<h2 id="istio-1-1-performance-enhancements">Istio 1.1 performance enhancements</h2>
|
||
<p>As members of the Istio Performance and Scalability workgroup, we have done extensive performance evaluations. We introduced many performance design features for Istio 1.1, in collaboration with other Istio contributors.
|
||
Some of the most visible performance enhancements in 1.1 include:</p>
|
||
<ul>
|
||
<li>Significant reduction in default collection of Envoy-generated statistics</li>
|
||
<li>Added load-shedding functionality to Mixer workloads</li>
|
||
<li>Improved the protocol between Envoy and Mixer</li>
|
||
<li>Namespace isolation, to reduce operational overhead</li>
|
||
<li>Configurable concurrent worker threads, which can improve overall throughput</li>
|
||
<li>Configurable filters that limit telemetry data</li>
|
||
<li>Removal of synchronization bottlenecks</li>
|
||
</ul>
|
||
<h2 id="continuous-code-quality-and-performance-verification">Continuous code quality and performance verification</h2>
|
||
<p>Regression Patrol drives continuous improvement in Istio performance and quality. Behind the scenes, the Regression Patrol helps Istio developers to identify and fix code issues. Daily builds are checked using a customer-centric benchmark, <a href="https://github.com/blueperf/">BluePerf</a>. The results are published to the <a href="https://ibmcloud-perf.istio.io/regpatrol/">Istio community web portal</a>. Various application configurations are evaluated to help provide insights on Istio component performance.</p>
|
||
<p>Another tool that is used to evaluate the performance of Istio’s builds is <a href="https://fortio.org/">Fortio</a>, which provides a synthetic end to end load testing benchmark.</p>
|
||
<h2 id="summary">Summary</h2>
|
||
<p>Istio 1.1 was designed for performance and scalability. The Istio Performance and Scalability workgroup measured significant performance improvements over 1.0.
|
||
Istio 1.1 introduces new features and optimizations to help harden the service mesh for enterprise microservice workloads. The Istio 1.1 Performance and Tuning Guide documents performance simulations, provides sizing and capacity planning guidance, and includes best practices for tuning custom use cases.</p>
|
||
<h2 id="useful-links">Useful links</h2>
|
||
<ul>
|
||
<li><a href="https://www.youtube.com/watch?time_continue=349&amp;v=G4F5aRFEXnU">Istio Service Mesh Performance (34:30)</a>, by Surya Duggirala, Laurent Demailly and Fawad Khaliq at Kubecon Europe 2018</li>
|
||
<li><a href="https://discuss.istio.io/c/performance-and-scalability">Istio Performance and Scalability discussion forum</a></li>
|
||
</ul>
|
||
<h2 id="disclaimer">Disclaimer</h2>
|
||
<p>The performance data contained herein was obtained in a controlled, isolated environment. Actual results that may be obtained in other operating environments may vary significantly. There is no guarantee that the same or similar results will be obtained elsewhere.</p></description><pubDate>Tue, 19 Mar 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/istio1.1_perf/</link><author>Surya V Duggirala (IBM), Mandar Jog (Google), Jose Nativio (IBM)</author><guid isPermaLink="true">/v1.1/blog/2019/istio1.1_perf/</guid><category>performance</category><category>scalability</category><category>scale</category><category>benchmarks</category></item><item><title>Announcing Istio 1.1</title><description><p>We are pleased to announce the release of Istio 1.1!</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.1.0"
|
||
data-updateadvice='Before you download 1.1, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.1.9'
|
||
data-updatehref="/v1.1/about/notes/1.1.9">
|
||
DOWNLOAD 1.1
|
||
</button>
|
||
<a class="btn" href="https://istio.io/docs">1.1 DOCS</a>
|
||
<a class="btn" href="/v1.1/about/notes/1.1/">1.1 RELEASE NOTES</a>
|
||
</div>
|
||
<p>Since we released 1.0 back in July, we’ve done a lot of work to help people get
|
||
into production. Not surprisingly, we had to do some <a href="/v1.1/about/notes">patch releases</a>
|
||
(6 so far!), but we’ve also been hard at work adding new features to the
|
||
product.</p>
|
||
<p>The theme for 1.1 is Enterprise Ready. We’ve been very pleased to see more and
|
||
more companies using Istio in production, but as some larger companies tried to
|
||
adopt Istio they hit some limits.</p>
|
||
<p>One of our prime areas of focus has been <a href="/v1.1/docs/concepts/performance-and-scalability/">performance and scalability</a>.
|
||
As people moved into production with larger clusters running more services at
|
||
higher volume, they hit some scaling and performance issues. The
|
||
<a href="/v1.1/docs/concepts/traffic-management/#sidecars">sidecars</a> took too many resources
|
||
and added too much latency. The control plane (especially
|
||
<a href="/v1.1/docs/concepts/traffic-management/#pilot-and-envoy">Pilot</a>) was overly
|
||
resource hungry.</p>
|
||
<p>We’ve done a lot of work to make both the data plane and the control plane more
|
||
efficient. You can find the details of our 1.1 performance testing and the
|
||
results in our updated <a href="/v1.1/docs/concepts/performance-and-scalability/">performance ans scalability concept</a>.</p>
|
||
<p>We’ve done work around namespace isolation as well. This lets you use
|
||
Kubernetes namespaces to enforce boundaries of control, and ensures that your
|
||
teams cannot interfere with each other.</p>
|
||
<p>We have also improved the <a href="/v1.1/docs/concepts/multicluster-deployments/">multicluster capabilities and usability</a>.
|
||
We listened to the community and improved defaults for traffic control and
|
||
policy. We introduced a new component called
|
||
<a href="/v1.1/docs/concepts/what-is-istio/#galley">Galley</a>. Galley validates that sweet,
|
||
sweet YAML, reducing the chance of configuration errors. Galley will also be
|
||
instrumental in <a href="/v1.1/docs/setup/kubernetes/install/multicluster/">multicluster setups</a>,
|
||
gathering service discovery information from each Kubernetes cluster. We are
|
||
also supporting additional multicluster topologies including <a href="/v1.1/docs/concepts/multicluster-deployments/#single-control-plane-topology">single control plane</a>
|
||
and <a href="/v1.1/docs/concepts/multicluster-deployments/#multiple-control-plane-topology">multiple synchronized control planes</a>
|
||
without requiring a flat network.</p>
|
||
<p>There is lots more &ndash; see the <a href="/v1.1/about/notes/1.1/">release notes</a> for complete
|
||
details.</p>
|
||
<p>There is more going on in the project as well. We know that Istio has a lot of
|
||
moving parts and can be a lot to take on. To help address that, we recently
|
||
formed a <a href="https://github.com/istio/community/blob/master/WORKING-GROUPS.md#working-group-meetings">Usability Working Group</a>
|
||
(feel free to join). There is also a lot happening in the <a href="https://github.com/istio/community#community-meeting">Community
|
||
Meeting</a> (Thursdays at
|
||
<code>11 a.m.</code>) and in the <a href="https://github.com/istio/community/blob/master/WORKING-GROUPS.md">Working
|
||
Groups</a>. And
|
||
if you haven’t yet joined the conversation at
|
||
<a href="https://discuss.istio.io">discuss.istio.io</a>, head over, log in with your
|
||
GitHub credentials and join us!</p>
|
||
<p>We are grateful to everyone who has worked hard on Istio over the last few
|
||
months &ndash; patching 1.0, adding features to 1.1, and, lately, doing tons of
|
||
testing on 1.1. Thanks especially to those companies and users who worked with
|
||
us installing and upgrading to the early builds and helping us catch problems
|
||
before the release.</p>
|
||
<p>So: now’s the time! Grab 1.1, check out <a href="/v1.1/docs/">the updated documentation</a>,
|
||
<a href="/v1.1/docs/setup/kubernetes/">install it</a> and&hellip;happy meshing!</p></description><pubDate>Tue, 19 Mar 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.1/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.1/</guid></item><item><title>Announcing Istio 1.0.6</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.0.6. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.0.6"
|
||
data-updateadvice='Before you download 1.0.6, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.0.8'
|
||
data-updatehref="/v1.1/about/notes/1.0.8">
|
||
DOWNLOAD 1.0.6
|
||
</button>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0.6 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.0.5...1.0.6">CHANGES IN 1.0.6</a>
|
||
</div>
|
||
<h2 id="security-vulnerability-fixes">Security vulnerability fixes</h2>
|
||
<ul>
|
||
<li>Updated Go <code>requests</code> and <code>urllib3</code> libraries in Bookinfo sample code per <a href="https://nvd.nist.gov/vuln/detail/CVE-2018-18074"><code>CVE-2018-18074</code></a> and <a href="https://nvd.nist.gov/vuln/detail/CVE-2018-20060"><code>CVE-2018-20060</code></a>.</li>
|
||
<li>Fixed username and password being exposed in <code>Grafana</code> and <code>Kiali</code> (<a href="https://github.com/istio/istio/issues/7476">Issue 7446</a>, <a href="https://github.com/istio/istio/issues/7447">Issue 7447</a>). If you have trouble to start the <code>Grafana</code> pod after upgrade to 1.0.6, please follow <a href="https://github.com/istio/istio/tree/release-1.1/install/kubernetes/helm/istio#installing-the-chart">the steps</a> to create the secrete first.</li>
|
||
<li>Removed in-memory service registry in Pilot. This allowed adding endpoints to proxy configurations from within the cluster through a Pilot debug API.</li>
|
||
</ul>
|
||
<h2 id="robustness-improvements">Robustness improvements</h2>
|
||
<ul>
|
||
<li>Fixed Pilot failing to push configuration under load (<a href="https://github.com/istio/istio/issues/10360">Issue 10360</a>).</li>
|
||
<li>Fixed a race condition that would lead Pilot to crash and restart (<a href="https://github.com/istio/istio/issues/10868">Issue 10868</a>).</li>
|
||
<li>Fixed a memory leak in Pilot (<a href="https://github.com/istio/istio/issues/10822">Issue 10822</a>).</li>
|
||
<li>Fixed a memory leak in Mixer (<a href="https://github.com/istio/istio/issues/10393">Issue 10393</a>).</li>
|
||
</ul></description><pubDate>Tue, 12 Feb 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-1.0.6/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-1.0.6/</guid></item><item><title>Version Routing in a Multicluster Service Mesh</title><description>
|
||
<p>If you&rsquo;ve spent any time looking at Istio, you&rsquo;ve probably noticed that it includes a lot of features that
|
||
can be demonstrated with simple <a href="/v1.1/docs/tasks/">tasks</a> and <a href="/v1.1/docs/examples/">examples</a>
|
||
running on a single Kubernetes cluster.
|
||
Because most, if not all, real-world cloud and microservices-based applications are not that simple
|
||
and will need to have the services distributed and running in more than one location, you may be
|
||
wondering if all these things will be just as simple in your real production environment.</p>
|
||
<p>Fortunately, Istio provides several ways to configure a service mesh so that applications
|
||
can, more-or-less transparently, be part of a mesh where the services are running
|
||
in more than one cluster, i.e., in a
|
||
<a href="/v1.1/docs/concepts/multicluster-deployments/#multicluster-service-mesh">multicluster service mesh</a>.
|
||
The simplest way to setup a multicluster mesh, because it has no special networking requirements,
|
||
is using a
|
||
<a href="/v1.1/docs/concepts/multicluster-deployments/#multiple-control-plane-topology">multiple control plane topology</a>.
|
||
In this configuration, each Kubernetes cluster contributing to the mesh has its own control plane,
|
||
but each control plane is synchronized and running under a single administrative control.</p>
|
||
<p>In this article we&rsquo;ll look at how one of the features of Istio,
|
||
<a href="/v1.1/docs/concepts/traffic-management/">traffic management</a>, works in a multicluster mesh with
|
||
a multiple control plane topology.
|
||
We&rsquo;ll show how to configure Istio route rules to call remote services in a multicluster service mesh
|
||
by deploying the <a href="https://github.com/istio/istio/tree/release-1.1/samples/bookinfo">Bookinfo sample</a> with version <code>v1</code> of the <code>reviews</code> service
|
||
running in one cluster, versions <code>v2</code> and <code>v3</code> running in a second cluster.</p>
|
||
<h2 id="setup-clusters">Setup clusters</h2>
|
||
<p>To start, you&rsquo;ll need two Kubernetes clusters, both running a slightly customized configuration of Istio.</p>
|
||
<ul>
|
||
<li><p>Set up a multicluster environment with two Istio clusters by following the
|
||
<a href="/v1.1/docs/setup/kubernetes/install/multicluster/gateways/">multiple control planes with gateways</a> instructions.</p></li>
|
||
<li><p>The <code>kubectl</code> command is used to access both clusters with the <code>--context</code> flag.
|
||
Use the following command to list your contexts:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl config get-contexts
|
||
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
|
||
* cluster1 cluster1 user@foo.com default
|
||
cluster2 cluster2 user@foo.com default
|
||
</code></pre></li>
|
||
<li><p>Export the following environment variables with the context names of your configuration:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export CTX_CLUSTER1=&lt;cluster1 context name&gt;
|
||
$ export CTX_CLUSTER2=&lt;cluster2 context name&gt;
|
||
</code></pre></li>
|
||
</ul>
|
||
<h2 id="deploy-version-v1-of-the-bookinfo-application-in-cluster1">Deploy version v1 of the <code>bookinfo</code> application in <code>cluster1</code></h2>
|
||
<p>Run the <code>productpage</code> and <code>details</code> services and version <code>v1</code> of the <code>reviews</code> service in <code>cluster1</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl label --context=$CTX_CLUSTER1 namespace default istio-injection=enabled
|
||
$ kubectl apply --context=$CTX_CLUSTER1 -f - &lt;&lt;EOF
|
||
apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: productpage
|
||
labels:
|
||
app: productpage
|
||
spec:
|
||
ports:
|
||
- port: 9080
|
||
name: http
|
||
selector:
|
||
app: productpage
|
||
---
|
||
apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: productpage-v1
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
app: productpage
|
||
version: v1
|
||
spec:
|
||
containers:
|
||
- name: productpage
|
||
image: istio/examples-bookinfo-productpage-v1:1.10.0
|
||
imagePullPolicy: IfNotPresent
|
||
ports:
|
||
- containerPort: 9080
|
||
---
|
||
apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: details
|
||
labels:
|
||
app: details
|
||
spec:
|
||
ports:
|
||
- port: 9080
|
||
name: http
|
||
selector:
|
||
app: details
|
||
---
|
||
apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: details-v1
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
app: details
|
||
version: v1
|
||
spec:
|
||
containers:
|
||
- name: details
|
||
image: istio/examples-bookinfo-details-v1:1.10.0
|
||
imagePullPolicy: IfNotPresent
|
||
ports:
|
||
- containerPort: 9080
|
||
---
|
||
apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: reviews
|
||
labels:
|
||
app: reviews
|
||
spec:
|
||
ports:
|
||
- port: 9080
|
||
name: http
|
||
selector:
|
||
app: reviews
|
||
---
|
||
apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: reviews-v1
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
app: reviews
|
||
version: v1
|
||
spec:
|
||
containers:
|
||
- name: reviews
|
||
image: istio/examples-bookinfo-reviews-v1:1.10.0
|
||
imagePullPolicy: IfNotPresent
|
||
ports:
|
||
- containerPort: 9080
|
||
EOF
|
||
</code></pre>
|
||
<h2 id="deploy-bookinfo-v2-and-v3-services-in-cluster2">Deploy <code>bookinfo</code> v2 and v3 services in <code>cluster2</code></h2>
|
||
<p>Run the <code>ratings</code> service and version <code>v2</code> and <code>v3</code> of the <code>reviews</code> service in <code>cluster2</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl label --context=$CTX_CLUSTER2 namespace default istio-injection=enabled
|
||
$ kubectl apply --context=$CTX_CLUSTER2 -f - &lt;&lt;EOF
|
||
apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: ratings
|
||
labels:
|
||
app: ratings
|
||
spec:
|
||
ports:
|
||
- port: 9080
|
||
name: http
|
||
selector:
|
||
app: ratings
|
||
---
|
||
apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: ratings-v1
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
app: ratings
|
||
version: v1
|
||
spec:
|
||
containers:
|
||
- name: ratings
|
||
image: istio/examples-bookinfo-ratings-v1:1.10.0
|
||
imagePullPolicy: IfNotPresent
|
||
ports:
|
||
- containerPort: 9080
|
||
---
|
||
apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: reviews
|
||
labels:
|
||
app: reviews
|
||
spec:
|
||
ports:
|
||
- port: 9080
|
||
name: http
|
||
selector:
|
||
app: reviews
|
||
---
|
||
apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: reviews-v2
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
app: reviews
|
||
version: v2
|
||
spec:
|
||
containers:
|
||
- name: reviews
|
||
image: istio/examples-bookinfo-reviews-v2:1.10.0
|
||
imagePullPolicy: IfNotPresent
|
||
ports:
|
||
- containerPort: 9080
|
||
---
|
||
apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: reviews-v3
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
app: reviews
|
||
version: v3
|
||
spec:
|
||
containers:
|
||
- name: reviews
|
||
image: istio/examples-bookinfo-reviews-v3:1.10.0
|
||
imagePullPolicy: IfNotPresent
|
||
ports:
|
||
- containerPort: 9080
|
||
EOF
|
||
</code></pre>
|
||
<h2 id="access-the-bookinfo-application">Access the <code>bookinfo</code> application</h2>
|
||
<p>Just like any application, we&rsquo;ll use an Istio gateway to access the <code>bookinfo</code> application.</p>
|
||
<ul>
|
||
<li><p>Create the <code>bookinfo</code> gateway in <code>cluster1</code>:</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/bookinfo-gateway.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply --context=$CTX_CLUSTER1 -f @samples/bookinfo/networking/bookinfo-gateway.yaml@
|
||
</code></pre></div></li>
|
||
<li><p>Follow the <a href="/v1.1/docs/examples/bookinfo/#determining-the-ingress-ip-and-port">Bookinfo sample instructions</a>
|
||
to determine the ingress IP and port and then point your browser to <code>http://$GATEWAY_URL/productpage</code>.</p></li>
|
||
</ul>
|
||
<p>You should see the <code>productpage</code> with reviews, but without ratings, because only <code>v1</code> of the <code>reviews</code> service
|
||
is running on <code>cluster1</code> and we have not yet configured access to <code>cluster2</code>.</p>
|
||
<h2 id="create-a-service-entry-and-destination-rule-on-cluster1-for-the-remote-reviews-service">Create a service entry and destination rule on <code>cluster1</code> for the remote reviews service</h2>
|
||
<p>As described in the <a href="/v1.1/docs/setup/kubernetes/install/multicluster/gateways/#setup-dns">setup instructions</a>,
|
||
remote services are accessed with a <code>.global</code> DNS name. In our case, it&rsquo;s <code>reviews.default.global</code>,
|
||
so we need to create a service entry and destination rule for that host.
|
||
The service entry will use the <code>cluster2</code> gateway as the endpoint address to access the service.
|
||
You can use the gateway&rsquo;s DNS name, if it has one, or its public IP, like this:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export CLUSTER2_GW_ADDR=$(kubectl get --context=$CTX_CLUSTER2 svc --selector=app=istio-ingressgateway \
|
||
-n istio-system -o jsonpath=&#34;{.items[0].status.loadBalancer.ingress[0].ip}&#34;)
|
||
</code></pre>
|
||
<p>Now create the service entry and destination rule using the following command:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply --context=$CTX_CLUSTER1 -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: reviews-default
|
||
spec:
|
||
hosts:
|
||
- reviews.default.global
|
||
location: MESH_INTERNAL
|
||
ports:
|
||
- name: http1
|
||
number: 9080
|
||
protocol: http
|
||
resolution: DNS
|
||
addresses:
|
||
- 127.255.0.3
|
||
endpoints:
|
||
- address: ${CLUSTER2_GW_ADDR}
|
||
labels:
|
||
cluster: cluster2
|
||
ports:
|
||
http1: 15443 # Do not change this port value
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: reviews-global
|
||
spec:
|
||
host: reviews.default.global
|
||
trafficPolicy:
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
subsets:
|
||
- name: v2
|
||
labels:
|
||
cluster: cluster2
|
||
- name: v3
|
||
labels:
|
||
cluster: cluster2
|
||
EOF
|
||
</code></pre>
|
||
<p>The address <code>127.255.0.3</code> of the service entry can be any arbitrary unallocated IP.
|
||
Using an IP from the loopback range 127.0.0.0/8 is a good choice.
|
||
Check out the
|
||
<a href="/v1.1/docs/examples/multicluster/gateways/#configure-the-example-services">gateway-connected multicluster example</a>
|
||
for more details.</p>
|
||
<p>Note that the labels of the subsets in the destination rule map to the service entry
|
||
endpoint label (<code>cluster: cluster2</code>) corresponding to the <code>cluster2</code> gateway.
|
||
Once the request reaches the destination cluster, a local destination rule will be used
|
||
to identify the actual pod labels (<code>version: v1</code> or <code>version: v2</code>) corresponding to the
|
||
requested subset.</p>
|
||
<h2 id="create-a-destination-rule-on-both-clusters-for-the-local-reviews-service">Create a destination rule on both clusters for the local reviews service</h2>
|
||
<p>Technically, we only need to define the subsets of the local service that are being used
|
||
in each cluster (i.e., <code>v1</code> in <code>cluster1</code>, <code>v2</code> and <code>v3</code> in <code>cluster2</code>), but for simplicity we&rsquo;ll
|
||
just define all three subsets in both clusters, since there&rsquo;s nothing wrong with defining subsets
|
||
for versions that are not actually deployed.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply --context=$CTX_CLUSTER1 -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: reviews
|
||
spec:
|
||
host: reviews.default.svc.cluster.local
|
||
trafficPolicy:
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
subsets:
|
||
- name: v1
|
||
labels:
|
||
version: v1
|
||
- name: v2
|
||
labels:
|
||
version: v2
|
||
- name: v3
|
||
labels:
|
||
version: v3
|
||
EOF
|
||
</code></pre>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply --context=$CTX_CLUSTER2 -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: reviews
|
||
spec:
|
||
host: reviews.default.svc.cluster.local
|
||
trafficPolicy:
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
subsets:
|
||
- name: v1
|
||
labels:
|
||
version: v1
|
||
- name: v2
|
||
labels:
|
||
version: v2
|
||
- name: v3
|
||
labels:
|
||
version: v3
|
||
EOF
|
||
</code></pre>
|
||
<h2 id="create-a-virtual-service-to-route-reviews-service-traffic">Create a virtual service to route reviews service traffic</h2>
|
||
<p>At this point, all calls to the <code>reviews</code> service will go to the local <code>reviews</code> pods (<code>v1</code>) because
|
||
if you look at the source code you will see that the <code>productpage</code> implementation is simply making
|
||
requests to <code>http://reviews:9080</code> (which expands to host <code>reviews.default.svc.cluster.local</code>), the
|
||
local version of the service.
|
||
The corresponding remote service is named <code>reviews.default.global</code>, so route rules are needed to
|
||
redirect requests to the global host.</p>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">Note that if all of the versions of the <code>reviews</code> service were remote, so there is no local <code>reviews</code>
|
||
service defined, the DNS would resolve <code>reviews</code> directly to <code>reviews.default.global</code>. In that case
|
||
we could call the remote <code>reviews</code> service without any route rules.</div>
|
||
</aside>
|
||
</div>
|
||
<p>Apply the following virtual service to direct traffic for user <code>jason</code> to <code>reviews</code> versions <code>v2</code> and <code>v3</code> (<sup>50</sup>&frasl;<sub>50</sub>)
|
||
which are running on <code>cluster2</code>. Traffic for any other user will go to <code>reviews</code> version <code>v1</code>.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply --context=$CTX_CLUSTER1 -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: reviews
|
||
spec:
|
||
hosts:
|
||
- reviews.default.svc.cluster.local
|
||
http:
|
||
- match:
|
||
- headers:
|
||
end-user:
|
||
exact: jason
|
||
route:
|
||
- destination:
|
||
host: reviews.default.global
|
||
subset: v2
|
||
weight: 50
|
||
- destination:
|
||
host: reviews.default.global
|
||
subset: v3
|
||
weight: 50
|
||
- route:
|
||
- destination:
|
||
host: reviews.default.svc.cluster.local
|
||
subset: v1
|
||
EOF
|
||
</code></pre>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">This 50/50 rule isn&rsquo;t a particularly realistic example. It&rsquo;s just a convenient way to demonstrate
|
||
accessing multiple subsets of a remote service.</div>
|
||
</aside>
|
||
</div>
|
||
<p>Return to your browser and login as user <code>jason</code>. If you refresh the page several times, you should see
|
||
the display alternating between black and red ratings stars (<code>v2</code> and <code>v3</code>). If you logout, you will
|
||
only see reviews without ratings (<code>v1</code>).</p>
|
||
<h2 id="summary">Summary</h2>
|
||
<p>In this article, we&rsquo;ve seen how to use Istio route rules to distribute the versions of a service
|
||
across clusters in a multicluster service mesh with a
|
||
<a href="/v1.1/docs/concepts/multicluster-deployments/#multiple-control-plane-topology">multiple control plane topology</a>.
|
||
In this example, we manually configured the <code>.global</code> service entry and destination rules needed to provide
|
||
connectivity to one remote service, <code>reviews</code>. In general, however, if we wanted to enable any service
|
||
to run either locally or remotely, we would need to create <code>.global</code> resources for every service.
|
||
Fortunately, this process could be automated and likely will be in a future Istio release.</p></description><pubDate>Thu, 07 Feb 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/multicluster-version-routing/</link><author>Frank Budinsky (IBM)</author><guid isPermaLink="true">/v1.1/blog/2019/multicluster-version-routing/</guid><category>traffic-management</category><category>multicluster</category></item><item><title>Sail the Blog!</title><description><p>Welcome to the Istio blog!</p>
|
||
<p>To make it easier to publish your content on our website, we
|
||
<a href="/v1.1/about/contribute/creating-and-editing-pages/#choosing-a-page-type">updated the content types guide</a>.</p>
|
||
<p>The goal of the updated guide is to make sharing and finding content easier.</p>
|
||
<p>We want to make sharing timely information on Istio easy and the Istio blog is
|
||
a good place to start.</p>
|
||
<p>We welcome your posts to the blog if you think your content falls in one of the
|
||
following four categories:</p>
|
||
<ul>
|
||
<li>Your post details your experience using and configuring Istio. Ideally, your
|
||
post shares a novel experience or perspective.</li>
|
||
<li>Your post highlights or announces Istio features.</li>
|
||
<li>Your post announces or recaps an Istio-related event.</li>
|
||
<li>Your post details how to accomplish a task or fulfill a specific use case
|
||
using Istio.</li>
|
||
</ul>
|
||
<p>Posting your blog is only <a href="/v1.1/about/contribute/github/#how-to-contribute">one PR away</a>
|
||
and, if you wish, you can <a href="/v1.1/about/contribute/github/#review">request a review</a>.</p>
|
||
<p>We look forward to reading about your Istio experience on the blog soon!</p></description><pubDate>Tue, 05 Feb 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/sail-the-blog/</link><author>Rigs Caballero, Google</author><guid isPermaLink="true">/v1.1/blog/2019/sail-the-blog/</guid><category>community</category><category>blog</category><category>contribution</category><category>guide</category><category>guideline</category><category>event</category></item><item><title>Egress Gateway Performance Investigation</title><description>
|
||
<p>The main objective of this investigation was to determine the impact on performance and resource utilization when an egress gateway is added in the service mesh to access an external service (MongoDB, in this case). The steps to configure an egress gateway for an external MongoDB are described in the blog <a href="/v1.1/blog/2018/egress-mongo/">Consuming External MongoDB Services</a>.</p>
|
||
<p>The application used for this investigation was the Java version of Acmeair, which simulates an airline reservation system. This application is used in the Performance Regression Patrol of Istio daily builds, but on that setup the microservices have been accessing the external MongoDB directly via their sidecars, without an egress gateway.</p>
|
||
<p>The diagram below illustrates how regression patrol currently runs with Acmeair and Istio:</p>
|
||
<figure style="width:70%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:62.69230769230769%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./acmeair_regpatrol3.png" title="Acmeair benchmark in the Istio performance regression patrol environment">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./acmeair_regpatrol3.png" alt="Acmeair benchmark in the Istio performance regression patrol environment" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Acmeair benchmark in the Istio performance regression patrol environment</figcaption>
|
||
</figure>
|
||
<p>Another difference is that the application communicates with the external DB with plain MongoDB protocol. The first change made for this study was to establish a TLS communication between the MongoDB and its clients running within the application, as this is a more realistic scenario.</p>
|
||
<p>Several cases for accessing the external database from the mesh were tested and described next.</p>
|
||
<h2 id="egress-traffic-cases">Egress traffic cases</h2>
|
||
<h3 id="case-1-bypassing-the-sidecar">Case 1: Bypassing the sidecar</h3>
|
||
<p>In this case, the sidecar does not intercept the communication between the application and the external DB. This is accomplished by setting the init container argument -x with the CIDR of the MongoDB, which makes the sidecar ignore messages to/from this IP address. For example:</p>
|
||
<pre><code> - -x
|
||
- &quot;169.47.232.211/32&quot;
|
||
</code></pre>
|
||
<figure style="width:70%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:76.45536869340232%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./case1_sidecar_bypass3.png" title="Traffic to external MongoDB by-passing the sidecar">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./case1_sidecar_bypass3.png" alt="Traffic to external MongoDB by-passing the sidecar" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Traffic to external MongoDB by-passing the sidecar</figcaption>
|
||
</figure>
|
||
<h3 id="case-2-through-the-sidecar-with-service-entry">Case 2: Through the sidecar, with service entry</h3>
|
||
<p>This is the default configuration when the sidecar is injected into the application pod. All messages are intercepted by the sidecar and routed to the destination according to the configured rules, including the communication with external services. The MongoDB was defined as a <code>ServiceEntry</code>.</p>
|
||
<figure style="width:70%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:74.41253263707573%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./case2_sidecar_passthru3.png" title="Sidecar intercepting traffic to external MongoDB">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./case2_sidecar_passthru3.png" alt="Sidecar intercepting traffic to external MongoDB" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Sidecar intercepting traffic to external MongoDB</figcaption>
|
||
</figure>
|
||
<h3 id="case-3-egress-gateway">Case 3: Egress gateway</h3>
|
||
<p>The egress gateway and corresponding destination rule and virtual service resources are defined for accessing MongoDB. All traffic to and from the external DB goes through the egress gateway (envoy).</p>
|
||
<figure style="width:70%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:62.309368191721134%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./case3_egressgw3.png" title="Introduction of the egress gateway to access MongoDB">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./case3_egressgw3.png" alt="Introduction of the egress gateway to access MongoDB" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Introduction of the egress gateway to access MongoDB</figcaption>
|
||
</figure>
|
||
<h3 id="case-4-mutual-tls-between-sidecars-and-the-egress-gateway">Case 4: Mutual TLS between sidecars and the egress gateway</h3>
|
||
<p>In this case, there is an extra layer of security between the sidecars and the gateway, so some impact in performance is expected.</p>
|
||
<figure style="width:70%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:63.968957871396896%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./case4_egressgw_mtls3.png" title="Enabling mutual TLS between sidecars and the egress gateway">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./case4_egressgw_mtls3.png" alt="Enabling mutual TLS between sidecars and the egress gateway" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Enabling mutual TLS between sidecars and the egress gateway</figcaption>
|
||
</figure>
|
||
<h3 id="case-5-egress-gateway-with-sni-proxy">Case 5: Egress gateway with SNI proxy</h3>
|
||
<p>This scenario is used to evaluate the case where another proxy is required to access wildcarded domains. This may be required due current limitations of envoy. An nginx proxy was created as sidecar in the egress gateway pod.</p>
|
||
<figure style="width:70%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:65.2762119503946%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./case5_egressgw_sni_proxy3.png" title="Egress gateway with additional SNI Proxy">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./case5_egressgw_sni_proxy3.png" alt="Egress gateway with additional SNI Proxy" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Egress gateway with additional SNI Proxy</figcaption>
|
||
</figure>
|
||
<h2 id="environment">Environment</h2>
|
||
<ul>
|
||
<li>Istio version: 1.0.2</li>
|
||
<li><code>K8s</code> version: <code>1.10.5_1517</code></li>
|
||
<li>Acmeair App: 4 services (1 replica of each), inter-services transactions, external Mongo DB, avg payload: 620 bytes.</li>
|
||
</ul>
|
||
<h2 id="results">Results</h2>
|
||
<p><code>Jmeter</code> was used to generate the workload which consisted in a sequence of 5-minute runs, each one using a growing number of clients making http requests. The number of clients used were 1, 5, 10, 20, 30, 40, 50 and 60.</p>
|
||
<h3 id="throughput">Throughput</h3>
|
||
<p>The chart below shows the throughput obtained for the different cases:</p>
|
||
<figure style="width:75%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:54.29638854296388%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./throughput3.png" title="Throughput obtained for the different cases">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./throughput3.png" alt="Throughput obtained for the different cases" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Throughput obtained for the different cases</figcaption>
|
||
</figure>
|
||
<p>As you can see, there is no major impact in having sidecars and the egress gateway between the application and the external MongoDB, but enabling mutual TLS and then adding the SNI proxy caused a degradation in the throughput of about 10% and 24%, respectively.</p>
|
||
<h3 id="response-time">Response time</h3>
|
||
<p>The average response times for the different requests were collected when traffic was being driven with 20 clients. The chart below shows the average, median, 90%, 95% and 99% average values for each case:</p>
|
||
<figure style="width:75%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:48.76783398184176%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./response_times3.png" title="Response times obtained for the different configurations">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./response_times3.png" alt="Response times obtained for the different configurations" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Response times obtained for the different configurations</figcaption>
|
||
</figure>
|
||
<p>Likewise, not much difference in the response times for the 3 first cases, but mutual TLS and the extra proxy adds noticeable latency.</p>
|
||
<h3 id="cpu-utilization">CPU utilization</h3>
|
||
<p>The CPU usage was collected for all Istio components as well as for the sidecars during the runs. For a fair comparison, CPU used by Istio was normalized by the throughput obtained for a given run. The results are shown in the following graph:</p>
|
||
<figure style="width:75%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:53.96174863387978%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/egress-performance/./cpu_usage3.png" title="CPU usage normalized by TPS">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/egress-performance/./cpu_usage3.png" alt="CPU usage normalized by TPS" />
|
||
</a>
|
||
</div>
|
||
<figcaption>CPU usage normalized by TPS</figcaption>
|
||
</figure>
|
||
<p>In terms of CPU consumption per transaction, Istio has used significantly more CPU only in the egress gateway + SNI proxy case.</p>
|
||
<h2 id="conclusion">Conclusion</h2>
|
||
<p>In this investigation, we tried different options to access an external TLS-enabled MongoDB to compare their performance. The introduction of the Egress Gateway did not have a significant impact on the performance nor meaningful additional CPU consumption. Only when enabling mutual TLS between sidecars and egress gateway or using an additional SNI proxy for wildcarded domains we could observe some degradation.</p></description><pubDate>Thu, 31 Jan 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/egress-performance/</link><author>Jose Nativio, IBM</author><guid isPermaLink="true">/v1.1/blog/2019/egress-performance/</guid><category>performance</category><category>traffic-management</category><category>egress</category><category>mongo</category></item><item><title>Demystifying Istio's Sidecar Injection Model</title><description>
|
||
<p>A simple overview of an Istio service-mesh architecture always starts with describing the control-plane and data-plane.</p>
|
||
<p><a href="/v1.1/docs/concepts/what-is-istio/#architecture">From Istio’s documentation:</a></p>
|
||
<div>
|
||
<aside class="callout quote">
|
||
<div class="type">
|
||
<svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-quote"/></svg>
|
||
</div>
|
||
<div class="content"><p>An Istio service mesh is logically split into a data plane and a control plane.</p>
|
||
<p>The data plane is composed of a set of intelligent proxies (Envoy) deployed as sidecars. These proxies mediate and control all network communication between microservices along with Mixer, a general-purpose policy and telemetry hub.</p>
|
||
<p>The control plane manages and configures the proxies to route traffic. Additionally, the control plane configures Mixers to enforce policies and collect telemetry.</p>
|
||
</div>
|
||
</aside>
|
||
</div>
|
||
<figure style="width:40%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:80%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2019/data-plane-setup/./arch-2.svg" title="Istio Architecture">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2019/data-plane-setup/./arch-2.svg" alt="The overall architecture of an Istio-based application." />
|
||
</a>
|
||
</div>
|
||
<figcaption>Istio Architecture</figcaption>
|
||
</figure>
|
||
<p>It is important to understand that the sidecar injection into the application pods happens automatically, though manual injection is also possible. Traffic is directed from the application services to and from these sidecars without developers needing to worry about it. Once the applications are connected to the Istio service mesh, developers can start using and reaping the benefits of all that the service mesh has to offer. However, how does the data plane plumbing happen and what is really required to make it work seamlessly? In this post, we will deep-dive into the specifics of the sidecar injection models to gain a very clear understanding of how sidecar injection works.</p>
|
||
<h2 id="sidecar-injection">Sidecar injection</h2>
|
||
<p>In simple terms, sidecar injection is adding the configuration of additional containers to the pod template. The added containers needed for the Istio service mesh are:</p>
|
||
<p><code>istio-init</code>
|
||
This <a href="https://kubernetes.io/docs/concepts/workloads/pods/init-containers/">init container</a> is used to setup the <code>iptables</code> rules so that inbound/outbound traffic will go through the sidecar proxy. An init container is different than an app container in following ways:</p>
|
||
<ul>
|
||
<li>It runs before an app container is started and it always runs to completion.</li>
|
||
<li>If there are many init containers, each should complete with success before the next container is started.</li>
|
||
</ul>
|
||
<p>So, you can see how this type of container is perfect for a set-up or initialization job which does not need to be a part of the actual application container. In this case, <code>istio-init</code> does just that and sets up the <code>iptables</code> rules.</p>
|
||
<p><code>istio-proxy</code>
|
||
This is the actual sidecar proxy (based on Envoy).</p>
|
||
<h3 id="manual-injection">Manual injection</h3>
|
||
<p>In the manual injection method, you can use <code>istioctl</code> to modify the pod template and add the configuration of the two containers previously mentioned. For both manual as well as automatic injection, Istio takes the configuration from the <code>istio-sidecar-injector</code> configuration map (configmap) and the mesh&rsquo;s <code>istio</code> configmap.</p>
|
||
<p>Let’s look at the configuration of the <code>istio-sidecar-injector</code> configmap, to get an idea of what actually is going on.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' data-outputis='yaml' >$ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath=&#39;{.data.config}&#39;
|
||
SNIPPET from the output:
|
||
policy: enabled
|
||
template: |-
|
||
initContainers:
|
||
- name: istio-init
|
||
image: docker.io/istio/proxy_init:1.0.2
|
||
args:
|
||
- &#34;-p&#34;
|
||
- [[ .MeshConfig.ProxyListenPort ]]
|
||
- &#34;-u&#34;
|
||
- 1337
|
||
.....
|
||
imagePullPolicy: IfNotPresent
|
||
securityContext:
|
||
capabilities:
|
||
add:
|
||
- NET_ADMIN
|
||
restartPolicy: Always
|
||
containers:
|
||
- name: istio-proxy
|
||
image: [[ if (isset .ObjectMeta.Annotations &#34;sidecar.istio.io/proxyImage&#34;) -]]
|
||
&#34;[[ index .ObjectMeta.Annotations &#34;sidecar.istio.io/proxyImage&#34; ]]&#34;
|
||
[[ else -]]
|
||
docker.io/istio/proxyv2:1.0.2
|
||
[[ end -]]
|
||
args:
|
||
- proxy
|
||
- sidecar
|
||
.....
|
||
env:
|
||
.....
|
||
- name: ISTIO_META_INTERCEPTION_MODE
|
||
value: [[ or (index .ObjectMeta.Annotations &#34;sidecar.istio.io/interceptionMode&#34;) .ProxyConfig.InterceptionMode.String ]]
|
||
imagePullPolicy: IfNotPresent
|
||
securityContext:
|
||
readOnlyRootFilesystem: true
|
||
[[ if eq (or (index .ObjectMeta.Annotations &#34;sidecar.istio.io/interceptionMode&#34;) .ProxyConfig.InterceptionMode.String) &#34;TPROXY&#34; -]]
|
||
capabilities:
|
||
add:
|
||
- NET_ADMIN
|
||
restartPolicy: Always
|
||
.....
|
||
</code></pre>
|
||
<p>As you can see, the configmap contains the configuration for both, the <code>istio-init</code> init container and the <code>istio-proxy</code> proxy container. The configuration includes the name of the container image and arguments like interception mode, capabilities, etc.</p>
|
||
<p>From a security point of view, it is important to note that <code>istio-init</code> requires <code>NET_ADMIN</code> capabilities to modify <code>iptables</code> within the pod&rsquo;s namespace and so does <code>istio-proxy</code> if configured in <code>TPROXY</code> mode. As this is restricted to a pod&rsquo;s namespace, there should be no problem. However, I have noticed that recent open-shift versions may have some issues with it and a workaround is needed. One such option is mentioned at the end of this post.</p>
|
||
<p>To modify the current pod template for sidecar injection, you can:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ istioctl kube-inject -f demo-red.yaml | kubectl apply -f -
|
||
</code></pre>
|
||
<p>OR</p>
|
||
<p>To use modified configmaps or local configmaps:</p>
|
||
<ul>
|
||
<li><p>Create <code>inject-config.yaml</code> and <code>mesh-config.yaml</code> from the configmaps</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath=&#39;{.data.config}&#39; &gt; inject-config.yaml
|
||
$ kubectl -n istio-system get configmap istio -o=jsonpath=&#39;{.data.mesh}&#39; &gt; mesh-config.yaml
|
||
</code></pre></li>
|
||
<li><p>Modify the existing pod template, in my case, <code>demo-red.yaml</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ istioctl kube-inject --injectConfigFile inject-config.yaml --meshConfigFile mesh-config.yaml --filename demo-red.yaml --output demo-red-injected.yaml
|
||
</code></pre></li>
|
||
<li><p>Apply the <code>demo-red-injected.yaml</code></p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f demo-red-injected.yaml
|
||
</code></pre></li>
|
||
</ul>
|
||
<p>As seen above, we create a new template using the <code>sidecar-injector</code> and the mesh configuration to then apply that new template using <code>kubectl</code>. If we look at the injected YAML file, it has the configuration of the Istio-specific containers, as we discussed above. Once we apply the injected YAML file, we see two containers running. One of them is the actual application container, and the other is the <code>istio-proxy</code> sidecar.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods | grep demo-red
|
||
demo-red-pod-8b5df99cc-pgnl7 2/2 Running 0 3d
|
||
</code></pre>
|
||
<p>The count is not 3 because the <code>istio-init</code> container is an init type container that exits after doing what it supposed to do, which is setting up the <code>iptable</code> rules within the pod. To confirm the init container exit, let’s look at the output of <code>kubectl describe</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' data-outputis='yaml' >$ kubectl describe pod demo-red-pod-8b5df99cc-pgnl7
|
||
SNIPPET from the output:
|
||
Name: demo-red-pod-8b5df99cc-pgnl7
|
||
Namespace: default
|
||
.....
|
||
Labels: app=demo-red
|
||
pod-template-hash=8b5df99cc
|
||
version=version-red
|
||
Annotations: sidecar.istio.io/status={&#34;version&#34;:&#34;3c0b8d11844e85232bc77ad85365487638ee3134c91edda28def191c086dc23e&#34;,&#34;initContainers&#34;:[&#34;istio-init&#34;],&#34;containers&#34;:[&#34;istio-proxy&#34;],&#34;volumes&#34;:[&#34;istio-envoy&#34;,&#34;istio-certs...
|
||
Status: Running
|
||
IP: 10.32.0.6
|
||
Controlled By: ReplicaSet/demo-red-pod-8b5df99cc
|
||
Init Containers:
|
||
istio-init:
|
||
Container ID: docker://bef731eae1eb3b6c9d926cacb497bb39a7d9796db49cd14a63014fc1a177d95b
|
||
Image: docker.io/istio/proxy_init:1.0.2
|
||
Image ID: docker-pullable://docker.io/istio/proxy_init@sha256:e16a0746f46cd45a9f63c27b9e09daff5432e33a2d80c8cc0956d7d63e2f9185
|
||
.....
|
||
State: Terminated
|
||
Reason: Completed
|
||
.....
|
||
Ready: True
|
||
Containers:
|
||
demo-red:
|
||
Container ID: docker://8cd9957955ff7e534376eb6f28b56462099af6dfb8b9bc37aaf06e516175495e
|
||
Image: chugtum/blue-green-image:v3
|
||
Image ID: docker-pullable://docker.io/chugtum/blue-green-image@sha256:274756dbc215a6b2bd089c10de24fcece296f4c940067ac1a9b4aea67cf815db
|
||
State: Running
|
||
Started: Sun, 09 Dec 2018 18:12:31 -0800
|
||
Ready: True
|
||
istio-proxy:
|
||
Container ID: docker://ca5d690be8cd6557419cc19ec4e76163c14aed2336eaad7ebf17dd46ca188b4a
|
||
Image: docker.io/istio/proxyv2:1.0.2
|
||
Image ID: docker-pullable://docker.io/istio/proxyv2@sha256:54e206530ba6ca9b3820254454e01b7592e9f986d27a5640b6c03704b3b68332
|
||
Args:
|
||
proxy
|
||
sidecar
|
||
.....
|
||
State: Running
|
||
Started: Sun, 09 Dec 2018 18:12:31 -0800
|
||
Ready: True
|
||
.....
|
||
</code></pre>
|
||
<p>As seen in the output, the <code>State</code> of the <code>istio-init</code> container is <code>Terminated</code> with the <code>Reason</code> being <code>Completed</code>. The only two containers running are the main application <code>demo-red</code> container and the <code>istio-proxy</code> container.</p>
|
||
<h3 id="automatic-injection">Automatic injection</h3>
|
||
<p>Most of the times, you don’t want to manually inject a sidecar every time you deploy an application, using the <code>istioctl</code> command, but would prefer that Istio automatically inject the sidecar to your pod. This is the recommended approach and for it to work, all you need to do is to label the namespace where you are deploying the app with <code>istio-injection=enabled</code>.</p>
|
||
<p>Once labeled, Istio injects the sidecar automatically for any pod you deploy in that namespace. In the following example, the sidecar gets automatically injected in the deployed pods in the <code>istio-dev</code> namespace.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get namespaces --show-labels
|
||
NAME STATUS AGE LABELS
|
||
default Active 40d &lt;none&gt;
|
||
istio-dev Active 19d istio-injection=enabled
|
||
istio-system Active 24d &lt;none&gt;
|
||
kube-public Active 40d &lt;none&gt;
|
||
kube-system Active 40d &lt;none&gt;
|
||
</code></pre>
|
||
<p>But how does this work? To get to the bottom of this, we need to understand Kubernetes admission controllers.</p>
|
||
<p><a href="https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/">From Kubernetes documentation:</a></p>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized. You can define two types of admission webhooks, validating admission Webhook and mutating admission webhook. With validating admission Webhooks, you may reject requests to enforce custom admission policies. With mutating admission Webhooks, you may change requests to enforce custom defaults.</div>
|
||
</aside>
|
||
</div>
|
||
<p>For automatic sidecar injection, Istio relies on <code>Mutating Admission Webhook</code>. Let’s look at the details of the <code>istio-sidecar-injector</code> mutating webhook configuration.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' data-outputis='yaml' >$ kubectl get mutatingwebhookconfiguration istio-sidecar-injector -o yaml
|
||
SNIPPET from the output:
|
||
apiVersion: admissionregistration.k8s.io/v1beta1
|
||
kind: MutatingWebhookConfiguration
|
||
metadata:
|
||
annotations:
|
||
kubectl.kubernetes.io/last-applied-configuration: |
|
||
{&#34;apiVersion&#34;:&#34;admissionregistration.k8s.io/v1beta1&#34;,&#34;kind&#34;:&#34;MutatingWebhookConfiguration&#34;,&#34;metadata&#34;:{&#34;annotations&#34;:{},&#34;labels&#34;:{&#34;app&#34;:&#34;istio-sidecar-injector&#34;,&#34;chart&#34;:&#34;sidecarInjectorWebhook-1.0.1&#34;,&#34;heritage&#34;:&#34;Tiller&#34;,&#34;release&#34;:&#34;istio-remote&#34;},&#34;name&#34;:&#34;istio-sidecar-injector&#34;,&#34;namespace&#34;:&#34;&#34;},&#34;webhooks&#34;:[{&#34;clientConfig&#34;:{&#34;caBundle&#34;:&#34;&#34;,&#34;service&#34;:{&#34;name&#34;:&#34;istio-sidecar-injector&#34;,&#34;namespace&#34;:&#34;istio-system&#34;,&#34;path&#34;:&#34;/inject&#34;}},&#34;failurePolicy&#34;:&#34;Fail&#34;,&#34;name&#34;:&#34;sidecar-injector.istio.io&#34;,&#34;namespaceSelector&#34;:{&#34;matchLabels&#34;:{&#34;istio-injection&#34;:&#34;enabled&#34;}},&#34;rules&#34;:[{&#34;apiGroups&#34;:[&#34;&#34;],&#34;apiVersions&#34;:[&#34;v1&#34;],&#34;operations&#34;:[&#34;CREATE&#34;],&#34;resources&#34;:[&#34;pods&#34;]}]}]}
|
||
creationTimestamp: 2018-12-10T08:40:15Z
|
||
generation: 2
|
||
labels:
|
||
app: istio-sidecar-injector
|
||
chart: sidecarInjectorWebhook-1.0.1
|
||
heritage: Tiller
|
||
release: istio-remote
|
||
name: istio-sidecar-injector
|
||
.....
|
||
webhooks:
|
||
- clientConfig:
|
||
service:
|
||
name: istio-sidecar-injector
|
||
namespace: istio-system
|
||
path: /inject
|
||
name: sidecar-injector.istio.io
|
||
namespaceSelector:
|
||
matchLabels:
|
||
istio-injection: enabled
|
||
rules:
|
||
- apiGroups:
|
||
- &#34;&#34;
|
||
apiVersions:
|
||
- v1
|
||
operations:
|
||
- CREATE
|
||
resources:
|
||
- pods
|
||
</code></pre>
|
||
<p>This is where you can see the webhook <code>namespaceSelector</code> label that is matched for sidecar injection with the label <code>istio-injection: enabled</code>. In this case, you also see the operations and resources for which this is done when the pods are created. When an <code>apiserver</code> receives a request that matches one of the rules, the <code>apiserver</code> sends an admission review request to the webhook service as specified in the <code>clientConfig:</code>configuration with the <code>name: istio-sidecar-injector</code> key-value pair. We should be able to see that this service is running in the <code>istio-system</code> namespace.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get svc --namespace=istio-system | grep sidecar-injector
|
||
istio-sidecar-injector ClusterIP 10.102.70.184 &lt;none&gt; 443/TCP 24d
|
||
</code></pre>
|
||
<p>This configuration ultimately does pretty much the same as we saw in manual injection. Just that it is done automatically during pod creation, so you won’t see the change in the deployment. You need to use <code>kubectl describe</code> to see the sidecar proxy and the init proxy. In case you want to change the default behavior, like the namespaces where Istio applies the injection, you can edit the <code>MutatingWebhookConfiguration</code> and restart the sidecar injector pod.</p>
|
||
<p>The automatic sidecar injection depends on the <code>namespaceSelector</code> webhook but also on the default injection policy and the per-pod override annotation.</p>
|
||
<p>If you look at the <code>istio-sidecar-injector</code> ConfigMap again, it has the default injection policy defined. In our case, it is enabled by default.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' data-outputis='yaml' >$ kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath=&#39;{.data.config}&#39;
|
||
SNIPPET from the output:
|
||
policy: enabled
|
||
template: |-
|
||
initContainers:
|
||
- name: istio-init
|
||
image: &#34;gcr.io/istio-release/proxy_init:1.0.2&#34;
|
||
args:
|
||
- &#34;-p&#34;
|
||
- [[ .MeshConfig.ProxyListenPort ]]
|
||
</code></pre>
|
||
<p>You can also use the annotation <code>sidecar.istio.io/inject</code> in the pod template to override the default policy. The following example disables the automatic injection of the sidecar for the pods in a <code>Deployment</code>.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: ignored
|
||
spec:
|
||
template:
|
||
metadata:
|
||
annotations:
|
||
sidecar.istio.io/inject: &#34;false&#34;
|
||
spec:
|
||
containers:
|
||
- name: ignored
|
||
image: tutum/curl
|
||
command: [&#34;/bin/sleep&#34;,&#34;infinity&#34;]
|
||
</code></pre>
|
||
<p>This example shows there are many variables, based on whether the automatic sidecar injection is controlled in your namespace, ConfigMap, or pod and they are:</p>
|
||
<ul>
|
||
<li>webhooks <code>namespaceSelector</code> (<code>istio-injection: enabled</code>)</li>
|
||
<li>default policy (Configured in the ConfigMap <code>istio-sidecar-injector</code>)</li>
|
||
<li>per-pod override annotation (<code>sidecar.istio.io/inject</code>)</li>
|
||
</ul>
|
||
<p>The <a href="../../../help/ops/setup/injection/">injection status table</a> shows a clear picture of the final injection status based on the value of the above variables.</p>
|
||
<h2 id="traffic-flow-from-application-container-to-sidecar-proxy">Traffic flow from application container to sidecar proxy</h2>
|
||
<p>Now that we are clear about how a sidecar container and an init container are injected into an application manifest, how does the sidecar proxy grab the inbound and outbound traffic to and from the container? We did briefly mention that it is done by setting up the <code>iptable</code> rules within the pod namespace, which in turn is done by the <code>istio-init</code> container. Now, it is time to verify what actually gets updated within the namespace.</p>
|
||
<p>Let’s get into the application pod namespace we deployed in the previous section and look at the configured iptables. I am going to show an example using <code>nsenter</code>. Alternatively, you can enter the container in a privileged mode to see the same information. For folks without access to the nodes, using <code>exec</code> to get into the sidecar and running <code>iptables</code> is more practical.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ docker inspect b8de099d3510 --format &#39;{{ .State.Pid }}&#39;
|
||
4125
|
||
</code></pre>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ nsenter -t 4215 -n iptables -t nat -S
|
||
-P PREROUTING ACCEPT
|
||
-P INPUT ACCEPT
|
||
-P OUTPUT ACCEPT
|
||
-P POSTROUTING ACCEPT
|
||
-N ISTIO_INBOUND
|
||
-N ISTIO_IN_REDIRECT
|
||
-N ISTIO_OUTPUT
|
||
-N ISTIO_REDIRECT
|
||
-A PREROUTING -p tcp -j ISTIO_INBOUND
|
||
-A OUTPUT -p tcp -j ISTIO_OUTPUT
|
||
-A ISTIO_INBOUND -p tcp -m tcp --dport 80 -j ISTIO_IN_REDIRECT
|
||
-A ISTIO_IN_REDIRECT -p tcp -j REDIRECT --to-ports 15001
|
||
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -j ISTIO_REDIRECT
|
||
-A ISTIO_OUTPUT -m owner --uid-owner 1337 -j RETURN
|
||
-A ISTIO_OUTPUT -m owner --gid-owner 1337 -j RETURN
|
||
-A ISTIO_OUTPUT -d 127.0.0.1/32 -j RETURN
|
||
-A ISTIO_OUTPUT -j ISTIO_REDIRECT
|
||
-A ISTIO_REDIRECT -p tcp -j REDIRECT --to-ports 15001
|
||
</code></pre>
|
||
<p>The output above clearly shows that all the incoming traffic to port 80, which is the port our <code>red-demo</code> application is listening, is now <code>REDIRECTED</code> to port <code>15001</code>, which is the port that the <code>istio-proxy</code>, an Envoy proxy, is listening. The same holds true for the outgoing traffic.</p>
|
||
<p>This brings us to the end of this post. I hope it helped to de-mystify how Istio manages to inject the sidecar proxies into an existing deployment and how Istio routes the traffic to the proxy.</p>
|
||
<div>
|
||
<aside class="callout idea">
|
||
<div class="type">
|
||
<svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-idea"/></svg>
|
||
</div>
|
||
<div class="content">Update: In place of <code>istio-init</code>, there now seems to be an option of using the new CNI, which removes the need for the init container and associated privileges. This <a href="https://github.com/istio/cni"><code>istio-cni</code></a> plugin sets up the pods&rsquo; networking to fulfill this requirement in place of the current Istio injected pod <code>istio-init</code> approach.</div>
|
||
</aside>
|
||
</div></description><pubDate>Thu, 31 Jan 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/data-plane-setup/</link><author>Manish Chugtu</author><guid isPermaLink="true">/v1.1/blog/2019/data-plane-setup/</guid><category>kubernetes</category><category>sidecar-injection</category><category>traffic-management</category></item><item><title>Sidestepping Dependency Ordering with AppSwitch</title><description>
|
||
<p>We are going through an interesting cycle of application decomposition and recomposition. While the microservice paradigm is driving monolithic applications to be broken into separate individual services, the service mesh approach is helping them to be connected back together into well-structured applications. As such, microservices are logically separate but not independent. They are usually closely interdependent and taking them apart introduces many new concerns such as need for mutual authentication between services. Istio directly addresses most of those issues.</p>
|
||
<h2 id="dependency-ordering-problem">Dependency ordering problem</h2>
|
||
<p>An issue that arises due to application decomposition and one that Istio doesn’t address is dependency ordering &ndash; bringing up individual services of an application in an order that guarantees that the application as a whole comes up quickly and correctly. In a monolithic application, with all its components built-in, dependency ordering between the components is enforced by internal locking mechanisms. But with individual services potentially scattered across the cluster in a service mesh, starting a service first requires checking that the services it depends on are up and available.</p>
|
||
<p>Dependency ordering is deceptively nuanced with a host of interrelated problems. Ordering individual services requires having the dependency graph of the services so that they can be brought up starting from leaf nodes back to the root nodes. It is not easy to construct such a graph and keep it updated over time as interdependencies evolve with the behavior of the application. Even if the dependency graph is somehow provided, enforcing the ordering itself is not easy. Simply starting the services in the specified order obviously won’t do. A service may have started but not be ready to accept connections yet. This is the problem with docker-compose&rsquo;s <code>depends-on</code> tag, for example.</p>
|
||
<p>Apart from introducing sufficiently long sleeps between service startups, a common pattern that is often used is to check for readiness of dependencies before starting a service. In Kubernetes, this could be done with a wait script as part of the init container of the pod. However that means that the entire application would be held up until all its dependencies come alive. Sometimes applications spend several minutes initializing themselves on startup before making their first outbound connection. Not allowing a service to start at all adds substantial overhead to overall startup time of the application. Also, the strategy of waiting on the init container won&rsquo;t work for the case of multiple interdependent services within the same pod.</p>
|
||
<h3 id="example-scenario-ibm-websphere-nd">Example scenario: IBM WebSphere ND</h3>
|
||
<p>Let us consider IBM WebSphere ND &ndash; a widely deployed application middleware &ndash; to grok these problems more closely. It is a fairly complex framework in itself and consists of a central component called deployment manager (<code>dmgr</code>) that manages a set of node instances. It uses UDP to negotiate cluster membership among the nodes and requires that deployment manager is up and operational before any of the node instances can come up and join the cluster.</p>
|
||
<p>Why are we talking about a traditional application in the modern cloud-native context? It turns out that there are significant gains to be had by enabling them to run on the Kubernetes and Istio platforms. Essentially it&rsquo;s a part of the modernization journey that allows running traditional apps alongside green-field apps on the same modern platform to facilitate interoperation between the two. In fact, WebSphere ND is a demanding application. It expects a consistent network environment with specific network interface attributes etc. AppSwitch is equipped to take care of those requirements. For the purpose of this blog however, I&rsquo;ll focus on the dependency ordering requirement and how AppSwitch addresses it.</p>
|
||
<p>Simply deploying <code>dmgr</code> and node instances as pods on a Kubernetes cluster does not work. <code>dmgr</code> and the node instances happen to have a lengthy initialization process that can take several minutes. If they are all co-scheduled, the application typically ends up in a funny state. When a node instance comes up and finds that <code>dmgr</code> is missing, it would take an alternate startup path. Instead, if it had exited immediately, Kubernetes crash-loop would have taken over and perhaps the application would have come up. But even in that case, it turns out that a timely startup is not guaranteed.</p>
|
||
<p>One <code>dmgr</code> along with its node instances is a basic deployment configuration for WebSphere ND. Applications like IBM Business Process Manager that are built on top of WebSphere ND running in production environments include several other services. In those configurations, there could be a chain of interdependencies. Depending on the applications hosted by the node instances, there may be an ordering requirement among them as well. With long service initialization times and crash-loop restarts, there is little chance for the application to start in any reasonable length of time.</p>
|
||
<h3 id="sidecar-dependency-in-istio">Sidecar dependency in Istio</h3>
|
||
<p>Istio itself is affected by a version of the dependency ordering problem. Since connections into and out of a service running under Istio are redirected through its sidecar proxy, an implicit dependency is created between the application service and its sidecar. Unless the sidecar is fully operational, all requests from and to the service get dropped.</p>
|
||
<h2 id="dependency-ordering-with-appswitch">Dependency ordering with AppSwitch</h2>
|
||
<p>So how do we go about addressing these issues? One way is to defer it to the applications and say that they are supposed to be &ldquo;well behaved&rdquo; and implement appropriate logic to make themselves immune to startup order issues. However, many applications (especially traditional ones) either timeout or deadlock if misordered. Even for new applications, implementing one off logic for each service is substantial additional burden that is best avoided. Service mesh needs to provide adequate support around these problems. After all, factoring out common patterns into an underlying framework is really the point of service mesh.</p>
|
||
<p><a href="http://appswitch.io">AppSwitch</a> explicitly addresses dependency ordering. It sits on the control path of the application’s network interactions between clients and services in a cluster and knows precisely when a service becomes a client by making the <code>connect</code> call and when a particular service becomes ready to accept connections by making the <code>listen</code> call. It&rsquo;s <em>service router</em> component disseminates information about these events across the cluster and arbitrates interactions among clients and servers. That is how AppSwitch implements functionality such as load balancing and isolation in a simple and efficient manner. Leveraging the same strategic location of the application&rsquo;s network control path, it is conceivable that the <code>connect</code> and <code>listen</code> calls made by those services can be lined up at a finer granularity rather than coarsely sequencing entire services as per a dependency graph. That would effectively solve the multilevel dependency problem and speedup application startup.</p>
|
||
<p>But that still requires a dependency graph. A number of products and tools exist to help with discovering service dependencies. But they are typically based on passive monitoring of network traffic and cannot provide the information beforehand for any arbitrary application. Network level obfuscation due to encryption and tunneling also makes them unreliable. The burden of discovering and specifying the dependencies ultimately falls to the developer or the operator of the application. As it is, even consistency checking a dependency specification is itself quite complex and any way to avoid requiring a dependency graph would be most desirable.</p>
|
||
<p>The point of a dependency graph is to know which clients depend on a particular service so that those clients can then be made to wait for the respective service to become live. But does it really matter which specific clients? Ultimately one tautology that always holds is that all clients of a service have an implicit dependency on the service. That’s what AppSwitch leverages to get around the requirement. In fact, that sidesteps dependency ordering altogether. All services of the application can be co-scheduled without regard to any startup order. Interdependencies among them automatically work themselves out at the granularity of individual requests and responses, resulting in quick and correct application startups.</p>
|
||
<h3 id="appswitch-model-and-constructs">AppSwitch model and constructs</h3>
|
||
<p>Now that we have a conceptual understanding of AppSwitch’s high-level approach, let’s look at the constructs involved. But first a quick summary of the usage model is in order. Even though it is written for a different context, reviewing my earlier <a href="/v1.1/blog/2018/delayering-istio/">blog</a> on this topic would be useful as well. For completeness, let me also note AppSwitch doesn’t bother with non-network dependencies. For example it may be possible for two services to interact using IPC mechanisms or through the shared file system. Processes with deep ties like that are typically part of the same service anyway and don’t require framework’s intervention for ordering.</p>
|
||
<p>At its core, AppSwitch is built on a mechanism that allows instrumenting the BSD socket API and other related calls like <code>fcntl</code> and <code>ioctl</code> that deal with sockets. As interesting as the details of its implementation are, it’s going to distract us from the main topic, so I’d just summarize the key properties that distinguish it from other implementations. (1) It’s fast. It uses a combination of <code>seccomp</code> filtering and binary instrumentation to aggressively limit intervening with application’s normal execution. AppSwitch is particularly suited for service mesh and application networking use cases given that it implements those features without ever having to actually touch the data. In contrast, network level approaches incur per-packet cost. Take a look at this <a href="/v1.1/blog/2018/delayering-istio/">blog</a> for some of the performance measurements. (2) It doesn’t require any kernel support, kernel module or a patch and works on standard distro kernels (3) It can run as regular user (no root). In fact, the mechanism can even make it possible to run <a href="https://linuxpiter.com/en/materials/2478">Docker daemon without root</a> by removing root requirement to network containers (4) It doesn’t require any changes to the applications whatsoever and works for any type of application &ndash; from WebSphere ND and SAP to custom C apps to statically linked Golang apps. Only requirement at this point is Linux/x86.</p>
|
||
<h3 id="decoupling-services-from-their-references">Decoupling services from their references</h3>
|
||
<p>AppSwitch is built on the fundamental premise that applications should be decoupled from their references. The identity of applications is traditionally derived from the identity of the host on which they run. However, applications and hosts are very different objects that need to be referenced independently. Detailed discussion around this topic along with a conceptual foundation of AppSwitch is presented in this <a href="http://cn.arxiv.org/abs/1711.02294">research paper</a>.</p>
|
||
<p>The central AppSwitch construct that achieves the decoupling between services objects and their identities is <em>service reference</em> (<em>reference</em>, for short). AppSwitch implements service references based on the API instrumentation mechanism outlined above. A service reference consists of an IP:port pair (and optionally a DNS name) and a label-selector that selects the service represented by the reference and the clients to which this reference applies. A reference supports a few key properties. (1) It can be named independently of the name of the object it refers to. That is, a service may be listening on an IP and port but a reference allows that service to be reached on any other IP and port chosen by the user. This is what allows AppSwitch to run traditional applications captured from their source environments with static IP configurations to run on Kubernetes by providing them with necessary IP addresses and ports regardless of the target network environment. (2) It remains unchanged even if the location of the target service changes. A reference automatically redirects itself as its label-selector now resolves to the new instance of the service (3) Most important for this discussion, a reference remains valid even as the target service is coming up.</p>
|
||
<p>To facilitate discovering services that can be accessed through service references, AppSwitch provides an <em>auto-curated service registry</em>. The registry is automatically kept up to date as services come and go across the cluster based on the network API that AppSwitch tracks. Each entry in the registry consists of the IP and port where the respective service is bound. Along with that, it includes a set of labels indicating the application to which this service belongs, the IP and port that the application passed through the socket API when creating the service, the IP and port where AppSwitch actually bound the service on the underlying host on behalf of the application etc. In addition, applications created under AppSwitch carry a set of labels passed by the user that describe the application together with a few default system labels indicating the user that created the application and the host where the application is running etc. These labels are all available to be expressed in the label-selector carried by a service reference. A service in the registry can be made accessible to clients by creating a service reference. A client would then be able to reach the service at the reference’s name (IP:port). Now let’s look at how AppSwitch guarantees that the reference remains valid even when the target service has not yet come up.</p>
|
||
<h3 id="non-blocking-requests">Non-blocking requests</h3>
|
||
<p>AppSwitch leverages the semantics of the BSD socket API to ensure that service references appear valid from the perspective of clients as corresponding services come up. When a client makes a blocking connect call to another service that has not yet come up, AppSwitch blocks the call for a certain time waiting for the target service to become live. Since it is known that the target service is a part of the application and is expected to come up shortly, making the client block rather than returning an error such as <code>ECONNREFUSED</code> prevents the application from failing to start. If the service doesn’t come up within time, an error is returned to the application so that framework-level mechanisms like Kubernetes crash-loop can kick in.</p>
|
||
<p>If the client request is marked as non-blocking, AppSwitch handles that by returning <code>EAGAIN</code> to inform the application to retry rather than give up. Once again, that is in-line with the semantics of socket API and prevents failures due to startup races. AppSwitch essentially enables the retry logic already built into applications in support of the BSD socket API to be transparently repurposed for dependency ordering.</p>
|
||
<h3 id="application-timeouts">Application timeouts</h3>
|
||
<p>What if the application times out based on its own internal timer? Truth be told, AppSwitch can also fake application’s perception of time if wanted but that would be overstepping and actually unnecessary. Application decides and knows best how long it should wait and it’s not appropriate for AppSwitch to mess with that. Application timeouts are conservatively long and if the target service still hasn’t come up in time, it is unlikely to be a dependency ordering issue. There must be something else going on that should not be masked.</p>
|
||
<h3 id="wildcard-service-references-for-sidecar-dependency">Wildcard service references for sidecar dependency</h3>
|
||
<p>Service references can be used to address the Istio sidecar dependency issue mentioned earlier. AppSwitch allows the IP:port specified as part of a service reference to be a wildcard. That is, the service reference IP address can be a netmask indicating the IP address range to be captured. If the label selector of the service reference points to the sidecar service, then all outgoing connections of any application for which this service reference is applied, will be transparently redirected to the sidecar. And of course, the service reference remains valid while sidecar is still coming up and the race is removed.</p>
|
||
<p>Using service references for sidecar dependency ordering also implicitly redirects application’s connections to the sidecar without requiring iptables and attendant privilege issues. Essentially it works as if the application is directly making connections to the sidecar rather than the target destination, leaving the sidecar in charge of what to do. AppSwitch would interject metadata about the original destination etc. into the data stream of the connection using the proxy protocol that the sidecar could decode before passing the connection through to the application. Some of these details were discussed <a href="/v1.1/blog/2018/delayering-istio/">here</a>. That takes care of outbound connections but what about incoming connections? With all services and their sidecars running under AppSwitch, any incoming connections that would have come from remote nodes would be redirected to their respective remote sidecars. So nothing special to do about incoming connections.</p>
|
||
<h2 id="summary">Summary</h2>
|
||
<p>Dependency ordering is a pesky problem. This is mostly due to lack of access to fine-grain application-level events around inter-service interactions. Addressing this problem would have normally required applications to implement their own internal logic. But AppSwitch makes those internal application events to be instrumented without requiring application changes. AppSwitch then leverages the ubiquitous support for the BSD socket API to sidestep the requirement of ordering dependencies.</p>
|
||
<h2 id="acknowledgements">Acknowledgements</h2>
|
||
<p>Thanks to Eric Herness and team for their insights and support with IBM WebSphere and BPM products as we modernized them onto the Kubernetes platform and to Mandar Jog, Martin Taillefer and Shriram Rajagopalan for reviewing early drafts of this blog.</p></description><pubDate>Mon, 14 Jan 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/appswitch/</link><author>Dinesh Subhraveti (AppOrbit and Columbia University)</author><guid isPermaLink="true">/v1.1/blog/2019/appswitch/</guid><category>appswitch</category><category>performance</category></item><item><title>Deploy a Custom Ingress Gateway Using Cert-Manager</title><description>
|
||
<p>This post provides instructions to manually create a custom ingress <a href="/v1.1/docs/reference/config/networking/v1alpha3/gateway/">gateway</a> with automatic provisioning of certificates based on cert-manager.</p>
|
||
<p>The creation of custom ingress gateway could be used in order to have different <code>loadbalancer</code> in order to isolate traffic.</p>
|
||
<h2 id="before-you-begin">Before you begin</h2>
|
||
<ul>
|
||
<li>Setup Istio by following the instructions in the
|
||
<a href="/v1.1/docs/setup/">Installation guide</a>.</li>
|
||
<li>Setup <code>cert-manager</code> with helm <a href="https://github.com/helm/charts/tree/master/stable/cert-manager#installing-the-chart">chart</a></li>
|
||
<li>We will use <code>demo.mydemo.com</code> for our example,
|
||
it must be resolved with your DNS</li>
|
||
</ul>
|
||
<h2 id="configuring-the-custom-ingress-gateway">Configuring the custom ingress gateway</h2>
|
||
<ol>
|
||
<li><p>Check if <a href="https://github.com/helm/charts/tree/master/stable/cert-manager">cert-manager</a> was installed using Helm with the following command:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ helm ls
|
||
</code></pre>
|
||
<p>The output should be similar to the example below and show cert-manager with a <code>STATUS</code> of <code>DEPLOYED</code>:</p>
|
||
<pre><code class='language-plain' data-expandlinks='true' >NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
|
||
istio 1 Thu Oct 11 13:34:24 2018 DEPLOYED istio-1.0.X 1.0.X istio-system
|
||
cert 1 Wed Oct 24 14:08:36 2018 DEPLOYED cert-manager-v0.6.0-dev.2 v0.6.0-dev.2 istio-system
|
||
</code></pre></li>
|
||
<li><p>To create the cluster&rsquo;s issuer, apply the following configuration:</p>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">Change the cluster&rsquo;s <a href="https://cert-manager.readthedocs.io/en/latest/reference/issuers.html#issuers">issuer</a> provider with your own configuration values. The example uses the values under <code>route53</code>.</div>
|
||
</aside>
|
||
</div>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: certmanager.k8s.io/v1alpha1
|
||
kind: ClusterIssuer
|
||
metadata:
|
||
name: letsencrypt-demo
|
||
namespace: kube-system
|
||
spec:
|
||
acme:
|
||
# The ACME server URL
|
||
server: https://acme-v02.api.letsencrypt.org/directory
|
||
# Email address used for ACME registration
|
||
email: &lt;REDACTED&gt;
|
||
# Name of a secret used to store the ACME account private key
|
||
privateKeySecretRef:
|
||
name: letsencrypt-demo
|
||
dns01:
|
||
# Here we define a list of DNS-01 providers that can solve DNS challenges
|
||
providers:
|
||
- name: your-dns
|
||
route53:
|
||
accessKeyID: &lt;REDACTED&gt;
|
||
region: eu-central-1
|
||
secretAccessKeySecretRef:
|
||
name: prod-route53-credentials-secret
|
||
key: secret-access-key
|
||
</code></pre></li>
|
||
<li><p>If you use the <code>route53</code> <a href="https://cert-manager.readthedocs.io/en/latest/tasks/acme/configuring-dns01/route53.html">provider</a>, you must provide a secret to perform DNS ACME Validation. To create the secret, apply the following configuration file:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: v1
|
||
kind: Secret
|
||
metadata:
|
||
name: prod-route53-credentials-secret
|
||
type: Opaque
|
||
data:
|
||
secret-access-key: &lt;REDACTED BASE64&gt;
|
||
</code></pre></li>
|
||
<li><p>Create your own certificate:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: certmanager.k8s.io/v1alpha1
|
||
kind: Certificate
|
||
metadata:
|
||
name: demo-certificate
|
||
namespace: istio-system
|
||
spec:
|
||
acme:
|
||
config:
|
||
- dns01:
|
||
provider: your-dns
|
||
domains:
|
||
- &#39;*.mydemo.com&#39;
|
||
commonName: &#39;*.mydemo.com&#39;
|
||
dnsNames:
|
||
- &#39;*.mydemo.com&#39;
|
||
issuerRef:
|
||
kind: ClusterIssuer
|
||
name: letsencrypt-demo
|
||
secretName: istio-customingressgateway-certs
|
||
</code></pre>
|
||
<p>Make a note of the value of <code>secretName</code> since a future step requires it.</p></li>
|
||
<li><p>To scale automatically, declare a new horizontal pod autoscaler with the following configuration:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: autoscaling/v1
|
||
kind: HorizontalPodAutoscaler
|
||
metadata:
|
||
name: my-ingressgateway
|
||
namespace: istio-system
|
||
spec:
|
||
maxReplicas: 5
|
||
minReplicas: 1
|
||
scaleTargetRef:
|
||
apiVersion: apps/v1beta1
|
||
kind: Deployment
|
||
name: my-ingressgateway
|
||
targetCPUUtilizationPercentage: 80
|
||
status:
|
||
currentCPUUtilizationPercentage: 0
|
||
currentReplicas: 1
|
||
desiredReplicas: 1
|
||
</code></pre></li>
|
||
<li><p>Apply your deployment with declaration provided in the <a href="/v1.1/blog/2019/custom-ingress-gateway/deployment-custom-ingress.yaml">yaml definition</a></p>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">The annotations used, for example <code>aws-load-balancer-type</code>, only apply for AWS.</div>
|
||
</aside>
|
||
</div>
|
||
</li>
|
||
<li><p>Create your service:</p>
|
||
<div>
|
||
<aside class="callout warning">
|
||
<div class="type">
|
||
<svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-warning"/></svg>
|
||
</div>
|
||
<div class="content">The <code>NodePort</code> used needs to be an available port.</div>
|
||
</aside>
|
||
</div>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: my-ingressgateway
|
||
annotations:
|
||
service.beta.kubernetes.io/aws-load-balancer-type: nlb
|
||
labels:
|
||
app: my-ingressgateway
|
||
istio: my-ingressgateway
|
||
spec:
|
||
type: LoadBalancer
|
||
selector:
|
||
app: my-ingressgateway
|
||
istio: my-ingressgateway
|
||
ports:
|
||
-
|
||
name: http2
|
||
nodePort: 32380
|
||
port: 80
|
||
targetPort: 80
|
||
-
|
||
name: https
|
||
nodePort: 32390
|
||
port: 443
|
||
-
|
||
name: tcp
|
||
nodePort: 32400
|
||
port: 31400
|
||
</code></pre></li>
|
||
<li><p>Create your Istio custom gateway configuration object:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: Gateway
|
||
metadata:
|
||
annotations:
|
||
name: istio-custom-gateway
|
||
namespace: default
|
||
spec:
|
||
selector:
|
||
istio: my-ingressgateway
|
||
servers:
|
||
- hosts:
|
||
- &#39;*.mydemo.com&#39;
|
||
port:
|
||
name: http
|
||
number: 80
|
||
protocol: HTTP
|
||
tls:
|
||
httpsRedirect: true
|
||
- hosts:
|
||
- &#39;*.mydemo.com&#39;
|
||
port:
|
||
name: https
|
||
number: 443
|
||
protocol: HTTPS
|
||
tls:
|
||
mode: SIMPLE
|
||
privateKey: /etc/istio/ingressgateway-certs/tls.key
|
||
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
|
||
</code></pre></li>
|
||
<li><p>Link your <code>istio-custom-gateway</code> with your <code>VirtualService</code>:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: my-virtualservice
|
||
spec:
|
||
hosts:
|
||
- &#34;demo.mydemo.com&#34;
|
||
gateways:
|
||
- istio-custom-gateway
|
||
http:
|
||
- route:
|
||
- destination:
|
||
host: my-demoapp
|
||
</code></pre></li>
|
||
<li><p>Correct certificate is returned by the server and it is successfully verified (<em>SSL certificate verify ok</em> is printed):</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ curl -v `https://demo.mydemo.com`
|
||
Server certificate:
|
||
SSL certificate verify ok.
|
||
</code></pre></li>
|
||
</ol>
|
||
<p><strong>Congratulations!</strong> You can now use your custom <code>istio-custom-gateway</code> <a href="/v1.1/docs/reference/config/networking/v1alpha3/gateway/">gateway</a> configuration object.</p></description><pubDate>Thu, 10 Jan 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/custom-ingress-gateway/</link><author>Julien Senon</author><guid isPermaLink="true">/v1.1/blog/2019/custom-ingress-gateway/</guid><category>ingress</category><category>traffic-management</category></item><item><title>Announcing discuss.istio.io</title><description><p>We in the Istio community have been working to find the right medium for users to engage with other members of the community &ndash; to ask questions,
|
||
to get help from other users, and to engage with developers working on the project.</p>
|
||
<p>We’ve tried several different avenues, but each has had some downsides. RocketChat was our most recent endeavor, but the lack of certain
|
||
features (for example, threading) meant it wasn’t ideal for any longer discussions around a single issue. It also led to a dilemma for
|
||
some users &ndash; when should I email istio-users@googlegroups.com and when should I use RocketChat?</p>
|
||
<p>We think we’ve found the right balance of features in a single platform, and we’re happy to announce
|
||
<a href="https://discuss.istio.io">discuss.istio.io</a>. It’s a full-featured forum where we will have discussions about Istio from here on out.
|
||
It will allow you to ask a question and get threaded replies! As a real bonus, you can use your GitHub identity.</p>
|
||
<p>If you prefer emails, you can configure it to send emails just like Google groups did.</p>
|
||
<p>We will be marking our Google groups &ldquo;read only&rdquo; so that the content remains, but we ask you to send further questions over to
|
||
<a href="https://discuss.istio.io">discuss.istio.io</a>. If you have any outstanding questions or discussions in the groups, please move the conversation over.</p>
|
||
<p>Happy meshing!</p></description><pubDate>Thu, 10 Jan 2019 00:00:00 +0000</pubDate><link>/v1.1/blog/2019/announcing-discuss.istio.io/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2019/announcing-discuss.istio.io/</guid></item><item><title>Announcing Istio 1.0.5</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.0.5. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.0.5"
|
||
data-updateadvice='Before you download 1.0.5, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.0.8'
|
||
data-updatehref="/v1.1/about/notes/1.0.8">
|
||
DOWNLOAD 1.0.5
|
||
</button>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0.5 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.0.4...1.0.5">CHANGES IN 1.0.5</a>
|
||
</div>
|
||
<h2 id="general">General</h2>
|
||
<ul>
|
||
<li><p>Disabled the precondition cache in the <code>istio-policy</code> service as it lead to invalid results. The
|
||
cache will be reintroduced in a later release.</p></li>
|
||
<li><p>Mixer now only generates spans when there is an enabled <code>tracespan</code> adapter, resulting in lower CPU overhead in normal cases.</p></li>
|
||
<li><p>Fixed a problem that could lead Pilot to hang.</p></li>
|
||
</ul></description><pubDate>Thu, 20 Dec 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/announcing-1.0.5/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2018/announcing-1.0.5/</guid></item><item><title>Incremental Istio Part 1, Traffic Management</title><description>
|
||
<p>Traffic management is one of the critical benefits provided by Istio. At the heart of Istio’s traffic management is the ability to decouple traffic flow and infrastructure scaling. This lets you control your traffic in ways that aren’t possible without a service mesh like Istio.</p>
|
||
<p>For example, let’s say you want to execute a <a href="https://martinfowler.com/bliki/CanaryRelease.html">canary deployment</a>. With Istio, you can specify that <strong>v1</strong> of a service receives 90% of incoming traffic, while <strong>v2</strong> of that service only receives 10%. With standard Kubernetes deployments, the only way to achieve this is to manually control the number of available Pods for each version, for example 9 Pods running v1 and 1 Pod running v2. This type of manual control is hard to implement, and over time may have trouble scaling. For more information, check out <a href="/v1.1/blog/2017/0.1-canary/">Canary Deployments using Istio</a>.</p>
|
||
<p>The same issue exists when deploying updates to existing services. While you can update deployments with Kubernetes, it requires replacing v1 Pods with v2 Pods. Using Istio, you can deploy v2 of your service and use built-in traffic management mechanisms to shift traffic to your updated services at a network level, then remove the v1 Pods.</p>
|
||
<p>In addition to canary deployments and general traffic shifting, Istio also gives you the ability to implement dynamic request routing (based on HTTP headers), failure recovery, retries, circuit breakers, and fault injection. For more information, check out the <a href="/v1.1/docs/concepts/traffic-management/">Traffic Management documentation</a>.</p>
|
||
<p>This post walks through a technique that highlights a particularly useful way that you can implement Istio incrementally &ndash; in this case, only the traffic management features &ndash; without having to individually update each of your Pods.</p>
|
||
<h2 id="setup-why-implement-istio-traffic-management-features">Setup: why implement Istio traffic management features?</h2>
|
||
<p>Of course, the first question is: Why would you want to do this?</p>
|
||
<p>If you’re part of one of the many organizations out there that have a large cluster with lots of teams deploying, the answer is pretty clear. Let’s say Team A is getting started with Istio and wants to start some canary deployments on Service A, but Team B hasn’t started using Istio, so they don’t have sidecars deployed.</p>
|
||
<p>With Istio, Team A can still implement their canaries by having Service B call Service A through Istio’s ingress gateway.</p>
|
||
<h2 id="background-traffic-routing-in-an-istio-mesh">Background: traffic routing in an Istio mesh</h2>
|
||
<p>But how can you use Istio’s traffic management capabilities without updating each of your applications’ Pods to include the Istio sidecar? Before answering that question, let’s take a quick high-level look at how traffic enters an Istio mesh and how it’s routed.</p>
|
||
<p>Pods that are part of the Istio mesh contain a sidecar proxy that is responsible for mediating all inbound and outbound traffic to the Pod. Within an Istio mesh, Pilot is responsible for converting high-level routing rules into configurations and propagating them to the sidecar proxies. That means when services communicate with one another, their routing decisions are determined from the client side.</p>
|
||
<p>Let’s say you have two services that are part of the Istio mesh, Service A and Service B. When A wants to communicate with B, the sidecar proxy of Pod A is responsible for directing traffic to Service B. For example, if you wanted to split traffic <sup>50</sup>&frasl;<sub>50</sub> across Service B v1 and v2, the traffic would flow as follows:</p>
|
||
<figure style="width:60%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:42.66666666666667%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/incremental-traffic-management/./fifty-fifty.png" title="50/50 Traffic Split">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/incremental-traffic-management/./fifty-fifty.png" alt="50/50 Traffic Split" />
|
||
</a>
|
||
</div>
|
||
<figcaption>50/50 Traffic Split</figcaption>
|
||
</figure>
|
||
<p>If Services A and B are not part of the Istio mesh, there is no sidecar proxy that knows how to route traffic to different versions of Service B. In that case you need to use another approach to get traffic from Service A to Service B, following the <sup>50</sup>&frasl;<sub>50</sub> rules you’ve setup.</p>
|
||
<p>Fortunately, a standard Istio deployment already includes a <a href="/v1.1/docs/concepts/traffic-management/#gateways">Gateway</a> that specifically deals with ingress traffic outside of the Istio mesh. This Gateway is used to allow ingress traffic from outside the cluster via an external load balancer, or to allow ingress traffic from within the Kubernetes cluster but outside the service mesh. It can be configured to proxy incoming ingress traffic to the appropriate Pods, even if they don’t have a sidecar proxy. While this approach allows you to leverage Istio’s traffic management features, it does mean that traffic going through the ingress gateway will incur an extra hop.</p>
|
||
<figure style="width:60%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:54.83870967741935%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/incremental-traffic-management/./fifty-fifty-ingress-gateway.png" title="50/50 Traffic Split using Ingress Gateway">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/incremental-traffic-management/./fifty-fifty-ingress-gateway.png" alt="50/50 Traffic Split using Ingress Gateway" />
|
||
</a>
|
||
</div>
|
||
<figcaption>50/50 Traffic Split using Ingress Gateway</figcaption>
|
||
</figure>
|
||
<h2 id="in-action-traffic-routing-with-istio">In action: traffic routing with Istio</h2>
|
||
<p>A simple way to see this type of approach in action is to first setup your Kubernetes environment using the <a href="/v1.1/docs/setup/kubernetes/prepare/platform-setup/">Platform Setup</a> instructions, and then install the <strong>minimal</strong> Istio profile using <a href="/v1.1/docs/setup/kubernetes/install/helm/">Helm</a>, including only the traffic management components (ingress gateway, egress gateway, Pilot). The following example uses <a href="https://cloud.google.com/gke">Google Kubernetes Engine</a>.</p>
|
||
<p>First, setup and configure <a href="/v1.1/docs/setup/kubernetes/prepare/platform-setup/gke/">GKE</a>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ gcloud container clusters create istio-inc --zone us-central1-f
|
||
$ gcloud container clusters get-credentials istio-inc
|
||
$ kubectl create clusterrolebinding cluster-admin-binding \
|
||
--clusterrole=cluster-admin \
|
||
--user=$(gcloud config get-value core/account)
|
||
</code></pre>
|
||
<p>Next, <a href="https://helm.sh/docs/securing_installation/">install Helm</a> and <a href="/v1.1/docs/setup/kubernetes/install/helm/">generate a minimal Istio install</a> &ndash; only traffic management components:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ helm template install/kubernetes/helm/istio \
|
||
--name istio \
|
||
--namespace istio-system \
|
||
--set security.enabled=false \
|
||
--set galley.enabled=false \
|
||
--set sidecarInjectorWebhook.enabled=false \
|
||
--set mixer.enabled=false \
|
||
--set prometheus.enabled=false \
|
||
--set pilot.sidecar=false &gt; istio-minimal.yaml
|
||
</code></pre>
|
||
<p>Then create the <code>istio-system</code> namespace and deploy Istio:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl create namespace istio-system
|
||
$ kubectl apply -f istio-minimal.yaml
|
||
</code></pre>
|
||
<p>Next, deploy the Bookinfo sample without the Istio sidecar containers:</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo.yaml@
|
||
</code></pre></div>
|
||
<p>Now, configure a new Gateway that allows access to the reviews service from outside the Istio mesh, a new <code>VirtualService</code> that splits traffic evenly between v1 and v2 of the reviews service, and a set of new <code>DestinationRule</code> resources that match destination subsets to service versions:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | kubectl apply -f -
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: Gateway
|
||
metadata:
|
||
name: reviews-gateway
|
||
spec:
|
||
selector:
|
||
istio: ingressgateway # use istio default controller
|
||
servers:
|
||
- port:
|
||
number: 80
|
||
name: http
|
||
protocol: HTTP
|
||
hosts:
|
||
- &#34;*&#34;
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: reviews
|
||
spec:
|
||
hosts:
|
||
- &#34;*&#34;
|
||
gateways:
|
||
- reviews-gateway
|
||
http:
|
||
- match:
|
||
- uri:
|
||
prefix: /reviews
|
||
route:
|
||
- destination:
|
||
host: reviews
|
||
subset: v1
|
||
weight: 50
|
||
- destination:
|
||
host: reviews
|
||
subset: v2
|
||
weight: 50
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: reviews
|
||
spec:
|
||
host: reviews
|
||
subsets:
|
||
- name: v1
|
||
labels:
|
||
version: v1
|
||
- name: v2
|
||
labels:
|
||
version: v2
|
||
- name: v3
|
||
labels:
|
||
version: v3
|
||
EOF
|
||
</code></pre>
|
||
<p>Finally, deploy a pod that you can use for testing with <code>curl</code> (and without the Istio sidecar container):</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/sleep/sleep.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/sleep/sleep.yaml@
|
||
</code></pre></div>
|
||
<h2 id="testing-your-deployment">Testing your deployment</h2>
|
||
<p>Now, you can test different behaviors using the <code>curl</code> commands via the sleep Pod.</p>
|
||
<p>The first example is to issue requests to the reviews service using standard Kubernetes service DNS behavior (<strong>note</strong>: <a href="https://stedolan.github.io/jq/"><code>jq</code></a> is used in the examples below to filter the output from <code>curl</code>):</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export SLEEP_POD=$(kubectl get pod -l app=sleep \
|
||
-o jsonpath={.items..metadata.name})
|
||
$ for i in `seq 3`; do \
|
||
kubectl exec -it $SLEEP_POD curl http://reviews:9080/reviews/0 | \
|
||
jq &#39;.reviews|.[]|.rating?&#39;; \
|
||
done
|
||
</code></pre>
|
||
<pre><code class='language-json' data-expandlinks='true' >{
|
||
&#34;stars&#34;: 5,
|
||
&#34;color&#34;: &#34;black&#34;
|
||
}
|
||
{
|
||
&#34;stars&#34;: 4,
|
||
&#34;color&#34;: &#34;black&#34;
|
||
}
|
||
null
|
||
null
|
||
{
|
||
&#34;stars&#34;: 5,
|
||
&#34;color&#34;: &#34;red&#34;
|
||
}
|
||
{
|
||
&#34;stars&#34;: 4,
|
||
&#34;color&#34;: &#34;red&#34;
|
||
}
|
||
</code></pre>
|
||
<p>Notice how we’re getting responses from all three versions of the reviews service (<code>null</code> is from reviews v1 which doesn’t have ratings) and not getting the even split across v1 and v2. This is expected behavior because the <code>curl</code> command is using Kubernetes service load balancing across all three versions of the reviews service. In order to access the reviews <sup>50</sup>&frasl;<sub>50</sub> split we need to access the service via the ingress Gateway:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ for i in `seq 4`; do \
|
||
kubectl exec -it $SLEEP_POD curl http://istio-ingressgateway.istio-system/reviews/0 | \
|
||
jq &#39;.reviews|.[]|.rating?&#39;; \
|
||
done
|
||
</code></pre>
|
||
<pre><code class='language-json' data-expandlinks='true' >{
|
||
&#34;stars&#34;: 5,
|
||
&#34;color&#34;: &#34;black&#34;
|
||
}
|
||
{
|
||
&#34;stars&#34;: 4,
|
||
&#34;color&#34;: &#34;black&#34;
|
||
}
|
||
null
|
||
null
|
||
{
|
||
&#34;stars&#34;: 5,
|
||
&#34;color&#34;: &#34;black&#34;
|
||
}
|
||
{
|
||
&#34;stars&#34;: 4,
|
||
&#34;color&#34;: &#34;black&#34;
|
||
}
|
||
null
|
||
null
|
||
</code></pre>
|
||
<p>Mission accomplished! This post showed how to deploy a minimal installation of Istio that only contains the traffic management components (Pilot, ingress Gateway), and then use those components to direct traffic to specific versions of the reviews service. And it wasn&rsquo;t necessary to deploy the Istio sidecar proxy to gain these capabilities, so there was little to no interruption of existing workloads or applications.</p>
|
||
<p>Using the built-in ingress gateway (along with some <code>VirtualService</code> and <code>DestinationRule</code> resources) this post showed how you can easily leverage Istio’s traffic management for cluster-external ingress traffic and cluster-internal service-to-service traffic. This technique is a great example of an incremental approach to adopting Istio, and can be especially useful in real-world cases where Pods are owned by different teams or deployed to different namespaces.</p></description><pubDate>Wed, 21 Nov 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/incremental-traffic-management/</link><author>Sandeep Parikh</author><guid isPermaLink="true">/v1.1/blog/2018/incremental-traffic-management/</guid><category>traffic-management</category><category>gateway</category></item><item><title>Announcing Istio 1.0.4</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.0.4. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.0.4"
|
||
data-updateadvice='Before you download 1.0.4, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.0.8'
|
||
data-updatehref="/v1.1/about/notes/1.0.8">
|
||
DOWNLOAD 1.0.4
|
||
</button>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0.4 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.0.3...1.0.4">CHANGES IN 1.0.4</a>
|
||
</div>
|
||
<h2 id="known-issues">Known issues</h2>
|
||
<ul>
|
||
<li>Pilot may deadlock when using <a href="/v1.1/docs/reference/commands/istioctl/#istioctl-proxy-status"><code>istioctl proxy-status</code></a> to get proxy synchronization status.
|
||
The work around is to <em>not use</em> <code>istioctl proxy-status</code>.
|
||
Once Pilot enters a deadlock, it exhibits continuous goroutine growth eventually running out of memory.</li>
|
||
</ul>
|
||
<h2 id="networking">Networking</h2>
|
||
<ul>
|
||
<li><p>Fixed the lack of removal of stale endpoints causing <code>503</code> errors.</p></li>
|
||
<li><p>Fixed sidecar injection when a pod label contains a <code>/</code>.</p></li>
|
||
</ul>
|
||
<h2 id="policy-and-telemetry">Policy and telemetry</h2>
|
||
<ul>
|
||
<li><p>Fixed occasional data corruption problem with out-of-process Mixer adapters leading to incorrect behavior.</p></li>
|
||
<li><p>Fixed excessive CPU usage by Mixer when waiting for missing CRDs.</p></li>
|
||
</ul></description><pubDate>Wed, 21 Nov 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/announcing-1.0.4/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2018/announcing-1.0.4/</guid></item><item><title>Consuming External MongoDB Services</title><description>
|
||
<p>In the <a href="/v1.1/blog/2018/egress-tcp/">Consuming External TCP Services</a> blog post, I described how external services
|
||
can be consumed by in-mesh Istio applications via TCP. In this post, I demonstrate consuming external MongoDB services.
|
||
You use the <a href="/v1.1/docs/examples/bookinfo/">Istio Bookinfo sample application</a>, the version in which the book
|
||
ratings data is persisted in a MongoDB database. You deploy this database outside the cluster and configure the
|
||
<em>ratings</em> microservice to use it. You will learn multiple options of controlling traffic to external MongoDB services and their
|
||
pros and cons.</p>
|
||
<h2 id="bookinfo-with-external-ratings-database">Bookinfo with external ratings database</h2>
|
||
<p>First, you set up a MongoDB database instance to hold book ratings data outside of your Kubernetes cluster. Then you
|
||
modify the <a href="/v1.1/docs/examples/bookinfo/">Bookinfo sample application</a> to use your database.</p>
|
||
<h3 id="setting-up-the-ratings-database">Setting up the ratings database</h3>
|
||
<p>For this task you set up an instance of <a href="https://www.mongodb.com">MongoDB</a>. You can use any MongoDB instance; I used
|
||
<a href="https://www.ibm.com/cloud/compose/mongodb">Compose for MongoDB</a>.</p>
|
||
<ol>
|
||
<li><p>Set an environment variable for the password of your <code>admin</code> user. To prevent the password from being preserved in
|
||
the Bash history, remove the command from the history immediately after running the command, using
|
||
<a href="https://www.gnu.org/software/bash/manual/html_node/Bash-History-Builtins.html#Bash-History-Builtins">history -d</a>.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export MONGO_ADMIN_PASSWORD=&lt;your MongoDB admin password&gt;
|
||
</code></pre></li>
|
||
<li><p>Set an environment variable for the password of the new user you will create, namely <code>bookinfo</code>.
|
||
Remove the command from the history using
|
||
<a href="https://www.gnu.org/software/bash/manual/html_node/Bash-History-Builtins.html#Bash-History-Builtins">history -d</a>.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export BOOKINFO_PASSWORD=&lt;password&gt;
|
||
</code></pre></li>
|
||
<li><p>Set environment variables for your MongoDB service, <code>MONGODB_HOST</code> and <code>MONGODB_PORT</code>.</p></li>
|
||
<li><p>Create the <code>bookinfo</code> user:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u admin -p $MONGO_ADMIN_PASSWORD --authenticationDatabase admin
|
||
use test
|
||
db.createUser(
|
||
{
|
||
user: &#34;bookinfo&#34;,
|
||
pwd: &#34;$BOOKINFO_PASSWORD&#34;,
|
||
roles: [ &#34;read&#34;]
|
||
}
|
||
);
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Create a <em>collection</em> to hold ratings. The following command sets both ratings to be equal <code>1</code> to provide a visual
|
||
clue when your database is used by the Bookinfo <em>ratings</em> service (the default Bookinfo <em>ratings</em> are <code>4</code> and <code>5</code>).</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u admin -p $MONGO_ADMIN_PASSWORD --authenticationDatabase admin
|
||
use test
|
||
db.createCollection(&#34;ratings&#34;);
|
||
db.ratings.insert(
|
||
[{rating: 1},
|
||
{rating: 1}]
|
||
);
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Check that <code>bookinfo</code> user can get ratings:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u bookinfo -p $BOOKINFO_PASSWORD --authenticationDatabase test
|
||
use test
|
||
db.ratings.find({});
|
||
EOF
|
||
</code></pre>
|
||
<p>The output should be similar to:</p>
|
||
<pre><code class='language-plain' data-expandlinks='true' >MongoDB server version: 3.4.10
|
||
switched to db test
|
||
{ &#34;_id&#34; : ObjectId(&#34;5b7c29efd7596e65b6ed2572&#34;), &#34;rating&#34; : 1 }
|
||
{ &#34;_id&#34; : ObjectId(&#34;5b7c29efd7596e65b6ed2573&#34;), &#34;rating&#34; : 1 }
|
||
bye
|
||
</code></pre></li>
|
||
</ol>
|
||
<h3 id="initial-setting-of-bookinfo-application">Initial setting of Bookinfo application</h3>
|
||
<p>To demonstrate the scenario of using an external database, you start with a Kubernetes cluster with <a href="/v1.1/docs/setup/kubernetes/install/kubernetes/#installation-steps">Istio installed</a>. Then you deploy the
|
||
<a href="/v1.1/docs/examples/bookinfo/">Istio Bookinfo sample application</a>, <a href="/v1.1/docs/examples/bookinfo/#apply-default-destination-rules">apply the default destination rules</a>, and
|
||
<a href="/v1.1/docs/tasks/traffic-management/egress/#change-to-the-blocking-by-default-policy">change Istio to the blocking-egress-by-default policy</a>.</p>
|
||
<p>This application uses the <code>ratings</code> microservice to fetch book ratings, a number between 1 and 5. The ratings are
|
||
displayed as stars for each review. There are several versions of the <code>ratings</code> microservice. You will deploy the
|
||
version that uses <a href="https://www.mongodb.com">MongoDB</a> as the ratings database in the next subsection.</p>
|
||
<p>The example commands in this blog post work with Istio 1.0.</p>
|
||
<p>As a reminder, here is the end-to-end architecture of the application from the
|
||
<a href="/v1.1/docs/examples/bookinfo/">Bookinfo sample application</a>.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.086918235567985%">
|
||
<a data-skipendnotes="true" href="/v1.1/docs/examples/bookinfo/withistio.svg" title="The original Bookinfo application">
|
||
<img class="element-to-stretch" src="/v1.1/docs/examples/bookinfo/withistio.svg" alt="The original Bookinfo application" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The original Bookinfo application</figcaption>
|
||
</figure>
|
||
<h3 id="use-the-external-database-in-bookinfo-application">Use the external database in Bookinfo application</h3>
|
||
<ol>
|
||
<li><p>Deploy the spec of the <em>ratings</em> microservice that uses a MongoDB database (<em>ratings v2</em>), while setting
|
||
<code>MONGO_DB_URL</code> environment variable of the spec:</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml@ --dry-run -o yaml | kubectl set env --local -f - &#34;MONGO_DB_URL=mongodb://bookinfo:$BOOKINFO_PASSWORD@$MONGODB_HOST:$MONGODB_PORT/test?authSource=test&amp;ssl=true&#34; -o yaml | kubectl apply -f -
|
||
deployment &#34;ratings-v2&#34; created
|
||
</code></pre></div></li>
|
||
<li><p>Route all the traffic destined to the <em>reviews</em> service to its <em>v3</em> version. You do this to ensure that the
|
||
<em>reviews</em> service always calls the <em>ratings</em> service. In addition, route all the traffic destined to the <em>ratings</em>
|
||
service to <em>ratings v2</em> that uses your database.</p>
|
||
<p>Specify the routing for both services above by adding two
|
||
<a href="/v1.1/docs/reference/config/networking/v1alpha3/virtual-service/">virtual services</a>. These virtual services are
|
||
specified in <code>samples/bookinfo/networking/virtual-service-ratings-mongodb.yaml</code> of an Istio release archive.
|
||
<strong><em>Important:</em></strong> make sure you
|
||
<a href="/v1.1/docs/examples/bookinfo/#apply-default-destination-rules">applied the default destination rules</a> before running the
|
||
following command.</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/virtual-service-ratings-db.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/networking/virtual-service-ratings-db.yaml@
|
||
</code></pre></div></li>
|
||
</ol>
|
||
<p>The updated architecture appears below. Note that the blue arrows inside the mesh mark the traffic configured according
|
||
to the virtual services we added. According to the virtual services, the traffic is sent to <em>reviews v3</em> and
|
||
<em>ratings v2</em>.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.314858206480224%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-mongo/./bookinfo-ratings-v2-mongodb-external.svg" title="The Bookinfo application with ratings v2 and an external MongoDB database">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-mongo/./bookinfo-ratings-v2-mongodb-external.svg" alt="The Bookinfo application with ratings v2 and an external MongoDB database" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The Bookinfo application with ratings v2 and an external MongoDB database</figcaption>
|
||
</figure>
|
||
<p>Note that the MongoDB database is outside the Istio service mesh, or more precisely outside the Kubernetes cluster. The
|
||
boundary of the service mesh is marked by a dashed line.</p>
|
||
<h3 id="access-the-webpage">Access the webpage</h3>
|
||
<p>Access the webpage of the application, after
|
||
<a href="/v1.1/docs/examples/bookinfo/#determining-the-ingress-ip-and-port">determining the ingress IP and port</a>.</p>
|
||
<p>Since you did not configure the egress traffic control yet, the access to the MongoDB service is blocked by Istio.
|
||
This is why instead of the rating stars, the message <em>&ldquo;Ratings service is currently unavailable&rdquo;</em> is currently
|
||
displayed below each review:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.18705035971223%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-mongo/./errorFetchingBookRating.png" title="The Ratings service error messages">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-mongo/./errorFetchingBookRating.png" alt="The Ratings service error messages" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The Ratings service error messages</figcaption>
|
||
</figure>
|
||
<p>In the following sections you will configure egress access to the external MongoDB service, using different options for
|
||
egress control in Istio.</p>
|
||
<h2 id="egress-control-for-tcp">Egress control for TCP</h2>
|
||
<p>Since <a href="https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/">MongoDB Wire Protocol</a> runs on top of TCP, you
|
||
can control the egress traffic to your MongoDB as traffic to any other <a href="/v1.1/blog/2018/egress-tcp/">external TCP service</a>. To
|
||
control TCP traffic, a block of IPs in the <a href="https://tools.ietf.org/html/rfc2317">CIDR</a> notation that includes the IP
|
||
address of your MongoDB host must be specified. The caveat here is that sometimes the IP of the MongoDB host is not
|
||
stable or known in advance.</p>
|
||
<p>In the cases when the IP of the MongoDB host is not stable, the egress traffic can either be
|
||
<a href="#egress-control-for-tls">controlled as TLS traffic</a>, or the traffic can be routed
|
||
<a href="/v1.1/docs/tasks/traffic-management/egress/#direct-access-to-external-services">directly</a>, bypassing the Istio sidecar
|
||
proxies.</p>
|
||
<p>Get the IP address of your MongoDB database instance. As an option, you can use the
|
||
<a href="https://linux.die.net/man/1/host">host</a> command:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export MONGODB_IP=$(host $MONGODB_HOST | grep &#34; has address &#34; | cut -d&#34; &#34; -f4)
|
||
</code></pre>
|
||
<h3 id="control-tcp-egress-traffic-without-a-gateway">Control TCP egress traffic without a gateway</h3>
|
||
<p>In case you do not need to direct the traffic through an
|
||
<a href="/v1.1/docs/examples/advanced-gateways/egress-gateway/#use-case">egress gateway</a>, for example if you do not have a
|
||
requirement that all the traffic that exists your mesh must exit through the gateway, follow the
|
||
instructions in this section. Alternatively, if you do want to direct your traffic through an egress gateway, proceed to
|
||
<a href="#direct-tcp-egress-traffic-through-an-egress-gateway">Direct TCP egress traffic through an egress gateway</a>.</p>
|
||
<ol>
|
||
<li><p>Define a TCP mesh-external service entry:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: mongo
|
||
spec:
|
||
hosts:
|
||
- my-mongo.tcp.svc
|
||
addresses:
|
||
- $MONGODB_IP/32
|
||
ports:
|
||
- number: $MONGODB_PORT
|
||
name: tcp
|
||
protocol: TCP
|
||
location: MESH_EXTERNAL
|
||
resolution: STATIC
|
||
endpoints:
|
||
- address: $MONGODB_IP
|
||
EOF
|
||
</code></pre>
|
||
<p>Note that the protocol <code>TCP</code> is specified instead of <code>MONGO</code> due to the fact that the traffic can be encrypted in
|
||
case <a href="https://docs.mongodb.com/manual/tutorial/configure-ssl/">the MongoDB protocol runs on top of TLS</a>.
|
||
If the traffic is encrypted, the encrypted MongoDB protocol cannot be parsed by the Istio proxy.</p>
|
||
<p>If you know that the plain MongoDB protocol is used, without encryption, you can specify the protocol as <code>MONGO</code> and
|
||
let the Istio proxy produce
|
||
<a href="https://www.envoyproxy.io/docs/envoy/latest/configuration/network_filters/mongo_proxy_filter#statistics">MongoDB related statistics</a>.
|
||
Also note that when the protocol <code>TCP</code> is specified, the configuration is not specific for MongoDB, but is the same
|
||
for any other database with the protocol on top of TCP.</p>
|
||
<p>Note that the host of your MongoDB is not used in TCP routing, so you can use any host, for example <code>my-mongo.tcp.svc</code>. Notice the <code>STATIC</code> resolution and the endpoint with the IP of your MongoDB service. Once you define such an endpoint, you can access MongoDB services that do not have a domain name.</p></li>
|
||
<li><p>Refresh the web page of the application. Now the application should display the ratings without error:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.69064748201439%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-mongo/./externalDBRatings.png" title="Book Ratings Displayed Correctly">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-mongo/./externalDBRatings.png" alt="Book Ratings Displayed Correctly" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Book Ratings Displayed Correctly</figcaption>
|
||
</figure>
|
||
<p>Note that you see a one-star rating for both displayed reviews, as expected. You set the ratings to be one star to
|
||
provide yourself with a visual clue that your external database is indeed being used.</p></li>
|
||
<li><p>If you want to direct the traffic through an egress gateway, proceed to the next section. Otherwise, perform
|
||
<a href="#cleanup-of-tcp-egress-traffic-control">cleanup</a>.</p></li>
|
||
</ol>
|
||
<h3 id="direct-tcp-egress-traffic-through-an-egress-gateway">Direct TCP Egress traffic through an egress gateway</h3>
|
||
<p>In this section you handle the case when you need to direct the traffic through an
|
||
<a href="/v1.1/docs/examples/advanced-gateways/egress-gateway/#use-case">egress gateway</a>. The sidecar proxy routes TCP
|
||
connections from the MongoDB client to the egress gateway, by matching the IP of the MongoDB host (a CIDR block of
|
||
length 32). The egress gateway forwards the traffic to the MongoDB host, by its hostname.</p>
|
||
<ol>
|
||
<li><p><a href="/v1.1/docs/examples/advanced-gateways/egress-gateway/#deploy-istio-egress-gateway">Deploy Istio egress gateway</a>.</p></li>
|
||
<li><p>If you did not perform the steps in <a href="#control-tcp-egress-traffic-without-a-gateway">the previous section</a>, perform them now.</p></li>
|
||
<li><p>Proceed to the following section.</p></li>
|
||
</ol>
|
||
<h4 id="configure-tcp-traffic-from-sidecars-to-the-egress-gateway">Configure TCP traffic from sidecars to the egress gateway</h4>
|
||
<ol>
|
||
<li><p>Define the <code>EGRESS_GATEWAY_MONGODB_PORT</code> environment variable to hold some port for directing traffic through
|
||
the egress gateway, e.g. <code>7777</code>. You must select a port that is not used for any other service in the mesh.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export EGRESS_GATEWAY_MONGODB_PORT=7777
|
||
</code></pre></li>
|
||
<li><p>Add the selected port to the <code>istio-egressgateway</code> service. You should use the same values you used for installing
|
||
Istio, in particular you have to specify all the ports of the <code>istio-egressgateway</code> service that you previously
|
||
configured.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ helm template install/kubernetes/helm/istio/ --name istio-egressgateway --namespace istio-system -x charts/gateways/templates/service.yaml --set gateways.istio-ingressgateway.enabled=false --set gateways.istio-egressgateway.enabled=true --set gateways.istio-egressgateway.ports[0].port=80 --set gateways.istio-egressgateway.ports[0].name=http --set gateways.istio-egressgateway.ports[1].port=443 --set gateways.istio-egressgateway.ports[1].name=https --set gateways.istio-egressgateway.ports[2].port=$EGRESS_GATEWAY_MONGODB_PORT --set gateways.istio-egressgateway.ports[2].name=mongo | kubectl apply -f -
|
||
</code></pre></li>
|
||
<li><p>Check that the <code>istio-egressgateway</code> service indeed has the selected port:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get svc istio-egressgateway -n istio-system
|
||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||
istio-egressgateway ClusterIP 172.21.202.204 &lt;none&gt; 80/TCP,443/TCP,7777/TCP 34d
|
||
</code></pre></li>
|
||
<li><p>Create an egress <code>Gateway</code> for your MongoDB service, and destination rules and a virtual service to direct the
|
||
traffic through the egress gateway and from the egress gateway to the external service.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: Gateway
|
||
metadata:
|
||
name: istio-egressgateway
|
||
spec:
|
||
selector:
|
||
istio: egressgateway
|
||
servers:
|
||
- port:
|
||
number: $EGRESS_GATEWAY_MONGODB_PORT
|
||
name: tcp
|
||
protocol: TCP
|
||
hosts:
|
||
- my-mongo.tcp.svc
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: egressgateway-for-mongo
|
||
spec:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
subsets:
|
||
- name: mongo
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: mongo
|
||
spec:
|
||
host: my-mongo.tcp.svc
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: direct-mongo-through-egress-gateway
|
||
spec:
|
||
hosts:
|
||
- my-mongo.tcp.svc
|
||
gateways:
|
||
- mesh
|
||
- istio-egressgateway
|
||
tcp:
|
||
- match:
|
||
- gateways:
|
||
- mesh
|
||
destinationSubnets:
|
||
- $MONGODB_IP/32
|
||
port: $MONGODB_PORT
|
||
route:
|
||
- destination:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
subset: mongo
|
||
port:
|
||
number: $EGRESS_GATEWAY_MONGODB_PORT
|
||
- match:
|
||
- gateways:
|
||
- istio-egressgateway
|
||
port: $EGRESS_GATEWAY_MONGODB_PORT
|
||
route:
|
||
- destination:
|
||
host: my-mongo.tcp.svc
|
||
port:
|
||
number: $MONGODB_PORT
|
||
weight: 100
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p><a href="#verify-that-egress-traffic-is-directed-through-the-egress-gateway">Verify that egress traffic is directed through the egress gateway</a>.</p></li>
|
||
</ol>
|
||
<h4 id="mutual-tls-between-the-sidecar-proxies-and-the-egress-gateway">Mutual TLS between the sidecar proxies and the egress gateway</h4>
|
||
<p>You may want to enable <a href="/v1.1/docs/tasks/security/mutual-tls/">mutual TLS Authentication</a> between the sidecar proxies of
|
||
your MongoDB clients and the egress gateway to let the egress gateway monitor the identity of the source pods and to
|
||
enable Mixer policy enforcement based on that identity. By enabling mutual TLS you also encrypt the traffic.</p>
|
||
<ol>
|
||
<li><p>Delete the configuration from the previous section:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete gateway istio-egressgateway --ignore-not-found=true
|
||
$ kubectl delete virtualservice direct-mongo-through-egress-gateway --ignore-not-found=true
|
||
$ kubectl delete destinationrule egressgateway-for-mongo mongo --ignore-not-found=true
|
||
</code></pre></li>
|
||
<li><p>Create an egress <code>Gateway</code> for your MongoDB service, and destination rules and a virtual service
|
||
to direct the traffic through the egress gateway and from the egress gateway to the external service.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: Gateway
|
||
metadata:
|
||
name: istio-egressgateway
|
||
spec:
|
||
selector:
|
||
istio: egressgateway
|
||
servers:
|
||
- port:
|
||
number: 443
|
||
name: tls
|
||
protocol: TLS
|
||
hosts:
|
||
- my-mongo.tcp.svc
|
||
tls:
|
||
mode: MUTUAL
|
||
serverCertificate: /etc/certs/cert-chain.pem
|
||
privateKey: /etc/certs/key.pem
|
||
caCertificates: /etc/certs/root-cert.pem
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: egressgateway-for-mongo
|
||
spec:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: ROUND_ROBIN
|
||
portLevelSettings:
|
||
- port:
|
||
number: 443
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
subsets:
|
||
- name: mongo
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: ROUND_ROBIN
|
||
portLevelSettings:
|
||
- port:
|
||
number: 443
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
sni: my-mongo.tcp.svc
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: mongo
|
||
spec:
|
||
host: my-mongo.tcp.svc
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: direct-mongo-through-egress-gateway
|
||
spec:
|
||
hosts:
|
||
- my-mongo.tcp.svc
|
||
gateways:
|
||
- mesh
|
||
- istio-egressgateway
|
||
tcp:
|
||
- match:
|
||
- gateways:
|
||
- mesh
|
||
destinationSubnets:
|
||
- $MONGODB_IP/32
|
||
port: $MONGODB_PORT
|
||
route:
|
||
- destination:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
subset: mongo
|
||
port:
|
||
number: 443
|
||
- match:
|
||
- gateways:
|
||
- istio-egressgateway
|
||
port: 443
|
||
route:
|
||
- destination:
|
||
host: my-mongo.tcp.svc
|
||
port:
|
||
number: $MONGODB_PORT
|
||
weight: 100
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Proceed to the next section.</p></li>
|
||
</ol>
|
||
<h4 id="verify-that-egress-traffic-is-directed-through-the-egress-gateway">Verify that egress traffic is directed through the egress gateway</h4>
|
||
<ol>
|
||
<li><p>Refresh the web page of the application again and verify that the ratings are still displayed correctly.</p></li>
|
||
<li><p><a href="/v1.1/docs/tasks/telemetry/logs/access-log/#enable-envoy-s-access-logging">Enable Envoy’s access logging</a></p></li>
|
||
<li><p>Check the log of the egress gateway&rsquo;s Envoy and see a line that corresponds to your
|
||
requests to the MongoDB service. If Istio is deployed in the <code>istio-system</code> namespace, the command to print the
|
||
log is:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl logs -l istio=egressgateway -n istio-system
|
||
[2019-04-14T06:12:07.636Z] &#34;- - -&#34; 0 - &#34;-&#34; 1591 4393 94 - &#34;-&#34; &#34;-&#34; &#34;-&#34; &#34;-&#34; &#34;&lt;Your MongoDB IP&gt;:&lt;your MongoDB port&gt;&#34; outbound|&lt;your MongoDB port&gt;||my-mongo.tcp.svc 172.30.146.119:59924 172.30.146.119:443 172.30.230.1:59206 -
|
||
</code></pre></li>
|
||
</ol>
|
||
<h3 id="cleanup-of-tcp-egress-traffic-control">Cleanup of TCP egress traffic control</h3>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry mongo
|
||
$ kubectl delete gateway istio-egressgateway --ignore-not-found=true
|
||
$ kubectl delete virtualservice direct-mongo-through-egress-gateway --ignore-not-found=true
|
||
$ kubectl delete destinationrule egressgateway-for-mongo mongo --ignore-not-found=true
|
||
</code></pre>
|
||
<h2 id="egress-control-for-tls">Egress control for TLS</h2>
|
||
<p>In the real life, most of the communication to the external services must be encrypted and
|
||
<a href="https://docs.mongodb.com/manual/tutorial/configure-ssl/">the MongoDB protocol runs on top of TLS</a>.
|
||
Also, the TLS clients usually send
|
||
<a href="https://en.wikipedia.org/wiki/Server_Name_Indication">Server Name Indication</a>, SNI, as part of their handshake. If your
|
||
MongoDB server runs TLS and your MongoDB client sends SNI as part of the handshake, you can control your MongoDB egress
|
||
traffic as any other TLS-with-SNI traffic. With TLS and SNI, you do not need to specify the IP addresses of your MongoDB
|
||
servers. You specify their host names instead, which is more convenient since you do not have to rely on the stability of
|
||
the IP addresses. You can also specify wildcards as a prefix of the host names, for example allowing access to any
|
||
server from the <code>*.com</code> domain.</p>
|
||
<p>To check if your MongoDB server supports TLS, run:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ openssl s_client -connect $MONGODB_HOST:$MONGODB_PORT -servername $MONGODB_HOST
|
||
</code></pre>
|
||
<p>If the command above prints a certificate returned by the server, the server supports TLS. If not, you have to control
|
||
your MongoDB egress traffic on the TCP level, as described in the previous sections.</p>
|
||
<h3 id="control-tls-egress-traffic-without-a-gateway">Control TLS egress traffic without a gateway</h3>
|
||
<p>In case you <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway/#use-case">do not need an egress gateway</a>, follow the
|
||
instructions in this section. If you want to direct your traffic through an egress gateway, proceed to
|
||
<a href="#direct-tcp-egress-traffic-through-an-egress-gateway">Direct TCP Egress traffic through an egress gateway</a>.</p>
|
||
<ol>
|
||
<li><p>Create a <code>ServiceEntry</code> for the MongoDB service:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: mongo
|
||
spec:
|
||
hosts:
|
||
- $MONGODB_HOST
|
||
ports:
|
||
- number: $MONGODB_PORT
|
||
name: tls
|
||
protocol: TLS
|
||
resolution: DNS
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Refresh the web page of the application. The application should display the ratings without error.</p></li>
|
||
</ol>
|
||
<h4 id="cleanup-of-the-egress-configuration-for-tls">Cleanup of the egress configuration for TLS</h4>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry mongo
|
||
</code></pre>
|
||
<h3 id="direct-tls-egress-traffic-through-an-egress-gateway">Direct TLS Egress traffic through an egress gateway</h3>
|
||
<p>In this section you handle the case when you need to direct the traffic through an
|
||
<a href="/v1.1/docs/examples/advanced-gateways/egress-gateway/#use-case">egress gateway</a>. The sidecar proxy routes TLS
|
||
connections from the MongoDB client to the egress gateway, by matching the SNI of the MongoDB host.
|
||
The egress gateway forwards the traffic to the MongoDB host. Note that the sidecar proxy rewrites the destination port
|
||
to be 443. The egress gateway accepts the MongoDB traffic on the port 443, matches the MongoDB host by SNI, and rewrites
|
||
the port again to be the port of the MongoDB server.</p>
|
||
<ol>
|
||
<li><p><a href="/v1.1/docs/examples/advanced-gateways/egress-gateway/#deploy-istio-egress-gateway">Deploy Istio egress gateway</a>.</p></li>
|
||
<li><p>Create a <code>ServiceEntry</code> for the MongoDB service:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: mongo
|
||
spec:
|
||
hosts:
|
||
- $MONGODB_HOST
|
||
ports:
|
||
- number: $MONGODB_PORT
|
||
name: tls
|
||
protocol: TLS
|
||
- number: 443
|
||
name: tls-port-for-egress-gateway
|
||
protocol: TLS
|
||
resolution: DNS
|
||
location: MESH_EXTERNAL
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Refresh the web page of the application and verify that the ratings are displayed correctly.</p></li>
|
||
<li><p>Create an egress <code>Gateway</code> for your MongoDB service, and destination rules and virtual services
|
||
to direct the traffic through the egress gateway and from the egress gateway to the external service.</p>
|
||
<p>If you want to enable <a href="/v1.1/docs/tasks/security/mutual-tls/">mutual TLS Authentication</a> between the sidecar proxies of
|
||
your application pods and the egress gateway, use the following command. (You may want to enable mutual TLS to let
|
||
the egress gateway monitor the identity of the source pods and to enable Mixer policy enforcement based on that
|
||
identity.)</p>
|
||
<div id="tabset-blog-2018-egress-mongo-2" role="tablist" class="tabset">
|
||
<div class="tab-strip" data-cookie-name="mtls"><button aria-selected="true" data-cookie-value="enabled"
|
||
aria-controls="tabset-blog-2018-egress-mongo-2-0-panel" id="tabset-blog-2018-egress-mongo-2-0-tab" role="tab"><span>mutual TLS enabled</span>
|
||
</button><button tabindex="-1" data-cookie-value="disabled"
|
||
aria-controls="tabset-blog-2018-egress-mongo-2-1-panel" id="tabset-blog-2018-egress-mongo-2-1-tab" role="tab"><span>mutual TLS disabled</span>
|
||
</button></div>
|
||
<div class="tab-content"><div id="tabset-blog-2018-egress-mongo-2-0-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2018-egress-mongo-2-0-tab"><pre><code class="language-bash">$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: Gateway
|
||
metadata:
|
||
name: istio-egressgateway
|
||
spec:
|
||
selector:
|
||
istio: egressgateway
|
||
servers:
|
||
- port:
|
||
number: 443
|
||
name: tls
|
||
protocol: TLS
|
||
hosts:
|
||
- $MONGODB_HOST
|
||
tls:
|
||
mode: MUTUAL
|
||
serverCertificate: /etc/certs/cert-chain.pem
|
||
privateKey: /etc/certs/key.pem
|
||
caCertificates: /etc/certs/root-cert.pem
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: egressgateway-for-mongo
|
||
spec:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: ROUND_ROBIN
|
||
portLevelSettings:
|
||
- port:
|
||
number: 443
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
subsets:
|
||
- name: mongo
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: ROUND_ROBIN
|
||
portLevelSettings:
|
||
- port:
|
||
number: 443
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
sni: $MONGODB_HOST
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: direct-mongo-through-egress-gateway
|
||
spec:
|
||
hosts:
|
||
- $MONGODB_HOST
|
||
gateways:
|
||
- mesh
|
||
- istio-egressgateway
|
||
tls:
|
||
- match:
|
||
- gateways:
|
||
- mesh
|
||
port: $MONGODB_PORT
|
||
sni_hosts:
|
||
- $MONGODB_HOST
|
||
route:
|
||
- destination:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
subset: mongo
|
||
port:
|
||
number: 443
|
||
tcp:
|
||
- match:
|
||
- gateways:
|
||
- istio-egressgateway
|
||
port: 443
|
||
route:
|
||
- destination:
|
||
host: $MONGODB_HOST
|
||
port:
|
||
number: $MONGODB_PORT
|
||
weight: 100
|
||
EOF
|
||
</code></pre></div><div hidden id="tabset-blog-2018-egress-mongo-2-1-panel" role="tabpanel" tabindex="0" aria-labelledby="tabset-blog-2018-egress-mongo-2-1-tab"><pre><code class="language-bash">$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: Gateway
|
||
metadata:
|
||
name: istio-egressgateway
|
||
spec:
|
||
selector:
|
||
istio: egressgateway
|
||
servers:
|
||
- port:
|
||
number: 443
|
||
name: tls
|
||
protocol: TLS
|
||
hosts:
|
||
- $MONGODB_HOST
|
||
tls:
|
||
mode: PASSTHROUGH
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: egressgateway-for-mongo
|
||
spec:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
subsets:
|
||
- name: mongo
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: direct-mongo-through-egress-gateway
|
||
spec:
|
||
hosts:
|
||
- $MONGODB_HOST
|
||
gateways:
|
||
- mesh
|
||
- istio-egressgateway
|
||
tls:
|
||
- match:
|
||
- gateways:
|
||
- mesh
|
||
port: $MONGODB_PORT
|
||
sni_hosts:
|
||
- $MONGODB_HOST
|
||
route:
|
||
- destination:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
subset: mongo
|
||
port:
|
||
number: 443
|
||
- match:
|
||
- gateways:
|
||
- istio-egressgateway
|
||
port: 443
|
||
sni_hosts:
|
||
- $MONGODB_HOST
|
||
route:
|
||
- destination:
|
||
host: $MONGODB_HOST
|
||
port:
|
||
number: $MONGODB_PORT
|
||
weight: 100
|
||
EOF
|
||
</code></pre></div></div>
|
||
</div>
|
||
</li>
|
||
<li><p><a href="#verify-that-egress-traffic-is-directed-through-the-egress-gateway">Verify that the traffic is directed though the egress gateway</a></p></li>
|
||
</ol>
|
||
<h4 id="cleanup-directing-tls-egress-traffic-through-an-egress-gateway">Cleanup directing TLS Egress traffic through an egress gateway</h4>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry mongo
|
||
$ kubectl delete gateway istio-egressgateway
|
||
$ kubectl delete virtualservice direct-mongo-through-egress-gateway
|
||
$ kubectl delete destinationrule egressgateway-for-mongo
|
||
</code></pre>
|
||
<h3 id="enable-mongodb-tls-egress-traffic-to-arbitrary-wildcarded-domains">Enable MongoDB TLS egress traffic to arbitrary wildcarded domains</h3>
|
||
<p>Sometimes you want to configure egress traffic to multiple hostnames from the same domain, for example traffic to all
|
||
MongoDB services from <code>*.&lt;your company domain&gt;.com</code>. You do not want to create multiple configuration items, one for
|
||
each and every MongoDB service in your company. To configure access to all the external services from the same domain by
|
||
a single configuration, you use <em>wildcarded</em> hosts.</p>
|
||
<p>In this section you configure egress traffic for a wildcarded domain. I used a MongoDB instance at <code>composedb.com</code>
|
||
domain, so configuring egress traffic for <code>*.com</code> worked for me (I could have used <code>*.composedb.com</code> as well).
|
||
You can pick a wildcarded domain according to your MongoDB host.</p>
|
||
<p>To configure egress gateway traffic for a wildcarded domain, you will first need to deploy a custom egress
|
||
gateway with
|
||
<a href="/v1.1/docs/examples/advanced-gateways/wildcard-egress-hosts/#wildcard-configuration-for-arbitrary-domains">an additional SNI proxy</a>.
|
||
This is needed due to current limitations of Envoy, the proxy used by the standard Istio egress gateway.</p>
|
||
<h4 id="prepare-a-new-egress-gateway-with-an-sni-proxy">Prepare a new egress gateway with an SNI proxy</h4>
|
||
<p>In this subsection you deploy an egress gateway with an SNI proxy, in addition to the standard Istio Envoy proxy. You
|
||
can use any SNI proxy that is capable of routing traffic according to arbitrary, not-preconfigured SNI values; we used
|
||
<a href="http://nginx.org">Nginx</a> to achieve this functionality.</p>
|
||
<ol>
|
||
<li><p>Create a configuration file for the Nginx SNI proxy. You may want to edit the file to specify additional Nginx
|
||
settings, if required.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF &gt; ./sni-proxy.conf
|
||
user www-data;
|
||
events {
|
||
}
|
||
stream {
|
||
log_format log_stream &#39;\$remote_addr [\$time_local] \$protocol [\$ssl_preread_server_name]&#39;
|
||
&#39;\$status \$bytes_sent \$bytes_received \$session_time&#39;;
|
||
access_log /var/log/nginx/access.log log_stream;
|
||
error_log /var/log/nginx/error.log;
|
||
# tcp forward proxy by SNI
|
||
server {
|
||
resolver 8.8.8.8 ipv6=off;
|
||
listen 127.0.0.1:$MONGODB_PORT;
|
||
proxy_pass \$ssl_preread_server_name:$MONGODB_PORT;
|
||
ssl_preread on;
|
||
}
|
||
}
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Create a Kubernetes <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/">ConfigMap</a>
|
||
to hold the configuration of the Nginx SNI proxy:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl create configmap egress-sni-proxy-configmap -n istio-system --from-file=nginx.conf=./sni-proxy.conf
|
||
</code></pre></li>
|
||
<li><p>The following command will generate <code>istio-egressgateway-with-sni-proxy.yaml</code> to edit and deploy.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | helm template install/kubernetes/helm/istio/ --name istio-egressgateway-with-sni-proxy --namespace istio-system -x charts/gateways/templates/deployment.yaml -x charts/gateways/templates/service.yaml -x charts/gateways/templates/serviceaccount.yaml -x charts/gateways/templates/autoscale.yaml -x charts/gateways/templates/clusterrole.yaml -x charts/gateways/templates/clusterrolebindings.yaml --set global.mtls.enabled=true --set global.istioNamespace=istio-system -f - &gt; ./istio-egressgateway-with-sni-proxy.yaml
|
||
gateways:
|
||
enabled: true
|
||
istio-ingressgateway:
|
||
enabled: false
|
||
istio-egressgateway:
|
||
enabled: false
|
||
istio-egressgateway-with-sni-proxy:
|
||
enabled: true
|
||
labels:
|
||
app: istio-egressgateway-with-sni-proxy
|
||
istio: egressgateway-with-sni-proxy
|
||
replicaCount: 1
|
||
autoscaleMin: 1
|
||
autoscaleMax: 5
|
||
cpu:
|
||
targetAverageUtilization: 80
|
||
serviceAnnotations: {}
|
||
type: ClusterIP
|
||
ports:
|
||
- port: 443
|
||
name: https
|
||
secretVolumes:
|
||
- name: egressgateway-certs
|
||
secretName: istio-egressgateway-certs
|
||
mountPath: /etc/istio/egressgateway-certs
|
||
- name: egressgateway-ca-certs
|
||
secretName: istio-egressgateway-ca-certs
|
||
mountPath: /etc/istio/egressgateway-ca-certs
|
||
configVolumes:
|
||
- name: sni-proxy-config
|
||
configMapName: egress-sni-proxy-configmap
|
||
additionalContainers:
|
||
- name: sni-proxy
|
||
image: nginx
|
||
volumeMounts:
|
||
- name: sni-proxy-config
|
||
mountPath: /etc/nginx
|
||
readOnly: true
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Deploy the new egress gateway:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f ./istio-egressgateway-with-sni-proxy.yaml
|
||
serviceaccount &#34;istio-egressgateway-with-sni-proxy-service-account&#34; created
|
||
clusterrole &#34;istio-egressgateway-with-sni-proxy-istio-system&#34; created
|
||
clusterrolebinding &#34;istio-egressgateway-with-sni-proxy-istio-system&#34; created
|
||
service &#34;istio-egressgateway-with-sni-proxy&#34; created
|
||
deployment &#34;istio-egressgateway-with-sni-proxy&#34; created
|
||
horizontalpodautoscaler &#34;istio-egressgateway-with-sni-proxy&#34; created
|
||
</code></pre></li>
|
||
<li><p>Verify that the new egress gateway is running. Note that the pod has two containers (one is the Envoy proxy and the
|
||
second one is the SNI proxy).</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pod -l istio=egressgateway-with-sni-proxy -n istio-system
|
||
NAME READY STATUS RESTARTS AGE
|
||
istio-egressgateway-with-sni-proxy-79f6744569-pf9t2 2/2 Running 0 17s
|
||
</code></pre></li>
|
||
<li><p>Create a service entry with a static address equal to 127.0.0.1 (<code>localhost</code>), and disable mutual TLS on the traffic directed to the new
|
||
service entry:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: sni-proxy
|
||
spec:
|
||
hosts:
|
||
- sni-proxy.local
|
||
location: MESH_EXTERNAL
|
||
ports:
|
||
- number: $MONGODB_PORT
|
||
name: tcp
|
||
protocol: TCP
|
||
resolution: STATIC
|
||
endpoints:
|
||
- address: 127.0.0.1
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: disable-mtls-for-sni-proxy
|
||
spec:
|
||
host: sni-proxy.local
|
||
trafficPolicy:
|
||
tls:
|
||
mode: DISABLE
|
||
EOF
|
||
</code></pre></li>
|
||
</ol>
|
||
<h4 id="configure-access-to-com-using-the-new-egress-gateway">Configure access to <code>*.com</code> using the new egress gateway</h4>
|
||
<ol>
|
||
<li><p>Define a <code>ServiceEntry</code> for <code>*.com</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | kubectl create -f -
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: mongo
|
||
spec:
|
||
hosts:
|
||
- &#34;*.com&#34;
|
||
ports:
|
||
- number: 443
|
||
name: tls
|
||
protocol: TLS
|
||
- number: $MONGODB_PORT
|
||
name: tls-mongodb
|
||
protocol: TLS
|
||
location: MESH_EXTERNAL
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Create an egress <code>Gateway</code> for <em>*.com</em>, port 443, protocol TLS, a destination rule to set the
|
||
<a href="https://en.wikipedia.org/wiki/Server_Name_Indication">SNI</a> for the gateway, and Envoy filters to prevent tampering
|
||
with SNI by a malicious application (the filters verify that the SNI issued by the application is the SNI reported
|
||
to Mixer).</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: Gateway
|
||
metadata:
|
||
name: istio-egressgateway-with-sni-proxy
|
||
spec:
|
||
selector:
|
||
istio: egressgateway-with-sni-proxy
|
||
servers:
|
||
- port:
|
||
number: 443
|
||
name: tls
|
||
protocol: TLS
|
||
hosts:
|
||
- &#34;*.com&#34;
|
||
tls:
|
||
mode: MUTUAL
|
||
serverCertificate: /etc/certs/cert-chain.pem
|
||
privateKey: /etc/certs/key.pem
|
||
caCertificates: /etc/certs/root-cert.pem
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: mtls-for-egress-gateway
|
||
spec:
|
||
host: istio-egressgateway-with-sni-proxy.istio-system.svc.cluster.local
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: ROUND_ROBIN
|
||
portLevelSettings:
|
||
- port:
|
||
number: 443
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
subsets:
|
||
- name: mongo
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: ROUND_ROBIN
|
||
portLevelSettings:
|
||
- port:
|
||
number: 443
|
||
tls:
|
||
mode: ISTIO_MUTUAL
|
||
---
|
||
# The following filter is used to forward the original SNI (sent by the application) as the SNI of the mutual TLS
|
||
# connection.
|
||
# The forwarded SNI will be reported to Mixer so that policies will be enforced based on the original SNI value.
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: EnvoyFilter
|
||
metadata:
|
||
name: forward-downstream-sni
|
||
spec:
|
||
filters:
|
||
- listenerMatch:
|
||
portNumber: $MONGODB_PORT
|
||
listenerType: SIDECAR_OUTBOUND
|
||
filterName: forward_downstream_sni
|
||
filterType: NETWORK
|
||
filterConfig: {}
|
||
---
|
||
# The following filter verifies that the SNI of the mutual TLS connection (the SNI reported to Mixer) is
|
||
# identical to the original SNI issued by the application (the SNI used for routing by the SNI proxy).
|
||
# The filter prevents Mixer from being deceived by a malicious application: routing to one SNI while
|
||
# reporting some other value of SNI. If the original SNI does not match the SNI of the mutual TLS connection, the
|
||
# filter will block the connection to the external service.
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: EnvoyFilter
|
||
metadata:
|
||
name: egress-gateway-sni-verifier
|
||
spec:
|
||
workloadLabels:
|
||
app: istio-egressgateway-with-sni-proxy
|
||
filters:
|
||
- listenerMatch:
|
||
portNumber: 443
|
||
listenerType: GATEWAY
|
||
filterName: sni_verifier
|
||
filterType: NETWORK
|
||
filterConfig: {}
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Route the traffic destined for <em>*.com</em> to the egress gateway and from the egress gateway to the SNI proxy.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: direct-mongo-through-egress-gateway
|
||
spec:
|
||
hosts:
|
||
- &#34;*.com&#34;
|
||
gateways:
|
||
- mesh
|
||
- istio-egressgateway-with-sni-proxy
|
||
tls:
|
||
- match:
|
||
- gateways:
|
||
- mesh
|
||
port: $MONGODB_PORT
|
||
sni_hosts:
|
||
- &#34;*.com&#34;
|
||
route:
|
||
- destination:
|
||
host: istio-egressgateway-with-sni-proxy.istio-system.svc.cluster.local
|
||
subset: mongo
|
||
port:
|
||
number: 443
|
||
weight: 100
|
||
tcp:
|
||
- match:
|
||
- gateways:
|
||
- istio-egressgateway-with-sni-proxy
|
||
port: 443
|
||
route:
|
||
- destination:
|
||
host: sni-proxy.local
|
||
port:
|
||
number: $MONGODB_PORT
|
||
weight: 100
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Refresh the web page of the application again and verify that the ratings are still displayed correctly.</p></li>
|
||
<li><p><a href="/v1.1/docs/tasks/telemetry/logs/access-log/#enable-envoy-s-access-logging">Enable Envoy’s access logging</a></p></li>
|
||
<li><p>Check the log of the egress gateway&rsquo;s Envoy proxy. If Istio is deployed in the <code>istio-system</code> namespace, the command
|
||
to print the log is:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl logs -l istio=egressgateway-with-sni-proxy -c istio-proxy -n istio-system
|
||
</code></pre>
|
||
<p>You should see lines similar to the following:</p>
|
||
<pre><code class='language-plain' data-expandlinks='true' >[2019-01-02T17:22:04.602Z] &#34;- - -&#34; 0 - 768 1863 88 - &#34;-&#34; &#34;-&#34; &#34;-&#34; &#34;-&#34; &#34;127.0.0.1:28543&#34; outbound|28543||sni-proxy.local 127.0.0.1:49976 172.30.146.115:443 172.30.146.118:58510 &lt;your MongoDB host&gt;
|
||
[2019-01-02T17:22:04.713Z] &#34;- - -&#34; 0 - 1534 2590 85 - &#34;-&#34; &#34;-&#34; &#34;-&#34; &#34;-&#34; &#34;127.0.0.1:28543&#34; outbound|28543||sni-proxy.local 127.0.0.1:49988 172.30.146.115:443 172.30.146.118:58522 &lt;your MongoDB host&gt;
|
||
</code></pre></li>
|
||
<li><p>Check the logs of the SNI proxy. If Istio is deployed in the <code>istio-system</code> namespace, the command to print the
|
||
log is:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl logs -l istio=egressgateway-with-sni-proxy -n istio-system -c sni-proxy
|
||
127.0.0.1 [23/Aug/2018:03:28:18 +0000] TCP [&lt;your MongoDB host&gt;]200 1863 482 0.089
|
||
127.0.0.1 [23/Aug/2018:03:28:18 +0000] TCP [&lt;your MongoDB host&gt;]200 2590 1248 0.095
|
||
</code></pre></li>
|
||
</ol>
|
||
<h4 id="understanding-what-happened">Understanding what happened</h4>
|
||
<p>In this section you configured egress traffic to your MongoDB host using a wildcarded domain. While for a single MongoDB
|
||
host there is no gain in using wildcarded domains (an exact hostname can be specified), it could be beneficial for
|
||
cases when the applications in the cluster access multiple MongoDB hosts that match some wildcarded domain. For example,
|
||
if the applications access <code>mongodb1.composedb.com</code>, <code>mongodb2.composedb.com</code> and <code>mongodb3.composedb.com</code>, the egress
|
||
traffic can be configured by a single configuration for the wildcarded domain <code>*.composedb.com</code>.</p>
|
||
<p>I will leave it as an exercise for the reader to verify that no additional Istio configuration is required when you
|
||
configure an app to use another instance of MongoDB with a hostname that matches the wildcarded domain used in this
|
||
section.</p>
|
||
<h4 id="cleanup-of-configuration-for-mongodb-tls-egress-traffic-to-arbitrary-wildcarded-domains">Cleanup of configuration for MongoDB TLS egress traffic to arbitrary wildcarded domains</h4>
|
||
<ol>
|
||
<li><p>Delete the configuration items for <em>*.com</em>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry mongo
|
||
$ kubectl delete gateway istio-egressgateway-with-sni-proxy
|
||
$ kubectl delete virtualservice direct-mongo-through-egress-gateway
|
||
$ kubectl delete destinationrule mtls-for-egress-gateway
|
||
$ kubectl delete envoyfilter forward-downstream-sni egress-gateway-sni-verifier
|
||
</code></pre></li>
|
||
<li><p>Delete the configuration items for the <code>egressgateway-with-sni-proxy</code> <code>Deployment</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry sni-proxy
|
||
$ kubectl delete destinationrule disable-mtls-for-sni-proxy
|
||
$ kubectl delete -f ./istio-egressgateway-with-sni-proxy.yaml
|
||
$ kubectl delete configmap egress-sni-proxy-configmap -n istio-system
|
||
</code></pre></li>
|
||
<li><p>Remove the configuration files you created:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ rm ./istio-egressgateway-with-sni-proxy.yaml
|
||
$ rm ./nginx-sni-proxy.conf
|
||
</code></pre></li>
|
||
</ol>
|
||
<h2 id="cleanup">Cleanup</h2>
|
||
<ol>
|
||
<li><p>Drop the <code>bookinfo</code> user:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u admin -p $MONGO_ADMIN_PASSWORD --authenticationDatabase admin
|
||
use test
|
||
db.dropUser(&#34;bookinfo&#34;);
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Drop the <em>ratings</em> collection:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | mongo --ssl --sslAllowInvalidCertificates $MONGODB_HOST:$MONGODB_PORT -u admin -p $MONGO_ADMIN_PASSWORD --authenticationDatabase admin
|
||
use test
|
||
db.ratings.drop();
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Unset the environment variables you used:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ unset MONGO_ADMIN_PASSWORD BOOKINFO_PASSWORD MONGODB_HOST MONGODB_PORT MONGODB_IP
|
||
</code></pre></li>
|
||
<li><p>Remove the virtual services:</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/virtual-service-ratings-db.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete -f @samples/bookinfo/networking/virtual-service-ratings-db.yaml@
|
||
Deleted config: virtual-service/default/reviews
|
||
Deleted config: virtual-service/default/ratings
|
||
</code></pre></div></li>
|
||
<li><p>Undeploy <em>ratings v2-mongodb</em>:</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete -f @samples/bookinfo/platform/kube/bookinfo-ratings-v2.yaml@
|
||
deployment &#34;ratings-v2&#34; deleted
|
||
</code></pre></div></li>
|
||
</ol>
|
||
<h2 id="conclusion">Conclusion</h2>
|
||
<p>In this blog post I demonstrated various options for MongoDB egress traffic control. You can control the MongoDB egress
|
||
traffic on a TCP or TLS level where applicable. In both TCP and TLS cases, you can direct the traffic from the sidecar
|
||
proxies directly to the external MongoDB host, or direct the traffic through an egress gateway, according to your
|
||
organization&rsquo;s security requirements. In the latter case, you can also decide to apply or disable mutual TLS
|
||
authentication between the sidecar proxies and the egress gateway. If you want to control MongoDB egress traffic on the
|
||
TLS level by specifying wildcarded domains like <code>*.com</code> and you need to direct the traffic through the egress gateway,
|
||
you must deploy a custom egress gateway with an SNI proxy.</p>
|
||
<p>Note that the configuration and considerations described in this blog post for MongoDB are rather the same for other
|
||
non-HTTP protocols on top of TCP/TLS.</p></description><pubDate>Fri, 16 Nov 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/egress-mongo/</link><author>Vadim Eisenberg</author><guid isPermaLink="true">/v1.1/blog/2018/egress-mongo/</guid><category>traffic-management</category><category>egress</category><category>tcp</category><category>mongo</category></item><item><title>Announcing Istio 1.0.3</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.0.3. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.0.3"
|
||
data-updateadvice='Before you download 1.0.3, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.0.8'
|
||
data-updatehref="/v1.1/about/notes/1.0.8">
|
||
DOWNLOAD 1.0.3
|
||
</button>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0.3 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.0.2...1.0.3">CHANGES IN 1.0.3</a>
|
||
</div>
|
||
<h2 id="behavior-changes">Behavior changes</h2>
|
||
<ul>
|
||
<li><p><a href="/v1.1/help/ops/setup/validation">Validating webhook</a> is now mandatory. Disabling it may result in Pilot crashes.</p></li>
|
||
<li><p><a href="/v1.1/docs/reference/config/networking/v1alpha3/service-entry/">Service entry</a> validation now rejects the wildcard hostname (<code>*</code>) when configuring DNS resolution. The API has never allowed this, however <code>ServiceEntry</code> was erroneously excluded from validation in the previous release. Use of wildcards as part of a hostname, e.g. <code>*.bar.com</code>, remains unchanged.</p></li>
|
||
<li><p>The core dump path for <code>istio-proxy</code> has changed to <code>/var/lib/istio</code>.</p></li>
|
||
</ul>
|
||
<h2 id="networking">Networking</h2>
|
||
<ul>
|
||
<li><p><a href="/v1.1/docs/tasks/security/mutual-tls">Mutual TLS</a> Permissive mode is enabled by default.</p></li>
|
||
<li><p>Pilot performance and scalability has been greatly enhanced. Pilot now delivers endpoint updates to 500 sidecars in under 1 second.</p></li>
|
||
<li><p>Default <a href="/v1.1/docs/tasks/telemetry/distributed-tracing/overview/#trace-sampling">trace sampling</a> is set to 1%.</p></li>
|
||
</ul>
|
||
<h2 id="policy-and-telemetry">Policy and telemetry</h2>
|
||
<ul>
|
||
<li><p>Mixer (<code>istio-telemetry</code>) now supports load shedding based on request rate and expected latency.</p></li>
|
||
<li><p>Mixer client (<code>istio-policy</code>) now supports <code>FAIL_OPEN</code> setting.</p></li>
|
||
<li><p>Istio Performance dashboard added to Grafana.</p></li>
|
||
<li><p>Reduced <code>istio-telemetry</code> CPU usage by 10%.</p></li>
|
||
<li><p>Eliminated <code>statsd-to-prometheus</code> deployment. Prometheus now directly scrapes from <code>istio-proxy</code>.</p></li>
|
||
</ul></description><pubDate>Tue, 30 Oct 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/announcing-1.0.3/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2018/announcing-1.0.3/</guid></item><item><title>Announcing Istio 1.0.2</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.0.2. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.0.2"
|
||
data-updateadvice='Before you download 1.0.2, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.0.8'
|
||
data-updatehref="/v1.1/about/notes/1.0.8">
|
||
DOWNLOAD 1.0.2
|
||
</button>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0.2 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.0.1...1.0.2">CHANGES IN 1.0.2</a>
|
||
</div>
|
||
<h2 id="general">General</h2>
|
||
<ul>
|
||
<li><p>Fixed bug in Envoy where the sidecar would crash if receiving normal traffic on the mutual TLS port.</p></li>
|
||
<li><p>Fixed bug with Pilot propagating incomplete updates to Envoy in a multicluster environment.</p></li>
|
||
<li><p>Added a few more Helm options for Grafana.</p></li>
|
||
<li><p>Improved Kubernetes service registry queue performance.</p></li>
|
||
<li><p>Fixed bug where <code>istioctl proxy-status</code> was not showing the patch version.</p></li>
|
||
<li><p>Add validation of virtual service SNI hosts.</p></li>
|
||
</ul></description><pubDate>Thu, 06 Sep 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/announcing-1.0.2/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2018/announcing-1.0.2/</guid></item><item><title>Announcing Istio 1.0.1</title><description><p>We&rsquo;re pleased to announce the availability of Istio 1.0.1. Please see below for what&rsquo;s changed.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.0.1"
|
||
data-updateadvice='Before you download 1.0.1, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.0.8'
|
||
data-updatehref="/v1.1/about/notes/1.0.8">
|
||
DOWNLOAD 1.0.1
|
||
</button>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0.1 DOCS</a>
|
||
<a class="btn" href="https://github.com/istio/istio/compare/1.0.0...1.0.1">CHANGES IN 1.0.1</a>
|
||
</div>
|
||
<h2 id="networking">Networking</h2>
|
||
<ul>
|
||
<li><p>Improved Pilot scalability and Envoy startup time.</p></li>
|
||
<li><p>Fixed virtual service host mismatch issue when adding a port.</p></li>
|
||
<li><p>Added limited support for <a href="/v1.1/help/ops/traffic-management/deploy-guidelines/#multiple-virtual-services-and-destination-rules-for-the-same-host">merging multiple virtual service or destination rule definitions</a> for the same host.</p></li>
|
||
<li><p>Allow <a href="https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/cluster/outlier_detection.proto">outlier</a> consecutive gateway failures when using HTTP.</p></li>
|
||
</ul>
|
||
<h2 id="environment">Environment</h2>
|
||
<ul>
|
||
<li><p>Made it possible to use Pilot standalone, for those users who want to only leverage Istio&rsquo;s traffic management functionality.</p></li>
|
||
<li><p>Introduced the convenient <code>values-istio-gateway.yaml</code> configuration that enables users to run standalone gateways.</p></li>
|
||
<li><p>Fixed a variety of Helm installation issues, including an issue with the <code>istio-sidecar-injector</code> configmap not being found.</p></li>
|
||
<li><p>Fixed the Istio installation error with Galley not being ready.</p></li>
|
||
<li><p>Fixed a variety of issues around mesh expansion.</p></li>
|
||
</ul>
|
||
<h2 id="policy-and-telemetry">Policy and Telemetry</h2>
|
||
<ul>
|
||
<li><p>Added an experimental metrics expiration configuration to the Mixer Prometheus adapter.</p></li>
|
||
<li><p>Updated Grafana to 5.2.2.</p></li>
|
||
</ul>
|
||
<h3 id="adapters">Adapters</h3>
|
||
<ul>
|
||
<li>Ability to specify sink options for the Stackdriver adapter.</li>
|
||
</ul>
|
||
<h2 id="galley">Galley</h2>
|
||
<ul>
|
||
<li>Improved configuration validation for health checks.</li>
|
||
</ul></description><pubDate>Wed, 29 Aug 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/announcing-1.0.1/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2018/announcing-1.0.1/</guid></item><item><title>All Day Istio Twitch Stream</title><description>
|
||
<p>To celebrate the 1.0 release and to promote the software to a wider audience, the Istio community is hosting an all day live stream on Twitch on August 17th.</p>
|
||
<h2 id="what-is-twitch">What is Twitch?</h2>
|
||
<p><a href="https://twitch.tv/">Twitch</a> is a popular video gaming live streaming platform and recently has seen a lot of coding content showing up. The IBM Advocates have been doing live coding and presentations there and it&rsquo;s been fun. While mostly used for gaming content, there is a <a href="https://www.twitch.tv/communities/programming">growing community</a> sharing and watching programming content on the site.</p>
|
||
<h2 id="what-does-this-have-to-do-with-istio">What does this have to do with Istio?</h2>
|
||
<p>The stream is going to be a full day of Istio content. Hopefully we&rsquo;ll have a good mix of deep technical content, beginner content and line-of-business content for our audience. We&rsquo;ll have developers, users, and evangelists on throughout the day to share their demos and stories. Expect live coding, q and a, and some surprises. We have stellar guests lined up from IBM, Google, Datadog, Pivotal, and more!</p>
|
||
<h2 id="recordings">Recordings</h2>
|
||
<p>Recordings are available <a href="https://www.youtube.com/playlist?list=PLzpeuWUENMK0V3dwpx5gPJun-SLG0USqU">here</a>.</p>
|
||
<h2 id="schedule">Schedule</h2>
|
||
<p>All times are <code>PDT</code>.</p>
|
||
<table>
|
||
<thead>
|
||
<tr>
|
||
<th>Time</th>
|
||
<th>Speaker</th>
|
||
<th>Affiliation</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td>10:00 - 10:30</td>
|
||
<td><code>Spencer Krum + Lisa-Marie Namphy</code></td>
|
||
<td><code>IBM / Portworx</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>10:30 - 11:00</td>
|
||
<td><code>Lin Sun / Spencer Krum / Sven Mawson</code></td>
|
||
<td><code>IBM / Google</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>11:00 - 11:10</td>
|
||
<td><code>Lin Sun / Spencer Krum</code></td>
|
||
<td><code>IBM</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>11:10 - 11:30</td>
|
||
<td><code>Jason Yee / Ilan Rabinovich</code></td>
|
||
<td><code>Datadog</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>11:30 - 11:50</td>
|
||
<td><code>April Nassl</code></td>
|
||
<td><code>Google</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>11:50 - 12:10</td>
|
||
<td><code>Spike Curtis</code></td>
|
||
<td><code>Tigera</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>12:10 - 12:30</td>
|
||
<td><code>Shannon Coen</code></td>
|
||
<td><code>Pivotal</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>12:30 - 1:00</td>
|
||
<td><code>Matt Klein</code></td>
|
||
<td><code>Lyft</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>1:00 - 1:20</td>
|
||
<td><code>Zach Jory</code></td>
|
||
<td><code>F5/Aspen Mesh</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>1:20 - 1:40</td>
|
||
<td><code>Dan Ciruli</code></td>
|
||
<td><code>Google</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>1:40 - 2:00</td>
|
||
<td><code>Isaiah Snell-Feikema</code> / <code>Greg Hanson</code></td>
|
||
<td><code>IBM</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>2:00 - 2:20</td>
|
||
<td><code>Zach Butcher</code></td>
|
||
<td><code>Tetrate</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>2:20 - 2:40</td>
|
||
<td><code>Ray Hudaihed</code></td>
|
||
<td><code>American Airlines</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>2:40 - 3:00</td>
|
||
<td><code>Christian Posta</code></td>
|
||
<td><code>Red Hat</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>3:00 - 3:20</td>
|
||
<td><code>Google/IBM China</code></td>
|
||
<td><code>Google / IBM</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>3:20 - 3:40</td>
|
||
<td><code>Colby Dyess</code></td>
|
||
<td><code>Tuffin</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td>3:40 - 4:00</td>
|
||
<td><code>Rohit Agarwalla</code></td>
|
||
<td><code>Cisco</code></td>
|
||
</tr>
|
||
</tbody>
|
||
</table></description><pubDate>Fri, 03 Aug 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/istio-twitch-stream/</link><author>Spencer Krum, IBM</author><guid isPermaLink="true">/v1.1/blog/2018/istio-twitch-stream/</guid></item><item><title>Istio a Game Changer for HP's FitStation Platform</title><description><p>The FitStation team at HP strongly believes in the future of Kubernetes, BPF and service-mesh as the next standards in cloud infrastructure. We are also very happy to see Istio coming to its official Istio 1.0 release &ndash; thanks to the joint collaboration that started at Google, IBM and Lyft beginning in May 2017.</p>
|
||
<p>Throughout the development of FitStation’s large scale and progressive cloud platform, Istio, Cilium and Kubernetes technologies have delivered a multitude of opportunities to make our systems more robust and scalable. Istio was a game changer in creating reliable and dynamic network communication.</p>
|
||
<p><a href="http://www.fitstation.com">FitStation powered by HP</a> is a technology platform that captures 3D biometric data to design personalized footwear to perfectly fit individual foot size and shape as well as gait profile. It uses 3D scanning, pressure sensing, 3D printing and variable density injection molding to create unique footwear. Footwear brands such as Brooks, Steitz Secura or Superfeet are connecting to FitStation to build their next generation of high performance sports, professional and medical shoes.</p>
|
||
<p>FitStation is built on the promise of ultimate security and privacy for users&rsquo; biometric data. ISTIO is the cornerstone to make that possible for data-at-flight within our cloud. By managing these aspects at the infrastructure level, we focused on solving business problems instead of spending time on individual implementations of secure service communication. Using Istio allowed us to dramatically reduce the complexity of maintaining a multitude of libraries and services to provide secure service communication.</p>
|
||
<p>As a bonus benefit of Istio 1.0, we gained network visibility, metrics and tracing out of the box. This radically improved decision-making and response quality for our development
|
||
and devops teams. The team got in-depth insight in the network communication across the entire platform, both for new as well as legacy applications. The integration of Cilium
|
||
with Envoy delivered a remarkable performance benefit on Istio service mesh communication, combined with a fine-grained kernel driven L7 network security layer. This was due to the powers of BPF brought to Istio by Cilium. We believe this will drive the future of Linux kernel security.</p>
|
||
<p>It has been very exciting to follow Istio’s growth. We have been able to see clear improvements of performance and stability over the different development versions. The improvements between version 0.7 and 0.8 made our teams feel comfortable with version 1.0, we can state that Istio is now ready for real production usage.</p>
|
||
<p>We are looking forward to the promising roadmaps of Istio, Envoy, Cilium and CNCF.</p></description><pubDate>Tue, 31 Jul 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/hp/</link><author>Steven Ceuppens, Chief Software Architect @ HP FitStation, Open Source Advocate & Contributor</author><guid isPermaLink="true">/v1.1/blog/2018/hp/</guid></item><item><title>Announcing Istio 1.0</title><description>
|
||
<p>Today, we’re excited to announce <a href="/v1.1/about/notes/1.0">Istio 1.0</a>. It’s been a little over a year since our initial 0.1 release. Since then, Istio has evolved significantly with the help of a thriving and growing community of contributors and users. We’ve now reached the point where many companies have successfully adopted Istio in production and have gotten real value from the insight and control it provides over their deployments. We’ve helped large enterprises and fast-moving startups like <a href="https://www.ebay.com/">eBay</a>, <a href="https://www.autotrader.co.uk/">Auto Trader UK</a>, <a href="http://www.descarteslabs.com/">Descartes Labs</a>, <a href="https://www.fitstation.com/">HP FitStation</a>, <a href="https://juspay.in">JUSPAY</a>, <a href="https://www.namely.com/">Namely</a>, <a href="https://www.pubnub.com/">PubNub</a> and <a href="https://www.trulia.com/">Trulia</a> use Istio to connect, manage and secure their services from the ground up. Shipping this release as 1.0 is recognition that we’ve built a core set of functionality that our users can rely on for production use.</p>
|
||
<div class="call-to-action">
|
||
<button class="btn update-notice"
|
||
data-title='Update Notice'
|
||
data-downloadhref="https://github.com/istio/istio/releases/tag/1.0.0"
|
||
data-updateadvice='Before you download 1.0, you should know that there&#39;s a newer patch release with the latest bug fixes and perf improvements.'
|
||
data-updatebutton='LEARN ABOUT ISTIO 1.0.8'
|
||
data-updatehref="/v1.1/about/notes/1.0.8">
|
||
DOWNLOAD 1.0
|
||
</button>
|
||
<a class="btn" href="https://archive.istio.io/v1.0">1.0 DOCS</a>
|
||
<a class="btn" href="/v1.1/about/notes/1.0/">1.0 RELEASE NOTES</a>
|
||
</div>
|
||
<h2 id="ecosystem">Ecosystem</h2>
|
||
<p>We’ve seen substantial growth in Istio&rsquo;s ecosystem in the last year. <a href="https://www.envoyproxy.io/">Envoy</a> continues its impressive growth and added numerous
|
||
features that are crucial for a production quality service mesh. Observability providers like <a href="https://www.datadoghq.com/">Datadog</a>,
|
||
<a href="https://www.solarwinds.com/">SolarWinds</a>, <a href="https://sysdig.com/blog/monitor-istio/">Sysdig</a>, <a href="https://cloud.google.com/stackdriver/">Google Stackdriver</a> and
|
||
<a href="https://aws.amazon.com/cloudwatch/">Amazon CloudWatch</a> have written plugins to integrate Istio with their products.
|
||
<a href="https://www.tigera.io/resources/using-network-policy-concert-istio-2/">Tigera</a>, <a href="https://www.aporeto.com/">Aporeto</a>, <a href="https://cilium.io/">Cilium</a>
|
||
and <a href="https://styra.com/">Styra</a> built extensions to our policy enforcement and networking capabilities. <a href="https://www.redhat.com/en">Red Hat</a> built <a href="https://www.kiali.io">Kiali</a> to wrap a nice user-experience around mesh management and observability. <a href="https://www.cloudfoundry.org/">Cloud Foundry</a> is building on Istio for it’s next generation traffic routing stack, the recently announced <a href="https://github.com/knative/docs">Knative</a> serverless project is doing the same and <a href="https://apigee.com/">Apigee</a> announced that they plan to use it in their API management solution. These are just some of the integrations the community has added in the last year.</p>
|
||
<h2 id="features">Features</h2>
|
||
<p>Since the 0.8 release we’ve added some important new features and more importantly marked many of our existing features as Beta signaling that they’re ready for production use. This is captured in more detail in the <a href="/v1.1/about/notes/1.0/">release notes</a> but it’s worth calling out some highlights</p>
|
||
<ul>
|
||
<li><p>Multiple Kubernetes clusters can now be <a href="/v1.1/docs/setup/kubernetes/install/multicluster/">added to a single mesh</a> and enabling cross-cluster communication and consistent policy enforcement. Multi-cluster support is now Beta.</p></li>
|
||
<li><p>Networking APIs that enable fine grained control over the flow of traffic through a mesh are now Beta. Explicitly modeling ingress and egress concerns using Gateways allows operators to <a href="/v1.1/blog/2018/v1alpha3-routing/">control the network topology</a> and meet access security requirements at the edge.</p></li>
|
||
<li><p>Mutual TLS can now be <a href="/v1.1/docs/tasks/security/mtls-migration">rolled out incrementally</a> without requiring all clients of a service to be updated. This is a critical feature that unblocks adoption in-place by existing production deployments.</p></li>
|
||
<li><p>Mixer now has support for <a href="https://github.com/istio/istio/wiki/Out-Of-Process-gRPC-Adapter-Dev-Guide">developing out-of-process adapters</a>. This will become the default way to extend Mixer over the coming releases and makes building adapters much simpler.</p></li>
|
||
<li><p><a href="/v1.1/docs/concepts/security/#authorization">Authorization policies</a> which control access to services are now entirely evaluated locally in Envoy increasing
|
||
their performance and reliability.</p></li>
|
||
<li><p><a href="/v1.1/docs/setup/kubernetes/install/helm/">Helm chart installation</a> is now the recommended install method offering rich customization options to adopt Istio on your terms.</p></li>
|
||
<li><p>We’ve put a lot of effort into performance including continuous regression testing, large scale environment simulation and targeted fixes. We’re very happy with the results and will share more on this in detail in the coming weeks.</p></li>
|
||
</ul>
|
||
<h2 id="what-s-next">What’s next?</h2>
|
||
<p>While this is a significant milestone for the project there’s lots more to do. In working with adopters we’ve gotten a lot of great feedback about what to focus next. We’ve heard consistent themes around support for hybrid-cloud, install modularity, richer networking features and scalability for massive deployments. We’ve already taken some of this feedback into account in the 1.0 release and we’ll continue to aggressively tackle this work in the coming months.</p>
|
||
<h2 id="getting-started">Getting Started</h2>
|
||
<p>If you’re new to Istio and looking to use it for your deployment we’d love to hear from you. Take a look at <a href="/v1.1/docs/">our docs</a> or stop by our
|
||
<a href="https://discuss.istio.io">chat forum</a>. If you’d like
|
||
to go deeper and <a href="/v1.1/about/community">contribute to the project</a> come to one of our community meetings and say hello.</p>
|
||
<h2 id="finally">Finally</h2>
|
||
<p>The Istio team would like to give huge thanks to everyone who has made a contribution to the project. It wouldn’t be where it is today without your help. The last year has been pretty amazing and we look forward to the next one with excitement about what we can achieve together as a community.</p></description><pubDate>Tue, 31 Jul 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/announcing-1.0/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2018/announcing-1.0/</guid></item><item><title>Delayering Istio with AppSwitch</title><description>
|
||
<div>
|
||
<aside class="callout quote">
|
||
<div class="type">
|
||
<svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-quote"/></svg>
|
||
</div>
|
||
<div class="content">All problems in computer science can be solved with another layer, except of course the problem of too many layers. &ndash; David Wheeler</div>
|
||
</aside>
|
||
</div>
|
||
<p>The sidecar proxy approach enables a lot of awesomeness. Squarely in the datapath between microservices, the sidecar can precisely tell what the application is trying to do. It can monitor and instrument protocol traffic, not in the bowels of the networking layers but at the application level, to enable deep visibility, access controls and traffic management.</p>
|
||
<p>If we look closely however, there are many intermediate layers that the data has to pass through before the high-value analysis of application-traffic can be performed. Most of those layers are part of the base plumbing infrastructure that are there just to push the data along. In doing so, they add latency to communication and complexity to the overall system.</p>
|
||
<p>Over the years, there has been much collective effort in implementing aggressive fine-grained optimizations within the layers of the network datapath. Each iteration may shave another few microseconds. But then the true necessity of those layers itself has not been questioned.</p>
|
||
<h2 id="don-t-optimize-layers-remove-them">Don’t optimize layers, remove them</h2>
|
||
<p>In my belief, optimizing something is a poor fallback to removing its requirement altogether. That was the goal of my <a href="https://apporbit.com/a-brief-history-of-containers-from-reality-to-hype/">initial work</a> on OS-level virtualization that led to Linux containers which effectively <a href="https://www.oreilly.com/ideas/the-unwelcome-guest-why-vms-arent-the-solution-for-next-gen-applications">removed virtual machines</a> by running applications directly on the host operating system without requiring an intermediate guest. For a long time the industry was fighting the wrong battle distracted by optimizing VMs rather than removing the additional layer altogether.</p>
|
||
<p>I see the same pattern repeat itself with the connectivity of microservices, and networking in general. The network has been going through the changes that physical servers have gone through a decade earlier. New set of layers and constructs are being introduced. They are being baked deep into the protocol stack and even silicon without adequately considering low-touch alternatives. Perhaps there is a way to remove those additional layers altogether.</p>
|
||
<p>I have been thinking about these problems for some time and believe that an approach similar in concept to containers can be applied to the network stack that would fundamentally simplify how application endpoints are connected across the complexity of many intermediate layers. I have reapplied the same principles from the original work on containers to create <a href="http://appswitch.io">AppSwitch</a>. Similar to the way containers provide an interface that applications can directly consume, AppSwitch plugs directly into well-defined and ubiquitous network API that applications currently use and directly connects application clients to appropriate servers, skipping all intermediate layers. In the end, that&rsquo;s what networking is all about.</p>
|
||
<p>Before going into the details of how AppSwitch promises to remove unnecessary layers from the Istio stack, let me give a very brief introduction to its architecture. Further details are available at the <a href="https://appswitch.readthedocs.io/en/latest/">documentation</a> page.</p>
|
||
<h2 id="appswitch">AppSwitch</h2>
|
||
<p>Not unlike the container runtime, AppSwitch consists of a client and a daemon that speak over HTTP via a REST API. Both the client and the daemon are built as one self-contained binary, <code>ax</code>. The client transparently plugs into the application and tracks its system calls related to network connectivity and notifies the daemon about their occurrences. As an example, let’s say an application makes the <code>connect(2)</code> system call to the service IP of a Kubernetes service. The AppSwitch client intercepts the connect call, nullifies it and notifies the daemon about its occurrence along with some context that includes the system call arguments. The daemon would then handle the system call, potentially by directly connecting to the Pod IP of the upstream server on behalf of the application.</p>
|
||
<p>It is important to note that no data is forwarded between AppSwitch client and daemon. They are designed to exchange file descriptors (FDs) over a Unix domain socket to avoid having to copy data. Note also that client is not a separate process. Rather it directly runs in the context of the application itself. There is no data copy between the application and AppSwitch client either.</p>
|
||
<h2 id="delayering-the-stack">Delayering the stack</h2>
|
||
<p>Now that we have an idea about what AppSwitch does, let’s look at the layers that it optimizes away from a standard service mesh.</p>
|
||
<h3 id="network-devirtualization">Network Devirtualization</h3>
|
||
<p>Kubernetes offers simple and well-defined network constructs to the microservice applications it runs. In order to support them however, it imposes specific <a href="https://kubernetes.io/docs/concepts/cluster-administration/networking/">requirements</a> on the underlying network. Meeting those requirements is often not easy. The go-to solution of adding another layer is typically adopted to satisfy the requirements. In most cases the additional layer consists of a network overlay that sits between Kubernetes and underlying network. Traffic produced by the applications is encapsulated at the source and decapsulated at the target, which not only costs network resources but also takes up compute cores.</p>
|
||
<p>Because AppSwitch arbitrates what the application sees through its touchpoints with the platform, it projects a consistent virtual view of the underlying network to the application similar to an overlay but without introducing an additional layer of processing along the datapath. Just to draw a parallel to containers, the inside of a container looks and feels like a VM. However the underlying implementation does not intervene along the high-incidence control paths of low-level interrupts etc.</p>
|
||
<p>AppSwitch can be injected into a standard Kubernetes manifest (similar to Istio injection) such that the application’s network is directly handled by AppSwitch bypassing any network overlay underneath. More details to follow in just a bit.</p>
|
||
<h3 id="artifacts-of-container-networking">Artifacts of Container Networking</h3>
|
||
<p>Extending network connectivity from host into the container has been a <a href="https://kubernetes.io/blog/2016/01/why-kubernetes-doesnt-use-libnetwork/">major challenge</a>. New layers of network plumbing were invented explicitly for that purpose. As such, an application running in a container is simply a process on the host. However due to a <a href="http://appswitch.io/blog/kubernetes_istio_and_network_function_devirtualization_with_appswitch/">fundamental misalignment</a> between the network abstraction expected by the application and the abstraction exposed by container network namespace, the process cannot directly access the host network. Applications think of networking in terms of sockets or sessions whereas network namespaces expose a device abstraction. Once placed in a network namespace, the process suddenly loses all connectivity. The notion of veth-pair and corresponding tooling were invented just to close that gap. The data would now have to go from a host interface into a virtual switch and then through a veth-pair to the virtual network interface of the container network namespace.</p>
|
||
<p>AppSwitch can effectively remove both the virtual switch and veth-pair layers on both ends of the connection. Since the connections are established by the daemon running on the host using the network that’s already available on the host, there is no need for additional plumbing to bridge host network into the container. The socket FDs created on the host are passed to the application running within the pod’s network namespace. By the time the application receives the FD, all control path work (security checks, connection establishment) is already done and the FD is ready for actual IO.</p>
|
||
<h3 id="skip-tcp-ip-for-colocated-endpoints">Skip TCP/IP for colocated endpoints</h3>
|
||
<p>TCP/IP is the universal protocol medium over which pretty much all communication occurs. But if application endpoints happen to be on the same host, is TCP/IP really required? After all, it does do quite a bit of work and it is quite complex. Unix sockets are explicitly designed for intrahost communication and AppSwitch can transparently switch the communication to occur over a Unix socket for colocated endpoints.</p>
|
||
<p>For each listening socket of an application, AppSwitch maintains two listening sockets, one each for TCP and Unix. When a client tries to connect to a server that happens to be colocated, AppSwitch daemon would choose to connect to the Unix listening socket of the server. The resulting Unix sockets on each end are passed into respective applications. Once a fully connected FD is returned, the application would simply treat it as a bit pipe. The protocol doesn’t really matter. The application may occasionally make protocol specific calls such as <code>getsockname(2)</code> and AppSwitch would handle them in kind. It would present consistent responses such that the application would continue to run on.</p>
|
||
<h3 id="data-pushing-proxy">Data Pushing Proxy</h3>
|
||
<p>As we continue to look for layers to remove, let us also reconsider the requirement of the proxy layer itself. There are times when the role of the proxy may degenerate into a plain data pusher:</p>
|
||
<ul>
|
||
<li>There may not be a need for any protocol decoding</li>
|
||
<li>The protocol may not be recognized by the proxy</li>
|
||
<li>The communication may be encrypted and the proxy cannot access relevant headers</li>
|
||
<li>The application (redis, memcached etc.) may be too latency-sensitive and cannot afford the cost of an intermediate proxy</li>
|
||
</ul>
|
||
<p>In all these cases, the proxy is not different from any low-level plumbing layer. In fact, the latency introduced can be far higher because the same level of optimizations won’t be available to a proxy.</p>
|
||
<p>To illustrate this with an example, consider the application shown below. It consists of a Python app and a set of memcached servers behind it. An upstream memcached server is selected based on connection time routing. Speed is the primary concern here.</p>
|
||
<figure style="width:75%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:38.63965267727931%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/delayering-istio/memcached.png" title="Latency-sensitive application scenario">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/delayering-istio/memcached.png" alt="Proxyless datapath" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Latency-sensitive application scenario</figcaption>
|
||
</figure>
|
||
<p>If we look at the data flow in this setup, the Python app makes a connection to the service IP of memcached. It is redirected to the client-side sidecar. The sidecar routes the connection to one of the memcached servers and copies the data between the two sockets &ndash; one connected to the app and another connected to memcached. And the same also occurs on the server side between the server-side sidecar and memcached. The role of proxy at that point is just boring shoveling of bits between the two sockets. However, it ends up adding substantial latency to the end-to-end connection.</p>
|
||
<p>Now let us imagine that the app is somehow made to connect directly to memcached, then the two intermediate proxies could be skipped. The data would flow directly between the app and memcached without any intermediate hops. AppSwitch can arrange for that by transparently tweaking the target address passed by the Python app when it makes the <code>connect(2)</code> system call.</p>
|
||
<h3 id="proxyless-protocol-decoding">Proxyless Protocol Decoding</h3>
|
||
<p>Things are going to get a bit strange here. We have seen that the proxy can be bypassed for cases that don’t involve looking into application traffic. But is there anything we can do even for those other cases? It turns out, yes.</p>
|
||
<p>In a typical communication between microservices, much of the interesting information is exchanged in the initial headers. Headers are followed by body or payload which typically represents bulk of the communication. And once again the proxy degenerates into a data pusher for this part of communication. AppSwitch provides a nifty mechanism to skip proxy for these cases.</p>
|
||
<p>Even though AppSwitch is not a proxy, it <em>does</em> arbitrate connections between application endpoints and it <em>does</em> have access to corresponding socket FDs. Normally, AppSwitch simply passes those FDs to the application. But it can also peek into the initial message received on the connection using the <code>MSG_PEEK</code> option of the <code>recvfrom(2)</code> system call on the socket. It allows AppSwitch to examine application traffic without actually removing it from the socket buffers. When AppSwitch returns the FD to the application and steps out of the datapath, the application would do an actual read on the connection. AppSwitch uses this technique to perform deeper analysis of application-level traffic and implement sophisticated network functions as discussed in the next section, all without getting into the datapath.</p>
|
||
<h3 id="zero-cost-load-balancer-firewall-and-network-analyzer">Zero-Cost Load Balancer, Firewall and Network Analyzer</h3>
|
||
<p>Typical implementations of network functions such as load balancers and firewalls require an intermediate layer that needs to tap into data/packet stream. Kubernetes&rsquo; implementation of load balancer (<code>kube-proxy</code>) for example introduces a probe into the packet stream through iptables and Istio implements the same at the proxy layer. But if all that is required is to redirect or drop connections based on policy, it is not really necessary to stay in the datapath during the entire course of the connection. AppSwitch can take care of that much more efficiently by simply manipulating the control path at the API level. Given its intimate proximity to the application, AppSwitch also has easy access to various pieces of application level metrics such as dynamics of stack and heap usage, precisely when a service comes alive, attributes of active connections etc., all of which could potentially form a rich signal for monitoring and analytics.</p>
|
||
<p>To go a step further, AppSwitch can also perform L7 load balancing and firewall functions based on the protocol data that it obtains from the socket buffers. It can synthesize the protocol data and various other signals with the policy information acquired from Pilot to implement a highly efficient form of routing and access control enforcement. It can essentially &ldquo;influence&rdquo; the application to connect to the right backend server without requiring any changes to the application or its configuration. It is as if the application itself is infused with policy and traffic-management intelligence. Except in this case, the application can&rsquo;t escape the influence.</p>
|
||
<p>There is some more black-magic possible that would actually allow modifying the application data stream without getting into the datapath but I am going to save that for a later post. Current implementation of AppSwitch uses a proxy if the use case requires application protocol traffic to be modified. For those cases, AppSwitch provides a highly optimal mechanism to attract traffic to the proxy as discussed in the next section.</p>
|
||
<h3 id="traffic-redirection">Traffic Redirection</h3>
|
||
<p>Before the sidecar proxy can look into application protocol traffic, it needs to first receive the connections. Redirection of connections coming into and going out of the application is currently done by a layer of packet filtering that rewrites packets such that they go to respective sidecars. Creating potentially large number of rules required to represent the redirection policy is tedious. And the process of applying the rules and updating them, as the target subnets to be captured by the sidecar change, is expensive.</p>
|
||
<p>While some of the performance concerns are being addressed by the Linux community, there is another concern related to privilege: iptables rules need to be updated whenever the policy changes. Given the current architecture, all privileged operations are performed in an init container that runs just once at the very beginning before privileges are dropped for the actual application. Since updating iptables rules requires root privileges, there is no way to do that without restarting the application.</p>
|
||
<p>AppSwitch provides a way to redirect application connections without root privilege. As such, an unprivileged application is already able to connect to any host (modulo firewall rules etc.) and the owner of the application should be allowed to change the host address passed by its application via <code>connect(2)</code> without requiring additional privilege.</p>
|
||
<h4 id="socket-delegation">Socket Delegation</h4>
|
||
<p>Let&rsquo;s see how AppSwitch could help redirect connections without using iptables. Imagine that the application somehow voluntarily passes the socket FDs that it uses for its communication to the sidecar, then there would be no need for iptables. AppSwitch provides a feature called <em>socket delegation</em> that does exactly that. It allows the sidecar to transparently gain access to copies of socket FDs that the application uses for its communication without any changes to the application itself.</p>
|
||
<p>Here are the sequence of steps that would achieve this in the context of the Python application example.</p>
|
||
<ol>
|
||
<li>The application initiates a connection request to the service IP of memcached service.</li>
|
||
<li>The connection request from client is forwarded to the daemon.</li>
|
||
<li>The daemon creates a pair of pre-connected Unix sockets (using <code>socketpair(2)</code> system call).</li>
|
||
<li>It passes one end of the socket pair into the application such that the application would use that socket FD for read/write. It also ensures that the application consistently sees it as a legitimate TCP socket as it expects by interposing all calls that query connection properties.</li>
|
||
<li>The other end is passed to sidecar over a different Unix socket where the daemon exposes its API. Information such as the original destination that the application was connecting to is also conveyed over the same interface.</li>
|
||
</ol>
|
||
<figure style="width:50%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:22.442748091603054%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/delayering-istio/socket-delegation.png" title="Socket delegation based connection redirection">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/delayering-istio/socket-delegation.png" alt="Socket delegation protocol" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Socket delegation based connection redirection</figcaption>
|
||
</figure>
|
||
<p>Once the application and sidecar are connected, the rest happens as usual. Sidecar would initiate a connection to upstream server and proxy data between the socket received from the daemon and the socket connected to upstream server. The main difference here is that sidecar would get the connection, not through the <code>accept(2)</code> system call as it is in the normal case, but from the daemon over the Unix socket. In addition to listening for connections from applications through the normal <code>accept(2)</code> channel, the sidecar proxy would connect to the AppSwitch daemon’s REST endpoint and receive sockets that way.</p>
|
||
<p>For completeness, here are the sequence of steps that would occur on the server side:</p>
|
||
<ol>
|
||
<li>The application receives a connection</li>
|
||
<li>AppSwitch daemon accepts the connection on behalf of the application</li>
|
||
<li>It creates a pair of pre-connected Unix sockets using <code>socketpair(2)</code> system call</li>
|
||
<li>One end of the socket pair is returned to the application through the <code>accept(2)</code> system call</li>
|
||
<li>The other end of the socket pair along with the socket originally accepted by the daemon on behalf of the application is sent to sidecar</li>
|
||
<li>Sidecar would extract the two socket FDs &ndash; a Unix socket FD connected to the application and a TCP socket FD connected to the remote client</li>
|
||
<li>Sidecar would read the metadata supplied by the daemon about the remote client and perform its usual operations</li>
|
||
</ol>
|
||
<h4 id="sidecar-aware-applications">&ldquo;Sidecar-Aware&rdquo; Applications</h4>
|
||
<p>Socket delegation feature can be very useful for applications that are explicitly aware of the sidecar and wish to take advantage of its features. They can voluntarily delegate their network interactions by passing their sockets to the sidecar using the same feature. In a way, AppSwitch transparently turns every application into a sidecar-aware application.</p>
|
||
<h2 id="how-does-it-all-come-together">How does it all come together?</h2>
|
||
<p>Just to step back, Istio offloads common connectivity concerns from applications to a sidecar proxy that performs those functions on behalf of the application. And AppSwitch simplifies and optimizes the service mesh by sidestepping intermediate layers and invoking the proxy only for cases where it is truly necessary.</p>
|
||
<p>In the rest of this section, I outline how AppSwitch may be integrated with Istio based on a very cursory initial implementation. This is not intended to be anything like a design doc &ndash; not every possible way of integration is explored and not every detail is worked out. The intent is to discuss high-level aspects of the implementation to present a rough idea of how the two systems may come together. The key is that AppSwitch would act as a cushion between Istio and a real proxy. It would serve as the &ldquo;fast-path&rdquo; for cases that can be performed more efficiently without invoking the sidecar proxy. And for the cases where the proxy is used, it would shorten the datapath by cutting through unnecessary layers. Look at this <a href="http://appswitch.io/blog/kubernetes_istio_and_network_function_devirtualization_with_appswitch/">blog</a> for a more detailed walk through of the integration.</p>
|
||
<h3 id="appswitch-client-injection">AppSwitch Client Injection</h3>
|
||
<p>Similar to Istio sidecar-injector, a simple tool called <code>ax-injector</code> injects AppSwitch client into a standard Kubernetes manifest. Injected client transparently monitors the application and intimates AppSwitch daemon of the control path network API events that the application produces.</p>
|
||
<p>It is possible to not require the injection and work with standard Kubernetes manifests if AppSwitch CNI plugin is used. In that case, the CNI plugin would perform necessary injection when it gets the initialization callback. Using injector does have some advantages, however: (1) It works in tightly-controlled environments like GKE (2) It can be easily extended to support other frameworks such as Mesos (3) Same cluster would be able to run standard applications alongside &ldquo;AppSwitch-enabled&rdquo; applications.</p>
|
||
<h3 id="appswitch-daemonset">AppSwitch <code>DaemonSet</code></h3>
|
||
<p>AppSwitch daemon can be configured to run as a <code>DaemonSet</code> or as an extension to the application that is directly injected into application manifest. In either case it handles network events coming in from the applications that it supports.</p>
|
||
<h3 id="agent-for-policy-acquisition">Agent for policy acquisition</h3>
|
||
<p>This is the component that conveys policy and configuration dictated by Istio to AppSwitch. It implements xDS API to listen from Pilot and calls appropriate AppSwitch APIs to program the daemon. For example, it allows the load balancing strategy, as specified by <code>istioctl</code>, to be translated into equivalent AppSwitch capability.</p>
|
||
<h3 id="platform-adapter-for-appswitch-auto-curated-service-registry">Platform Adapter for AppSwitch &ldquo;Auto-Curated&rdquo; Service Registry</h3>
|
||
<p>Given that AppSwitch is in the control path of applications’ network APIs, it has ready access to the topology of services across the cluster. AppSwitch exposes that information in the form of a service registry that is automatically and (almost) synchronously updated as applications and their services come and go. A new platform adapter for AppSwitch alongside Kubernetes, Eureka etc. would provide the details of upstream services to Istio. This is not strictly necessary but it does make it easier to correlate service endpoints received from Pilot by AppSwitch agent above.</p>
|
||
<h3 id="proxy-integration-and-chaining">Proxy integration and chaining</h3>
|
||
<p>Connections that do require deep scanning and mutation of application traffic are handed off to an external proxy through the socket delegation mechanism discussed earlier. It uses an extended version of <a href="https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt">proxy protocol</a>. In addition to the simple parameters supported by the proxy protocol, a variety of other metadata (including the initial protocol headers obtained from the socket buffers) and live socket FDs (representing application connections) are forwarded to the proxy.</p>
|
||
<p>The proxy can look at the metadata and decide how to proceed. It could respond by accepting the connection to do the proxying or by directing AppSwitch to allow the connection and use the fast-path or to just drop the connection.</p>
|
||
<p>One of the interesting aspects of the mechanism is that, when the proxy accepts a socket from AppSwitch, it can in turn delegate the socket to another proxy. In fact that is how AppSwitch currently works. It uses a simple built-in proxy to examine the metadata and decide whether to handle the connection internally or to hand it off to an external proxy (Envoy). The same mechanism can be potentially extended to allow for a chain of plugins, each looking for a specific signature, with the last one in the chain doing the real proxy work.</p>
|
||
<h2 id="it-s-not-just-about-performance">It&rsquo;s not just about performance</h2>
|
||
<p>Removing intermediate layers along the datapath is not just about improving performance. Performance is a great side effect, but it <em>is</em> a side effect. There are a number of important advantages to an API level approach.</p>
|
||
<h3 id="automatic-application-onboarding-and-policy-authoring">Automatic application onboarding and policy authoring</h3>
|
||
<p>Before microservices and service mesh, traffic management was done by load balancers and access controls were enforced by firewalls. Applications were identified by IP addresses and DNS names which were relatively static. In fact, that&rsquo;s still the status quo in most environments. Such environments stand to benefit immensely from service mesh. However a practical and scalable bridge to the new world needs to be provided. The difficulty in transformation is not as much due to lack of features and functionality but the investment required to rethink and reimplement the entire application infrastructure. Currently most of the policy and configuration exists in the form of load balancer and firewall rules. Somehow that existing context needs to be leveraged in providing a scalable path to adopting the service mesh model.</p>
|
||
<p>AppSwitch can substantially ease the onboarding process. It can project the same network environment to the application at the target as its current source environment. Not having any assistance here is typically a non-starter in case of traditional applications which have complex configuration files with static IP addresses or specific DNS names hard-coded in them. AppSwitch could help capture those applications along with their existing configuration and connect them over a service mesh without requiring any changes.</p>
|
||
<h3 id="broader-application-and-protocol-support">Broader application and protocol support</h3>
|
||
<p>HTTP clearly dominates the modern application landscapes but once we talk about traditional applications and environments, we&rsquo;d encounter all kinds of protocols and transports. Particularly, support for UDP becomes unavoidable. Traditional application servers such as IBM WebSphere rely extensively on UDP. Most multimedia applications use UDP media streams. Of course DNS is probably the most widely used UDP &ldquo;application&rdquo;. AppSwitch supports UDP at the API level much the same way as TCP and when it detects a UDP connection, it can transparently handle it in its &ldquo;fast-path&rdquo; rather than delegating it to the proxy.</p>
|
||
<h3 id="client-ip-preservation-and-end-to-end-principle">Client IP preservation and end-to-end principle</h3>
|
||
<p>The same mechanism that preserves the source network environment can also preserve client IP addresses as seen by the servers. With a sidecar proxy in place, connection requests come from the proxy rather than the client. As a result, the peer address (IP:port) of the connection as seen by the server would be that of the proxy rather than the client. AppSwitch ensures that the server sees correct address of the client, logs it correctly and any decisions made based on the client address remain valid. More generally, AppSwitch preserves the <a href="https://en.wikipedia.org/wiki/End-to-end_principle">end-to-end principle</a> which is otherwise broken by intermediate layers that obfuscate the true underlying context.</p>
|
||
<h3 id="enhanced-application-signal-with-access-to-encrypted-headers">Enhanced application signal with access to encrypted headers</h3>
|
||
<p>Encrypted traffic completely undermines the ability of the service mesh to analyze application traffic. API level interposition could potentially offer a way around it. Current implementation of AppSwitch gains access to application&rsquo;s network API at the system call level. However it is possible in principle to influence the application at an API boundary, higher in the stack where application data is not yet encrypted or already decrypted. Ultimately the data is always produced in the clear by the application and then encrypted at some point before it goes out. Since AppSwitch directly runs within the memory context of the application, it is possible to tap into the data higher on the stack where it is still held in clear. Only requirement for this to work is that the API used for encryption should be well-defined and amenable for interposition. Particularly, it requires access to the symbol table of the application binaries. Just to be clear, AppSwitch doesn&rsquo;t implement this today.</p>
|
||
<h2 id="so-what-s-the-net">So what’s the net?</h2>
|
||
<p>AppSwitch removes a number of layers and processing from the standard service mesh stack. What does all that translate to in terms of performance?</p>
|
||
<p>We ran some initial experiments to characterize the extent of the opportunity for optimization based on the initial integration of AppSwitch discussed earlier. The experiments were run on GKE using <code>fortio-0.11.0</code>, <code>istio-0.8.0</code> and <code>appswitch-0.4.0-2</code>. In case of the proxyless test, AppSwitch daemon was run as a <code>DaemonSet</code> on the Kubernetes cluster and the Fortio pod spec was modified to inject AppSwitch client. These were the only two changes made to the setup. The test was configured to measure the latency of GRPC requests across 100 concurrent connections.</p>
|
||
<figure style="width:100%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:54.66034755134282%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/delayering-istio/perf.png" title="Latency with and without AppSwitch">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/delayering-istio/perf.png" alt="Performance comparison" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Latency with and without AppSwitch</figcaption>
|
||
</figure>
|
||
<p>Initial results indicate a difference of over 18x in p50 latency with and without AppSwitch (3.99ms vs 72.96ms). The difference was around 8x when mixer and access logs were disabled. Clearly the difference was due to sidestepping all those intermediate layers along the datapath. Unix socket optimization wasn&rsquo;t triggered in case of AppSwitch because client and server pods were scheduled to separate hosts. End-to-end latency of AppSwitch case would have been even lower if the client and server happened to be colocated. Essentially the client and server running in their respective pods of the Kubernetes cluster are directly connected over a TCP socket going over the GKE network &ndash; no tunneling, bridge or proxies.</p>
|
||
<h2 id="net-net">Net Net</h2>
|
||
<p>I started out with David Wheeler&rsquo;s seemingly reasonable quote that says adding another layer is not a solution for the problem of too many layers. And I argued through most of the blog that current network stack already has too many layers and that they should be removed. But isn&rsquo;t AppSwitch itself a layer?</p>
|
||
<p>Yes, AppSwitch is clearly another layer. However it is one that can remove multiple other layers. In doing so, it seamlessly glues the new service mesh layer with existing layers of traditional network environments. It offsets the cost of sidecar proxy and as Istio graduates to 1.0, it provides a bridge for existing applications and their network environments to transition to the new world of service mesh.</p>
|
||
<p>Perhaps Wheeler’s quote should read:</p>
|
||
<div>
|
||
<aside class="callout quote">
|
||
<div class="type">
|
||
<svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-quote"/></svg>
|
||
</div>
|
||
<div class="content">All problems in computer science can be solved with another layer, <strong>even</strong> the problem of too many layers!</div>
|
||
</aside>
|
||
</div>
|
||
<h2 id="acknowledgements">Acknowledgements</h2>
|
||
<p>Thanks to Mandar Jog (Google) for several discussions about the value of AppSwitch for Istio and to the following individuals (in alphabetical order) for their review of early drafts of this blog.</p>
|
||
<ul>
|
||
<li>Frank Budinsky (IBM)</li>
|
||
<li>Lin Sun (IBM)</li>
|
||
<li>Shriram Rajagopalan (VMware)</li>
|
||
</ul></description><pubDate>Mon, 30 Jul 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/delayering-istio/</link><author>Dinesh Subhraveti (AppOrbit and Columbia University)</author><guid isPermaLink="true">/v1.1/blog/2018/delayering-istio/</guid><category>appswitch</category><category>performance</category></item><item><title>Micro-Segmentation with Istio Authorization</title><description>
|
||
<p>Micro-segmentation is a security technique that creates secure zones in cloud deployments and allows organizations to
|
||
isolate workloads from one another and secure them individually.
|
||
<a href="/v1.1/docs/concepts/security/#authorization">Istio&rsquo;s authorization feature</a>, also known as Istio Role Based Access Control,
|
||
provides micro-segmentation for services in an Istio mesh. It features:</p>
|
||
<ul>
|
||
<li>Authorization at different levels of granularity, including namespace level, service level, and method level.</li>
|
||
<li>Service-to-service and end-user-to-service authorization.</li>
|
||
<li>High performance, as it is enforced natively on Envoy.</li>
|
||
<li>Role-based semantics, which makes it easy to use.</li>
|
||
<li>High flexibility as it allows users to define conditions using
|
||
<a href="/v1.1/docs/reference/config/authorization/constraints-and-properties/">combinations of attributes</a>.</li>
|
||
</ul>
|
||
<p>In this blog post, you&rsquo;ll learn about the main authorization features and how to use them in different situations.</p>
|
||
<h2 id="characteristics">Characteristics</h2>
|
||
<h3 id="rpc-level-authorization">RPC level authorization</h3>
|
||
<p>Authorization is performed at the level of individual RPCs. Specifically, it controls &ldquo;who can access my <code>bookstore</code> service”,
|
||
or &ldquo;who can access method <code>getBook</code> in my <code>bookstore</code> service”. It is not designed to control access to application-specific
|
||
resource instances, like access to &ldquo;storage bucket X” or access to &ldquo;3rd book on 2nd shelf”. Today this kind of application
|
||
specific access control logic needs to be handled by the application itself.</p>
|
||
<h3 id="role-based-access-control-with-conditions">Role-based access control with conditions</h3>
|
||
<p>Authorization is a <a href="https://en.wikipedia.org/wiki/Role-based_access_control">role-based access control (RBAC)</a> system,
|
||
contrast this to an <a href="https://en.wikipedia.org/wiki/Attribute-based_access_control">attribute-based access control (ABAC)</a>
|
||
system. Compared to ABAC, RBAC has the following advantages:</p>
|
||
<ul>
|
||
<li><p><strong>Roles allow grouping of attributes.</strong> Roles are groups of permissions, which specifies the actions you are allowed
|
||
to perform on a system. Users are grouped based on the roles within an organization. You can define the roles and reuse
|
||
them for different cases.</p></li>
|
||
<li><p><strong>It is easier to understand and reason about who has access.</strong> The RBAC concepts map naturally to business concepts.
|
||
For example, a DB admin may have all access to DB backend services, while a web client may only be able to view the
|
||
frontend service.</p></li>
|
||
<li><p><strong>It reduces unintentional errors.</strong> RBAC policies make otherwise complex security changes easier. You won&rsquo;t have
|
||
duplicate configurations in multiple places and later forget to update some of them when you need to make changes.</p></li>
|
||
</ul>
|
||
<p>On the other hand, Istio&rsquo;s authorization system is not a traditional RBAC system. It also allows users to define <strong>conditions</strong> using
|
||
<a href="/v1.1/docs/reference/config/authorization/constraints-and-properties/">combinations of attributes</a>. This gives Istio
|
||
flexibility to express complex access control policies. In fact, <strong>the &ldquo;RBAC + conditions” model
|
||
that Istio authorization adopts, has all the benefits an RBAC system has, and supports the level of flexibility that
|
||
normally an ABAC system provides.</strong> You&rsquo;ll see some <a href="#examples">examples</a> below.</p>
|
||
<h3 id="high-performance">High performance</h3>
|
||
<p>Because of its simple semantics, Istio authorization is enforced on Envoy as a native authorization support. At runtime, the
|
||
authorization decision is completely done locally inside an Envoy filter, without dependency to any external module.
|
||
This allows Istio authorization to achieve high performance and availability.</p>
|
||
<h3 id="work-with-without-primary-identities">Work with/without primary identities</h3>
|
||
<p>Like any other RBAC system, Istio authorization is identity aware. In Istio authorization policy, there is a primary
|
||
identity called <code>user</code>, which represents the principal of the client.</p>
|
||
<p>In addition to the primary identity, you can also specify any conditions that define the identities. For example,
|
||
you can specify the client identity as &ldquo;user Alice calling from Bookstore frontend service”, in which case,
|
||
you have a combined identity of the calling service (<code>Bookstore frontend</code>) and the end user (<code>Alice</code>).</p>
|
||
<p>To improve security, you should enable <a href="/v1.1/docs/concepts/security/#authentication">authentication features</a>,
|
||
and use authenticated identities in authorization policies. However, strongly authenticated identity is not required
|
||
for using authorization. Istio authorization works with or without identities. If you are working with a legacy system,
|
||
you may not have mutual TLS or JWT authentication setup for your mesh. In this case, the only way to identify the client is, for example,
|
||
through IP. You can still use Istio authorization to control which IP addresses or IP ranges are allowed to access your service.</p>
|
||
<h2 id="examples">Examples</h2>
|
||
<p>The <a href="/v1.1/docs/tasks/security/authz-http/">authorization task</a> shows you how to
|
||
use Istio&rsquo;s authorization feature to control namespace level and service level access using the
|
||
<a href="/v1.1/docs/examples/bookinfo/">Bookinfo application</a>. In this section, you&rsquo;ll see more examples on how to achieve
|
||
micro-segmentation with Istio authorization.</p>
|
||
<h3 id="namespace-level-segmentation-via-rbac-conditions">Namespace level segmentation via RBAC + conditions</h3>
|
||
<p>Suppose you have services in the <code>frontend</code> and <code>backend</code> namespaces. You would like to allow all your services
|
||
in the <code>frontend</code> namespace to access all services that are marked <code>external</code> in the <code>backend</code> namespace.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: &#34;rbac.istio.io/v1alpha1&#34;
|
||
kind: ServiceRole
|
||
metadata:
|
||
name: external-api-caller
|
||
namespace: backend
|
||
spec:
|
||
rules:
|
||
- services: [&#34;*&#34;]
|
||
methods: [&#34;*”]
|
||
constraints:
|
||
- key: &#34;destination.labels[visibility]”
|
||
values: [&#34;external&#34;]
|
||
---
|
||
apiVersion: &#34;rbac.istio.io/v1alpha1&#34;
|
||
kind: ServiceRoleBinding
|
||
metadata:
|
||
name: external-api-caller
|
||
namespace: backend
|
||
spec:
|
||
subjects:
|
||
- properties:
|
||
source.namespace: &#34;frontend”
|
||
roleRef:
|
||
kind: ServiceRole
|
||
name: &#34;external-api-caller&#34;
|
||
</code></pre>
|
||
<p>The <code>ServiceRole</code> and <code>ServiceRoleBinding</code> above expressed &ldquo;<em>who</em> is allowed to do <em>what</em> under *which conditions*”
|
||
(RBAC + conditions). Specifically:</p>
|
||
<ul>
|
||
<li><strong>&ldquo;who”</strong> are the services in the <code>frontend</code> namespace.</li>
|
||
<li><strong>&ldquo;what”</strong> is to call services in <code>backend</code> namespace.</li>
|
||
<li><strong>&ldquo;conditions”</strong> is the <code>visibility</code> label of the destination service having the value <code>external</code>.</li>
|
||
</ul>
|
||
<h3 id="service-method-level-isolation-with-without-primary-identities">Service/method level isolation with/without primary identities</h3>
|
||
<p>Here is another example that demonstrates finer grained access control at service/method level. The first step
|
||
is to define a <code>book-reader</code> <code>ServiceRole</code> that allows READ access to <code>/books/*</code> resource in <code>bookstore</code> service.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: &#34;rbac.istio.io/v1alpha1&#34;
|
||
kind: ServiceRole
|
||
metadata:
|
||
name: book-reader
|
||
namespace: default
|
||
spec:
|
||
rules:
|
||
- services: [&#34;bookstore.default.svc.cluster.local&#34;]
|
||
paths: [&#34;/books/*”]
|
||
methods: [&#34;GET”]
|
||
</code></pre>
|
||
<h4 id="using-authenticated-client-identities">Using authenticated client identities</h4>
|
||
<p>Suppose you want to grant this <code>book-reader</code> role to your <code>bookstore-frontend</code> service. If you have enabled
|
||
<a href="/v1.1/docs/concepts/security/#mutual-tls-authentication">mutual TLS authentication</a> for your mesh, you can use a
|
||
service account to identify your <code>bookstore-frontend</code> service. Granting the <code>book-reader</code> role to the <code>bookstore-frontend</code>
|
||
service can be done by creating a <code>ServiceRoleBinding</code> as shown below:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: &#34;rbac.istio.io/v1alpha1&#34;
|
||
kind: ServiceRoleBinding
|
||
metadata:
|
||
name: book-reader
|
||
namespace: default
|
||
spec:
|
||
subjects:
|
||
- user: &#34;cluster.local/ns/default/sa/bookstore-frontend”
|
||
roleRef:
|
||
kind: ServiceRole
|
||
name: &#34;book-reader&#34;
|
||
</code></pre>
|
||
<p>You may want to restrict this further by adding a condition that &ldquo;only users who belong to the <code>qualified-reviewer</code> group are
|
||
allowed to read books”. The <code>qualified-reviewer</code> group is the end user identity that is authenticated by
|
||
<a href="/v1.1/docs/concepts/security/#authentication">JWT authentication</a>. In this case, the combination of the client service identity
|
||
(<code>bookstore-frontend</code>) and the end user identity (<code>qualified-reviewer</code>) is used in the authorization policy.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: &#34;rbac.istio.io/v1alpha1&#34;
|
||
kind: ServiceRoleBinding
|
||
metadata:
|
||
name: book-reader
|
||
namespace: default
|
||
spec:
|
||
subjects:
|
||
- user: &#34;cluster.local/ns/default/sa/bookstore-frontend”
|
||
properties:
|
||
request.auth.claims[group]: &#34;qualified-reviewer”
|
||
roleRef:
|
||
kind: ServiceRole
|
||
name: &#34;book-reader&#34;
|
||
</code></pre>
|
||
<h4 id="client-does-not-have-identity">Client does not have identity</h4>
|
||
<p>Using authenticated identities in authorization policies is strongly recommended for security. However, if you have a
|
||
legacy system that does not support authentication, you may not have authenticated identities for your services.
|
||
You can still use Istio authorization to protect your services even without authenticated identities. The example below
|
||
shows that you can specify allowed source IP range in your authorization policy.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: &#34;rbac.istio.io/v1alpha1&#34;
|
||
kind: ServiceRoleBinding
|
||
metadata:
|
||
name: book-reader
|
||
namespace: default
|
||
spec:
|
||
subjects:
|
||
- properties:
|
||
source.ip: 10.20.0.0/9
|
||
roleRef:
|
||
kind: ServiceRole
|
||
name: &#34;book-reader&#34;
|
||
</code></pre>
|
||
<h2 id="summary">Summary</h2>
|
||
<p>Istio’s authorization feature provides authorization at namespace-level, service-level, and method-level granularity.
|
||
It adopts &ldquo;RBAC + conditions” model, which makes it easy to use and understand as an RBAC system, while providing the level of
|
||
flexibility that an ABAC system normally provides. Istio authorization achieves high performance as it is enforced
|
||
natively on Envoy. While it provides the best security by working together with
|
||
<a href="/v1.1/docs/concepts/security/#authentication">Istio authentication features</a>, Istio authorization can also be used to
|
||
provide access control for legacy systems that do not have authentication.</p></description><pubDate>Fri, 20 Jul 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/istio-authorization/</link><author>Limin Wang</author><guid isPermaLink="true">/v1.1/blog/2018/istio-authorization/</guid><category>authorization</category><category>rbac</category><category>security</category></item><item><title>Exporting Logs to BigQuery, GCS, Pub/Sub through Stackdriver</title><description>
|
||
<p>This post shows how to direct Istio logs to <a href="https://cloud.google.com/stackdriver/">Stackdriver</a>
|
||
and export those logs to various configured sinks such as such as
|
||
<a href="https://cloud.google.com/bigquery/">BigQuery</a>, <a href="https://cloud.google.com/storage/">Google Cloud Storage</a>
|
||
or <a href="https://cloud.google.com/pubsub/">Cloud Pub/Sub</a>. At the end of this post you can perform
|
||
analytics on Istio data from your favorite places such as BigQuery, GCS or Cloud Pub/Sub.</p>
|
||
<p>The <a href="/v1.1/docs/examples/bookinfo/">Bookinfo</a> sample application is used as the example
|
||
application throughout this task.</p>
|
||
<h2 id="before-you-begin">Before you begin</h2>
|
||
<p><a href="/v1.1/docs/setup/">Install Istio</a> in your cluster and deploy an application.</p>
|
||
<h2 id="configuring-istio-to-export-logs">Configuring Istio to export logs</h2>
|
||
<p>Istio exports logs using the <code>logentry</code> <a href="/v1.1/docs/reference/config/policy-and-telemetry/templates/logentry">template</a>.
|
||
This specifies all the variables that are available for analysis. It
|
||
contains information like source service, destination service, auth
|
||
metrics (coming..) among others. Following is a diagram of the pipeline:</p>
|
||
<figure style="width:75%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:75%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/export-logs-through-stackdriver/./istio-analytics-using-stackdriver.png" title="Exporting logs from Istio to Stackdriver for analysis">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/export-logs-through-stackdriver/./istio-analytics-using-stackdriver.png" alt="Exporting logs from Istio to Stackdriver for analysis" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Exporting logs from Istio to Stackdriver for analysis</figcaption>
|
||
</figure>
|
||
<p>Istio supports exporting logs to Stackdriver which can in turn be configured to export
|
||
logs to your favorite sink like BigQuery, Pub/Sub or GCS. Please follow the steps
|
||
below to setup your favorite sink for exporting logs first and then Stackdriver
|
||
in Istio.</p>
|
||
<h3 id="setting-up-various-log-sinks">Setting up various log sinks</h3>
|
||
<p>Common setup for all sinks:</p>
|
||
<ol>
|
||
<li>Enable <a href="https://cloud.google.com/monitoring/api/enable-api">Stackdriver Monitoring API</a> for the project.</li>
|
||
<li>Make sure <code>principalEmail</code> that would be setting up the sink has write access to the project and Logging Admin role permissions.</li>
|
||
<li>Make sure the <code>GOOGLE_APPLICATION_CREDENTIALS</code> environment variable is set. Please follow instructions <a href="https://cloud.google.com/docs/authentication/getting-started">here</a> to set it up.</li>
|
||
</ol>
|
||
<h4 id="bigquery">BigQuery</h4>
|
||
<ol>
|
||
<li><a href="https://cloud.google.com/bigquery/docs/datasets">Create a BigQuery dataset</a> as a destination for the logs export.</li>
|
||
<li>Record the ID of the dataset. It will be needed to configure the Stackdriver handler.
|
||
It would be of the form <code>bigquery.googleapis.com/projects/[PROJECT_ID]/datasets/[DATASET_ID]</code></li>
|
||
<li>Give <a href="https://cloud.google.com/logging/docs/api/tasks/exporting-logs#writing_to_the_destination">sink’s writer identity</a>: <code>cloud-logs@system.gserviceaccount.com</code> BigQuery Data Editor role in IAM.</li>
|
||
<li>If using <a href="/v1.1/docs/setup/kubernetes/prepare/platform-setup/gke/">Google Kubernetes Engine</a>, make sure <code>bigquery</code> <a href="https://cloud.google.com/sdk/gcloud/reference/container/clusters/create">Scope</a> is enabled on the cluster.</li>
|
||
</ol>
|
||
<h4 id="google-cloud-storage-gcs">Google Cloud Storage (GCS)</h4>
|
||
<ol>
|
||
<li><a href="https://cloud.google.com/storage/docs/creating-buckets">Create a GCS bucket</a> where you would like logs to get exported in GCS.</li>
|
||
<li>Recode the ID of the bucket. It will be needed to configure Stackdriver.
|
||
It would be of the form <code>storage.googleapis.com/[BUCKET_ID]</code></li>
|
||
<li>Give <a href="https://cloud.google.com/logging/docs/api/tasks/exporting-logs#writing_to_the_destination">sink’s writer identity</a>: <code>cloud-logs@system.gserviceaccount.com</code> Storage Object Creator role in IAM.</li>
|
||
</ol>
|
||
<h4 id="google-cloud-pub-sub">Google Cloud Pub/Sub</h4>
|
||
<ol>
|
||
<li><a href="https://cloud.google.com/pubsub/docs/admin">Create a topic</a> where you would like logs to get exported in Google Cloud Pub/Sub.</li>
|
||
<li>Recode the ID of the topic. It will be needed to configure Stackdriver.
|
||
It would be of the form <code>pubsub.googleapis.com/projects/[PROJECT_ID]/topics/[TOPIC_ID]</code></li>
|
||
<li>Give <a href="https://cloud.google.com/logging/docs/api/tasks/exporting-logs#writing_to_the_destination">sink’s writer identity</a>: <code>cloud-logs@system.gserviceaccount.com</code> Pub/Sub Publisher role in IAM.</li>
|
||
<li>If using <a href="/v1.1/docs/setup/kubernetes/prepare/platform-setup/gke/">Google Kubernetes Engine</a>, make sure <code>pubsub</code> <a href="https://cloud.google.com/sdk/gcloud/reference/container/clusters/create">Scope</a> is enabled on the cluster.</li>
|
||
</ol>
|
||
<h3 id="setting-up-stackdriver">Setting up Stackdriver</h3>
|
||
<p>A Stackdriver handler must be created to export data to Stackdriver. The configuration for
|
||
a Stackdriver handler is described <a href="/v1.1/docs/reference/config/policy-and-telemetry/adapters/stackdriver/">here</a>.</p>
|
||
<ol>
|
||
<li><p>Save the following yaml file as <code>stackdriver.yaml</code>. Replace <code>&lt;project_id&gt;,
|
||
&lt;sink_id&gt;, &lt;sink_destination&gt;, &lt;log_filter&gt;</code> with their specific values.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: stackdriver
|
||
metadata:
|
||
name: handler
|
||
namespace: istio-system
|
||
spec:
|
||
# We&#39;ll use the default value from the adapter, once per minute, so we don&#39;t need to supply a value.
|
||
# pushInterval: 1m
|
||
# Must be supplied for the Stackdriver adapter to work
|
||
project_id: &#34;&lt;project_id&gt;&#34;
|
||
# One of the following must be set; the preferred method is `appCredentials`, which corresponds to
|
||
# Google Application Default Credentials.
|
||
# If none is provided we default to app credentials.
|
||
# appCredentials:
|
||
# apiKey:
|
||
# serviceAccountPath:
|
||
# Describes how to map Istio logs into Stackdriver.
|
||
logInfo:
|
||
accesslog.logentry.istio-system:
|
||
payloadTemplate: &#39;{{or (.sourceIp) &#34;-&#34;}} - {{or (.sourceUser) &#34;-&#34;}} [{{or (.timestamp.Format &#34;02/Jan/2006:15:04:05 -0700&#34;) &#34;-&#34;}}] &#34;{{or (.method) &#34;-&#34;}} {{or (.url) &#34;-&#34;}} {{or (.protocol) &#34;-&#34;}}&#34; {{or (.responseCode) &#34;-&#34;}} {{or (.responseSize) &#34;-&#34;}}&#39;
|
||
httpMapping:
|
||
url: url
|
||
status: responseCode
|
||
requestSize: requestSize
|
||
responseSize: responseSize
|
||
latency: latency
|
||
localIp: sourceIp
|
||
remoteIp: destinationIp
|
||
method: method
|
||
userAgent: userAgent
|
||
referer: referer
|
||
labelNames:
|
||
- sourceIp
|
||
- destinationIp
|
||
- sourceService
|
||
- sourceUser
|
||
- sourceNamespace
|
||
- destinationIp
|
||
- destinationService
|
||
- destinationNamespace
|
||
- apiClaims
|
||
- apiKey
|
||
- protocol
|
||
- method
|
||
- url
|
||
- responseCode
|
||
- responseSize
|
||
- requestSize
|
||
- latency
|
||
- connectionMtls
|
||
- userAgent
|
||
- responseTimestamp
|
||
- receivedBytes
|
||
- sentBytes
|
||
- referer
|
||
sinkInfo:
|
||
id: &#39;&lt;sink_id&gt;&#39;
|
||
destination: &#39;&lt;sink_destination&gt;&#39;
|
||
filter: &#39;&lt;log_filter&gt;&#39;
|
||
---
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: rule
|
||
metadata:
|
||
name: stackdriver
|
||
namespace: istio-system
|
||
spec:
|
||
match: &#34;true&#34; # If omitted match is true.
|
||
actions:
|
||
- handler: handler.stackdriver
|
||
instances:
|
||
- accesslog.logentry
|
||
---
|
||
</code></pre></li>
|
||
<li><p>Push the configuration</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f stackdriver.yaml
|
||
stackdriver &#34;handler&#34; created
|
||
rule &#34;stackdriver&#34; created
|
||
logentry &#34;stackdriverglobalmr&#34; created
|
||
metric &#34;stackdriverrequestcount&#34; created
|
||
metric &#34;stackdriverrequestduration&#34; created
|
||
metric &#34;stackdriverrequestsize&#34; created
|
||
metric &#34;stackdriverresponsesize&#34; created
|
||
</code></pre></li>
|
||
<li><p>Send traffic to the sample application.</p>
|
||
<p>For the Bookinfo sample, visit <code>http://$GATEWAY_URL/productpage</code> in your web
|
||
browser or issue the following command:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ curl http://$GATEWAY_URL/productpage
|
||
</code></pre></li>
|
||
<li><p>Verify that logs are flowing through Stackdriver to the configured sink.</p>
|
||
<ul>
|
||
<li>Stackdriver: Navigate to the <a href="https://pantheon.corp.google.com/logs/viewer">Stackdriver Logs
|
||
Viewer</a> for your project
|
||
and look under &ldquo;GKE Container&rdquo; -&gt; &ldquo;Cluster Name&rdquo; -&gt; &ldquo;Namespace Id&rdquo; for
|
||
Istio Access logs.</li>
|
||
<li>BigQuery: Navigate to the <a href="https://bigquery.cloud.google.com/">BigQuery
|
||
Interface</a> for your project and you
|
||
should find a table with prefix <code>accesslog_logentry_istio</code> in your sink
|
||
dataset.</li>
|
||
<li>GCS: Navigate to the <a href="https://pantheon.corp.google.com/storage/browser/">Storage
|
||
Browser</a> for your
|
||
project and you should find a bucket named
|
||
<code>accesslog.logentry.istio-system</code> in your sink bucket.</li>
|
||
<li>Pub/Sub: Navigate to the <a href="https://pantheon.corp.google.com/cloudpubsub/topicList">Pub/Sub
|
||
Topic List</a> for
|
||
your project and you should find a topic for <code>accesslog</code> in your sink
|
||
topic.</li>
|
||
</ul></li>
|
||
</ol>
|
||
<h2 id="understanding-what-happened">Understanding what happened</h2>
|
||
<p><code>Stackdriver.yaml</code> file above configured Istio to send access logs to
|
||
Stackdriver and then added a sink configuration where these logs could be
|
||
exported. In detail as follows:</p>
|
||
<ol>
|
||
<li><p>Added a handler of kind <code>stackdriver</code></p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: stackdriver
|
||
metadata:
|
||
name: handler
|
||
namespace: &lt;your defined namespace&gt;
|
||
</code></pre></li>
|
||
<li><p>Added logInfo in spec</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >spec:
|
||
logInfo: accesslog.logentry.istio-system:
|
||
labelNames:
|
||
- sourceIp
|
||
- destinationIp
|
||
...
|
||
...
|
||
sinkInfo:
|
||
id: &#39;&lt;sink_id&gt;&#39;
|
||
destination: &#39;&lt;sink_destination&gt;&#39;
|
||
filter: &#39;&lt;log_filter&gt;&#39;
|
||
</code></pre>
|
||
<p>In the above configuration sinkInfo contains information about the sink where you want
|
||
the logs to get exported to. For more information on how this gets filled for different sinks please refer
|
||
<a href="https://cloud.google.com/logging/docs/export/#sink-terms">here</a>.</p></li>
|
||
<li><p>Added a rule for Stackdriver</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: rule
|
||
metadata:
|
||
name: stackdriver
|
||
namespace: istio-system spec:
|
||
match: &#34;true&#34; # If omitted match is true
|
||
actions:
|
||
- handler: handler.stackdriver
|
||
instances:
|
||
- accesslog.logentry
|
||
</code></pre></li>
|
||
</ol>
|
||
<h2 id="cleanup">Cleanup</h2>
|
||
<ul>
|
||
<li><p>Remove the new Stackdriver configuration:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete -f stackdriver.yaml
|
||
</code></pre></li>
|
||
<li><p>If you are not planning to explore any follow-on tasks, refer to the
|
||
<a href="/v1.1/docs/examples/bookinfo/#cleanup">Bookinfo cleanup</a> instructions to shutdown
|
||
the application.</p></li>
|
||
</ul>
|
||
<h2 id="availability-of-logs-in-export-sinks">Availability of logs in export sinks</h2>
|
||
<p>Export to BigQuery is within minutes (we see it to be almost instant), GCS can
|
||
have a delay of 2 to 12 hours and Pub/Sub is almost immediately.</p></description><pubDate>Mon, 09 Jul 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/export-logs-through-stackdriver/</link><author>Nupur Garg and Douglas Reid</author><guid isPermaLink="true">/v1.1/blog/2018/export-logs-through-stackdriver/</guid></item><item><title>Monitoring and Access Policies for HTTP Egress Traffic</title><description>
|
||
<p>While Istio&rsquo;s main focus is management of traffic between microservices inside a service mesh, Istio can also manage
|
||
ingress (from outside into the mesh) and egress (from the mesh outwards) traffic. Istio can uniformly enforce access
|
||
policies and aggregate telemetry data for mesh-internal, ingress and egress traffic.</p>
|
||
<p>In this blog post, we show how to apply monitoring and access policies to HTTP egress traffic with Istio.</p>
|
||
<h2 id="use-case">Use case</h2>
|
||
<p>Consider an organization that runs applications that process content from <em>cnn.com</em>. The applications are decomposed
|
||
into microservices deployed in an Istio service mesh. The applications access pages of various topics from <em>cnn.com</em>: <a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a>, <a href="https://edition.cnn.com/sport">edition.cnn.com/sport</a> and <a href="https://edition.cnn.com/health">edition.cnn.com/health</a>. The organization <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway-tls-origination/">configures Istio to allow access to edition.cnn.com</a> and everything works fine. However, at some
|
||
point in time, the organization decides to banish politics. Practically, it means blocking access to
|
||
<a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> and allowing access to
|
||
<a href="https://edition.cnn.com/sport">edition.cnn.com/sport</a> and <a href="https://edition.cnn.com/health">edition.cnn.com/health</a>
|
||
only. The organization will grant permissions to individual applications and to particular users to access <a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a>, on a case-by-case basis.</p>
|
||
<p>To achieve that goal, the organization&rsquo;s operations people monitor access to the external services and
|
||
analyze Istio logs to verify that no unauthorized request was sent to
|
||
<a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a>. They also configure Istio to prevent access to
|
||
<a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> automatically.</p>
|
||
<p>The organization is resolved to prevent any tampering with the new policy. It decides to put mechanisms in place that
|
||
will prevent any possibility for a malicious application to access the forbidden topic.</p>
|
||
<h2 id="related-tasks-and-examples">Related tasks and examples</h2>
|
||
<ul>
|
||
<li>The <a href="/v1.1/docs/tasks/traffic-management/egress/">Control Egress Traffic</a> task demonstrates how external (outside the
|
||
Kubernetes cluster) HTTP and HTTPS services can be accessed by applications inside the mesh.</li>
|
||
<li>The <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway/">Configure an Egress Gateway</a> example describes how to configure
|
||
Istio to direct egress traffic through a dedicated gateway service called <em>egress gateway</em>.</li>
|
||
<li>The <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway-tls-origination/">Egress Gateway with TLS Origination</a> example
|
||
demonstrates how to allow applications to send HTTP requests to external servers that require HTTPS, while directing
|
||
traffic through egress gateway.</li>
|
||
<li>The <a href="/v1.1/docs/tasks/telemetry/metrics/collecting-metrics/">Collecting Metrics</a> task describes how to configure metrics for services in a mesh.</li>
|
||
<li>The <a href="/v1.1/docs/tasks/telemetry/metrics/using-istio-dashboard/">Visualizing Metrics with Grafana</a>
|
||
describes the Istio Dashboard to monitor mesh traffic.</li>
|
||
<li>The <a href="/v1.1/docs/tasks/policy-enforcement/denial-and-list/">Basic Access Control</a> task shows how to control access to
|
||
in-mesh services.</li>
|
||
<li>The <a href="/v1.1/docs/tasks/policy-enforcement/denial-and-list/">Denials and White/Black Listing</a> task shows how to configure
|
||
access policies using black or white list checkers.</li>
|
||
</ul>
|
||
<p>As opposed to the telemetry and security tasks above, this blog post describes Istio&rsquo;s monitoring and access policies
|
||
applied exclusively to the egress traffic.</p>
|
||
<h2 id="before-you-begin">Before you begin</h2>
|
||
<p>Follow the steps in the <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway-tls-origination/">Egress Gateway with TLS Origination</a> example, <strong>with mutual TLS authentication enabled</strong>, without
|
||
the <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway-tls-origination//#cleanup">Cleanup</a> step.
|
||
After completing that example, you can access <a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> from an in-mesh container with <code>curl</code> installed. This blog post assumes that the <code>SOURCE_POD</code> environment variable contains the source pod&rsquo;s name and that the container&rsquo;s name is <code>sleep</code>.</p>
|
||
<h2 id="configure-monitoring-and-access-policies">Configure monitoring and access policies</h2>
|
||
<p>Since you want to accomplish your tasks in a <em>secure way</em>, you should direct egress traffic through
|
||
<em>egress gateway</em>, as described in the <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway-tls-origination/">Egress Gateway with TLS Origination</a>
|
||
task. The <em>secure way</em> here means that you want to prevent malicious applications from bypassing Istio monitoring and
|
||
policy enforcement.</p>
|
||
<p>According to our scenario, the organization performed the instructions in the
|
||
<a href="#before-you-begin">Before you begin</a> section, enabled HTTP traffic to <em>edition.cnn.com</em>, and configured that traffic
|
||
to pass through the egress gateway. The egress gateway performs TLS origination to <em>edition.cnn.com</em>, so the traffic
|
||
leaves the mesh encrypted. At this point, the organization is ready to configure Istio to monitor and apply access policies for
|
||
the traffic to <em>edition.cnn.com</em>.</p>
|
||
<h3 id="logging">Logging</h3>
|
||
<p>Configure Istio to log access to <em>*.cnn.com</em>. You create a <code>logentry</code> and two
|
||
<a href="/v1.1/docs/reference/config/policy-and-telemetry/adapters/stdio/">stdio</a> <code>handlers</code>, one for logging forbidden access
|
||
(<em>error</em> log level) and another one for logging all access to <em>*.cnn.com</em> (<em>info</em> log level). Then you create <code>rules</code> to
|
||
direct your <code>logentry</code> instances to your <code>handlers</code>. One rule directs access to <em>*.cnn.com/politics</em> to the handler for
|
||
logging forbidden access, another rule directs log entries to the handler that outputs each access to <em>*.cnn.com</em> as an
|
||
<em>info</em> log entry. To understand the Istio <code>logentries</code>, <code>rules</code>, and <code>handlers</code>, see
|
||
<a href="/v1.1/blog/2017/adapter-model/">Istio Adapter Model</a>. A diagram with the involved entities and dependencies between them
|
||
appears below:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:46.46700562636976%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-monitoring-access-control/egress-adapters-monitoring.svg" title="Instances, rules and handlers for egress monitoring">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-monitoring-access-control/egress-adapters-monitoring.svg" alt="Instances, rules and handlers for egress monitoring" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Instances, rules and handlers for egress monitoring</figcaption>
|
||
</figure>
|
||
<ol>
|
||
<li><p>Create the <code>logentry</code>, <code>rules</code> and <code>handlers</code>. Note that you specify <code>context.reporter.uid</code> as
|
||
<code>kubernetes://istio-egressgateway</code> in the rules to get logs from the egress gateway only.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | kubectl apply -f -
|
||
# Log entry for egress access
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: logentry
|
||
metadata:
|
||
name: egress-access
|
||
namespace: istio-system
|
||
spec:
|
||
severity: &#39;&#34;info&#34;&#39;
|
||
timestamp: request.time
|
||
variables:
|
||
destination: request.host | &#34;unknown&#34;
|
||
path: request.path | &#34;unknown&#34;
|
||
responseCode: response.code | 0
|
||
responseSize: response.size | 0
|
||
reporterUID: context.reporter.uid | &#34;unknown&#34;
|
||
sourcePrincipal: source.principal | &#34;unknown&#34;
|
||
monitored_resource_type: &#39;&#34;UNSPECIFIED&#34;&#39;
|
||
---
|
||
# Handler for error egress access entries
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: stdio
|
||
metadata:
|
||
name: egress-error-logger
|
||
namespace: istio-system
|
||
spec:
|
||
severity_levels:
|
||
info: 2 # output log level as error
|
||
outputAsJson: true
|
||
---
|
||
# Rule to handle access to *.cnn.com/politics
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: rule
|
||
metadata:
|
||
name: handle-politics
|
||
namespace: istio-system
|
||
spec:
|
||
match: request.host.endsWith(&#34;cnn.com&#34;) &amp;&amp; request.path.startsWith(&#34;/politics&#34;) &amp;&amp; context.reporter.uid.startsWith(&#34;kubernetes://istio-egressgateway&#34;)
|
||
actions:
|
||
- handler: egress-error-logger.stdio
|
||
instances:
|
||
- egress-access.logentry
|
||
---
|
||
# Handler for info egress access entries
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: stdio
|
||
metadata:
|
||
name: egress-access-logger
|
||
namespace: istio-system
|
||
spec:
|
||
severity_levels:
|
||
info: 0 # output log level as info
|
||
outputAsJson: true
|
||
---
|
||
# Rule to handle access to *.cnn.com
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: rule
|
||
metadata:
|
||
name: handle-cnn-access
|
||
namespace: istio-system
|
||
spec:
|
||
match: request.host.endsWith(&#34;.cnn.com&#34;) &amp;&amp; context.reporter.uid.startsWith(&#34;kubernetes://istio-egressgateway&#34;)
|
||
actions:
|
||
- handler: egress-access-logger.stdio
|
||
instances:
|
||
- egress-access.logentry
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Send three HTTP requests to <em>cnn.com</em>, to <a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a>, <a href="https://edition.cnn.com/sport">edition.cnn.com/sport</a> and <a href="https://edition.cnn.com/health">edition.cnn.com/health</a>.
|
||
All three should return <em>200 OK</em>.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &#39;curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/health&#39;
|
||
200
|
||
200
|
||
200
|
||
</code></pre></li>
|
||
<li><p>Query the Mixer log and see that the information about the requests appears in the log:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep egress-access | grep cnn | tail -4
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T07:43:24.611462Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/politics&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:200,&#34;responseSize&#34;:1883355,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/sleep&#34;}
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T07:43:24.886316Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/sport&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:200,&#34;responseSize&#34;:2094561,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/sleep&#34;}
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T07:43:25.369663Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/health&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:200,&#34;responseSize&#34;:2157009,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/sleep&#34;}
|
||
{&#34;level&#34;:&#34;error&#34;,&#34;time&#34;:&#34;2019-01-29T07:43:24.611462Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/politics&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:200,&#34;responseSize&#34;:1883355,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/sleep&#34;}
|
||
</code></pre>
|
||
<p>You see four log entries related to your three requests. Three <em>info</em> entries about the access to <em>edition.cnn.com</em>
|
||
and one <em>error</em> entry about the access to <em>edition.cnn.com/politics</em>. The service mesh operators can see all the
|
||
access instances, and can also search the log for <em>error</em> log entries that represent forbidden accesses. This is the
|
||
first security measure the organization can apply before blocking the forbidden accesses automatically, namely
|
||
logging all the forbidden access instances as errors. In some settings this can be a sufficient security measure.</p>
|
||
<p>Note the attributes:</p>
|
||
<ul>
|
||
<li><code>destination</code>, <code>path</code>, <code>responseCode</code>, <code>responseSize</code> are related to HTTP parameters of the requests</li>
|
||
<li><code>sourcePrincipal</code>:<code>cluster.local/ns/default/sa/sleep</code> - a string that represents the <code>sleep</code> service account in
|
||
the <code>default</code> namespace</li>
|
||
<li><code>reporterUID</code>: <code>kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system</code> - a UID of the reporting pod, in
|
||
this case <code>istio-egressgateway-747b6764b8-44rrh</code> in the <code>istio-system</code> namespace</li>
|
||
</ul></li>
|
||
</ol>
|
||
<h3 id="access-control-by-routing">Access control by routing</h3>
|
||
<p>After enabling logging of access to <em>edition.cnn.com</em>, automatically enforce an access policy, namely allow
|
||
accessing <em>/health</em> and <em>/sport</em> URL paths only. Such a simple policy control can be implemented with Istio routing.</p>
|
||
<ol>
|
||
<li><p>Redefine your <code>VirtualService</code> for <em>edition.cnn.com</em>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | kubectl apply -f -
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: direct-cnn-through-egress-gateway
|
||
spec:
|
||
hosts:
|
||
- edition.cnn.com
|
||
gateways:
|
||
- istio-egressgateway
|
||
- mesh
|
||
http:
|
||
- match:
|
||
- gateways:
|
||
- mesh
|
||
port: 80
|
||
route:
|
||
- destination:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
subset: cnn
|
||
port:
|
||
number: 443
|
||
weight: 100
|
||
- match:
|
||
- gateways:
|
||
- istio-egressgateway
|
||
port: 443
|
||
uri:
|
||
regex: &#34;/health|/sport&#34;
|
||
route:
|
||
- destination:
|
||
host: edition.cnn.com
|
||
port:
|
||
number: 443
|
||
weight: 100
|
||
EOF
|
||
</code></pre>
|
||
<p>Note that you added a <code>match</code> by <code>uri</code> condition that checks that the URL path is
|
||
either <em>/health</em> or <em>/sport</em>. Also note that this condition is added to the <code>istio-egressgateway</code>
|
||
section of the <code>VirtualService</code>, since the egress gateway is a hardened component in terms of security (see
|
||
<a href="/v1.1/docs/examples/advanced-gateways/egress-gateway/#additional-security-considerations">egress gateway security considerations</a>). You don&rsquo;t want any tampering
|
||
with your policies.</p></li>
|
||
<li><p>Send the previous three HTTP requests to <em>cnn.com</em>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &#39;curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/health&#39;
|
||
404
|
||
200
|
||
200
|
||
</code></pre>
|
||
<p>The request to <a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> returned <em>404 Not Found</em>, while requests
|
||
to <a href="https://edition.cnn.com/sport">edition.cnn.com/sport</a> and
|
||
<a href="https://edition.cnn.com/health">edition.cnn.com/health</a> returned <em>200 OK</em>, as expected.</p>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">You may need to wait several seconds for the update of the <code>VirtualService</code> to propagate to the egress
|
||
gateway.</div>
|
||
</aside>
|
||
</div>
|
||
</li>
|
||
<li><p>Query the Mixer log and see that the information about the requests appears again in the log:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep egress-access | grep cnn | tail -4
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T07:55:59.686082Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/politics&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:404,&#34;responseSize&#34;:0,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/sleep&#34;}
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T07:55:59.697565Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/sport&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:200,&#34;responseSize&#34;:2094561,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/sleep&#34;}
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T07:56:00.264498Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/health&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:200,&#34;responseSize&#34;:2157009,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/sleep&#34;}
|
||
{&#34;level&#34;:&#34;error&#34;,&#34;time&#34;:&#34;2019-01-29T07:55:59.686082Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/politics&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:404,&#34;responseSize&#34;:0,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/sleep&#34;}
|
||
</code></pre>
|
||
<p>You still get info and error messages regarding accesses to
|
||
<a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a>, however this time the <code>responseCode</code> is <code>404</code>, as
|
||
expected.</p></li>
|
||
</ol>
|
||
<p>While implementing access control using Istio routing worked for us in this simple case, it would not suffice for more
|
||
complex cases. For example, the organization may want to allow access to
|
||
<a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> under certain conditions, so more complex policy logic than
|
||
just filtering by URL paths will be required. You may want to apply <a href="/v1.1/blog/2017/adapter-model/">Istio Mixer Adapters</a>,
|
||
for example
|
||
<a href="/v1.1/docs/tasks/policy-enforcement/denial-and-list/#attribute-based-whitelists-or-blacklists">white lists or black lists</a>
|
||
of allowed/forbidden URL paths, respectively.
|
||
<a href="/v1.1/docs/reference/config/policy-and-telemetry/istio.policy.v1beta1/">Policy Rules</a> allow specifying complex conditions,
|
||
specified in a <a href="/v1.1/docs/reference/config/policy-and-telemetry/expression-language/">rich expression language</a>, which
|
||
includes AND and OR logical operators. The rules can be reused for both logging and policy checks. More advanced users
|
||
may want to apply <a href="/v1.1/docs/concepts/security/#authorization">Istio Role-Based Access Control</a>.</p>
|
||
<p>An additional aspect is integration with remote access policy systems. If the organization in our use case operates some
|
||
<a href="https://en.wikipedia.org/wiki/Identity_management">Identity and Access Management</a> system, you may want to configure
|
||
Istio to use access policy information from such a system. You implement this integration by applying
|
||
<a href="/v1.1/blog/2017/adapter-model/">Istio Mixer Adapters</a>.</p>
|
||
<p>Cancel the access control by routing you used in this section and implement access control by Mixer policy checks
|
||
in the next section.</p>
|
||
<ol>
|
||
<li><p>Replace the <code>VirtualService</code> for <em>edition.cnn.com</em> with your previous version from the <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway-tls-origination/#perform-tls-origination-with-an-egress-gateway">Configure an Egress Gateway</a> example:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | kubectl apply -f -
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: direct-cnn-through-egress-gateway
|
||
spec:
|
||
hosts:
|
||
- edition.cnn.com
|
||
gateways:
|
||
- istio-egressgateway
|
||
- mesh
|
||
http:
|
||
- match:
|
||
- gateways:
|
||
- mesh
|
||
port: 80
|
||
route:
|
||
- destination:
|
||
host: istio-egressgateway.istio-system.svc.cluster.local
|
||
subset: cnn
|
||
port:
|
||
number: 443
|
||
weight: 100
|
||
- match:
|
||
- gateways:
|
||
- istio-egressgateway
|
||
port: 443
|
||
route:
|
||
- destination:
|
||
host: edition.cnn.com
|
||
port:
|
||
number: 443
|
||
weight: 100
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Send the previous three HTTP requests to <em>cnn.com</em>, this time you should get three <em>200 OK</em> responses as
|
||
previously:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &#39;curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/health&#39;
|
||
200
|
||
200
|
||
200
|
||
</code></pre></li>
|
||
</ol>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">You may need to wait several seconds for the update of the <code>VirtualService</code> to propagate to the egress
|
||
gateway.</div>
|
||
</aside>
|
||
</div>
|
||
<h3 id="access-control-by-mixer-policy-checks">Access control by Mixer policy checks</h3>
|
||
<p>In this step you use a Mixer
|
||
<a href="/v1.1/docs/reference/config/policy-and-telemetry/adapters/list/"><code>Listchecker</code> adapter</a>, its whitelist
|
||
variety. You define a <code>listentry</code> with the URL path of the request and a <code>listchecker</code> to check the <code>listentry</code> using a
|
||
static list of allowed URL paths, specified by the <code>overrides</code> field. For an external <a href="https://en.wikipedia.org/wiki/Identity_management">Identity and Access Management</a> system, use the <code>providerurl</code> field instead. The updated
|
||
diagram of the instances, rules and handlers appears below. Note that you reuse the same policy rule, <code>handle-cnn-access</code>
|
||
both for logging and for access policy checks.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:52.79420593027812%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-monitoring-access-control/egress-adapters-monitoring-policy.svg" title="Instances, rules and handlers for egress monitoring and access policies">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-monitoring-access-control/egress-adapters-monitoring-policy.svg" alt="Instances, rules and handlers for egress monitoring and access policies" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Instances, rules and handlers for egress monitoring and access policies</figcaption>
|
||
</figure>
|
||
<ol>
|
||
<li><p>Define <code>path-checker</code> and <code>request-path</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | kubectl create -f -
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: listchecker
|
||
metadata:
|
||
name: path-checker
|
||
namespace: istio-system
|
||
spec:
|
||
overrides: [&#34;/health&#34;, &#34;/sport&#34;] # overrides provide a static list
|
||
blacklist: false
|
||
---
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: listentry
|
||
metadata:
|
||
name: request-path
|
||
namespace: istio-system
|
||
spec:
|
||
value: request.path
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Modify the <code>handle-cnn-access</code> policy rule to send <code>request-path</code> instances to the <code>path-checker</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | kubectl apply -f -
|
||
# Rule handle egress access to cnn.com
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: rule
|
||
metadata:
|
||
name: handle-cnn-access
|
||
namespace: istio-system
|
||
spec:
|
||
match: request.host.endsWith(&#34;.cnn.com&#34;) &amp;&amp; context.reporter.uid.startsWith(&#34;kubernetes://istio-egressgateway&#34;)
|
||
actions:
|
||
- handler: egress-access-logger.stdio
|
||
instances:
|
||
- egress-access.logentry
|
||
- handler: path-checker.listchecker
|
||
instances:
|
||
- request-path.listentry
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Perform your usual test by sending HTTP requests to
|
||
<a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a>, <a href="https://edition.cnn.com/sport">edition.cnn.com/sport</a>
|
||
and <a href="https://edition.cnn.com/health">edition.cnn.com/health</a>. As expected, the request to
|
||
<a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> returns <em>403</em> (Forbidden).</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &#39;curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/health&#39;
|
||
403
|
||
200
|
||
200
|
||
</code></pre></li>
|
||
</ol>
|
||
<h3 id="access-control-by-mixer-policy-checks-part-2">Access control by Mixer policy checks, part 2</h3>
|
||
<p>After the organization in our use case managed to configure logging and access control, it decided to extend its access
|
||
policy by allowing the applications with a special
|
||
<a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/">Service Account</a> to access any topic of <em>cnn.com</em>, without being monitored. You&rsquo;ll see how this requirement can be configured in Istio.</p>
|
||
<ol>
|
||
<li><p>Start the <a href="https://github.com/istio/istio/tree/release-1.1/samples/sleep">sleep</a> sample with the <code>politics</code> service account.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ sed &#39;s/: sleep/: politics/g&#39; samples/sleep/sleep.yaml | kubectl create -f -
|
||
serviceaccount &#34;politics&#34; created
|
||
service &#34;politics&#34; created
|
||
deployment &#34;politics&#34; created
|
||
</code></pre></li>
|
||
<li><p>Define the <code>SOURCE_POD_POLITICS</code> shell variable to hold the name of the source pod with the <code>politics</code> service
|
||
account, for sending requests to external services.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export SOURCE_POD_POLITICS=$(kubectl get pod -l app=politics -o jsonpath={.items..metadata.name})
|
||
</code></pre></li>
|
||
<li><p>Perform your usual test of sending three HTTP requests this time from <code>SOURCE_POD_POLITICS</code>.
|
||
The request to <a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> returns <em>403</em>, since you did not configure
|
||
the exception for the <em>politics</em> namespace.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl exec -it $SOURCE_POD_POLITICS -c politics -- sh -c &#39;curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/health&#39;
|
||
403
|
||
200
|
||
200
|
||
</code></pre></li>
|
||
<li><p>Query the Mixer log and see that the information about the requests from the <em>politics</em> namespace appears in
|
||
the log:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep egress-access | grep cnn | tail -4
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T08:04:42.559812Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/politics&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:403,&#34;responseSize&#34;:84,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/politics&#34;}
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T08:04:42.568424Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/sport&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:200,&#34;responseSize&#34;:2094561,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/politics&#34;}
|
||
{&#34;level&#34;:&#34;error&#34;,&#34;time&#34;:&#34;2019-01-29T08:04:42.559812Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/politics&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:403,&#34;responseSize&#34;:84,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/politics&#34;}
|
||
{&#34;level&#34;:&#34;info&#34;,&#34;time&#34;:&#34;2019-01-29T08:04:42.615641Z&#34;,&#34;instance&#34;:&#34;egress-access.logentry.istio-system&#34;,&#34;destination&#34;:&#34;edition.cnn.com&#34;,&#34;path&#34;:&#34;/health&#34;,&#34;reporterUID&#34;:&#34;kubernetes://istio-egressgateway-747b6764b8-44rrh.istio-system&#34;,&#34;responseCode&#34;:200,&#34;responseSize&#34;:2157009,&#34;sourcePrincipal&#34;:&#34;cluster.local/ns/default/sa/politics&#34;}
|
||
</code></pre>
|
||
<p>Note that <code>sourcePrincipal</code> is <code>cluster.local/ns/default/sa/politics</code> which represents the <code>politics</code> service
|
||
account in the <code>default</code> namespace.</p></li>
|
||
<li><p>Redefine <code>handle-cnn-access</code> and <code>handle-politics</code> policy rules, to make the applications in the <em>politics</em>
|
||
namespace exempt from monitoring and policy enforcement.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat &lt;&lt;EOF | kubectl apply -f -
|
||
# Rule to handle access to *.cnn.com/politics
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: rule
|
||
metadata:
|
||
name: handle-politics
|
||
namespace: istio-system
|
||
spec:
|
||
match: request.host.endsWith(&#34;cnn.com&#34;) &amp;&amp; context.reporter.uid.startsWith(&#34;kubernetes://istio-egressgateway&#34;) &amp;&amp; request.path.startsWith(&#34;/politics&#34;) &amp;&amp; source.principal != &#34;cluster.local/ns/default/sa/politics&#34;
|
||
actions:
|
||
- handler: egress-error-logger.stdio
|
||
instances:
|
||
- egress-access.logentry
|
||
---
|
||
# Rule handle egress access to cnn.com
|
||
apiVersion: &#34;config.istio.io/v1alpha2&#34;
|
||
kind: rule
|
||
metadata:
|
||
name: handle-cnn-access
|
||
namespace: istio-system
|
||
spec:
|
||
match: request.host.endsWith(&#34;.cnn.com&#34;) &amp;&amp; context.reporter.uid.startsWith(&#34;kubernetes://istio-egressgateway&#34;) &amp;&amp; source.principal != &#34;cluster.local/ns/default/sa/politics&#34;
|
||
actions:
|
||
- handler: egress-access-logger.stdio
|
||
instances:
|
||
- egress-access.logentry
|
||
- handler: path-checker.listchecker
|
||
instances:
|
||
- request-path.listentry
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Perform your usual test from <code>SOURCE_POD</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c &#39;curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/health&#39;
|
||
403
|
||
200
|
||
200
|
||
</code></pre>
|
||
<p>Since <code>SOURCE_POD</code> does not have <code>politics</code> service account, access to
|
||
<a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> is forbidden, as previously.</p></li>
|
||
<li><p>Perform the previous test from <code>SOURCE_POD_POLITICS</code>:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl exec -it $SOURCE_POD_POLITICS -c politics -- sh -c &#39;curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/politics; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/sport; curl -sL -o /dev/null -w &#34;%{http_code}\n&#34; http://edition.cnn.com/health&#39;
|
||
200
|
||
200
|
||
200
|
||
</code></pre>
|
||
<p>Access to all the topics of <em>edition.cnn.com</em> is allowed.</p></li>
|
||
<li><p>Examine the Mixer log and see that no more requests with <code>sourcePrincipal</code> equal
|
||
<code>cluster.local/ns/default/sa/politics</code> appear in the log.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep egress-access | grep cnn | tail -4
|
||
</code></pre></li>
|
||
</ol>
|
||
<h2 id="comparison-with-https-egress-traffic-control">Comparison with HTTPS egress traffic control</h2>
|
||
<p>In this use case the applications use HTTP and Istio Egress Gateway performs TLS origination for them. Alternatively,
|
||
the applications could originate TLS themselves by issuing HTTPS requests to <em>edition.cnn.com</em>. In this section we
|
||
describe both approaches and their pros and cons.</p>
|
||
<p>In the HTTP approach, the requests are sent unencrypted on the local host, intercepted by the Istio sidecar proxy and
|
||
forwarded to the egress gateway. Since you configure Istio to use mutual TLS between the sidecar proxy and the egress
|
||
gateway, the traffic leaves the pod encrypted. The egress gateway decrypts the traffic, inspects the URL path, the
|
||
HTTP method and headers, reports telemetry and performs policy checks. If the request is not blocked by some policy
|
||
check, the egress gateway performs TLS origination to the external destination (<em>cnn.com</em> in our case), so the request
|
||
is encrypted again and sent encrypted to the external destination. The diagram below demonstrates the network flow of
|
||
this approach. The HTTP protocol inside the gateway designates the protocol as seen by the gateway after decryption.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:64.81718469808756%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-monitoring-access-control/http-to-gateway.svg" title="HTTP egress traffic through an egress gateway">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-monitoring-access-control/http-to-gateway.svg" alt="HTTP egress traffic through an egress gateway" />
|
||
</a>
|
||
</div>
|
||
<figcaption>HTTP egress traffic through an egress gateway</figcaption>
|
||
</figure>
|
||
<p>The drawback of this approach is that the requests are sent unencrypted inside the pod, which may be against security
|
||
policies in some organizations. Also some SDKs have external service URLs hard-coded, including the protocol, so
|
||
sending HTTP requests could be impossible. The advantage of this approach is the ability to inspect HTTP methods,
|
||
headers and URL paths, and to apply policies based on them.</p>
|
||
<p>In the HTTPS approach, the requests are encrypted end-to-end, from the application to the external destination. The
|
||
diagram below demonstrates the network flow of this approach. The HTTPS protocol inside the gateway designates the
|
||
protocol as seen by the gateway.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:64.81718469808756%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-monitoring-access-control/https-to-gateway.svg" title="HTTPS egress traffic through an egress gateway">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-monitoring-access-control/https-to-gateway.svg" alt="HTTPS egress traffic through an egress gateway" />
|
||
</a>
|
||
</div>
|
||
<figcaption>HTTPS egress traffic through an egress gateway</figcaption>
|
||
</figure>
|
||
<p>The end-to-end HTTPS is considered a better approach from the security point of view. However, since the traffic is
|
||
encrypted the Istio proxies and the egress gateway can only see the source and destination IPs and the <a href="https://en.wikipedia.org/wiki/Server_Name_Indication">SNI</a> of the destination. Since you configure Istio to use mutual TLS between the sidecar proxy
|
||
and the egress gateway, the <a href="/v1.1/docs/concepts/security/#istio-identity">identity of the source</a> is also known.
|
||
The gateway is unable to inspect the URL path, the HTTP method and the headers of the requests, so no monitoring and
|
||
policies based on the HTTP information can be possible.
|
||
In our use case, the organization would be able to allow access to <em>edition.cnn.com</em> and to specify which applications
|
||
are allowed to access <em>edition.cnn.com</em>.
|
||
However, it will not be possible to allow or block access to specific URL paths of <em>edition.cnn.com</em>.
|
||
Neither blocking access to <a href="https://edition.cnn.com/politics">edition.cnn.com/politics</a> nor monitoring such access are
|
||
possible with the HTTPS approach.</p>
|
||
<p>We guess that each organization will consider the pros and cons of the two approaches and choose the one most
|
||
appropriate to its needs.</p>
|
||
<h2 id="summary">Summary</h2>
|
||
<p>In this blog post we showed how different monitoring and policy mechanisms of Istio can be applied to HTTP egress
|
||
traffic. Monitoring can be implemented by configuring a logging adapter. Access
|
||
policies can be implemented by configuring <code>VirtualServices</code> or by configuring various policy check adapters. We
|
||
demonstrated a simple policy that allowed certain URL paths only. We also showed a more complex policy that extended the
|
||
simple policy by making an exemption to the applications with a certain service account. Finally, we compared
|
||
HTTP-with-TLS-origination egress traffic with HTTPS egress traffic, in terms of control possibilities by Istio.</p>
|
||
<h2 id="cleanup">Cleanup</h2>
|
||
<ol>
|
||
<li><p>Perform the instructions in <a href="/v1.1/docs/examples/advanced-gateways/egress-gateway//#cleanup">Cleanup</a> section of the
|
||
<a href="/v1.1/docs/examples/advanced-gateways/egress-gateway//">Configure an Egress Gateway</a> example.</p></li>
|
||
<li><p>Delete the logging and policy checks configuration:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete logentry egress-access -n istio-system
|
||
$ kubectl delete stdio egress-error-logger -n istio-system
|
||
$ kubectl delete stdio egress-access-logger -n istio-system
|
||
$ kubectl delete rule handle-politics -n istio-system
|
||
$ kubectl delete rule handle-cnn-access -n istio-system
|
||
$ kubectl delete -n istio-system listchecker path-checker
|
||
$ kubectl delete -n istio-system listentry request-path
|
||
</code></pre></li>
|
||
<li><p>Delete the <em>politics</em> source pod:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ sed &#39;s/: sleep/: politics/g&#39; samples/sleep/sleep.yaml | kubectl delete -f -
|
||
serviceaccount &#34;politics&#34; deleted
|
||
service &#34;politics&#34; deleted
|
||
deployment &#34;politics&#34; deleted
|
||
</code></pre></li>
|
||
</ol></description><pubDate>Fri, 22 Jun 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/egress-monitoring-access-control/</link><author>Vadim Eisenberg and Ronen Schaffer (IBM)</author><guid isPermaLink="true">/v1.1/blog/2018/egress-monitoring-access-control/</guid><category>egress</category><category>traffic-management</category><category>access-control</category><category>monitoring</category></item><item><title>Introducing the Istio v1alpha3 routing API</title><description>
|
||
<p>Up until now, Istio has provided a simple API for traffic management using four configuration resources:
|
||
<code>RouteRule</code>, <code>DestinationPolicy</code>, <code>EgressRule</code>, and (Kubernetes) <code>Ingress</code>.
|
||
With this API, users have been able to easily manage the flow of traffic in an Istio service mesh.
|
||
The API has allowed users to route requests to specific versions of services, inject delays and failures for resilience
|
||
testing, add timeouts and circuit breakers, and more, all without changing the application code itself.</p>
|
||
<p>While this functionality has proven to be a very compelling part of Istio, user feedback has also shown that this API does
|
||
have some shortcomings, specifically when using it to manage very large applications containing thousands of services, and
|
||
when working with protocols other than HTTP. Furthermore, the use of Kubernetes <code>Ingress</code> resources to configure external
|
||
traffic has proven to be woefully insufficient for our needs.</p>
|
||
<p>To address these, and other concerns, a new traffic management API, a.k.a. <code>v1alpha3</code>, is being introduced, which will
|
||
completely replace the previous API going forward. Although the <code>v1alpha3</code> model is fundamentally the same, it is not
|
||
backward compatible and will require manual conversion from the old API.</p>
|
||
<p>To justify this disruption, the <code>v1alpha3</code> API has gone through a long and painstaking community
|
||
review process that has hopefully resulted in a greatly improved API that will stand the test of time. In this article,
|
||
we will introduce the new configuration model and attempt to explain some of the motivation and design principles that
|
||
influenced it.</p>
|
||
<h2 id="design-principles">Design principles</h2>
|
||
<p>A few key design principles played a role in the routing model redesign:</p>
|
||
<ul>
|
||
<li>Explicitly model infrastructure as well as intent. For example, in addition to configuring an ingress gateway, the
|
||
component (controller) implementing it can also be specified.</li>
|
||
<li>The authoring model should be &ldquo;producer oriented&rdquo; and &ldquo;host centric&rdquo; as opposed to compositional. For example, all
|
||
rules associated with a particular host are configured together, instead of individually.</li>
|
||
<li>Clear separation of routing from post-routing behaviors.</li>
|
||
</ul>
|
||
<h2 id="configuration-resources-in-v1alpha3">Configuration resources in v1alpha3</h2>
|
||
<p>A typical mesh will have one or more load balancers (we call them gateways)
|
||
that terminate TLS from external networks and allow traffic into the mesh.
|
||
Traffic then flows through internal services via sidecar gateways.
|
||
It is also common for applications to consume external
|
||
services (e.g., Google Maps API). These may be called directly or, in certain deployments, all traffic
|
||
exiting the mesh may be forced through dedicated egress gateways. The following diagram depicts
|
||
this mental model.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:35.204472660409245%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/v1alpha3-routing/./gateways.svg" title="Gateways in an Istio service mesh">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/v1alpha3-routing/./gateways.svg" alt="Role of gateways in the mesh" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Gateways in an Istio service mesh</figcaption>
|
||
</figure>
|
||
<p>With the above setup in mind, <code>v1alpha3</code> introduces the following new
|
||
configuration resources to control traffic routing into, within, and out of the mesh.</p>
|
||
<ol>
|
||
<li><code>Gateway</code></li>
|
||
<li><code>VirtualService</code></li>
|
||
<li><code>DestinationRule</code></li>
|
||
<li><code>ServiceEntry</code></li>
|
||
</ol>
|
||
<p><code>VirtualService</code>, <code>DestinationRule</code>, and <code>ServiceEntry</code> replace <code>RouteRule</code>,
|
||
<code>DestinationPolicy</code>, and <code>EgressRule</code> respectively. The <code>Gateway</code> is a
|
||
platform independent abstraction to model the traffic flowing into
|
||
dedicated middleboxes.</p>
|
||
<p>The figure below depicts the flow of control across configuration
|
||
resources.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:41.164966727369595%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/v1alpha3-routing/./virtualservices-destrules.svg" title="Relationship between different v1alpha3 elements">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/v1alpha3-routing/./virtualservices-destrules.svg" alt="Relationship between different v1alpha3 elements" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Relationship between different v1alpha3 elements</figcaption>
|
||
</figure>
|
||
<h3 id="gateway"><code>Gateway</code></h3>
|
||
<p>A <a href="/v1.1/docs/reference/config/networking/v1alpha3/gateway/"><code>Gateway</code></a>
|
||
configures a load balancer for HTTP/TCP traffic, regardless of
|
||
where it will be running. Any number of gateways can exist within the mesh
|
||
and multiple different gateway implementations can co-exist. In fact, a
|
||
gateway configuration can be bound to a particular workload by specifying
|
||
the set of workload (pod) labels as part of the configuration, allowing
|
||
users to reuse off the shelf network appliances by writing a simple gateway
|
||
controller.</p>
|
||
<p>For ingress traffic management, you might ask: <em>Why not reuse Kubernetes Ingress APIs</em>?
|
||
The Ingress APIs proved to be incapable of expressing Istio&rsquo;s routing needs.
|
||
By trying to draw a common denominator across different HTTP proxies, the
|
||
Ingress is only able to support the most basic HTTP routing and ends up
|
||
pushing every other feature of modern proxies into non-portable
|
||
annotations.</p>
|
||
<p>Istio <code>Gateway</code> overcomes the <code>Ingress</code> shortcomings by separating the
|
||
L4-L6 spec from L7. It only configures the L4-L6 functions (e.g., ports to
|
||
expose, TLS configuration) that are uniformly implemented by all good L7
|
||
proxies. Users can then use standard Istio rules to control HTTP
|
||
requests as well as TCP traffic entering a <code>Gateway</code> by binding a
|
||
<code>VirtualService</code> to it.</p>
|
||
<p>For example, the following simple <code>Gateway</code> configures a load balancer
|
||
to allow external https traffic for host <code>bookinfo.com</code> into the mesh:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: Gateway
|
||
metadata:
|
||
name: bookinfo-gateway
|
||
spec:
|
||
servers:
|
||
- port:
|
||
number: 443
|
||
name: https
|
||
protocol: HTTPS
|
||
hosts:
|
||
- bookinfo.com
|
||
tls:
|
||
mode: SIMPLE
|
||
serverCertificate: /tmp/tls.crt
|
||
privateKey: /tmp/tls.key
|
||
</code></pre>
|
||
<p>To configure the corresponding routes, a <code>VirtualService</code> (described in the <a href="#virtualservice">following section</a>)
|
||
must be defined for the same host and bound to the <code>Gateway</code> using
|
||
the <code>gateways</code> field in the configuration:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: bookinfo
|
||
spec:
|
||
hosts:
|
||
- bookinfo.com
|
||
gateways:
|
||
- bookinfo-gateway # &lt;---- bind to gateway
|
||
http:
|
||
- match:
|
||
- uri:
|
||
prefix: /reviews
|
||
route:
|
||
...
|
||
</code></pre>
|
||
<p>The <code>Gateway</code> can be used to model an edge-proxy or a purely internal proxy
|
||
as shown in the first figure. Irrespective of the location, all gateways
|
||
can be configured and controlled in the same way.</p>
|
||
<h3 id="virtualservice"><code>VirtualService</code></h3>
|
||
<p>Replacing route rules with something called &ldquo;virtual services” might seem peculiar at first, but in reality it’s
|
||
fundamentally a much better name for what is being configured, especially after redesigning the API to address the
|
||
scalability issues with the previous model.</p>
|
||
<p>In effect, what has changed is that instead of configuring routing using a set of individual configuration resources
|
||
(rules) for a particular destination service, each containing a precedence field to control the order of evaluation, we
|
||
now configure the (virtual) destination itself, with all of its rules in an ordered list within a corresponding
|
||
<a href="/v1.1/docs/reference/config/networking/v1alpha3/virtual-service/"><code>VirtualService</code></a> resource.
|
||
For example, where previously we had two <code>RouteRule</code> resources for the
|
||
<a href="/v1.1/docs/examples/bookinfo/">Bookinfo</a> application’s <code>reviews</code> service, like this:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: config.istio.io/v1alpha2
|
||
kind: RouteRule
|
||
metadata:
|
||
name: reviews-default
|
||
spec:
|
||
destination:
|
||
name: reviews
|
||
precedence: 1
|
||
route:
|
||
- labels:
|
||
version: v1
|
||
---
|
||
apiVersion: config.istio.io/v1alpha2
|
||
kind: RouteRule
|
||
metadata:
|
||
name: reviews-test-v2
|
||
spec:
|
||
destination:
|
||
name: reviews
|
||
precedence: 2
|
||
match:
|
||
request:
|
||
headers:
|
||
cookie:
|
||
regex: &#34;^(.*?;)?(user=jason)(;.*)?$&#34;
|
||
route:
|
||
- labels:
|
||
version: v2
|
||
</code></pre>
|
||
<p>In <code>v1alpha3</code>, we provide the same configuration in a single <code>VirtualService</code> resource:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: reviews
|
||
spec:
|
||
hosts:
|
||
- reviews
|
||
http:
|
||
- match:
|
||
- headers:
|
||
cookie:
|
||
regex: &#34;^(.*?;)?(user=jason)(;.*)?$&#34;
|
||
route:
|
||
- destination:
|
||
host: reviews
|
||
subset: v2
|
||
- route:
|
||
- destination:
|
||
host: reviews
|
||
subset: v1
|
||
</code></pre>
|
||
<p>As you can see, both of the rules for the <code>reviews</code> service are consolidated in one place, which at first may or may not
|
||
seem preferable. However, if you look closer at this new model, you’ll see there are fundamental differences that make
|
||
<code>v1alpha3</code> vastly more functional.</p>
|
||
<p>First of all, notice that the destination service for the <code>VirtualService</code> is specified using a <code>hosts</code> field (repeated field, in fact) and is then again specified in a <code>destination</code> field of each of the route specifications. This is a
|
||
very important difference from the previous model.</p>
|
||
<p>A <code>VirtualService</code> describes the mapping between one or more user-addressable destinations to the actual destination workloads inside the mesh. In our example, they are the same, however, the user-addressed hosts can be any DNS
|
||
names with optional wildcard prefix or CIDR prefix that will be used to address the service. This can be particularly
|
||
useful in facilitating turning monoliths into a composite service built out of distinct microservices without requiring the
|
||
consumers of the service to adapt to the transition.</p>
|
||
<p>For example, the following rule allows users to address both the <code>reviews</code> and <code>ratings</code> services of the Bookinfo application
|
||
as if they are parts of a bigger (virtual) service at <code>http://bookinfo.com/</code>:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: bookinfo
|
||
spec:
|
||
hosts:
|
||
- bookinfo.com
|
||
http:
|
||
- match:
|
||
- uri:
|
||
prefix: /reviews
|
||
route:
|
||
- destination:
|
||
host: reviews
|
||
- match:
|
||
- uri:
|
||
prefix: /ratings
|
||
route:
|
||
- destination:
|
||
host: ratings
|
||
...
|
||
</code></pre>
|
||
<p>The hosts of a <code>VirtualService</code> do not actually have to be part of the service registry, they are simply virtual
|
||
destinations. This allows users to model traffic for virtual hosts that do not have routable entries inside the mesh.
|
||
These hosts can be exposed outside the mesh by binding the <code>VirtualService</code> to a <code>Gateway</code> configuration for the same host
|
||
(as described in the <a href="#gateway">previous section</a>).</p>
|
||
<p>In addition to this fundamental restructuring, <code>VirtualService</code> includes several other important changes:</p>
|
||
<ol>
|
||
<li><p>Multiple match conditions can be expressed inside the <code>VirtualService</code> configuration, reducing the need for redundant
|
||
rules.</p></li>
|
||
<li><p>Each service version has a name (called a service subset). The set of pods/VMs belonging to a subset is defined in a
|
||
<code>DestinationRule</code>, described in the following section.</p></li>
|
||
<li><p><code>VirtualService</code> hosts can be specified using wildcard DNS prefixes to create a single rule for all matching services.
|
||
For example, in Kubernetes, to apply the same rewrite rule for all services in the <code>foo</code> namespace, the <code>VirtualService</code>
|
||
would use <code>*.foo.svc.cluster.local</code> as the host.</p></li>
|
||
</ol>
|
||
<h3 id="destinationrule"><code>DestinationRule</code></h3>
|
||
<p>A <a href="/v1.1/docs/reference/config/networking/v1alpha3/destination-rule/"><code>DestinationRule</code></a>
|
||
configures the set of policies to be applied while forwarding traffic to a service. They are
|
||
intended to be authored by service owners, describing the circuit breakers, load balancer settings, TLS settings, etc..
|
||
<code>DestinationRule</code> is more or less the same as its predecessor, <code>DestinationPolicy</code>, with the following exceptions:</p>
|
||
<ol>
|
||
<li>The <code>host</code> of a <code>DestinationRule</code> can include wildcard prefixes, allowing a single rule to be specified for many actual
|
||
services.</li>
|
||
<li>A <code>DestinationRule</code> defines addressable <code>subsets</code> (i.e., named versions) of the corresponding destination host. These
|
||
subsets are used in <code>VirtualService</code> route specifications when sending traffic to specific versions of the service.
|
||
Naming versions this way allows us to cleanly refer to them across different virtual services, simplify the stats that
|
||
Istio proxies emit, and to encode subsets in SNI headers.</li>
|
||
</ol>
|
||
<p>A <code>DestinationRule</code> that configures policies and subsets for the reviews service might look something like this:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: reviews
|
||
spec:
|
||
host: reviews
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: RANDOM
|
||
subsets:
|
||
- name: v1
|
||
labels:
|
||
version: v1
|
||
- name: v2
|
||
labels:
|
||
version: v2
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: ROUND_ROBIN
|
||
- name: v3
|
||
labels:
|
||
version: v3
|
||
</code></pre>
|
||
<p>Notice that, unlike <code>DestinationPolicy</code>, multiple policies (e.g., default and v2-specific) are specified in a single
|
||
<code>DestinationRule</code> configuration.</p>
|
||
<h3 id="serviceentry"><code>ServiceEntry</code></h3>
|
||
<p><a href="/v1.1/docs/reference/config/networking/v1alpha3/service-entry/"><code>ServiceEntry</code></a>
|
||
is used to add additional entries into the service registry that Istio maintains internally.
|
||
It is most commonly used to allow one to model traffic to external dependencies of the mesh
|
||
such as APIs consumed from the web or traffic to services in legacy infrastructure.</p>
|
||
<p>Everything you could previously configure using an <code>EgressRule</code> can just as easily be done with a <code>ServiceEntry</code>.
|
||
For example, access to a simple external service from inside the mesh can be enabled using a configuration
|
||
something like this:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: foo-ext
|
||
spec:
|
||
hosts:
|
||
- foo.com
|
||
ports:
|
||
- number: 80
|
||
name: http
|
||
protocol: HTTP
|
||
</code></pre>
|
||
<p>That said, <code>ServiceEntry</code> has significantly more functionality than its predecessor.
|
||
First of all, a <code>ServiceEntry</code> is not limited to external service configuration,
|
||
it can be of two types: mesh-internal or mesh-external.
|
||
Mesh-internal entries are like all other internal services but are used to explicitly add services
|
||
to the mesh. They can be used to add services as part of expanding the service mesh to include unmanaged infrastructure
|
||
(e.g., VMs added to a Kubernetes-based service mesh).
|
||
Mesh-external entries represent services external to the mesh.
|
||
For them, mutual TLS authentication is disabled and policy enforcement is performed on the client-side,
|
||
instead of on the usual server-side for internal service requests.</p>
|
||
<p>Because a <code>ServiceEntry</code> configuration simply adds a destination to the internal service registry, it can be
|
||
used in conjunction with a <code>VirtualService</code> and/or <code>DestinationRule</code>, just like any other service in the registry.
|
||
The following <code>DestinationRule</code>, for example, can be used to initiate mutual TLS connections for an external service:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: foo-ext
|
||
spec:
|
||
host: foo.com
|
||
trafficPolicy:
|
||
tls:
|
||
mode: MUTUAL
|
||
clientCertificate: /etc/certs/myclientcert.pem
|
||
privateKey: /etc/certs/client_private_key.pem
|
||
caCertificates: /etc/certs/rootcacerts.pem
|
||
</code></pre>
|
||
<p>In addition to its expanded generality, <code>ServiceEntry</code> provides several other improvements over <code>EgressRule</code>
|
||
including the following:</p>
|
||
<ol>
|
||
<li>A single <code>ServiceEntry</code> can configure multiple service endpoints, which previously would have required multiple
|
||
<code>EgressRules</code>.</li>
|
||
<li>The resolution mode for the endpoints is now configurable (<code>NONE</code>, <code>STATIC</code>, or <code>DNS</code>).</li>
|
||
<li>Additionally, we are working on addressing another pain point: the need to access secure external services over plain
|
||
text ports (e.g., <code>http://google.com:443</code>). This should be fixed in the coming weeks, allowing you to directly access
|
||
<code>https://google.com</code> from your application. Stay tuned for an Istio patch release (0.8.x) that addresses this limitation.</li>
|
||
</ol>
|
||
<h2 id="creating-and-deleting-v1alpha3-route-rules">Creating and deleting v1alpha3 route rules</h2>
|
||
<p>Because all route rules for a given destination are now stored together as an ordered
|
||
list in a single <code>VirtualService</code> resource, adding a second and subsequent rules for a particular destination
|
||
is no longer done by creating a new (<code>RouteRule</code>) resource, but instead by updating the one-and-only <code>VirtualService</code>
|
||
resource for the destination.</p>
|
||
<p>old routing rules:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f my-second-rule-for-destination-abc.yaml
|
||
</code></pre>
|
||
<p><code>v1alpha3</code> routing rules:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f my-updated-rules-for-destination-abc.yaml
|
||
</code></pre>
|
||
<p>Deleting route rules other than the last one for a particular destination is also done by updating
|
||
the existing resource using <code>kubectl apply</code>.</p>
|
||
<p>When adding or removing routes that refer to service versions, the <code>subsets</code> will need to be updated in
|
||
the service&rsquo;s corresponding <code>DestinationRule</code>.
|
||
As you might have guessed, this is also done using <code>kubectl apply</code>.</p>
|
||
<h2 id="summary">Summary</h2>
|
||
<p>The Istio <code>v1alpha3</code> routing API has significantly more functionality than
|
||
its predecessor, but unfortunately is not backwards compatible, requiring a
|
||
one time manual conversion. The previous configuration resources,
|
||
<code>RouteRule</code>, <code>DesintationPolicy</code>, and <code>EgressRule</code>, will not be supported
|
||
from Istio 0.9 onwards. Kubernetes users can continue to use <code>Ingress</code> to
|
||
configure their edge load balancers for basic routing. However, advanced
|
||
routing features (e.g., traffic split across two versions) will require use
|
||
of <code>Gateway</code>, a significantly more functional and highly
|
||
recommended <code>Ingress</code> replacement.</p>
|
||
<h2 id="acknowledgments">Acknowledgments</h2>
|
||
<p>Credit for the routing model redesign and implementation work goes to the
|
||
following people (in alphabetical order):</p>
|
||
<ul>
|
||
<li>Frank Budinsky (IBM)</li>
|
||
<li>Zack Butcher (Google)</li>
|
||
<li>Greg Hanson (IBM)</li>
|
||
<li>Costin Manolache (Google)</li>
|
||
<li>Martin Ostrowski (Google)</li>
|
||
<li>Shriram Rajagopalan (VMware)</li>
|
||
<li>Louis Ryan (Google)</li>
|
||
<li>Isaiah Snell-Feikema (IBM)</li>
|
||
<li>Kuat Yessenov (Google)</li>
|
||
</ul></description><pubDate>Wed, 25 Apr 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/v1alpha3-routing/</link><author>Frank Budinsky (IBM) and Shriram Rajagopalan (VMware)</author><guid isPermaLink="true">/v1.1/blog/2018/v1alpha3-routing/</guid><category>traffic-management</category></item><item><title>Configuring Istio Ingress with AWS NLB</title><description>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">This post was updated on January 16, 2019 to include some usage warnings.</div>
|
||
</aside>
|
||
</div>
|
||
<p>This post provides instructions to use and configure ingress Istio with <a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html">AWS Network Load Balancer</a>.</p>
|
||
<p>Network load balancer (NLB) could be used instead of classical load balancer. You can see the <a href="https://aws.amazon.com/elasticloadbalancing/details/#Product_comparisons">comparison</a> between different AWS <code>loadbalancer</code> for more explanation.</p>
|
||
<h2 id="prerequisites">Prerequisites</h2>
|
||
<p>The following instructions require a Kubernetes <strong>1.9.0 or newer</strong> cluster.</p>
|
||
<div>
|
||
<aside class="callout warning">
|
||
<div class="type">
|
||
<svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-warning"/></svg>
|
||
</div>
|
||
<div class="content"><p>Usage of AWS <code>nlb</code> on Kubernetes is an Alpha feature and not recommended for production clusters.</p>
|
||
<p>Usage of AWS <code>nlb</code> does not support the creation of two or more Kubernetes clusters running Istio in the same zone as a result of <a href="https://github.com/kubernetes/kubernetes/issues/69264">Kubernetes Bug #69264</a>.</p>
|
||
</div>
|
||
</aside>
|
||
</div>
|
||
<h2 id="iam-policy">IAM Policy</h2>
|
||
<p>You need to apply policy on the master role in order to be able to provision network load balancer.</p>
|
||
<ol>
|
||
<li><p>In AWS <code>iam</code> console click on policies and click on create a new one:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:52.430278884462155%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/aws-nlb/./createpolicystart.png" title="Create a new policy">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/aws-nlb/./createpolicystart.png" alt="Create a new policy" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Create a new policy</figcaption>
|
||
</figure>
|
||
</li>
|
||
<li><p>Select <code>json</code>:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:50.63492063492063%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/aws-nlb/./createpolicyjson.png" title="Select json">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/aws-nlb/./createpolicyjson.png" alt="Select json" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Select json</figcaption>
|
||
</figure>
|
||
</li>
|
||
<li><p>Copy/paste text below:</p>
|
||
<pre><code class='language-json' data-expandlinks='true' >{
|
||
&#34;Version&#34;: &#34;2012-10-17&#34;,
|
||
&#34;Statement&#34;: [
|
||
{
|
||
&#34;Sid&#34;: &#34;kopsK8sNLBMasterPermsRestrictive&#34;,
|
||
&#34;Effect&#34;: &#34;Allow&#34;,
|
||
&#34;Action&#34;: [
|
||
&#34;ec2:DescribeVpcs&#34;,
|
||
&#34;elasticloadbalancing:AddTags&#34;,
|
||
&#34;elasticloadbalancing:CreateListener&#34;,
|
||
&#34;elasticloadbalancing:CreateTargetGroup&#34;,
|
||
&#34;elasticloadbalancing:DeleteListener&#34;,
|
||
&#34;elasticloadbalancing:DeleteTargetGroup&#34;,
|
||
&#34;elasticloadbalancing:DescribeListeners&#34;,
|
||
&#34;elasticloadbalancing:DescribeLoadBalancerPolicies&#34;,
|
||
&#34;elasticloadbalancing:DescribeTargetGroups&#34;,
|
||
&#34;elasticloadbalancing:DescribeTargetHealth&#34;,
|
||
&#34;elasticloadbalancing:ModifyListener&#34;,
|
||
&#34;elasticloadbalancing:ModifyTargetGroup&#34;,
|
||
&#34;elasticloadbalancing:RegisterTargets&#34;,
|
||
&#34;elasticloadbalancing:SetLoadBalancerPoliciesOfListener&#34;
|
||
],
|
||
&#34;Resource&#34;: [
|
||
&#34;*&#34;
|
||
]
|
||
},
|
||
{
|
||
&#34;Effect&#34;: &#34;Allow&#34;,
|
||
&#34;Action&#34;: [
|
||
&#34;ec2:DescribeVpcs&#34;,
|
||
&#34;ec2:DescribeRegions&#34;
|
||
],
|
||
&#34;Resource&#34;: &#34;*&#34;
|
||
}
|
||
]
|
||
}
|
||
</code></pre></li>
|
||
<li><p>Click review policy, fill all fields and click create policy:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:60.08097165991902%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/aws-nlb/./create_policy.png" title="Validate policy">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/aws-nlb/./create_policy.png" alt="Validate policy" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Validate policy</figcaption>
|
||
</figure>
|
||
</li>
|
||
<li><p>Click on roles, select you master role nodes, and click attach policy:</p>
|
||
<figure style="width:100%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:30.328324986087924%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/aws-nlb/./roles_summary.png" title="Attach policy">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/aws-nlb/./roles_summary.png" alt="Attach policy" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Attach policy</figcaption>
|
||
</figure>
|
||
</li>
|
||
<li><p>Your policy is now attach to your master node.</p></li>
|
||
</ol>
|
||
<h2 id="generate-the-istio-manifest">Generate the Istio manifest</h2>
|
||
<p>To use an AWS <code>nlb</code> load balancer, it is necessary to add an AWS specific
|
||
annotation to the Istio installation. These instructions explain how to
|
||
add the annotation.</p>
|
||
<p>Save this as the file <code>override.yaml</code>:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >gateways:
|
||
istio-ingressgateway:
|
||
serviceAnnotations:
|
||
service.beta.kubernetes.io/aws-load-balancer-type: &#34;nlb&#34;
|
||
</code></pre>
|
||
<p>Generate a manifest with Helm:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ helm template install/kubernetes/helm/istio --namespace istio -f override.yaml &gt; $HOME/istio.yaml
|
||
</code></pre></description><pubDate>Fri, 20 Apr 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/aws-nlb/</link><author>Julien SENON</author><guid isPermaLink="true">/v1.1/blog/2018/aws-nlb/</guid><category>ingress</category><category>traffic-management</category><category>aws</category></item><item><title>Istio Soft Multi-Tenancy Support</title><description>
|
||
<p>Multi-tenancy is commonly used in many environments across many different applications,
|
||
but the implementation details and functionality provided on a per tenant basis does not
|
||
follow one model in all environments. The <a href="https://github.com/kubernetes/community/blob/master/wg-multitenancy/README.md">Kubernetes multi-tenancy working group</a>
|
||
is working to define the multi-tenant use cases and functionality that should be available
|
||
within Kubernetes. However, from their work so far it is clear that only &ldquo;soft multi-tenancy&rdquo;
|
||
is possible due to the inability to fully protect against malicious containers or workloads
|
||
gaining access to other tenant&rsquo;s pods or kernel resources.</p>
|
||
<h2 id="soft-multi-tenancy">Soft multi-tenancy</h2>
|
||
<p>For this blog, &ldquo;soft multi-tenancy&rdquo; is defined as having a single Kubernetes control plane
|
||
with multiple Istio control planes and multiple meshes, one control plane and one mesh
|
||
per tenant. The cluster administrator gets control and visibility across all the Istio
|
||
control planes, while the tenant administrator only gets control of a specific Istio
|
||
instance. Separation between the tenants is provided by Kubernetes namespaces and RBAC.</p>
|
||
<p>One use case for this deployment model is a shared corporate infrastructure where malicious
|
||
actions are not expected, but a clean separation of the tenants is still required.</p>
|
||
<p>Potential future Istio multi-tenant deployment models are described at the bottom of this
|
||
blog.</p>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">This blog is a high-level description of how to deploy Istio in a
|
||
limited multi-tenancy environment. The <a href="/v1.1/docs/">docs</a> section will be updated
|
||
when official multi-tenancy support is provided.</div>
|
||
</aside>
|
||
</div>
|
||
<h2 id="deployment">Deployment</h2>
|
||
<h3 id="multiple-istio-control-planes">Multiple Istio control planes</h3>
|
||
<p>Deploying multiple Istio control planes starts by replacing all <code>namespace</code> references
|
||
in a manifest file with the desired namespace. Using <code>istio.yaml</code> as an example, if two tenant
|
||
level Istio control planes are required; the first can use the <code>istio.yaml</code> default name of
|
||
<code>istio-system</code> and a second control plane can be created by generating a new yaml file with
|
||
a different namespace. As an example, the following command creates a yaml file with
|
||
the Istio namespace of <code>istio-system1</code>.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ cat istio.yaml | sed s/istio-system/istio-system1/g &gt; istio-system1.yaml
|
||
</code></pre>
|
||
<p>The <code>istio.yaml</code> file contains the details of the Istio control plane deployment, including the
|
||
pods that make up the control plane (Mixer, Pilot, Ingress, Galley, CA). Deploying the two Istio
|
||
control plane yaml files:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f install/kubernetes/istio.yaml
|
||
$ kubectl apply -f install/kubernetes/istio-system1.yaml
|
||
</code></pre>
|
||
<p>Results in two Istio control planes running in two namespaces.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods --all-namespaces
|
||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||
istio-system istio-ca-ffbb75c6f-98w6x 1/1 Running 0 15d
|
||
istio-system istio-ingress-68d65fc5c6-dnvfl 1/1 Running 0 15d
|
||
istio-system istio-mixer-5b9f8dffb5-8875r 3/3 Running 0 15d
|
||
istio-system istio-pilot-678fc976c8-b8tv6 2/2 Running 0 15d
|
||
istio-system1 istio-ca-5f496fdbcd-lqhlk 1/1 Running 0 15d
|
||
istio-system1 istio-ingress-68d65fc5c6-2vldg 1/1 Running 0 15d
|
||
istio-system1 istio-mixer-7d4f7b9968-66z44 3/3 Running 0 15d
|
||
istio-system1 istio-pilot-5bb6b7669c-779vb 2/2 Running 0 15d
|
||
</code></pre>
|
||
<p>The Istio <a href="/v1.1/docs/setup/kubernetes/additional-setup/sidecar-injection/">sidecar</a>
|
||
and <a href="/v1.1/docs/tasks/telemetry/">addons</a>, if required, manifests must also be
|
||
deployed to match the configured <code>namespace</code> in use by the tenant&rsquo;s Istio
|
||
control plane.</p>
|
||
<p>The execution of these two yaml files is the responsibility of the cluster
|
||
administrator, not the tenant level administrator. Additional RBAC restrictions will also
|
||
need to be configured and applied by the cluster administrator, limiting the tenant
|
||
administrator to only the assigned namespace.</p>
|
||
<h3 id="split-common-and-namespace-specific-resources">Split common and namespace specific resources</h3>
|
||
<p>The manifest files in the Istio repositories create both common resources that would
|
||
be used by all Istio control planes as well as resources that are replicated per control
|
||
plane. Although it is a simple matter to deploy multiple control planes by replacing the
|
||
<code>istio-system</code> namespace references as described above, a better approach is to split the
|
||
manifests into a common part that is deployed once for all tenants and a tenant
|
||
specific part. For the <a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions">Custom Resource Definitions</a>, the roles and the role
|
||
bindings should be separated out from the provided Istio manifests. Additionally, the
|
||
roles and role bindings in the provided Istio manifests are probably unsuitable for a
|
||
multi-tenant environment and should be modified or augmented as described in the next
|
||
section.</p>
|
||
<h3 id="kubernetes-rbac-for-istio-control-plane-resources">Kubernetes RBAC for Istio control plane resources</h3>
|
||
<p>To restrict a tenant administrator to a single Istio namespace, the cluster
|
||
administrator would create a manifest containing, at a minimum, a <code>Role</code> and <code>RoleBinding</code>
|
||
similar to the one below. In this example, a tenant administrator named <em>sales-admin</em>
|
||
is limited to the namespace <code>istio-system1</code>. A completed manifest would contain many
|
||
more <code>apiGroups</code> under the <code>Role</code> providing resource access to the tenant administrator.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >kind: Role
|
||
apiVersion: rbac.authorization.k8s.io/v1
|
||
metadata:
|
||
namespace: istio-system1
|
||
name: ns-access-for-sales-admin-istio-system1
|
||
rules:
|
||
- apiGroups: [&#34;&#34;] # &#34;&#34; indicates the core API group
|
||
resources: [&#34;*&#34;]
|
||
verbs: [&#34;*&#34;]
|
||
---
|
||
kind: RoleBinding
|
||
apiVersion: rbac.authorization.k8s.io/v1
|
||
metadata:
|
||
name: access-all-istio-system1
|
||
namespace: istio-system1
|
||
subjects:
|
||
- kind: User
|
||
name: sales-admin
|
||
apiGroup: rbac.authorization.k8s.io
|
||
roleRef:
|
||
kind: Role
|
||
name: ns-access-for-sales-admin-istio-system1
|
||
apiGroup: rbac.authorization.k8s.io
|
||
</code></pre>
|
||
<h3 id="watching-specific-namespaces-for-service-discovery">Watching specific namespaces for service discovery</h3>
|
||
<p>In addition to creating RBAC rules limiting the tenant administrator&rsquo;s access to a specific
|
||
Istio control plane, the Istio manifest must be updated to specify the application namespace
|
||
that Pilot should watch for creation of its xDS cache. This is done by starting the Pilot
|
||
component with the additional command line arguments <code>--appNamespace, ns-1</code>. Where <em>ns-1</em>
|
||
is the namespace that the tenant’s application will be deployed in. An example snippet from
|
||
the <code>istio-system1.yaml</code> file is shown below.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: istio-pilot
|
||
namespace: istio-system1
|
||
annotations:
|
||
sidecar.istio.io/inject: &#34;false&#34;
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
istio: pilot
|
||
spec:
|
||
serviceAccountName: istio-pilot-service-account
|
||
containers:
|
||
- name: discovery
|
||
image: docker.io/&lt;user ID&gt;/pilot:&lt;tag&gt;
|
||
imagePullPolicy: IfNotPresent
|
||
args: [&#34;discovery&#34;, &#34;-v&#34;, &#34;2&#34;, &#34;--admission-service&#34;, &#34;istio-pilot&#34;, &#34;--appNamespace&#34;, &#34;ns-1&#34;]
|
||
ports:
|
||
- containerPort: 8080
|
||
- containerPort: 443
|
||
</code></pre>
|
||
<h3 id="deploying-the-tenant-application-in-a-namespace">Deploying the tenant application in a namespace</h3>
|
||
<p>Now that the cluster administrator has created the tenant&rsquo;s namespace (ex. <code>istio-system1</code>) and
|
||
Pilot&rsquo;s service discovery has been configured to watch for a specific application
|
||
namespace (ex. <em>ns-1</em>), create the application manifests to deploy in that tenant&rsquo;s specific
|
||
namespace. For example:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: v1
|
||
kind: Namespace
|
||
metadata:
|
||
name: ns-1
|
||
</code></pre>
|
||
<p>And add the namespace reference to each resource type included in the application&rsquo;s manifest
|
||
file. For example:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: details
|
||
labels:
|
||
app: details
|
||
namespace: ns-1
|
||
</code></pre>
|
||
<p>Although not shown, the application namespaces will also have RBAC settings limiting access
|
||
to certain resources. These RBAC settings could be set by the cluster administrator and/or
|
||
the tenant administrator.</p>
|
||
<h3 id="using-kubectl-in-a-multi-tenant-environment">Using <code>kubectl</code> in a multi-tenant environment</h3>
|
||
<p>When defining <a href="https://archive.istio.io/v0.7/docs/reference/config/istio.routing.v1alpha1/#RouteRule">route rules</a>
|
||
or <a href="https://archive.istio.io/v0.7/docs/reference/config/istio.routing.v1alpha1/#DestinationPolicy">destination policies</a>,
|
||
it is necessary to ensure that the <code>kubectl</code> command is scoped to
|
||
the namespace the Istio control plane is running in to ensure the resource is created
|
||
in the proper namespace. Additionally, the rule itself must be scoped to the tenant&rsquo;s namespace
|
||
so that it will be applied properly to that tenant&rsquo;s mesh. The <em>-i</em> option is used to create
|
||
(or get or describe) the rule in the namespace that the Istio control plane is deployed in.
|
||
The <em>-n</em> option will scope the rule to the tenant&rsquo;s mesh and should be set to the namespace that
|
||
the tenant&rsquo;s app is deployed in. Note that the <em>-n</em> option can be skipped on the command line if
|
||
the .yaml file for the resource scopes it properly instead.</p>
|
||
<p>For example, the following command would be required to add a route rule to the <code>istio-system1</code>
|
||
namespace:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl –i istio-system1 apply -n ns-1 -f route_rule_v2.yaml
|
||
</code></pre>
|
||
<p>And can be displayed using the command:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl -i istio-system1 -n ns-1 get routerule
|
||
NAME KIND NAMESPACE
|
||
details-Default RouteRule.v1alpha2.config.istio.io ns-1
|
||
productpage-default RouteRule.v1alpha2.config.istio.io ns-1
|
||
ratings-default RouteRule.v1alpha2.config.istio.io ns-1
|
||
reviews-default RouteRule.v1alpha2.config.istio.io ns-1
|
||
</code></pre>
|
||
<p>See the <a href="/v1.1/blog/2018/soft-multitenancy/#multiple-istio-control-planes">Multiple Istio control planes</a> section of this document for more details on <code>namespace</code> requirements in a
|
||
multi-tenant environment.</p>
|
||
<h3 id="test-results">Test results</h3>
|
||
<p>Following the instructions above, a cluster administrator can create an environment limiting,
|
||
via RBAC and namespaces, what a tenant administrator can deploy.</p>
|
||
<p>After deployment, accessing the Istio control plane pods assigned to a specific tenant
|
||
administrator is permitted:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods -n istio-system
|
||
NAME READY STATUS RESTARTS AGE
|
||
grafana-78d649479f-8pqk9 1/1 Running 0 1d
|
||
istio-ca-ffbb75c6f-98w6x 1/1 Running 0 1d
|
||
istio-ingress-68d65fc5c6-dnvfl 1/1 Running 0 1d
|
||
istio-mixer-5b9f8dffb5-8875r 3/3 Running 0 1d
|
||
istio-pilot-678fc976c8-b8tv6 2/2 Running 0 1d
|
||
istio-sidecar-injector-7587bd559d-5tgk6 1/1 Running 0 1d
|
||
prometheus-cf8456855-hdcq7 1/1 Running 0 1d
|
||
</code></pre>
|
||
<p>However, accessing all the cluster&rsquo;s pods is not permitted:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods --all-namespaces
|
||
Error from server (Forbidden): pods is forbidden: User &#34;dev-admin&#34; cannot list pods at the cluster scope
|
||
</code></pre>
|
||
<p>And neither is accessing another tenant&rsquo;s namespace:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods -n istio-system1
|
||
Error from server (Forbidden): pods is forbidden: User &#34;dev-admin&#34; cannot list pods in the namespace &#34;istio-system1&#34;
|
||
</code></pre>
|
||
<p>The tenant administrator can deploy applications in the application namespace configured for
|
||
that tenant. As an example, updating the <a href="/v1.1/docs/examples/bookinfo/">Bookinfo</a>
|
||
manifests and then deploying under the tenant&rsquo;s application namespace of <em>ns-0</em>, listing the
|
||
pods in use by this tenant&rsquo;s namespace is permitted:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods -n ns-0
|
||
NAME READY STATUS RESTARTS AGE
|
||
details-v1-64b86cd49-b7rkr 2/2 Running 0 1d
|
||
productpage-v1-84f77f8747-rf2mt 2/2 Running 0 1d
|
||
ratings-v1-5f46655b57-5b4c5 2/2 Running 0 1d
|
||
reviews-v1-ff6bdb95b-pm5lb 2/2 Running 0 1d
|
||
reviews-v2-5799558d68-b989t 2/2 Running 0 1d
|
||
reviews-v3-58ff7d665b-lw5j9 2/2 Running 0 1d
|
||
</code></pre>
|
||
<p>But accessing another tenant&rsquo;s application namespace is not:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods -n ns-1
|
||
Error from server (Forbidden): pods is forbidden: User &#34;dev-admin&#34; cannot list pods in the namespace &#34;ns-1&#34;
|
||
</code></pre>
|
||
<p>If the <a href="/v1.1/docs/tasks/telemetry/">add-on tools</a>, example
|
||
<a href="/v1.1/docs/tasks/telemetry/metrics/querying-metrics/">Prometheus</a>, are deployed
|
||
(also limited by an Istio <code>namespace</code>) the statistical results returned would represent only
|
||
that traffic seen from that tenant&rsquo;s application namespace.</p>
|
||
<h2 id="conclusion">Conclusion</h2>
|
||
<p>The evaluation performed indicates Istio has sufficient capabilities and security to meet a
|
||
small number of multi-tenant use cases. It also shows that Istio and Kubernetes <strong>cannot</strong>
|
||
provide sufficient capabilities and security for other use cases, especially those use
|
||
cases that require complete security and isolation between untrusted tenants. The improvements
|
||
required to reach a more secure model of security and isolation require work in container
|
||
technology, ex. Kubernetes, rather than improvements in Istio capabilities.</p>
|
||
<h2 id="issues">Issues</h2>
|
||
<ul>
|
||
<li>The CA (Certificate Authority) and Mixer pod logs from one tenant&rsquo;s Istio control
|
||
plane (e.g. <code>istio-system</code> namespace) contained &lsquo;info&rsquo; messages from a second tenant&rsquo;s
|
||
Istio control plane (e.g. <code>istio-system1</code> namespace).</li>
|
||
</ul>
|
||
<h2 id="challenges-with-other-multi-tenancy-models">Challenges with other multi-tenancy models</h2>
|
||
<p>Other multi-tenancy deployment models were considered:</p>
|
||
<ol>
|
||
<li><p>A single mesh with multiple applications, one for each tenant on the mesh. The cluster
|
||
administrator gets control and visibility mesh wide and across all applications, while the
|
||
tenant administrator only gets control of a specific application.</p></li>
|
||
<li><p>A single Istio control plane with multiple meshes, one mesh per tenant. The cluster
|
||
administrator gets control and visibility across the entire Istio control plane and all
|
||
meshes, while the tenant administrator only gets control of a specific mesh.</p></li>
|
||
<li><p>A single cloud environment (cluster controlled), but multiple Kubernetes control planes
|
||
(tenant controlled).</p></li>
|
||
</ol>
|
||
<p>These options either can&rsquo;t be properly supported without code changes or don&rsquo;t fully
|
||
address the use cases.</p>
|
||
<p>Current Istio capabilities are poorly suited to support the first model as it lacks
|
||
sufficient RBAC capabilities to support cluster versus tenant operations. Additionally,
|
||
having multiple tenants under one mesh is too insecure with the current mesh model and the
|
||
way Istio drives configuration to the Envoy proxies.</p>
|
||
<p>Regarding the second option, the current Istio paradigm assumes a single mesh per Istio control
|
||
plane. The needed changes to support this model are substantial. They would require
|
||
finer grained scoping of resources and security domains based on namespaces, as well as,
|
||
additional Istio RBAC changes. This model will likely be addressed by future work, but not
|
||
currently possible.</p>
|
||
<p>The third model doesn’t satisfy most use cases, as most cluster administrators prefer
|
||
a common Kubernetes control plane which they provide as a
|
||
<a href="https://en.wikipedia.org/wiki/Platform_as_a_service">PaaS</a> to their tenants.</p>
|
||
<h2 id="future-work">Future work</h2>
|
||
<p>Allowing a single Istio control plane to control multiple meshes would be an obvious next
|
||
feature. An additional improvement is to provide a single mesh that can host different
|
||
tenants with some level of isolation and security between the tenants. This could be done
|
||
by partitioning within a single control plane using the same logical notion of namespace as
|
||
Kubernetes. A <a href="https://docs.google.com/document/d/14Hb07gSrfVt5KX9qNi7FzzGwB_6WBpAnDpPG6QEEd9Q">document</a>
|
||
has been started within the Istio community to define additional use cases and the
|
||
Istio functionality required to support those use cases.</p>
|
||
<h2 id="references">References</h2>
|
||
<ul>
|
||
<li>Video on Kubernetes multi-tenancy support, <a href="https://www.youtube.com/watch?v=ahwCkJGItkU">Multi-Tenancy Support &amp; Security Modeling with RBAC and Namespaces</a>, and the <a href="https://schd.ws/hosted_files/kccncna17/21/Multi-tenancy%20Support%20%26%20Security%20Modeling%20with%20RBAC%20and%20Namespaces.pdf">supporting slide deck</a>.</li>
|
||
<li>Kubecon talk on security that discusses Kubernetes support for &ldquo;Cooperative soft multi-tenancy&rdquo;, <a href="https://www.youtube.com/watch?v=YRR-kZub0cA">Building for Trust: How to Secure Your Kubernetes</a>.</li>
|
||
<li>Kubernetes documentation on <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">RBAC</a> and <a href="https://kubernetes.io/docs/tasks/administer-cluster/namespaces-walkthrough/">namespaces</a>.</li>
|
||
<li>Kubecon slide deck on <a href="https://schd.ws/hosted_files/kccncna17/a9/kubecon-multitenancy.pdf">Multi-tenancy Deep Dive</a>.</li>
|
||
<li>Google document on <a href="https://docs.google.com/document/d/15w1_fesSUZHv-vwjiYa9vN_uyc--PySRoLKTuDhimjc">Multi-tenancy models for Kubernetes</a>. (Requires permission)</li>
|
||
<li>Cloud Foundry WIP document, <a href="https://docs.google.com/document/d/14Hb07gSrfVt5KX9qNi7FzzGwB_6WBpAnDpPG6QEEd9Q">Multi-cloud and Multi-tenancy</a></li>
|
||
<li><a href="https://docs.google.com/document/d/12F183NIRAwj2hprx-a-51ByLeNqbJxK16X06vwH5OWE">Istio Auto Multi-Tenancy 101</a></li>
|
||
</ul></description><pubDate>Thu, 19 Apr 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/soft-multitenancy/</link><author>John Joyce and Rich Curran</author><guid isPermaLink="true">/v1.1/blog/2018/soft-multitenancy/</guid><category>tenancy</category></item><item><title>Traffic Mirroring with Istio for Testing in Production</title><description><p>Trying to enumerate all the possible combinations of test cases for testing services in non-production/test environments can be daunting. In some cases, you&rsquo;ll find that all of the effort that goes into cataloging these use cases doesn&rsquo;t match up to real production use cases. Ideally, we could use live production use cases and traffic to help illuminate all of the feature areas of the service under test that we might miss in more contrived testing environments.</p>
|
||
<p>Istio can help here. With the release of <a href="/v1.1/about/notes/older/0.5/">Istio 0.5.0</a>, Istio can mirror traffic to help test your services. You can write route rules similar to the following to enable traffic mirroring:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: config.istio.io/v1alpha2
|
||
kind: RouteRule
|
||
metadata:
|
||
name: mirror-traffic-to-httbin-v2
|
||
spec:
|
||
destination:
|
||
name: httpbin
|
||
precedence: 11
|
||
route:
|
||
- labels:
|
||
version: v1
|
||
weight: 100
|
||
- labels:
|
||
version: v2
|
||
weight: 0
|
||
mirror:
|
||
name: httpbin
|
||
labels:
|
||
version: v2
|
||
</code></pre>
|
||
<p>A few things to note here:</p>
|
||
<ul>
|
||
<li>When traffic gets mirrored to a different service, that happens outside the critical path of the request</li>
|
||
<li>Responses to any mirrored traffic is ignored; traffic is mirrored as &ldquo;fire-and-forget&rdquo;</li>
|
||
<li>You&rsquo;ll need to have the 0-weighted route to hint to Istio to create the proper Envoy cluster under the covers; <a href="https://github.com/istio/istio/issues/3270">this should be ironed out in future releases</a>.</li>
|
||
</ul>
|
||
<p>Learn more about mirroring by visiting the <a href="/v1.1/docs/tasks/traffic-management/mirroring/">Mirroring Task</a> and see a more
|
||
<a href="https://blog.christianposta.com/microservices/traffic-shadowing-with-istio-reduce-the-risk-of-code-release/">comprehensive treatment of this scenario on my blog</a>.</p></description><pubDate>Thu, 08 Feb 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/traffic-mirroring/</link><author>Christian Posta</author><guid isPermaLink="true">/v1.1/blog/2018/traffic-mirroring/</guid><category>traffic-management</category><category>mirroring</category></item><item><title>Consuming External TCP Services</title><description>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">This blog post was updated on July 23, 2018 to use the new
|
||
<a href="/v1.1/blog/2018/v1alpha3-routing/">v1alpha3 traffic management API</a>. If you need to use the old version, follow these <a href="https://archive.istio.io/v0.7/blog/2018/egress-tcp.html">docs</a>.</div>
|
||
</aside>
|
||
</div>
|
||
<p>In my previous blog post, <a href="/v1.1/blog/2018/egress-https/">Consuming External Web Services</a>, I described how external services
|
||
can be consumed by in-mesh Istio applications via HTTPS. In this post, I demonstrate consuming external services
|
||
over TCP. You will use the <a href="/v1.1/docs/examples/bookinfo/">Istio Bookinfo sample application</a>, the version in which the book
|
||
ratings data is persisted in a MySQL database. You deploy this database outside the cluster and configure the
|
||
<em>ratings</em> microservice to use it. You define a
|
||
<a href="/v1.1/docs/reference/config/networking/v1alpha3/service-entry/">Service Entry</a> to allow the in-mesh applications to
|
||
access the external database.</p>
|
||
<h2 id="bookinfo-sample-application-with-external-ratings-database">Bookinfo sample application with external ratings database</h2>
|
||
<p>First, you set up a MySQL database instance to hold book ratings data outside of your Kubernetes cluster. Then you
|
||
modify the <a href="/v1.1/docs/examples/bookinfo/">Bookinfo sample application</a> to use your database.</p>
|
||
<h3 id="setting-up-the-database-for-ratings-data">Setting up the database for ratings data</h3>
|
||
<p>For this task you set up an instance of <a href="https://www.mysql.com">MySQL</a>. You can use any MySQL instance; I used
|
||
<a href="https://www.ibm.com/cloud/compose/mysql">Compose for MySQL</a>. I used <code>mysqlsh</code>
|
||
(<a href="https://dev.mysql.com/doc/mysql-shell/en/">MySQL Shell</a>) as a MySQL client to feed the ratings data.</p>
|
||
<ol>
|
||
<li><p>Set the <code>MYSQL_DB_HOST</code> and <code>MYSQL_DB_PORT</code> environment variables:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export MYSQL_DB_HOST=&lt;your MySQL database host&gt;
|
||
$ export MYSQL_DB_PORT=&lt;your MySQL database port&gt;
|
||
</code></pre>
|
||
<p>In case of a local MySQL database with the default port, the values are <code>localhost</code> and <code>3306</code>, respectively.</p></li>
|
||
<li><p>To initialize the database, run the following command entering the password when prompted. The command is
|
||
performed with the credentials of the <code>admin</code> user, created by default by
|
||
<a href="https://www.ibm.com/cloud/compose/mysql">Compose for MySQL</a>.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ curl -s https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/src/mysql/mysqldb-init.sql | mysqlsh --sql --ssl-mode=REQUIRED -u admin -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT
|
||
</code></pre>
|
||
<p><em><strong>OR</strong></em></p>
|
||
<p>When using the <code>mysql</code> client and a local MySQL database, run:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ curl -s https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/src/mysql/mysqldb-init.sql | mysql -u root -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT
|
||
</code></pre></li>
|
||
<li><p>Create a user with the name <code>bookinfo</code> and grant it <em>SELECT</em> privilege on the <code>test.ratings</code> table:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ mysqlsh --sql --ssl-mode=REQUIRED -u admin -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &#34;CREATE USER &#39;bookinfo&#39; IDENTIFIED BY &#39;&lt;password you choose&gt;&#39;; GRANT SELECT ON test.ratings to &#39;bookinfo&#39;;&#34;
|
||
</code></pre>
|
||
<p><em><strong>OR</strong></em></p>
|
||
<p>For <code>mysql</code> and the local database, the command is:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ mysql -u root -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &#34;CREATE USER &#39;bookinfo&#39; IDENTIFIED BY &#39;&lt;password you choose&gt;&#39;; GRANT SELECT ON test.ratings to &#39;bookinfo&#39;;&#34;
|
||
</code></pre>
|
||
<p>Here you apply the <a href="https://en.wikipedia.org/wiki/Principle_of_least_privilege">principle of least privilege</a>. This
|
||
means that you do not use your <code>admin</code> user in the Bookinfo application. Instead, you create a special user for the
|
||
Bookinfo application , <code>bookinfo</code>, with minimal privileges. In this case, the <em>bookinfo</em> user only has the <code>SELECT</code>
|
||
privilege on a single table.</p>
|
||
<p>After running the command to create the user, you may want to clean your bash history by checking the number of the last
|
||
command and running <code>history -d &lt;the number of the command that created the user&gt;</code>. You don&rsquo;t want the password of the
|
||
new user to be stored in the bash history. If you&rsquo;re using <code>mysql</code>, remove the last command from
|
||
<code>~/.mysql_history</code> file as well. Read more about password protection of the newly created user in <a href="https://dev.mysql.com/doc/refman/5.5/en/create-user.html">MySQL documentation</a>.</p></li>
|
||
<li><p>Inspect the created ratings to see that everything worked as expected:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ mysqlsh --sql --ssl-mode=REQUIRED -u bookinfo -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &#34;select * from test.ratings;&#34;
|
||
Enter password:
|
||
+----------+--------+
|
||
| ReviewID | Rating |
|
||
+----------+--------+
|
||
| 1 | 5 |
|
||
| 2 | 4 |
|
||
+----------+--------+
|
||
</code></pre>
|
||
<p><em><strong>OR</strong></em></p>
|
||
<p>For <code>mysql</code> and the local database:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ mysql -u bookinfo -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &#34;select * from test.ratings;&#34;
|
||
Enter password:
|
||
+----------+--------+
|
||
| ReviewID | Rating |
|
||
+----------+--------+
|
||
| 1 | 5 |
|
||
| 2 | 4 |
|
||
+----------+--------+
|
||
</code></pre></li>
|
||
<li><p>Set the ratings temporarily to <code>1</code> to provide a visual clue when our database is used by the Bookinfo <em>ratings</em>
|
||
service:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ mysqlsh --sql --ssl-mode=REQUIRED -u admin -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &#34;update test.ratings set rating=1; select * from test.ratings;&#34;
|
||
Enter password:
|
||
Rows matched: 2 Changed: 2 Warnings: 0
|
||
+----------+--------+
|
||
| ReviewID | Rating |
|
||
+----------+--------+
|
||
| 1 | 1 |
|
||
| 2 | 1 |
|
||
+----------+--------+
|
||
</code></pre>
|
||
<p><em><strong>OR</strong></em></p>
|
||
<p>For <code>mysql</code> and the local database:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ mysql -u root -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &#34;update test.ratings set rating=1; select * from test.ratings;&#34;
|
||
Enter password:
|
||
+----------+--------+
|
||
| ReviewID | Rating |
|
||
+----------+--------+
|
||
| 1 | 1 |
|
||
| 2 | 1 |
|
||
+----------+--------+
|
||
</code></pre>
|
||
<p>You used the <code>admin</code> user (and <code>root</code> for the local database) in the last command since the <code>bookinfo</code> user does not
|
||
have the <code>UPDATE</code> privilege on the <code>test.ratings</code> table.</p></li>
|
||
</ol>
|
||
<p>Now you are ready to deploy a version of the Bookinfo application that will use your database.</p>
|
||
<h3 id="initial-setting-of-bookinfo-application">Initial setting of Bookinfo application</h3>
|
||
<p>To demonstrate the scenario of using an external database, you start with a Kubernetes cluster with <a href="/v1.1/docs/setup/kubernetes/install/kubernetes/#installation-steps">Istio installed</a>. Then you deploy the
|
||
<a href="/v1.1/docs/examples/bookinfo/">Istio Bookinfo sample application</a>, <a href="/v1.1/docs/examples/bookinfo/#apply-default-destination-rules">apply the default destination rules</a>, and <a href="/v1.1/docs/tasks/traffic-management/egress/#change-to-the-blocking-by-default-policy">change Istio to the blocking-egress-by-default policy</a>.</p>
|
||
<p>This application uses the <code>ratings</code> microservice to fetch
|
||
book ratings, a number between 1 and 5. The ratings are displayed as stars for each review. There are several versions
|
||
of the <code>ratings</code> microservice. Some use <a href="https://www.mongodb.com">MongoDB</a>, others use <a href="https://www.mysql.com">MySQL</a>
|
||
as their database.</p>
|
||
<p>The example commands in this blog post work with Istio 0.8+, with or without
|
||
<a href="/v1.1/docs/concepts/security/#mutual-tls-authentication">mutual TLS</a> enabled.</p>
|
||
<p>As a reminder, here is the end-to-end architecture of the application from the
|
||
<a href="/v1.1/docs/examples/bookinfo/">Bookinfo sample application</a>.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.086918235567985%">
|
||
<a data-skipendnotes="true" href="/v1.1/docs/examples/bookinfo/withistio.svg" title="The original Bookinfo application">
|
||
<img class="element-to-stretch" src="/v1.1/docs/examples/bookinfo/withistio.svg" alt="The original Bookinfo application" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The original Bookinfo application</figcaption>
|
||
</figure>
|
||
<h3 id="use-the-database-for-ratings-data-in-bookinfo-application">Use the database for ratings data in Bookinfo application</h3>
|
||
<ol>
|
||
<li><p>Modify the deployment spec of a version of the <em>ratings</em> microservice that uses a MySQL database, to use your
|
||
database instance. The spec is in <a href="https://github.com/istio/istio/blob/release-1.1/samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml"><code>samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml</code></a>
|
||
of an Istio release archive. Edit the following lines:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >- name: MYSQL_DB_HOST
|
||
value: mysqldb
|
||
- name: MYSQL_DB_PORT
|
||
value: &#34;3306&#34;
|
||
- name: MYSQL_DB_USER
|
||
value: root
|
||
- name: MYSQL_DB_PASSWORD
|
||
value: password
|
||
</code></pre>
|
||
<p>Replace the values in the snippet above, specifying the database host, port, user, and password. Note that the
|
||
correct way to work with passwords in container&rsquo;s environment variables in Kubernetes is <a href="https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables">to use secrets</a>. For this
|
||
example task only, you may want to write the password directly in the deployment spec. <strong>Do not do it</strong> in a real
|
||
environment! I also assume everyone realizes that <code>&quot;password&quot;</code> should not be used as a password&hellip;</p></li>
|
||
<li><p>Apply the modified spec to deploy the version of the <em>ratings</em> microservice, <em>v2-mysql</em>, that will use your
|
||
database.</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml@
|
||
deployment &#34;ratings-v2-mysql&#34; created
|
||
</code></pre></div></li>
|
||
<li><p>Route all the traffic destined to the <em>reviews</em> service to its <em>v3</em> version. You do this to ensure that the
|
||
<em>reviews</em> service always calls the <em>ratings</em> service. In addition, route all the traffic destined to the <em>ratings</em>
|
||
service to <em>ratings v2-mysql</em> that uses your database.</p>
|
||
<p>Specify the routing for both services above by adding two
|
||
<a href="/v1.1/docs/reference/config/networking/v1alpha3/virtual-service/">virtual services</a>. These virtual services are
|
||
specified in <code>samples/bookinfo/networking/virtual-service-ratings-mysql.yaml</code> of an Istio release archive.
|
||
<strong><em>Important:</em></strong> make sure you
|
||
<a href="/v1.1/docs/examples/bookinfo/#apply-default-destination-rules">applied the default destination rules</a> before running the
|
||
following command.</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/virtual-service-ratings-mysql.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/networking/virtual-service-ratings-mysql.yaml@
|
||
</code></pre></div></li>
|
||
</ol>
|
||
<p>The updated architecture appears below. Note that the blue arrows inside the mesh mark the traffic configured according
|
||
to the virtual services we added. According to the virtual services, the traffic is sent to <em>reviews v3</em> and
|
||
<em>ratings v2-mysql</em>.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.314858206480224%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-tcp/./bookinfo-ratings-v2-mysql-external.svg" title="The Bookinfo application with ratings v2-mysql and an external MySQL database">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-tcp/./bookinfo-ratings-v2-mysql-external.svg" alt="The Bookinfo application with ratings v2-mysql and an external MySQL database" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The Bookinfo application with ratings v2-mysql and an external MySQL database</figcaption>
|
||
</figure>
|
||
<p>Note that the MySQL database is outside the Istio service mesh, or more precisely outside the Kubernetes cluster. The
|
||
boundary of the service mesh is marked by a dashed line.</p>
|
||
<h3 id="access-the-webpage">Access the webpage</h3>
|
||
<p>Access the webpage of the application, after
|
||
<a href="/v1.1/docs/examples/bookinfo/#determining-the-ingress-ip-and-port">determining the ingress IP and port</a>.</p>
|
||
<p>You have a problem&hellip; Instead of the rating stars, the message <em>&ldquo;Ratings service is currently unavailable&rdquo;</em> is currently
|
||
displayed below each review:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.18705035971223%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-tcp/./errorFetchingBookRating.png" title="The Ratings service error messages">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-tcp/./errorFetchingBookRating.png" alt="The Ratings service error messages" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The Ratings service error messages</figcaption>
|
||
</figure>
|
||
<p>As in <a href="/v1.1/blog/2018/egress-https/">Consuming External Web Services</a>, you experience <strong>graceful service degradation</strong>,
|
||
which is good. The application did not crash due to the error in the <em>ratings</em> microservice. The webpage of the
|
||
application correctly displayed the book information, the details, and the reviews, just without the rating stars.</p>
|
||
<p>You have the same problem as in <a href="/v1.1/blog/2018/egress-https/">Consuming External Web Services</a>, namely all the traffic
|
||
outside the Kubernetes cluster, both TCP and HTTP, is blocked by default by the sidecar proxies. To enable such traffic
|
||
for TCP, a mesh-external service entry for TCP must be defined.</p>
|
||
<h3 id="mesh-external-service-entry-for-an-external-mysql-instance">Mesh-external service entry for an external MySQL instance</h3>
|
||
<p>TCP mesh-external service entries come to our rescue.</p>
|
||
<ol>
|
||
<li><p>Get the IP address of your MySQL database instance. As an option, you can use the
|
||
<a href="https://linux.die.net/man/1/host">host</a> command:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ export MYSQL_DB_IP=$(host $MYSQL_DB_HOST | grep &#34; has address &#34; | cut -d&#34; &#34; -f4)
|
||
</code></pre>
|
||
<p>For a local database, set <code>MYSQL_DB_IP</code> to contain the IP of your machine, accessible from your cluster.</p></li>
|
||
<li><p>Define a TCP mesh-external service entry:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: mysql-external
|
||
spec:
|
||
hosts:
|
||
- $MYSQL_DB_HOST
|
||
addresses:
|
||
- $MYSQL_DB_IP/32
|
||
ports:
|
||
- name: tcp
|
||
number: $MYSQL_DB_PORT
|
||
protocol: tcp
|
||
location: MESH_EXTERNAL
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Review the service entry you just created and check that it contains the correct values:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get serviceentry mysql-external -o yaml
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
...
|
||
</code></pre></li>
|
||
</ol>
|
||
<p>Note that for a TCP service entry, you specify <code>tcp</code> as the protocol of a port of the entry. Also note that you have to
|
||
specify the IP of the external service in the list of addresses, as a <a href="https://tools.ietf.org/html/rfc2317">CIDR</a> block
|
||
with suffix <code>32</code>.</p>
|
||
<p>I will talk more about TCP service entries
|
||
<a href="#service-entries-for-tcp-traffic">below</a>. For now, verify that the service entry we added fixed the problem. Access the
|
||
webpage and see if the stars are back.</p>
|
||
<p>It worked! Accessing the web page of the application displays the ratings without error:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.69064748201439%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-tcp/./externalMySQLRatings.png" title="Book Ratings Displayed Correctly">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-tcp/./externalMySQLRatings.png" alt="Book Ratings Displayed Correctly" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Book Ratings Displayed Correctly</figcaption>
|
||
</figure>
|
||
<p>Note that you see a one-star rating for both displayed reviews, as expected. You changed the ratings to be one star to
|
||
provide us with a visual clue that our external database is indeed being used.</p>
|
||
<p>As with service entries for HTTP/HTTPS, you can delete and create service entries for TCP using <code>kubectl</code>, dynamically.</p>
|
||
<h2 id="motivation-for-egress-tcp-traffic-control">Motivation for egress TCP traffic control</h2>
|
||
<p>Some in-mesh Istio applications must access external services, for example legacy systems. In many cases, the access is
|
||
not performed over HTTP or HTTPS protocols. Other TCP protocols are used, such as database-specific protocols like
|
||
<a href="https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/">MongoDB Wire Protocol</a> and <a href="https://dev.mysql.com/doc/internals/en/client-server-protocol.html">MySQL Client/Server Protocol</a> to communicate with external databases.</p>
|
||
<p>Next let me provide more details about the service entries for TCP traffic.</p>
|
||
<h2 id="service-entries-for-tcp-traffic">Service entries for TCP traffic</h2>
|
||
<p>The service entries for enabling TCP traffic to a specific port must specify <code>TCP</code> as the protocol of the port.
|
||
Additionally, for the <a href="https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/">MongoDB Wire Protocol</a>, the
|
||
protocol can be specified as <code>MONGO</code>, instead of <code>TCP</code>.</p>
|
||
<p>For the <code>addresses</code> field of the entry, a block of IPs in <a href="https://tools.ietf.org/html/rfc2317">CIDR</a>
|
||
notation must be used. Note that the <code>hosts</code> field is ignored for TCP service entries.</p>
|
||
<p>To enable TCP traffic to an external service by its hostname, all the IPs of the hostname must be specified. Each IP
|
||
must be specified by a CIDR block.</p>
|
||
<p>Note that all the IPs of an external service are not always known. To enable egress TCP traffic, only the IPs that are
|
||
used by the applications must be specified.</p>
|
||
<p>Also note that the IPs of an external service are not always static, for example in the case of
|
||
<a href="https://en.wikipedia.org/wiki/Content_delivery_network">CDNs</a>. Sometimes the IPs are static most of the time, but can
|
||
be changed from time to time, for example due to infrastructure changes. In these cases, if the range of the possible
|
||
IPs is known, you should specify the range by CIDR blocks. If the range of the possible IPs is not known, service
|
||
entries for TCP cannot be used and
|
||
<a href="/v1.1/docs/tasks/traffic-management/egress/#direct-access-to-external-services">the external services must be called directly</a>,
|
||
bypassing the sidecar proxies.</p>
|
||
<h2 id="relation-to-mesh-expansion">Relation to mesh expansion</h2>
|
||
<p>Note that the scenario described in this post is different from the mesh expansion scenario, described in the
|
||
<a href="/v1.1/docs/examples/integrating-vms/">Integrating Virtual Machines</a> example. In that scenario, a MySQL instance runs on an
|
||
external
|
||
(outside the cluster) machine (a bare metal or a VM), integrated with the Istio service mesh. The MySQL service becomes
|
||
a first-class citizen of the mesh with all the beneficial features of Istio applicable. Among other things, the service
|
||
becomes addressable by a local cluster domain name, for example by <code>mysqldb.vm.svc.cluster.local</code>, and the communication
|
||
to it can be secured by
|
||
<a href="/v1.1/docs/concepts/security/#mutual-tls-authentication">mutual TLS authentication</a>. There is no need to create a service
|
||
entry to access this service; however, the service must be registered with Istio. To enable such integration, Istio
|
||
components (<em>Envoy proxy</em>, <em>node-agent</em>, <code>_istio-agent_</code>) must be installed on the machine and the Istio control plane
|
||
(<em>Pilot</em>, <em>Mixer</em>, <em>Citadel</em>) must be accessible from it. See the
|
||
<a href="/v1.1/docs/setup/kubernetes/additional-setup/mesh-expansion/">Istio Mesh Expansion</a> instructions for more details.</p>
|
||
<p>In our case, the MySQL instance can run on any machine or can be provisioned as a service by a cloud provider. There is
|
||
no requirement to integrate the machine with Istio. The Istio control plane does not have to be accessible from the
|
||
machine. In the case of MySQL as a service, the machine which MySQL runs on may be not accessible and installing on it
|
||
the required components may be impossible. In our case, the MySQL instance is addressable by its global domain name,
|
||
which could be beneficial if the consuming applications expect to use that domain name. This is especially relevant when
|
||
that expected domain name cannot be changed in the deployment configuration of the consuming applications.</p>
|
||
<h2 id="cleanup">Cleanup</h2>
|
||
<ol>
|
||
<li><p>Drop the <code>test</code> database and the <code>bookinfo</code> user:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ mysqlsh --sql --ssl-mode=REQUIRED -u admin -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &#34;drop database test; drop user bookinfo;&#34;
|
||
</code></pre>
|
||
<p><em><strong>OR</strong></em></p>
|
||
<p>For <code>mysql</code> and the local database:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ mysql -u root -p --host $MYSQL_DB_HOST --port $MYSQL_DB_PORT -e &#34;drop database test; drop user bookinfo;&#34;
|
||
</code></pre></li>
|
||
<li><p>Remove the virtual services:</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/virtual-service-ratings-mysql.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete -f @samples/bookinfo/networking/virtual-service-ratings-mysql.yaml@
|
||
Deleted config: virtual-service/default/reviews
|
||
Deleted config: virtual-service/default/ratings
|
||
</code></pre></div></li>
|
||
<li><p>Undeploy <em>ratings v2-mysql</em>:</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete -f @samples/bookinfo/platform/kube/bookinfo-ratings-v2-mysql.yaml@
|
||
deployment &#34;ratings-v2-mysql&#34; deleted
|
||
</code></pre></div></li>
|
||
<li><p>Delete the service entry:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry mysql-external -n default
|
||
Deleted config: serviceentry mysql-external
|
||
</code></pre></li>
|
||
</ol>
|
||
<h2 id="conclusion">Conclusion</h2>
|
||
<p>In this blog post, I demonstrated how the microservices in an Istio service mesh can consume external services via TCP.
|
||
By default, Istio blocks all the traffic, TCP and HTTP, to the hosts outside the cluster. To enable such traffic for
|
||
TCP, TCP mesh-external service entries must be created for the service mesh.</p></description><pubDate>Tue, 06 Feb 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/egress-tcp/</link><author>Vadim Eisenberg</author><guid isPermaLink="true">/v1.1/blog/2018/egress-tcp/</guid><category>traffic-management</category><category>egress</category><category>tcp</category></item><item><title>Consuming External Web Services</title><description>
|
||
<p>In many cases, not all the parts of a microservices-based application reside in a <em>service mesh</em>. Sometimes, the
|
||
microservices-based applications use functionality provided by legacy systems that reside outside the mesh. You may want
|
||
to migrate these systems to the service mesh gradually. Until these systems are migrated, they must be accessed by the
|
||
applications inside the mesh. In other cases, the applications use web services provided by third parties.</p>
|
||
<p>In this blog post, I modify the <a href="/v1.1/docs/examples/bookinfo/">Istio Bookinfo Sample Application</a> to fetch book details from
|
||
an external web service (<a href="https://developers.google.com/books/docs/v1/getting_started">Google Books APIs</a>). I show how
|
||
to enable egress HTTPS traffic in Istio by using <em>mesh-external service entries</em>. I provide two options for egress
|
||
HTTPS traffic and describe the pros and cons of each of the options.</p>
|
||
<h2 id="initial-setting">Initial setting</h2>
|
||
<p>To demonstrate the scenario of consuming an external web service, I start with a Kubernetes cluster with <a href="/v1.1/docs/setup/kubernetes/install/kubernetes/#installation-steps">Istio installed</a>. Then I deploy
|
||
<a href="/v1.1/docs/examples/bookinfo/">Istio Bookinfo Sample Application</a>. This application uses the <em>details</em> microservice to fetch
|
||
book details, such as the number of pages and the publisher. The original <em>details</em> microservice provides the book
|
||
details without consulting any external service.</p>
|
||
<p>The example commands in this blog post work with Istio 1.0+, with or without
|
||
<a href="/v1.1/docs/concepts/security/#mutual-tls-authentication">mutual TLS</a> enabled. The Bookinfo configuration files reside in the
|
||
<code>samples/bookinfo</code> directory of the Istio release archive.</p>
|
||
<p>Here is a copy of the end-to-end architecture of the application from the original
|
||
<a href="/v1.1/docs/examples/bookinfo/">Bookinfo sample application</a>.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.086918235567985%">
|
||
<a data-skipendnotes="true" href="/v1.1/docs/examples/bookinfo/withistio.svg" title="The Original Bookinfo Application">
|
||
<img class="element-to-stretch" src="/v1.1/docs/examples/bookinfo/withistio.svg" alt="The Original Bookinfo Application" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The Original Bookinfo Application</figcaption>
|
||
</figure>
|
||
<p>Perform the steps in the
|
||
<a href="/v1.1/docs/examples/bookinfo/#deploying-the-application">Deploying the application</a>,
|
||
<a href="/v1.1/docs/examples/bookinfo/#confirm-the-app-is-accessible-from-outside-the-cluster">Confirm the app is running</a>,
|
||
<a href="/v1.1/docs/examples/bookinfo/#apply-default-destination-rules">Apply default destination rules</a>
|
||
sections, and
|
||
<a href="/v1.1/docs/tasks/traffic-management/egress/#change-to-the-blocking-by-default-policy">change Istio to the blocking-egress-by-default policy</a>.</p>
|
||
<h2 id="bookinfo-with-https-access-to-a-google-books-web-service">Bookinfo with HTTPS access to a Google Books web service</h2>
|
||
<p>Deploy a new version of the <em>details</em> microservice, <em>v2</em>, that fetches the book details from <a href="https://developers.google.com/books/docs/v1/getting_started">Google Books APIs</a>. Run the following command; it sets the
|
||
<code>DO_NOT_ENCRYPT</code> environment variable of the service&rsquo;s container to <code>false</code>. This setting will instruct the deployed
|
||
service to use HTTPS (instead of HTTP) to access to the external service.</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo-details-v2.yaml@ --dry-run -o yaml | kubectl set env --local -f - &#39;DO_NOT_ENCRYPT=false&#39; -o yaml | kubectl apply -f -
|
||
</code></pre></div>
|
||
<p>The updated architecture of the application now looks as follows:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:65.1654485092242%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-https/./bookinfo-details-v2.svg" title="The Bookinfo Application with details V2">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-https/./bookinfo-details-v2.svg" alt="The Bookinfo Application with details V2" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The Bookinfo Application with details V2</figcaption>
|
||
</figure>
|
||
<p>Note that the Google Books web service is outside the Istio service mesh, the boundary of which is marked by a dashed
|
||
line.</p>
|
||
<p>Now direct all the traffic destined to the <em>details</em> microservice, to <em>details version v2</em>.</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/virtual-service-details-v2.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/networking/virtual-service-details-v2.yaml@
|
||
</code></pre></div>
|
||
<p>Note that the virtual service relies on a destination rule that you created in the <a href="/v1.1/docs/examples/bookinfo/#apply-default-destination-rules">Apply default destination rules</a> section.</p>
|
||
<p>Access the web page of the application, after
|
||
<a href="/v1.1/docs/examples/bookinfo/#determining-the-ingress-ip-and-port">determining the ingress IP and port</a>.</p>
|
||
<p>Oops&hellip; Instead of the book details you have the <em>Error fetching product details</em> message displayed:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:36.18649965205289%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-https/./errorFetchingBookDetails.png" title="The Error Fetching Product Details Message">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-https/./errorFetchingBookDetails.png" alt="The Error Fetching Product Details Message" />
|
||
</a>
|
||
</div>
|
||
<figcaption>The Error Fetching Product Details Message</figcaption>
|
||
</figure>
|
||
<p>The good news is that your application did not crash. With a good microservice design, you do not have <strong>failure
|
||
propagation</strong>. In your case, the failing <em>details</em> microservice does not cause the <code>productpage</code> microservice to fail.
|
||
Most of the functionality of the application is still provided, despite the failure in the <em>details</em> microservice. You
|
||
have <strong>graceful service degradation</strong>: as you can see, the reviews and the ratings are displayed correctly, and the
|
||
application is still useful.</p>
|
||
<p>So what might have gone wrong? Ah&hellip; The answer is that I forgot to tell you to enable traffic from inside the mesh to
|
||
an external service, in this case to the Google Books web service. By default, the Istio sidecar proxies
|
||
(<a href="https://www.envoyproxy.io">Envoy proxies</a>) <strong>block all the traffic to destinations outside the cluster</strong>. To enable
|
||
such traffic, you must define a
|
||
<a href="/v1.1/docs/reference/config/networking/v1alpha3/service-entry/">mesh-external service entry</a>.</p>
|
||
<h3 id="enable-https-access-to-a-google-books-web-service">Enable HTTPS access to a Google Books web service</h3>
|
||
<p>No worries, define a <strong>mesh-external service entry</strong> and fix your application. You must also define a <em>virtual
|
||
service</em> to perform routing by <a href="https://en.wikipedia.org/wiki/Server_Name_Indication">SNI</a> to the external service.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: googleapis
|
||
spec:
|
||
hosts:
|
||
- www.googleapis.com
|
||
ports:
|
||
- number: 443
|
||
name: https
|
||
protocol: HTTPS
|
||
location: MESH_EXTERNAL
|
||
resolution: DNS
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: googleapis
|
||
spec:
|
||
hosts:
|
||
- www.googleapis.com
|
||
tls:
|
||
- match:
|
||
- port: 443
|
||
sni_hosts:
|
||
- www.googleapis.com
|
||
route:
|
||
- destination:
|
||
host: www.googleapis.com
|
||
port:
|
||
number: 443
|
||
weight: 100
|
||
EOF
|
||
</code></pre>
|
||
<p>Now accessing the web page of the application displays the book details without error:</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:34.82831114225648%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-https/./externalBookDetails.png" title="Book Details Displayed Correctly">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-https/./externalBookDetails.png" alt="Book Details Displayed Correctly" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Book Details Displayed Correctly</figcaption>
|
||
</figure>
|
||
<p>You can query your service entries:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get serviceentries
|
||
NAME AGE
|
||
googleapis 8m
|
||
</code></pre>
|
||
<p>You can delete your service entry:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry googleapis
|
||
serviceentry &#34;googleapis&#34; deleted
|
||
</code></pre>
|
||
<p>and see in the output that the service entry is deleted.</p>
|
||
<p>Accessing the web page after deleting the service entry produces the same error that you experienced before, namely
|
||
<em>Error fetching product details</em>. As you can see, the service entries are defined <strong>dynamically</strong>, as are many other
|
||
Istio configuration artifacts. The Istio operators can decide dynamically which domains they allow the microservices to
|
||
access. They can enable and disable traffic to the external domains on the fly, without redeploying the microservices.</p>
|
||
<h3 id="cleanup-of-https-access-to-a-google-books-web-service">Cleanup of HTTPS access to a Google Books web service</h3>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml'>Zip</a><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/virtual-service-details-v2.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry googleapis
|
||
$ kubectl delete virtualservice googleapis
|
||
$ kubectl delete -f @samples/bookinfo/networking/virtual-service-details-v2.yaml@
|
||
$ kubectl delete -f @samples/bookinfo/platform/kube/bookinfo-details-v2.yaml@
|
||
</code></pre></div>
|
||
<h2 id="tls-origination-by-istio">TLS origination by Istio</h2>
|
||
<p>There is a caveat to this story. Suppose you want to monitor which specific set of
|
||
<a href="https://developers.google.com/apis-explorer/">Google APIs</a> your microservices use
|
||
(<a href="https://developers.google.com/books/docs/v1/getting_started">Books</a>,
|
||
<a href="https://developers.google.com/calendar/">Calendar</a>, <a href="https://developers.google.com/tasks/">Tasks</a> etc.)
|
||
Suppose you want to enforce a policy that using only
|
||
<a href="https://developers.google.com/books/docs/v1/getting_started">Books APIs</a> is allowed. Suppose you want to monitor the
|
||
book identifiers that your microservices access. For these monitoring and policy tasks you need to know the URL path.
|
||
Consider for example the URL
|
||
<a href="https://www.googleapis.com/books/v1/volumes?q=isbn:0486424618"><code>www.googleapis.com/books/v1/volumes?q=isbn:0486424618</code></a>.
|
||
In that URL, <a href="https://developers.google.com/books/docs/v1/getting_started">Books APIs</a> is specified by the path segment
|
||
<code>/books</code>, and the <a href="https://en.wikipedia.org/wiki/International_Standard_Book_Number">ISBN</a> number by the path segment
|
||
<code>/volumes?q=isbn:0486424618</code>. However, in HTTPS, all the HTTP details (hostname, path, headers etc.) are encrypted and
|
||
such monitoring and policy enforcement by the sidecar proxies is not possible. Istio can only know the server name of
|
||
the encrypted requests by the <a href="https://tools.ietf.org/html/rfc3546#section-3.1">SNI</a> (<em>Server Name Indication</em>) field,
|
||
in this case <code>www.googleapis.com</code>.</p>
|
||
<p>To allow Istio to perform monitoring and policy enforcement of egress requests based on HTTP details, the microservices
|
||
must issue HTTP requests. Istio then opens an HTTPS connection to the destination (performs TLS origination). The code
|
||
of the microservices must be written differently or configured differently, according to whether the microservice runs
|
||
inside or outside an Istio service mesh. This contradicts the Istio design goal of <a href="/v1.1/docs/concepts/what-is-istio/#design-goals">maximizing transparency</a>. Sometimes you need to compromise&hellip;</p>
|
||
<p>The diagram below shows two options for sending HTTPS traffic to external services. On the top, a microservice sends
|
||
regular HTTPS requests, encrypted end-to-end. On the bottom, the same microservice sends unencrypted HTTP requests
|
||
inside a pod, which are intercepted by the sidecar Envoy proxy. The sidecar proxy performs TLS origination, so the
|
||
traffic between the pod and the external service is encrypted.</p>
|
||
<figure style="width:60%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:95.1355088590701%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2018/egress-https/./https_from_the_app.svg" title="HTTPS traffic to external services, with TLS originated by the microservice vs. by the sidecar proxy">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2018/egress-https/./https_from_the_app.svg" alt="HTTPS traffic to external services, with TLS originated by the microservice vs. by the sidecar proxy" />
|
||
</a>
|
||
</div>
|
||
<figcaption>HTTPS traffic to external services, with TLS originated by the microservice vs. by the sidecar proxy</figcaption>
|
||
</figure>
|
||
<p>Here is how both patterns are supported in the
|
||
<a href="https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/src/details/details.rb">Bookinfo details microservice code</a>, using the Ruby
|
||
<a href="https://docs.ruby-lang.org/en/2.0.0/Net/HTTP.html">net/http module</a>:</p>
|
||
<pre><code class='language-ruby' data-expandlinks='true' >uri = URI.parse(&#39;https://www.googleapis.com/books/v1/volumes?q=isbn:&#39; + isbn)
|
||
http = Net::HTTP.new(uri.host, ENV[&#39;DO_NOT_ENCRYPT&#39;] === &#39;true&#39; ? 80:443)
|
||
...
|
||
unless ENV[&#39;DO_NOT_ENCRYPT&#39;] === &#39;true&#39; then
|
||
http.use_ssl = true
|
||
end
|
||
</code></pre>
|
||
<p>When the <code>DO_NOT_ENCRYPT</code> environment variable is defined, the request is performed without SSL (plain HTTP) to port 80.</p>
|
||
<p>You can set the <code>DO_NOT_ENCRYPT</code> environment variable to <em>&ldquo;true&rdquo;</em> in the
|
||
<a href="https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml">Kubernetes deployment spec of details v2</a>,
|
||
the <code>container</code> section:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >env:
|
||
- name: DO_NOT_ENCRYPT
|
||
value: &#34;true&#34;
|
||
</code></pre>
|
||
<p>In the next section you will configure TLS origination for accessing an external web service.</p>
|
||
<h2 id="bookinfo-with-tls-origination-to-a-google-books-web-service">Bookinfo with TLS origination to a Google Books web service</h2>
|
||
<ol>
|
||
<li><p>Deploy a version of <em>details v2</em> that sends an HTTP request to
|
||
<a href="https://developers.google.com/books/docs/v1/getting_started">Google Books APIs</a>. The <code>DO_NOT_ENCRYPT</code> variable
|
||
is set to true in
|
||
<a href="https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml"><code>bookinfo-details-v2.yaml</code></a>.</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo-details-v2.yaml@
|
||
</code></pre></div></li>
|
||
<li><p>Direct the traffic destined to the <em>details</em> microservice, to <em>details version v2</em>.</p>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/virtual-service-details-v2.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f @samples/bookinfo/networking/virtual-service-details-v2.yaml@
|
||
</code></pre></div></li>
|
||
<li><p>Create a mesh-external service entry for <code>www.google.apis</code> , a virtual service to rewrite the destination port from
|
||
80 to 443, and a destination rule to perform TLS origination.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: ServiceEntry
|
||
metadata:
|
||
name: googleapis
|
||
spec:
|
||
hosts:
|
||
- www.googleapis.com
|
||
ports:
|
||
- number: 80
|
||
name: http
|
||
protocol: HTTP
|
||
- number: 443
|
||
name: https
|
||
protocol: HTTPS
|
||
resolution: DNS
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: rewrite-port-for-googleapis
|
||
spec:
|
||
hosts:
|
||
- www.googleapis.com
|
||
http:
|
||
- match:
|
||
- port: 80
|
||
route:
|
||
- destination:
|
||
host: www.googleapis.com
|
||
port:
|
||
number: 443
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: originate-tls-for-googleapis
|
||
spec:
|
||
host: www.googleapis.com
|
||
trafficPolicy:
|
||
loadBalancer:
|
||
simple: ROUND_ROBIN
|
||
portLevelSettings:
|
||
- port:
|
||
number: 443
|
||
tls:
|
||
mode: SIMPLE # initiates HTTPS when accessing www.googleapis.com
|
||
EOF
|
||
</code></pre></li>
|
||
<li><p>Access the web page of the application and verify that the book details are displayed without errors.</p></li>
|
||
<li><p><a href="/v1.1/docs/tasks/telemetry/logs/access-log/#enable-envoy-s-access-logging">Enable Envoy’s access logging</a></p></li>
|
||
<li><p>Check the log of of the sidecar proxy of <em>details v2</em> and see the HTTP request.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl logs $(kubectl get pods -l app=details -l version=v2 -o jsonpath=&#39;{.items[0].metadata.name}&#39;) istio-proxy | grep googleapis
|
||
[2018-08-09T11:32:58.171Z] &#34;GET /books/v1/volumes?q=isbn:0486424618 HTTP/1.1&#34; 200 - 0 1050 264 264 &#34;-&#34; &#34;Ruby&#34; &#34;b993bae7-4288-9241-81a5-4cde93b2e3a6&#34; &#34;www.googleapis.com:80&#34; &#34;172.217.20.74:80&#34;
|
||
EOF
|
||
</code></pre>
|
||
<p>Note the URL path in the log, the path can be monitored and access policies can be applied based on it. To read more
|
||
about monitoring and access policies for HTTP egress traffic, check out <a href="https://archive.istio.io/v0.8/blog/2018/egress-monitoring-access-control/#logging">this blog post</a>.</p></li>
|
||
</ol>
|
||
<h3 id="cleanup-of-tls-origination-to-a-google-books-web-service">Cleanup of TLS origination to a Google Books web service</h3>
|
||
<div><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/platform/kube/bookinfo-details-v2.yaml'>Zip</a><a data-skipendnotes='true' style='display:none' href='https://raw.githubusercontent.com/istio/istio/release-1.1/samples/bookinfo/networking/virtual-service-details-v2.yaml'>Zip</a><pre><code class='language-bash' data-expandlinks='true' >$ kubectl delete serviceentry googleapis
|
||
$ kubectl delete virtualservice rewrite-port-for-googleapis
|
||
$ kubectl delete destinationrule originate-tls-for-googleapis
|
||
$ kubectl delete -f @samples/bookinfo/networking/virtual-service-details-v2.yaml@
|
||
$ kubectl delete -f @samples/bookinfo/platform/kube/bookinfo-details-v2.yaml@
|
||
</code></pre></div>
|
||
<h3 id="relation-to-istio-mutual-tls">Relation to Istio mutual TLS</h3>
|
||
<p>Note that the TLS origination in this case is unrelated to
|
||
<a href="/v1.1/docs/concepts/security/#mutual-tls-authentication">the mutual TLS</a> applied by Istio. The TLS origination for the
|
||
external services will work, whether the Istio mutual TLS is enabled or not. The <strong>mutual</strong> TLS secures
|
||
service-to-service communication <strong>inside</strong> the service mesh and provides each service with a strong identity. The
|
||
<strong>external services</strong> in this blog post were accessed using <strong>one-way TLS</strong>, the same mechanism used to secure communication between a
|
||
web browser and a web server. TLS is applied to the communication with external services to verify the identity of the
|
||
external server and to encrypt the traffic.</p>
|
||
<h2 id="conclusion">Conclusion</h2>
|
||
<p>In this blog post I demonstrated how microservices in an Istio service mesh can consume external web services by
|
||
HTTPS. By default, Istio blocks all the traffic to the hosts outside the cluster. To enable such traffic, mesh-external
|
||
service entries must be created for the service mesh. It is possible to access the external sites either by
|
||
issuing HTTPS requests, or by issuing HTTP requests with Istio performing TLS origination. When the microservices issue
|
||
HTTPS requests, the traffic is encrypted end-to-end, however Istio cannot monitor HTTP details like the URL paths of the
|
||
requests. When the microservices issue HTTP requests, Istio can monitor the HTTP details of the requests and enforce
|
||
HTTP-based access policies. However, in that case the traffic between microservice and the sidecar proxy is unencrypted.
|
||
Having part of the traffic unencrypted can be forbidden in organizations with very strict security requirements.</p></description><pubDate>Wed, 31 Jan 2018 00:00:00 +0000</pubDate><link>/v1.1/blog/2018/egress-https/</link><author>Vadim Eisenberg</author><guid isPermaLink="true">/v1.1/blog/2018/egress-https/</guid><category>traffic-management</category><category>egress</category><category>https</category></item><item><title>Mixer and the SPOF Myth</title><description>
|
||
<p>As <a href="/v1.1/docs/concepts/policies-and-telemetry/">Mixer</a> is in the request path, it is natural to question how it impacts
|
||
overall system availability and latency. A common refrain we hear when people first glance at Istio architecture diagrams is
|
||
&ldquo;Isn&rsquo;t this just introducing a single point of failure?&rdquo;</p>
|
||
<p>In this post, we’ll dig deeper and cover the design principles that underpin Mixer and the surprising fact Mixer actually
|
||
increases overall mesh availability and reduces average request latency.</p>
|
||
<p>Istio&rsquo;s use of Mixer has two main benefits in terms of overall system availability and latency:</p>
|
||
<ul>
|
||
<li><p><strong>Increased SLO</strong>. Mixer insulates proxies and services from infrastructure backend failures, enabling higher effective mesh availability. The mesh as a whole tends to experience a lower rate of failure when interacting with the infrastructure backends than if Mixer were not present.</p></li>
|
||
<li><p><strong>Reduced Latency</strong>. Through aggressive use of shared multi-level caches and sharding, Mixer reduces average observed latencies across the mesh.</p></li>
|
||
</ul>
|
||
<p>We&rsquo;ll explain this in more detail below.</p>
|
||
<h2 id="how-we-got-here">How we got here</h2>
|
||
<p>For many years at Google, we’ve been using an internal API &amp; service management system to handle the many APIs exposed by Google. This system has been fronting the world’s biggest services (Google Maps, YouTube, Gmail, etc) and sustains a peak rate of hundreds of millions of QPS. Although this system has served us well, it had problems keeping up with Google’s rapid growth, and it became clear that a new architecture was needed in order to tamp down ballooning operational costs.</p>
|
||
<p>In 2014, we started an initiative to create a replacement architecture that would scale better. The result has proven extremely successful and has been gradually deployed throughout Google, saving in the process millions of dollars a month in ops costs.</p>
|
||
<p>The older system was built around a centralized fleet of fairly heavy proxies into which all incoming traffic would flow, before being forwarded to the services where the real work was done. The newer architecture jettisons the shared proxy design and instead consists of a very lean and efficient distributed sidecar proxy sitting next to service instances, along with a shared fleet of sharded control plane intermediaries:</p>
|
||
<figure style="width:75%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:74.79295535770372%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2017/mixer-spof-myth/./mixer-spof-myth-1.svg" title="Google System Topology">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2017/mixer-spof-myth/./mixer-spof-myth-1.svg" alt="Google System Topology" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Google&#39;s API &amp; Service Management System</figcaption>
|
||
</figure>
|
||
<p>Look familiar? Of course: it’s just like Istio! Istio was conceived as a second generation of this distributed proxy architecture. We took the core lessons from this internal system, generalized many of the concepts by working with our partners, and created Istio.</p>
|
||
<h2 id="architecture-recap">Architecture recap</h2>
|
||
<p>As shown in the diagram below, Mixer sits between the mesh and the infrastructure backends that support it:</p>
|
||
<figure style="width:75%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:65.89948886170049%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2017/mixer-spof-myth/./mixer-spof-myth-2.svg" title="Istio Topology">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2017/mixer-spof-myth/./mixer-spof-myth-2.svg" alt="Istio Topology" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Istio Topology</figcaption>
|
||
</figure>
|
||
<p>The Envoy sidecar logically calls Mixer before each request to perform precondition checks, and after each request to report telemetry.
|
||
The sidecar has local caching such that a relatively large percentage of precondition checks can be performed from cache. Additionally, the
|
||
sidecar buffers outgoing telemetry such that it only actually needs to call Mixer once for every several thousands requests. Whereas precondition
|
||
checks are synchronous to request processing, telemetry reports are done asynchronously with a fire-and-forget pattern.</p>
|
||
<p>At a high level, Mixer provides:</p>
|
||
<ul>
|
||
<li><p><strong>Backend Abstraction</strong>. Mixer insulates the Istio components and services within the mesh from the implementation details of individual infrastructure backends.</p></li>
|
||
<li><p><strong>Intermediation</strong>. Mixer allows operators to have fine-grained control over all interactions between the mesh and the infrastructure backends.</p></li>
|
||
</ul>
|
||
<p>However, even beyond these purely functional aspects, Mixer has other characteristics that provide the system with additional benefits.</p>
|
||
<h2 id="mixer-slo-booster">Mixer: SLO booster</h2>
|
||
<p>Contrary to the claim that Mixer is a SPOF and can therefore lead to mesh outages, we believe it in fact improves the effective availability of a mesh. How can that be? There are three basic characteristics at play:</p>
|
||
<ul>
|
||
<li><p><strong>Statelessness</strong>. Mixer is stateless in that it doesn’t manage any persistent storage of its own.</p></li>
|
||
<li><p><strong>Hardening</strong>. Mixer proper is designed to be a highly reliable component. The design intent is to achieve &gt; 99.999% uptime for any individual Mixer instance.</p></li>
|
||
<li><p><strong>Caching and Buffering</strong>. Mixer is designed to accumulate a large amount of transient ephemeral state.</p></li>
|
||
</ul>
|
||
<p>The sidecar proxies that sit next to each service instance in the mesh must necessarily be frugal in terms of memory consumption, which constrains the possible amount of local caching and buffering. Mixer, however, lives independently and can use considerably larger caches and output buffers. Mixer thus acts as a highly-scaled and highly-available second-level cache for the sidecars.</p>
|
||
<p>Mixer’s expected availability is considerably higher than most infrastructure backends (those often have availability of perhaps 99.9%). Its local caches and buffers help mask infrastructure backend failures by being able to continue operating even when a backend has become unresponsive.</p>
|
||
<h2 id="mixer-latency-slasher">Mixer: Latency slasher</h2>
|
||
<p>As we explained above, the Istio sidecars generally have fairly effective first-level caching. They can serve the majority of their traffic from cache. Mixer provides a much greater shared pool of second-level cache, which helps Mixer contribute to a lower average per-request latency.</p>
|
||
<p>While it’s busy cutting down latency, Mixer is also inherently cutting down the number of calls your mesh makes to infrastructure backends. Depending on how you’re paying for these backends, this might end up saving you some cash by cutting down the effective QPS to the backends.</p>
|
||
<h2 id="work-ahead">Work ahead</h2>
|
||
<p>We have opportunities ahead to continue improving the system in many ways.</p>
|
||
<h3 id="configuration-canaries">Configuration canaries</h3>
|
||
<p>Mixer is highly scaled so it is generally resistant to individual instance failures. However, Mixer is still susceptible to cascading
|
||
failures in the case when a poison configuration is deployed which causes all Mixer instances to crash basically at the same time
|
||
(yeah, that would be a bad day). To prevent this from happening, configuration changes can be canaried to a small set of Mixer instances,
|
||
and then more broadly rolled out.</p>
|
||
<p>Mixer doesn’t yet do canarying of configuration changes, but we expect this to come online as part of Istio’s ongoing work on reliable
|
||
configuration distribution.</p>
|
||
<h3 id="cache-tuning">Cache tuning</h3>
|
||
<p>We have yet to fine-tune the sizes of the sidecar and Mixer caches. This work will focus on achieving the highest performance possible using the least amount of resources.</p>
|
||
<h3 id="cache-sharing">Cache sharing</h3>
|
||
<p>At the moment, each Mixer instance operates independently of all other instances. A request handled by one Mixer instance will not leverage data cached in a different instance. We will eventually experiment with a distributed cache such as memcached or Redis in order to provide a much larger mesh-wide shared cache, and further reduce the number of calls to infrastructure backends.</p>
|
||
<h3 id="sharding">Sharding</h3>
|
||
<p>In very large meshes, the load on Mixer can be great. There can be a large number of Mixer instances, each straining to keep caches primed to
|
||
satisfy incoming traffic. We expect to eventually introduce intelligent sharding such that Mixer instances become slightly specialized in
|
||
handling particular data streams in order to increase the likelihood of cache hits. In other words, sharding helps improve cache
|
||
efficiency by routing related traffic to the same Mixer instance over time, rather than randomly dispatching to
|
||
any available Mixer instance.</p>
|
||
<h2 id="conclusion">Conclusion</h2>
|
||
<p>Practical experience at Google showed that the model of a slim sidecar proxy and a large shared caching control plane intermediary hits a sweet
|
||
spot, delivering excellent perceived availability and latency. We’ve taken the lessons learned there and applied them to create more sophisticated and
|
||
effective caching, prefetching, and buffering strategies in Istio. We’ve also optimized the communication protocols to reduce overhead when a cache miss does occur.</p>
|
||
<p>Mixer is still young. As of Istio 0.3, we haven’t really done significant performance work within Mixer itself. This means when a request misses the sidecar
|
||
cache, we spend more time in Mixer to respond to requests than we should. We’re doing a lot of work to improve this in coming months to reduce the overhead
|
||
that Mixer imparts in the synchronous precondition check case.</p>
|
||
<p>We hope this post makes you appreciate the inherent benefits that Mixer brings to Istio.
|
||
Don’t hesitate to post comments or questions to <a href="https://groups.google.com/forum/#!forum/istio-policies-and-telemetry">istio-policies-and-telemetry@</a>.</p></description><pubDate>Thu, 07 Dec 2017 00:00:00 +0000</pubDate><link>/v1.1/blog/2017/mixer-spof-myth/</link><author>Martin Taillefer</author><guid isPermaLink="true">/v1.1/blog/2017/mixer-spof-myth/</guid><category>adapters</category><category>mixer</category><category>policies</category><category>telemetry</category><category>availability</category><category>latency</category></item><item><title>Mixer Adapter Model</title><description>
|
||
<p>Istio 0.2 introduced a new Mixer adapter model which is intended to increase Mixer’s flexibility to address a varied set of infrastructure backends. This post intends to put the adapter model in context and explain how it works.</p>
|
||
<h2 id="why-adapters">Why adapters?</h2>
|
||
<p>Infrastructure backends provide support functionality used to build services. They include such things as access control systems, telemetry capturing systems, quota enforcement systems, billing systems, and so forth. Services traditionally directly integrate with these backend systems, creating a hard coupling and baking-in specific semantics and usage options.</p>
|
||
<p>Mixer serves as an abstraction layer between Istio and an open-ended set of infrastructure backends. The Istio components and services that run within the mesh can interact with these backends, while not being coupled to the backends’ specific interfaces.</p>
|
||
<p>In addition to insulating application-level code from the details of infrastructure backends, Mixer provides an intermediation model that allows operators to inject and control policies between application code and backends. Operators can control which data is reported to which backend, which backend to consult for authorization, and much more.</p>
|
||
<p>Given that individual infrastructure backends each have different interfaces and operational models, Mixer needs custom
|
||
code to deal with each and we call these custom bundles of code <a href="https://github.com/istio/istio/wiki/Mixer-Compiled-In-Adapter-Dev-Guide"><em>adapters</em></a>.</p>
|
||
<p>Adapters are Go packages that are directly linked into the Mixer binary. It’s fairly simple to create custom Mixer binaries linked with specialized sets of adapters, in case the default set of adapters is not sufficient for specific use cases.</p>
|
||
<h2 id="philosophy">Philosophy</h2>
|
||
<p>Mixer is essentially an attribute processing and routing machine. The proxy sends it <a href="/v1.1/docs/concepts/policies-and-telemetry/#attributes">attributes</a> as part of doing precondition checks and telemetry reports, which it turns into a series of calls into adapters. The operator supplies configuration which describes how to map incoming attributes to inputs for the adapters.</p>
|
||
<figure style="width:60%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:42.60859894197215%">
|
||
<a data-skipendnotes="true" href="/v1.1/docs/concepts/policies-and-telemetry/machine.svg" title="Attribute Machine">
|
||
<img class="element-to-stretch" src="/v1.1/docs/concepts/policies-and-telemetry/machine.svg" alt="Attribute Machine" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Attribute Machine</figcaption>
|
||
</figure>
|
||
<p>Configuration is a complex task. In fact, evidence shows that the overwhelming majority of service outages are caused by configuration errors. To help combat this, Mixer’s configuration model enforces a number of constraints designed to avoid errors. For example, the configuration model uses strong typing to ensure that only meaningful attributes or attribute expressions are used in any given context.</p>
|
||
<h2 id="handlers-configuring-adapters">Handlers: configuring adapters</h2>
|
||
<p>Each adapter that Mixer uses requires some configuration to operate. Typically, adapters need things like the URL to their backend, credentials, caching options, and so forth. Each adapter defines the exact configuration data it needs via a <a href="https://developers.google.com/protocol-buffers/">protobuf</a> message.</p>
|
||
<p>You configure each adapter by creating <a href="/v1.1/docs/concepts/policies-and-telemetry/#handlers"><em>handlers</em></a> for them. A handler is a
|
||
configuration resource which represents a fully configured adapter ready for use. There can be any number of handlers for a single adapter, making it possible to reuse an adapter in different scenarios.</p>
|
||
<h2 id="templates-adapter-input-schema">Templates: adapter input schema</h2>
|
||
<p>Mixer is typically invoked twice for every incoming request to a mesh service, once for precondition checks and once for telemetry reporting. For every such call, Mixer invokes one or more adapters. Different adapters need different pieces of data as input in order to do their work. A logging adapter needs a log entry, a metric adapter needs a metric, an authorization adapter needs credentials, etc.
|
||
Mixer <a href="/v1.1/docs/reference/config/policy-and-telemetry/templates/"><em>templates</em></a> are used to describe the exact data that an adapter consumes at request time.</p>
|
||
<p>Each template is specified as a <a href="https://developers.google.com/protocol-buffers/">protobuf</a> message. A single template describes a bundle of data that is delivered to one or more adapters at runtime. Any given adapter can be designed to support any number of templates, the specific templates the adapter supports is determined by the adapter developer.</p>
|
||
<p><a href="/v1.1/docs/reference/config/policy-and-telemetry/templates/metric/"><code>metric</code></a> and <a href="/v1.1/docs/reference/config/policy-and-telemetry/templates/logentry/"><code>logentry</code></a> are two of the most essential templates used within Istio. They represent respectively the payload to report a single metric and a single log entry to appropriate backends.</p>
|
||
<h2 id="instances-attribute-mapping">Instances: attribute mapping</h2>
|
||
<p>You control which data is delivered to individual adapters by creating
|
||
<a href="/v1.1/docs/concepts/policies-and-telemetry/#instances"><em>instances</em></a>.
|
||
Instances control how Mixer uses the <a href="/v1.1/docs/concepts/policies-and-telemetry/#attributes">attributes</a> delivered
|
||
by the proxy into individual bundles of data that can be routed to different adapters.</p>
|
||
<p>Creating instances generally requires using <a href="/v1.1/docs/concepts/policies-and-telemetry/#attribute-expressions">attribute expressions</a>. The point of these expressions is to use any attribute or literal value in order to produce a result that can be assigned to an instance’s field.</p>
|
||
<p>Every instance field has a type, as defined in the template, every attribute has a
|
||
<a href="https://github.com/istio/api/blob/master/policy/v1beta1/value_type.proto">type</a>, and every attribute expression has a type.
|
||
You can only assign type-compatible expressions to any given instance fields. For example, you can’t assign an integer expression
|
||
to a string field. This kind of strong typing is designed to minimize the risk of creating bogus configurations.</p>
|
||
<h2 id="rules-delivering-data-to-adapters">Rules: delivering data to adapters</h2>
|
||
<p>The last piece to the puzzle is telling Mixer which instances to send to which handler and when. This is done by
|
||
creating <a href="/v1.1/docs/concepts/policies-and-telemetry/#rules"><em>rules</em></a>. Each rule identifies a specific handler and the set of
|
||
instances to send to that handler. Whenever Mixer processes an incoming call, it invokes the indicated handler and gives it the specific set of instances for processing.</p>
|
||
<p>Rules contain matching predicates. A predicate is an attribute expression which returns a true/false value. A rule only takes effect if its predicate expression returns true. Otherwise, it’s like the rule didn’t exist and the indicated handler isn’t invoked.</p>
|
||
<h2 id="future">Future</h2>
|
||
<p>We are working to improve the end to end experience of using and developing adapters. For example, several new features are planned to make templates more expressive. Additionally, the expression language is being substantially enhanced to be more powerful and well-rounded.</p>
|
||
<p>Longer term, we are evaluating ways to support adapters which aren’t directly linked into the main Mixer binary. This would simplify deployment and composition.</p>
|
||
<h2 id="conclusion">Conclusion</h2>
|
||
<p>The refreshed Mixer adapter model is designed to provide a flexible framework to support an open-ended set of infrastructure backends.</p>
|
||
<p>Handlers provide configuration data for individual adapters, templates determine exactly what kind of data different adapters want to consume at runtime, instances let operators prepare this data, rules direct the data to one or more handlers.</p>
|
||
<p>You can learn more about Mixer&rsquo;s overall architecture <a href="/v1.1/docs/concepts/policies-and-telemetry/">here</a>, and learn the specifics of templates, handlers,
|
||
and rules <a href="/v1.1/docs/reference/config/policy-and-telemetry">here</a>. You can find many examples of Mixer configuration resources in the Bookinfo sample
|
||
<a href="https://github.com/istio/istio/tree/release-1.1/samples/bookinfo">here</a>.</p></description><pubDate>Fri, 03 Nov 2017 00:00:00 +0000</pubDate><link>/v1.1/blog/2017/adapter-model/</link><author>Martin Taillefer</author><guid isPermaLink="true">/v1.1/blog/2017/adapter-model/</guid><category>adapters</category><category>mixer</category><category>policies</category><category>telemetry</category></item><item><title>Announcing Istio 0.2</title><description>
|
||
<p>We launched Istio; an open platform to connect, manage, monitor, and secure microservices, on May 24, 2017. We have been humbled by the incredible interest, and
|
||
rapid community growth of developers, operators, and partners. Our 0.1 release was focused on showing all the concepts of Istio in Kubernetes.</p>
|
||
<p>Today we are happy to announce the 0.2 release which improves stability and performance, allows for cluster wide deployment and automated injection of sidecars in Kubernetes, adds policy and authentication for TCP services, and enables expansion of the mesh to include services deployed in virtual machines. In addition, Istio can now run outside Kubernetes, leveraging Consul/Nomad or Eureka. Beyond core features, Istio is now ready for extensions to be written by third party companies and developers.</p>
|
||
<h2 id="highlights-for-the-0-2-release">Highlights for the 0.2 release</h2>
|
||
<h3 id="usability-improvements">Usability improvements</h3>
|
||
<ul>
|
||
<li><p><em>Multiple namespace support</em>: Istio now works cluster-wide, across multiple namespaces and this was one of the top requests from community from 0.1 release.</p></li>
|
||
<li><p><em>Policy and security for TCP services</em>: In addition to HTTP, we have added transparent mutual TLS authentication and policy enforcement for TCP services as well. This will allow you to secure more of your
|
||
Kubernetes deployment, and get Istio features like telemetry, policy and security.</p></li>
|
||
<li><p><em>Automated sidecar injection</em>: By leveraging the alpha <a href="https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/">initializer</a> feature provided by Kubernetes 1.7, envoy sidecars can now be automatically injected into application deployments when your cluster has the initializer enabled. This enables you to deploy microservices using <code>kubectl</code>, the exact same command that you normally use for deploying the microservices without Istio.</p></li>
|
||
<li><p><em>Extending Istio</em>: An improved Mixer design that lets vendors write Mixer adapters to implement support for their own systems, such as application
|
||
management or policy enforcement. The
|
||
<a href="https://github.com/istio/istio/wiki/Mixer-Compiled-In-Adapter-Dev-Guide">Mixer Adapter Developer&rsquo;s Guide</a> can help
|
||
you easily integrate your solution with Istio.</p></li>
|
||
<li><p><em>Bring your own CA certificates</em>: Allows users to provide their own key and certificate for Istio CA and persistent CA key/certificate Storage. Enables storing signing key/certificates in persistent storage to facilitate CA restarts.</p></li>
|
||
<li><p><em>Improved routing &amp; metrics</em>: Support for WebSocket, MongoDB and Redis protocols. You can apply resilience features like circuit breakers on traffic to third party services. In addition to Mixer’s metrics, hundreds of metrics from Envoy are now visible inside Prometheus for all traffic entering, leaving and within Istio mesh.</p></li>
|
||
</ul>
|
||
<h3 id="cross-environment-support">Cross environment support</h3>
|
||
<ul>
|
||
<li><p><em>Mesh expansion</em>: Istio mesh can now span services running outside of Kubernetes - like those running in virtual machines while enjoying benefits such as automatic mutual TLS authentication, traffic management, telemetry, and policy enforcement across the mesh.</p></li>
|
||
<li><p><em>Running outside Kubernetes</em>: We know many customers use other service registry and orchestration solutions like <a href="/v1.1/docs/setup/consul/quick-start/">Consul/Nomad</a> and Eureka. Istio Pilot can now run standalone outside Kubernetes, consuming information from these systems, and manage the Envoy fleet in VMs or containers.</p></li>
|
||
</ul>
|
||
<h2 id="get-involved-in-shaping-the-future-of-istio">Get involved in shaping the future of Istio</h2>
|
||
<p>We have a growing <a href="/v1.1/about/feature-stages/">roadmap</a> ahead of us, full of great features to implement. Our focus next release is going to be on stability, reliability, integration with third party tools and multicluster use cases.</p>
|
||
<p>To learn how to get involved and contribute to Istio&rsquo;s future, check out our <a href="https://github.com/istio/community">community</a> GitHub repository which
|
||
will introduce you to our working groups, our mailing lists, our various community meetings, our general procedures and our guidelines.</p>
|
||
<p>We want to thank our fantastic community for field testing new versions, filing bug reports, contributing code, helping out other community members, and shaping Istio by participating in countless productive discussions. This has enabled the project to accrue 3000 stars on GitHub since launch and hundreds of active community members on Istio mailing lists.</p>
|
||
<p>Thank you</p></description><pubDate>Tue, 10 Oct 2017 00:00:00 +0000</pubDate><link>/v1.1/blog/2017/0.2-announcement/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2017/0.2-announcement/</guid></item><item><title>Using Network Policy with Istio</title><description>
|
||
<p>The use of Network Policy to secure applications running on Kubernetes is a now a widely accepted industry best practice. Given that Istio also supports policy, we want to spend some time explaining how Istio policy and Kubernetes Network Policy interact and support each other to deliver your application securely.</p>
|
||
<p>Let’s start with the basics: why might you want to use both Istio and Kubernetes Network Policy? The short answer is that they are good at different things. Consider the main differences between Istio and Network Policy (we will describe &ldquo;typical” implementations, e.g. Calico, but implementation details can vary with different network providers):</p>
|
||
<table>
|
||
<thead>
|
||
<tr>
|
||
<th></th>
|
||
<th>Istio Policy</th>
|
||
<th>Network Policy</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><strong>Layer</strong></td>
|
||
<td>&ldquo;Service&rdquo; &mdash; L7</td>
|
||
<td>&ldquo;Network&rdquo; &mdash; L3-4</td>
|
||
</tr>
|
||
<tr>
|
||
<td><strong>Implementation</strong></td>
|
||
<td>User space</td>
|
||
<td>Kernel</td>
|
||
</tr>
|
||
<tr>
|
||
<td><strong>Enforcement Point</strong></td>
|
||
<td>Pod</td>
|
||
<td>Node</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<h2 id="layer">Layer</h2>
|
||
<p>Istio policy operates at the &ldquo;service” layer of your network application. This is Layer 7 (Application) from the perspective of the OSI model, but the de facto model of cloud native applications is that Layer 7 actually consists of at least two layers: a service layer and a content layer. The service layer is typically HTTP, which encapsulates the actual application data (the content layer). It is at this service layer of HTTP that the Istio’s Envoy proxy operates. In contrast, Network Policy operates at Layers 3 (Network) and 4 (Transport) in the OSI model.</p>
|
||
<p>Operating at the service layer gives the Envoy proxy a rich set of attributes to base policy decisions on, for protocols it understands, which at present includes HTTP/1.1 &amp; HTTP/2 (gRPC operates over HTTP/2). So, you can apply policy based on virtual host, URL, or other HTTP headers. In the future, Istio will support a wide range of Layer 7 protocols, as well as generic TCP and UDP transport.</p>
|
||
<p>In contrast, operating at the network layer has the advantage of being universal, since all network applications use IP. At the network layer you can apply policy regardless of the layer 7 protocol: DNS, SQL databases, real-time streaming, and a plethora of other services that do not use HTTP can be secured. Network Policy isn’t limited to a classic firewall’s tuple of IP addresses, proto, and ports. Both Istio and Network Policy are aware of rich Kubernetes labels to describe pod endpoints.</p>
|
||
<h2 id="implementation">Implementation</h2>
|
||
<p>The Istio’s proxy is based on <a href="https://envoyproxy.github.io/envoy/">Envoy</a>, which is implemented as a user space daemon in the data plane that
|
||
interacts with the network layer using standard sockets. This gives it a large amount of flexibility in processing, and allows it to be
|
||
distributed (and upgraded!) in a container.</p>
|
||
<p>Network Policy data plane is typically implemented in kernel space (e.g. using iptables, eBPF filters, or even custom kernel modules). Being in kernel space
|
||
allows them to be extremely fast, but not as flexible as the Envoy proxy.</p>
|
||
<h2 id="enforcement-point">Enforcement Point</h2>
|
||
<p>Policy enforcement using the Envoy proxy is implemented inside the pod, as a sidecar container in the same network namespace. This allows a simple deployment model. Some containers are given permission to reconfigure the networking inside their pod (CAP_NET_ADMIN). If such a service instance is compromised, or misbehaves (as in a malicious tenant) the proxy can be bypassed.</p>
|
||
<p>While this won’t let an attacker access other Istio-enabled pods, so long as they are correctly configured, it opens several attack vectors:</p>
|
||
<ul>
|
||
<li>Attacking unprotected pods</li>
|
||
<li>Attempting to deny service to protected pods by sending lots of traffic</li>
|
||
<li>Exfiltrating data collected in the pod</li>
|
||
<li>Attacking the cluster infrastructure (servers or Kubernetes services)</li>
|
||
<li>Attacking services outside the mesh, like databases, storage arrays, or legacy systems.</li>
|
||
</ul>
|
||
<p>Network Policy is typically enforced at the host node, outside the network namespace of the guest pods. This means that compromised or misbehaving pods must break into the root namespace to avoid enforcement. With the addition of egress policy due in Kubernetes 1.8, this difference makes Network Policy a key part of protecting your infrastructure from compromised workloads.</p>
|
||
<h2 id="examples">Examples</h2>
|
||
<p>Let’s walk through a few examples of what you might want to do with Kubernetes Network Policy for an Istio-enabled application. Consider the Bookinfo sample application. We’re going to cover the following use cases for Network Policy:</p>
|
||
<ul>
|
||
<li>Reduce attack surface of the application ingress</li>
|
||
<li>Enforce fine-grained isolation within the application</li>
|
||
</ul>
|
||
<h3 id="reduce-attack-surface-of-the-application-ingress">Reduce attack surface of the application ingress</h3>
|
||
<p>Our application ingress controller is the main entry-point to our application from the outside world. A quick peek at <code>istio.yaml</code> (used to install Istio) defines the Istio ingress like this:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: istio-ingress
|
||
labels:
|
||
istio: ingress
|
||
spec:
|
||
type: LoadBalancer
|
||
ports:
|
||
- port: 80
|
||
name: http
|
||
- port: 443
|
||
name: https
|
||
selector:
|
||
istio: ingress
|
||
</code></pre>
|
||
<p>The <code>istio-ingress</code> exposes ports 80 and 443. Let’s limit incoming traffic to just these two ports. Envoy has a <a href="https://www.envoyproxy.io/docs/envoy/latest/operations/admin.html#operations-admin-interface">built-in administrative interface</a>, and we don’t want a misconfigured <code>istio-ingress</code> image to accidentally expose our admin interface to the outside world. This is an example of defense in depth: a properly configured image should not expose the interface, and a properly configured Network Policy will prevent anyone from connecting to it. Either can fail or be misconfigured and we are still protected.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.k8s.io/v1
|
||
kind: NetworkPolicy
|
||
metadata:
|
||
name: istio-ingress-lockdown
|
||
namespace: default
|
||
spec:
|
||
podSelector:
|
||
matchLabels:
|
||
istio: ingress
|
||
ingress:
|
||
- ports:
|
||
- protocol: TCP
|
||
port: 80
|
||
- protocol: TCP
|
||
port: 443
|
||
</code></pre>
|
||
<h3 id="enforce-fine-grained-isolation-within-the-application">Enforce fine-grained isolation within the application</h3>
|
||
<p>Here is the service graph for the Bookinfo application.</p>
|
||
<figure style="width:80%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:59.086918235567985%">
|
||
<a data-skipendnotes="true" href="/v1.1/docs/examples/bookinfo/withistio.svg" title="Bookinfo Service Graph">
|
||
<img class="element-to-stretch" src="/v1.1/docs/examples/bookinfo/withistio.svg" alt="Bookinfo Service Graph" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Bookinfo Service Graph</figcaption>
|
||
</figure>
|
||
<p>This graph shows every connection that a correctly functioning application should be allowed to make. All other connections, say from the Istio Ingress directly to the Rating service, are not part of the application. Let’s lock out those extraneous connections so they cannot be used by an attacker. Imagine, for example, that the Ingress pod is compromised by an exploit that allows an attacker to run arbitrary code. If we only allow connections to the Product Page pods using Network Policy, the attacker has gained no more access to my application backends <em>even though they have compromised a member of the service mesh</em>.</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: networking.k8s.io/v1
|
||
kind: NetworkPolicy
|
||
metadata:
|
||
name: product-page-ingress
|
||
namespace: default
|
||
spec:
|
||
podSelector:
|
||
matchLabels:
|
||
app: productpage
|
||
ingress:
|
||
- ports:
|
||
- protocol: TCP
|
||
port: 9080
|
||
from:
|
||
- podSelector:
|
||
matchLabels:
|
||
istio: ingress
|
||
</code></pre>
|
||
<p>You can and should write a similar policy for each service to enforce which other pods are allowed to access each.</p>
|
||
<h2 id="summary">Summary</h2>
|
||
<p>Our take is that Istio and Network Policy have different strengths in applying policy. Istio is application-protocol aware and highly flexible, making it ideal for applying policy in support of operational goals, like service routing, retries, circuit-breaking, etc, and for security that operates at the application layer, such as token validation. Network Policy is universal, highly efficient, and isolated from the pods, making it ideal for applying policy in support of network security goals. Furthermore, having policy that operates at different layers of the network stack is a really good thing as it gives each layer specific context without commingling of state and allows separation of responsibility.</p>
|
||
<p>This post is based on the three part blog series by Spike Curtis, one of the Istio team members at Tigera. The full series can be found here: <a href="https://www.projectcalico.org/using-network-policy-in-concert-with-istio/">https://www.projectcalico.org/using-network-policy-in-concert-with-istio/</a></p></description><pubDate>Thu, 10 Aug 2017 00:00:00 +0000</pubDate><link>/v1.1/blog/2017/0.1-using-network-policy/</link><author>Spike Curtis</author><guid isPermaLink="true">/v1.1/blog/2017/0.1-using-network-policy/</guid></item><item><title>Canary Deployments using Istio</title><description>
|
||
<div>
|
||
<aside class="callout tip">
|
||
<div class="type"><svg class="large-icon"><use xlink:href="/v1.1/img/icons.svg#callout-tip"/></svg></div>
|
||
<div class="content">This post was updated on May 16, 2018 to use the latest version of the traffic management model.</div>
|
||
</aside>
|
||
</div>
|
||
<p>One of the benefits of the <a href="/v1.1/">Istio</a> project is that it provides the control needed to deploy canary services. The idea behind
|
||
canary deployment (or rollout) is to introduce a new version of a service by first testing it using a small percentage of user
|
||
traffic, and then if all goes well, increase, possibly gradually in increments, the percentage while simultaneously phasing out
|
||
the old version. If anything goes wrong along the way, we abort and rollback to the previous version. In its simplest form,
|
||
the traffic sent to the canary version is a randomly selected percentage of requests, but in more sophisticated schemes it
|
||
can be based on the region, user, or other properties of the request.</p>
|
||
<p>Depending on your level of expertise in this area, you may wonder why Istio&rsquo;s support for canary deployment is even needed, given that platforms like Kubernetes already provide a way to do <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment">version rollout</a> and <a href="https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments">canary deployment</a>. Problem solved, right? Well, not exactly. Although doing a rollout this way works in simple cases, it’s very limited, especially in large scale cloud environments receiving lots of (and especially varying amounts of) traffic, where autoscaling is needed.</p>
|
||
<h2 id="canary-deployment-in-kubernetes">Canary deployment in Kubernetes</h2>
|
||
<p>As an example, let&rsquo;s say we have a deployed service, <strong>helloworld</strong> version <strong>v1</strong>, for which we would like to test (or simply rollout) a new version, <strong>v2</strong>. Using Kubernetes, you can rollout a new version of the <strong>helloworld</strong> service by simply updating the image in the service’s corresponding <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployment</a> and letting the <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment">rollout</a> happen automatically. If we take particular care to ensure that there are enough <strong>v1</strong> replicas running when we start and <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#pausing-and-resuming-a-deployment">pause</a> the rollout after only one or two <strong>v2</strong> replicas have been started, we can keep the canary’s effect on the system very small. We can then observe the effect before deciding to proceed or, if necessary, <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-back-a-deployment">rollback</a>. Best of all, we can even attach a <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment">horizontal pod autoscaler</a> to the Deployment and it will keep the replica ratios consistent if, during the rollout process, it also needs to scale replicas up or down to handle traffic load.</p>
|
||
<p>Although fine for what it does, this approach is only useful when we have a properly tested version that we want to deploy, i.e., more of a blue/green, a.k.a. red/black, kind of upgrade than a &ldquo;dip your feet in the water&rdquo; kind of canary deployment. In fact, for the latter (for example, testing a canary version that may not even be ready or intended for wider exposure), the canary deployment in Kubernetes would be done using two Deployments with <a href="https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#using-labels-effectively">common pod labels</a>. In this case, we can’t use autoscaling anymore because it’s now being done by two independent autoscalers, one for each Deployment, so the replica ratios (percentages) may vary from the desired ratio, depending purely on load.</p>
|
||
<p>Whether we use one deployment or two, canary management using deployment features of container orchestration platforms like Docker, Mesos/Marathon, or Kubernetes has a fundamental problem: the use of instance scaling to manage the traffic; traffic version distribution and replica deployment are not independent in these systems. All replica pods, regardless of version, are treated the same in the <code>kube-proxy</code> round-robin pool, so the only way to manage the amount of traffic that a particular version receives is by controlling the replica ratio. Maintaining canary traffic at small percentages requires many replicas (e.g., 1% would require a minimum of 100 replicas). Even if we ignore this problem, the deployment approach is still very limited in that it only supports the simple (random percentage) canary approach. If, instead, we wanted to limit the visibility of the canary to requests based on some specific criteria, we still need another solution.</p>
|
||
<h2 id="enter-istio">Enter Istio</h2>
|
||
<p>With Istio, traffic routing and replica deployment are two completely independent functions. The number of pods implementing services are free to scale up and down based on traffic load, completely orthogonal to the control of version traffic routing. This makes managing a canary version in the presence of autoscaling a much simpler problem. Autoscalers may, in fact, respond to load variations resulting from traffic routing changes, but they are nevertheless functioning independently and no differently than when loads change for other reasons.</p>
|
||
<p>Istio’s <a href="/v1.1/docs/concepts/traffic-management/#rule-configuration">routing rules</a> also provide other important advantages; you can easily control
|
||
fine grain traffic percentages (e.g., route 1% of traffic without requiring 100 pods) and you can control traffic using other criteria (e.g., route traffic for specific users to the canary version). To illustrate, let’s look at deploying the <strong>helloworld</strong> service and see how simple the problem becomes.</p>
|
||
<p>We begin by defining the <strong>helloworld</strong> Service, just like any other Kubernetes service, something like this:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >apiVersion: v1
|
||
kind: Service
|
||
metadata:
|
||
name: helloworld
|
||
labels:
|
||
app: helloworld
|
||
spec:
|
||
selector:
|
||
app: helloworld
|
||
...
|
||
</code></pre>
|
||
<p>We then add 2 Deployments, one for each version (<strong>v1</strong> and <strong>v2</strong>), both of which include the service selector’s <code>app: helloworld</code> label:</p>
|
||
<pre><code class='language-yaml' data-expandlinks='true' >kind: Deployment
|
||
metadata:
|
||
name: helloworld-v1
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
app: helloworld
|
||
version: v1
|
||
spec:
|
||
containers:
|
||
- image: helloworld-v1
|
||
...
|
||
---
|
||
apiVersion: extensions/v1beta1
|
||
kind: Deployment
|
||
metadata:
|
||
name: helloworld-v2
|
||
spec:
|
||
replicas: 1
|
||
template:
|
||
metadata:
|
||
labels:
|
||
app: helloworld
|
||
version: v2
|
||
spec:
|
||
containers:
|
||
- image: helloworld-v2
|
||
...
|
||
</code></pre>
|
||
<p>Note that this is exactly the same way we would do a <a href="https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments">canary deployment</a> using plain Kubernetes, but in that case we would need to adjust the number of replicas of each Deployment to control the distribution of traffic. For example, to send 10% of the traffic to the canary version (<strong>v2</strong>), the replicas for <strong>v1</strong> and <strong>v2</strong> could be set to 9 and 1, respectively.</p>
|
||
<p>However, since we are going to deploy the service in an <a href="/v1.1/docs/setup/">Istio enabled</a> cluster, all we need to do is set a routing
|
||
rule to control the traffic distribution. For example if we want to send 10% of the traffic to the canary, we could use <code>kubectl</code>
|
||
to set a routing rule something like this:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: helloworld
|
||
spec:
|
||
hosts:
|
||
- helloworld
|
||
http:
|
||
- route:
|
||
- destination:
|
||
host: helloworld
|
||
subset: v1
|
||
weight: 90
|
||
- destination:
|
||
host: helloworld
|
||
subset: v2
|
||
weight: 10
|
||
---
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: DestinationRule
|
||
metadata:
|
||
name: helloworld
|
||
spec:
|
||
host: helloworld
|
||
subsets:
|
||
- name: v1
|
||
labels:
|
||
version: v1
|
||
- name: v2
|
||
labels:
|
||
version: v2
|
||
EOF
|
||
</code></pre>
|
||
<p>After setting this rule, Istio will ensure that only one tenth of the requests will be sent to the canary version, regardless of how many replicas of each version are running.</p>
|
||
<h2 id="autoscaling-the-deployments">Autoscaling the deployments</h2>
|
||
<p>Because we don’t need to maintain replica ratios anymore, we can safely add Kubernetes <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/">horizontal pod autoscalers</a> to manage the replicas for both version Deployments:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl autoscale deployment helloworld-v1 --cpu-percent=50 --min=1 --max=10
|
||
deployment &#34;helloworld-v1&#34; autoscaled
|
||
</code></pre>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl autoscale deployment helloworld-v2 --cpu-percent=50 --min=1 --max=10
|
||
deployment &#34;helloworld-v2&#34; autoscaled
|
||
</code></pre>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get hpa
|
||
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
|
||
Helloworld-v1 Deployment/helloworld-v1 50% 47% 1 10 17s
|
||
Helloworld-v2 Deployment/helloworld-v2 50% 40% 1 10 15s
|
||
</code></pre>
|
||
<p>If we now generate some load on the <strong>helloworld</strong> service, we would notice that when scaling begins, the <strong>v1</strong> autoscaler will scale up its replicas significantly higher than the <strong>v2</strong> autoscaler will for its replicas because <strong>v1</strong> pods are handling 90% of the load.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods | grep helloworld
|
||
helloworld-v1-3523621687-3q5wh 0/2 Pending 0 15m
|
||
helloworld-v1-3523621687-73642 2/2 Running 0 11m
|
||
helloworld-v1-3523621687-7hs31 2/2 Running 0 19m
|
||
helloworld-v1-3523621687-dt7n7 2/2 Running 0 50m
|
||
helloworld-v1-3523621687-gdhq9 2/2 Running 0 11m
|
||
helloworld-v1-3523621687-jxs4t 0/2 Pending 0 15m
|
||
helloworld-v1-3523621687-l8rjn 2/2 Running 0 19m
|
||
helloworld-v1-3523621687-wwddw 2/2 Running 0 15m
|
||
helloworld-v1-3523621687-xlt26 0/2 Pending 0 19m
|
||
helloworld-v2-4095161145-963wt 2/2 Running 0 50m
|
||
</code></pre>
|
||
<p>If we then change the routing rule to send 50% of the traffic to <strong>v2</strong>, we should, after a short delay, notice that the <strong>v1</strong> autoscaler will scale down the replicas of <strong>v1</strong> while the <strong>v2</strong> autoscaler will perform a corresponding scale up.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods | grep helloworld
|
||
helloworld-v1-3523621687-73642 2/2 Running 0 35m
|
||
helloworld-v1-3523621687-7hs31 2/2 Running 0 43m
|
||
helloworld-v1-3523621687-dt7n7 2/2 Running 0 1h
|
||
helloworld-v1-3523621687-gdhq9 2/2 Running 0 35m
|
||
helloworld-v1-3523621687-l8rjn 2/2 Running 0 43m
|
||
helloworld-v2-4095161145-57537 0/2 Pending 0 21m
|
||
helloworld-v2-4095161145-9322m 2/2 Running 0 21m
|
||
helloworld-v2-4095161145-963wt 2/2 Running 0 1h
|
||
helloworld-v2-4095161145-c3dpj 0/2 Pending 0 21m
|
||
helloworld-v2-4095161145-t2ccm 0/2 Pending 0 17m
|
||
helloworld-v2-4095161145-v3v9n 0/2 Pending 0 13m
|
||
</code></pre>
|
||
<p>The end result is very similar to the simple Kubernetes Deployment rollout, only now the whole process is not being orchestrated and managed in one place. Instead, we’re seeing several components doing their jobs independently, albeit in a cause and effect manner.
|
||
What&rsquo;s different, however, is that if we now stop generating load, the replicas of both versions will eventually scale down to their minimum (1), regardless of what routing rule we set.</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl get pods | grep helloworld
|
||
helloworld-v1-3523621687-dt7n7 2/2 Running 0 1h
|
||
helloworld-v2-4095161145-963wt 2/2 Running 0 1h
|
||
</code></pre>
|
||
<h2 id="focused-canary-testing">Focused canary testing</h2>
|
||
<p>As mentioned above, the Istio routing rules can be used to route traffic based on specific criteria, allowing more sophisticated canary deployment scenarios. Say, for example, instead of exposing the canary to an arbitrary percentage of users, we want to try it out on internal users, maybe even just a percentage of them. The following command could be used to send 50% of traffic from users at <em>some-company-name.com</em> to the canary version, leaving all other users unaffected:</p>
|
||
<pre><code class='language-bash' data-expandlinks='true' >$ kubectl apply -f - &lt;&lt;EOF
|
||
apiVersion: networking.istio.io/v1alpha3
|
||
kind: VirtualService
|
||
metadata:
|
||
name: helloworld
|
||
spec:
|
||
hosts:
|
||
- helloworld
|
||
http:
|
||
- match:
|
||
- headers:
|
||
cookie:
|
||
regex: &#34;^(.*?;)?(email=[^;]*@some-company-name.com)(;.*)?$&#34;
|
||
route:
|
||
- destination:
|
||
host: helloworld
|
||
subset: v1
|
||
weight: 50
|
||
- destination:
|
||
host: helloworld
|
||
subset: v2
|
||
weight: 50
|
||
- route:
|
||
- destination:
|
||
host: helloworld
|
||
subset: v1
|
||
EOF
|
||
</code></pre>
|
||
<p>As before, the autoscalers bound to the 2 version Deployments will automatically scale the replicas accordingly, but that will have no affect on the traffic distribution.</p>
|
||
<h2 id="summary">Summary</h2>
|
||
<p>In this article we’ve seen how Istio supports general scalable canary deployments, and how this differs from the basic deployment support in Kubernetes. Istio’s service mesh provides the control necessary to manage traffic distribution with complete independence from deployment scaling. This allows for a simpler, yet significantly more functional, way to do canary test and rollout.</p>
|
||
<p>Intelligent routing in support of canary deployment is just one of the many features of Istio that will make the production deployment of large-scale microservices-based applications much simpler. Check out <a href="/v1.1/">istio.io</a> for more information and to try it out.
|
||
The sample code used in this article can be found <a href="https://github.com/istio/istio/tree/release-1.1/samples/helloworld">here</a>.</p></description><pubDate>Wed, 14 Jun 2017 00:00:00 +0000</pubDate><link>/v1.1/blog/2017/0.1-canary/</link><author>Frank Budinsky</author><guid isPermaLink="true">/v1.1/blog/2017/0.1-canary/</guid><category>traffic-management</category><category>canary</category></item><item><title>Using Istio to Improve End-to-End Security</title><description>
|
||
<p>Conventional network security approaches fail to address security threats to distributed applications deployed in dynamic production environments. Today, we describe how Istio Auth enables enterprises to transform their security posture from just protecting the edge to consistently securing all inter-service communications deep within their applications. With Istio Auth, developers and operators can protect services with sensitive data against unauthorized insider access and they can achieve this without any changes to the application code!</p>
|
||
<p>Istio Auth is the security component of the broader <a href="/v1.1/">Istio platform</a>. It incorporates the learnings of securing millions of microservice
|
||
endpoints in Google’s production environment.</p>
|
||
<h2 id="background">Background</h2>
|
||
<p>Modern application architectures are increasingly based on shared services that are deployed and scaled dynamically on cloud platforms. Traditional network edge security (e.g. firewall) is too coarse-grained and allows access from unintended clients. An example of a security risk is stolen authentication tokens that can be replayed from another client. This is a major risk for companies with sensitive data that are concerned about insider threats. Other network security approaches like IP whitelists have to be statically defined, are hard to manage at scale, and are unsuitable for dynamic production environments.</p>
|
||
<p>Thus, security administrators need a tool that enables them to consistently, and by default, secure all communication between services across diverse production environments.</p>
|
||
<h2 id="solution-strong-service-identity-and-authentication">Solution: strong service identity and authentication</h2>
|
||
<p>Google has, over the years, developed architecture and technology to uniformly secure millions of microservice endpoints in its production environment against
|
||
external
|
||
attacks and insider threats. Key security principles include trusting the endpoints and not the network, strong mutual authentication based on service identity and service level authorization. Istio Auth is based on the same principles.</p>
|
||
<p>The version 0.1 release of Istio Auth runs on Kubernetes and provides the following features:</p>
|
||
<ul>
|
||
<li><p>Strong identity assertion between services</p></li>
|
||
<li><p>Access control to limit the identities that can access a service (and its data)</p></li>
|
||
<li><p>Automatic encryption of data in transit</p></li>
|
||
<li><p>Management of keys and certificates at scale</p></li>
|
||
</ul>
|
||
<p>Istio Auth is based on industry standards like mutual TLS and X.509. Furthermore, Google is actively contributing to an open, community-driven service security framework called <a href="https://spiffe.io/">SPIFFE</a>. As the <a href="https://spiffe.io/">SPIFFE</a> specifications mature, we intend for Istio Auth to become a reference implementation of the same.</p>
|
||
<p>The diagram below provides an overview of the Istio Auth service authentication architecture on Kubernetes.</p>
|
||
<figure style="width:100%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2017/0.1-auth/./istio_auth_overview.svg" title="Istio Auth Overview">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2017/0.1-auth/./istio_auth_overview.svg" alt="Istio Auth Overview" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Istio Auth Overview</figcaption>
|
||
</figure>
|
||
<p>The above diagram illustrates three key security features:</p>
|
||
<h3 id="strong-identity">Strong identity</h3>
|
||
<p>Istio Auth uses <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/">Kubernetes service accounts</a> to identify who the service runs as. The identity is used to establish trust and define service level access policies. The identity is assigned at service deployment time and encoded in the SAN (Subject Alternative Name) field of an X.509 certificate. Using a service account as the identity has the following advantages:</p>
|
||
<ul>
|
||
<li><p>Administrators can configure who has access to a Service Account by using the <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">RBAC</a> feature introduced in Kubernetes 1.6</p></li>
|
||
<li><p>Flexibility to identify a human user, a service, or a group of services</p></li>
|
||
<li><p>Stability of the service identity for dynamically placed and auto-scaled workloads</p></li>
|
||
</ul>
|
||
<h3 id="communication-security">Communication security</h3>
|
||
<p>Service-to-service communication is tunneled through high performance client side and server side <a href="https://envoyproxy.github.io/envoy/">Envoy</a> proxies. The communication between the proxies is secured using mutual TLS. The benefit of using mutual TLS is that the service identity is not expressed as a bearer token that can be stolen or replayed from another source. Istio Auth also introduces the concept of Secure Naming to protect from a server spoofing attacks - the client side proxy verifies that the authenticated server&rsquo;s service account is allowed to run the named service.</p>
|
||
<h3 id="key-management-and-distribution">Key management and distribution</h3>
|
||
<p>Istio Auth provides a per-cluster CA (Certificate Authority) and automated key &amp; certificate management. In this context, Istio Auth:</p>
|
||
<ul>
|
||
<li><p>Generates a key and certificate pair for each service account.</p></li>
|
||
<li><p>Distributes keys and certificates to the appropriate pods using <a href="https://kubernetes.io/docs/concepts/configuration/secret/">Kubernetes Secrets</a>.</p></li>
|
||
<li><p>Rotates keys and certificates periodically.</p></li>
|
||
<li><p>Revokes a specific key and certificate pair when necessary (future).</p></li>
|
||
</ul>
|
||
<p>The following diagram explains the end to end Istio Auth authentication workflow on Kubernetes:</p>
|
||
<figure style="width:100%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:56.25%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2017/0.1-auth/./istio_auth_workflow.svg" title="Istio Auth Workflow">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2017/0.1-auth/./istio_auth_workflow.svg" alt="Istio Auth Workflow" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Istio Auth Workflow</figcaption>
|
||
</figure>
|
||
<p>Istio Auth is part of the broader security story for containers. Red Hat, a partner on the development of Kubernetes, has identified <a href="https://www.redhat.com/en/resources/container-security-openshift-cloud-devops-whitepaper">10 Layers</a> of container security. Istio and Istio Auth addresses two of these layers: &ldquo;Network Isolation&rdquo; and &ldquo;API and Service Endpoint Management&rdquo;. As cluster federation evolves on Kubernetes and other platforms, our intent is for Istio to secure communications across services spanning multiple federated clusters.</p>
|
||
<h2 id="benefits-of-istio-auth">Benefits of Istio Auth</h2>
|
||
<p><strong>Defense in depth</strong>: When used in conjunction with Kubernetes (or infrastructure) network policies, users achieve higher levels of confidence, knowing that pod-to-pod or service-to-service communication is secured both at network and application layers.</p>
|
||
<p><strong>Secure by default</strong>: When used with Istio’s proxy and centralized policy engine, Istio Auth can be configured during deployment with minimal or no application change. Administrators and operators can thus ensure that service communications are secured by default and that they can enforce these policies consistently across diverse protocols and runtimes.</p>
|
||
<p><strong>Strong service authentication</strong>: Istio Auth secures service communication using mutual TLS to ensure that the service identity is not expressed as a bearer token that can be stolen or replayed from another source. This ensures that services with sensitive data can only be accessed from strongly authenticated and authorized clients.</p>
|
||
<h2 id="join-us-in-this-journey">Join us in this journey</h2>
|
||
<p>Istio Auth is the first step towards providing a full stack of capabilities to protect services with sensitive data from external attacks and insider
|
||
threats. While the initial version runs on Kubernetes, our goal is to enable Istio Auth to secure services across diverse production environments. We encourage the
|
||
community to <a href="https://github.com/istio/istio/tree/release-1.1/security">join us</a> in making robust service security easy and ubiquitous across different application
|
||
stacks and runtime platforms.</p></description><pubDate>Thu, 25 May 2017 00:00:00 +0000</pubDate><link>/v1.1/blog/2017/0.1-auth/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2017/0.1-auth/</guid></item><item><title>Introducing Istio</title><description>
|
||
<p>Google, IBM, and Lyft are proud to announce the first public release of <a href="/v1.1/">Istio</a>: an open source project that provides a uniform way to connect, secure, manage and monitor microservices. Our current release is targeted at the <a href="https://kubernetes.io/">Kubernetes</a> environment; we intend to add support for other environments such as virtual machines and Cloud Foundry in the coming months.
|
||
Istio adds traffic management to microservices and creates a basis for value-add capabilities like security, monitoring, routing, connectivity management and policy. The software is built using the battle-tested <a href="https://envoyproxy.github.io/envoy/">Envoy</a> proxy from Lyft, and gives visibility and control over traffic <em>without requiring any changes to application code</em>. Istio gives CIOs a powerful tool to enforce security, policy and compliance requirements across the enterprise.</p>
|
||
<h2 id="background">Background</h2>
|
||
<p>Writing reliable, loosely coupled, production-grade applications based on microservices can be challenging. As monolithic applications are decomposed into microservices, software teams have to worry about the challenges inherent in integrating services in distributed systems: they must account for service discovery, load balancing, fault tolerance, end-to-end monitoring, dynamic routing for feature experimentation, and perhaps most important of all, compliance and security.</p>
|
||
<p>Inconsistent attempts at solving these challenges, cobbled together from libraries, scripts and Stack Overflow snippets leads to solutions that vary wildly across languages and runtimes, have poor observability characteristics and can often end up compromising security.</p>
|
||
<p>One solution is to standardize implementations on a common RPC library like <a href="https://grpc.io">gRPC</a>, but this can be costly for organizations to adopt wholesale
|
||
and leaves out brownfield applications which may be practically impossible to change. Operators need a flexible toolkit to make their microservices secure, compliant, trackable and highly available, and developers need the ability to experiment with different features in production, or deploy canary releases, without impacting the system as a whole.</p>
|
||
<h2 id="solution-service-mesh">Solution: Service Mesh</h2>
|
||
<p>Imagine if we could transparently inject a layer of infrastructure between a service and the network that gives operators the controls they need while freeing developers from having to bake solutions to distributed system problems into their code. This uniform layer of infrastructure combined with service deployments is commonly referred to as a <strong><em>service mesh</em></strong>. Just as microservices help to decouple feature teams from each other, a service mesh helps to decouple operators from application feature development and release processes. Istio turns disparate microservices into an integrated service mesh by systemically injecting a proxy into the network paths among them.</p>
|
||
<p>Google, IBM and Lyft joined forces to create Istio from a desire to provide a reliable substrate for microservice development and maintenance, based on our common experiences building and operating massive scale microservices for internal and enterprise customers. Google and IBM have extensive experience with these large scale microservices in their own applications and with their enterprise customers in sensitive/regulated environments, while Lyft developed Envoy to address their internal operability challenges. <a href="https://eng.lyft.com/announcing-envoy-c-l7-proxy-and-communication-bus-92520b6c8191">Lyft open sourced Envoy</a> after successfully using it in production for over a year to manage more than 100 services spanning 10,000 VMs, processing 2M requests/second.</p>
|
||
<h2 id="benefits-of-istio">Benefits of Istio</h2>
|
||
<p><strong>Fleet-wide Visibility</strong>: Failures happen, and operators need tools to stay on top of the health of clusters and their graphs of microservices. Istio produces detailed monitoring data about application and network behaviors that is rendered using <a href="https://prometheus.io/">Prometheus</a> &amp; <a href="https://github.com/grafana/grafana">Grafana</a>, and can be easily extended to send metrics and logs to any collection, aggregation and querying system. Istio enables analysis of performance hotspots and diagnosis of distributed failure modes with <a href="https://github.com/openzipkin/zipkin">Zipkin</a> tracing.</p>
|
||
<figure style="width:100%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:55.425531914893625%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2017/0.1-announcement/./istio_grafana_dashboard-new.png" title="Grafana Dashboard with Response Size">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2017/0.1-announcement/./istio_grafana_dashboard-new.png" alt="Grafana Dashboard with Response Size" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Grafana Dashboard with Response Size</figcaption>
|
||
</figure>
|
||
<figure style="width:100%">
|
||
<div class="wrapper-with-intrinsic-ratio" style="padding-bottom:29.912663755458514%">
|
||
<a data-skipendnotes="true" href="/v1.1/blog/2017/0.1-announcement/./istio_zipkin_dashboard.png" title="Zipkin Dashboard">
|
||
<img class="element-to-stretch" src="/v1.1/blog/2017/0.1-announcement/./istio_zipkin_dashboard.png" alt="Zipkin Dashboard" />
|
||
</a>
|
||
</div>
|
||
<figcaption>Zipkin Dashboard</figcaption>
|
||
</figure>
|
||
<p><strong>Resiliency and efficiency</strong>: When developing microservices, operators need to assume that the network will be unreliable. Operators can use retries, load balancing, flow-control (HTTP/2), and circuit-breaking to compensate for some of the common failure modes due to an unreliable network. Istio provides a uniform approach to configuring these features, making it easier to operate a highly resilient service mesh.</p>
|
||
<p><strong>Developer productivity</strong>: Istio provides a significant boost to developer productivity by letting them focus on building service features in their language of choice, while Istio handles resiliency and networking challenges in a uniform way. Developers are freed from having to bake solutions to distributed systems problems into their code. Istio further improves productivity by providing common functionality supporting A/B testing, canarying, and fault injection.</p>
|
||
<p><strong>Policy Driven Ops</strong>: Istio empowers teams with different areas of concern to operate independently. It decouples cluster operators from the feature development cycle, allowing improvements to security, monitoring, scaling, and service topology to be rolled out <em>without</em> code changes. Operators can route a precise subset of production traffic to qualify a new service release. They can inject failures or delays into traffic to test the resilience of the service mesh, and set up rate limits to prevent services from being overloaded. Istio can also be used to enforce compliance rules, defining ACLs between services to allow only authorized services to talk to each other.</p>
|
||
<p><strong>Secure by default</strong>: It is a common fallacy of distributed computing that the network is secure. Istio enables operators to authenticate and secure all communication between services using a mutual TLS connection, without burdening the developer or the operator with cumbersome certificate management tasks. Our security framework is aligned with the emerging <a href="https://spiffe.github.io/">SPIFFE</a> specification, and is based on similar systems that have been tested extensively inside Google.</p>
|
||
<p><strong>Incremental Adoption</strong>: We designed Istio to be completely transparent to the services running in the mesh, allowing teams to incrementally adopt features of Istio over time. Adopters can start with enabling fleet-wide visibility and once they’re comfortable with Istio in their environment they can switch on other features as needed.</p>
|
||
<h2 id="join-us-in-this-journey">Join us in this journey</h2>
|
||
<p>Istio is a completely open development project. Today we are releasing version 0.1, which works in a Kubernetes cluster, and we plan to have major new
|
||
releases every 3 months, including support for additional environments. Our goal is to enable developers and operators to rollout and operate microservices
|
||
with agility, complete visibility of the underlying network, and uniform control and security in all environments. We look forward to working with the Istio
|
||
community and our partners towards these goals, following our <a href="/v1.1/about/feature-stages/">roadmap</a>.</p>
|
||
<p>Visit <a href="https://github.com/istio/istio/releases">here</a> to get the latest released bits.</p>
|
||
<p>View the <a href="/v1.1/talks/istio_talk_gluecon_2017.pdf">presentation</a> from GlueCon 2017, where Istio was unveiled.</p>
|
||
<h2 id="community">Community</h2>
|
||
<p>We are excited to see early commitment to support the project from many companies in the community:
|
||
<a href="https://blog.openshift.com/red-hat-istio-launch/">Red Hat</a> with Red Hat OpenShift and OpenShift Application Runtimes,
|
||
Pivotal with <a href="https://content.pivotal.io/blog/pivotal-and-istio-advancing-the-ecosystem-for-microservices-in-the-enterprise">Pivotal Cloud Foundry</a>,
|
||
WeaveWorks with <a href="https://www.weave.works/blog/istio-weave-cloud/">Weave Cloud</a> and Weave Net 2.0,
|
||
<a href="https://www.projectcalico.org/welcoming-istio-to-the-kubernetes-networking-community">Tigera</a> with the Project Calico Network Policy Engine
|
||
and <a href="https://www.datawire.io/istio-and-datawire-ecosystem/">Datawire</a> with the Ambassador project. We hope to see many more companies join us in
|
||
this journey.</p>
|
||
<p>To get involved, connect with us via any of these channels:</p>
|
||
<ul>
|
||
<li><p>[istio.io]() for documentation and examples.</p></li>
|
||
<li><p>The <a href="https://discuss.istio.io">Istio discussion board</a> general discussions,</p></li>
|
||
<li><p><a href="https://stackoverflow.com/questions/tagged/istio">Stack Overflow</a> for curated questions and answers</p></li>
|
||
<li><p><a href="https://github.com/istio/istio/issues">GitHub</a> for filing issues</p></li>
|
||
<li><p><a href="https://twitter.com/IstioMesh">@IstioMesh</a> on Twitter</p></li>
|
||
</ul>
|
||
<p>From everyone working on Istio, welcome aboard!</p></description><pubDate>Wed, 24 May 2017 00:00:00 +0000</pubDate><link>/v1.1/blog/2017/0.1-announcement/</link><author>The Istio Team</author><guid isPermaLink="true">/v1.1/blog/2017/0.1-announcement/</guid></item></channel></rss> |