istio
/
istio.io
mirror of https://github.com/istio/istio.io.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

wasm: use wasm API for attributegen (#12765) 

* wasm: use wasm API for attributegen

Signed-off-by: Kuat Yessenov <kuat@google.com>

* fix links

Signed-off-by: Kuat Yessenov <kuat@google.com>

* cleanup

Signed-off-by: Kuat Yessenov <kuat@google.com>

* typo

Signed-off-by: Kuat Yessenov <kuat@google.com>

---------

Signed-off-by: Kuat Yessenov <kuat@google.com>
This commit is contained in:
Kuat 2023-03-06 08:17:46 -08:00 committed by GitHub
parent d18ce3a976
commit 2e8eff90e6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 31 additions and 595 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
content/en/blog/2021/wasm-progress/index.md
View File

@ -23,7 +23,7 @@ This was a significant milestone, since it indicates that:
## `wasm-extensions` Ecosystem Repository ## `wasm-extensions` Ecosystem Repository
As an early adopter of the Envoy Wasm runtime, the Istio Extensions and Telemetry working group gained a lot of experience in developing extensions. We built several first-class extensions, including [metadata exchange](/docs/reference/config/proxy_extensions/metadata_exchange/), [Prometheus stats](/docs/reference/config/proxy_extensions/stats/), and [attribute generation](/docs/reference/config/proxy_extensions/attributegen/). As an early adopter of the Envoy Wasm runtime, the Istio Extensions and Telemetry working group gained a lot of experience in developing extensions. We built several first-class extensions, including [metadata exchange](/docs/reference/config/proxy_extensions/metadata_exchange/), [Prometheus stats](https://archive.istio.io/v1.17/docs/reference/config/proxy_extensions/stats/), and [attribute generation](https://archive.istio.io/v1.17/docs/reference/config/proxy_extensions/attributegen/).
In order to share our learning more broadly, we created a [`wasm-extensions` repository](https://github.com/istio-ecosystem/wasm-extensions) in the `istio-ecosystem` organization. This repository serves two purposes: In order to share our learning more broadly, we created a [`wasm-extensions` repository](https://github.com/istio-ecosystem/wasm-extensions) in the `istio-ecosystem` organization. This repository serves two purposes:
* It provides canonical example extensions, covering several highly demanded features (such as [basic authentication](https://github.com/istio-ecosystem/wasm-extensions/tree/master/extensions/basic_auth)). * It provides canonical example extensions, covering several highly demanded features (such as [basic authentication](https://github.com/istio-ecosystem/wasm-extensions/tree/master/extensions/basic_auth)).

261
content/en/docs/reference/config/proxy_extensions/attributegen/index.html
View File

@ -1,261 +0,0 @@
---
WARNING: THIS IS AN AUTO-GENERATED FILE, DO NOT EDIT. PLEASE MODIFY THE ORIGINAL SOURCE IN THE 'https://github.com/istio/proxy' REPO
source_repo: https://github.com/istio/proxy
title: AttributeGen Config
description: Configuration for Attribute Generation plugin.
location: https://istio.io/docs/reference/config/proxy_extensions/attributegen.html
layout: protoc-gen-docs
generator: protoc-gen-docs
schema: istio.attributegen
weight: 20
number_of_entries: 3
---
<p>AttributeGen plugin uses <a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes">builtin
attributes</a>
as inputs and produces new attributes that can be used by downstream plugins.</p>
<p>The following is an example of a configuration that produces one attribute
named <code>istio_operationId</code> using <code>request.url_path</code> and <code>request.method</code>.</p>
<p>{{<tabset category-name="example">}}
{{<tab name="v1alpha3" category-value="v1alpha3">}}</p>
<pre><code class="language-yaml">{
&quot;attributes&quot;: [
{
&quot;output_attribute&quot;: &quot;istio_operationId&quot;,
&quot;match&quot;: [
{
&quot;value&quot;: &quot;ListBooks&quot;,
&quot;condition&quot;: &quot;request.url_path == '/books' &amp;&amp; request.method ==
'GET'&quot;
},
{
&quot;value&quot;: &quot;GetBook&quot;,
&quot;condition&quot;:
&quot;request.url_path.matches('^/shelves/[[:alnum:]]*/books/[[:alnum:]]*$')
&amp;&amp; request.method == 'GET'&quot;
},
{
&quot;value&quot;: &quot;CreateBook&quot;,
&quot;condition&quot;: &quot;request.url_path == '/books/' &amp;&amp; request.method ==
'POST'&quot;
}
]
}
]
}
</code></pre>
<p>{{</tab>}}
{{</tabset>}}</p>
<p>If the Stats plugin runs after AttributeGen, it can use <code>istio_operationId</code>
to populate a dimension on a metric.</p>
<p>The following is an example of response codes being mapped into a smaller
number of response classes as the <code>istio_responseClass</code> attribute. For
example, all response codes in 200s are mapped to <code>2xx</code>.</p>
<p>{{<tabset category-name="example">}}
{{<tab name="v1alpha3" category-value="v1alpha3">}}</p>
<pre><code class="language-yaml">{
&quot;attributes&quot;: [
{
&quot;output_attribute&quot;: &quot;istio_responseClass&quot;,
&quot;match&quot;: [
{
&quot;value&quot;: &quot;2xx&quot;,
&quot;condition&quot;: &quot;response.code &gt;= 200 &amp;&amp; response.code &lt;= 299&quot;
},
{
&quot;value&quot;: &quot;3xx&quot;,
&quot;condition&quot;: &quot;response.code &gt;= 300 &amp;&amp; response.code &lt;= 399&quot;
},
{
&quot;value&quot;: &quot;404&quot;,
&quot;condition&quot;: &quot;response.code == 404&quot;
},
{
&quot;value&quot;: &quot;429&quot;,
&quot;condition&quot;: &quot;response.code == 429&quot;
},
{
&quot;value&quot;: &quot;503&quot;,
&quot;condition&quot;: &quot;response.code == 503&quot;
},
{
&quot;value&quot;: &quot;5xx&quot;,
&quot;condition&quot;: &quot;response.code &gt;= 500 &amp;&amp; response.code &lt;= 599&quot;
},
{
&quot;value&quot;: &quot;4xx&quot;,
&quot;condition&quot;: &quot;response.code &gt;= 400 &amp;&amp; response.code &lt;= 499&quot;
}
]
}
]
}
</code></pre>
<p>{{</tab>}}
{{</tabset>}}</p>
<p>If multiple AttributeGen configurations produce the same attribute, the
result of the last configuration will be visible to downstream filters.</p>
<h2 id="PluginConfig">PluginConfig</h2>
<section>
<p>Top level configuration to generate new attributes based on attributes of the
proxied traffic.</p>
<table class="message-fields">
<thead>
<tr>
<th>Field</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody>
<tr id="PluginConfig-debug">
<td><code>debug</code></td>
<td><code>bool</code></td>
<td>
<p>The following settings should be rarely used.
Enable debug for this filter.</p>
</td>
<td>
No
</td>
</tr>
<tr id="PluginConfig-attributes">
<td><code>attributes</code></td>
<td><code><a href="#AttributeGeneration">AttributeGeneration[]</a></code></td>
<td>
<p>Multiple independent attribute generation configurations.</p>
</td>
<td>
No
</td>
</tr>
</tbody>
</table>
</section>
<h2 id="AttributeGeneration">AttributeGeneration</h2>
<section>
<p>AttributeGeneration define generation of one attribute.</p>
<table class="message-fields">
<thead>
<tr>
<th>Field</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody>
<tr id="AttributeGeneration-output_attribute">
<td><code>output_attribute</code></td>
<td><code>string</code></td>
<td>
<p>The name of the attribute that is populated on a successful match.
An attribute name SHOULD NOT contain a <code>.</code>. You may use underscores for
namespacing instead.</p>
<p>Example: <code>istio_operationId</code></p>
<p><code>istio_</code> attribute namespace is reserved by Istio.</p>
<p>AttributeGeneration may fail to evaluate when an attribute is not
available. For example, <code>response.code</code> may not be available when a request
ends abruptly. When attribute generation fails, it will not populate the
attribute.</p>
<p>If the generated attribute is used by an authz plugin, it should account
for the possibility that the attribute may be missing. Use
<code>has(attribute_name)</code> function to check for presence of an attribute before
using its value, and provide appropriate defaults. For example the
following is a safe use of <code>response.code</code></p>
<p><code>has(response.code)?response.code:200</code></p>
</td>
<td>
No
</td>
</tr>
<tr id="AttributeGeneration-match">
<td><code>match</code></td>
<td><code><a href="#Match">Match[]</a></code></td>
<td>
<p>Matches are evaluated in order until the first successful match.
The value specified by the successful match is assgined to the
output_attribute.</p>
</td>
<td>
No
</td>
</tr>
</tbody>
</table>
</section>
<h2 id="Match">Match</h2>
<section>
<p>If the condition evaluates to true then the Match returns the specified
value.</p>
<table class="message-fields">
<thead>
<tr>
<th>Field</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody>
<tr id="Match-condition">
<td><code>condition</code></td>
<td><code>string</code></td>
<td>
<p>The condition is a <a href="https://github.com/google/cel-spec/blob/master/doc/langdef.md">CEL
expression</a>
that may use <a href="https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#attributes">builtin
attributes</a>.</p>
<p>Example:</p>
<p>{{<tabset category-name="example">}}
{{<tab name="attribute-match" >}}</p>
<pre><code class="language-yaml"> {
&quot;value&quot;: &quot;GetBook&quot;,
&quot;condition&quot;:
&quot;request.url_path.matches('^/shelves/[[:alnum:]]*/books/[[:alnum:]]*$')
&amp;&amp; request.method == 'GET'&quot;
},
</code></pre>
<p>Note: CEL uses <a href="https://github.com/google/re2/wiki/Syntax">re2</a> regex
library. Use anchors <code>{^, $}</code> to ensure that the regex evaluates
efficiently.</p>
<p>Note: <code>request.url_path</code> is normalized and stripped of query params.</p>
<p>a Read only operation on books</p>
<pre><code class="language-yaml">{ &quot;value&quot;: &quot;ReadOnlyBooks&quot;,
&quot;condition&quot;: &quot;request.url_path.startsWith('/books/') &amp;&amp;
in(request.method, ['GET', 'HEAD'])&quot;}
</code></pre>
<p>{{</tab>}}
{{</tabset>}}</p>
<p>An empty condition evaluates to <code>true</code> and should be used to provide a
default value.</p>
</td>
<td>
No
</td>
</tr>
<tr id="Match-value">
<td><code>value</code></td>
<td><code>string</code></td>
<td>
<p>If condition evaluates to true, return the <code>value</code>.</p>
</td>
<td>
No
</td>
</tr>
</tbody>
</table>
</section>

271
content/en/docs/reference/config/proxy_extensions/stats/index.html
View File

@ -1,271 +0,0 @@
---
WARNING: THIS IS AN AUTO-GENERATED FILE, DO NOT EDIT. PLEASE MODIFY THE ORIGINAL SOURCE IN THE 'https://github.com/istio/proxy' REPO
source_repo: https://github.com/istio/proxy
title: Stats Config
description: Configuration for Stats Filter.
location: https://istio.io/docs/reference/config/proxy_extensions/stats.html
layout: protoc-gen-docs
generator: protoc-gen-docs
weight: 20
number_of_entries: 5
---
<h2 id="MetricConfig">MetricConfig</h2>
<section>
<p>Metric instance configuration overrides.
The metric value and the metric type are optional and permit changing the
reported value for an existing metric.
The standard metrics are optimized and reported through a &ldquo;fast-path&rdquo;.
The customizations allow full configurability, at the cost of a &ldquo;slower&rdquo;
path.</p>
<table class="message-fields">
<thead>
<tr>
<th>Field</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody>
<tr id="MetricConfig-dimensions">
<td><code>dimensions</code></td>
<td><code>map&lt;string,&nbsp;string&gt;</code></td>
<td>
<p>(Optional) Collection of tag names and tag expressions to include in the
metric. Conflicts are resolved by the tag name by overriding previously
supplied values.</p>
</td>
<td>
No
</td>
</tr>
<tr id="MetricConfig-name">
<td><code>name</code></td>
<td><code>string</code></td>
<td>
<p>(Optional) Metric name to restrict the override to a metric. If not
specified, applies to all.</p>
</td>
<td>
No
</td>
</tr>
<tr id="MetricConfig-tags_to_remove">
<td><code>tags_to_remove</code></td>
<td><code>string[]</code></td>
<td>
<p>(Optional) A list of tags to remove.</p>
</td>
<td>
No
</td>
</tr>
<tr id="MetricConfig-match">
<td><code>match</code></td>
<td><code>string</code></td>
<td>
<p>NOT IMPLEMENTED. (Optional) Conditional enabling the override.</p>
</td>
<td>
No
</td>
</tr>
<tr id="MetricConfig-drop">
<td><code>drop</code></td>
<td><code>bool</code></td>
<td>
<p>(Optional) If this is set to true, the metric(s) selected by this
configuration will not be generated or reported.</p>
</td>
<td>
No
</td>
</tr>
</tbody>
</table>
</section>
<h2 id="MetricDefinition">MetricDefinition</h2>
<section>
<table class="message-fields">
<thead>
<tr>
<th>Field</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody>
<tr id="MetricDefinition-name">
<td><code>name</code></td>
<td><code>string</code></td>
<td>
<p>Metric name.</p>
</td>
<td>
No
</td>
</tr>
<tr id="MetricDefinition-value">
<td><code>value</code></td>
<td><code>string</code></td>
<td>
<p>Metric value expression.</p>
</td>
<td>
No
</td>
</tr>
<tr id="MetricDefinition-type">
<td><code>type</code></td>
<td><code><a href="#MetricType">MetricType</a></code></td>
<td>
<p>NOT IMPLEMENTED (Optional) Metric type.</p>
</td>
<td>
No
</td>
</tr>
</tbody>
</table>
</section>
<h2 id="PluginConfig">PluginConfig</h2>
<section>
<table class="message-fields">
<thead>
<tr>
<th>Field</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody>
<tr id="PluginConfig-disable_host_header_fallback">
<td><code>disable_host_header_fallback</code></td>
<td><code>bool</code></td>
<td>
<p>Optional: Disable using host header as a fallback if destination service is
not available from the controlplane. Disable the fallback if the host
header originates outsides the mesh, like at ingress.</p>
</td>
<td>
No
</td>
</tr>
<tr id="PluginConfig-tcp_reporting_duration">
<td><code>tcp_reporting_duration</code></td>
<td><code><a href="https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#duration">Duration</a></code></td>
<td>
<p>Optional. Allows configuration of the time between calls out to for TCP
metrics reporting. The default duration is <code>15s</code>.</p>
</td>
<td>
No
</td>
</tr>
<tr id="PluginConfig-metrics">
<td><code>metrics</code></td>
<td><code><a href="#MetricConfig">MetricConfig[]</a></code></td>
<td>
<p>Metric overrides.</p>
</td>
<td>
No
</td>
</tr>
<tr id="PluginConfig-definitions">
<td><code>definitions</code></td>
<td><code><a href="#MetricDefinition">MetricDefinition[]</a></code></td>
<td>
<p>Metric definitions.</p>
</td>
<td>
No
</td>
</tr>
<tr id="PluginConfig-reporter">
<td><code>reporter</code></td>
<td><code><a href="#Reporter">Reporter</a></code></td>
<td>
<p>Proxy deployment type.</p>
</td>
<td>
No
</td>
</tr>
</tbody>
</table>
</section>
<h2 id="MetricType">MetricType</h2>
<section>
<table class="enum-values">
<thead>
<tr>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr id="MetricType-COUNTER">
<td><code>COUNTER</code></td>
<td>
</td>
</tr>
<tr id="MetricType-GAUGE">
<td><code>GAUGE</code></td>
<td>
</td>
</tr>
<tr id="MetricType-HISTOGRAM">
<td><code>HISTOGRAM</code></td>
<td>
</td>
</tr>
</tbody>
</table>
</section>
<h2 id="Reporter">Reporter</h2>
<section>
<p>Specifies the proxy deployment type.</p>
<table class="enum-values">
<thead>
<tr>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr id="Reporter-UNSPECIFIED">
<td><code>UNSPECIFIED</code></td>
<td>
<p>Default value is inferred from the listener direction, as either client or
server sidecar.</p>
</td>
</tr>
<tr id="Reporter-SERVER_GATEWAY">
<td><code>SERVER_GATEWAY</code></td>
<td>
<p>Shared server gateway, e.g. &ldquo;waypoint&rdquo;.</p>
</td>
</tr>
</tbody>
</table>
</section>

86
content/en/docs/tasks/observability/metrics/classify-metrics/index.md
View File

@ -20,7 +20,7 @@ Counting the number of review requests must account for the unbounded element
requests to get reviews. requests to get reviews.
Istio lets you create classification rules using the Istio lets you create classification rules using the
[AttributeGen plugin](/docs/reference/config/proxy_extensions/attributegen/) that groups requests AttributeGen plugin that groups requests
into a fixed number of logical operations. For example, you can create an operation named into a fixed number of logical operations. For example, you can create an operation named
`GetReviews`, which is a common way to identify operations using the `GetReviews`, which is a common way to identify operations using the
[`Open API Spec operationId`](https://swagger.io/docs/specification/paths-and-operations/). [`Open API Spec operationId`](https://swagger.io/docs/specification/paths-and-operations/).
@ -30,9 +30,6 @@ You can use the attribute as a dimension in Istio standard metrics. Similarly,
you can track metrics based on other operations like `ListReviews` and you can track metrics based on other operations like `ListReviews` and
`CreateReviews`. `CreateReviews`.
For more information, see the
[reference content](/docs/reference/config/proxy_extensions/attributegen/).
Istio uses the Envoy proxy to generate metrics and provides its configuration in Istio uses the Envoy proxy to generate metrics and provides its configuration in
the `EnvoyFilter` at the `EnvoyFilter` at
[manifests/charts/istio-control/istio-discovery/templates/telemetryv2_{{< istio_version >}}.yaml]({{<github_blob>}}/manifests/charts/istio-control/istio-discovery/templates/telemetryv2_{{< istio_version >}}.yaml). [manifests/charts/istio-control/istio-discovery/templates/telemetryv2_{{< istio_version >}}.yaml]({{<github_blob>}}/manifests/charts/istio-control/istio-discovery/templates/telemetryv2_{{< istio_version >}}.yaml).
@ -53,63 +50,38 @@ You can classify requests based on their type, for example `ListReview`,
service-specific. service-specific.
{{< text yaml >}} {{< text yaml >}}
apiVersion: networking.istio.io/v1alpha3 apiVersion: extensions.istio.io/v1alpha1
kind: EnvoyFilter kind: WasmPlugin
metadata: metadata:
name: istio-attributegen-filter name: istio-attributegen-filter
spec: spec:
workloadSelector: selector:
labels: matchLabels:
app: reviews app: reviews
configPatches: url: https://storage.googleapis.com/istio-build/proxy/attributegen-359dcd3a19f109c50e97517fe6b1e2676e870c4d.wasm
- applyTo: HTTP_FILTER imagePullPolicy: Always
match: phase: AUTHN
context: SIDECAR_INBOUND pluginConfig: {
proxy: "attributes": [
proxyVersion: '1\.9.*' {
listener: "output_attribute": "istio_operationId",
filterChain: "match": [
filter: {
name: "envoy.filters.network.http_connection_manager" "value": "ListReviews",
subFilter: "condition": "request.url_path == '/reviews' && request.method == 'GET'"
name: "istio.stats" },
patch: {
operation: INSERT_BEFORE "value": "GetReview",
value: "condition": "request.url_path.matches('^/reviews/[[:alnum:]]*$') && request.method == 'GET'"
name: istio.attributegen },
typed_config: {
"@type": type.googleapis.com/udpa.type.v1.TypedStruct "value": "CreateReview",
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm "condition": "request.url_path == '/reviews/' && request.method == 'POST'"
value: }
config: ]
configuration: }
"@type": type.googleapis.com/google.protobuf.StringValue ]
value: | }
{
"attributes": [
{
"output_attribute": "istio_operationId",
"match": [
{
"value": "ListReviews",
"condition": "request.url_path == '/reviews' && request.method == 'GET'"
},
{
"value": "GetReview",
"condition": "request.url_path.matches('^/reviews/[[:alnum:]]*$') && request.method == 'GET'"
},
{
"value": "CreateReview",
"condition": "request.url_path == '/reviews/' && request.method == 'POST'"
}
]
}
]
}
vm_config:
runtime: envoy.wasm.runtime.null
code:
local: { inline_string: "envoy.wasm.attributegen" }
{{< /text >}} {{< /text >}}
1. Apply your changes using the following command: 1. Apply your changes using the following command:

4
content/en/docs/tasks/observability/metrics/customize-metrics/index.md
View File

@ -36,8 +36,6 @@ configuration settings are also exposed as istioctl installation options, which
allow you to customize different metrics for gateways and sidecars as well as allow you to customize different metrics for gateways and sidecars as well as
for the inbound or outbound direction. for the inbound or outbound direction.
For more information, see [Stats Config reference](/docs/reference/config/proxy_extensions/stats/).
## Before you begin ## Before you begin
[Install Istio](/docs/setup/) in your cluster and deploy an application. [Install Istio](/docs/setup/) in your cluster and deploy an application.
@ -200,5 +198,3 @@ Peer metadata is available as attributes `upstream_peer` for outbound and `downs
For example, the expression for the peer `app` label to be used in an outbound configuration is For example, the expression for the peer `app` label to be used in an outbound configuration is
`upstream_peer.labels['app'].value`. `upstream_peer.labels['app'].value`.
For more information, see [configuration reference](/docs/reference/config/proxy_extensions/stats/).

2
content/en/docs/tasks/observability/metrics/tcp-metrics/index.md
View File

@ -146,7 +146,7 @@ In this task, you used Istio configuration to
automatically generate and report metrics for all traffic to a TCP service automatically generate and report metrics for all traffic to a TCP service
within the mesh. within the mesh.
TCP Metrics for all active connections are recorded every `15s` by default and this timer is configurable TCP Metrics for all active connections are recorded every `15s` by default and this timer is configurable
via [`tcpReportingDuration`](/docs/reference/config/proxy_extensions/stats/#PluginConfig). via `tcpReportingDuration`.
Metrics for a connection are also recorded at the end of the connection. Metrics for a connection are also recorded at the end of the connection.
### TCP attributes ### TCP attributes