Add documentations for SkyWalking integration and task (#11683)

* Add documentations for SkyWalking integration and task

* Add script to undeploy skywalking

* Clean up istio namespace

* Update index.md

* Address review comments

* Apply suggestions from code review

Co-authored-by: Eric Van Norman <ericvn@us.ibm.com>

Co-authored-by: Eric Van Norman <ericvn@us.ibm.com>

.spelling

@@ -684,6 +684,7 @@ NUL
 NULs
 Nurmamat
 O'Neal
+oap
 OAuth
 OAuth2
 oc

content/en/docs/ops/integrations/skywalking/index.md

@@ -0,0 +1,55 @@
+---
+title: Apache SkyWalking
+description: How to integrate with Apache SkyWalking.
+weight: 32
+keywords: [integration,skywalking,tracing]
+owner: istio/wg-environments-maintainers
+test: no
+---
+
+[Apache SkyWalking](http://skywalking.apache.org) is an application performance monitoring (APM) system, especially designed for
+microservices, cloud native and container-based architectures. SkyWalking is a one-stop solution for observability that not only
+provides distributed tracing ability like Jaeger and Zipkin, metrics ability like Prometheus and Grafana, logging ability like Kiali,
+it also extends the observability to many other scenarios, such as associating the logs with traces, collecting system events and
+associating the events with metrics, service performance profiling based on eBPF, etc.
+
+## Installation
+
+### Option 1: Quick start
+
+Istio provides a basic sample installation to quickly get SkyWalking up and running:
+
+{{< text bash >}}
+$ kubectl apply -f @samples/addons/extras/skywalking.yaml@
+{{< /text >}}
+
+This will deploy SkyWalking into your cluster. This is intended for demonstration only, and is not tuned for performance or security.
+
+Istio proxies by default don't send traces to SkyWalking. You will also need to enable the SkyWalking tracing extension provider by adding
+the following fields to your configuration:
+
+{{< text yaml >}}
+apiVersion: install.istio.io/v1alpha1
+kind: IstioOperator
+spec:
+  meshConfig:
+    extensionProviders:
+      - skywalking:
+          service: tracing.istio-system.svc.cluster.local
+          port: 11800
+        name: skywalking
+    defaultProviders:
+        tracing:
+        - "skywalking"
+{{< /text >}}
+
+### Option 2: Customizable install
+
+Consult the [SkyWalking documentation](http://skywalking.apache.org) to get started. No special changes are needed for SkyWalking to work with Istio.
+
+Once SkyWalking is installed, remember to modify the option `--set meshConfig.extensionProviders[0].skywalking.service` to point to the `skywalking-oap` deployment.
+See [`ProxyConfig.Tracing`](/docs/reference/config/istio.mesh.v1alpha1/#Tracing) for advanced configuration such as TLS settings.
+
+## Usage
+
+For more information on using SkyWalking, please refer to the [SkyWalking task](/docs/tasks/observability/distributed-tracing/skywalking/).

content/en/docs/tasks/observability/distributed-tracing/skywalking/index.md

@@ -0,0 +1,110 @@
+---
+title: Apache SkyWalking
+description: Learn how to configure the proxies to send tracing requests to Apache SkyWalking.
+weight: 10
+keywords: [telemetry,tracing,skywalking,span,port-forwarding]
+owner: istio/wg-policies-and-telemetry-maintainers
+test: yes
+---
+
+After completing this task, you will understand how to have your application participate in tracing with [Apache SkyWalking](https://skywalking.apache.org),
+regardless of the language, framework, or platform you use to build it.
+
+This task uses the [Bookinfo](/docs/examples/bookinfo/) sample as the example application.
+
+To learn how Istio handles tracing, visit the [Distributed Tracing Overview](../overview/) section.
+
+## Configure tracing
+
+If you used an `IstioOperator` CR to install Istio, add the following field to your configuration:
+
+{{< text yaml >}}
+apiVersion: install.istio.io/v1alpha1
+kind: IstioOperator
+spec:
+  meshConfig:
+    defaultProviders:
+      tracing:
+      - "skywalking"
+    enableTracing: true
+    extensionProviders:
+    - name: "skywalking"
+      skywalking:
+        service: tracing.istio-system.svc.cluster.local
+        port: 11800
+{{< /text >}}
+
+With this configuration, Istio is installed with a SkyWalking Agent as the default tracer. Trace data will be sent to a SkyWalking backend.
+
+In the default profile, the sampling rate is 1%. Increase it to 100% using the [Telemetry API](/docs/tasks/observability/telemetry/):
+
+{{< text bash >}}
+$ kubectl apply -f - <<EOF
+apiVersion: telemetry.istio.io/v1alpha1
+kind: Telemetry
+metadata:
+  name: mesh-default
+  namespace: istio-system
+spec:
+  tracing:
+  - randomSamplingPercentage: 100.00
+EOF
+{{< /text >}}
+
+## Deploy the SkyWalking Collector
+
+Follow the [SkyWalking installation](/docs/ops/integrations/skywalking/#installation) documentation to deploy SkyWalking into your cluster.
+
+## Deploy the Bookinfo Application
+
+Deploy the [Bookinfo](/docs/examples/bookinfo/#deploying-the-application) sample application.
+
+## Accessing the dashboard
+
+[Remotely Accessing Telemetry Addons](/docs/tasks/observability/gateways) details how to configure access to the Istio addons through a gateway.
+
+For testing (and temporary access), you may also use port-forwarding. Use the following, assuming you've deployed SkyWalking to the `istio-system` namespace:
+
+{{< text bash >}}
+$ istioctl dashboard skywalking
+{{< /text >}}
+
+## Generating traces using the Bookinfo sample
+
+1.  When the Bookinfo application is up and running, access `http://$GATEWAY_URL/productpage` one or more times
+    to generate trace information.
+
+    {{< boilerplate trace-generation >}}
+
+1.  From the "General Service" panel, you can see the service list.
+
+    {{< image link="./istio-service-list-skywalking.png" caption="Service List" >}}
+
+1.  Select the `Trace` tab in the main content. You can see the trace list in the left sidebar and the trace details in the right panel:
+
+    {{< image link="./istio-tracing-list-skywalking.png" caption="Trace View" >}}
+
+1.  The trace is comprised of a set of spans,
+    where each span corresponds to a Bookinfo service, invoked during the execution of a `/productpage` request, or
+    internal Istio component, for example: `istio-ingressgateway`.
+
+## Explore SkyWalking's official demo application
+
+In this tutorial, we use the [Bookinfo](/docs/examples/bookinfo/#deploying-the-application) sample application, in this sample application
+there is no SkyWalking agent installed to the services, all the traces are generated by the sidecar proxies.
+
+If you want to explore more about [SkyWalking's language agents](https://skywalking.apache.org/docs/#Agent), the SkyWalking team also provides
+[a demo application](http://github.com/apache/skywalking-showcase) that is integrated with the language agents and you can have more detailed traces as well as other language agent specific
+features such as profiling.
+
+## Cleanup
+
+1.  Remove any `istioctl` processes that may still be running using control-C or:
+
+    {{< text bash >}}
+    $ killall istioctl
+    {{< /text >}}
+
+1.  If you are not planning to explore any follow-on tasks, refer to the
+    [Bookinfo cleanup](/docs/examples/bookinfo/#cleanup) instructions
+    to shutdown the application.

content/en/docs/tasks/observability/distributed-tracing/skywalking/snips.sh

@@ -0,0 +1,59 @@
+#!/bin/bash
+# shellcheck disable=SC2034,SC2153,SC2155,SC2164
+
+# Copyright Istio Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+####################################################################################################
+# WARNING: THIS IS AN AUTO-GENERATED FILE, DO NOT EDIT. PLEASE MODIFY THE ORIGINAL MARKDOWN FILE:
+#          docs/tasks/observability/distributed-tracing/skywalking/index.md
+####################################################################################################
+source "content/en/boilerplates/snips/trace-generation.sh"
+
+! read -r -d '' snip_configure_tracing_1 <<\ENDSNIP
+apiVersion: install.istio.io/v1alpha1
+kind: IstioOperator
+spec:
+  meshConfig:
+    defaultProviders:
+      tracing:
+      - "skywalking"
+    enableTracing: true
+    extensionProviders:
+    - name: "skywalking"
+      skywalking:
+        service: tracing.istio-system.svc.cluster.local
+        port: 11800
+ENDSNIP
+
+snip_configure_tracing_2() {
+kubectl apply -f - <<EOF
+apiVersion: telemetry.istio.io/v1alpha1
+kind: Telemetry
+metadata:
+  name: mesh-default
+  namespace: istio-system
+spec:
+  tracing:
+  - randomSamplingPercentage: 100.00
+EOF
+}
+
+snip_accessing_the_dashboard_1() {
+istioctl dashboard skywalking
+}
+
+snip_cleanup_1() {
+killall istioctl
+}

content/en/docs/tasks/observability/distributed-tracing/skywalking/test.sh

@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+# shellcheck disable=SC1090,SC2154,SC2155,SC2034,SC2016
+
+# Copyright Istio Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -e
+set -u
+set -o pipefail
+
+source "tests/util/samples.sh"
+source "tests/util/addons.sh"
+
+# @setup profile=none
+echo "$snip_configure_tracing_1" | istioctl install -y -r skywalkingagent -f -
+snip_configure_tracing_2
+
+_deploy_and_wait_for_addons skywalking
+
+## Setting up application
+kubectl label namespace default istio-injection=enabled --overwrite
+
+# Install Bookinfo application
+startup_bookinfo_sample
+
+snip_accessing_the_dashboard_1 &
+
+_set_ingress_environment_variables
+GATEWAY_URL="$INGRESS_HOST:$INGRESS_PORT"
+bpsnip_trace_generation__1
+
+function access_skywalking_with_portforward() {
+  local skywalking_url='http://localhost:8080/graphql'
+  local product_svc_id="cHJvZHVjdHBhZ2UuZGVmYXVsdA==.1"
+  local now=$(date +%s)
+  local _15min_ago=$((now - 15 * 60))
+  local now=$(date +'%Y-%m-%d %H%M' -d @"$now")
+  local _15min_ago=$(date +'%Y-%m-%d %H%M' -d @"$_15min_ago")
+
+  curl -s "$skywalking_url" \
+    -X 'POST' \
+    -H 'Content-Type: application/json' \
+    --data-binary '{
+      "query": "query queryTraces($condition: TraceQueryCondition) { result: queryBasicTraces(condition: $condition) { traces { key: segmentId endpointNames duration start isError traceIds } } }",
+      "variables": {
+        "condition": {
+          "queryDuration": {
+            "start": "'"$_15min_ago"'",
+            "end": "'"$now"'",
+            "step": "MINUTE"
+          },
+          "traceState": "ALL",
+          "queryOrder": "BY_DURATION",
+          "paging": {
+            "pageNum": 1,
+            "pageSize": 20
+          },
+          "serviceId": "'"$product_svc_id"'",
+          "minTraceDuration": null,
+          "maxTraceDuration": null
+        }
+      }
+    }' | jq '.data.result.traces | length'
+}
+
+_verify_same access_skywalking_with_portforward 20
+pgrep istioctl | xargs kill
+
+# @cleanup
+cleanup_bookinfo_sample
+
+# TODO: Fix issue with using killall. Also why do we need to do this in setup and cleanup?
+pgrep istioctl | xargs kill
+_undeploy_addons skywalking
+
+kubectl delete telemetries.telemetry.istio.io -n istio-system mesh-default
+istioctl uninstall -r skywalkingagent --skip-confirmation
+kubectl label namespace default istio-injection-
+kubectl delete ns istio-system

tests/util/addons.sh

@@ -32,12 +32,16 @@ function _deploy_and_wait_for_addons() {
                  _wait_for_deployment istio-system jaeger
                  ;;
     kiali)      kubectl apply -f samples/addons/kiali.yaml || true # ignore first errors
-                 kubectl apply -f samples/addons/kiali.yaml # Need to apply twice due to a reace condition
+                 kubectl apply -f samples/addons/kiali.yaml # Need to apply twice due to a race condition
                  _wait_for_deployment istio-system kiali
                  ;;
     prometheus) kubectl apply -f samples/addons/prometheus.yaml
                  _wait_for_deployment istio-system prometheus
                  ;;
+    skywalking) kubectl apply -f samples/addons/extras/skywalking.yaml
+                 _wait_for_deployment istio-system skywalking-oap
+                 _wait_for_deployment istio-system skywalking-ui
+                 ;;
     *)           echo "unknown parameter $arg"
                  exit 1
     esac
@@ -59,6 +63,8 @@ function _undeploy_addons() {
                   ;;
     prometheus)  kubectl delete -f samples/addons/prometheus.yaml
                   ;;
+    skywalking)  kubectl delete -f samples/addons/extras/skywalking.yaml
+                  ;;
     *)            echo "unknown parameter $arg"
                   exit 1
     esac