diff --git a/config/nav.yml b/config/nav.yml index bc82ad2eb..51cb8c5e5 100644 --- a/config/nav.yml +++ b/config/nav.yml @@ -103,6 +103,7 @@ nav: - Configuring certificate class: serving/services/certificate-class.md - Configuring custom domains: serving/services/custom-domains.md - Using a custom TLS certificate for DomainMapping: serving/services/byo-certificate.md + - Using extensions enabled by QPOptions: serving/services/using-queue-extensions.md # TODO: Add security section to docs? - Configure resource requests and limits: serving/services/configure-requests-limits-services.md - HTTPS redirection: serving/services/http-protocol.md @@ -126,6 +127,7 @@ nav: - Configuring the ingress gateway: serving/setting-up-custom-ingress-gateway.md - Configuring domain names: serving/using-a-custom-domain.md - Converting a Kubernetes Deployment to a Knative Service: serving/convert-deployment-to-knative-service.md + - Extending Queue Proxy image with QPOptions: serving/queue-extensions.md # Serving config - Serving configuration: - Configure Deployment resources: serving/configuration/deployment.md diff --git a/docs/serving/configuration/feature-flags.md b/docs/serving/configuration/feature-flags.md index 206078aa5..ab26f3860 100644 --- a/docs/serving/configuration/feature-flags.md +++ b/docs/serving/configuration/feature-flags.md @@ -295,13 +295,13 @@ When this extension is `enabled`, the server always runs this validation. When this extension is `allowed`, the server does not run this validation by default. -When this extension is `allowed`, you can run this validation for individual Services, by adding the `features.knative.dev/podspec-dryrun":"enabled"` annotation: +When this extension is `allowed`, you can run this validation for individual Services, by adding the `features.knative.dev/podspec-dryrun: enabled` annotation: ```yaml apiVersion: serving.knative.dev/v1 kind: Service metadata: - annotations: features.knative.dev/podspec-dryrun":"enabled" + annotations: features.knative.dev/podspec-dryrun: enabled ... ``` @@ -415,3 +415,24 @@ spec: command: ['sh', '-c', "service_setup.sh"] ... ``` + +### Queue Proxy Pod Info + +* **Type**: Extension +* **ConfigMap key:** `queueproxy.mount-podinfo` + +You must set this feature to either "enabled or "allowed" when using QPOptions. The flag controls whether Knative mounts the `pod-info` volume to the `queue-proxy` container. + +Mounting the `pod-info` volume allows extensions that use QPOptions to access the Service annotations, by reading the `/etc/podinfo/annnotations` file. See [Extending Queue Proxy image with QPOptions](../queue-extensions.md) for more details. + +When this feature is `enabled`, the `pod-info` volume is always mounted. This is helpful in case where all or most of the cluster Services are required to use extensions that rely on QPOptions. + +When this feature is `allowed`, the `pod-info` volume is not mounted by default. Instead, the volume is mounted only for Services that add the `features.knative.dev/queueproxy-podinfo: enabled` annotation as shown below: + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + annotations: features.knative.dev/queueproxy-podinfo: enabled +... +``` diff --git a/docs/serving/queue-extensions.md b/docs/serving/queue-extensions.md new file mode 100644 index 000000000..ac85f2a2e --- /dev/null +++ b/docs/serving/queue-extensions.md @@ -0,0 +1,40 @@ +# Extending Queue Proxy image with QPOptions + +Knative service pods include two containers: + +- The user main service container, which is named `user-container` +- The Queue Proxy - a sidecar named `queue-proxy` that serves as a reverse proxy in front of the `user-container` + +You can extend Queue Proxy to offer additional features. The QPOptions feature of Queue Proxy allows additional runtime packages to extend Queue Proxy capabilities. + +For example, the [security-guard](https://knative.dev/security-guard/README.md) repository provides an extension that uses the QPOptions feature. The [QPOption](https://knative.dev/security-guard/pkg/qpoption/README.md) package enables users to add additional security features to Queue Proxy. + +The runtime features available are determined when the Queue Proxy image is built. Queue Proxy defines an orderly manner to activate and to configure extensions. + +## Additional information + +- [Enabling Queue Proxy Pod Info](./configuration/feature-flags.md#queue-proxy-pod-info) - discussing a necessary step to enable the use of extensions. +- [Using extensions enabled by QPOptions](./services/using-queue-extensions.md) - discussing how to configure a service to use features implemented in extensions. + +## Adding extensions + +You can add extensions by replacing the `cmd/queue/main.go` file before the Queue Proxy image is built. The following example shows a `cmd/queue/main.go` file that adds the `test-gate` extension: + +```go + package main + + import "os" + + import "knative.dev/serving/pkg/queue/sharedmain" + import "knative.dev/security-guard/pkg/qpoption" + import _ "knative.dev/security-guard/pkg/test-gate" + + func main() { + qOpt := qpoption.NewQPSecurityPlugs() + defer qOpt.Shutdown() + + if sharedmain.Main(qOpt.Setup) != nil { + os.Exit(1) + } + } +``` diff --git a/docs/serving/services/using-queue-extensions.md b/docs/serving/services/using-queue-extensions.md new file mode 100644 index 000000000..004bae9df --- /dev/null +++ b/docs/serving/services/using-queue-extensions.md @@ -0,0 +1,82 @@ +# Using extensions enabled by QPOptions + +QPOptions is a Queue Proxy feature that enables extending Queue Proxy with additional Go packages. For example, the [security-guard](https://knative.dev/security-guard/README.md) repository extends Queue Proxy by adding runtime security features to protect user services. + +Once your cluster is setup with extensions enabled by QPOptions, a Service can decide which extensions it wish to use and how to configure such extensions. Activating and configuring extensions is described here. + +## Overview + +A Service can activate and configure extensions by adding `qpoption.knative.dev/*` annotations under the: `spec.template.metadata` of the Service Custom Resource Definition (CRD). + +Setting a value of: `qpoption.knative.dev/-activate: "enable"` activates the extension. + +Setting a value of: `qpoption.knative.dev/-config-: ""` adds a configuration of `key: value` to the extension. + +In addition, the Service must ensure that the Pod Info volume is mounted by adding the `features.knative.dev/queueproxy-podinfo: enabled` annotation under the: `spec.template.metadata` of the Service CRD. + +You can create a Knative Service by applying a YAML file or by using the `kn service create` CLI command. + +## Prerequisites + +Before you can use extensions enabled by QPOptions, you must: + +- Prepare your cluster: + - Make sure you are using a Queue Proxy image that was built with the extensions that you wish to use - See [Extending Queue Proxy image with QPOptions](../queue-extensions.md). + - Make sure that the cluster config-features is set with `queueproxy.mount-podinfo: allowed`. See [Enabling Queue Proxy Pod Info](../configuration/feature-flags.md#queue-proxy-pod-info) for more details. +- Meet the prerequisites in [Creating a Service](./creating-services.md) + +## Procedure + +!!! tip + + The following commands create a `helloworld-go` sample Service while activating and configuring the `test-gate` extension for this Service. You can modify these commands, including the extension(s) to be activated and the extension configuration. + +Create a sample Service: + + +=== "Apply YAML" + + 1. Create a YAML file using the following example: + + ```yaml + apiVersion: serving.knative.dev/v1 + kind: Service + metadata: + name: helloworld-go + namespace: default + spec: + template: + metadata: + annotations: + features.knative.dev/queueproxy-podinfo: enabled + qpoption.knative.dev/testgate-activate: enable + qpoption.knative.dev/testgate-config-response: CU + qpoption.knative.dev/testgate-config-sender: Joe + spec: + containers: + - image: gcr.io/knative-samples/helloworld-go + env: + - name: TARGET + value: "World" + ``` + + 1. Apply the YAML file by running the command: + + ```bash + kubectl apply -f .yaml + ``` + Where `` is the name of the file you created in the previous step. + +=== "kn CLI" + + ``` + kn service create helloworld-go \ + --image gcr.io/knative-samples/helloworld-go \ + --env TARGET=World \ + --annotation features.knative.dev/queueproxy-podinfo=enabled \ + --annotation qpoption.knative.dev/testgate-activate=enable \ + --annotation qpoption.knative.dev/testgate-config-response=Goodbye \ + --annotation qpoption.knative.dev/testgate-config-sender=Joe + ``` + +After the Service has been created, Knative propagates the annotations to the podSpec of the Service deployment. When a Service pod is created, the Queue Proxy sidecar will mount a volume that contains the pod annotations and activate the `testgate` extension. This occurs if the `testgate` extension is available in the Queue Proxy image. The `testgate` extension will then be configured with the configuration: `{ sender: "Joe", response: "CU"}`.