346 lines
18 KiB
Markdown
346 lines
18 KiB
Markdown
---
|
|
title: Migrate from PodSecurityPolicy to the Built-In PodSecurity Admission Controller
|
|
reviewers:
|
|
- tallclair
|
|
- liggitt
|
|
content_type: task
|
|
min-kubernetes-server-version: v1.22
|
|
weight: 260
|
|
---
|
|
|
|
<!-- overview -->
|
|
|
|
This page describes the process of migrating from PodSecurityPolicies to the built-in PodSecurity
|
|
admission controller. This can be done effectively using a combination of dry-run and `audit` and
|
|
`warn` modes, although this becomes harder if mutating PSPs are used.
|
|
|
|
## {{% heading "prerequisites" %}}
|
|
|
|
{{% version-check %}}
|
|
|
|
If you are currently running a version of Kubernetes other than
|
|
{{< skew currentVersion >}}, you may want to switch to viewing this
|
|
page in the documentation for the version of Kubernetes that you
|
|
are actually running.
|
|
|
|
This page assumes you are already familiar with the basic [Pod Security Admission](/docs/concepts/security/pod-security-admission/)
|
|
concepts.
|
|
|
|
<!-- body -->
|
|
|
|
## Overall approach
|
|
|
|
There are multiple strategies you can take for migrating from PodSecurityPolicy to Pod Security
|
|
Admission. The following steps are one possible migration path, with a goal of minimizing both the
|
|
risks of a production outage and of a security gap.
|
|
|
|
<!-- Keep section header numbering in sync with this list. -->
|
|
0. Decide whether Pod Security Admission is the right fit for your use case.
|
|
1. Review namespace permissions
|
|
2. Simplify & standardize PodSecurityPolicies
|
|
3. Update namespaces
|
|
1. Identify an appropriate Pod Security level
|
|
2. Verify the Pod Security level
|
|
3. Enforce the Pod Security level
|
|
4. Bypass PodSecurityPolicy
|
|
4. Review namespace creation processes
|
|
5. Disable PodSecurityPolicy
|
|
|
|
## 0. Decide whether Pod Security Admission is right for you {#is-psa-right-for-you}
|
|
|
|
Pod Security Admission was designed to meet the most common security needs out of the box, and to
|
|
provide a standard set of security levels across clusters. However, it is less flexible than
|
|
PodSecurityPolicy. Notably, the following features are supported by PodSecurityPolicy but not Pod
|
|
Security Admission:
|
|
|
|
- **Setting default security constraints** - Pod Security Admission is a non-mutating admission
|
|
controller, meaning it won't modify pods before validating them. If you were relying on this
|
|
aspect of PSP, you will need to either modify your workloads to meet the Pod Security constraints,
|
|
or use a [Mutating Admission Webhook](/docs/reference/access-authn-authz/extensible-admission-controllers/)
|
|
to make those changes. See [Simplify & Standardize PodSecurityPolicies](#simplify-psps) below for more detail.
|
|
- **Fine-grained control over policy definition** - Pod Security Admission only supports
|
|
[3 standard levels](/docs/concepts/security/pod-security-standards/).
|
|
If you require more control over specific constraints, then you will need to use a
|
|
[Validating Admission Webhook](/docs/reference/access-authn-authz/extensible-admission-controllers/)
|
|
to enforce those policies.
|
|
- **Sub-namespace policy granularity** - PodSecurityPolicy lets you bind different policies to
|
|
different Service Accounts or users, even within a single namespace. This approach has many
|
|
pitfalls and is not recommended, but if you require this feature anyway you will
|
|
need to use a 3rd party webhook instead. The exception to this is if you only need to completely exempt
|
|
specific users or [RuntimeClasses](/docs/concepts/containers/runtime-class/), in which case Pod
|
|
Security Admission does expose some
|
|
[static configuration for exemptions](/docs/concepts/security/pod-security-admission/#exemptions).
|
|
|
|
Even if Pod Security Admission does not meet all of your needs it was designed to be _complementary_
|
|
to other policy enforcement mechanisms, and can provide a useful fallback running alongside other
|
|
admission webhooks.
|
|
|
|
|
|
## 1. Review namespace permissions {#review-namespace-permissions}
|
|
|
|
Pod Security Admission is controlled by [labels on
|
|
namespaces](/docs/concepts/security/pod-security-admission/#pod-security-admission-labels-for-namespaces).
|
|
This means that anyone who can update (or patch or create) a namespace can also modify the Pod
|
|
Security level for that namespace, which could be used to bypass a more restrictive policy. Before
|
|
proceeding, ensure that only trusted, privileged users have these namespace permissions. It is not
|
|
recommended to grant these powerful permissions to users that shouldn't have elevated permissions,
|
|
but if you must you will need to use an
|
|
[admission webhook](/docs/reference/access-authn-authz/extensible-admission-controllers/)
|
|
to place additional restrictions on setting Pod Security labels on Namespace objects.
|
|
|
|
## 2. Simplify & standardize PodSecurityPolicies {#simplify-psps}
|
|
|
|
In this section, you will reduce mutating PodSecurityPolicies and remove options that are outside
|
|
the scope of the Pod Security Standards. You should make the changes recommended here to an offline
|
|
copy of the original PodSecurityPolicy being modified. The cloned PSP should have a different
|
|
name that is alphabetically before the original (for example, prepend a `0` to it). Do not create the
|
|
new policies in Kubernetes yet - that will be covered in the [Rollout the updated
|
|
policies](#psp-update-rollout) section below.
|
|
|
|
### 2.a. Eliminate purely mutating fields {#eliminate-mutating-fields}
|
|
|
|
If a PodSecurityPolicy is mutating pods, then you could end up with pods that don't meet the Pod
|
|
Security level requirements when you finally turn PodSecurityPolicy off. In order to avoid this, you
|
|
should eliminate all PSP mutation prior to switching over. Unfortunately PSP does not cleanly
|
|
separate mutating & validating fields, so this is not a straightforward migration.
|
|
|
|
You can start by eliminating the fields that are purely mutating, and don't have any bearing on the
|
|
validating policy. These fields (also listed in the
|
|
[Mapping PodSecurityPolicies to Pod Security Standards](/docs/reference/access-authn-authz/psp-to-pod-security-standards/)
|
|
reference) are:
|
|
|
|
- `.spec.defaultAllowPrivilegeEscalation`
|
|
- `.spec.runtimeClass.defaultRuntimeClassName`
|
|
- `.metadata.annotations['seccomp.security.alpha.kubernetes.io/defaultProfileName']`
|
|
- `.metadata.annotations['apparmor.security.beta.kubernetes.io/defaultProfileName']`
|
|
- `.spec.defaultAddCapabilities` - Although technically a mutating & validating field, these should
|
|
be merged into `.spec.allowedCapabilities` which performs the same validation without mutation.
|
|
|
|
{{< caution >}}
|
|
Removing these could result in workloads missing required configuration, and cause problems. See
|
|
[Rollout the updated policies](#psp-update-rollout) below for advice on how to roll these changes
|
|
out safely.
|
|
{{< /caution >}}
|
|
|
|
### 2.b. Eliminate options not covered by the Pod Security Standards {#eliminate-non-standard-options}
|
|
|
|
There are several fields in PodSecurityPolicy that are not covered by the Pod Security Standards. If
|
|
you must enforce these options, you will need to supplement Pod Security Admission with an
|
|
[admission webhook](/docs/reference/access-authn-authz/extensible-admission-controllers/),
|
|
which is outside the scope of this guide.
|
|
|
|
First, you can remove the purely validating fields that the Pod Security Standards do not cover.
|
|
These fields (also listed in the
|
|
[Mapping PodSecurityPolicies to Pod Security Standards](/docs/reference/access-authn-authz/psp-to-pod-security-standards/)
|
|
reference with "no opinion") are:
|
|
|
|
- `.spec.allowedHostPaths`
|
|
- `.spec.allowedFlexVolumes`
|
|
- `.spec.allowedCSIDrivers`
|
|
- `.spec.forbiddenSysctls`
|
|
- `.spec.runtimeClass`
|
|
|
|
You can also remove the following fields, that are related to POSIX / UNIX group controls.
|
|
|
|
{{< caution >}}
|
|
If any of these use the `MustRunAs` strategy they may be mutating! Removing these could result in
|
|
workloads not setting the required groups, and cause problems. See
|
|
[Rollout the updated policies](#psp-update-rollout) below for advice on how to roll these changes
|
|
out safely.
|
|
{{< /caution >}}
|
|
|
|
- `.spec.runAsGroup`
|
|
- `.spec.supplementalGroups`
|
|
- `.spec.fsGroup`
|
|
|
|
The remaining mutating fields are required to properly support the Pod Security Standards, and will
|
|
need to be handled on a case-by-case basis later:
|
|
|
|
- `.spec.requiredDropCapabilities` - Required to drop `ALL` for the Restricted profile.
|
|
- `.spec.seLinux` - (Only mutating with the `MustRunAs` rule) required to enforce the SELinux
|
|
requirements of the Baseline & Restricted profiles.
|
|
- `.spec.runAsUser` - (Non-mutating with the `RunAsAny` rule) required to enforce `RunAsNonRoot` for
|
|
the Restricted profile.
|
|
- `.spec.allowPrivilegeEscalation` - (Only mutating if set to `false`) required for the Restricted
|
|
profile.
|
|
|
|
### 2.c. Rollout the updated PSPs {#psp-update-rollout}
|
|
|
|
Next, you can rollout the updated policies to your cluster. You should proceed with caution, as
|
|
removing the mutating options may result in workloads missing required configuration.
|
|
|
|
For each updated PodSecurityPolicy:
|
|
|
|
1. Identify pods running under the original PSP. This can be done using the `kubernetes.io/psp`
|
|
annotation. For example, using kubectl:
|
|
```sh
|
|
PSP_NAME="original" # Set the name of the PSP you're checking for
|
|
kubectl get pods --all-namespaces -o jsonpath="{range .items[?(@.metadata.annotations.kubernetes\.io\/psp=='$PSP_NAME')]}{.metadata.namespace} {.metadata.name}{'\n'}{end}"
|
|
```
|
|
2. Compare these running pods against the original pod spec to determine whether PodSecurityPolicy
|
|
has modified the pod. For pods created by a [workload resource](/docs/concepts/workloads/controllers/)
|
|
you can compare the pod with the PodTemplate in the controller resource. If any changes are
|
|
identified, the original Pod or PodTemplate should be updated with the desired configuration.
|
|
The fields to review are:
|
|
- `.metadata.annotations['container.apparmor.security.beta.kubernetes.io/*']` (replace * with each container name)
|
|
- `.spec.runtimeClassName`
|
|
- `.spec.securityContext.fsGroup`
|
|
- `.spec.securityContext.seccompProfile`
|
|
- `.spec.securityContext.seLinuxOptions`
|
|
- `.spec.securityContext.supplementalGroups`
|
|
- On containers, under `.spec.containers[*]` and `.spec.initContainers[*]`:
|
|
- `.securityContext.allowPrivilegeEscalation`
|
|
- `.securityContext.capabilities.add`
|
|
- `.securityContext.capabilities.drop`
|
|
- `.securityContext.readOnlyRootFilesystem`
|
|
- `.securityContext.runAsGroup`
|
|
- `.securityContext.runAsNonRoot`
|
|
- `.securityContext.runAsUser`
|
|
- `.securityContext.seccompProfile`
|
|
- `.securityContext.seLinuxOptions`
|
|
3. Create the new PodSecurityPolicies. If any Roles or ClusterRoles are granting `use` on all PSPs
|
|
this could cause the new PSPs to be used instead of their mutating counter-parts.
|
|
4. Update your authorization to grant access to the new PSPs. In RBAC this means updating any Roles
|
|
or ClusterRoles that grant the `use` permission on the original PSP to also grant it to the
|
|
updated PSP.
|
|
5. Verify: after some soak time, rerun the command from step 1 to see if any pods are still using
|
|
the original PSPs. Note that pods need to be recreated after the new policies have been rolled
|
|
out before they can be fully verified.
|
|
6. (optional) Once you have verified that the original PSPs are no longer in use, you can delete
|
|
them.
|
|
|
|
## 3. Update Namespaces {#update-namespaces}
|
|
|
|
The following steps will need to be performed on every namespace in the cluster. Commands referenced
|
|
in these steps use the `$NAMESPACE` variable to refer to the namespace being updated.
|
|
|
|
### 3.a. Identify an appropriate Pod Security level {#identify-appropriate-level}
|
|
|
|
Start reviewing the [Pod Security Standards](/docs/concepts/security/pod-security-standards/) and
|
|
familiarizing yourself with the 3 different levels.
|
|
|
|
There are several ways to choose a Pod Security level for your namespace:
|
|
|
|
1. **By security requirements for the namespace** - If you are familiar with the expected access
|
|
level for the namespace, you can choose an appropriate level based on those requirements, similar
|
|
to how one might approach this on a new cluster.
|
|
2. **By existing PodSecurityPolicies** - Using the
|
|
[Mapping PodSecurityPolicies to Pod Security Standards](/docs/reference/access-authn-authz/psp-to-pod-security-standards/)
|
|
reference you can map each
|
|
PSP to a Pod Security Standard level. If your PSPs aren't based on the Pod Security Standards, you
|
|
may need to decide between choosing a level that is at least as permissive as the PSP, and a
|
|
level that is at least as restrictive. You can see which PSPs are in use for pods in a given
|
|
namespace with this command:
|
|
```sh
|
|
kubectl get pods -n $NAMESPACE -o jsonpath="{.items[*].metadata.annotations.kubernetes\.io\/psp}" | tr " " "\n" | sort -u
|
|
```
|
|
3. **By existing pods** - Using the strategies under [Verify the Pod Security level](#verify-pss-level),
|
|
you can test out both the Baseline and Restricted levels to see
|
|
whether they are sufficiently permissive for existing workloads, and chose the least-privileged
|
|
valid level.
|
|
|
|
{{< caution >}}
|
|
Options 2 & 3 above are based on _existing_ pods, and may miss workloads that aren't currently
|
|
running, such as CronJobs, scale-to-zero workloads, or other workloads that haven't rolled out.
|
|
{{< /caution >}}
|
|
|
|
### 3.b. Verify the Pod Security level {#verify-pss-level}
|
|
|
|
Once you have selected a Pod Security level for the namespace (or if you're trying several), it's a
|
|
good idea to test it out first (you can skip this step if using the Privileged level). Pod Security
|
|
includes several tools to help test and safely roll out profiles.
|
|
|
|
First, you can dry-run the policy, which will evaluate pods currently running in the namespace
|
|
against the applied policy, without making the new policy take effect:
|
|
```sh
|
|
# $LEVEL is the level to dry-run, either "baseline" or "restricted".
|
|
kubectl label --dry-run=server --overwrite ns $NAMESPACE pod-security.kubernetes.io/enforce=$LEVEL
|
|
```
|
|
This command will return a warning for any _existing_ pods that are not valid under the proposed
|
|
level.
|
|
|
|
The second option is better for catching workloads that are not currently running: audit mode. When
|
|
running under audit-mode (as opposed to enforcing), pods that violate the policy level are recorded
|
|
in the audit logs, which can be reviewed later after some soak time, but are not forbidden. Warning
|
|
mode works similarly, but returns the warning to the user immediately. You can set the audit level
|
|
on a namespace with this command:
|
|
```sh
|
|
kubectl label --overwrite ns $NAMESPACE pod-security.kubernetes.io/audit=$LEVEL
|
|
```
|
|
|
|
If either of these approaches yield unexpected violations, you will need to either update the
|
|
violating workloads to meet the policy requirements, or relax the namespace Pod Security level.
|
|
|
|
### 3.c. Enforce the Pod Security level {#enforce-pod-security-level}
|
|
|
|
When you are satisfied that the chosen level can safely be enforced on the namespace, you can update
|
|
the namespace to enforce the desired level:
|
|
|
|
```sh
|
|
kubectl label --overwrite ns $NAMESPACE pod-security.kubernetes.io/enforce=$LEVEL
|
|
```
|
|
|
|
### 3.d. Bypass PodSecurityPolicy {#bypass-psp}
|
|
|
|
Finally, you can effectively bypass PodSecurityPolicy at the namespace level by binding the fully
|
|
{{< example file="policy/privileged-psp.yaml" >}}privileged PSP{{< /example >}} to all service
|
|
accounts in the namespace.
|
|
|
|
```sh
|
|
# The following cluster-scoped commands are only needed once.
|
|
kubectl apply -f privileged-psp.yaml
|
|
kubectl create clusterrole privileged-psp --verb use --resource podsecuritypolicies.policy --resource-name privileged
|
|
|
|
# Per-namespace disable
|
|
kubectl create -n $NAMESPACE rolebinding disable-psp --clusterrole privileged-psp --group system:serviceaccounts:$NAMESPACE
|
|
```
|
|
|
|
Since the privileged PSP is non-mutating, and the PSP admission controller always
|
|
prefers non-mutating PSPs, this will ensure that pods in this namespace are no longer being modified
|
|
or restricted by PodSecurityPolicy.
|
|
|
|
The advantage to disabling PodSecurityPolicy on a per-namespace basis like this is if a problem
|
|
arises you can easily roll the change back by deleting the RoleBinding. Just make sure the
|
|
pre-existing PodSecurityPolicies are still in place!
|
|
|
|
```sh
|
|
# Undo PodSecurityPolicy disablement.
|
|
kubectl delete -n $NAMESPACE rolebinding disable-psp
|
|
```
|
|
|
|
## 4. Review namespace creation processes {#review-namespace-creation-process}
|
|
|
|
Now that existing namespaces have been updated to enforce Pod Security Admission, you should ensure
|
|
that your processes and/or policies for creating new namespaces are updated to ensure that an
|
|
appropriate Pod Security profile is applied to new namespaces.
|
|
|
|
You can also statically configure the Pod Security admission controller to set a default enforce,
|
|
audit, and/or warn level for unlabeled namespaces. See
|
|
[Configure the Admission Controller](/docs/tasks/configure-pod-container/enforce-standards-admission-controller/#configure-the-admission-controller)
|
|
for more information.
|
|
|
|
## 5. Disable PodSecurityPolicy {#disable-psp}
|
|
|
|
Finally, you're ready to disable PodSecurityPolicy. To do so, you will need to modify the admission
|
|
configuration of the API server:
|
|
[How do I turn off an admission controller?](/docs/reference/access-authn-authz/admission-controllers/#how-do-i-turn-off-an-admission-controller).
|
|
|
|
To verify that the PodSecurityPolicy admission controller is no longer enabled, you can manually run
|
|
a test by impersonating a user without access to any PodSecurityPolicies (see the
|
|
[PodSecurityPolicy example](/docs/concepts/security/pod-security-policy/#example)), or by verifying in
|
|
the API server logs. At startup, the API server outputs log lines listing the loaded admission
|
|
controller plugins:
|
|
|
|
```
|
|
I0218 00:59:44.903329 13 plugins.go:158] Loaded 16 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,ExtendedResourceToleration,PersistentVolumeLabel,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
|
|
I0218 00:59:44.903350 13 plugins.go:161] Loaded 14 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,DenyServiceExternalIPs,ValidatingAdmissionWebhook,ResourceQuota.
|
|
```
|
|
|
|
You should see `PodSecurity` (in the validating admission controllers), and neither list should
|
|
contain `PodSecurityPolicy`.
|
|
|
|
Once you are certain the PSP admission controller is disabled (and after sufficient soak time to be
|
|
confident you won't need to roll back), you are free to delete your PodSecurityPolicies and any
|
|
associated Roles, ClusterRoles, RoleBindings and ClusterRoleBindings (just make sure they don't
|
|
grant any other unrelated permissions).
|