chore(kyverno): Adding kyverno pod security policies (#3216)
* chore(kyverno): Adding kyverno pod security policies Signed-off-by: shubham chaudhary <shubham@chaosnative.com> * chore(psp): updating the psp path Signed-off-by: shubham chaudhary <shubham@chaosnative.com> * Apply suggestions from code review Co-authored-by: Karthik Satchitanand <karthik.s@mayadata.io> Co-authored-by: Karthik Satchitanand <karthik.s@mayadata.io>
This commit is contained in:
Shubham Chaudhary
GitHub
parent
fa74c7bea0
commit
ed12182164
|
|
@ -3,8 +3,8 @@
|
|||
You can execute Litmus AWS experiments to target different AWS services from the EKS cluster itself, for this we need to authenticate Litmus with the AWS platform, we can do this in two different ways:
|
||||
|
||||
<ul>
|
||||
<li> **Using secrets:** It is one of the common ways to authenticate litmus with AWS irrespective of the Kubernetes cluster used for the deployment. In other words, it is Kubernetes’ native way for the authentication of litmus with the AWS platform. </li>
|
||||
<li> **IAM Integration:** It can be used when we’ve deployed Litmus on <b>EKS cluster</b>, we can associate an IAM role with a Kubernetes service account. This service account can then provide AWS permissions to the experiment pod that uses that service account. We’ll discuss more this method in the below sections.</li>
|
||||
<li> <b>Using secrets:</b> It is one of the common ways to authenticate litmus with AWS irrespective of the Kubernetes cluster used for the deployment. In other words, it is Kubernetes’ native way for the authentication of litmus with the AWS platform. </li>
|
||||
<li> <b>IAM Integration:</b> It can be used when we’ve deployed Litmus on <code>EKS cluster</code>, we can associate an IAM role with a Kubernetes service account. This service account can then provide AWS permissions to the experiment pod that uses that service account. We’ll discuss more this method in the below sections.</li>
|
||||
</ul>
|
||||
|
||||
## Why should we use IAM integration for AWS authentication?
|
||||
|
|
@ -16,7 +16,7 @@ The IAM roles for service accounts feature provides the following benefits:
|
|||
<li> <b>Credential isolation:</b> The experiment can only retrieve credentials for the IAM role that is associated with the service account to which it belongs. The experiment never has access to credentials that are intended for another experiment that belongs to another pod.</li>
|
||||
</ul>
|
||||
|
||||
# Enable service accounts to access AWS resources:
|
||||
## Enable service accounts to access AWS resources:
|
||||
|
||||
#### Step 1: Create an IAM OIDC provider for your cluster
|
||||
|
||||
|
|
@ -32,13 +32,13 @@ root@demo> aws eks describe-cluster --name <litmus-demo> --query "cluster.identi
|
|||
```
|
||||
**Output:**
|
||||
|
||||
```
|
||||
```bash
|
||||
https://oidc.eks.us-west-1.amazonaws.com/id/D054E55B6947B1A7B3F200297789662C
|
||||
```
|
||||
|
||||
Now List the IAM OIDC providers in your account
|
||||
|
||||
Command:
|
||||
<i>Command:</i>
|
||||
|
||||
```bash
|
||||
aws iam list-open-id-connect-providers | grep <EXAMPLED539D4633E53DE1B716D3041E>
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
[Kyverno policies](https://kyverno.io/policies/pod-security/) blocks configurations that don't match a policy (enforce mode) or can generate policy violations (audit mode). It scans existing configurations and reports violations in the cluster.
|
||||
Litmus recommends using the provided policy configuration to enable the execution of all supported (out-of-the-box) experiments listed in the chaoshub. Having said that, this is recommendatory in nature and left to user discretion/choice depending upon experiments desired.
|
||||
|
||||
The details listed here are expected to aid users of Kyverno. If you are using alternate means to enforce runtime security, such as native Kubernetes PSPs (pod security policies), refer this section: <>
|
||||
|
||||
## Policies in Litmus
|
||||
|
||||
Litmus recommends using the following policies:
|
||||
|
||||
1. [Add Capabilities](https://github.com/litmuschaos/chaos-charts/blob/kyverno-policies/security/kyverno-policies/allow-capabilities.yaml): It restricts add capabilities except the `NET_ADMIN` and `SYS_ADMIN` for the pods that use runtime API
|
||||
1. [Host Namespaces](https://github.com/litmuschaos/chaos-charts/blob/kyverno-policies/security/kyverno-policies/allow-host-namespaces.yaml): It validates following host namespaces for the pods that use runtime API.
|
||||
1. HostPID: It allows hostPID. It should be set to `true`.
|
||||
1. HostIPC: It restricts the host IPC. It should be set to `false`.
|
||||
1. HostNetwork: It restricts the hostNetwork. It should be set to `false`.
|
||||
1. [Host Paths](https://github.com/litmuschaos/chaos-charts/blob/kyverno-policies/security/kyverno-policies/allow-host-paths.yaml): It restricts hostPath except the `socket-path` & `container-path` host paths for the pods that uses runtime API. It allows hostPaths for service-kill experiments.
|
||||
1. [Privilege Escalation](https://github.com/litmuschaos/chaos-charts/blob/kyverno-policies/security/kyverno-policies/allow-privilege-escalation.yaml): It restricts privilege escalation except for the pods that use runtime API
|
||||
1. [Privilege Container](https://github.com/litmuschaos/chaos-charts/blob/kyverno-policies/security/kyverno-policies/allow-privileged-containers.yaml): It restricts privileged containers except for the pods that use runtime API
|
||||
1. [User Groups](https://github.com/litmuschaos/chaos-charts/blob/kyverno-policies/security/kyverno-policies/allow-user-groups.yaml): It allows users groups for all the experiment pods
|
||||
|
||||
## Install Policies
|
||||
|
||||
These Kyverno policies are based on the [Kubernetes Pod Security Standards](https://kubernetes.io/docs/concepts/security/pod-security-standards/) definitons. To apply all pod security policies (recommended) [install Kyverno](https://kyverno.io/docs/installation/) and [kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/binaries/), then run:
|
||||
|
||||
```bash
|
||||
kustomize build https://github.com/litmuschaos/chaos-charts/kyverno-policies/security/kyverno | kubectl apply -f -
|
||||
```
|
||||
|
||||
## Pod Security Policies in restricted setup
|
||||
|
||||
If setup contains restricted policies which don't allow execution of litmus experiments by default. For Example [deny-privilege-escalation](https://kyverno.io/policies/pod-security/restricted/deny-privilege-escalation/deny-privilege-escalation/) policy doesn't allow privileged escalation. It deny all the pods to use privileged escalation.
|
||||
|
||||
To allow litmus pods to use the privileged escalation. Add the litmus serviceAcccount or ClusterRole/Role inside the exclude block as :
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/litmus/master/mkdocs/docs/experiments/concepts/security/restricted-policies.yaml yaml)
|
||||
```yaml
|
||||
apiVersion: kyverno.io/v1
|
||||
kind: ClusterPolicy
|
||||
metadata:
|
||||
name: deny-privilege-escalation
|
||||
annotations:
|
||||
policies.kyverno.io/category: Pod Security Standards (Restricted)
|
||||
policies.kyverno.io/severity: medium
|
||||
policies.kyverno.io/subject: Pod
|
||||
policies.kyverno.io/description: >-
|
||||
Privilege escalation, such as via set-user-ID or set-group-ID file mode, should not be allowed.
|
||||
This policy ensures the `allowPrivilegeEscalation` fields are either undefined
|
||||
or set to `false`.
|
||||
spec:
|
||||
background: true
|
||||
validationFailureAction: enforce
|
||||
rules:
|
||||
- name: deny-privilege-escalation
|
||||
match:
|
||||
resources:
|
||||
kinds:
|
||||
- Pod
|
||||
exclude:
|
||||
clusterRoles:
|
||||
# add litmus cluster roles here
|
||||
- litmus-admin
|
||||
roles:
|
||||
# add litmus roles here
|
||||
- litmus-roles
|
||||
subjects:
|
||||
# add serviceAccount name here
|
||||
- kind: ServiceAccount
|
||||
name: pod-network-loss-sa
|
||||
validate:
|
||||
message: >-
|
||||
Privilege escalation is disallowed. The fields
|
||||
spec.containers[*].securityContext.allowPrivilegeEscalation, and
|
||||
spec.initContainers[*].securityContext.allowPrivilegeEscalation must
|
||||
be undefined or set to `false`.
|
||||
pattern:
|
||||
spec:
|
||||
=(initContainers):
|
||||
- =(securityContext):
|
||||
=(allowPrivilegeEscalation): "false"
|
||||
containers:
|
||||
- =(securityContext):
|
||||
=(allowPrivilegeEscalation): "false"
|
||||
```
|
||||
|
|
@ -15,7 +15,7 @@ opting for the default ["restricted"](https://kubernetes.io/docs/concepts/policy
|
|||
|
||||
- To run the litmus pods with operating characteristics described above, first create a custom PodSecurityPolicy that allows the same:
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/pod-security-policy/psp-litmus.yaml yaml)
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/security/pod-security-policy/psp-litmus.yaml yaml)
|
||||
```yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
apiVersion: kyverno.io/v1
|
||||
kind: ClusterPolicy
|
||||
metadata:
|
||||
name: deny-privilege-escalation
|
||||
annotations:
|
||||
policies.kyverno.io/category: Pod Security Standards (Restricted)
|
||||
policies.kyverno.io/severity: medium
|
||||
policies.kyverno.io/subject: Pod
|
||||
policies.kyverno.io/description: >-
|
||||
Privilege escalation, such as via set-user-ID or set-group-ID file mode, should not be allowed.
|
||||
This policy ensures the `allowPrivilegeEscalation` fields are either undefined
|
||||
or set to `false`.
|
||||
spec:
|
||||
background: true
|
||||
validationFailureAction: enforce
|
||||
rules:
|
||||
- name: deny-privilege-escalation
|
||||
match:
|
||||
resources:
|
||||
kinds:
|
||||
- Pod
|
||||
exclude:
|
||||
clusterRoles:
|
||||
# add litmus cluster roles here
|
||||
- litmus-admin
|
||||
roles:
|
||||
# add litmus roles here
|
||||
- litmus-roles
|
||||
subjects:
|
||||
# add serviceAccount name here
|
||||
- kind: ServiceAccount
|
||||
name: pod-network-loss-sa
|
||||
validate:
|
||||
message: >-
|
||||
Privilege escalation is disallowed. The fields
|
||||
spec.containers[*].securityContext.allowPrivilegeEscalation, and
|
||||
spec.initContainers[*].securityContext.allowPrivilegeEscalation must
|
||||
be undefined or set to `false`.
|
||||
pattern:
|
||||
spec:
|
||||
=(initContainers):
|
||||
- =(securityContext):
|
||||
=(allowPrivilegeEscalation): "false"
|
||||
containers:
|
||||
- =(securityContext):
|
||||
=(allowPrivilegeEscalation): "false"
|
||||
|
|
@ -167,8 +167,10 @@ nav:
|
|||
- K8S Probe: experiments/concepts/chaos-resources/probes/k8sProbe.md
|
||||
- Prometheus Probe: experiments/concepts/chaos-resources/probes/promProbe.md
|
||||
- Probe Chaining: experiments/concepts/chaos-resources/probes/probe-chaining.md
|
||||
- Security: experiments/concepts/psp.md
|
||||
- AWS IAM Integration: experiments/concepts/awsIamIntegration.md
|
||||
- Security:
|
||||
- Pod Security Policies: experiments/concepts/security/psp.md
|
||||
- Kyverno Policies: experiments/concepts/security/kyverno-policies.md
|
||||
- AWS IAM Integration: experiments/concepts/IAM/awsIamIntegration.md
|
||||
- Litmus FAQ:
|
||||
- Contents: experiments/faq/content.md
|
||||
- FAQ:
|
||||
|
|
|
|||
Loading…
Reference in New Issue