23 KiB
catalog.cattle.io/v1
ClusterRepo
Validation Checks
Invalid Fields - Create
Users cannot create a ClusterRepo which violates the following constraints:
- Fields GitRepo and URL are mutually exclusive and so both cannot be filled at once.
Invalid Fields - Update
Users cannot update a ClusterRepo which violates the following constraints:
- Fields GitRepo and URL are mutually exclusive and so both cannot be filled at once.
cluster.cattle.io/v3
ClusterAuthToken
Validation Checks
Invalid Fields - Create
When a ClusterAuthToken is created, the following checks take place:
- If set,
lastUsedAtmust be a valid date time according to RFC3339 (e.g.2023-11-29T00:00:00Z).
Invalid Fields - Update
When a ClusterAuthToken is updated, the following checks take place:
- If set,
lastUsedAtmust be a valid date time according to RFC3339 (e.g.2023-11-29T00:00:00Z).
core/v1
Namespace
Validation Checks
Note: The kube-system namespace, unlike other namespaces, has a failPolicy of ignore on update calls.
Project annotation
Verifies that the annotation field.cattle.io/projectId value can only be updated by users with the manage-namespaces
verb on the project specified in the annotation.
PSA Label Validation
Validates that users who create or edit a PSA enforcement label on a namespace have the updatepsa verb on projects
in management.cattle.io/v3. See the upstream docs
for more information on the effect of these labels.
The following labels are considered relevant for PSA enforcement:
- pod-security.kubernetes.io/enforce
- pod-security.kubernetes.io/enforce-version
- pod-security.kubernetes.io/audit
- pod-security.kubernetes.io/audit-version
- pod-security.kubernetes.io/warn
- pod-security.kubernetes.io/warn-version
Namespace resource limit validation
Validation ensures that the limits for cpu/memory must not be less than the requests for cpu/memory.
Secret
Validation Checks
A secret cannot be deleted if its deletion request has an orphan policy, and the secret has roles or role bindings dependent on it.
Mutation Checks
On create
For all secrets of type provisioning.cattle.io/cloud-credential,
places a field.cattle.io/creatorId annotation with the name of the user as the value.
If field.cattle.io/no-creator-rbac annotation is set, field.cattle.io/creatorId does not get set.
On delete
Checks if there are any RoleBindings owned by this secret which provide access to a role granting access to this secret. If yes, the webhook redacts the role, so that it only grants a deletion permission.
management.cattle.io/v3
Cluster
Validation Checks
Annotations validation
When a cluster is created and field.cattle.io/creator-principal-name annotation is set then field.cattle.io/creatorId annotation must be set as well. The value of field.cattle.io/creator-principal-name should match the creator's user principal id.
When a cluster is updated field.cattle.io/creator-principal-name and field.cattle.io/creatorId annotations must stay the same or removed.
If field.cattle.io/no-creator-rbac annotation is set, field.cattle.io/creatorId cannot be set.
ClusterProxyConfig
Validation Checks
On create
When creating a clusterproxyconfig, we check to make sure that one does not already exist for the given cluster. Only 1 clusterproxyconfig per downstream cluster is ever permitted.
ClusterRoleTemplateBinding
Validation Checks
Escalation Prevention
Users can only create/update ClusterRoleTemplateBindings which grant permissions to RoleTemplates with rights less than or equal to those they currently possess. This is to prevent privilege escalation.
For external RoleTemplates (RoleTemplates with external set to true), if the external-rules feature flag is enabled and ExternalRules is specified in the roleTemplate in RoleTemplateName,
ExternalRules will be used for authorization. Otherwise (if the feature flag is off or ExternalRules are nil), the rules from the backing ClusterRole in the local cluster will be used.
Invalid Fields - Create
Users cannot create ClusterRoleTemplateBindings which violate the following constraints:
- Either a user subject (through
UserNameorUserPrincipalName) or a group subject (throughGroupNameorGroupPrincipalName) must be specified; both a user subject and a group subject cannot be specified ClusterNamemust:- Be provided as a non-empty value
- Match the namespace of the ClusterRoleTemplateBinding
- Refer to an existing cluster
- The roleTemplate indicated in
RoleTemplateNamemust be:- Provided as a non-empty value
- Valid (i.e. is an existing
roleTemplateobject of given name in themanagement.cattle.io/v3API group) - Not locked (i.e.
roleTemplate.Lockedmust befalse) - Associated with its appropriate context (
roleTemplate.Contextmust be equal to "cluster")
- If the label indicating ownership by a GlobalRoleBinding (
authz.management.cattle.io/grb-owner) exists, it must refer to a valid (existing and not deleting) GlobalRoleBinding
Invalid Fields - Update
Users cannot update the following fields after creation:
- RoleTemplateName
- ClusterName
- The label indicating ownership by a GlobalRoleBinding (
authz.management.cattle.io/grb-owner)
Users can update the following fields if they have not been set, but after they have been set they cannot be changed:
- UserName
- UserPrincipalName
- GroupName
- GroupPrincipalName
In addition, as in the create validation, both a user subject and a group subject cannot be specified.
Feature
Validation Checks
On update
The desired value must not change on new spec unless it's equal to the lockedValue or lockedValue is nil.
Due to the security impact of the external-rules feature flag, only users with admin permissions (* verbs on * resources in * APIGroups in all namespaces) can enable or disable this feature flag.
FleetWorkspace
Validation Checks
A FleetWorkspace cannot be created if a namespace with the same name already exists.
Mutation Checks
On create
When a FleetWorkspace is created, it will create the following resources:
Namespace. It will have the same name as theFleetWorkspace.ClusterRole. It will create the cluster role that has * permission only to the current workspace.- Two
RoleBindingsto bind the current user to fleet-admin roles andFleetWorkspaceroles.
GlobalRole
Validation Checks
Note: all checks are bypassed if the GlobalRole is being deleted, or if only the metadata fields are being updated.
Invalid Fields - Create and Update
On create or update, the following checks take place:
- The webhook validates each rule using the standard Kubernetes RBAC checks (see next section).
- Each new RoleTemplate referred to in
inheritedClusterRolesmust have a context ofclusterand not belocked. This validation is skipped for RoleTemplates ininheritedClusterRolesfor the prior version of this object.
Rules Without Verbs, Resources, API groups
Rules without verbs, resources, or apigroups are not permitted. The rules included in a GlobalRole are of the same type as the rules used by standard Kubernetes RBAC types (such as Roles from rbac.authorization.k8s.io/v1). Because of this, they inherit the same restrictions as these types, including this one.
Escalation Prevention
Escalation checks are bypassed if a user has the escalate verb on the GlobalRole that they are attempting to update or create. This can also be given through a wildcard permission (i.e. the * verb also gives escalate).
Users can only change GlobalRoles with rights less than or equal to those they currently possess. This is to prevent privilege escalation. This includes the rules in the RoleTemplates referred to in inheritedClusterRoles and the rules in inheritedFleetWorkspacePermissions.
Users can only grant rules in the NamespacedRules field with rights less than or equal to those they currently possess. This works on a per namespace basis, meaning that the user must have the permission
in the namespace specified. The Rules field apply to every namespace, which means a user can create NamespacedRules in any namespace that are equal to or less than the Rules they currently possess.
Builtin Validation
The globalroles.builtin field is immutable, and new builtIn GlobalRoles cannot be created.
If globalroles.builtin is true then all fields are immutable except metadata and newUserDefault.
If globalroles.builtin is true then the GlobalRole can not be deleted.
GlobalRoleBinding
Validation Checks
Note: all checks are bypassed if the GlobalRoleBinding is being deleted, or if only the metadata fields are being updated.
Escalation Prevention
Users can only create/update GlobalRoleBindings with rights less than or equal to those they currently possess. This is to prevent privilege escalation.
Valid Global Role Reference
GlobalRoleBindings must refer to a valid global role (i.e. an existing GlobalRole object in the management.cattle.io/v3 apiGroup).
This escalation check is bypassed if a user has the bind verb on the GlobalRole that they are trying to bind to (through creating or updating a GlobalRoleBinding to that GlobalRole). This can also be given through a wildcard permission (i.e. the * verb also gives bind).
Invalid Fields - Update
Users cannot update the following fields after creation:
userNamegroupPrincipalNameglobalRoleName
Invalid Fields - Create
GlobalRoleBindings must have either userName or groupPrincipalName, but not both.
All RoleTemplates which are referred to in the inheritedClusterRoles field must exist and not be locked.
Mutation Checks
On create
When a GlobalRoleBinding is created an owner reference is created on the binding referring to the backing GlobalRole defined by globalRoleName.
NodeDriver
Validation Checks
Note: checks only run if a node driver is being disabled or deleted
Machine Deletion Prevention
This admission webhook prevents the disabling or deletion of a NodeDriver if there are any Nodes that are under management by said driver. If there are any nodes that use the driver the request will be denied.
Project
Validation Checks
ClusterName validation
ClusterName must be equal to the namespace, and must refer to an existing management.cattle.io/v3.Cluster object. In addition, users cannot update the field after creation.
Protects system project
The system project cannot be deleted.
Quota validation
Project quotas and default limits must be consistent with one another and must be sufficient for the requirements of active namespaces.
Container default resource limit validation
Validation mimics the upstream behavior of the Kubernetes API server when it validates LimitRanges. The container default resource configuration must have properly formatted quantities for all requests and limits.
Limits for any resource must not be less than requests.
Annotations validation
When a project is created and field.cattle.io/creator-principal-name annotation is set then field.cattle.io/creatorId annotation must be set as well. The value of field.cattle.io/creator-principal-name should match the creator's user principal id.
When a project is updated field.cattle.io/creator-principal-name and field.cattle.io/creatorId annotations must stay the same or removed.
If field.cattle.io/no-creator-rbac annotation is set, field.cattle.io/creatorId cannot be set.
Mutations
On create
Adds the authz.management.cattle.io/creator-role-bindings annotation.
ProjectRoleTemplateBinding
Validation Checks
Escalation Prevention
Users can only create/update ProjectRoleTemplateBindings with rights less than or equal to those they currently possess.
This is to prevent privilege escalation.
For external RoleTemplates (RoleTemplates with external set to true), if the external-rules feature flag is enabled and ExternalRules is specified in the roleTemplate in RoleTemplateName,
ExternalRules will be used for authorization. Otherwise, if ExternalRules are nil when the feature flag is on, the rules from the backing ClusterRole in the local cluster will be used.
Invalid Fields - Create
Users cannot create ProjectRoleTemplateBindings that violate the following constraints:
- The
ProjectNamefield must be:- Provided as a non-empty value
- Specified using the format of
clusterName:projectName;clusterNameis themetadata.nameof a cluster, andprojectNameis themetadata.nameof a project - The
projectNamepart of the field must match the namespace of the ProjectRoleTemplateBinding - Refer to a valid project and cluster (both must exist and project.Spec.ClusterName must equal the cluster)
- Either a user subject (through
UserNameorUserPrincipalName), or a group subject (throughGroupNameorGroupPrincipalName), or a service account subject (throughServiceAccount) must be specified. Exactly one subject type of the three must be provided. - The roleTemplate indicated in
RoleTemplateNamemust be:- Provided as a non-empty value
- Valid (there must exist a
roleTemplateobject of given name in themanagement.cattle.io/v3API group) - Not locked (
roleTemplate.Lockedmust befalse) - Associated with its appropriate context (
roleTemplate.Contextmust be equal to "project")
Invalid Fields - Update
Users cannot update the following fields after creation:
- RoleTemplateName
- ProjectName
- ServiceAccount
Users can update the following fields if they had not been set. But after getting initial values, the fields cannot be changed:
- UserName
- UserPrincipalName
- GroupName
- GroupPrincipalName
In addition, as in the create validation, both a user subject and a group subject cannot be specified.
RoleTemplate
Validation Checks
Note: all checks are bypassed if the RoleTemplate is being deleted
Circular Reference
Circular references to a RoleTemplate (a inherits b, b inherits a) are not allowed. More specifically, if "roleTemplate1" is included in the roleTemplateNames of "roleTemplate2", then "roleTemplate2" must not be included in the roleTemplateNames of "roleTemplate1". This check prevents the creation of roles whose end-state cannot be resolved.
Rules Without Verbs, Resources, API groups
Rules without verbs, resources, or apigroups are not permitted. The rules and externalRules included in a RoleTemplate are of the same type as the rules used by standard Kubernetes RBAC types (such as Roles from rbac.authorization.k8s.io/v1). Because of this, they inherit the same restrictions as these types, including this one.
Escalation Prevention
Users can only change RoleTemplates with rights less than or equal to those they currently possess. This prevents privilege escalation.
Users can't create external RoleTemplates (or update existing RoleTemplates) with ExternalRules without having the escalate verb on that RoleTemplate.
Context Validation
The roletemplates.context field must be one of the following values ["cluster", "project", ""].
If the roletemplates.administrative is set to true the context must equal "cluster".
If the roletemplate.ProjectCreatorDefault is true, context must equal "project"
Builtin Validation
The roletemplates.builtin field is immutable, and new builtIn RoleTemplates cannot be created.
If roletemplates.builtin is true then all fields are immutable except:
metadataclusterCreatorDefaultprojectCreatorDefaultlocked
Deletion check
RoleTemplate can not be deleted if they are referenced by other RoleTemplates via roletemplates.roleTemplateNames or by GlobalRoles via globalRoles.inheritedClusterRoles
Setting
Validation Checks
Create and Update
When settings are created or updated, the following common checks take place:
- If set,
disable-inactive-user-aftermust be zero or a positive duration (e.g.240h). - If set,
delete-inactive-user-aftermust be zero or a positive duration and can't be less than336h(e.g.336h). - If set,
user-last-login-defaultmust be a date time according to RFC3339 (e.g.2023-11-29T00:00:00Z). - If set,
user-retention-cronmust be a valid standard cron expression (e.g.0 0 * * 0). - The
auth-user-session-ttl-minutesmust be a positive integer and can't be greater thandisable-inactive-user-afterordelete-inactive-user-afterif those values are set.
Update
When settings are updated, the following additional checks take place:
- If
agent-tls-modehasdefaultorvalueupdated fromsystem-storetostrict, then all non-local clusters must have a status conditionAgentTlsStrictCheckset toTrue, unless the new setting has an overriding annotationcattle.io/force=true.
Token
Validation Checks
Invalid Fields - Create
When a Token is created, the following checks take place:
- If set,
lastUsedAtmust be a valid date time according to RFC3339 (e.g.2023-11-29T00:00:00Z).
Invalid Fields - Update
When a Token is updated, the following checks take place:
- If set,
lastUsedAtmust be a valid date time according to RFC3339 (e.g.2023-11-29T00:00:00Z).
UserAttribute
Validation Checks
Invalid Fields - Create
When a UserAttribute is created, the following checks take place:
- If set,
lastLoginmust be a valid date time according to RFC3339 (e.g.2023-11-29T00:00:00Z). - If set,
disableAftermust be zero or a positive duration (e.g.240h). - If set,
deleteAftermust be zero or a positive duration (e.g.240h).
Invalid Fields - Update
When a UserAttribute is updated, the following checks take place:
- If set,
lastLoginmust be a valid date time according to RFC3339 (e.g.2023-11-29T00:00:00Z). - If set,
disableAftermust be zero or a positive duration (e.g.240h). - If set,
deleteAftermust be zero or a positive duration (e.g.240h).
provisioning.cattle.io/v1
Cluster
Validation Checks
On Create
Creator ID Annotation
The annotation field.cattle.io/creatorId must be set to the Username of the User that initiated the request.
If field.cattle.io/no-creator-rbac annotation is set, field.cattle.io/creatorId cannot be set.
Data Directories
Prevent the creation of new objects with an env var (under spec.agentEnvVars) with a name of CATTLE_AGENT_VAR_DIR.
Prevent the creation of new objects with an invalid data directory. An invalid data directory is defined as the
following:
- Is not an absolute path (i.e. does not start with
/) - Attempts to include environment variables (e.g.
$VARIABLEor${VARIABLE}) - Attempts to include shell expressions (e.g.
$(command)or`command`) - Equal to another data directory
- Attempts to nest another data directory
On Update
Creator ID Annotation
The annotation field.cattle.io/creatorId cannot be changed, but it can be removed.
If field.cattle.io/no-creator-rbac annotation is set, field.cattle.io/creatorId cannot be set.
Data Directories
On update, prevent new env vars with this name from being added but allow them to be removed. Rancher will perform
a one-time migration to move the system-agent data dir definition to the top level field from the AgentEnvVars
section. A secondary validator will ensure that the effective data directory for the system-agent is not different
from the one chosen during cluster creation. Additionally, the changing of a data directory for the system-agent,
kubernetes distro (RKE2/K3s), and CAPR components is also prohibited.
cluster.spec.clusterAgentDeploymentCustomization and cluster.spec.fleetAgentDeploymentCustomization
The DeploymentCustomization fields are of 3 types:
appendTolerations: adds tolerations to the appropriate deployment (cluster-agent/fleet-agent)affinity: adds various affinities to the deployments, which include the followingnodeAffinity: where to schedule the workloadpodAffinitityandpodAntiAffinity: pods to avoid or prefer when scheduling the workload
A Toleration is matched to a regex which is provided by upstream apimachinery here but it boils down to this regex on the label:
([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]
For the Affinity based rules, the podAffinity/podAntiAffinity are validated via label selectors via this apimachinery function whereas the nodeAffinity nodeSelectorTerms are validated via the same Toleration function.
Mutation Checks
On Create
When a cluster is created field.cattle.io/creatorId is set to the Username from the request.
If field.cattle.io/no-creator-rbac annotation is set, field.cattle.io/creatorId does not get set.
On Update
Dynamic Schema Drop
Check for the presence of the provisioning.cattle.io/allow-dynamic-schema-drop annotation. If the value is "true",
perform no mutations. If the value is not present or not "true", compare the value of the dynamicSchemaSpec field
for each machinePool, to its' previous value. If the values are not identical, revert the value for the
dynamicSchemaSpec for the specific machinePool, but do not reject the request.
rbac.authorization.k8s.io/v1
ClusterRole
Validation Checks
Invalid Fields - Update
Users cannot update or remove the following label after it has been added:
- authz.management.cattle.io/gr-owner
ClusterRoleBinding
Validation Checks
Invalid Fields - Update
Users cannot update or remove the following label after it has been added:
- authz.management.cattle.io/grb-owner
Role
Validation Checks
Invalid Fields - Update
Users cannot update or remove the following label after it has been added:
- authz.management.cattle.io/gr-owner
RoleBinding
Validation Checks
Invalid Fields - Update
Users cannot update or remove the following label after it has been added:
- authz.management.cattle.io/grb-owner
rke-machine-config.cattle.io/v1
MachineConfig
Validation Checks
Creator ID Annotation
The annotation field.cattle.io/creatorId must be set to the Username of the User that initiated the request.
The annotation field.cattle.io/creatorId cannot be changed, but it can be removed.
If field.cattle.io/no-creator-rbac annotation is set, field.cattle.io/creatorId cannot be set.
Mutation Checks
Creator ID Annotion
When a cluster is created field.cattle.io/creatorId is set to the Username from the request.
If field.cattle.io/no-creator-rbac annotation is set, field.cattle.io/creatorId does not get set.