Pod security admission username exemtion not working

[Mapmo]

Environmental Info:
RKE2 Version:
v1.29.9+rke2r

Node(s) CPU architecture, OS, and Version:
various CPUs, Ubuntu 22.04.2

Cluster Configuration:

3 servers, 5 agents

Describe the bug:

I am using pod security admission to restrict my pods to the baseline PodSecurity level for the entire cluster. Due to having some resources that require exemption, I tried using the username approach, but the controller fails to create new pods because that violates the PodSecurity baseline rule. When using namespace exemption the problem is gone.

Steps To Reproduce:

On the cluster nodes

• Create PodSecurity configuration file

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: DefaultPodSecurity
  configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1alpha1
    kind: PodSecurityConfiguration
    defaults:
      enforce: "baseline"
      enforce-version: "v1.29"
    exemptions:
      usernames: ["nginx"]
      runtimeClassNames: []
      namespaces: []

• Add the configuration file in rke2 config file as pod-security-admission-config-file

On the cluster

• Create a serviceAccount called nginx
• Create a deployment that has serviceAccountName: nginx and has a hostPath volume (the hostPath violates baseline rules), e.g:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      serviceAccountName: nginx
      containers:
      - name: nginx
        image: nginx:latest
        volumeMounts:
        - name: host-volume
          mountPath: /test
      volumes:
      - name: host-volume
        hostPath:
          path: /home

Expected behavior:

The controller is successfully creating pod(s) for the deployment
Actual behavior:

The controller fails to create pods because it's forbidden due to violating the PodSecurity rule
Additional context / logs:

[brandond]

Converted this to a discussion since this is core Kubernetes functionality that you're asking about, and not anything specific to RKE2.

    exemptions:
      usernames: ["nginx"]

serviceAccountName: nginx

I don't think that the username is just the literal service account name. If this was the case then service accounts in different namespaces would be indistinguishable. I think that the actual "user" for the serviceaccount is derived in the format
system:serviceaccount:<namespace>:<serviceaccountname>
so that's what you'd need to reference in your configuration.

Again, this is just core Kubernetes stuff that you should be able to find covered in the upstream docs:
https://kubernetes.io/docs/reference/access-authn-authz/rbac/#referring-to-subjects

ServiceAccounts have names prefixed with system:serviceaccount:, and belong to groups that have names prefixed with system:serviceaccounts:.

Note:
system:serviceaccount: (singular) is the prefix for service account usernames.
system:serviceaccounts: (plural) is the prefix for service account groups.

https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#serviceaccount-token-volume-projection

"sub": "system:serviceaccount:kube-system:coredns"

[Mapmo]

That wasn't the issue. As explained here https://docs.microfocus.com/doc/SMAX/24.4/KubernetesPSA, the issue was that it's the replicaset controller user that needs exemption in order for this to work (which is a bad practice)

[brandond]

I didn't look closely at what you were trying to do, and assumed that for some reason you were intentionally trying to allow the nginx SA to create pods. If that's not the case then it sounds like you had multiple issues.

[Mapmo]

Most pods are created by a controller in response to a workload resource, meaning that exempting an end user will only exempt them from enforcement when creating pods directly, but not when creating a workload resource. Controller service accounts (such as system:serviceaccount:kube-system:replicaset-controller) should generally not be exempted, as doing so would implicitly exempt any user that can create the corresponding workload resource.

https://docs.microfocus.com/doc/SMAX/24.4/KubernetesPSA