Chapter 8: Security and RBAC

Haiyue
25min

Chapter 8: Security and RBAC

Learning Objectives
  • Understand Kubernetes security model and threat analysis
  • Master RBAC permission control configuration methods
  • Learn to configure Pod security policies and network policies
  • Become proficient in security auditing and vulnerability detection

Key Concepts

Kubernetes Security Model

🔄 正在渲染 Mermaid 图表...

Security Threat Analysis

🔄 正在渲染 Mermaid 图表...

Authentication

Authentication Methods Overview

Authentication MethodUse CaseDescription
X509 Client CertificatesUser/Component authenticationIdentifies users and groups through certificate CN and O fields
Bearer TokenServiceAccountDefault method for Pod access to API
Bootstrap TokenNode joining clusterTemporary tokens used by kubeadm
OIDCEnterprise SSO integrationIntegrate with IdP (like Okta, Auth0)
WebhookCustom authenticationCall external service for authentication

ServiceAccount

# Create ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-service-account
  namespace: default
automountServiceAccountToken: true  # Whether to auto-mount token
---
# Create Secret associated with ServiceAccount (K8s 1.24+ requires manual creation)
apiVersion: v1
kind: Secret
metadata:
  name: my-service-account-token
  annotations:
    kubernetes.io/service-account.name: my-service-account
type: kubernetes.io/service-account-token
# Create ServiceAccount
kubectl create serviceaccount my-sa

# View ServiceAccount
kubectl get serviceaccount
kubectl describe serviceaccount my-sa

# Get ServiceAccount token
kubectl create token my-sa

# Run Pod with ServiceAccount
kubectl run test-pod --image=nginx --serviceaccount=my-sa

Using ServiceAccount

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-sa
spec:
  serviceAccountName: my-service-account
  automountServiceAccountToken: true
  containers:
  - name: app
    image: nginx
    # Token is mounted to /var/run/secrets/kubernetes.io/serviceaccount/
    # Contains: ca.crt, namespace, token

X509 Certificate Authentication

# Generate user private key
openssl genrsa -out developer.key 2048

# Generate certificate signing request (CSR)
# CN is username, O is group name
openssl req -new -key developer.key -out developer.csr \
  -subj "/CN=developer/O=dev-team"

# Create CertificateSigningRequest
cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: developer-csr
spec:
  request: $(cat developer.csr | base64 | tr -d '\n')
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
EOF

# Approve CSR
kubectl certificate approve developer-csr

# Get certificate
kubectl get csr developer-csr -o jsonpath='{.status.certificate}' | base64 -d > developer.crt

# Configure kubeconfig
kubectl config set-credentials developer \
  --client-certificate=developer.crt \
  --client-key=developer.key

kubectl config set-context developer-context \
  --cluster=kubernetes \
  --namespace=default \
  --user=developer

# Use new user
kubectl config use-context developer-context
kubectl get pods  # Will fail due to no permissions

RBAC Authorization

RBAC Core Concepts

🔄 正在渲染 Mermaid 图表...

Role and ClusterRole

# Role: Namespace-level permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]  # Core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get"]

---
# ClusterRole: Cluster-level permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "watch", "list"]

---
# More complex ClusterRole example
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: developer-role
rules:
# Pod-related permissions
- apiGroups: [""]
  resources: ["pods", "pods/log", "pods/exec"]
  verbs: ["get", "list", "watch", "create", "delete"]
# Deployment-related permissions
- apiGroups: ["apps"]
  resources: ["deployments", "replicasets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
# Service-related permissions
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "watch", "create", "update", "delete"]
# ConfigMap and Secret read-only
- apiGroups: [""]
  resources: ["configmaps", "secrets"]
  verbs: ["get", "list", "watch"]
# Specific resource name restrictions
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["my-config"]
  verbs: ["update"]

RoleBinding and ClusterRoleBinding

# RoleBinding: Bind Role to user
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
# Bind to user
- kind: User
  name: developer
  apiGroup: rbac.authorization.k8s.io
# Bind to group
- kind: Group
  name: dev-team
  apiGroup: rbac.authorization.k8s.io
# Bind to ServiceAccount
- kind: ServiceAccount
  name: my-service-account
  namespace: default
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

---
# ClusterRoleBinding: Cluster-level binding
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-admin-binding
subjects:
- kind: User
  name: admin
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: cluster-admin  # Built-in super admin role
  apiGroup: rbac.authorization.k8s.io

---
# Use ClusterRole in specific namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: dev-team-edit
  namespace: development
subjects:
- kind: Group
  name: dev-team
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: edit  # Built-in edit role
  apiGroup: rbac.authorization.k8s.io

Built-in ClusterRoles

ClusterRoleDescription
cluster-adminSuper administrator, full control permissions
adminNamespace administrator, cannot modify quotas and namespaces
editRead/write most resources, cannot view/modify Roles and RoleBindings
viewRead-only permissions, cannot view Secrets

RBAC Permission Verification

# Check current user permissions
kubectl auth can-i create pods
kubectl auth can-i delete deployments
kubectl auth can-i '*' '*'  # Is super administrator

# Check specific user permissions
kubectl auth can-i create pods --as developer
kubectl auth can-i delete pods --as system:serviceaccount:default:my-sa

# Check permissions in specific namespace
kubectl auth can-i list secrets -n production --as developer

# List all permissions
kubectl auth can-i --list
kubectl auth can-i --list --as developer

# View user role bindings
kubectl get rolebindings,clusterrolebindings -A | grep developer

Pod Security

Security Context

apiVersion: v1
kind: Pod
metadata:
  name: secure-pod
spec:
  securityContext:
    # Pod-level security settings
    runAsUser: 1000           # Run user ID
    runAsGroup: 3000          # Run group ID
    fsGroup: 2000             # Filesystem group
    runAsNonRoot: true        # Must run as non-root
    seccompProfile:
      type: RuntimeDefault    # Use runtime default seccomp profile
  containers:
  - name: app
    image: nginx
    securityContext:
      # Container-level security settings
      allowPrivilegeEscalation: false  # Disable privilege escalation
      readOnlyRootFilesystem: true     # Read-only root filesystem
      runAsNonRoot: true
      runAsUser: 1000
      capabilities:
        drop:
        - ALL                          # Drop all capabilities
        add:
        - NET_BIND_SERVICE             # Only add necessary capabilities
    volumeMounts:
    - name: tmp
      mountPath: /tmp
    - name: cache
      mountPath: /var/cache/nginx
  volumes:
  - name: tmp
    emptyDir: {}
  - name: cache
    emptyDir: {}

Pod Security Standards (PSS)

Kubernetes 1.25+ uses Pod Security Standards to replace PSP.

# Enable Pod Security Standards through namespace labels
apiVersion: v1
kind: Namespace
metadata:
  name: restricted-ns
  labels:
    # enforce: Pods violating policy are rejected
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    # warn: Warning issued when policy is violated
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/warn-version: latest
    # audit: Audit log recorded when policy is violated
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/audit-version: latest

Three Security Levels

🔄 正在渲染 Mermaid 图表...
# Baseline level Pod example
apiVersion: v1
kind: Pod
metadata:
  name: baseline-pod
spec:
  containers:
  - name: app
    image: nginx
    securityContext:
      allowPrivilegeEscalation: false
    # Baseline allows running as root
    # But prohibits privileged, hostNetwork, etc.

---
# Restricted level Pod example
apiVersion: v1
kind: Pod
metadata:
  name: restricted-pod
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    image: nginx
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop:
        - ALL

Network Security

Network Policy Basics

# Default deny all ingress traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: production
spec:
  podSelector: {}  # Select all Pods
  policyTypes:
  - Ingress

---
# Default deny all egress traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-egress
  namespace: production
spec:
  podSelector: {}
  policyTypes:
  - Egress

---
# Default deny all traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Fine-grained Network Policies

# Policy allowing specific traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-server-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api-server
  policyTypes:
  - Ingress
  - Egress
  ingress:
  # Allow traffic from frontend
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080
  # Allow Prometheus scraping from monitoring namespace
  - from:
    - namespaceSelector:
        matchLabels:
          name: monitoring
      podSelector:
        matchLabels:
          app: prometheus
    ports:
    - protocol: TCP
      port: 9090
  egress:
  # Allow access to database
  - to:
    - podSelector:
        matchLabels:
          app: database
    ports:
    - protocol: TCP
      port: 5432
  # Allow DNS queries
  - to:
    - namespaceSelector: {}
      podSelector:
        matchLabels:
          k8s-app: kube-dns
    ports:
    - protocol: UDP
      port: 53
  # Allow access to external API
  - to:
    - ipBlock:
        cidr: 203.0.113.0/24
    ports:
    - protocol: TCP
      port: 443

Secret Security

Secret Encryption at Rest

# Configure etcd encryption (in API Server configuration)
# /etc/kubernetes/encryption-config.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  # Use aescbc encryption
  - aescbc:
      keys:
      - name: key1
        secret: <base64-encoded-32-byte-key>
  # Try plaintext read if decryption fails (for migration)
  - identity: {}
# Generate encryption key
head -c 32 /dev/urandom | base64

# Verify Secret is encrypted
kubectl get secret my-secret -o yaml
# Content still shows base64, but stored encrypted in etcd

# Check actual storage in etcd
ETCDCTL_API=3 etcdctl get /registry/secrets/default/my-secret | hexdump -C

External Key Management

# Using External Secrets Operator
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: aws-secrets-manager
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        secretRef:
          accessKeyIDSecretRef:
            name: aws-credentials
            key: access-key-id
          secretAccessKeySecretRef:
            name: aws-credentials
            key: secret-access-key

---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: db-secret
    creationPolicy: Owner
  data:
  - secretKey: username
    remoteRef:
      key: production/database
      property: username
  - secretKey: password
    remoteRef:
      key: production/database
      property: password

Audit Logging

Configuring Audit Policy

# /etc/kubernetes/audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
# Don't log response bodies for read-only requests
- level: Metadata
  verbs: ["get", "list", "watch"]

# Log all Secret operations (but not content)
- level: Metadata
  resources:
  - group: ""
    resources: ["secrets"]

# Log full request/response for authentication
- level: RequestResponse
  resources:
  - group: "authentication.k8s.io"
    resources: ["tokenreviews"]

# Log RBAC changes
- level: RequestResponse
  resources:
  - group: "rbac.authorization.k8s.io"
    resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"]

# Log Pod creation/deletion
- level: Request
  verbs: ["create", "delete", "patch", "update"]
  resources:
  - group: ""
    resources: ["pods"]

# Default level
- level: Metadata
  omitStages:
  - "RequestReceived"

Configuring API Server to Use Auditing

# Add these parameters to kube-apiserver configuration
# --audit-policy-file=/etc/kubernetes/audit-policy.yaml
# --audit-log-path=/var/log/kubernetes/audit.log
# --audit-log-maxage=30
# --audit-log-maxbackup=10
# --audit-log-maxsize=100

Audit Log Analysis

# View audit logs
tail -f /var/log/kubernetes/audit.log | jq .

# Filter specific user operations
cat audit.log | jq 'select(.user.username == "developer")'

# Find Secret access
cat audit.log | jq 'select(.objectRef.resource == "secrets")'

# Find failed requests
cat audit.log | jq 'select(.responseStatus.code >= 400)'

Image Security

Image Scanning

# Use Trivy to scan image vulnerabilities
trivy image nginx:latest
trivy image --severity HIGH,CRITICAL myapp:v1.0

# Scan images in Kubernetes cluster
trivy k8s --report summary cluster

Image Admission Control

# Use OPA Gatekeeper to restrict image sources
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8sallowedrepos
spec:
  crd:
    spec:
      names:
        kind: K8sAllowedRepos
      validation:
        openAPIV3Schema:
          type: object
          properties:
            repos:
              type: array
              items:
                type: string
  targets:
  - target: admission.k8s.gatekeeper.sh
    rego: |
      package k8sallowedrepos

      violation[{"msg": msg}] {
        container := input.review.object.spec.containers[_]
        satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
        not any(satisfied)
        msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
      }

---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRepos
metadata:
  name: allowed-repos
spec:
  match:
    kinds:
    - apiGroups: [""]
      kinds: ["Pod"]
    namespaces:
    - production
  parameters:
    repos:
    - "gcr.io/my-project/"
    - "docker.io/library/"

Image Signature Verification

# Use Cosign and Policy Controller to verify image signatures
apiVersion: policy.sigstore.dev/v1alpha1
kind: ClusterImagePolicy
metadata:
  name: verify-signatures
spec:
  images:
  - glob: "gcr.io/my-project/*"
  authorities:
  - keyless:
      url: https://fulcio.sigstore.dev
      identities:
      - issuer: https://accounts.google.com
        subject: build@my-project.iam.gserviceaccount.com

Practical Exercise

Multi-tenant Security Configuration

# Create tenant namespace
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-a
  labels:
    tenant: a
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/warn: restricted

---
# Tenant ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tenant-a-admin
  namespace: tenant-a

---
# Tenant admin role
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: tenant-a
  name: tenant-admin
rules:
- apiGroups: ["", "apps", "batch"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: ["networking.k8s.io"]
  resources: ["networkpolicies", "ingresses"]
  verbs: ["*"]

---
# Bind role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tenant-a-admin-binding
  namespace: tenant-a
subjects:
- kind: ServiceAccount
  name: tenant-a-admin
  namespace: tenant-a
- kind: Group
  name: tenant-a-admins
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tenant-admin
  apiGroup: rbac.authorization.k8s.io

---
# Resource quota
apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-a-quota
  namespace: tenant-a
spec:
  hard:
    requests.cpu: "10"
    requests.memory: 20Gi
    limits.cpu: "20"
    limits.memory: 40Gi
    pods: "50"
    services: "10"
    secrets: "20"
    configmaps: "20"
    persistentvolumeclaims: "10"

---
# Network isolation
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: tenant-isolation
  namespace: tenant-a
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
  ingress:
  # Only allow traffic from same namespace
  - from:
    - podSelector: {}
  # Allow Ingress Controller
  - from:
    - namespaceSelector:
        matchLabels:
          name: ingress-nginx
  egress:
  # Allow access to same namespace
  - to:
    - podSelector: {}
  # Allow DNS
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      podSelector:
        matchLabels:
          k8s-app: kube-dns
    ports:
    - protocol: UDP
      port: 53
  # Allow external access
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.0.0.0/8
        - 172.16.0.0/12
        - 192.168.0.0/16

CI/CD Service Account Configuration

# CI/CD dedicated ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: cicd-deployer
  namespace: cicd

---
# Deployment permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: cicd-deployer
rules:
# Deployment and related resources
- apiGroups: ["apps"]
  resources: ["deployments", "replicasets"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
# Pod (for status checking)
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list", "watch"]
# Service
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
# ConfigMap (only update specific ones)
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
# Ingress
- apiGroups: ["networking.k8s.io"]
  resources: ["ingresses"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]

---
# Bind to specific namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cicd-deployer-production
  namespace: production
subjects:
- kind: ServiceAccount
  name: cicd-deployer
  namespace: cicd
roleRef:
  kind: ClusterRole
  name: cicd-deployer
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cicd-deployer-staging
  namespace: staging
subjects:
- kind: ServiceAccount
  name: cicd-deployer
  namespace: cicd
roleRef:
  kind: ClusterRole
  name: cicd-deployer
  apiGroup: rbac.authorization.k8s.io

Security Checklist

# Use kube-bench to check CIS benchmarks
docker run --rm -v /etc:/host/etc:ro -v /var:/host/var:ro \
  aquasec/kube-bench run --targets=master,node

# Use kubeaudit to check security configuration
kubeaudit all

# Check RBAC configuration
kubectl auth can-i --list --as system:anonymous
kubectl get clusterrolebindings -o json | jq '.items[] | select(.subjects[]?.name == "system:anonymous")'

# Check privileged Pods
kubectl get pods -A -o json | jq '.items[] | select(.spec.containers[].securityContext.privileged == true) | .metadata.name'

# Check hostNetwork Pods
kubectl get pods -A -o json | jq '.items[] | select(.spec.hostNetwork == true) | .metadata.name'

# Check Pods without resource limits
kubectl get pods -A -o json | jq '.items[] | select(.spec.containers[].resources.limits == null) | .metadata.name'
Security Best Practices
  1. Principle of least privilege: Only grant necessary permissions
  2. Use namespace isolation: Different tenants/environments use different namespaces
  3. Enable network policies: Default deny, explicitly allow
  4. Pod security standards: Use at least Baseline level
  5. Audit logging: Record sensitive operations
  6. Image security: Scan vulnerabilities, verify signatures
  7. Regular audits: Periodically review permissions and configurations

Summary

Through this chapter, you should have mastered:

  • Authentication mechanisms: ServiceAccount, certificate authentication, OIDC integration
  • RBAC authorization: Role, ClusterRole, binding configuration
  • Pod security: SecurityContext, Pod Security Standards
  • Network security: NetworkPolicy configuration and best practices
  • Audit logging: Configuring and analyzing audit logs

In the next chapter, we will learn about CI/CD and GitOps, mastering continuous delivery practices for cloud-native applications.