Chapter 8: Security and RBAC
Haiyue
25min
Chapter 8: Security and RBAC
Learning Objectives
- Understand Kubernetes security model and threat analysis
- Master RBAC permission control configuration methods
- Learn to configure Pod security policies and network policies
- Become proficient in security auditing and vulnerability detection
Key Concepts
Kubernetes Security Model
🔄 正在渲染 Mermaid 图表...
Security Threat Analysis
🔄 正在渲染 Mermaid 图表...
Authentication
Authentication Methods Overview
| Authentication Method | Use Case | Description |
|---|---|---|
| X509 Client Certificates | User/Component authentication | Identifies users and groups through certificate CN and O fields |
| Bearer Token | ServiceAccount | Default method for Pod access to API |
| Bootstrap Token | Node joining cluster | Temporary tokens used by kubeadm |
| OIDC | Enterprise SSO integration | Integrate with IdP (like Okta, Auth0) |
| Webhook | Custom authentication | Call external service for authentication |
ServiceAccount
# Create ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-service-account
namespace: default
automountServiceAccountToken: true # Whether to auto-mount token
---
# Create Secret associated with ServiceAccount (K8s 1.24+ requires manual creation)
apiVersion: v1
kind: Secret
metadata:
name: my-service-account-token
annotations:
kubernetes.io/service-account.name: my-service-account
type: kubernetes.io/service-account-token
# Create ServiceAccount
kubectl create serviceaccount my-sa
# View ServiceAccount
kubectl get serviceaccount
kubectl describe serviceaccount my-sa
# Get ServiceAccount token
kubectl create token my-sa
# Run Pod with ServiceAccount
kubectl run test-pod --image=nginx --serviceaccount=my-sa
Using ServiceAccount
apiVersion: v1
kind: Pod
metadata:
name: pod-with-sa
spec:
serviceAccountName: my-service-account
automountServiceAccountToken: true
containers:
- name: app
image: nginx
# Token is mounted to /var/run/secrets/kubernetes.io/serviceaccount/
# Contains: ca.crt, namespace, token
X509 Certificate Authentication
# Generate user private key
openssl genrsa -out developer.key 2048
# Generate certificate signing request (CSR)
# CN is username, O is group name
openssl req -new -key developer.key -out developer.csr \
-subj "/CN=developer/O=dev-team"
# Create CertificateSigningRequest
cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: developer-csr
spec:
request: $(cat developer.csr | base64 | tr -d '\n')
signerName: kubernetes.io/kube-apiserver-client
usages:
- client auth
EOF
# Approve CSR
kubectl certificate approve developer-csr
# Get certificate
kubectl get csr developer-csr -o jsonpath='{.status.certificate}' | base64 -d > developer.crt
# Configure kubeconfig
kubectl config set-credentials developer \
--client-certificate=developer.crt \
--client-key=developer.key
kubectl config set-context developer-context \
--cluster=kubernetes \
--namespace=default \
--user=developer
# Use new user
kubectl config use-context developer-context
kubectl get pods # Will fail due to no permissions
RBAC Authorization
RBAC Core Concepts
🔄 正在渲染 Mermaid 图表...
Role and ClusterRole
# Role: Namespace-level permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: pod-reader
rules:
- apiGroups: [""] # Core API group
resources: ["pods"]
verbs: ["get", "watch", "list"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get"]
---
# ClusterRole: Cluster-level permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: secret-reader
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "watch", "list"]
---
# More complex ClusterRole example
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: developer-role
rules:
# Pod-related permissions
- apiGroups: [""]
resources: ["pods", "pods/log", "pods/exec"]
verbs: ["get", "list", "watch", "create", "delete"]
# Deployment-related permissions
- apiGroups: ["apps"]
resources: ["deployments", "replicasets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
# Service-related permissions
- apiGroups: [""]
resources: ["services"]
verbs: ["get", "list", "watch", "create", "update", "delete"]
# ConfigMap and Secret read-only
- apiGroups: [""]
resources: ["configmaps", "secrets"]
verbs: ["get", "list", "watch"]
# Specific resource name restrictions
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["my-config"]
verbs: ["update"]
RoleBinding and ClusterRoleBinding
# RoleBinding: Bind Role to user
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-pods
namespace: default
subjects:
# Bind to user
- kind: User
name: developer
apiGroup: rbac.authorization.k8s.io
# Bind to group
- kind: Group
name: dev-team
apiGroup: rbac.authorization.k8s.io
# Bind to ServiceAccount
- kind: ServiceAccount
name: my-service-account
namespace: default
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
---
# ClusterRoleBinding: Cluster-level binding
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-admin-binding
subjects:
- kind: User
name: admin
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: cluster-admin # Built-in super admin role
apiGroup: rbac.authorization.k8s.io
---
# Use ClusterRole in specific namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: dev-team-edit
namespace: development
subjects:
- kind: Group
name: dev-team
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: edit # Built-in edit role
apiGroup: rbac.authorization.k8s.io
Built-in ClusterRoles
| ClusterRole | Description |
|---|---|
| cluster-admin | Super administrator, full control permissions |
| admin | Namespace administrator, cannot modify quotas and namespaces |
| edit | Read/write most resources, cannot view/modify Roles and RoleBindings |
| view | Read-only permissions, cannot view Secrets |
RBAC Permission Verification
# Check current user permissions
kubectl auth can-i create pods
kubectl auth can-i delete deployments
kubectl auth can-i '*' '*' # Is super administrator
# Check specific user permissions
kubectl auth can-i create pods --as developer
kubectl auth can-i delete pods --as system:serviceaccount:default:my-sa
# Check permissions in specific namespace
kubectl auth can-i list secrets -n production --as developer
# List all permissions
kubectl auth can-i --list
kubectl auth can-i --list --as developer
# View user role bindings
kubectl get rolebindings,clusterrolebindings -A | grep developer
Pod Security
Security Context
apiVersion: v1
kind: Pod
metadata:
name: secure-pod
spec:
securityContext:
# Pod-level security settings
runAsUser: 1000 # Run user ID
runAsGroup: 3000 # Run group ID
fsGroup: 2000 # Filesystem group
runAsNonRoot: true # Must run as non-root
seccompProfile:
type: RuntimeDefault # Use runtime default seccomp profile
containers:
- name: app
image: nginx
securityContext:
# Container-level security settings
allowPrivilegeEscalation: false # Disable privilege escalation
readOnlyRootFilesystem: true # Read-only root filesystem
runAsNonRoot: true
runAsUser: 1000
capabilities:
drop:
- ALL # Drop all capabilities
add:
- NET_BIND_SERVICE # Only add necessary capabilities
volumeMounts:
- name: tmp
mountPath: /tmp
- name: cache
mountPath: /var/cache/nginx
volumes:
- name: tmp
emptyDir: {}
- name: cache
emptyDir: {}
Pod Security Standards (PSS)
Kubernetes 1.25+ uses Pod Security Standards to replace PSP.
# Enable Pod Security Standards through namespace labels
apiVersion: v1
kind: Namespace
metadata:
name: restricted-ns
labels:
# enforce: Pods violating policy are rejected
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: latest
# warn: Warning issued when policy is violated
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
# audit: Audit log recorded when policy is violated
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/audit-version: latest
Three Security Levels
🔄 正在渲染 Mermaid 图表...
# Baseline level Pod example
apiVersion: v1
kind: Pod
metadata:
name: baseline-pod
spec:
containers:
- name: app
image: nginx
securityContext:
allowPrivilegeEscalation: false
# Baseline allows running as root
# But prohibits privileged, hostNetwork, etc.
---
# Restricted level Pod example
apiVersion: v1
kind: Pod
metadata:
name: restricted-pod
spec:
securityContext:
runAsNonRoot: true
runAsUser: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: app
image: nginx
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
Network Security
Network Policy Basics
# Default deny all ingress traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-ingress
namespace: production
spec:
podSelector: {} # Select all Pods
policyTypes:
- Ingress
---
# Default deny all egress traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-egress
namespace: production
spec:
podSelector: {}
policyTypes:
- Egress
---
# Default deny all traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: production
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
Fine-grained Network Policies
# Policy allowing specific traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-server-policy
namespace: production
spec:
podSelector:
matchLabels:
app: api-server
policyTypes:
- Ingress
- Egress
ingress:
# Allow traffic from frontend
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- protocol: TCP
port: 8080
# Allow Prometheus scraping from monitoring namespace
- from:
- namespaceSelector:
matchLabels:
name: monitoring
podSelector:
matchLabels:
app: prometheus
ports:
- protocol: TCP
port: 9090
egress:
# Allow access to database
- to:
- podSelector:
matchLabels:
app: database
ports:
- protocol: TCP
port: 5432
# Allow DNS queries
- to:
- namespaceSelector: {}
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
# Allow access to external API
- to:
- ipBlock:
cidr: 203.0.113.0/24
ports:
- protocol: TCP
port: 443
Secret Security
Secret Encryption at Rest
# Configure etcd encryption (in API Server configuration)
# /etc/kubernetes/encryption-config.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
# Use aescbc encryption
- aescbc:
keys:
- name: key1
secret: <base64-encoded-32-byte-key>
# Try plaintext read if decryption fails (for migration)
- identity: {}
# Generate encryption key
head -c 32 /dev/urandom | base64
# Verify Secret is encrypted
kubectl get secret my-secret -o yaml
# Content still shows base64, but stored encrypted in etcd
# Check actual storage in etcd
ETCDCTL_API=3 etcdctl get /registry/secrets/default/my-secret | hexdump -C
External Key Management
# Using External Secrets Operator
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: aws-secrets-manager
spec:
provider:
aws:
service: SecretsManager
region: us-east-1
auth:
secretRef:
accessKeyIDSecretRef:
name: aws-credentials
key: access-key-id
secretAccessKeySecretRef:
name: aws-credentials
key: secret-access-key
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: db-credentials
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secrets-manager
kind: SecretStore
target:
name: db-secret
creationPolicy: Owner
data:
- secretKey: username
remoteRef:
key: production/database
property: username
- secretKey: password
remoteRef:
key: production/database
property: password
Audit Logging
Configuring Audit Policy
# /etc/kubernetes/audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
# Don't log response bodies for read-only requests
- level: Metadata
verbs: ["get", "list", "watch"]
# Log all Secret operations (but not content)
- level: Metadata
resources:
- group: ""
resources: ["secrets"]
# Log full request/response for authentication
- level: RequestResponse
resources:
- group: "authentication.k8s.io"
resources: ["tokenreviews"]
# Log RBAC changes
- level: RequestResponse
resources:
- group: "rbac.authorization.k8s.io"
resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"]
# Log Pod creation/deletion
- level: Request
verbs: ["create", "delete", "patch", "update"]
resources:
- group: ""
resources: ["pods"]
# Default level
- level: Metadata
omitStages:
- "RequestReceived"
Configuring API Server to Use Auditing
# Add these parameters to kube-apiserver configuration
# --audit-policy-file=/etc/kubernetes/audit-policy.yaml
# --audit-log-path=/var/log/kubernetes/audit.log
# --audit-log-maxage=30
# --audit-log-maxbackup=10
# --audit-log-maxsize=100
Audit Log Analysis
# View audit logs
tail -f /var/log/kubernetes/audit.log | jq .
# Filter specific user operations
cat audit.log | jq 'select(.user.username == "developer")'
# Find Secret access
cat audit.log | jq 'select(.objectRef.resource == "secrets")'
# Find failed requests
cat audit.log | jq 'select(.responseStatus.code >= 400)'
Image Security
Image Scanning
# Use Trivy to scan image vulnerabilities
trivy image nginx:latest
trivy image --severity HIGH,CRITICAL myapp:v1.0
# Scan images in Kubernetes cluster
trivy k8s --report summary cluster
Image Admission Control
# Use OPA Gatekeeper to restrict image sources
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
name: k8sallowedrepos
spec:
crd:
spec:
names:
kind: K8sAllowedRepos
validation:
openAPIV3Schema:
type: object
properties:
repos:
type: array
items:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8sallowedrepos
violation[{"msg": msg}] {
container := input.review.object.spec.containers[_]
satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
not any(satisfied)
msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
}
---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRepos
metadata:
name: allowed-repos
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Pod"]
namespaces:
- production
parameters:
repos:
- "gcr.io/my-project/"
- "docker.io/library/"
Image Signature Verification
# Use Cosign and Policy Controller to verify image signatures
apiVersion: policy.sigstore.dev/v1alpha1
kind: ClusterImagePolicy
metadata:
name: verify-signatures
spec:
images:
- glob: "gcr.io/my-project/*"
authorities:
- keyless:
url: https://fulcio.sigstore.dev
identities:
- issuer: https://accounts.google.com
subject: build@my-project.iam.gserviceaccount.com
Practical Exercise
Multi-tenant Security Configuration
# Create tenant namespace
apiVersion: v1
kind: Namespace
metadata:
name: tenant-a
labels:
tenant: a
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/warn: restricted
---
# Tenant ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
name: tenant-a-admin
namespace: tenant-a
---
# Tenant admin role
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: tenant-a
name: tenant-admin
rules:
- apiGroups: ["", "apps", "batch"]
resources: ["*"]
verbs: ["*"]
- apiGroups: ["networking.k8s.io"]
resources: ["networkpolicies", "ingresses"]
verbs: ["*"]
---
# Bind role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tenant-a-admin-binding
namespace: tenant-a
subjects:
- kind: ServiceAccount
name: tenant-a-admin
namespace: tenant-a
- kind: Group
name: tenant-a-admins
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: tenant-admin
apiGroup: rbac.authorization.k8s.io
---
# Resource quota
apiVersion: v1
kind: ResourceQuota
metadata:
name: tenant-a-quota
namespace: tenant-a
spec:
hard:
requests.cpu: "10"
requests.memory: 20Gi
limits.cpu: "20"
limits.memory: 40Gi
pods: "50"
services: "10"
secrets: "20"
configmaps: "20"
persistentvolumeclaims: "10"
---
# Network isolation
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: tenant-isolation
namespace: tenant-a
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress:
# Only allow traffic from same namespace
- from:
- podSelector: {}
# Allow Ingress Controller
- from:
- namespaceSelector:
matchLabels:
name: ingress-nginx
egress:
# Allow access to same namespace
- to:
- podSelector: {}
# Allow DNS
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
# Allow external access
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
CI/CD Service Account Configuration
# CI/CD dedicated ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
name: cicd-deployer
namespace: cicd
---
# Deployment permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cicd-deployer
rules:
# Deployment and related resources
- apiGroups: ["apps"]
resources: ["deployments", "replicasets"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
# Pod (for status checking)
- apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["get", "list", "watch"]
# Service
- apiGroups: [""]
resources: ["services"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
# ConfigMap (only update specific ones)
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
# Ingress
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
# Bind to specific namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cicd-deployer-production
namespace: production
subjects:
- kind: ServiceAccount
name: cicd-deployer
namespace: cicd
roleRef:
kind: ClusterRole
name: cicd-deployer
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cicd-deployer-staging
namespace: staging
subjects:
- kind: ServiceAccount
name: cicd-deployer
namespace: cicd
roleRef:
kind: ClusterRole
name: cicd-deployer
apiGroup: rbac.authorization.k8s.io
Security Checklist
# Use kube-bench to check CIS benchmarks
docker run --rm -v /etc:/host/etc:ro -v /var:/host/var:ro \
aquasec/kube-bench run --targets=master,node
# Use kubeaudit to check security configuration
kubeaudit all
# Check RBAC configuration
kubectl auth can-i --list --as system:anonymous
kubectl get clusterrolebindings -o json | jq '.items[] | select(.subjects[]?.name == "system:anonymous")'
# Check privileged Pods
kubectl get pods -A -o json | jq '.items[] | select(.spec.containers[].securityContext.privileged == true) | .metadata.name'
# Check hostNetwork Pods
kubectl get pods -A -o json | jq '.items[] | select(.spec.hostNetwork == true) | .metadata.name'
# Check Pods without resource limits
kubectl get pods -A -o json | jq '.items[] | select(.spec.containers[].resources.limits == null) | .metadata.name'
Security Best Practices
- Principle of least privilege: Only grant necessary permissions
- Use namespace isolation: Different tenants/environments use different namespaces
- Enable network policies: Default deny, explicitly allow
- Pod security standards: Use at least Baseline level
- Audit logging: Record sensitive operations
- Image security: Scan vulnerabilities, verify signatures
- Regular audits: Periodically review permissions and configurations
Summary
Through this chapter, you should have mastered:
- Authentication mechanisms: ServiceAccount, certificate authentication, OIDC integration
- RBAC authorization: Role, ClusterRole, binding configuration
- Pod security: SecurityContext, Pod Security Standards
- Network security: NetworkPolicy configuration and best practices
- Audit logging: Configuring and analyzing audit logs
In the next chapter, we will learn about CI/CD and GitOps, mastering continuous delivery practices for cloud-native applications.