Chapter 4: Service Discovery and Load Balancing

Key Concepts

Why Services are Needed

In Kubernetes, Pods are dynamic and can be created or destroyed at any time. Each Pod has its own IP address, but these IP addresses are not fixed. Services provide a stable access endpoint for a group of Pods.

flowchart LR
    subgraph Problem["Problem: Direct Pod Access"]
        Client1[Client] --> Pod1[Pod 10.1.1.1]
        Client1 -.-> Pod2[Pod 10.1.1.2 Deleted]
        Client1 -.-> Pod3[Pod 10.1.1.3 Newly Created]
    end

    subgraph Solution["Solution: Access via Service"]
        Client2[Client] --> Service[Service<br/>ClusterIP: 10.96.0.1]
        Service --> PodA[Pod A]
        Service --> PodB[Pod B]
        Service --> PodC[Pod C]
    end

How Services Work

Services select backend Pods through Label Selectors, and kube-proxy forwards traffic to the correct Pods.

flowchart TB
    subgraph ControlPlane["Control Plane"]
        API[API Server]
        EP[Endpoints Controller]
    end

    subgraph Node1["Node 1"]
        KP1[kube-proxy]
        IPT1[iptables/IPVS rules]
        Pod1[Pod app=web]
        Pod2[Pod app=web]
    end

    subgraph Node2["Node 2"]
        KP2[kube-proxy]
        IPT2[iptables/IPVS rules]
        Pod3[Pod app=web]
    end

    API -->|Watch Service/Endpoints| KP1
    API -->|Watch Service/Endpoints| KP2
    EP -->|Update Endpoints| API

    KP1 -->|Configure| IPT1
    KP2 -->|Configure| IPT2

    Client[Client] -->|Access Service IP| IPT1
    IPT1 -->|Forward| Pod1
    IPT1 -->|Forward| Pod2
    IPT1 -->|Forward| Pod3

Four Service Types

mindmap
  root((Service Types))
    ClusterIP
      Default type
      Internal cluster access
      Virtual IP
    NodePort
      Expose via node port
      External access
      Port range 30000-32767
    LoadBalancer
      Cloud provider load balancer
      Auto-assign external IP
      Common in production
    ExternalName
      CNAME mapping
      Access external services
      No proxy no Selector

ClusterIP Service

ClusterIP is the default Service type, accessible only within the cluster.

Creating ClusterIP Service

# web-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx:1.20
        ports:
        - containerPort: 80
---
# web-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  type: ClusterIP  # Default type, can be omitted
  selector:
    app: web       # Select Pods with label app=web
  ports:
  - name: http
    port: 80       # Service port
    targetPort: 80 # Pod port
    protocol: TCP

# Create Deployment and Service
kubectl apply -f web-deployment.yaml
kubectl apply -f web-service.yaml

# View Service
kubectl get svc web-service
# NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
# web-service   ClusterIP   10.96.123.45    <none>        80/TCP    5s

# View Service details
kubectl describe svc web-service

# View Endpoints (backend Pod list)
kubectl get endpoints web-service
# NAME          ENDPOINTS                                      AGE
# web-service   10.244.1.5:80,10.244.2.6:80,10.244.3.7:80     5s

# Test access within cluster
kubectl run test-pod --image=busybox --rm -it --restart=Never -- wget -qO- http://web-service

Multi-Port Service

apiVersion: v1
kind: Service
metadata:
  name: multi-port-service
spec:
  selector:
    app: myapp
  ports:
  - name: http
    port: 80
    targetPort: 8080
  - name: https
    port: 443
    targetPort: 8443
  - name: metrics
    port: 9090
    targetPort: 9090

Headless Service

Headless Service does not assign a ClusterIP and directly returns the list of backend Pod IPs, commonly used with StatefulSet.

apiVersion: v1
kind: Service
metadata:
  name: headless-service
spec:
  clusterIP: None  # Key: set to None
  selector:
    app: database
  ports:
  - port: 3306
    targetPort: 3306

# View Headless Service
kubectl get svc headless-service
# NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
# headless-service   ClusterIP   None         <none>        3306/TCP   5s

# DNS query returns all Pod IPs
kubectl run test --image=busybox --rm -it --restart=Never -- nslookup headless-service
# Server:    10.96.0.10
# Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
#
# Name:      headless-service
# Address 1: 10.244.1.10 pod-0.headless-service.default.svc.cluster.local
# Address 2: 10.244.2.11 pod-1.headless-service.default.svc.cluster.local
# Address 3: 10.244.3.12 pod-2.headless-service.default.svc.cluster.local

NodePort Service

NodePort opens a port on each node, allowing external cluster access to the Service.

flowchart LR
    External[External Client] -->|NodeIP:30080| Node1[Node 1<br/>192.168.1.10:30080]
    External -->|NodeIP:30080| Node2[Node 2<br/>192.168.1.11:30080]
    External -->|NodeIP:30080| Node3[Node 3<br/>192.168.1.12:30080]

    Node1 --> Service[Service<br/>ClusterIP:80]
    Node2 --> Service
    Node3 --> Service

    Service --> Pod1[Pod 1]
    Service --> Pod2[Pod 2]
    Service --> Pod3[Pod 3]

Creating NodePort Service

apiVersion: v1
kind: Service
metadata:
  name: nodeport-service
spec:
  type: NodePort
  selector:
    app: web
  ports:
  - port: 80          # Service port (internal cluster access)
    targetPort: 80    # Pod port
    nodePort: 30080   # Node port (optional, auto-assigned if not specified)

# Apply configuration
kubectl apply -f nodeport-service.yaml

# View Service
kubectl get svc nodeport-service
# NAME               TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
# nodeport-service   NodePort   10.96.200.100   <none>        80:30080/TCP   5s

# Get node IP
kubectl get nodes -o wide

# Access from external (using any node IP)
curl http://<NodeIP>:30080

NodePort Configuration Options

apiVersion: v1
kind: Service
metadata:
  name: nodeport-advanced
spec:
  type: NodePort
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80
    nodePort: 30080
  # Traffic policy control
  externalTrafficPolicy: Local  # Local: forward only to Pods on this node, Cluster: can forward across nodes

LoadBalancer Service

LoadBalancer type automatically creates a cloud provider’s load balancer in cloud environments.

flowchart TB
    Internet[Internet] --> LB[Cloud Load Balancer<br/>External IP: 203.0.113.10]

    LB --> Node1[Node 1]
    LB --> Node2[Node 2]
    LB --> Node3[Node 3]

    Node1 --> Service[Service<br/>ClusterIP]
    Node2 --> Service
    Node3 --> Service

    Service --> Pod1[Pod 1]
    Service --> Pod2[Pod 2]
    Service --> Pod3[Pod 3]

Creating LoadBalancer Service

apiVersion: v1
kind: Service
metadata:
  name: loadbalancer-service
  annotations:
    # Cloud provider specific annotations (AWS example)
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-internal: "false"
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80
  # Optional: specify load balancer IP (requires cloud provider support)
  # loadBalancerIP: 203.0.113.10

# Apply configuration
kubectl apply -f loadbalancer-service.yaml

# View Service (wait for EXTERNAL-IP assignment)
kubectl get svc loadbalancer-service -w
# NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
# loadbalancer-service   LoadBalancer   10.96.100.50    <pending>       80:31234/TCP   5s
# loadbalancer-service   LoadBalancer   10.96.100.50    203.0.113.10    80:31234/TCP   30s

# Access via external IP
curl http://203.0.113.10

Local LoadBalancer Testing (MetalLB)

For local Kubernetes clusters (like Minikube, Kind), MetalLB can provide LoadBalancer functionality.

# Install MetalLB
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml

# Wait for MetalLB to be ready
kubectl wait --namespace metallb-system \
  --for=condition=ready pod \
  --selector=app=metallb \
  --timeout=90s

# metallb-config.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.240-192.168.1.250  # Available IP range
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default
  namespace: metallb-system
spec:
  ipAddressPools:
  - default-pool

ExternalName Service

ExternalName maps a Service to an external DNS name, commonly used to access external cluster services.

apiVersion: v1
kind: Service
metadata:
  name: external-database
spec:
  type: ExternalName
  externalName: database.example.com  # External service DNS name

# Pods within cluster can access external database via external-database
kubectl run test --image=busybox --rm -it --restart=Never -- nslookup external-database
# Name:      external-database
# Address 1: <IP address of database.example.com>

Accessing External Fixed IP Services

If the external service is a fixed IP rather than a domain name, you need to manually create Endpoints.

# Service without Selector
apiVersion: v1
kind: Service
metadata:
  name: external-service
spec:
  ports:
  - port: 3306
    targetPort: 3306
---
# Manually create Endpoints
apiVersion: v1
kind: Endpoints
metadata:
  name: external-service  # Must match Service name
subsets:
- addresses:
  - ip: 192.168.100.10  # External service IP
  - ip: 192.168.100.11
  ports:
  - port: 3306

Kubernetes DNS

CoreDNS is Kubernetes’ DNS server, providing DNS resolution for Services and Pods.

DNS Resolution Rules

flowchart TB
    subgraph DNS["DNS Resolution Format"]
        SVC["Service DNS"]
        POD["Pod DNS"]
    end

    SVC --> S1["<service>.<namespace>.svc.cluster.local"]
    SVC --> S2["<service>.<namespace>.svc"]
    SVC --> S3["<service>.<namespace>"]
    SVC --> S4["<service> (same namespace)"]

    POD --> P1["<pod-ip-dashed>.<namespace>.pod.cluster.local"]

    subgraph Example["Examples"]
        E1["web-service.default.svc.cluster.local"]
        E2["10-244-1-5.default.pod.cluster.local"]
    end

DNS Hands-on Testing

# Create test Service and Deployment
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80

# Create test Pod for DNS queries
kubectl run dns-test --image=busybox:1.28 --rm -it --restart=Never -- sh

# Execute following commands in container
# View DNS configuration
cat /etc/resolv.conf
# nameserver 10.96.0.10
# search default.svc.cluster.local svc.cluster.local cluster.local
# options ndots:5

# Resolve Service
nslookup nginx
nslookup nginx.default
nslookup nginx.default.svc.cluster.local

# Resolve kubernetes API Service
nslookup kubernetes
nslookup kubernetes.default.svc.cluster.local

Cross-Namespace Access

# Create Service in production namespace
apiVersion: v1
kind: Service
metadata:
  name: backend-api
  namespace: production
spec:
  selector:
    app: backend
  ports:
  - port: 8080

# Access production namespace Service from default namespace Pod
kubectl run test --image=busybox --rm -it --restart=Never -- wget -qO- http://backend-api.production:8080

Custom DNS Configuration

apiVersion: v1
kind: Pod
metadata:
  name: custom-dns-pod
spec:
  containers:
  - name: app
    image: nginx
  dnsPolicy: "None"  # Fully custom DNS
  dnsConfig:
    nameservers:
    - 8.8.8.8
    - 8.8.4.4
    searches:
    - my-domain.com
    options:
    - name: ndots
      value: "2"

Ingress Controller

Ingress provides Layer 7 (HTTP/HTTPS) load balancing, supporting domain name and path-based routing.

Ingress Architecture

flowchart TB
    Internet[Internet] --> Ingress[Ingress Controller<br/>nginx/traefik/istio]

    Ingress -->|host: api.example.com| SVC1[api-service]
    Ingress -->|host: web.example.com| SVC2[web-service]
    Ingress -->|path: /admin| SVC3[admin-service]

    SVC1 --> Pod1[API Pods]
    SVC2 --> Pod2[Web Pods]
    SVC3 --> Pod3[Admin Pods]

Installing Nginx Ingress Controller

# Install using Helm
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace

# Or install using kubectl
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.1/deploy/static/provider/cloud/deploy.yaml

# Verify installation
kubectl get pods -n ingress-nginx
kubectl get svc -n ingress-nginx

Creating Ingress Rules

# First create backend services
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80
---
# Create Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx  # Specify Ingress Controller
  rules:
  - host: web.example.com  # Domain name
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

# Apply configuration
kubectl apply -f web-ingress.yaml

# View Ingress
kubectl get ingress
# NAME          CLASS   HOSTS             ADDRESS         PORTS   AGE
# web-ingress   nginx   web.example.com   192.168.1.240   80      5s

# Local testing (add hosts entry)
echo "192.168.1.240 web.example.com" | sudo tee -a /etc/hosts

# Access test
curl http://web.example.com

Multi-Domain and Path Routing

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-host-ingress
spec:
  ingressClassName: nginx
  rules:
  # First domain
  - host: api.example.com
    http:
      paths:
      - path: /v1
        pathType: Prefix
        backend:
          service:
            name: api-v1-service
            port:
              number: 80
      - path: /v2
        pathType: Prefix
        backend:
          service:
            name: api-v2-service
            port:
              number: 80
  # Second domain
  - host: admin.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: admin-service
            port:
              number: 80

HTTPS Configuration

# Generate self-signed certificate (production environments recommend using cert-manager)
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout tls.key -out tls.crt \
  -subj "/CN=web.example.com"

# Create Secret
kubectl create secret tls web-tls-secret \
  --cert=tls.crt \
  --key=tls.key

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: https-ingress
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - web.example.com
    secretName: web-tls-secret  # TLS certificate Secret
  rules:
  - host: web.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Advanced Ingress Configuration

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: advanced-ingress
  annotations:
    # Path rewrite
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    # Rate limiting
    nginx.ingress.kubernetes.io/limit-rps: "10"
    nginx.ingress.kubernetes.io/limit-connections: "5"
    # Timeout configuration
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
    # Request body size limit
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
    # Enable CORS
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
spec:
  ingressClassName: nginx
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /api(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80

NetworkPolicy

NetworkPolicy controls network traffic between Pods, implementing network isolation.

NetworkPolicy Working Principle

flowchart TB
    subgraph NS1["Namespace: frontend"]
        FE[Frontend Pod<br/>app=frontend]
    end

    subgraph NS2["Namespace: backend"]
        BE[Backend Pod<br/>app=backend]
        NP[NetworkPolicy]
    end

    subgraph NS3["Namespace: database"]
        DB[Database Pod<br/>app=db]
    end

    FE -->|✅ Allowed| BE
    BE -->|✅ Allowed| DB
    FE -.->|❌ Denied| DB

    NP -->|Rule: Only backend can access| DB

Creating NetworkPolicy

# Deny all ingress traffic (default isolation)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
  namespace: production
spec:
  podSelector: {}  # Select all Pods
  policyTypes:
  - Ingress
  # No ingress rules means deny all ingress

---
# Allow specific traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
  - Ingress
  - Egress
  ingress:
  # Allow traffic from frontend
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080
  egress:
  # Allow access to database
  - to:
    - podSelector:
        matchLabels:
          app: database
    ports:
    - protocol: TCP
      port: 5432
  # Allow DNS queries
  - to:
    - namespaceSelector: {}
      podSelector:
        matchLabels:
          k8s-app: kube-dns
    ports:
    - protocol: UDP
      port: 53

Common NetworkPolicy Scenarios

# Scenario 1: Only allow Pods in same namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: same-namespace-only
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector: {}  # All Pods in same namespace

---
# Scenario 2: Allow access from specific namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-from-monitoring
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: web
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: monitoring
    ports:
    - protocol: TCP
      port: 9090

---
# Scenario 3: Allow external IP access
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-external-ip
spec:
  podSelector:
    matchLabels:
      app: public-api
  policyTypes:
  - Ingress
  ingress:
  - from:
    - ipBlock:
        cidr: 203.0.113.0/24
        except:
        - 203.0.113.100/32
    ports:
    - protocol: TCP
      port: 443

Summary

Through this chapter, you should have mastered:

• Service Types: ClusterIP (internal cluster), NodePort (node port), LoadBalancer (cloud load balancer), ExternalName (external mapping)
• Service Discovery: Understanding how Kubernetes DNS works and resolution rules
• Ingress: Configuring Layer 7 load balancing for domain and path routing
• Network Policies: Using NetworkPolicy to control network access between Pods
• Best Practices: Network design and security isolation for microservice architectures

In the next chapter, we will learn about configuration management and storage, including the use of ConfigMap, Secret, and persistent storage.