What Are the kube-proxy Modes and How Do They Work?

Detailed Answer

What kube-proxy Does

kube-proxy is a network component that runs on every node in the cluster, typically as a DaemonSet. Its job is to implement the virtual IP mechanism for Kubernetes Services. When you create a Service, kube-proxy programs rules on every node so that traffic destined for the Service's ClusterIP gets forwarded to one of the backend Pods.

kube-proxy watches the Kubernetes API server for changes to Service and EndpointSlice objects and updates forwarding rules in real time.

The Three Modes

1. iptables Mode (Default)

In iptables mode, kube-proxy creates a chain of iptables rules for each Service. When a packet arrives for a Service ClusterIP, iptables uses the statistic module to randomly select a backend Pod.

# View iptables rules created by kube-proxy
iptables -t nat -L KUBE-SERVICES -n | head -20

# Example rule chain for a Service
# KUBE-SVC-XXXX -> KUBE-SEP-AAAA (33% probability)
#               -> KUBE-SEP-BBBB (50% of remaining)
#               -> KUBE-SEP-CCCC (100% of remaining)

Pros: Simple, reliable, well-tested default. Cons: O(n) rule traversal. With thousands of Services, rule updates become slow and CPU-intensive.

2. IPVS Mode

IPVS (IP Virtual Server) mode uses the Linux kernel's IPVS subsystem, which is specifically designed for load balancing. It stores rules in a hash table, giving O(1) lookup performance regardless of the number of Services.

# kube-proxy ConfigMap for IPVS mode
apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-proxy
  namespace: kube-system
data:
  config.conf: |
    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    kind: KubeProxyConfiguration
    mode: "ipvs"
    ipvs:
      scheduler: "rr"  # round-robin; options: rr, lc, dh, sh, sed, nq

# Check current kube-proxy mode
kubectl logs -n kube-system -l k8s-app=kube-proxy | grep "Using"

# View IPVS rules
ipvsadm -Ln

IPVS supports multiple load-balancing algorithms:

• rr (round-robin) - default
• lc (least connections)
• sh (source hashing)
• sed (shortest expected delay)

Pros: O(1) performance, multiple scheduling algorithms, better throughput. Cons: Requires IPVS kernel modules (ip_vs, ip_vs_rr, etc.) to be loaded on every node.

3. Userspace Mode (Legacy)

In userspace mode, kube-proxy opens a proxy port on each node and forwards traffic through a user-space process. This is the original implementation and is significantly slower because every packet crosses the kernel-userspace boundary twice.

This mode is deprecated and should never be used in production.

How Service Routing Works End-to-End

1. A client Pod sends traffic to my-service.default.svc.cluster.local on port 80.
2. CoreDNS resolves this to the Service ClusterIP, e.g., 10.96.45.12.
3. The packet reaches the node's network stack.
4. kube-proxy's iptables/IPVS rules intercept the packet and DNAT it to a backend Pod IP, e.g., 10.244.1.15:8080.
5. The CNI plugin routes the packet to the correct node and Pod.

Checking and Switching Modes

# Check current proxy mode
kubectl get configmap kube-proxy -n kube-system -o yaml | grep mode

# Edit kube-proxy ConfigMap to switch modes
kubectl edit configmap kube-proxy -n kube-system
# Change mode: "" to mode: "ipvs"

# Restart kube-proxy DaemonSet to apply
kubectl rollout restart daemonset kube-proxy -n kube-system

# Verify IPVS modules are loaded (required for IPVS mode)
lsmod | grep ip_vs

eBPF as a kube-proxy Replacement

Modern CNI plugins like Cilium can replace kube-proxy entirely by implementing service routing using eBPF programs attached to network interfaces. This eliminates iptables/IPVS overhead and provides better performance, especially for NodePort and externalTrafficPolicy handling.

# Install Cilium with kube-proxy replacement
helm install cilium cilium/cilium \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=<API_SERVER_IP> \
  --set k8sServicePort=6443

Performance Considerations

In clusters with fewer than 1,000 Services, iptables mode performs adequately. Beyond that threshold, IPVS mode or an eBPF-based replacement becomes important to avoid increased latency during rule updates and packet processing. Monitoring kube-proxy sync latency metrics (kubeproxy_sync_proxy_rules_duration_seconds) helps identify when to switch modes.