How Does mTLS Work in a Service Mesh?

Detailed Answer

Mutual TLS (mTLS) is a protocol where both the client and server authenticate each other using X.509 certificates before establishing an encrypted connection. In a service mesh, mTLS is applied transparently to all pod-to-pod communication.

How mTLS Differs from Standard TLS

| Aspect | Standard TLS | Mutual TLS | |---|---|---| | Server authenticates to client | Yes | Yes | | Client authenticates to server | No | Yes | | Encryption | Yes | Yes | | Use case | HTTPS websites | Service-to-service in zero-trust | | Certificate requirement | Server only | Both client and server |

How It Works in a Service Mesh

Pod A                                            Pod B
┌──────────────┐                    ┌──────────────┐
│  App (plain   │                    │  App (plain   │
│  HTTP :8080)  │                    │  HTTP :8080)  │
│      │        │                    │      ▲        │
│      ▼        │                    │      │        │
│  Sidecar      │ ◄── mTLS ──────► │  Sidecar      │
│  Proxy        │    (encrypted)     │  Proxy        │
│  (Envoy)      │                    │  (Envoy)      │
└──────────────┘                    └──────────────┘

1. App A sends a plaintext HTTP request to localhost
2. The sidecar proxy intercepts the outbound request
3. The proxy initiates a TLS handshake with Pod B's sidecar
4. Both proxies exchange and verify X.509 certificates
5. An encrypted tunnel is established
6. The request is forwarded encrypted
7. Pod B's sidecar decrypts and forwards to the app on localhost

Enabling mTLS in Istio

Permissive Mode (Migration)

Accept both plaintext and mTLS:

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  mtls:
    mode: PERMISSIVE

Strict Mode (Production)

Require mTLS for all communication:

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  mtls:
    mode: STRICT

Per-Service Override

Disable mTLS for a specific workload (e.g., a legacy service):

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: legacy-exception
  namespace: production
spec:
  selector:
    matchLabels:
      app: legacy-service
  mtls:
    mode: DISABLE

Enabling mTLS in Linkerd

Linkerd enables mTLS by default for all meshed workloads with no additional configuration:

# Inject the Linkerd proxy
kubectl get deploy api -o yaml | linkerd inject - | kubectl apply -f -

# Verify mTLS is active
linkerd viz tap deploy/api
# req id=0:0 proxy=in  src=10.244.1.5:38472 dst=10.244.2.8:8080 tls=true

Certificate Lifecycle

The service mesh automates certificate management:

1. Pod starts → Sidecar requests certificate from mesh CA
2. CA issues short-lived X.509 certificate (e.g., 24h validity)
3. Certificate contains workload identity:
   spiffe://cluster.local/ns/production/sa/api-service
4. Sidecar uses certificate for all mTLS connections
5. Before expiry → Sidecar automatically requests renewal
6. Old certificate → Rotated out seamlessly

# View certificate details in Istio
istioctl proxy-config secret deploy/api -o json | jq '.dynamicActiveSecrets[0].secret.tlsCertificate'

# Check certificate expiration
istioctl proxy-config secret deploy/api | grep VALID
# default    2026-03-19T12:00:00Z   2026-03-20T12:00:00Z   ACTIVE

SPIFFE Identity

Service meshes use the SPIFFE standard for workload identity:

spiffe://cluster.local/ns/<namespace>/sa/<service-account>

This identity is embedded in the X.509 certificate and used for authorization decisions:

# Istio AuthorizationPolicy using SPIFFE identity
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: api-authz
  namespace: production
spec:
  selector:
    matchLabels:
      app: api
  rules:
    - from:
        - source:
            principals:
              - "cluster.local/ns/production/sa/frontend"
      to:
        - operation:
            methods: ["GET"]

Performance Considerations

| Factor | Impact | Mitigation | |---|---|---| | TLS handshake latency | 1-3ms per new connection | Connection pooling, keep-alive | | CPU for encryption | 5-10% overhead | Hardware AES acceleration | | Memory for sidecar | 50-100MB per Pod | Tune sidecar resource limits | | Certificate rotation | Negligible | Handled in background |

Verifying mTLS

# Istio: Check if mTLS is active between services
istioctl authn tls-check deploy/api.production

# Check for plaintext connections (should be none in strict mode)
istioctl proxy-config listeners deploy/api --port 8080

# Linkerd: Verify mTLS with tap
linkerd viz tap deploy/api --to deploy/database