What Are Image Scanning Best Practices for Kubernetes?

intermediate|securitydevopssrebackend developerCKACKAD
TL;DR

Image scanning detects known vulnerabilities (CVEs), misconfigurations, and secrets in container images before they run in a cluster. Best practices include scanning in CI/CD pipelines, blocking vulnerable images with admission control, and using minimal base images.

Detailed Answer

Image scanning analyzes container images for known vulnerabilities, misconfigurations, exposed secrets, and license compliance issues. It is a critical component of Kubernetes security that should be automated throughout the software delivery lifecycle.

Where to Scan

Development → CI/CD Pipeline → Registry → Admission → Runtime
     ↓              ↓              ↓           ↓          ↓
  IDE plugin    Build-time      Scheduled   Webhook    Continuous
  (developer    scan (block     registry    (block     monitoring
   feedback)    the build)      scan        deployment) (new CVEs)

Popular Scanning Tools

| Tool | Type | Strengths | |------|------|-----------| | Trivy | CLI/CI/Operator | Fast, comprehensive, Kubernetes-native | | Grype | CLI/CI | Fast, Syft-based SBOM integration | | Snyk | SaaS/CLI | Developer-friendly, fix suggestions | | Clair | Registry scanner | Self-hosted, integrates with registries | | Prisma Cloud (Twistlock) | Enterprise | Full lifecycle, runtime protection |

CI/CD Pipeline Scanning with Trivy

# GitHub Actions example
name: Build and Scan
on: push

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build image
        run: docker build -t myapp:${{ github.sha }} .

      - name: Scan with Trivy
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: myapp:${{ github.sha }}
          format: table
          exit-code: 1                    # Fail the build
          severity: CRITICAL,HIGH         # Only fail on critical/high
          ignore-unfixed: true            # Skip CVEs without fixes
# CLI scanning
trivy image --severity HIGH,CRITICAL --exit-code 1 myapp:latest

# Scan with output for CI systems
trivy image --format json --output scan-results.json myapp:latest

# Scan for secrets and misconfigurations too
trivy image --scanners vuln,secret,misconfig myapp:latest

Registry Scanning

Configure your container registry to scan images on push:

  • AWS ECR: Enable "scan on push" in repository settings
  • GCR/Artifact Registry: Automatic scanning with Container Analysis API
  • Harbor: Built-in Trivy integration
  • Docker Hub: Docker Scout scanning

Admission-Time Enforcement

Block vulnerable images from running using a validating webhook or policy engine:

# Kyverno policy to require scanned images
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-image-scan
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-vulnerabilities
      match:
        any:
          - resources:
              kinds:
                - Pod
      verifyImages:
        - imageReferences:
            - "*"
          attestations:
            - predicateType: cosign.sigstore.dev/attestation/vuln/v1
              conditions:
                - all:
                    - key: "{{ scanner.result.summary.criticalCount }}"
                      operator: Equals
                      value: "0"

Image Hardening Best Practices

1. Use Minimal Base Images

# Bad: Full OS with hundreds of packages and CVEs
FROM ubuntu:22.04

# Better: Minimal Alpine
FROM alpine:3.19

# Best: Distroless (no shell, no package manager)
FROM gcr.io/distroless/static-debian12

# Or: Scratch for statically compiled binaries
FROM scratch
COPY myapp /myapp
ENTRYPOINT ["/myapp"]

Vulnerability count comparison:

  • ubuntu:22.04 — ~100-200+ CVEs
  • alpine:3.19 — ~10-20 CVEs
  • distroless/static — ~0-5 CVEs

2. Pin Image Versions

# Bad: Tag can change, no reproducibility
image: nginx:latest

# Better: Specific version
image: nginx:1.27.0

# Best: SHA256 digest (immutable)
image: nginx@sha256:abc123...

3. Multi-Stage Builds

# Build stage (contains build tools, test deps)
FROM golang:1.22 AS builder
WORKDIR /app
COPY . .
RUN CGO_ENABLED=0 go build -o /myapp

# Production stage (minimal)
FROM gcr.io/distroless/static-debian12
COPY --from=builder /myapp /myapp
ENTRYPOINT ["/myapp"]

The production image contains only the compiled binary — no Go compiler, no build tools, no source code.

4. Run as Non-Root

FROM alpine:3.19
RUN adduser -D -u 1000 appuser
USER appuser
COPY --chown=appuser:appuser myapp /app/myapp
ENTRYPOINT ["/app/myapp"]

Continuous Monitoring with Trivy Operator

The Trivy Operator runs inside Kubernetes and continuously scans running workloads:

helm install trivy-operator aquasecurity/trivy-operator \
  --namespace trivy-system --create-namespace

It creates VulnerabilityReport CRDs for each workload:

# Check vulnerability reports
kubectl get vulnerabilityreports -A

# View details for a specific workload
kubectl get vulnerabilityreport -n production \
  -l trivy-operator.resource.name=api-server -o yaml

Scanning Checklist

  1. Scan in CI/CD pipeline — fail builds on critical/high CVEs with available fixes
  2. Scan on registry push — catch images that bypass CI/CD
  3. Enforce via admission control — block unscanned or vulnerable images
  4. Monitor continuously — detect newly published CVEs in running images
  5. Use minimal base images — reduce attack surface
  6. Pin image digests — prevent tag mutation attacks
  7. Generate and store SBOMs — know what is inside every image
  8. Set up alerting — notify teams when new critical CVEs affect their images

Why Interviewers Ask This

Vulnerable container images are one of the top attack vectors in Kubernetes. This question tests whether you integrate security into the software delivery pipeline rather than treating it as an afterthought.

Common Follow-Up Questions

What is the difference between scanning in CI/CD vs. at runtime?
CI/CD scanning catches known vulnerabilities before deployment. Runtime scanning detects newly discovered CVEs in already-running images. Both are needed for defense in depth.
Should you block all images with vulnerabilities?
No — block only critical and high severity CVEs with known fixes. Many CVEs have no fix or are not exploitable in your context. Use an exception process for accepted risks.
What are distroless images and why do they help?
Distroless images contain only the application and its runtime dependencies — no shell, package manager, or OS utilities. This dramatically reduces the attack surface.

Key Takeaways

  • Scan images in CI/CD to catch vulnerabilities before they reach the cluster.
  • Use admission control to block images with critical CVEs from running.
  • Use minimal base images (distroless, Alpine) to reduce the vulnerability surface area.

Related Questions

You Might Also Like