What Are Custom Resource Definitions (CRDs) in Kubernetes?

Detailed Answer

Why Custom Resource Definitions?

Kubernetes ships with built-in resources like Pods, Services, and Deployments. But real-world platforms need to manage databases, certificates, message queues, and other complex systems. CRDs let you teach Kubernetes about new resource types that represent these systems.

Once a CRD is registered, the API server handles storage, API endpoints, RBAC, kubectl integration, and watch events for the custom resource, just like any built-in resource.

Creating a CRD

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databases.example.com
spec:
  group: example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              required:
                - engine
                - version
                - storage
              properties:
                engine:
                  type: string
                  enum:
                    - postgres
                    - mysql
                    - mongodb
                version:
                  type: string
                storage:
                  type: string
                  pattern: "^[0-9]+Gi$"
                replicas:
                  type: integer
                  minimum: 1
                  maximum: 5
                  default: 1
            status:
              type: object
              properties:
                phase:
                  type: string
                endpoint:
                  type: string
      subresources:
        status: {}
      additionalPrinterColumns:
        - name: Engine
          type: string
          jsonPath: .spec.engine
        - name: Version
          type: string
          jsonPath: .spec.version
        - name: Status
          type: string
          jsonPath: .status.phase
        - name: Age
          type: date
          jsonPath: .metadata.creationTimestamp
  scope: Namespaced
  names:
    plural: databases
    singular: database
    kind: Database
    shortNames:
      - db

# Register the CRD
kubectl apply -f database-crd.yaml

# Verify the CRD exists
kubectl get crd databases.example.com

Creating a Custom Resource

Once the CRD is registered, you can create instances:

apiVersion: example.com/v1
kind: Database
metadata:
  name: production-db
  namespace: default
spec:
  engine: postgres
  version: "16.2"
  storage: 100Gi
  replicas: 3

# Create the custom resource
kubectl apply -f production-db.yaml

# List all databases (using the short name)
kubectl get db
# NAME            ENGINE    VERSION   STATUS   AGE
# production-db   postgres  16.2               5s

# Describe the resource
kubectl describe db production-db

# Edit the resource
kubectl edit db production-db

# Delete the resource
kubectl delete db production-db

Schema Validation

The OpenAPI v3 schema in the CRD enforces validation at the API server level:

# This will be rejected because "oracle" is not in the enum
kubectl apply -f - <<EOF
apiVersion: example.com/v1
kind: Database
metadata:
  name: bad-db
spec:
  engine: oracle
  version: "19c"
  storage: 50Gi
EOF
# Error: spec.engine: Unsupported value: "oracle": supported values: "postgres", "mysql", "mongodb"

The Operator Pattern

A CRD alone is just a data structure stored in etcd. To make it useful, you pair it with a controller (together called an operator) that watches for changes to the custom resource and takes action.

The reconciliation loop:

1. User creates/updates a Database CR
2. Controller detects the change via watch
3. Controller compares desired state (CR spec) with actual state
4. Controller takes action (create StatefulSet, PVC, Service, etc.)
5. Controller updates the CR status with the current state
6. Repeat on any change

Here is a simplified operator structure:

// Simplified reconcile function
func (r *DatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    // Fetch the Database CR
    var db examplev1.Database
    if err := r.Get(ctx, req.NamespacedName, &db); err != nil {
        return ctrl.Result{}, client.IgnoreNotFound(err)
    }

    // Create a StatefulSet for the database
    sts := buildStatefulSet(&db)
    if err := r.Create(ctx, sts); err != nil {
        if !errors.IsAlreadyExists(err) {
            return ctrl.Result{}, err
        }
    }

    // Update status
    db.Status.Phase = "Running"
    db.Status.Endpoint = fmt.Sprintf("%s.%s.svc:5432", db.Name, db.Namespace)
    r.Status().Update(ctx, &db)

    return ctrl.Result{}, nil
}

Popular Operators in Production

| Operator | CRD | Purpose | |---|---|---| | cert-manager | Certificate, Issuer | TLS certificate automation | | Prometheus Operator | Prometheus, ServiceMonitor | Monitoring stack management | | Strimzi | Kafka, KafkaTopic | Kafka cluster management | | CloudNativePG | Cluster | PostgreSQL management | | Argo CD | Application | GitOps deployment |

Building Operators

The two most popular frameworks for building operators:

# Kubebuilder
kubebuilder init --domain example.com
kubebuilder create api --group apps --version v1 --kind Database

# Operator SDK (supports Go, Ansible, Helm)
operator-sdk init --domain example.com
operator-sdk create api --group apps --version v1 --kind Database

RBAC for Custom Resources

Custom resources integrate with Kubernetes RBAC:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: database-admin
rules:
  - apiGroups: ["example.com"]
    resources: ["databases"]
    verbs: ["get", "list", "watch", "create", "update", "delete"]
  - apiGroups: ["example.com"]
    resources: ["databases/status"]
    verbs: ["get", "update"]

Best Practices

1. Always include OpenAPI validation schemas in CRDs to catch errors at admission time.
2. Use the status subresource to separate desired state (spec) from actual state (status).
3. Add printer columns for useful kubectl get output.
4. Use finalizers in controllers to handle cleanup when a CR is deleted.
5. Version your CRDs and provide conversion webhooks when migrating between versions.