Securing Kubernetes Pods with OPA Gatekeeper

Introduction: The Importance of Pod Security

Last year, a fintech startup suffered a major outage after a developer unintentionally deployed a Kubernetes pod with host-level privileges. Within minutes, a malicious actor exploited the misconfiguration, spinning up crypto-mining containers that silently drained cloud resources and forced a full cluster shutdown.

This story isn’t unique. As Kubernetes adoption grows, so does the risk of privilege escalation and resource abuse. Modern clusters often host hundreds—sometimes thousands—of pods managed by distributed teams. One misconfigured YAML file can open the floodgates to attackers or cripple your infrastructure bills.

Who is this article for?
This article is for cloud-native engineers, platform teams, and DevOps professionals who want to take control of Kubernetes security at scale—without bottlenecking development velocity.

What will you learn?

• Why pod security is notoriously hard in Kubernetes
• How to enforce robust, custom security policies cluster-wide
• How to implement and migrate pod security controls using OPA Gatekeeper, a powerful policy engine for Kubernetes
• Best practices, common pitfalls, and practical code examples for real-world deployments

Security Challenges: Privilege Escalation & Resource Abuse

Kubernetes offers flexibility, but with it comes risk. Two of the most critical threats are:

• Privilege Escalation:
  Attackers or misconfigured pods can gain extra privileges—like accessing the host filesystem, running as root, or escaping the container sandbox.

• Resource Abuse:
  Pods without proper limits can starve other workloads, rack up cloud costs, or enable DoS attacks (intentional or accidental).

Real-World Example:
Imagine a team deploying a batch job without CPU/memory limits. The pod spikes to 32 CPUs, throttling production traffic and impacting SLAs—just because one YAML field was left out.

Overview of Kubernetes Admission Controllers

Kubernetes doesn’t enforce security by default. Instead, it delegates enforcement to admission controllers—plugins that intercept API requests before objects (like pods) are persisted.

Types:

• Validating Admission Controllers: Reject/allow objects based on custom logic.
• Mutating Admission Controllers: Modify objects (e.g., inject sidecars).

Why are they important?
They’re your gatekeepers: enforcing organizational policies, blocking dangerous manifests, and ensuring compliance—before anything touches the cluster.

PodSecurityPolicy (PSP) deprecated:
Kubernetes’ built-in PodSecurityPolicy was complex and is now deprecated. Modern clusters need more flexible and maintainable policy engines.

OPA Gatekeeper Architecture: Policy Engine and Constraint Framework

OPA Gatekeeper is a CNCF project that acts as a customizable admission controller for Kubernetes. It lets you codify policies in Rego (a powerful policy language) and apply them cluster-wide.

Key Components:

• OPA (Open Policy Agent): Evaluates policies written in Rego.
• Gatekeeper Controller: Integrates OPA with Kubernetes admission control.
• ConstraintTemplates: Define reusable policy logic.
• Constraints: Apply those templates to specific resources/namespaces.

How it works:

1. User submits a manifest (e.g., a Pod).
2. Admission request hits the Gatekeeper webhook.
3. Gatekeeper evaluates all relevant constraints and policies.
4. Request is accepted or rejected based on matches.

Architecture Diagram:

[User] --> [Kubernetes API Server] --> [Gatekeeper Admission Webhook] --> [OPA Rego Policies] --> [Allow/Deny]

Installing OPA Gatekeeper in Your Cluster

Gatekeeper deploys natively via YAML manifests or Helm.

Installation with kubectl

kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/release-3.14/deploy/gatekeeper.yaml

• Namespace: gatekeeper-system
• Components: Admission webhook, controller manager, audit

Verify Installation:

kubectl -n gatekeeper-system get pods

You should see pods for gatekeeper-controller-manager, audit, etc., all running.

(Optional) Installation with Helm

helm repo add gatekeeper https://open-policy-agent.github.io/gatekeeper/charts
helm install gatekeeper/gatekeeper --generate-name

Defining and Deploying ConstraintTemplates

ConstraintTemplates are how you define reusable, parameterized policies.

Example: Prohibit Privileged Pods

ConstraintTemplate:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8sprohibitprivileged
spec:
  crd:
    spec:
      names:
        kind: K8sProhibitPrivileged
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sprohibitprivileged

        violation[{"msg": msg}] {
          input.review.object.spec.containers[_].securityContext.privileged == true
          msg := "Privileged mode is not allowed."
        }

Deploy the template:

kubectl apply -f k8sprohibitprivileged_template.yaml

Implementing Custom Policies: Resource Limits, Security Contexts, Image Policies

Let’s walk through common, real-world security policies.

1. Enforce Resource Limits

ConstraintTemplate:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequirelimits
spec:
  crd:
    spec:
      names:
        kind: K8sRequireLimits
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequirelimits

        violation[{"msg": msg}] {
          c := input.review.object.spec.containers[_]
          not c.resources.limits.cpu
          msg := sprintf("Container <%v> must set cpu limits.", [c.name])
        }

        violation[{"msg": msg}] {
          c := input.review.object.spec.containers[_]
          not c.resources.limits.memory
          msg := sprintf("Container <%v> must set memory limits.", [c.name])
        }

Constraint:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequireLimits
metadata:
  name: require-limits
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]

Apply:

kubectl apply -f k8srequirelimits_template.yaml
kubectl apply -f require-limits_constraint.yaml

2. Enforce Non-root Containers

ConstraintTemplate:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequirednonroot
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredNonRoot
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequirednonroot

        violation[{"msg": msg}] {
          c := input.review.object.spec.containers[_]
          not c.securityContext.runAsNonRoot
          msg := sprintf("Container <%v> must set runAsNonRoot: true.", [c.name])
        }

3. Restrict Allowed Container Images

ConstraintTemplate:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8sallowedimages
spec:
  crd:
    spec:
      names:
        kind: K8sAllowedImages
      validation:
        openAPIV3Schema:
          properties:
            allowedRepos:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sallowedimages

        violation[{"msg": msg}] {
          allowed := {repo | repo := input.parameters.allowedRepos[_]}
          image := input.review.object.spec.containers[_].image
          not startswith(image, allowed[_])
          msg := sprintf("Image <%v> is not from an allowed repository.", [image])
        }

Constraint:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedImages
metadata:
  name: only-allow-approved-repos
spec:
  parameters:
    allowedRepos:
      - "myregistry.company.com/"
      - "gcr.io/trusted/"
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]

Testing and Validating Policies: Workflow and Violation Handling

1. Attempt to Deploy a Violating Pod

Example: Pod without resource limits.

apiVersion: v1
kind: Pod
metadata:
  name: badpod
spec:
  containers:
    - name: app
      image: nginx

kubectl apply -f badpod.yaml

Result:

Error from server ([require-limits] Container <app> must set cpu limits.): admission webhook "validation.gatekeeper.sh" denied the request...

2. Audit Existing Resources

Gatekeeper can audit existing resources and report violations.

kubectl get constrainttemplates
kubectl get k8srequirelimits

Check for violations:

kubectl get constraintstatus

3. Real-World Use Case: Staging vs. Production

• In staging, you might allow broader access for experimentation.
• In production, Gatekeeper can enforce strict policies—e.g., only images from signed, trusted registries, all pods must set resource limits, and no privileged containers.

Recommended workflow:

• Test all policies in a non-production cluster.
• Use Gatekeeper's dry-run and audit modes to preview impacts.
• Monitor audit reports; iterate and tune constraints before strict enforcement.

Migrating from PodSecurityPolicy to OPA Gatekeeper

With PodSecurityPolicy deprecated, Gatekeeper is the recommended path forward.

Migration steps:

1. Inventory existing PSPs
   List your current PodSecurityPolicies and capture their intent.

2. Map PSP rules to ConstraintTemplates
   For common patterns (privileged, hostPath mounts, capabilities), see psp-migration templates.

   Example: Migrate a PSP enforcing runAsNonRoot:

   PodSecurityPolicy	Gatekeeper ConstraintTemplate
   runAsNonRoot: true	K8sRequiredNonRoot ConstraintTemplate

3. Apply, Test, and Audit
   Deploy templates and constraints incrementally. Use audit mode to avoid blocking workloads unintentionally.

4. Decommission PSP
   Once policies are enforced and validated, remove PSP from your cluster.

Gotcha:
OPA Gatekeeper and PSP differ in syntax and behavior—always test migrated policies to avoid regressions.

Best Practices and Common Pitfalls

Best Practices:

• Start with audit mode. Avoid breaking existing apps by enforcing new policies gradually.
• Use version control for policies. Treat ConstraintTemplates and constraints as code—PR reviews, CI/CD, rollbacks.
• Document intent. Each policy should have a clear business/security reason.
• Scope policies appropriately. Use match fields to target only needed namespaces or resources.
• Monitor Gatekeeper health. Admission webhooks are critical-path; monitor latency and availability.

Common Pitfalls:

• Overly broad constraints: Can block essential system pods or third-party controllers.
• Policy drift: If teams bypass constraints (e.g., by applying manifests directly to the API server), you’ll lose enforcement.
• Neglecting updates: Gatekeeper and Rego have regular updates; old policies may become incompatible.
• Performance: Hundreds of complex constraints can increase admission latency. Profile and optimize as needed.

Conclusion and Next Steps

Securing Kubernetes pods is non-trivial, but with OPA Gatekeeper, you can codify organizational policies as code and enforce them at scale—reducing risks of privilege escalation and resource abuse.

Key takeaways:

• Kubernetes admission controllers are your first line of defense.
• OPA Gatekeeper decouples security intent from engineering velocity.
• Real-world policies: enforce resource limits, block privileged pods, restrict images.
• Migrate carefully from deprecated PodSecurityPolicies using audit and gradual rollout.
• Treat security policies as code—review, monitor, and iterate.

Explore next:

• OPA Gatekeeper official docs
• Open-source policy libraries
• Integrating Gatekeeper with CI/CD for policy testing
• Advanced policy authoring with Rego

Secure your cluster—before someone else does it for you.