SafeOps Academy

Chapter 15: Admission Policy Guardrails (Advanced)

Chapter 15: Admission Policy Guardrails (Advanced)

Why This Chapter Exists

Local checks (pre-commit, CI, review) reduce risk but can be bypassed. Admission control is the last enforcement point before runtime.

This chapter focuses on policy-as-code guardrails that block risky workloads even when upstream checks fail.

Learning Objectives

By the end of this chapter, learners can:

  • explain why cluster-side policy is mandatory in production systems
  • roll out Kyverno rules with Audit -> Enforce safely
  • troubleshoot deny events and remediate manifests correctly
  • run controlled break-glass exceptions with expiry and audit trail

The Incident Hook

A workload is deployed during incident pressure with missing limits, mutable tags, and weak security context. Workstation hooks were skipped and review focused on speed. The pod starts in a risky configuration and causes noisy-neighbor impact. Recovery is slowed because the team lacks clear deny/exception discipline.

What AI Would Propose (Brave Junior)

  • “Disable the policy engine temporarily.”
  • “Allow privileged mode now, fix later.”
  • “Create a broad exception for the whole namespace.”

Why this sounds reasonable:

  • immediate progress under pressure
  • lower friction in the moment

Why This Is Dangerous

  • Security and stability regressions reach runtime.
  • “Temporary” exceptions become long-term drift.
  • Platform trust model is weakened for all teams.

Guardrails That Stop It

  • Policy engine always-on (Kyverno).
  • Default rollout path: Audit then selective Enforce.
  • Exceptions must be scoped, time-bound, and approved.
  • Deny evidence is mandatory before policy changes.

Current Platform State

  • Kyverno engine is active via Flux: flux/infrastructure/policy/kyverno/
  • Chapter 15 policies are scaffolded and inactive by default: flux/infrastructure/policy/packs/chapter-15-admission-guardrails/

Repository Mapping

  • Engine:
    • flux/infrastructure/policy/kyverno/kustomization.yaml
    • flux/infrastructure/policy/kyverno/release.yaml
  • Starter pack templates:
    • flux/infrastructure/policy/packs/chapter-15-admission-guardrails/disallow-latest-tag.example.yaml
    • flux/infrastructure/policy/packs/chapter-15-admission-guardrails/require-security-context.example.yaml
    • flux/infrastructure/policy/packs/chapter-15-admission-guardrails/require-requests-limits.example.yaml
  • Workloads under control: flux/apps/**/

Safe Workflow (Step-by-Step)

  1. Enable selected policies in Audit.
  2. Trigger known violations intentionally in develop.
  3. Review policy reports and event messages.
  4. Fix manifests, not engine settings.
  5. Move stable rules to Enforce in non-production.
  6. Promote enforcement gradually across environments.

Lab Files

  • lab.md
  • runbook-admission-policy.md
  • quiz.md

Done When

  • learner demonstrates Audit -> Enforce with clear evidence
  • learner can perform deny triage and manifest remediation
  • learner can apply a safe exception process without global bypass

Lab: Admission Guardrails in Audit and Enforce Modes (Advanced)

Lab: Admission Guardrails in Audit and Enforce Modes (Advanced)

Goal

Run a practical Kyverno workflow in develop:

  • enable starter policies in Audit
  • trigger and inspect violations
  • switch one policy to Enforce
  • validate deny -> fix -> admit behavior

Prerequisites

  • Kyverno engine running in cluster
  • access to develop namespace
  • starter templates available:
    • flux/infrastructure/policy/packs/chapter-15-admission-guardrails/disallow-latest-tag.example.yaml
    • flux/infrastructure/policy/packs/chapter-15-admission-guardrails/require-security-context.example.yaml
    • flux/infrastructure/policy/packs/chapter-15-admission-guardrails/require-requests-limits.example.yaml

Quick checks:

kubectl -n kyverno get pods
kubectl get ns develop
kubectl get cpol

Step 1: Apply Starter Policies in Audit Mode

kubectl apply -f flux/infrastructure/policy/packs/chapter-15-admission-guardrails/disallow-latest-tag.example.yaml
kubectl apply -f flux/infrastructure/policy/packs/chapter-15-admission-guardrails/require-security-context.example.yaml
kubectl apply -f flux/infrastructure/policy/packs/chapter-15-admission-guardrails/require-requests-limits.example.yaml

Confirm mode is Audit:

Quiz: Chapter 15 (Admission Policy Guardrails)

Quiz: Chapter 15 (Admission Policy Guardrails)

Questions

  1. Why is admission policy called the “last safety gate”?

  2. What is the safest rollout order for new blocking policies?

  3. What does validationFailureAction: Audit do?

  4. What does validationFailureAction: Enforce do?

  5. Which statement is correct?

  • A) If pre-commit passes, admission control is unnecessary.
  • B) Admission control can still block risky manifests after merge.
  • C) Fastest fix is always disabling policy engine.

  1. Name two risky patterns from Chapter 15 starter pack.

    Runbook: Admission Policy Operations (Advanced)

    Runbook: Admission Policy Operations (Advanced)

    Purpose

    Operate Kyverno guardrails safely while keeping deployment velocity and strong auditability.

    Scope

    This runbook covers:

    • policy engine health checks
    • Audit and Enforce rollout operations
    • deny triage and manifest remediation
    • break-glass exception discipline

    Health and Inventory Checks

    kubectl -n kyverno get pods
kubectl get cpol
kubectl get policyreport -A

    If engine pods are not healthy, pause policy mode changes.

    Operational Policy Modes

    • Audit: violations are reported but admission is allowed.
    • Enforce: violating resources are blocked at admission.

    Check policy mode: