Advanced Track Do this after finishing Chapters 01-14.

Estimated Time

  • Reading: 30-40 min
  • Lab: 60-90 min
  • Quiz: 15-20 min

Prerequisites

  • Core track (Chapters 01-14) completed.
  • GitOps promotion and observability workflows available.

Source Code References

  • verify-attestations.example.yaml Members
  • verify-images.example.yaml Members

Sign in to view source code.

What You Will Produce

A go/no-go evidence package: rollout results, remediation notes, and explicit rollback conditions.

Chapter 15: Supply Chain Security

Learning Objectives

By the end of this chapter, you will be able to:

  • Generate SBOM and sign container images with attestation evidence
  • Verify image attestation at admission time before deployment
  • Plan an audit-first rollout toward enforceable supply-chain trust
  • Trace the supply chain trust boundary from build to deploy

Start with the video for the concept overview, then work through each lesson section.

A successful CI build does not guarantee runtime trust. If you cannot prove where an image came from or what is inside it, you cannot trust your production environment. In this chapter, we enforce a strict production rule: only verifiable artifacts may run.


1. The Problem: The “Workstation Rebuild”

An urgent fix is rebuilt from a developer’s workstation and pushed with a familiar tag. During an incident, the team realizes they cannot prove the provenance or safety of that binary. Without cryptographic trust, every “quick fix” is a potential supply-chain vulnerability.

2. The Concept: Verifiable Artifacts

We extend our GitOps model into the world of cryptographic signatures and attestations.

  1. Build Once, Promote Many: The exact same digest is tested and promoted across all environments.
  2. Attestations: We attach verifiable evidence (like SBOMs and build provenance) to our artifacts.
  3. Cluster Enforcement: The Kubernetes cluster becomes the final gatekeeper, rejecting any image that lacks a valid signature from an authorized signer.

3. The Code: Signature Verification Policy

Our sre/ repo uses Kyverno to enforce supply-chain integrity. The image verification policy is our contract for runtime trust.

Image verification policy

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-signed-images-example
spec:
  validationFailureAction: Audit
  background: false
  rules:
    - name: verify-ghcr-signatures
      match:
        any:
          - resources:
              kinds:
                - Pod
      verifyImages:
        - imageReferences:
            - "${image_registry}/*"
          mutateDigest: true
          attestors:
            - entries:
                - keyless:
                    # Replace with your CI issuer and subject constraints.
                    issuer: "https://token.actions.githubusercontent.com"
                    subject: "https://github.com/${git_owner}/*"

4. The Guardrail: Audit-to-Enforce

We never block production deployments without evidence. We roll out supply-chain protections in two phases:

  • Audit Mode: Violations are recorded in reports but allowed to run. This allows us to catch configuration gaps without causing downtime.
  • Enforce Mode: Once the baseline is clean, we move to blocking all untrusted artifacts at admission.

Attestation verification policy

Show the attestation verification policy
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-sbom-attestation-example
spec:
  validationFailureAction: Audit
  background: false
  rules:
    - name: verify-spdx-attestation
      match:
        any:
          - resources:
              kinds:
                - Pod
      verifyImages:
        - imageReferences:
            - "${image_registry}/*"
          attestations:
            - type: "https://spdx.dev/Document"
              attestors:
                - entries:
                    - keyless:
                        issuer: "https://token.actions.githubusercontent.com"
                        subject: "https://github.com/${git_owner}/*"

5. Verification: Did I Get It?

Verify your artifact identity and check for policy compliance:

# Check the digest of your running image
kubectl get pods -n develop -o jsonpath='{.items[*].status.containerStatuses[*].imageID}'
# Verify the signature locally
cosign verify <image-reference> --key <public-key>

Expected Output: A successful verification showing the signer’s identity and the artifact’s digest.


Detailed Lessons

Hands-On Materials

Labs, quizzes, and runbooks — available to course members.

  • Lab: Signed Artifacts and Verification Policy Rollout (Advanced) Members
  • Quiz: Chapter 15 (Supply Chain Security) Members
  • Runbook: Supply Chain Verification (Advanced) Members
  • Supply Chain Security Scorecard (Template) Members

The Incident: Workstation Rebuild

Result: Rollback confidence drops because the trust lineage of the running artifact is uncertain. Recovery delay shifts from a technical problem to a trust problem. Observed Symptoms What the team sees first: The …

Investigation & Containment

Safe investigation sequence: Verify Digest &amp; Signature: Check the digest and cryptographic signature for the running image. Confirm SBOM Match: Ensure the SBOM belongs to the exact same immutable artifact. Compare …

Workflow & Verification

Kyverno: The policy engine installed and reconciled via Flux. Cosign: For signing and verifying container images. Attestation Templates: Pre-defined policies for provenance and SBOM checks. Image verification policy …

Lab & Completion

Done When You have completed this chapter when: You can explain why &ldquo;build once, promote many&rdquo; is required for provenance. You have successfully signed and attested a container image. You have demonstrated …