Chapter 04: GitOps & Version Promotion
Learning Objectives
By the end of this chapter, you will be able to:
- Describe the Flux reconciliation model and environment overlays
- Promote immutable images across develop, staging, and production without rebuild
- Execute rollback by Git evidence rather than ad-hoc rebuilds
- Configure Kustomize overlays for multi-environment deployment
Start with the video for the concept overview, then work through each lesson section.
Production safety depends on controlled promotion, not ad-hoc rebuilds. In this chapter, we implement a deployment model where every change is auditable, immutable, and automated via FluxCD.
1. The Problem: The “Rebuild” Trap
During an incident, rebuilding code for production introduces the risk of dependency drift or build-time variance. This creates ambiguity: the version running in production might not be identical to the one tested in staging, making rollbacks unpredictable.
2. The Concept: Immutable Promotion
We build an artifact once, test it in lower environments, and promote the exact same image to production. Git remains the single source of truth, and all cluster updates are triggered by Git commits rather than manual commands.
3. The Code: Automated Image Policies
We use Flux ImageRepository and ImagePolicy objects to discover new tags and select the correct one for each environment based on immutable versioning patterns.
Image automation layout
---
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
name: backend
namespace: flux-system
spec:
image: ${image_registry}/backend
interval: 1m0s
secretRef:
name: ghcr-credentials-docker
accessFrom:
namespaceSelectors:
- matchLabels:
environment: development
- matchLabels:
environment: staging
- matchLabels:
environment: production
4. The Guardrail: GitOps Rollback
Because Flux commits all image updates back to Git, your rollback path is a simple git revert. This ensures that even during a high-pressure incident, you have a clean, auditable path to restore a known-good state.
GitOps workflow guide
Show the GitOps promotion guide
## Overview
As of February 16, 2026, the active deployment model is:
- **Develop** namespace auto-updated from env-tagged `develop-*` images.
- **Staging** namespace auto-updated from env-tagged `staging-*` images.
- **Production** namespace auto-updated from env-tagged `production-*` images created by manual promotion workflows.
Flux sync source:
- Git repository branch: `main`
- Path: `./flux/bootstrap/flux-system`
## Actual Image Tagging Strategy
Backend and frontend build workflows publish multiple tags per build:
- Environment alias: `develop` or `staging`
- Immutable env/version tag: `<env>-v<major>.<minor>.<patch>-<short_sha>-<unix_ts>`
- Commit tag: `<short_sha>`
Examples:
- `develop-v0.0.1-a1b2c3d-1738860000`
- `staging-v0.0.1-a1b2c3d-1738860123`
- `production-v0.0.1-a1b2c3d-1738861000` (from promotion workflow)
Production promotion workflows also maintain alias tag `production`.
## CI/CD to Flux Flow
### 1. Build (develop branch)
- Trigger: push to `develop` in service repos (`backend` or `frontend`).
- Workflow builds and pushes `develop-*` tags to GHCR.
- Flux `ImagePolicy` in namespace `develop` selects latest matching tag by extracted timestamp.
- Flux `ImageUpdateAutomation` commits setter updates into this repo (`main`).
- Flux applies the new image tag to `develop`.
### 2. Build (main branch)
- Trigger: push to `main` in service repos.
- Workflow builds and pushes `staging-*` tags to GHCR.
- Flux `ImagePolicy` in namespace `staging` selects latest matching tag.
- Flux writes the updated tag to Git and reconciles `staging`.
### 3. Promotion to production
- Trigger: manual `workflow_dispatch` in service repo (`promote-production.yml`).
- Workflow chooses a `staging-*` tag (explicit input or latest), then retags to:
- `production`
- `production-v<major>.<minor>.<patch>-<short_sha>-<unix_ts>`
- Flux `ImagePolicy` in namespace `production` matches `production-*` and deploys automatically.
- The promotion workflow also creates/publishes GitHub Release metadata and bumps next version tag.
## Flux Objects Used (Current State)
The active implementation uses Flux Image Automation (Git write-back), not ResourceSet runtime mutation.
- `ImageRepository` objects in `flux-system`:
- `flux/bootstrap/infrastructure/image-automation/backend-image-repo.yaml`
- `flux/bootstrap/infrastructure/image-automation/frontend-image-repo.yaml`
- `ImagePolicy` objects per env:
- backend: `flux/apps/backend/develop|staging|production/image-policy.yaml`
- frontend: `flux/apps/frontend/overlays/develop|staging|production/image-policy.yaml`
- `ImageUpdateAutomation` objects per env:
- backend: `flux/apps/backend/develop|staging|production/image-automation.yaml`
- frontend: `flux/apps/frontend/overlays/develop|staging|production/image-automation.yaml`
- `GitRepository` source for write-back:
- `flux/bootstrap/infrastructure/image-automation/git-repository.yaml`
Note: ResourceSet examples are currently commented out in `flux/bootstrap/apps/*`.
## Regex Policies in Use
Backend and frontend use the same tag filters per environment:
- develop: `^develop-v[0-9]+\.[0-9]+\.[0-9]+-[a-f0-9]+-(?P<ts>[0-9]+)$`
- staging: `^staging-v[0-9]+\.[0-9]+\.[0-9]+-[a-f0-9]+-(?P<ts>[0-9]+)$`
- production: `^production-v[0-9]+\.[0-9]+\.[0-9]+-[a-f0-9]+-(?P<ts>[0-9]+)$`
Policies extract `ts` and choose the latest numerically.
## Deployment Verification
```bash
## Rollback Paths
Preferred rollback is GitOps-first:
1. Revert the Flux bot commit in this repository (`main`) that bumped the image tag.
2. Let Flux reconcile the reverted manifest.
Emergency rollback can use `kubectl rollout undo`, but that may drift from Git and should be reconciled back via Git immediately after.
## Troubleshooting
```bash
## Security Notes
1. Use least-privilege credentials for Flux Git and registry access.
2. Keep network isolation between `develop`, `staging`, and `production`.
3. Keep auditability: all image changes should be traceable through Git commits and workflow runs.
## Additional Resources
- [Flux Documentation](https://fluxcd.io/flux/)
- [FluxCD Image Automation](https://fluxcd.io/flux/guides/image-update/)
- [GitHub Actions Documentation](https://docs.github.com/en/actions)
- [Kustomize Documentation](https://kubectl.docs.kubernetes.io/references/kustomize/)
5. Verification: Did I Get It?
Verify the current state of image automation and the running versions in your cluster:
flux get images all -A
kubectl -n staging get deploy backend -o jsonpath='{.spec.template.spec.containers[0].image}'