GitOps with Argo CD — Every Change Auditable

What Is GitOps?

GitOps is a deployment model with one rule: git is the single source of truth for cluster state.

From a security perspective, GitOps enforces change control at the infrastructure level. Every cluster change is a git commit with author, timestamp, and reviewer — meeting the audit requirements of PCI-DSS, SOC 2, and ISO 27001.

Install Argo CD

kubectl create namespace argocd

kubectl apply -n argocd -f \
  https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Wait for Argo CD to be ready
kubectl wait --for=condition=Ready pod \
  -l app.kubernetes.io/name=argocd-server \
  -n argocd \
  --timeout=180s

Connect to Argo CD

# Port-forward the Argo CD API server
kubectl port-forward svc/argocd-server -n argocd 8080:443 &>/dev/null &
sleep 3

# Get the initial admin password
ARGOCD_PASSWORD=$(kubectl get secret argocd-initial-admin-secret \
  -n argocd \
  -o jsonpath='{.data.password}' | base64 -d)

echo "Password: $ARGOCD_PASSWORD"

# Login
argocd login localhost:8080 \
  --username admin \
  --password "$ARGOCD_PASSWORD" \
  --insecure

Create a Namespace for the Application

kubectl create namespace gitops-demo

Deploy an Application via Argo CD

We will use a public example app. In production, this would be your private git repository:

argocd app create demo-app \
  --repo https://github.com/argoproj/argocd-example-apps.git \
  --path guestbook \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace gitops-demo \
  --sync-policy automated \
  --auto-prune \
  --self-heal

# Watch the application sync
argocd app get demo-app
argocd app wait demo-app --health --timeout 120

Inspect the Audit Trail

# See all resources Argo CD manages
argocd app resources demo-app

# See the sync history (every deployment is recorded)
argocd app history demo-app

In a production environment, the sync history ties every cluster state to a specific git commit SHA — giving you the exact code, the deployer's identity, and the timestamp for every change.

The Problem with Manual kubectl in Production

Every kubectl apply that happens outside of git creates drift: the cluster state diverges from the declared state. Without GitOps, drift is invisible and accumulates silently.

Common drift sources:

• A developer patches a bug directly in prod under pressure
• An ops team member scales a deployment manually and forgets to update the YAML
• An attacker who gains kubectl access modifies a deployment to run their code

With Argo CD's --self-heal flag enabled, drift is not just detected — it is automatically corrected.

Simulate Drift: A Manual kubectl Change

The application is synced. Now make an unauthorised change directly to the cluster:

# Scale the deployment to 0 (simulate an attacker or mistake)
kubectl scale deployment guestbook-ui -n gitops-demo --replicas=0

# Immediately check what Argo CD sees
sleep 5
argocd app get demo-app | grep -E "Sync Status|Health|OutOfSync"

Expected: Sync Status: OutOfSync

Argo CD detects within seconds that the cluster no longer matches git.

Watch Self-Heal in Action

Because we created the app with --self-heal, Argo CD automatically reconciles:

# Watch the app sync back
argocd app wait demo-app --sync --health --timeout 60
kubectl get deployment guestbook-ui -n gitops-demo

Expected: Deployment restored to the replica count declared in git. Your manual change was overwritten.

This is the security property: an attacker with kubectl access cannot make a persistent change if Argo CD is watching. They can scale down a deployment, but it comes back within seconds.

The Security Audit Log

# See every sync with timestamp and result
argocd app history demo-app

# Get detailed events for the last sync
kubectl get events -n gitops-demo --sort-by='.lastTimestamp' | tail -20

In production, Argo CD sends sync events to a webhook (Slack, PagerDuty, SIEM). An unexpected sync — especially one that was not triggered by a git commit — is an alert worth investigating.

GitOps Security Checklist

✓ All cluster changes flow through git (PRs with review)
✓ --self-heal enabled (drift auto-corrected)
✓ --auto-prune enabled (deleted from git = deleted from cluster)
✓ Argo CD RBAC configured (not everyone can sync to prod)
✓ Argo CD sync webhook notifies security team
✓ kubectl direct access to prod restricted (RBAC + audit logging)

Manual Sync When You Need It

Self-heal auto-corrects drift — but sometimes you need a deliberate manual sync (e.g., after a git push):

# Trigger a manual sync
argocd app sync demo-app

# Sync a specific resource only
argocd app sync demo-app --resource apps:Deployment:guestbook-ui

Cleanup

argocd app delete demo-app --yes
kubectl delete namespace gitops-demo
# Leave argocd running for Kyverno lab