Lesson  in  DevSecOps in Practice: Container & Kubernetes Security

ConfigMaps & Secrets — The Right and Wrong Way

Sensitive configuration baked into a container image is a breach waiting to happen. Walk through the full progression: hardcoded credentials → ConfigMap (still visible) → Kubernetes Secret (base64 is not encryption — live proof) → correct secret consumption patterns. This lab sets up the "why" before the companion course shows you Vault and the External Secrets Operator.

The Wrong Ways — Hardcoded, ConfigMap, and the base64 Lie

Why Configuration Management Is a Security Problem

Every application needs configuration: database hostnames, API endpoints, feature flags, and credentials. Where and how you store that configuration determines your attack surface.

This unit walks through three progressively better — but still flawed — approaches, ending with a live demonstration of why Kubernetes Secrets are not encrypted.

Create the Demo Namespace

kubectl create namespace config-demo

Step 1: Hardcoded Config (The Worst)

cat > hardcoded-pod.yaml << 'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: hardcoded-app
  namespace: config-demo
spec:
  containers:
  - name: app
    image: public.ecr.aws/docker/library/busybox:1.36
    command: ["sh", "-c", "echo 'DB: postgres://admin:S3cr3tP@ssw0rd@db:5432/prod'; sleep 3600"]
    env:
    - name: DB_PASSWORD
      value: "S3cr3tP@ssw0rd"
    resources:
      limits: {cpu: "100m", memory: "64Mi"}
EOF

kubectl apply -f hardcoded-pod.yaml
kubectl wait --for=condition=Ready pod/hardcoded-app -n config-demo --timeout=60s

Find the password with read-only cluster access

# Anyone with "get pods" access can extract the password
kubectl get pod hardcoded-app -n config-demo -o yaml | grep -A2 "env:"

# It's also in the container's environment
kubectl exec -n config-demo hardcoded-app -- env | grep DB_PASSWORD

Expected: DB_PASSWORD=S3cr3tP@ssw0rd — plaintext, visible to anyone who can read pod specs.

This also ends up in git history if the YAML is committed. Credentials committed to git persist forever in history even after deletion — this is what Gitleaks (from the companion course) hunts for.

Step 2: ConfigMap (Better, But Not for Secrets)

cat > app-config.yaml << 'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: config-demo
data:
  DB_HOST: "postgres.config-demo.svc.cluster.local"
  DB_PORT: "5432"
  DB_NAME: "production"
  APP_LOG_LEVEL: "info"
  APP_PORT: "8080"
EOF

kubectl apply -f app-config.yaml
kubectl describe configmap app-config -n config-demo

ConfigMaps are appropriate for non-sensitive configuration. The values are plaintext in etcd and visible to anyone with get configmaps access.

Never put passwords or tokens in a ConfigMap.

Step 3: Kubernetes Secret — The base64 Lie

kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=S3cr3tP@ssw0rd \
  -n config-demo

Prove base64 is NOT encryption

# Get the "secret" password
ENCODED=$(kubectl get secret db-credentials -n config-demo \
  -o jsonpath='{.data.password}')

echo "Encoded:  $ENCODED"
echo "Decoded:  $(echo $ENCODED | base64 -d)"

Expected:

Encoded:  UzNjcjN0UEBzc3cwcmQ=
Decoded:  S3cr3tP@ssw0rd

The password is recoverable by anyone with get secrets access using a one-liner. Kubernetes Secrets are base64-encoded, not encrypted — base64 is an encoding, not a cipher.

Note: etcd encryption at rest is a cluster-admin configuration that encrypts Secrets on disk. But it does not protect against a user with kubectl get secret access — the API server decrypts and returns the value regardless.

Consuming Secrets Correctly — Files, Not Environment Variables

Why Env Vars Are the Wrong Way to Consume Secrets

Mounting a Secret as an environment variable is better than hardcoding — but it still has problems:

  1. Process listings expose env vars — ps e or /proc/<pid>/environ reveals all env vars to any process on the same host
  2. Application crash dumps include env vars — secrets end up in crash reports and logs
  3. Child processes inherit env vars — any subprocess spawned by your app gets the secret
  4. Env vars are visible in kubectl describe pod output

The correct approach: mount the Secret as a file, in a tmpfs volume, with restrictive permissions. The application reads it directly from disk when it needs it.

Deploy with Secret Mounted as a File

cat > secure-config-app.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: secure-config
  namespace: config-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: secure-config
  template:
    metadata:
      labels:
        app: secure-config
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
      containers:
      - name: app
        image: public.ecr.aws/docker/library/busybox:1.36
        command:
        - sh
        - -c
        - |
          echo "=== Config from ConfigMap ==="
          echo "DB_HOST: $DB_HOST"
          echo "DB_PORT: $DB_PORT"
          echo ""
          echo "=== Secret from file (correct) ==="
          echo "Password file exists: $(test -f /run/secrets/db/password && echo YES)"
          echo "Password length: $(wc -c < /run/secrets/db/password) chars"
          echo "Password NOT in env: $(env | grep -c DB_PASSWORD || echo 0) matches"
          sleep 3600
        envFrom:
        - configMapRef:
            name: app-config
        resources:
          requests: {cpu: "50m", memory: "32Mi"}
          limits: {cpu: "100m", memory: "64Mi"}
        volumeMounts:
        - name: db-secrets
          mountPath: /run/secrets/db
          readOnly: true
      volumes:
      - name: db-secrets
        secret:
          secretName: db-credentials
          defaultMode: 0400
EOF

kubectl apply -f secure-config-app.yaml
kubectl rollout status deployment/secure-config -n config-demo --timeout=60s

Verify the Correct Consumption Pattern

POD=$(kubectl get pods -n config-demo -l app=secure-config \
  -o jsonpath='{.items[0].metadata.name}')

# Non-sensitive config comes from ConfigMap as env var — fine
kubectl exec -n config-demo "$POD" -- env | grep DB_HOST

# Secret is NOT in the environment
kubectl exec -n config-demo "$POD" -- env | grep -i password || echo "Not in env — correct!"

# Secret is readable from the volume mount
kubectl exec -n config-demo "$POD" -- cat /run/secrets/db/password

# File permissions — read-only for owner only (0400)
kubectl exec -n config-demo "$POD" -- ls -la /run/secrets/db/

Expected:

DB_HOST=postgres.config-demo.svc.cluster.local
Not in env — correct!
S3cr3tP@ssw0rd
-r--------    1 1000     1000  14 ... password
-r--------    1 1000     1000   5 ... username

The Limits of This Approach

Kubernetes Secrets mounted as files are a significant improvement over env vars and hardcoding. But they still have two weaknesses:

  1. Rotation requires a pod restart — when the Secret object is updated, the mounted file updates within ~60s, but most applications do not detect and reload the file automatically
  2. etcd stores the secret in base64 — anyone with etcd access can read all secrets

This is where HashiCorp Vault + External Secrets Operator (covered in the companion course) comes in:

  • Vault stores secrets encrypted, with audit logs for every access
  • ESO automatically syncs Vault secrets into Kubernetes Secrets
  • Rotation is handled by Vault; applications get the new value without restarts

Configuration Decision Tree

Is this value sensitive (password, token, API key)?
├── YES → Kubernetes Secret (volume mount, not env var)
│         └── Production: Vault + External Secrets Operator
└── NO  → ConfigMap
          ├── Per-environment config: env var injection
          └── Config files: volume mount

Cleanup

kubectl delete namespace config-demo
Previous lesson
Health Probes & Self-Healing
Next lesson
Observability — Detecting Security Anomalies with Prometheus
Start lesson