Lesson  in  DevSecOps in Practice: Container & Kubernetes Security

Supply-Chain Security — Image Signing & Verification

Even if your image passes Trivy scanning, an attacker who gains registry write access can push a malicious image with the same tag. Image signing with cosign adds a cryptographic signature — and a Kyverno policy enforces that only signed images run in your cluster. An unsigned image is rejected at admission time, even if the tag and digest look correct.

Signing Images with cosign

The Gap Scanning Cannot Fill

Trivy scans images for CVEs — but it cannot protect against this attack:

An attacker gains registry write access and pushes a malicious image with the same tag. Trivy scans it, finds no CVEs (the malicious image was crafted to be clean), and passes it. The tag and digest look correct. But the image runs a cryptominer.

Image signing with cosign closes this gap. Every image your CI pipeline builds is signed with a private key. The cluster refuses to run any image without a valid signature from that key — even if the tag and digest match.

How cosign Works

CI Pipeline                      Kubernetes Cluster
───────────                      ──────────────────
1. Build image                   4. Kyverno webhook calls cosign verify
2. Push to registry              5. Valid signature  → pod starts
3. cosign sign (private key)        No signature     → pod rejected
   → stores signature in registry

Generate a Signing Key Pair

# Empty passphrase is fine for labs
COSIGN_PASSWORD="" cosign generate-key-pair
ls ~/cosign.key ~/cosign.pub
  • cosign.key — private signing key. In production: store in Vault, AWS KMS, or GCP KMS. Never commit to git.
  • cosign.pub — public verification key. Safe to share and commit.

Set Your Lab ID

We use ttl.sh — a free, anonymous HTTPS registry that requires no account. Images expire after 2 hours, which is fine for a lab session.

export LAB_ID=$(openssl rand -hex 4)
echo "LAB_ID=${LAB_ID}"
echo "${LAB_ID}" > ~/lab_id.txt

Push and Sign an Image

SIGNED_IMAGE="ttl.sh/lab-${LAB_ID}-signed:2h"

docker pull public.ecr.aws/nginx/nginx:1.25-alpine
docker tag public.ecr.aws/nginx/nginx:1.25-alpine ${SIGNED_IMAGE}
docker push ${SIGNED_IMAGE}

# Sign the image (--tlog-upload=false skips Rekor transparency log — fine for labs)
COSIGN_PASSWORD="" cosign sign \
  --key ~/cosign.key \
  --tlog-upload=false \
  ${SIGNED_IMAGE}

echo "${SIGNED_IMAGE}" > ~/signed_image.txt
echo "Signed: ${SIGNED_IMAGE}"

Verify the Signature

COSIGN_PASSWORD="" cosign verify \
  --key ~/cosign.pub \
  --insecure-ignore-tlog=true \
  ${SIGNED_IMAGE}

Expected: JSON output confirming the signature is valid.

Enforcing Signatures — Blocking Unsigned Images with Kyverno

Write the Signature Verification Policy

The policy requires every image deployed to signing-demo to carry a valid cosign signature from our key. No signature — no pod.

export LAB_ID=$(cat ~/lab_id.txt)
PUBLIC_KEY=$(cat ~/cosign.pub)

cat > ~/verify-signatures-policy.yaml << EOF
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-image-signatures
spec:
  validationFailureAction: Enforce
  background: false
  rules:
  - name: check-image-signature
    match:
      any:
      - resources:
          kinds: [Pod]
          namespaces: [signing-demo]
    verifyImages:
    - imageReferences: ["ttl.sh/lab-${LAB_ID}-*:*"]
      attestors:
      - count: 1
        entries:
        - keys:
            publicKeys: |-
$(echo "$PUBLIC_KEY" | sed 's/^/              /')
            rekor:
              ignoreTlog: true
      required: true
EOF

kubectl apply -f ~/verify-signatures-policy.yaml
kubectl get clusterpolicy verify-image-signatures

Deploy the Signed Image — Should Succeed

kubectl create namespace signing-demo

export SIGNED_IMAGE=$(cat ~/signed_image.txt)

kubectl run signed-app \
  --image=${SIGNED_IMAGE} \
  --restart=Never \
  -n signing-demo \
  --overrides='{"spec":{"resources":{"limits":{"cpu":"100m","memory":"64Mi"}}}}'

kubectl wait --for=condition=Ready pod/signed-app -n signing-demo --timeout=60s
kubectl get pod signed-app -n signing-demo

Expected: Pod starts. Kyverno verified the cosign signature before allowing it.

Deploy an Unsigned Image — Should Be Blocked

kubectl run unsigned-app \
  --image=public.ecr.aws/nginx/nginx:1.25-alpine \
  --restart=Never \
  -n signing-demo 2>&1 || true

Expected:

Error from server: admission webhook "mutate.kyverno.svc-fail" denied the request:
  check-image-signature: image not signed / no matching signature

The nginx:1.25-alpine image has no cosign signature from our key — Kyverno rejects it before it is ever scheduled or pulled.

The Full Supply-Chain Picture

Developer pushes code
        │
        ▼
CI Pipeline
  ├── trivy image      → block on CRITICAL CVEs
  ├── docker build + push
  └── cosign sign      → signature stored in registry
        │
        ▼
kubectl apply / Argo CD sync
        │
        ▼
Kyverno admission webhook
  ├── disallow-latest-tag        → block unpinned tags
  ├── require-resource-limits    → block unlimited containers
  ├── require-non-root           → block root containers
  └── verify-image-signatures    → block unsigned images
        │
        ▼
Pod scheduled → Falco runtime detection

Every arrow is a gate. An attacker must bypass all of them to run arbitrary code in your cluster.

Cleanup

kubectl delete pod signed-app -n signing-demo --ignore-not-found
kubectl delete namespace signing-demo
kubectl delete clusterpolicy verify-image-signatures
Previous lesson
Helm — Packaging Security Into Your App Template
Next lesson
Secret Scanning with Gitleaks
Start lesson