Coming soon ๐ง Bundling and overlays of signed SBOB Fragments
1. Install kubescape with the right images
Chart 1.40.3-sign-rc3 (helm-charts signature-overlays) pins ghcr.io/k8sstormcenter/node-agent:v0.3.193 + ghcr.io/k8sstormcenter/storage:v0.3.177.
From the repo root THIS IS ALREADY PREINSTALLED NO NEED TO RUN IT
make kubescape
make alertmanager
make alertmanager is what ยง4b reads its alerts from; the rest of the demo reports through the node-agent stdout exporter and does not need it.
Two ways to install the trust bundle
The trust policy is a root-signed artifact โ about 2.5KB of JSON carrying a certificate and a signature. There are two ways to get it onto the cluster, and the demo works identically with either.
A. Inline in values (what make kubescape does). kubescape/values.yaml holds the artifact under nodeAgent.bundleSigning.trustPolicy, and the chart renders the ConfigMap. The chart owns the object, so the policy is whatever the values say โ a helm upgrade re-asserts it, which is what you want when the values are your source of truth. To avoid pasting the artifact by hand you can pass it at install time instead:
helm upgrade --install kubescape \
https://github.com/k8sstormcenter/helm-charts/releases/download/kubescape-operator-1.40.3-sign-rc3/kubescape-operator-1.40.3-sign-rc3.tgz \
-n honey --create-namespace --values kubescape/values.yaml \
--set-file nodeAgent.bundleSigning.trustPolicy=example/redis/distros/signed-bundles/trust-policy.signed.json
DONT RUN, ITS AN OPTION
B. Mounted from a ConfigMap you own.
make kubescape-mounted
This creates kubescape-trust-bundle from trust-policy.signed.json and installs with nodeAgent.bundleSigning.existingConfigMap=kubescape-trust-bundle. The chart mounts that ConfigMap and renders none, so the policy comes straight from your signing process and is rotated with kubectl apply on the ConfigMap โ no helm upgrade, no re-pasting.
Either way node-agent reads /etc/bundle/trust-policy.json. The ConfigMap is mounted as a directory, so kubelet propagates updates and node-agent applies a rotated policy on its own, within a reconcile interval โ see ยง9(a). A replacement that does not verify against the root is refused and the policy already in force is kept, so neither path lets an unsigned policy take effect.
2. Signed-bundle support boots with the chart
Root-signed policy verified against the root public key compiled into the image. No private key deployed, no restart, nothing to patch.
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "signed bundle overlays enabled"
# โ signed bundle overlays enabled in alert mode
One global state, carried in the policy: alert reports, enforce refuses. A mounted policy is never silent โ no explicit mode = alert.
Expected startup warning โ the anchor is the published demo root key; the one thing to change for real use:
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "DEMO root key"
See "Bring your own root key"; under enforce, node-agent refuses the demo root outright.
The shipped policy carries ruleClasses โ rule signing is on from the first boot: every Rules object must verify or its rules are dropped whole. An unsigned baseline = no runtime detections, said loudly on every sync. make kubescape therefore ships rules/baseline-rules-signed.yaml (31-rule baseline, base-class, demo operator key) โ to the chart via nodeAgent.bundleSigning.signedDefaultRules and as a direct apply. Confirm before deploying anything:
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "signed rule fragments enabled"
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep -c "detection is effectively OFF"
# โ signed rule fragments enabled, and the count must be 0
2b. Signing modes โ OFF / ALERT / ENFORCE
One switch, in the root-signed policy ("mode"), plus the master values toggle. The full contract, per mode:
OFF (bundleSigning.enabled: false) | ALERT (default) | ENFORCE ("mode": "enforce") | |
|---|---|---|---|
| boot log | none (no bundle lines) | signed bundle overlays enabled in alert mode | signed bundle overlays enabled in ENFORCE mode: unsigned and unverifiable artifacts are refused |
| demo root (compiled anchor) | n/a | warning, keeps running | refused โ mount your own root |
| invalid policy at boot | n/a | trust policy invalid at startup: signed bundle overlays DISABLED โฆ โ agent runs, keeps polling; first valid mount enables signing, no restart | same |
policy without ruleClasses | n/a | rule signing DISABLED: โฆ ANY Rules object in ANY namespace will load without a signature check | same |
| policy reload (valid change) | n/a | applied within ~1 min, trust policy reloaded without restart + inForceDigest; Rules admission re-evaluated immediately, no watch event | same |
| policy reload (unverifiable) | n/a | reload REFUSED naming BOTH digests (sha256sum on the mounted file matches); in-force policy kept | same |
policy reload (older policyVersion) | n/a | refused as rollback, in-force kept | same |
| reload that narrows scope | n/a | trust policy reloaded with REDUCED scope | same |
| unsigned user profile | loads | loads (refused only with requireSignedObjects, ยง12) | refused |
unsigned Rules object | loads | rejected when ruleClasses present; zero admitted โ detection is effectively OFF every sync | rejected, same |
| tampered signed content | takes effect silently โ nothing verifies anything | R1016 + bundle overlay refused โฆ keeping the last verified composite | same |
| stored-spec edit on a signed object | takes effect (it IS the spec) | inert + warning stored spec is display-only and is NOT enforced, once per distinct edit | inert + same warning + alert R1017 Signed profile drift (never R1016) |
| bundle shadows a same-named profile | n/a | shadows a ContainerProfile of the same name: the bundle is enforced, the named profile is not, once per root change | same |
| bundle fails, no prior projection | n/a | NO fallback to a ContainerProfile of the same name: container runs with no user-defined profile | same |
| what is enforced? | the objects themselves | curl :7888/policyz โ digest, mode, root anchor, rules admitted/rejected; or grep inForceDigest | same |
Switching modes = edit "mode" in trust-policy.json, re-sign with the root key, rotate (ยง9a โ no restart). ENFORCE on this demo requires the mounted-root path ("Bring your own root key") because the compiled-in anchor is the demo root.
Rows above the recorded block that mention reload digests, policyVersion, DISABLED, divergence, R1017, shadowing, no-fallback wording, and /policyz are the source strings of the current branch โ they ship with the next image; re-record then.
Recorded on a clean k3s v1.36 cluster, 2026-08-14, images node-agent v0.3.193 (2b recorded block: v0.3.192) / storage v0.3.177.
$NA = kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1
; timestamps trimmed.
OFF โ zero signing plumbing, tamper is silent:
$NA | grep -c "signed bundle" # โ 0
$NA | grep "RulesWatcher - synced rules" | tail -1
# โ "enabledRules":28,"totalRules":1 (unsigned baseline loads)
# after the ยง7d content tamper:
$NA --since=5m | grep -cE "R1016|verif|bundle" # โ 0
ALERT (shipped defaults) โ healthy boot:
{"level":"info","msg":"signed bundle overlays enabled in alert mode"}
{"level":"warn","msg":"signed bundle overlays anchored to the PUBLISHED DEMO root key: this authenticates nothing, mount a real root before relying on signatures"}
{"level":"info","msg":"signed rule fragments enabled"}
{"level":"info","msg":"RulesWatcher - signed rule fragments","admitted":1,"rejected":0}
{"level":"info","msg":"assembled signed bundle overlay","bundle":"redis","fragments":2,"root":"97df8151โฆ"}
ALERT โ the detection outage and its recovery (delete the signed baseline, apply the unsigned one):
{"level":"warn","msg":"rules fragment rejected","name":"default-rules","error":"fragment is not signed: \"default-rules\""}
{"level":"info","msg":"RulesWatcher - signed rule fragments","admitted":0,"rejected":1}
{"level":"error","msg":"RulesWatcher - signing enabled but NO rule fragment admitted while rules objects exist: detection is effectively OFF; sign the baseline ruleset as a base-class fragment or correct the trust policy","rulesObjects":1,"rejected":1}
{"level":"info","msg":"RulesWatcher - synced rules from cluster","enabledRules":0,"totalRules":1}
# exec `id` in redis-master during the outage โ 0 R0001 alerts. Re-apply
# rules/baseline-rules-signed.yaml โ admitted:1 on the watch event, the same
# exec fires R0001 again โ no agent restart (restartCount stayed 0).
ALERT and ENFORCE โ content tamper (ยง7d), identical in both modes:
R1016 "Signed profile tampered", severity 10
{"level":"warn","msg":"signed bundle overlay refused: a verified member no longer verifies; keeping the last verified composite","bundle":"redis","members":"redis-ops-overlay","error":"fragment signature does not verify (tampered): โฆ"}
ENFORCE (mounted root) โ boot + unsigned flat profile refused:
{"level":"info","msg":"signed bundle overlays enabled in ENFORCE mode: unsigned and unverifiable artifacts are refused"}
{"level":"warn","msg":"signed bundle overlays anchored to a MOUNTED root public key: the trust anchor is cluster-mutable, protect it with an immutable ConfigMap and tight RBAC","fingerprint":"key:d0cc7f2eโฆ"}
{"level":"warn","msg":"user-defined ContainerProfile refused: signature verification is required and the profile is unsigned","profile":"redis-client"}
`
**ENFORCE without a mounted root** (compiled-in demo anchor) โ refused outright, signing stays off:
{"level":"warn","msg":"signed bundle overlays disabled: trust policy signature invalid","error":"enforce mode refuses the published demo root key: mount a real root at /etc/bundle/root.pub"}
## 3. The vendor ships SIGNED fragments โ before any workload exists
Offline signing: `sign-object --embed-content` embeds the signed bytes in `signature.kubescape.io/content` โ survives storage normalisation. The cluster never sees an unsigned fragment. No admission fragment yet โ the client does not exist:
cd example/redis/distros/signed-bundles # if you cd'd to the repo root for ยง1 ./sign-fragment.sh fragments/frag-base-redis.yaml keys/vendor.pem ./sign-fragment.sh fragments/frag-overlay-ops.yaml keys/operator.pem
No bundle object exists: grouping is the signed `signature.kubescape.io/bundle: redis` label.
## 4. Deploy redis (pinned distros install, sbob binding)
```sh
(cd .. && ./deploy-distros.sh redis)
kubectl -n redis patch sts redis-master -p '{"spec":{"template":{"metadata":{"labels":{"kubescape.io/user-defined-profile":"redis"}}}}}'
The sbob toggle labels the statefulset with kubescape.io/user-defined-profile: redis, exactly as in the distros demo.
Because the signed fragments are already in place, the pod starts protected โ there is no window where the workload runs without its profile.
node-agent verifies each leaf and assembles the composite in memory, with no signing and no key:
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "assembled signed bundle overlay"
# โ bundle=redis fragments=2 root=<merkle-root-A>
4b. Functional suite, then attacks
A signed profile must behave like the learned one it replaces: benign quiet, attacks alert.
Capture TSTART BEFORE the suite โ anything alerting before it is deployment noise, not a functional FP:
export TSTART=$(date -u +%Y-%m-%dT%H:%M:%S.%3NZ) NS=redis
(cd .. && bobctl test --functional-tests functional/redis-oss.yaml -n redis)
sleep 2; export T0=$(date -u +%Y-%m-%dT%H:%M:%S.%3NZ)
(cd .. && bobctl attack --attack-suite attacks/redis-oss.yaml -n redis)
Split alerts into pre-suite / functional / attack, with DETAILS on every functional FP โ the printed comm + startsAt decide whether an FP is a missing startup exec in the base fragment, a runc:[2:INIT] runtime-init attribution, or a timestamp artifact:
kubectl -n honey port-forward svc/alertmanager 9093:9093 &
sleep 5
curl -s localhost:9093/api/v2/alerts | python3 -c '
import json,sys,os
a=json.load(sys.stdin); ns=os.environ["NS"]
t0=os.environ.get("T0",""); ts=os.environ.get("TSTART","")
al=[x for x in a if x["labels"].get("namespace")==ns]
pre=[x for x in al if x.get("startsAt","")<ts]
fp=[x for x in al if ts<=x.get("startsAt","")<t0]
tp=sorted({x["labels"].get("rule_id") for x in al if x.get("startsAt","")>=t0})
print("pre-suite noise:", len(pre))
print("functional FPs:", len(fp) or "CLEAN")
for x in fp: L=x["labels"]; print(" FP", L.get("rule_id"), "comm="+str(L.get("comm")), "container="+str(L.get("container_name")), "startsAt="+x.get("startsAt",""), x.get("annotations",{}).get("message","")[:80])
print("attack TPs (distinct rules):", len(tp), tp)'
A non-empty FP line means the base fragment does not cover workload startup โ regenerate it from a learned profile taken across a full cold start โ or the split timestamps are wrong; the printed startsAt decides which.
Expected (recorded 2026-08-14 on a clean cluster, current fragments):
functional FPs: CLEAN
attack TPs (distinct rules): 12 ['R0001', 'R0002', 'R0005', 'R0006', 'R0008', 'R0010', 'R0011', 'R0040', 'R1004', 'R1008', 'R1010', 'R1012']
kubectl exec into either workload alerts as the exec'd process, never as runc:[2:INIT] โ the profiles cover container-runtime setup (the client SBoB carries the exec-session init opens/caps and runc rulePolicies).
5. A client appears โ unexpected ingress fires
Client with its own SBoB (its redis-cli/egress stay quiet):
kubectl apply -f ../sbobs/cp-redis-client.yaml
kubectl apply -f ../../client.yaml
Server composite has no ingress โ R0012 on the node hosting redis-master:
MNODE=$(kubectl -n redis get pod redis-master-0 -o jsonpath='{.spec.nodeName}')
NA=$(kubectl -n honey get pod -o wide --field-selector spec.nodeName=$MNODE --no-headers | awk '/node-agent/{print $1;exit}')
timeout 120 kubectl -n honey logs -f $NA | grep -m3 "Unexpected ingress network"
6. Allowlist the client LATER โ signed fragment
The ingress entry ships as a standalone signed admission fragment:
./sign-fragment.sh fragments/frag-admission-redis-client.yaml keys/operator.pem
Within ~1 min: re-assembly, 3 fragments, new root:
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "assembled signed bundle overlay" | tail -1
# โ bundle=redis fragments=3 root=<merkle-root-B> (โ root-A)
R0012 stops. The admission key can only add ingress/egress โ execs in this fragment would fail the whole bundle.
7. Verify the composite โ without touching the leaves
(a) Merkle-bound assembly: the root commits to the exact verified leaf set โ root-A โ root-B just showed it.
(b) Leaves untouched โ assembly never rewrites fragments:
export CP=containerprofiles.spdx.softwarecomposition.kubescape.io
for f in redis-base redis-ops-overlay redis-client-ingress; do
kubectl -n redis get $CP $f -o yaml > /tmp/leaf.yaml
./sign-object verify --file /tmp/leaf.yaml --strict=false && echo "leaf $f: OK"
done
(c) Behaviour proves the union โ each check exercises one fragment:
# base fragment enforced: an exec outside every fragment alerts (R0001)
kubectl -n redis exec sts/redis-master -- id
# overlay fragment honoured: df -h is allowed โ no alert
kubectl -n redis exec sts/redis-master -- df -h
# admission fragment honoured: R0012 stopped in ยง6
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=5m | grep -oE '"RuleID":"R[0-9]+"' | sort | uniq -c
(d) Tamper the signed content โ refused + R1016. Stored-spec edits are inert (enforcement binds embedded content); attacking the embedded content breaks the signature:
kubectl -n redis get $CP redis-ops-overlay -o jsonpath='{.metadata.annotations.signature\.kubescape\.io/content}' \
| python3 -c 'import sys,base64,gzip; d=gzip.decompress(base64.b64decode(sys.stdin.read())); d=d.replace(b"/usr/bin/df", b"/bin/backdoor"); print(base64.b64encode(gzip.compress(d)).decode())' \
| xargs -I{} kubectl -n redis annotate $CP redis-ops-overlay --overwrite signature.kubescape.io/content={}
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=3m | grep '"RuleID":"R1016"' | tail -1
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=3m | grep "bundle overlay refused" | tail -1
Workload keeps the last verified composite โ tamper reports and is rejected without turning every exec into an alert.
Recover โ re-ship the vendor artifact:
./sign-fragment.sh fragments/frag-overlay-ops.yaml keys/operator.pem
Identical leaf set = identical root (logs at debug, no new-root transition). Enforcement-resumed proof is behavioural:
kubectl -n redis exec sts/redis-master -- uname -a
sleep 30
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=2m | grep '"RuleID":"R0001"' | tail -1
How admissibility is decided (reference)
All must hold per fragment, or the bundle is rejected:
signature.kubescape.io/fragment-classlabel present, class exists in the policy;- signature verifies over the embedded signed content (
metadata{name,labels}+spec) โ stored spec irrelevant; - signer fingerprint (
key:<sha256(PKIX(pub))>) listed for its class; - only spec paths its class allows (e.g.
admissionโingress/egress).
New key: sign-object generate-keypair --output my.pem โ fingerprint:
echo "key:$(openssl pkey -pubin -in my.pem.pub -outform DER | sha256sum | cut -d' ' -f1)"
Public keys are never stored on the cluster โ each artifact carries its certificate; the policy holds only the fingerprint it must match.
metadata.namespace is NOT signed: a vendor cannot know the install namespace, and per-customer re-signing would defeat offline signing. Confinement = the signed bundle + fragment-class labels; namespace placement is ordinary RBAC.
How kubescape.io/user-defined-profile resolves. Under a trust policy, kubescape.io/user-defined-profile resolves as a bundle name first; a verifying bundle shadows the identically named ContainerProfile; a failing bundle never falls back to it. The three outcomes, each with its own log line:
- No fragment carries the name โ the ContainerProfile of that name is fetched as before (pre-bundle profiles keep working unchanged).
- Verifying fragments exist โ the composite is enforced, the same-named profile is never read:
signed bundle overlay shadows a ContainerProfile of the same name: the bundle is enforced, the named profile is not(once per root transition). - Fragments exist but fail verification โ NO fallback:
signed bundle failed verification and there is NO fallback to a ContainerProfile of the same name: container runs with no user-defined profile until the bundle verifies. Falling back would let anyone who can corrupt one fragment downgrade a signed bundle to an unsigned profile.
RBAC consequence: once bundles are enabled, create on containerprofiles in a workload's namespace is security-relevant โ an object labelled into a bundle cannot forge a profile, but outcome 3 means it can deny one. Restrict that verb where the profile matters.
Bring your own root key (rotating the trust anchor)
The published image ships a demo root key you are meant to replace.
./sign-object generate-keypair --output root.pem # writes root.pem + root.pem.pub
./sign-object sign-policy --policy trust-policy.json --key root.pem --output trust-policy.signed.json
Keep the root private key offline in escrow โ adding/rotating a signer later means re-signing.
Mounted (no rebuild): set nodeAgent.bundleSigning.rootPublicKey to root.pem.pub alongside the signed trustPolicy; verification uses /etc/bundle/root.pub. The anchor is then a cluster ConfigMap โ protect it with an immutable ConfigMap + tight RBAC; node-agent warns when a mounted anchor is in use.
Embedded (rebuild): replace DefaultRootPublicKeyPEM in pkg/signature/bundle/root.go, rebuild the image. The anchor cannot be swapped on a running cluster.
8. Robustness โ the adversarial cases
Stored-spec edit is inert:
kubectl -n redis patch $CP redis-ops-overlay --type json \
-p '[{"op":"add","path":"/spec/execs/-","value":{"path":"/bin/backdoor"}}]'
# โ no R1016, composite root unchanged; df -h still the ONLY overlay exec enforced
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=2m | grep -c '"RuleID":"R1016"'
# โ 0
The inert edit is not silent: node-agent warns signed fragment stored spec diverges from the signed content: the stored spec is display-only and is NOT enforced naming the diverging paths โ once per distinct stored content. It cuts both ways: an exec added by patching is NOT allowed until re-signed. To see what IS enforced, decode the signed content (the ยง7d jsonpath + gzip one-liner, minus the tamper step).
Signed-content edit is caught โ R1016, fail closed (ยง7d).
Unsigned fragment rejected:
kubectl -n redis delete $CP redis-ops-overlay # drop the signed object
kubectl -n redis apply -f fragments/frag-overlay-ops.yaml # re-create from the UNSIGNED source
# โ refused: "fragment is not signed"; the workload KEEPS its last verified composite
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=2m | grep "no longer verifies" | tail -1
A verified member that stops verifying = refused, last verified composite kept. (apply alone would not unsign โ 3-way merge keeps the annotations.) Re-ship before continuing:
./sign-fragment.sh fragments/frag-overlay-ops.yaml keys/operator.pem
A never-verified object is a non-member, not a bundle failure. Membership is authenticated โ labels alone don't make a member. Unsigned / untrusted-key / class-violating labelled objects are dropped; the bundle assembles from genuine members:
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=2m | grep "dropped non-member"
Otherwise one injected object would fail the bundle closed for every bound workload โ a namespace-writer DoS. Alert deduped per bundle, not per object.
No rollback. Fragments carry a monotonic signature.kubescape.io/version inside the signed content; anything below the accepted high-water mark for that bundle/class/name is refused. Marks are in-memory, reset on restart.
Signer identity = the fingerprint the signature verified against โ never the spoofable OIDC annotations.
9. Signed rules โ the vendor ships rules with the bundle
Profiles: what a workload may do. Rules: what the agent alerts on. Unsigned reality: any Rules object in any namespace merges into one ruleset keyed by rule ID โ anyone who can create one can redefine R0001 with enabled: false cluster-wide. Signed rule fragments close that.
Same two labels as profile fragments โ one bundle carries both halves:
| Object | Labels | Applies |
|---|---|---|
ContainerProfile | bundle: redis, fragment-class: base | workloads bound to bundle redis |
Rules | bundle: redis, fragment-class: overlay | same workloads, overriding the base rule with the same ID |
Rules (baseline) | fragment-class: base | cluster-wide, no bundle |
Bundle + class are signed โ no re-classing, no re-targeting. Namespace is not signed โ same artifact installs anywhere.
Rule classes invert profile roles: base = the user's cluster-wide baseline, overlay = vendor bundle-scoped rules. The policy names signers and permitted rule IDs per class:
"ruleClasses": {
"base": {"signers": ["key:<user>"], "allowedRuleIDs": ["*"]},
"overlay": {"signers": ["key:<vendor>"], "allowedRuleIDs": ["R0001","R0002"]}
}
Scenario: redis is the cache tier โ an unexpected process there is possible compromise, not drift โ so the vendor ships R0001 at severity 10 with a redis-specific message.
If the policy in force has no ruleClasses at all, rule signing is off and node-agent says so at startup and on every reload โ grep "rule signing DISABLED". After a refused reload, the enforced policy is identified by digest: grep "inForceDigest" against sha256sum trust-policy.signed.json, or curl :7888/policyz.
(a) Rule signing is already on
kubescape/values.yaml ships the full root-signed policy, ruleClasses included โ on from ยง1.
Change the policy in
values.yaml, never withkubectlon the ConfigMap. The chart ownsnode-agent-bundle-policyโ a ConfigMap edit survives only until the nexthelm upgrade, and a policy missingruleClassesswitches rule signing off with no error. Edittrust-policy.json, re-sign, paste into values, re-run:./sign-object sign-policy --policy trust-policy.json --key keys/root.pem --output trust-policy.signed.json # paste trust-policy.signed.json into nodeAgent.bundleSigning.trustPolicy in kubescape/values.yaml, then: (cd ../../../.. && make kubescape)No restart โ directory mount, reload within a reconcile interval:
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "trust policy reloaded"A replacement failing root verification is refused, the policy in force kept:
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "trust policy reload REFUSED"With
make kubescape-mounted, rotate the ConfigMap directly instead:kubectl -n honey create configmap kubescape-trust-bundle \ --from-file=trust-policy.json=trust-policy.signed.json --dry-run=client -o yaml | kubectl apply -f -
Every Rules object must verify or its rules drop whole. The install ships the pre-signed baseline (ยง2) โ sign it yourself only when it is yours to sign (own keys, or an edited ruleset):
./sign-rules.sh rules/baseline-rules.yaml keys/operator.pem
kubectl -n honey rollout restart daemonset node-agent
kubectl -n honey rollout status daemonset node-agent --timeout=300s
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "signed rule fragments enabled"
(b) The vendor ships the bundle's rules
./sign-rules.sh rules/rules-redis.yaml keys/vendor.pem
rules/rules-redis.yaml: bundle: redis + fragment-class: overlay, R0001 at severity 10, message REDIS TIER CRITICAL: ....
(c) The override follows the bundle, not the namespace
kubectl -n redis exec sts/redis-master -- id
sleep 30
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=2m | grep "REDIS TIER CRITICAL" | tail -1
Same namespace, different binding โ the client never gets the override:
kubectl -n redis exec deploy/redis-client -- id
sleep 30
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=3m | grep "REDIS TIER CRITICAL" | grep -o '"containerName":"[^"]*"' | sort | uniq -c
(d) The adversarial cases
Rogue key ships a disable-detection fragment:
./sign-object generate-keypair --output /tmp/rogue.pem
sed 's/enabled: true/enabled: false/' rules/rules-redis.yaml > /tmp/rogue-rules.yaml
./sign-rules.sh /tmp/rogue-rules.yaml /tmp/rogue.pem
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=2m | grep "rules fragment rejected"
# โ signer not permitted for this fragment class; the fragment's rules are dropped whole
# โ RulesWatcher - signed rule fragments admitted=1 rejected=1
Rogue rules never load; redis falls back to the base R0001 (severity 1 instead of 10). Signing protects rule content, not object presence โ create/delete on rules.kubescape.io can still lose the tightening; restrict that verb with RBAC. Re-ship the operator artifact to restore.
Unsigned rules object dropped:
kubectl -n redis delete rules.kubescape.io rules-redis
kubectl apply -f rules/rules-redis.yaml # the UNSIGNED source
# โ rules fragment rejected: fragment is not signed
Class-confined IDs: a vendor overlay setting R0007 โ rule ID not allowed for this class.
Overlay must declare a bundle โ no bundle label โ rejected, never applies everywhere.
No re-targeting โ the bundle label is signed; pointing an overlay at another bundle breaks the signature.
10. The same signed artifact, any namespace
kubectl create ns redis-staging
sed 's/^ namespace: redis$/ namespace: redis-staging/' fragments/frag-base-redis-signed.yaml | kubectl create -f -
export CP=containerprofiles.spdx.softwarecomposition.kubescape.io
kubectl -n redis-staging get $CP redis-base -o yaml > /tmp/moved.yaml
./sign-object verify --file /tmp/moved.yaml --strict=false && echo "verified in redis-staging"
Rules likewise: an overlay signed for bundle: redis follows the binding, not the namespace.
10b. A single signed profile, without a bundle
One party owning the whole profile signs it directly โ no bundle/class labels:
cat > /tmp/flat-cp.yaml <<'YAML'
apiVersion: spdx.softwarecomposition.kubescape.io/v1beta1
kind: ContainerProfile
metadata:
name: redis-solo
namespace: redis
annotations:
kubescape.io/managed-by: User
spec:
architectures: ["amd64"]
execs:
- path: /opt/bitnami/redis/bin/redis-server
args: ["redis-server"]
YAML
./sign-fragment.sh /tmp/flat-cp.yaml keys/vendor.pem
kubectl -n redis get $CP redis-solo -o yaml > /tmp/solo.yaml
./sign-object verify --file /tmp/solo.yaml --strict=false
Learned profiles are separate objects, unsigned by design. Bundles are for the multi-party case.
11. The trust anchor itself is checked
A policy not signed by the root is refused โ editing the ConfigMap cannot name a new signer:
./sign-object sign-policy --policy trust-policy.json --key keys/operator.pem --output /tmp/policy-badsigner.json
kubectl -n honey create cm node-agent-bundle-policy --from-file=trust-policy.json=/tmp/policy-badsigner.json --dry-run=client -o yaml | kubectl apply -f -
kubectl -n honey rollout restart daemonset node-agent
kubectl -n honey rollout status daemonset node-agent --timeout=300s
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 | grep "trust policy signature invalid"
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=2m | grep -c "signed bundle overlays enabled"
Restore:
kubectl -n honey create cm node-agent-bundle-policy --from-file=trust-policy.json=trust-policy.signed.json --dry-run=client -o yaml | kubectl apply -f -
kubectl -n honey rollout restart daemonset node-agent
kubectl -n honey rollout status daemonset node-agent --timeout=300s
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=2m | grep "signed bundle overlays enabled"
12. Require signatures on user-supplied profiles
Default: unsigned user-defined profiles load (signing opt-in per object). Flip it:
(cd ../../../.. && { grep -q requireSignedObjects kubescape/values.yaml || sed -i '/^ bundleSigning:/a\ requireSignedObjects: true' kubescape/values.yaml; } && make kubescape)
kubectl -n honey rollout status daemonset node-agent --timeout=300s
kubectl -n honey get cm node-agent -o jsonpath='{.data.config\.json}' | grep enableSignatureVerification
The client's SBoB from ยง5 is a flat unsigned user profile โ refused as soon as this is on:
kubectl -n honey logs -l app=node-agent -c node-agent --tail=-1 --since=3m | grep "is unsigned" | tail -1
# โ user-defined ContainerProfile refused: signature verification is required and the profile is unsigned โฆ profile: redis-client
Sign it back:
./sign-fragment.sh ../sbobs/cp-redis-client.yaml keys/operator.pem
Flat path โ bundle path: an unsigned flat profile is refused outright; a bundle member that stops verifying keeps the last verified composite (ยง8). Learned profiles are unaffected โ generated in-cluster, never on this path.
- Previous lesson
- Consume an application incl. its SBOB on your own infrastructure