Chain of Trust: How a Verifier Decides to Believe a Certificate

This course is about replacing shared secrets with identities a workload can prove. To see why that matters, start with the thing we are replacing.

The problem, in your own terminal

Install a database, give an app a password, and put that password where every app keeps it: in a file.

sudo apt-get update && sudo apt-get install -y mariadb-server
sudo systemctl start mariadb
sudo mariadb <<'SQL'
DELETE FROM mysql.user WHERE User='';
CREATE USER 'app'@'%' IDENTIFIED BY 's3cr3t-in-a-file';
GRANT ALL PRIVILEGES ON *.* TO 'app'@'%';
FLUSH PRIVILEGES;
SQL
mkdir -p ~/app && printf 'DB_USER=app\nDB_PASSWORD=s3cr3t-in-a-file\n' > ~/app/.env

Now be the attacker who found the file:

grep -r PASSWORD ~/app
PW=$(grep DB_PASSWORD ~/app/.env | cut -d= -f2)
mariadb --protocol=tcp -h 127.0.0.1 -u app -p"$PW" -e "SELECT CURRENT_USER();"

You are in. That secret leaks the moment anyone reads the file, it never rotates on its own, and it does not actually prove who connected, only that someone knew the string. Every lab after this one chips away at those three failures. But certificates, the tool we will use, only work if a verifier can decide to trust them, so that is what this lab builds. We will use the clearest verifier there is: curl against a web server, exactly how a browser checks a site.

Step 1: Build the trust root

A real CA never signs server certs with its root key. The root signs one intermediate and then goes cold; the intermediate does the day-to-day signing. Build both with cfssl.

sudo apt-get install -y golang-cfssl nginx
mkdir -p ~/pki && cd ~/pki

cat > root-csr.json <<'JSON'
{ "CN": "Demo Root CA", "key": {"algo":"ecdsa","size":256}, "names":[{"O":"Demo"}], "ca": {"expiry":"87600h"} }
JSON
cat > intermediate-csr.json <<'JSON'
{ "CN": "Demo Intermediate CA", "key": {"algo":"ecdsa","size":256}, "names":[{"O":"Demo"}], "ca": {"expiry":"43800h"} }
JSON
cat > ca-config.json <<'JSON'
{ "signing": { "default": {"expiry":"8760h"}, "profiles": {
    "intermediate": {"expiry":"43800h","usages":["cert sign","crl sign"],"ca_constraint":{"is_ca":true,"max_path_len":0}},
    "server":       {"expiry":"8760h","usages":["signing","key encipherment","server auth"]}
  } } }
JSON

cfssl gencert -initca root-csr.json         | cfssljson -bare root
cfssl gencert -initca intermediate-csr.json | cfssljson -bare intermediate
cfssl sign -ca root.pem -ca-key root-key.pem -config ca-config.json \
  -profile intermediate intermediate.csr    | cfssljson -bare intermediate

Check it. The intermediate is signed by the root, not self-signed:

openssl x509 -in intermediate.pem -noout -subject -issuer

The issuer should read Demo Root CA while the subject reads Demo Intermediate CA. That gap is the chain's first link.

Step 2: Issue a server certificate and serve it

Issue a cert for app.local from the intermediate, and build the file nginx serves: the leaf followed by the intermediate, so a client can walk the chain up to the root.

cd ~/pki
echo '{"CN":"app.local","hosts":["app.local"],"key":{"algo":"ecdsa","size":256}}' \
  | cfssl gencert -ca intermediate.pem -ca-key intermediate-key.pem -config ca-config.json -profile server - \
  | cfssljson -bare server
cat server.pem intermediate.pem > fullchain.pem

echo "127.0.0.1 app.local" | sudo tee -a /etc/hosts > /dev/null
sudo mkdir -p /var/www/app && echo "hello over TLS" | sudo tee /var/www/app/index.html > /dev/null
sudo tee /etc/nginx/sites-available/app > /dev/null <<CONF
server {
    listen 443 ssl;
    server_name app.local;
    ssl_certificate     $HOME/pki/fullchain.pem;
    ssl_certificate_key $HOME/pki/server-key.pem;
    root /var/www/app;
}
CONF
sudo ln -sf /etc/nginx/sites-available/app /etc/nginx/sites-enabled/app
sudo nginx -t && sudo systemctl restart nginx

Check it. The cert is bound to the name app.local through its SAN, the field a verifier actually checks:

openssl x509 -in server.pem -noout -ext subjectAltName

Step 3: Watch the verifier decide

This is the heart of the lab. curl is the verifier; trust is the decision it makes.

First, with no trust configured:

curl https://app.local/

It refuses: unable to get local issuer certificate. The server's cert chains to a root curl has never heard of, so it will not believe it. Now hand curl your root CA and let it decide again:

curl --cacert ~/pki/root.pem https://app.local/

Check it. The second command prints hello over TLS. Nothing about the server changed; the only difference is that the verifier now holds the root the chain leads to. That is the entire mechanism: trust is the verifier matching a chain against a root it was told to trust.

Step 4: The intermediate gap

A chain only connects if every link is present. Serve the leaf alone, with the intermediate missing:

sudo sed -i 's#/fullchain.pem#/server.pem#' /etc/nginx/sites-available/app
sudo nginx -t && sudo systemctl restart nginx
curl --cacert ~/pki/root.pem https://app.local/

It fails again with unable to get local issuer certificate, even though you trust the root, because the client cannot bridge the leaf to the root without the intermediate. Put the chain back:

sudo sed -i 's#/server.pem#/fullchain.pem#' /etc/nginx/sites-available/app
sudo nginx -t && sudo systemctl restart nginx
curl --cacert ~/pki/root.pem https://app.local/

Check it. It works again. The lesson is operational: a server must serve its full chain (leaf plus intermediates), never just its leaf. Most "works in my browser but not in curl" cert bugs are a missing intermediate.

Step 5: Names are part of identity

A valid chain is not enough if the cert is for the wrong name. Hit the same server by IP instead of app.local:

curl --cacert ~/pki/root.pem https://127.0.0.1/

Check it. It refuses: no alternative certificate subject name matches target host name '127.0.0.1'. The chain is perfect, but the cert says app.local and you asked for 127.0.0.1. A verifier checks both that the chain is valid and that the name matches its SAN. Hold onto that second check; in Lab 2 it becomes the whole game.

Step 6: You are Mozilla

So far you pass --cacert by hand every time. In a real fleet you instead install your root into each verifier's trust store once, and then trust is automatic. Do that for this whole machine:

sudo cp ~/pki/root.pem /usr/local/share/ca-certificates/demo-root.crt
sudo update-ca-certificates
curl https://app.local/

Check it. No --cacert, and it still prints hello over TLS. You just did what Mozilla does for the public web, for your own CA: distributed your root to a verifier so everything you sign is trusted. That distribution step is the foundation the rest of the course builds identity on.

Recap

You saw the problem the course exists to solve, then built the foundation for solving it:

• A plaintext password let anyone holding a file become the app. That is the secret we spend the course replacing.
• A verifier trusts a certificate when its chain leads to a root the verifier holds, which is why you serve the full chain and why distributing your root is the key act.
• Identity is bound to a name and checked against the SAN, not just the chain.

Right now trust runs one way: the client checks the server. In Lab 2 we turn it around, the server demands a certificate from the client too, and discover that a valid certificate from your CA still does not prove which client it is.