kube-scheduler

What's the difference between Kubernetes and Docker?

This is one of the most common questions asked by newcomers to Kubernetes.

Kubernetes is a container orchestration platform that manages the deployment, scaling, and operation of containerized applications across a cluster of machines.

This multi-machine capability (among other things) is what sets Kubernetes apart from single-host container runtimes like Docker. Kubernetes can distribute workloads across multiple nodes, ensuring applications are deployed and scaled efficiently and reliably.

However, Kubernetes cannot randomly decide where to place Pods. It needs to make informed decisions, otherwise this could lead to resource contention, performance degradation, or application failure.

In Kubernetes, scheduling refers to making sure that Pods are assigned to Nodes so that kubelet can run them.

A scheduler watches for newly created Pods that have no assigned Node. For every unscheduled Pod, the scheduler becomes responsible for finding the best Node for that Pod to run on.

kube-scheduler is the default scheduler for Kubernetes and runs as part of the control plane. It considers various factors including resource availability, node affinity, pod priorities, and more to make optimal scheduling decisions.

kube-scheduler in a Kubernetes cluster

Before installing kube-scheduler, let's explore how Kubernetes assigns Pods to Nodes.

Create a (fake) Node object that will serve as the scheduling target:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Node
metadata:
  name: worker-01
status:
  conditions:
    - type: Ready
      status: "True"
EOF

The simplest way to assign a Pod to a specific Node is to set the nodeName field directly in the Pod specification:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: podinfo-node-name
spec:
  nodeName: worker-01
  automountServiceAccountToken: false
  containers:
    - name: podinfo
      image: ghcr.io/stefanprodan/podinfo:latest
EOF

Verify that the Pod was assigned to the selected Node:

kubectl get pod podinfo-node-name -o jsonpath='{.spec.nodeName}'

While this approach works when you need to place a Pod on a specific node, manually assigning Pods to Nodes isn't scalable, may not be possible in all scenarios and kinda defeats the purpose of container orchestration.

However, there's a more fundamental problem with regards to scheduling: Pods are (mostly) immutable objects. Once created, a Pod's specification cannot be modified, including the nodeName field.

This immutability raises an important question: if a Pod is created without a nodeName (unscheduled) and Pods cannot be modified after creation, how does a scheduler assign it to a Node?

The answer lies in a special Kubernetes resource called Binding. It is an internal subresource of Pods that allows the assignment of Pods to Nodes without modifying the Pod specification itself.

Let's see this in action by creating a Pod without specifying a nodeName:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: podinfo-unscheduled
spec:
  automountServiceAccountToken: false
  containers:
    - name: podinfo
      image: ghcr.io/stefanprodan/podinfo:latest
EOF

At this point, the Pod exists but remains unscheduled (it has no assigned Node):

kubectl get pod podinfo-unscheduled -o jsonpath='{.spec.nodeName}'

Simulate what kube-scheduler does by creating a Binding object. Since kubectl doesn't support Binding creation directly (it's an internal operation), use the Kubernetes API directly:

curl -f -k https://127.0.0.1:6443/api/v1/namespaces/default/pods/podinfo-unscheduled/binding \
    --cacert /etc/kubernetes/pki/ca.crt \
    --cert /etc/kubernetes/pki/admin.crt \
    --key /etc/kubernetes/pki/admin.key \
    -X POST \
    -H "Content-Type: application/json" \
    -d '{
      "apiVersion": "v1",
      "kind": "Binding",
      "metadata": {
        "name": "podinfo-unscheduled"
      },
      "target": {
        "apiVersion": "v1",
        "kind": "Node",
        "name": "worker-01"
      }
    }'

Verify that the Binding successfully assigned the Pod to the Node:

kubectl get pod podinfo-unscheduled -o jsonpath='{.spec.nodeName}'

Check the status of both Pods to see the final result:

kubectl get pods -o wide

Follow these steps to install kube-scheduler:

Set the version to install:

KUBE_VERSION=v1.33.3

if command -v kube-apiserver > /dev/null; then
KUBE_VERSION=$(kube-apiserver --version | cut -d ' ' -f 2)
else
KUBE_VERSION=$(curl -fsL https://dl.k8s.io/release/stable.txt)
fi

Download and install kube-scheduler from the official Kubernetes releases:

curl -fsSLO https://dl.k8s.io/${KUBE_VERSION?}/bin/linux/amd64/kube-scheduler
sudo install -m 755 kube-scheduler /usr/local/bin

Download the systemd unit file to configure kube-scheduler as a systemd service:

sudo wget -O /etc/systemd/system/kube-scheduler.service https://labs.iximiuz.com/content/files/courses/kubernetes-the-very-hard-way-0cbfd997.beta/03-control-plane/03-kube-scheduler/__static__/kube-scheduler.service?v=1756470614

Before you can start the kube-scheduler service, you need to configure authentication.

Generate a certificate and key for kube-scheduler:

cd /etc/kubernetes/pki

sudo openssl genrsa -out kube-scheduler.key 2048
sudo openssl req -new -key kube-scheduler.key -out kube-scheduler.csr -subj "/CN=system:kube-scheduler"
sudo openssl x509 -req -in kube-scheduler.csr -out kube-scheduler.crt \
  -CA ca.crt -CAkey ca.key \
  -days 365

Create a kubeconfig for kube-scheduler:

sudo kubectl config set-cluster default \
    --kubeconfig=/etc/kubernetes/kube-scheduler.conf \
    --certificate-authority=/etc/kubernetes/pki/ca.crt \
    --embed-certs=true \
    --server=https://127.0.0.1:6443

sudo kubectl config set-credentials kube-scheduler \
    --kubeconfig=/etc/kubernetes/kube-scheduler.conf \
    --client-certificate=/etc/kubernetes/pki/kube-scheduler.crt \
    --client-key=/etc/kubernetes/pki/kube-scheduler.key \
    --embed-certs=true

sudo kubectl config set-context default \
    --kubeconfig=/etc/kubernetes/kube-scheduler.conf \
    --cluster=default \
    --user=kube-scheduler

sudo kubectl config use-context default \
    --kubeconfig=/etc/kubernetes/kube-scheduler.conf

Configure kube-scheduler to use the kubeconfig file:

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration

clientConnection:
    kubeconfig: /etc/kubernetes/kube-scheduler.conf

sudo mkdir -p /etc/kubernetes/scheduler
sudoedit /etc/kubernetes/scheduler/config.yaml

Reload the systemd daemon and start the kube-scheduler service:

sudo systemctl daemon-reload
sudo systemctl enable --now kube-scheduler

Congratulations! You have successfully installed kube-scheduler. 🎉

With the kube-scheduler installed, Pods can now be scheduled automatically onto Nodes in the cluster.

Create a new Pod to test scheduling:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: podinfo-scheduler
spec:
  nodeName: worker-01
  automountServiceAccountToken: false
  containers:
    - name: podinfo
      image: ghcr.io/stefanprodan/podinfo:latest
EOF

Verify that the Pod was assigned to a Node:

kubectl get pod podinfo-scheduler -o jsonpath='{.spec.nodeName}'

In this lesson, you learned about kube-scheduler, the default scheduler component that assigns Pods to Nodes in a Kubernetes cluster.

Key takeaways:

• Pod placement: kube-scheduler watches for newly created Pods without assigned Nodes and makes informed decisions about optimal Node placement based on resource availability, node affinity, pod priorities, and other factors
• Binding mechanism: Since Pods are immutable objects, kube-scheduler uses the internal Binding subresource to assign unscheduled Pods to Nodes without modifying the Pod specification itself
• Extensible design: As the default scheduler, kube-scheduler can be replaced with custom schedulers when specialized scheduling logic is required

With kube-scheduler now running alongside kube-apiserver, you have established the core scheduling capability that enables Kubernetes to automatically distribute workloads across your cluster nodes.

🧪 Playgrounds

• kube-apiserver

TODO