User-defined Playground

ELK Stack Playground

This is a hands-on ELK (Elasticsearch, Logstash, Kibana) playground enhanced with Docker and Beats to simulate real-world log and observability scenarios. It includes an Elastic Cluster (es-node-01 and es-node-02) for storing and indexing data, a test-node-03 running Docker with Filebeat, Metricbeat, and Heartbeat for log and metric collection, and kibana-node-04 for visualizing everything via Kibana UI—perfect for learning and experimentation.

ELK Stack playground: This is a hands-on ELK (Elasticsearch, Logstash, Kibana) playground enhanced with Docker and Beats to simulate real-world log and observability scenarios. It includes an Elastic Cluster (es-node-01 and es-node-02) for storing and indexing data, a test-node-03 running Docker with Filebeat, Metricbeat, and Heartbeat for log and metric collection, and kibana-node-04 for visualizing everything via Kibana UI—perfect for learning and experimentation.

🔧 Dev Machine Setup Guide

This guide helps you spin up Filebeat, Metricbeat, and Heartbeat containers on your dev machine using Docker Compose. You can also run beats manually on the Kibana node.

🐳 On dev-machine (node-03)

1. Switch to Docker group

newgrp docker

2. Start Beats with Docker Compose

docker compose up -d

✅ This will spin up the following containers:

  • filebeat
  • metricbeat
  • heartbeat

3. Verify containers are running

docker ps

Output example:

CONTAINER ID   IMAGE                                      NAME
d0df8abd650e   docker.elastic.co/beats/heartbeat:9.0.0    heartbeat
00b825fe814a   docker.elastic.co/beats/filebeat:9.0.0     filebeat
c5866c25e314   docker.elastic.co/beats/metricbeat:9.0.0   metricbeat

📊 On Kibana Machine (node-04)

You can manually run individual Beats if needed:

Run Metricbeat

cd metricbeat-9.0.0-linux-x86_64/
sudo ./metricbeat -e

Run Filebeat

cd filebeat-9.0.0-linux-x86_64/
sudo ./filebeat -e

Run Heartbeat

cd heartbeat-9.0.0-linux-x86_64/
sudo ./heartbeat -e