Grafana + Prometheus Monitoring Stack Setup Guide

Architecture Overview

Monitoring Server

• Runs Grafana (visualization)
• Runs Prometheus (metrics)
• 2GB RAM minimum
• 4GB RAM recommended

→ View VPS Plans

Client Servers

• Runs Node Exporter
• Exposes system metrics
• Minimal overhead
• 1GB+ RAM

Install Prometheus

Install Prometheus on your monitoring server:

Create Prometheus User and Directories

# Create prometheus user
sudo useradd --no-create-home --shell /bin/false prometheus

# Create directories
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus

Download and Install Prometheus

cd /tmp
wget https://github.com/prometheus/prometheus/releases/download/v2.47.0/prometheus-2.47.0.linux-amd64.tar.gz
tar -xvf prometheus-2.47.0.linux-amd64.tar.gz
cd prometheus-2.47.0.linux-amd64

# Copy binaries
sudo cp prometheus /usr/local/bin/
sudo cp promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool

# Copy configuration files
sudo cp -r consoles /etc/prometheus
sudo cp -r console_libraries /etc/prometheus
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries

Configure Prometheus

Create Prometheus configuration:

Create prometheus.yml

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node_exporter'
    static_configs:
      - targets: ['CLIENT_IP:9100']
        labels:
          alias: 'client-server'

⚠️ Important: Replace CLIENT_IP with your monitored server's IP address.

Set Permissions

sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

Create Prometheus Service

Create systemd service for Prometheus:

Create Service File

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
  --config.file /etc/prometheus/prometheus.yml \
  --storage.tsdb.path /var/lib/prometheus/ \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

Enable and Start Prometheus

sudo systemctl daemon-reload
sudo systemctl start prometheus
sudo systemctl enable prometheus
sudo systemctl status prometheus

✅ Verify at: http://YOUR_SERVER_IP:9090

Install Grafana

Install Grafana for visualization:

Install Grafana

# Add repository
sudo apt-get install -y software-properties-common
sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -

# Install
sudo apt-get update
sudo apt-get install grafana

# Start service
sudo systemctl start grafana-server
sudo systemctl enable grafana-server
sudo systemctl status grafana-server

💡 Default credentials: admin / admin (change on first login)
Access at: http://YOUR_SERVER_IP:3000

Configure Grafana Data Source

Connect Grafana to Prometheus:

Log in to Grafana at http://YOUR_SERVER_IP:3000
Click Configuration (gear icon) → Data Sources
Click "Add data source"
Select "Prometheus"
Set URL to: http://localhost:9090
Click "Save & Test"

Import Dashboard: Use ID 1860 (Node Exporter Full)

Install Node Exporter on Clients

Run on each server you want to monitor:

Install Node Exporter

# Create user
sudo useradd --no-create-home --shell /bin/false node_exporter

# Download
cd /tmp
wget https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz
tar -xvf node_exporter-1.6.1.linux-amd64.tar.gz
cd node_exporter-1.6.1.linux-amd64

# Install
sudo cp node_exporter /usr/local/bin/
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Create Service File

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

Start Node Exporter

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo systemctl status node_exporter

Configure Firewall

Set up firewall rules:

Monitoring Server

Allow Grafana & Prometheus

sudo ufw allow 3000/tcp
sudo ufw allow 9090/tcp
sudo ufw reload

Client Servers

Allow Node Exporter

sudo ufw allow from PROMETHEUS_IP to any port 9100
sudo ufw reload

Verification

Check Prometheus Targets

Visit: http://YOUR_SERVER_IP:9090/targets

All targets should show as UP

View Grafana Dashboard

Navigate to Dashboards → Node Exporter Full to see CPU, memory, disk, and network metrics

Add Multiple Servers

Monitor additional servers by updating Prometheus config:

Update prometheus.yml

- job_name: 'node_exporter'
  static_configs:
    - targets: ['CLIENT1_IP:9100']
      labels:
        alias: 'client-server-1'
    - targets: ['CLIENT2_IP:9100']
      labels:
        alias: 'client-server-2'

Restart Prometheus

sudo systemctl restart prometheus

Troubleshooting Common Issues

Prometheus not scraping metrics

• Check firewall rules on client
• Verify Node Exporter is running
• Check Prometheus logs: sudo journalctl -u prometheus -f

Grafana shows no data

• Verify Prometheus data source is configured
• Check Prometheus can reach targets
• Review time range in dashboard

Security Best Practices

Use reverse proxy (Nginx) with SSL for Grafana
Restrict firewall rules to trusted IPs
Change default Grafana password immediately
Keep software updated regularly