Introduction
Prometheus is the gold standard for metrics collection, but it was never designed for long-term retention. By default, data is stored locally and purged after 15 days. That's fine for short-term alerting, but useless when you want to analyze trends across weeks or months.
Thanos solves this by layering on top of your existing Prometheus setup, federating metrics across multiple instances, and offloading historical data to object storage. The result: unlimited retention, global query views, and downsampling for faster long-range queries.
Architecture Overview
Prometheus (scraping targets)
|
+-- Thanos Sidecar (reads TSDB blocks, exposes gRPC Store API)
|
+-- Object Storage (MinIO on-disk)
|
+-- Thanos Store Gateway (serves historical blocks)
+-- Thanos Compactor (downsamples + deduplicates)
|
Thanos Querier (merges live + historical data)
|
Grafana (visualization)The Sidecar runs alongside Prometheus and ships completed TSDB blocks to object storage every two hours. The Store Gateway makes those blocks queryable. The Querier fans out requests across both the Sidecar (recent data) and Store Gateway (historical data), giving you a unified view.
Prerequisites
- A RamNode KVM VPS running Ubuntu 22.04 (4GB RAM minimum; 8GB recommended)
- Prometheus already installed and running
- Docker and Docker Compose installed
- A domain or static IP for your VPS
- Ports 9090, 10901–10907, 9000, and 9001 available
If Prometheus is not yet installed, see our Prometheus Getting Started guide first.
Reconfigure Prometheus for Thanos
Thanos requires Prometheus to store data in a location the Sidecar can read. You also need to set block durations so Thanos, not Prometheus, handles compaction.
sudo nano /etc/systemd/system/prometheus.serviceExecStart=/usr/local/bin/prometheus \
--config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/var/lib/prometheus/data \
--storage.tsdb.min-block-duration=2h \
--storage.tsdb.max-block-duration=2h \
--storage.tsdb.retention.time=6h \
--web.enable-lifecycle \
--web.listen-address=0.0.0.0:9090Key points: Setting both block durations to 2h disables Prometheus's own compaction — Thanos Compactor handles this from object storage. Retention of 6h keeps only recent data in Prometheus; Thanos serves everything older.
sudo systemctl daemon-reload
sudo systemctl restart prometheus
sudo systemctl status prometheusCreate Object Storage with MinIO
MinIO gives you an S3-compatible store running entirely on your RamNode VPS. All metrics blocks will be written here.
sudo mkdir -p /opt/thanos
cd /opt/thanosversion: "3.8"
services:
minio:
image: quay.io/minio/minio:latest
container_name: minio
restart: unless-stopped
ports:
- "9000:9000"
- "9001:9001"
environment:
MINIO_ROOT_USER: thanosadmin
MINIO_ROOT_PASSWORD: changeme-use-a-strong-password
volumes:
- /opt/thanos/minio-data:/data
command: server /data --console-address ":9001"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 10s
retries: 3Security note: Change MINIO_ROOT_PASSWORD before going to production. Do not expose port 9000 or 9001 publicly without authentication.
sudo docker compose up -d minio
# Install MinIO client if needed
curl -sSL https://dl.min.io/client/mc/release/linux-amd64/mc -o /usr/local/bin/mc
chmod +x /usr/local/bin/mc
# Configure alias and create bucket
mc alias set local http://localhost:9000 thanosadmin changeme-use-a-strong-password
mc mb local/thanos-metrics
mc ls local/Create Object Store Configuration
Thanos uses a YAML config file to authenticate with your object store.
type: S3
config:
bucket: thanos-metrics
endpoint: localhost:9000
access_key: thanosadmin
secret_key: changeme-use-a-strong-password
insecure: truesudo chmod 600 /etc/thanos-objstore.ymlDeploy the Thanos Sidecar
The Sidecar runs next to Prometheus, uploads TSDB blocks to object storage, and exposes a gRPC Store API endpoint so the Querier can read recent data directly from Prometheus.
thanos-sidecar:
image: quay.io/thanos/thanos:v0.35.1
container_name: thanos-sidecar
restart: unless-stopped
network_mode: host
volumes:
- /var/lib/prometheus/data:/prometheus-data
- /etc/thanos-objstore.yml:/etc/thanos-objstore.yml:ro
command:
- sidecar
- --tsdb.path=/prometheus-data
- --prometheus.url=http://localhost:9090
- --objstore.config-file=/etc/thanos-objstore.yml
- --http-address=0.0.0.0:10902
- --grpc-address=0.0.0.0:10901The Sidecar uses network_mode: host so it can reach Prometheus on localhost:9090. Adjust if you run Prometheus in Docker as well.
cd /opt/thanos
sudo docker compose up -d thanos-sidecar
sudo docker compose logs -f thanos-sidecarWait for a log line like Starting shipper. The first block upload happens after two hours.
Deploy the Store Gateway
The Store Gateway reads blocks from MinIO and exposes them via the same gRPC Store API. This gives the Querier access to historical data.
thanos-store:
image: quay.io/thanos/thanos:v0.35.1
container_name: thanos-store
restart: unless-stopped
ports:
- "10903:10903"
- "10904:10904"
volumes:
- /opt/thanos/store-data:/store-data
- /etc/thanos-objstore.yml:/etc/thanos-objstore.yml:ro
command:
- store
- --objstore.config-file=/etc/thanos-objstore.yml
- --data-dir=/store-data
- --http-address=0.0.0.0:10904
- --grpc-address=0.0.0.0:10903sudo mkdir -p /opt/thanos/store-data
sudo docker compose up -d thanos-storeDeploy the Compactor
The Compactor downsamples blocks for faster long-range queries and deduplicates data from multiple Prometheus replicas. Only run one Compactor per bucket.
thanos-compactor:
image: quay.io/thanos/thanos:v0.35.1
container_name: thanos-compactor
restart: unless-stopped
ports:
- "10905:10905"
volumes:
- /opt/thanos/compactor-data:/compactor-data
- /etc/thanos-objstore.yml:/etc/thanos-objstore.yml:ro
command:
- compact
- --objstore.config-file=/etc/thanos-objstore.yml
- --data-dir=/compactor-data
- --http-address=0.0.0.0:10905
- --retention.resolution-raw=30d
- --retention.resolution-5m=90d
- --retention.resolution-1h=365d
- --waitRetention Policy
| Resolution | Retention |
|---|---|
| Raw (15s/30s scrape) | 30 days |
| 5-minute downsamples | 90 days |
| 1-hour downsamples | 1 year |
sudo mkdir -p /opt/thanos/compactor-data
sudo docker compose up -d thanos-compactorDeploy the Querier
The Querier is the component Grafana talks to. It fans out PromQL queries across all configured Store API endpoints and deduplicates overlapping data.
thanos-querier:
image: quay.io/thanos/thanos:v0.35.1
container_name: thanos-querier
restart: unless-stopped
ports:
- "10906:10906"
- "10907:10907"
command:
- query
- --http-address=0.0.0.0:10906
- --grpc-address=0.0.0.0:10907
- --store=host.docker.internal:10901
- --store=thanos-store:10903
- --query.replica-label=prometheus_replica
extra_hosts:
- "host.docker.internal:host-gateway"The --store flags point to the Sidecar (host network) and the Store Gateway container.
sudo docker compose up -d thanos-querierVerify at http://your-vps-ip:10906. Navigate to Stores and confirm both endpoints show as UP.
Connect Grafana
Add a new Prometheus data source in Grafana pointing to the Thanos Querier instead of Prometheus directly:
| Setting | Value |
|---|---|
| URL | http://your-vps-ip:10906 |
| Type | Prometheus |
This is the only change needed. All existing dashboards continue to work, but long-range queries now transparently pull from object storage.
Verify the Full Pipeline
# Check all containers are running
sudo docker compose -f /opt/thanos/docker-compose.yml ps
# Query via the Thanos HTTP API for a known metric
curl -s "http://localhost:10906/api/v1/query?query=up" | python3 -m json.tool | head -40
# Check block uploads in MinIO
mc ls local/thanos-metrics/After the first two-hour TSDB block cycle completes, you should see block directories appear in local/thanos-metrics/. The Store Gateway will index them automatically.
Resource Expectations
On a 4-vCPU / 8GB RAM RamNode KVM VPS with roughly 50 scrape targets at 15s intervals:
| Component | RAM | CPU (idle) |
|---|---|---|
| Prometheus | ~600 MB | Low |
| Thanos Sidecar | ~80 MB | Low |
| Thanos Store Gateway | ~200 MB | Low (spikes on query) |
| Thanos Querier | ~100 MB | Low (spikes on query) |
| Thanos Compactor | ~300 MB | Periodic bursts |
| MinIO | ~150 MB | Low |
Total: roughly 1.5 GB RAM at steady state, leaving plenty of headroom on an 8GB node. On a 4GB VPS, consider running the Compactor on a schedule rather than with --wait.
Troubleshooting
| Issue | Solution |
|---|---|
| Sidecar can't connect to Prometheus | Confirm Prometheus listens on 0.0.0.0:9090, not 127.0.0.1 |
| Store endpoints show as DOWN | Check firewall: sudo ufw allow from 172.16.0.0/12 to any port 10901 |
| MinIO bucket not found | Re-run mc mb local/thanos-metrics and verify name in objstore config |
| Compactor: "too many blocks" | Normal on first run — let it work through the backlog with --wait |
| Disk space growing quickly | Tighten --retention.resolution-raw or reduce scrape frequency |
Thanos Long-Term Metrics Deployed!
You now have unlimited Prometheus metrics retention with Thanos on your RamNode VPS. Next steps to consider:
- Add a second Prometheus replica for HA — the Compactor handles deduplication automatically
- Enable TLS on gRPC endpoints if accessible outside your private network
- Set up Thanos Ruler for recording rules and alerts over long-range data
- Monitor Thanos itself — each component exposes
/metricson its HTTP port
