Part 1 of 6

30 min

Introduction & Installation

Deploy a modern CI/CD pipeline platform on your RamNode VPS with Docker Compose or binary installation

What is Concourse CI?

Concourse CI is an open-source, container-native continuous integration and delivery platform. Unlike traditional CI tools like Jenkins, Concourse treats pipelines as first-class citizens with a declarative YAML-based configuration that's version-controlled alongside your code.

Key Differentiators

Containerized by Default

Every task runs in an isolated container, ensuring reproducibility and eliminating "works on my machine" issues.

Stateless Workers

Workers don't maintain state between builds, making horizontal scaling straightforward and debugging predictable.

Resource-Based Model

External dependencies (git repos, S3 buckets, Docker registries) are abstracted as "resources" that can be versioned and tracked.

Visual Pipeline UI

The web UI renders your entire pipeline as an interactive graph, making complex workflows easy to understand.

Prerequisites

Before installing Concourse, ensure your RamNode VPS meets these requirements:

Operating System: Ubuntu 22.04 LTS or 24.04 LTS (recommended)
RAM: Minimum 2GB (4GB+ recommended for production)
Storage: 20GB+ available disk space
CPU: 2+ vCPUs recommended
Network: Public IP address with ports 80, 443, and 8080 accessible

Recommended RamNode Plans

Use Case	Plan	Specs
Development/Testing	Standard 4GB	4GB RAM, 2 vCPU, 80GB SSD
Small Team Production	Standard 8GB	8GB RAM, 4 vCPU, 160GB SSD
Enterprise/High Volume	Premium 16GB	16GB RAM, 6 vCPU, 320GB NVMe

Docker Compose Installation (Recommended)

Docker Compose provides the simplest path to a working Concourse installation with minimal configuration.

Step 1: Install Docker and Docker Compose

Install Docker

# Update system packages
sudo apt update && sudo apt upgrade -y

# Install required packages
sudo apt install -y ca-certificates curl gnupg lsb-release

# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add Docker repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Add your user to docker group
sudo usermod -aG docker $USER
newgrp docker

Step 2: Create Concourse Directory Structure

mkdir -p ~/concourse/keys
cd ~/concourse

Step 3: Generate Required Keys

Concourse requires several key pairs for secure communication between components:

Generate keys

# Generate session signing key (for web authentication)
ssh-keygen -t rsa -b 4096 -f ./keys/session_signing_key -N ''

# Generate TSA host key (for worker registration)
ssh-keygen -t rsa -b 4096 -f ./keys/tsa_host_key -N ''

# Generate worker key
ssh-keygen -t rsa -b 4096 -f ./keys/worker_key -N ''

# Copy worker public key for authorization
cp ./keys/worker_key.pub ./keys/authorized_worker_keys

Step 4: Create Docker Compose Configuration

docker-compose.yml

cat > docker-compose.yml << 'EOF'
version: '3'

services:
  concourse-db:
    image: postgres:15
    environment:
      POSTGRES_DB: concourse
      POSTGRES_USER: concourse_user
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: unless-stopped

  concourse-web:
    image: concourse/concourse:7.11
    command: web
    depends_on:
      - concourse-db
    ports:
      - "8080:8080"
    environment:
      CONCOURSE_POSTGRES_HOST: concourse-db
      CONCOURSE_POSTGRES_USER: concourse_user
      CONCOURSE_POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      CONCOURSE_POSTGRES_DATABASE: concourse
      CONCOURSE_EXTERNAL_URL: ${CONCOURSE_EXTERNAL_URL}
      CONCOURSE_ADD_LOCAL_USER: ${CONCOURSE_ADMIN_USER}:${CONCOURSE_ADMIN_PASSWORD}
      CONCOURSE_MAIN_TEAM_LOCAL_USER: ${CONCOURSE_ADMIN_USER}
      CONCOURSE_SESSION_SIGNING_KEY: /keys/session_signing_key
      CONCOURSE_TSA_HOST_KEY: /keys/tsa_host_key
      CONCOURSE_TSA_AUTHORIZED_KEYS: /keys/authorized_worker_keys
    volumes:
      - ./keys:/keys:ro
    restart: unless-stopped

  concourse-worker:
    image: concourse/concourse:7.11
    command: worker
    privileged: true
    depends_on:
      - concourse-web
    environment:
      CONCOURSE_TSA_HOST: concourse-web:2222
      CONCOURSE_TSA_PUBLIC_KEY: /keys/tsa_host_key.pub
      CONCOURSE_TSA_WORKER_PRIVATE_KEY: /keys/worker_key
      CONCOURSE_RUNTIME: containerd
      CONCOURSE_BAGGAGECLAIM_DRIVER: overlay
    volumes:
      - ./keys:/keys:ro
    restart: unless-stopped

volumes:
  pgdata:
EOF

Step 5: Create Environment File

.env

cat > .env << EOF
POSTGRES_PASSWORD=$(openssl rand -base64 32 | tr -dc 'a-zA-Z0-9' | head -c 24)
CONCOURSE_EXTERNAL_URL=http://$(curl -s ifconfig.me):8080
CONCOURSE_ADMIN_USER=admin
CONCOURSE_ADMIN_PASSWORD=$(openssl rand -base64 32 | tr -dc 'a-zA-Z0-9' | head -c 16)
EOF

# Display credentials (save these!)
echo "=== SAVE THESE CREDENTIALS ==="
cat .env
echo "=============================="

Step 6: Launch Concourse

docker compose up -d

# Verify all containers are running
docker compose ps

# Check logs if needed
docker compose logs -f concourse-web

Binary Installation (Alternative)

For environments where Docker isn't available or you need more control, Concourse can be installed directly.

Step 1: Install PostgreSQL

sudo apt update
sudo apt install -y postgresql postgresql-contrib

# Create database and user
sudo -u postgres psql << EOF
CREATE USER concourse WITH PASSWORD 'your_secure_password';
CREATE DATABASE concourse OWNER concourse;
EOF

Step 2: Download Concourse Binaries

# Create installation directory
sudo mkdir -p /opt/concourse/bin
cd /opt/concourse

# Download latest Concourse
CONCOURSE_VERSION="7.11.2"
sudo curl -L -o bin/concourse "https://github.com/concourse/concourse/releases/download/v${CONCOURSE_VERSION}/concourse-${CONCOURSE_VERSION}-linux-amd64.tgz"
sudo tar -xzf bin/concourse -C bin/ --strip-components=1
sudo rm bin/concourse
sudo chmod +x bin/concourse-*

Step 3: Generate Keys

sudo mkdir -p /opt/concourse/keys

cd /opt/concourse/keys
sudo ssh-keygen -t rsa -b 4096 -f session_signing_key -N ''
sudo ssh-keygen -t rsa -b 4096 -f tsa_host_key -N ''
sudo ssh-keygen -t rsa -b 4096 -f worker_key -N ''
sudo cp worker_key.pub authorized_worker_keys

Step 4: Create Systemd Services

concourse-web.service

sudo cat > /etc/systemd/system/concourse-web.service << 'EOF'
[Unit]
Description=Concourse CI Web
After=postgresql.service

[Service]
Type=simple
ExecStart=/opt/concourse/bin/concourse web \
  --postgres-host=127.0.0.1 \
  --postgres-user=concourse \
  --postgres-password=your_secure_password \
  --postgres-database=concourse \
  --external-url=http://YOUR_SERVER_IP:8080 \
  --session-signing-key=/opt/concourse/keys/session_signing_key \
  --tsa-host-key=/opt/concourse/keys/tsa_host_key \
  --tsa-authorized-keys=/opt/concourse/keys/authorized_worker_keys \
  --add-local-user=admin:your_admin_password \
  --main-team-local-user=admin
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
EOF

concourse-worker.service

sudo cat > /etc/systemd/system/concourse-worker.service << 'EOF'
[Unit]
Description=Concourse CI Worker
After=concourse-web.service

[Service]
Type=simple
ExecStart=/opt/concourse/bin/concourse worker \
  --tsa-host=127.0.0.1:2222 \
  --tsa-public-key=/opt/concourse/keys/tsa_host_key.pub \
  --tsa-worker-private-key=/opt/concourse/keys/worker_key
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
EOF

Step 5: Start Services

sudo systemctl daemon-reload
sudo systemctl enable --now concourse-web concourse-worker
sudo systemctl status concourse-web concourse-worker

Installing the Fly CLI

The fly command-line tool is essential for interacting with Concourse. It's used to configure pipelines, trigger builds, and manage teams.

Download and install fly

# Download fly (matches your Concourse version)
curl -L "http://YOUR_SERVER_IP:8080/api/v1/cli?arch=amd64&platform=linux" -o fly
chmod +x fly
sudo mv fly /usr/local/bin/

# Verify installation
fly --version

Authenticate with Your Concourse Instance

# Login to your Concourse (creates a "target" alias)
fly -t main login -c http://YOUR_SERVER_IP:8080

# You'll be prompted to authenticate via browser or enter credentials
# Enter admin credentials from your .env file

Verifying Your Installation

Check Web UI Access

Open your browser and navigate to http://YOUR_SERVER_IP:8080. You should see the Concourse login page.

Verify Worker Connection

fly -t main workers

# Should output something like:
# name              containers  platform  tags  team  state    version  age
# abc123def456      0           linux     none  none  running  2.5      5m

Run a Test Pipeline

Create a simple test pipeline to verify everything works:

hello-world.yml

cat > hello-world.yml << 'EOF'
jobs:
- name: hello-world-job
  plan:
  - task: say-hello
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: alpine
          tag: latest
      run:
        path: echo
        args: ["Hello from Concourse on RamNode!"]
EOF

# Set the pipeline
fly -t main set-pipeline -p hello-world -c hello-world.yml

# Unpause and trigger
fly -t main unpause-pipeline -p hello-world
fly -t main trigger-job -j hello-world/hello-world-job --watch

Troubleshooting Common Issues

Workers Not Connecting

Check that the TSA port (2222) is accessible between web and worker nodes:

# On worker, test connection to web
nc -zv concourse-web 2222

Verify key permissions:

ls -la ~/concourse/keys/
# Keys should be readable by the container user

Database Connection Errors

Verify PostgreSQL is running and accepting connections:

# Docker Compose
docker compose logs concourse-db

# Binary installation
sudo systemctl status postgresql
sudo -u postgres psql -c "SELECT 1"

Web UI Not Loading

Check that port 8080 isn't blocked by firewall:

sudo ufw allow 8080/tcp
# Or for firewalld:
sudo firewall-cmd --add-port=8080/tcp --permanent
sudo firewall-cmd --reload

Installation Complete!

Your Concourse CI instance is now running on your RamNode VPS

In Part 2, we'll dive deep into Concourse's core concepts—understanding pipelines, jobs, tasks, and resources—which form the foundation for everything you'll build.

Series Overview Part 2: Core Concepts