Troubleshooting an Instance That Will Not Boot

All RamNode KVM VPS plans include VNC console access through the client portal. This gives you a view into the virtual screen output of your server, exactly as if you were sitting in front of a physical machine.

To open the VNC console:

1. Log in to the RamNode Client Portal
2. Navigate to Services and select your VPS
3. Click VNC Console (or Launch Console depending on your panel view)
4. A browser-based VNC session will open — wait a few seconds for the display to populate

When your VPS fails to boot, the VNC console is your primary diagnostic tool. Do not reboot repeatedly before reading what is on screen — the error message displayed at the point of failure tells you almost everything you need to know.

What you should see during a healthy boot

GRUB loading...
Welcome to GRUB!

[Linux kernel version line]
[  OK  ] Started Journal Service.
[  OK  ] Reached target Local File Systems.
[  OK  ] Started OpenSSH server daemon.
...
Ubuntu 22.04.3 LTS hostname tty1
hostname login:

Each stage either completes successfully or produces an error. Your goal is to identify exactly where the sequence stops.

Reading failure output

Before diving into manual recovery steps, try one quick option that resolves some issues automatically.

Check for an older kernel in the GRUB menu

If GRUB loads and shows a boot menu, look for Advanced options or a list of older kernel versions. A bad kernel update is a common cause of boot failures, and older kernels are often still installed alongside the new one.

1. When the GRUB menu appears, do not let it auto-boot. Press any key to pause the countdown
2. Select Advanced options for Ubuntu (or your distribution equivalent)
3. Choose a kernel from before the failed update — typically the second or third entry in the list
4. If the system boots successfully with the older kernel, the issue is a bad kernel update. See Scenario 2 below

Symptoms

• System drops to emergency mode: You are in emergency mode. After logging in, type "journalctl -xb" to view system logs
• Console shows: ALERT! /dev/disk/by-uuid/... does not exist. Dropping to a shell
• A mount-related failure appears in the boot sequence before the system goes interactive
• Boot hangs indefinitely on a A start job is running for /etc/fstab... line

What happened

The /etc/fstab file tells the kernel which filesystems to mount and where. If an entry references a device or UUID that does not exist — or if you added a line with incorrect syntax — the boot process fails when it cannot satisfy that mount requirement. Common triggers include adding a network share, an additional block device, or a swap file entry that is no longer valid.

Recovery steps

If you are dropped to an emergency shell (you will see a # prompt), remount the filesystem as read-write:

mount -o remount,rw /

nano /etc/fstab

To find the correct UUIDs for your actual block devices:

blkid

The safest fix for a problematic non-essential mount entry is to either comment it out or add the nofail option:

# Before (causes boot failure if device missing):
UUID=abc123  /mnt/data  ext4  defaults  0  2

# After (boots normally even if device is absent):
UUID=abc123  /mnt/data  ext4  defaults,nofail  0  2

reboot

Symptoms

• System was rebooted after running apt upgrade, yum update, or similar, and now does not boot
• Kernel loads but immediately produces a kernel panic
• System hangs shortly after the kernel version line prints, before systemd starts
• Older kernel entries in the GRUB menu boot successfully

What happened

A kernel update occasionally ships with a bug, an incompatible module, or a broken initramfs image. This is more common when using distribution-provided kernels on non-standard hardware configurations, or when a partial upgrade was interrupted.

Option A: Boot the previous kernel (no data risk)

1. Reboot the VPS from the portal
2. Watch the VNC console and press any key when the GRUB menu appears to pause it
3. Select Advanced options and choose the previous kernel version
4. If the system boots successfully, set the old kernel as default and remove the broken one:

# List installed kernels
dpkg --list | grep linux-image

# Set default kernel (Ubuntu/Debian - use exact kernel name from dpkg output)
sudo nano /etc/default/grub
# Change GRUB_DEFAULT=0 to the exact menuentry string, or use a saved entry:
# GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.15.0-91-generic"

sudo update-grub

# Optionally remove the broken kernel
sudo apt remove linux-image-6.x.x-xx-generic
sudo update-grub

Option B: Rebuild the initramfs

Sometimes the kernel update succeeds but the initramfs image is corrupted or was not regenerated properly:

# Check which initramfs files exist
ls -lh /boot/initrd.img-*

# Regenerate for a specific kernel version
sudo update-initramfs -u -k 6.x.x-xx-generic

# Or regenerate all installed kernels
sudo update-initramfs -u -k all

sudo update-grub
reboot

Option C: If GRUB menu is hidden or times out too fast

GRUB_TIMEOUT=10
GRUB_TIMEOUT_STYLE=menu

sudo update-grub
reboot

Symptoms

• Boot log shows errors like No space left on device or Input/output error
• System enters emergency mode and the root filesystem is mounted read-only
• Journald or other early-boot services fail to start because they cannot write to disk
• Prior to the boot failure, you may have seen df -h showing / at 100% use

What happened

When a Linux filesystem reaches 100% capacity, the kernel cannot write temporary files, logs, or PID files that services require to start. Common causes include runaway log files, a filled /tmp directory, a large Docker image cache, or accumulated package files.

Recovery steps

mount -o remount,rw /

df -h
du -sh /* 2>/dev/null | sort -rh | head -20

# Clear old package cache (Debian/Ubuntu)
apt-get clean
apt-get autoremove

# Clear journal logs older than 2 days
journalctl --vacuum-time=2d

# Find and remove large log files
find /var/log -name "*.log" -size +50M
find /var/log -name "*.gz" -delete

# Check for large files in /tmp
du -sh /tmp && rm -rf /tmp/*

# If Docker is installed - check image/container disk usage
docker system df
docker system prune -f

If the disk is critically full, even running commands can be difficult. Find the single largest file directly:

find / -xdev -type f -printf '%s %p\n' 2>/dev/null | sort -rn | head -5

Preventing this in the future

• Set up disk usage monitoring through RamNode's provided metrics or a tool like Netdata or Prometheus node exporter
• Configure logrotate to cap log sizes: /etc/logrotate.conf
• Add a cron job to alert when disk usage exceeds 80%:

# Add to root's crontab (crontab -e)
0 * * * * df / | awk 'NR==2 {if ($5+0 > 80) print "Disk at "$5}' | mail -s "Disk Warning" you@example.com

Symptoms

• VNC shows a blank screen with a grub> or grub rescue> prompt and nothing else
• Error message: error: no such partition or error: unknown filesystem
• Error message: minimal BASH-like line editing is supported (GRUB emergency shell)
• GRUB splash screen appears but all menu entries fail with file not found errors

What happened

GRUB is the bootloader responsible for loading the Linux kernel. It can fail when the MBR or EFI boot entry was overwritten, the GRUB configuration file was deleted or corrupted, the boot partition UUID changed, or a partial GRUB update left inconsistent files.

Recovery: From the GRUB rescue prompt

If you see grub rescue>, GRUB loaded minimally but cannot find its configuration. Identify the correct partition:

grub rescue> ls
(hd0) (hd0,msdos1) (hd0,msdos2)

grub rescue> ls (hd0,msdos1)/
# Try each partition until you see /boot/grub/ listed

grub rescue> set root=(hd0,msdos1)
grub rescue> set prefix=(hd0,msdos1)/boot/grub
grub rescue> insmod normal
grub rescue> normal

After booting, reinstall GRUB properly

# Determine your root disk (not partition - usually /dev/vda)
lsblk

# Reinstall GRUB to the disk (Debian/Ubuntu, MBR/BIOS systems)
sudo grub-install /dev/vda
sudo update-grub

# For UEFI systems
sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu
sudo update-grub

Recovery: Reinstall GRUB from rescue mode

If the GRUB rescue prompt approach does not work, boot into rescue mode (see below), then chroot:

# Mount necessary filesystems for chroot
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys

# If UEFI, also mount the EFI partition
mount /dev/vda1 /mnt/boot/efi   # adjust partition as needed

# Enter chroot
chroot /mnt

# Reinstall GRUB
grub-install /dev/vda
update-grub

# Exit and reboot
exit
reboot

When your system cannot boot far enough to give you a recovery shell, you need to access the disk from outside the broken OS. RamNode provides two paths for this.

Option A: Boot from a RamNode rescue image

1. Log in to the RamNode Client Portal
2. Select your VPS and look for Rescue Mode, Boot ISO, or Reinstall options
3. Select a rescue or live Linux image (Debian or Ubuntu rescue images are recommended)
4. Boot the server and connect via VNC — you will be in a minimal live environment
5. Mount your root partition to access your files:

# Find your root partition
lsblk
fdisk -l

# Mount the root volume
mkdir -p /mnt/root
mount /dev/vda1 /mnt/root   # adjust partition name as shown by lsblk

# If you have a separate /boot partition, mount it too
mount /dev/vda2 /mnt/root/boot

Make your repairs (edit fstab, clear disk space, reinstall GRUB via chroot), then unmount and reboot:

umount /mnt/root
reboot

Option B: Single-user / recovery mode via GRUB

If GRUB loads and you can access the boot menu:

1. Highlight your default kernel entry and press e to edit it
2. Find the line starting with linux and locate ro quiet splash near the end
3. Replace ro quiet splash with rw init=/bin/bash or add single at the end of the linux line
4. Press Ctrl+X or F10 to boot with these parameters
5. You will be dropped to a root shell with the filesystem mounted read-write

Most boot failures are recoverable using the steps above. However, contact RamNode Support if:

• The VNC console shows nothing at all and the server does not respond to power cycles from the portal
• You see hardware-level errors such as I/O error on device vda — this may indicate an issue with the underlying storage that requires intervention on our end
• The rescue image fails to load or VNC is unresponsive after multiple attempts
• You are not comfortable performing chroot operations and want assistance recovering the system

When submitting a ticket, include:

• Your VPS hostname or IP address
• A screenshot of the VNC output at the point of failure
• The approximate time the issue started
• Any recent changes you made before the failure (updates, disk operations, config edits)