Tuning sysctl for High-Connection Workloads

Reach for these changes when you observe one or more of the following symptoms:

• Connections being refused or dropped under load while CPU and memory still have headroom
• dmesg showing entries like nf_conntrack: table full, dropping packet, TCP: request_sock_TCP: Possible SYN flooding, or Out of socket memory
• ss -s reporting large counts of sockets in TIME-WAIT, or netstat -s reporting connection reset and failed connection attempt counts climbing quickly
• Latency spikes on otherwise idle servers under bursts of new connections
• Applications logging EMFILE (too many open files) or EADDRNOTAVAIL (cannot assign requested address) errors

Persistent kernel parameters belong in /etc/sysctl.d/ rather than directly in /etc/sysctl.conf. Create a dedicated file for your workload tuning so it is easy to review and roll back:

sudo nano /etc/sysctl.d/99-high-connection.conf

After editing, load the changes with:

sudo sysctl --system

You can preview a single setting without persisting it using sysctl -w net.core.somaxconn=4096. Anything set this way is lost on reboot, which makes it the safest way to test a value before committing it to disk.

Every TCP socket consumes a file descriptor. Before any network tuning matters, the kernel and your process both need to be allowed to hold enough of them.

System-wide ceiling in /etc/sysctl.d/99-high-connection.conf:

fs.file-max = 2097152
fs.nr_open = 1048576

Per-process limits live outside sysctl. For traditional sysvinit-style services, edit /etc/security/limits.conf:

*    soft    nofile    1048576
*    hard    nofile    1048576
root soft    nofile    1048576
root hard    nofile    1048576

For systemd-managed services (Nginx, HAProxy, PostgreSQL, custom units), the limits.conf values are ignored. Add a drop-in instead:

sudo systemctl edit nginx.service

[Service]
LimitNOFILE=1048576

Reload and restart the service for the new limit to take effect. Verify with cat /proc/<pid>/limits | grep "open files".

When connections arrive faster than your application calls accept(), the kernel buffers them. Two queues are involved: the SYN queue (half-open handshakes) and the accept queue (completed handshakes waiting to be picked up). Both have small defaults.

net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.netdev_max_backlog = 16384

somaxconn is the upper bound on the accept queue size that an application can request via listen(). Most applications pass a small value like 511 or 1024 and silently get clamped to whatever somaxconn allows. After raising the kernel limit, you usually also need to raise the application-level value: in Nginx that is listen ... backlog=65535;, in HAProxy it is tune.maxaccept and the backlog parameter on the bind line.

netdev_max_backlog controls how many packets the kernel buffers per interface when the CPU cannot drain them fast enough. Raise this on hosts handling heavy small-packet traffic or many parallel TLS handshakes.

Outbound connections, including those from a reverse proxy to upstream servers, consume ephemeral ports. The default range gives you roughly 28,000 ports, which is easy to exhaust on a busy proxy.

net.ipv4.ip_local_port_range = 1024 65535

If your application connects to one or two upstream IP:port pairs, you can hit the limit even with the full range expanded, because the four-tuple of (src IP, src port, dst IP, dst port) must be unique. The fix in that case is to add upstream IP addresses, use Unix sockets, or enable connection pooling so you reuse existing sockets rather than opening new ones.

Closed TCP sockets sit in TIME_WAIT for twice the maximum segment lifetime (typically 60 seconds on Linux). On a server fielding many short-lived client connections this is normal and not a problem; the kernel handles millions of TIME_WAIT entries efficiently. On the client side of a connection (your reverse proxy talking to upstreams, for example), TIME_WAIT can pin ephemeral ports and cause EADDRNOTAVAIL.

net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_tw_buckets = 2000000

tcp_tw_reuse lets the kernel reuse a socket in TIME_WAIT for a new outbound connection when it is safe to do so. This is the right knob for client-side TIME_WAIT pressure. Do not enable the old tcp_tw_recycle option; it was removed in kernel 4.12 and was unsafe behind NAT even before that.

tcp_max_tw_buckets caps the total number of TIME_WAIT entries. Raising it prevents the kernel from logging TCP: time wait bucket table overflow and recycling sockets aggressively.

SYN cookies let the kernel survive a SYN flood without filling the SYN queue. Modern kernels enable them by default, but it is worth confirming and pairing them with a reasonable backlog:

net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_syn_retries = 3

Reducing the retry counts shortens how long the kernel waits on half-open connections, which frees queue slots faster during a flood. The trade-off is slightly faster failure for legitimate clients on lossy networks.

For workloads that move a lot of data over each connection (file downloads, video streaming, database replication, internal service meshes), the TCP send and receive buffers gate throughput. The defaults are tuned for modest LAN traffic.

net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.ipv4.tcp_rmem = 4096 262144 16777216
net.ipv4.tcp_wmem = 4096 262144 16777216
net.ipv4.tcp_mem = 786432 1048576 1572864

The three values in tcp_rmem and tcp_wmem are minimum, default, and maximum. The kernel auto-tunes within that range. For workloads with many small connections (HTTP APIs, Redis, memcached), the defaults are usually fine and raising buffers wastes memory; for long-lived streams or high bandwidth-delay product paths, the larger maximums matter.

tcp_mem is in 4 KB pages and sets soft and hard memory ceilings across all TCP sockets. On a 4 GB VPS the values above are a sensible starting point; scale up roughly with available RAM.

If you use a stateful firewall, including iptables -m state, nftables with stateful rules, firewalld, or ufw, every connection consumes a conntrack entry. The default table is small and overflow drops packets silently except for a dmesg warning.

net.netfilter.nf_conntrack_max = 1048576
net.netfilter.nf_conntrack_tcp_timeout_established = 86400
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30

If nf_conntrack is not loaded, these keys will not exist and sysctl will warn on apply. Load the module with modprobe nf_conntrack and persist it via /etc/modules-load.d/. You can also adjust the hash table size at module load time using /etc/modprobe.d/conntrack.conf:

options nf_conntrack hashsize=262144

A reasonable rule of thumb is hashsize equal to nf_conntrack_max / 4. The memory cost is around 300 bytes per conntrack entry plus 8 bytes per hash bucket.

For workloads that do not need stateful filtering, the cleanest answer is to bypass conntrack entirely. Add a NOTRACK rule in the raw table for the relevant ports, or move the firewall rules off the loopback and inter-service paths.

Defaults wait two hours before sending the first keepalive probe, which is far too long for detecting dead peers on long-lived connections such as WebSockets, MQTT brokers, or database connection pools.

net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 5

With these values a dead peer is detected in roughly 7.5 minutes (300 + 5 × 30 seconds). Tighten further if your application relies on the kernel rather than its own heartbeat to surface dead connections. Note that applications must explicitly enable SO_KEEPALIVE on a socket for these timers to apply.

BBR generally outperforms the default cubic congestion control for long-lived flows and high-bandwidth paths. It ships in mainline kernels and is safe to enable:

net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

Confirm BBR is available with sysctl net.ipv4.tcp_available_congestion_control. If bbr is not listed, load the module: modprobe tcp_bbr and add tcp_bbr to /etc/modules-load.d/bbr.conf to persist it.

For short-lived HTTP traffic the difference is small; for video, large file transfers, and replication links it can be substantial.

Below is a complete profile suitable for a busy Nginx or HAProxy front-end on a 4 to 8 GB RamNode VPS. Save it as /etc/sysctl.d/99-high-connection.conf, review every value against the explanations above, and adjust before applying:

# File descriptors
fs.file-max = 2097152
fs.nr_open = 1048576

# Listen queues
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 16384
net.ipv4.tcp_max_syn_backlog = 65535

# Ephemeral ports and TIME_WAIT
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_tw_buckets = 2000000

# SYN flood handling
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_syn_retries = 3

# Socket buffers
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 262144 16777216
net.ipv4.tcp_wmem = 4096 262144 16777216

# Keepalive
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 5

# Congestion control
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

# Conntrack (only if a stateful firewall is in use)
net.netfilter.nf_conntrack_max = 1048576
net.netfilter.nf_conntrack_tcp_timeout_established = 86400
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30

Apply with:

sudo sysctl --system

After applying, confirm each value is live:

sysctl net.core.somaxconn net.ipv4.tcp_tw_reuse net.ipv4.ip_local_port_range

Check the listening socket actually picked up the new backlog. The Recv-Q column under LISTEN state shows the configured backlog:

ss -ltn

Watch for kernel warnings during load tests:

sudo dmesg -wT

And track network counters over time. The nstat tool gives you per-counter deltas, which is far more useful than the cumulative numbers in netstat -s:

nstat -a; sleep 60; nstat

Because every change above lives in a single file under /etc/sysctl.d/, rollback is a one-line operation:

sudo mv /etc/sysctl.d/99-high-connection.conf /root/
sudo sysctl --system

Some settings (notably congestion control) persist on existing sockets until they close, so a full effect rollback may require restarting the affected service or rebooting.

Once the kernel side is no longer the bottleneck, the next round of tuning typically lives in your application layer: worker counts, event-loop sizing, upstream connection pools, TLS session caches, and HTTP/2 multiplexing. Kernel tuning gives the application room to scale; the application still has to use that room well.

If you run into a specific symptom you cannot resolve, open a ticket with our support team and include the output of ss -s, nstat -a, sysctl -a | grep -E "tcp_|somaxconn|netdev", and recent dmesg lines. That set of data is usually enough to pinpoint which sysctl group needs further adjustment.

Related Reading

• Performance Optimization — right-sizing CPU, RAM, and disk
• Diagnosing High CPU Usage
• Noisy Neighbor Symptoms vs. Real Performance Issues
• Network Troubleshooting Toolkit