•3 min read•533 words

Zero-Downtime Load Balancing with Nginx and HAProxy in Production

Architectural patterns for high-availability Layer 4 and Layer 7 load balancing, active health checks, graceful connection draining, and kernel TCP tuning.

NginxHAProxyNetworkingLinuxSRE

In high-throughput production environments, rolling out application updates or surviving upstream node failures without dropping active requests comes down to how your reverse proxies and load balancers handle connection draining, health probing, and keepalive pools.

Here is how we architect HAProxy and Nginx for zero-downtime traffic management.

1. Layer 4 vs. Layer 7 Load Balancing#

Choosing between HAProxy and Nginx—or combining both in a two-tier ingress topology—depends on your protocol requirements and failure domains:

CapabilityHAProxy (L4 / L7)Nginx (L7 Reverse Proxy & Web Server)
Primary StrengthUltra-fast TCP/HTTP load balancing & rich health checksStatic asset serving, caching, TLS termination, routing
ObservabilityBuilt-in real-time Prometheus exporter & stats socketStub status module (detailed metrics via exporter)
Dynamic DrainInstant weight/drain control via runtime UNIX socketGraceful worker reload (nginx -s reload)

2. Production HAProxy Configuration with Active Health Checks#

A resilient HAProxy tier should configure explicit connection timeouts, HTTP health checks, and redispatch so in-flight connection attempts transparently retry on a healthy backend if a node goes down mid-deploy:

/etc/haproxy/haproxy.cfg
haproxy
1global
2    log /dev/log local0 info
3    maxconn 50000
4    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
5    user haproxy
6    group haproxy
7    daemon
8
9defaults
10    mode http
11    log global
12    option httplog
13    option dontlognull
14    option redispatch
15    retries 3
16    timeout connect 3s
17    timeout client  30s
18    timeout server  30s
19    timeout http-keep-alive 10s
20
21backend app_nodes
22    balance leastconn
23    option httpchk
24    http-check send meth GET uri /healthz ver HTTP/1.1 hdr Host geenodh.me
25    http-check expect status 200
26    default-server inter 2s fall 3 rise 2 maxconn 1000
27    server app01 10.20.1.11:8080 check
28    server app02 10.20.1.12:8080 check
29    server app03 10.20.1.13:8080 check

3. Graceful Draining During Rolling Deployments#

Before stopping an application container or systemd service on app01, put the server into drain mode via the HAProxy runtime socket. This stops new sessions from routing to app01 while allowing existing in-flight requests to complete cleanly:

Rolling Drain Workflow via socat
bash
$echo 'set server app_nodes/app01 state drain' | socat stdio /run/haproxy/admin.sock
$echo 'show stat' | socat stdio /run/haproxy/admin.sock | grep app01
$echo 'set server app_nodes/app01 state ready' | socat stdio /run/haproxy/admin.sock

4. Upstream Keepalive Tuning in Nginx#

One of the most common misconfigurations in Nginx is omitting keepalive in the upstream block. Without it, Nginx opens a brand-new ephemeral TCP connection to your backend for every single incoming HTTP request, leading to ephemeral port exhaustion (TIME_WAIT accumulation) under load.

/etc/nginx/conf.d/upstream.conf
nginx
1upstream backend_service {
2    least_conn;
3    server 10.20.1.11:8080 max_fails=3 fail_timeout=10s;
4    server 10.20.1.12:8080 max_fails=3 fail_timeout=10s;
5    keepalive 64;
6}
7
8server {
9    listen 443 ssl http2;
10    server_name geenodh.me;
11
12    location /api/ {
13        proxy_pass http://backend_service;
14        proxy_http_version 1.1;
15        proxy_set_header Connection "";
16        proxy_set_header Host $host;
17        proxy_set_header X-Real-IP $remote_addr;
18        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
19        proxy_next_upstream error timeout http_502 http_503 http_504;
20    }
21}