Network
Nginx — reverse proxy, TLS termination, and debugging 502s
Fast-lookup reference for Nginx as a reverse proxy: server/location blocks, proxy_pass header forwarding, TLS termination, upstream load balancing, static files, gzip, rate limiting, and the test-then-reload cycle — centered on diagnosing 502/504 errors.
Quick reference — reach for these first
nginx -tParse every config file and the vhosts; prints the first syntax/path error with file:line. ALWAYS run before reload.sudo systemctl reload nginxGraceful reload: workers re-read config with zero dropped connections. Config edits do nothing until this runs. PROD-CHANGING.location / {
proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}Pass requests to an upstream app and preserve the original host, client IP, and scheme.sudo tail -f /var/log/nginx/error.logPrimary signal for 502/504/permission faults — look for 'connect() failed', 'upstream timed out', '(13: Permission denied)'.curl -I -H 'Host: app.example.com' http://127.0.0.1Hit a specific server_name without DNS — confirms which server block answers and its status code.Request flow: client → Nginx (TLS + reverse proxy) → upstream pool
Text equivalent: A client sends an HTTPS request to Nginx. Nginx terminates TLS on port 443, sets Host, X-Forwarded-For and X-Forwarded-Proto headers, then proxies plain HTTP to an upstream pool of two app servers on ports 3000 and 3001. A note shows that when the upstream connection is refused the client gets 502, and when it times out the client gets 504.
Common tasks
Serve & proxy · TLS & performance · Operate
| Serve & proxy | TLS & performance | Operate |
|---|---|---|
| Check the installed version and modules | nginx -V 2>&1 | tr ' ' '\n' | grep -E 'nginx|ssl|module' | Prints 'nginx version: nginx/1.x.y' and the compiled-in modules (e.g. --with-http_ssl_module). |
| Define a minimal server block for a vhost | server {
listen 80;
server_name app.example.com;
root /var/www/app;
} | curl -I -H 'Host: app.example.com' http://127.0.0.1 returns 200/301/404 from THIS block, not the default_server. |
| Reverse-proxy requests to an upstream app | location / { proxy_pass http://127.0.0.1:3000; } | Request reaches the app; its access log shows the hit and the browser gets the app's response (not 502). |
| Forward Host and real client IP to the app | proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; | App logs show the real client IP (via X-Forwarded-For) and the original Host, not 127.0.0.1 / the upstream address. |
| Terminate TLS on port 443 | server {
listen 443 ssl;
server_name app.example.com;
ssl_certificate /etc/ssl/app/fullchain.pem;
ssl_certificate_key /etc/ssl/app/privkey.pem;
} | curl -I https://app.example.com succeeds; openssl s_client -connect app.example.com:443 shows the cert chain and 'Verify return code: 0 (ok)'. |
| Redirect all HTTP to HTTPS | server {
listen 80;
server_name app.example.com;
return 301 https://$host$request_uri;
} | curl -I http://app.example.com returns '301 Moved Permanently' with a Location: https://... header. |
| Define an upstream pool with load balancing | upstream app {
least_conn;
server 127.0.0.1:3000;
server 127.0.0.1:3001;
}
# then: proxy_pass http://app; | Repeated requests are spread across both backends; stopping one still serves traffic from the other. |
| Serve static files with a SPA-safe fallback | location / {
root /var/www/app;
try_files $uri $uri/ /index.html;
} | Existing files return 200; an unknown deep path (e.g. /dashboard) serves index.html (200), not 404. |
| Enable gzip for text responses | gzip on;
gzip_types text/plain text/css application/json application/javascript; | curl -s -H 'Accept-Encoding: gzip' -D - -o /dev/null https://app.example.com/app.js shows 'Content-Encoding: gzip'. Use a GET (-D -), NOT -I/HEAD — nginx does not gzip HEAD responses, so -I makes a working config look broken. |
| Rate-limit a sensitive endpoint | # http {}:
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
# location:
limit_req zone=api burst=20 nodelay; | A rapid burst past the limit returns 503 and error.log records 'limiting requests, excess:' for that client IP. |
| Validate, then gracefully reload (PROD-CHANGING) | sudo nginx -t && sudo systemctl reload nginx | nginx -t prints 'test is successful'; reload exits 0; systemctl status nginx shows 'active (running)' and the new behavior is live. |
| Tail both logs while reproducing an issue | sudo tail -f /var/log/nginx/access.log /var/log/nginx/error.log | A fresh request appears in access.log with its status code; any fault (upstream/permission) appears in error.log in the same instant. |
Debug a 502 Bad Gateway end-to-end
502
curl -I https://app.example.com
# HTTP/1.1 502 Bad Gateway
# Server: nginxsudo tail -n 20 /var/log/nginx/error.log
# connect() failed (111: Connection refused) while connecting to upstream,
# upstream: "http://127.0.0.1:3000/", host: "app.example.com"curl -i 127.0.0.1:3000
# curl: (7) Failed to connect to 127.0.0.1 port 3000: Connection refusedsudo ss -ltnp | grep ':3000'
# (no output — nothing is bound to :3000, so the app is down)grep -rn proxy_pass /etc/nginx/
# /etc/nginx/sites-enabled/app:14: proxy_pass http://127.0.0.1:3000;sudo systemctl start myapp && sudo ss -ltnp | grep ':3000'
# LISTEN 0 511 127.0.0.1:3000 ... users:(("node",pid=...))sudo nginx -t
# nginx: configuration file /etc/nginx/nginx.conf test is successfulsudo systemctl reload nginx
# (exit 0 — zero dropped connections)curl -I https://app.example.com
# HTTP/1.1 200 OKTroubleshooting — symptom → first check → next check
| Symptom | First check | Establishes | Next check |
|---|---|---|---|
| 502 Bad Gateway on a proxied route | sudo tail -n 20 /var/log/nginx/error.log then curl -i 127.0.0.1:PORT | Whether Nginx reached the upstream. 'connect() failed (111: Connection refused)' = the app is down or not bound to that host:port; 'no live upstreams' = every pool member is marked failed. | Start the app / bind it to the exact host:port in proxy_pass (127.0.0.1 vs 0.0.0.0 matters); fix the port; confirm with curl to the upstream, then nginx -t and reload. |
| 504 Gateway Timeout under load or on slow requests | grep 'upstream timed out' /var/log/nginx/error.log and time the upstream directly: curl -w '%{time_total}\n' -o /dev/null -s 127.0.0.1:PORT/slow | The upstream accepts the connection but does not respond within proxy_read_timeout (default 60s) — a slow backend, not a down one. | Fix the slow backend first; if legitimately long, raise proxy_read_timeout / proxy_connect_timeout in that location, then nginx -t and reload. Do not mask a real latency bug with a huge timeout. |
| 403 Forbidden or 404 Not Found on static files | Check the resolved path and permissions: namei -l /var/www/app/index.html and read error.log for '(13: Permission denied)' or '(2: No such file)' | 403 = Nginx found the path but the worker user (www-data/nginx) can't traverse/read it, or directory listing is off; 404 = root/try_files computed a path that doesn't exist. | For 403: chmod o+x each parent dir and make files readable by the worker user. For 404: fix root/alias and the try_files fallback. Reload after nginx -t. |
| Config edit has no effect in the browser | sudo nginx -t then confirm a reload actually ran: systemctl status nginx | grep -i reload | Either nginx -t failed (so the old config is still live) or the file was edited but never reloaded — Nginx serves the last successfully loaded config until reload/restart. | Fix any nginx -t error (it names file:line), ensure you edited the file actually included by nginx.conf, then sudo systemctl reload nginx. Bypass caches with curl, not the browser. |
| TLS/HTTPS fails to start or serves the wrong/no cert | sudo nginx -t and openssl s_client -connect app.example.com:443 -servername app.example.com </dev/null | nginx -t flags a bad ssl_certificate/ssl_certificate_key path or key mismatch; '(13: Permission denied)' on the key means the worker user can't read it. s_client shows which cert (if any) is actually served per SNI. | Point ssl_certificate at the FULLCHAIN (leaf+intermediates), match the correct key, and make the key readable by nginx (root:nginx, 640). Verify cert/key pair, nginx -t, then reload. |
References
- Nginx Documentation (index)nginx.org
- Beginner's Guidenginx.org
- ngx_http_proxy_module (proxy_pass, proxy_set_header, timeouts)nginx.org
- ngx_http_upstream_module (load balancing)nginx.org
- Configuring HTTPS serversnginx.org
Category kept EXACTLY as "Network" per the brief, even though the relayed user request mentioned higher-level groupings like "AI SRE" — that grouping is a hub-navigation concern applied across sheets, not a per-sheet category override. Config examples use placeholder hosts/ports (app.example.com, 127.0.0.1:3000/3001) and Debian-style paths (/var/log/nginx, www-data worker user); on RHEL/Alpine the worker user and paths differ.