Opscraft
Nginx & reverse proxies · Lesson 5 of 6 · 5 min read

Nginx in Kubernetes and debugging

Ingress controllers and the Gateway API

An Ingress controller is a reverse proxy that configures itself from Kubernetes Ingress objects. For years the most common one was the community ingress-nginx project, configured mostly through annotations such as nginx.ingress.kubernetes.io/proxy-body-size.

The Kubernetes project retired ingress-nginx in March 2026: no more releases, bug fixes or security patches. Clusters still running it should migrate. Note that F5's NGINX Ingress Controller (nginx/kubernetes-ingress) is a separate, still-maintained product with different annotations, so "we use Nginx ingress" needs a follow-up question.

The recommended direction is the Gateway API, the successor to Ingress:

Ingress Gateway API
One object mixes infrastructure and routing GatewayClass (which implementation), Gateway (listeners, owned by the platform team), HTTPRoute (routes, owned by app teams)
Features via controller-specific annotations Header matching, traffic weights, redirects and rewrites in the standard spec
HTTP only in the core spec HTTP, gRPC, TLS and TCP routes

Implementations include Envoy Gateway, Istio, Cilium, Kong, Traefik, NGINX Gateway Fabric and, on AWS, the AWS Load Balancer Controller and VPC Lattice. A migration plan: inventory every Ingress and its annotations, map each annotation to a Gateway API feature or a policy resource, run the new controller alongside the old one, move host names one at a time by DNS, then remove ingress-nginx.

Enlarge diagramGatewayClasswhich implementation (Envoy Gateway, Istio, AWS…)Gatewaylisteners, TLS, owned by the platform teamHTTPRoutehosts, paths, weights, owned by an app teamServicethe app’s pods
Figure 1. The Gateway API splits ownership: the platform team owns the Gateway, app teams own their routes.
Swipe sideways to see all of it, or tap to open it full screen.

Debugging with logs

The access log tells you what happened; the error log tells you why. Add timing fields to the access log:

log_format timed '$remote_addr "$request" $status $body_bytes_sent '
                 'rt=$request_time uct=$upstream_connect_time urt=$upstream_response_time '
                 'ua="$upstream_addr" us=$upstream_status';

$request_time is the total time; $upstream_response_time is the backend's part. A big gap between them points at a slow client or buffering, not the app.

Symptom Error log says Usual cause
502 Bad Gateway connect() failed (111: Connection refused) while connecting to upstream App not listening, wrong port, container still starting
502 upstream prematurely closed connection App crashed mid-request, or its keep-alive timeout is shorter than Nginx's
502 no live upstreams Every server marked failed by max_fails
504 Gateway Timeout upstream timed out (110: Connection timed out) while reading response header App too slow for proxy_read_timeout, or a blocked connection
413 client intended to send too large body Raise client_max_body_size (and the Ingress annotation)
499 (in access log) — The client gave up before Nginx answered (often a shorter timeout in front)

Workflow: reproduce with curl -v -H 'Host: shop.example.com' http://127.0.0.1/path, read the matching error-log line, check nginx -T to see which server and location really apply, then test the upstream directly with curl from the proxy host.

Enlarge diagram502 Bad Gateway504 Gateway Timeoutconnection refused: nothing listeningupstream closed early: crash or keep-alivemismatchno live upstreams: all marked failedconnected, but no response in timeslow query, lock or overloaded apptune proxy_read_timeout last, fix the slownessfirst
Figure 2. Read the error log line, not just the status code.
Swipe sideways to see all of it, or tap to open it full screen.

Alternatives

Proxy Strength Typical use
Nginx Fast static serving, mature, simple config Web servers, edge proxies, small ingress
HAProxy Excellent L4/L7 balancing, active health checks, detailed stats High-traffic load balancing, TCP services
Envoy Dynamic config over APIs (xDS), rich observability, gRPC, retries Service meshes, Gateway API implementations
Traefik Auto-discovers routes from Docker and Kubernetes labels Small clusters, Docker hosts
Caddy Automatic HTTPS with almost no config Small sites, internal tools