Fixing Node.js 502 errors during cloud rolling deploys

Teams migrating Node.js applications to containerized cloud environments frequently encounter brief but disruptive bursts of 502 errors during every deployment. These outages occur because standard Node.js processes do not inherently coordinate with load balancers when receiving shutdown signals, causing active requests to drop. Implementing specific health check endpoints and graceful shutdown logic resolves these timing mismatches between the orchestrator and the infrastructure.

What happened

When a service moves from a single virtual machine to a managed container environment like Kubernetes or ECS, the deployment strategy typically shifts to rolling updates. In this model, new instances start while old ones are stopped. The core issue arises from a race condition: the orchestrator sends a SIGTERM signal to the old container and simultaneously instructs the load balancer to remove it from the rotation. However, the load balancer often lags by several seconds in updating its routing table. Since Node.js exits immediately upon receiving SIGTERM by default, the process dies before the load balancer stops sending it traffic, resulting in failed requests.

The problem is compounded by startup behaviors and connection management. New containers may receive traffic before their database connections are fully established if no readiness check is in place. Additionally, mismatched keep-alive timeouts between Node.js and cloud load balancers cause further errors. For instance, AWS Application Load Balancers hold idle connections for 60 seconds, while Node.js defaults to 5 seconds. When the balancer attempts to reuse a socket that Node has already closed, users experience 502 errors even outside of deployment windows.

Key details

• Rolling deploys trigger SIGTERM signals that kill Node.js processes before load balancers stop routing traffic to them.
• Liveness probes should only check if the event loop is responsive, never checking database connectivity to avoid cascading restarts.
• Readiness probes must verify dependency availability and return a 503 status during shutdown to drain traffic safely.
• Node.js keepAliveTimeout must be set higher than the load balancer’s idle timeout to prevent socket reuse errors.
• Using CMD npm start in Docker prevents proper signal forwarding; developers should use exec form or an init process like tini.
• Database pools and other resources must be closed only after the HTTP server has finished processing in-flight requests.

Background

Understanding the distinction between liveness and readiness is critical for stable cloud operations. Liveness checks answer whether a process is stuck and needs restarting, while readiness checks determine if an instance can accept traffic. Confusing these two leads to common pitfalls, such as restarting all instances simultaneously when a database slows down, which turns a minor latency spike into a total outage. Graceful shutdown is the practice of stopping new work, finishing existing tasks, and then exiting, ensuring no data is lost or corrupted during the transition.

Signal handling in Linux containers also requires attention to process IDs. In many Docker configurations, the shell or package manager runs as PID 1, which does not reliably forward termination signals to the child Node.js process. Without proper signal propagation, the application ignores the shutdown request until the orchestrator forces a kill after the grace period expires, bypassing any cleanup code entirely.

Why it matters

Ignoring these brief deployment errors creates a hidden tax on engineering velocity and system reliability. Teams often tolerate small blips, leading to less frequent deployments and larger, riskier release batches. This hesitation slows down feedback loops and makes rollbacks more difficult. Furthermore, client-side retries triggered by random 502 errors can duplicate non-idempotent actions, such as creating multiple orders or charging customers twice, turning a technical glitch into a business incident.

For IT leads and DevOps engineers, these noise-induced alerts degrade monitoring effectiveness. When real incidents are buried under routine deployment failures, teams loosen alert thresholds, reducing their ability to detect genuine problems. Ensuring zero-downtime deployments restores confidence in the release process, allowing teams to ship smaller changes more frequently without fear of disrupting users or corrupting data.

What you can do

• Implement separate /healthz and /readyz endpoints, ensuring liveness checks remain lightweight and independent of external dependencies.
• Configure your Node.js server to listen for SIGTERM, set a flag to reject new requests, and wait for the load balancer deregistration delay before closing connections.
• Set server.keepAliveTimeout to a value exceeding your load balancer’s idle timeout, such as 65 seconds for AWS ALB.
• Use the exec form CMD ["node", "server.js"] in your Dockerfile or include an init process to ensure SIGTERM reaches your application.
• Close database pools and background job consumers only after the HTTP server has confirmed all in-flight requests are complete.
• Validate your setup by running load tests during a rollout restart, aiming for zero non-2xx responses throughout the process.