AWS ALB 502 Bad Gateway: all targets failing health checks and how to fix it

Symptoms

ALB returns HTTP 502 to all requests. The target group health checks show all registered targets as unhealthy:

# Check target health status
aws elbv2 describe-target-health \
  --target-group-arn arn:aws:elasticloadbalancing:eu-central-1:123456789012:targetgroup/my-app/abc123

# Output shows all targets unhealthy:
# {
#   "TargetHealthDescriptions": [
#     {
#       "Target": { "Id": "10.0.1.42", "Port": 8080 },
#       "TargetHealth": {
#         "State": "unhealthy",
#         "Reason": "Target.ResponseCodeMismatch",
#         "Description": "Health checks failed with these codes: [503]"
#       }
#     }
#   ]
# }

# CloudWatch shows ELB 5xx spike
aws cloudwatch get-metric-statistics \
  --namespace AWS/ApplicationELB \
  --metric-name HTTPCode_ELB_502_Count \
  --dimensions Name=LoadBalancer,Value=app/my-alb/abc123 \
  --start-time 2026-07-07T08:00:00Z \
  --end-time 2026-07-07T09:00:00Z \
  --period 60 \
  --statistics Sum

The application itself may be running fine - the problem is that the ALB cannot confirm target health and removes all targets from rotation.

Cause

ALB requires at least one healthy target to forward traffic. When all targets fail health checks, it returns 502. The four most common causes:

• Security group misconfiguration. The ALB security group can reach the targets on the health check port, but the target security group does not allow inbound from the ALB SG. Traffic from the ALB is blocked before it ever hits the application.
• Health check path returns non-200. The configured health check path (/health, /, or custom) returns 4xx or 5xx. This happens when the app requires authentication on the health endpoint, when the path does not exist, or when the app has not finished booting.
• Health check timeout too short. The timeout is shorter than the app’s response time under load. The ALB marks targets as unhealthy because they respond too slowly, not because they are down.
• Target deregistration during rolling deploys. ECS or ASG drains old targets before new ones pass health checks. If deregistration_delay is too short or new task startup time is too long, there is a window where zero healthy targets exist.

Fix

Step 1: Verify security group connectivity

The ALB must be able to reach targets on the health check port. Check both security groups:

# Find ALB security group
aws elbv2 describe-load-balancers \
  --names my-alb \
  --query 'LoadBalancers[0].SecurityGroups'

# Find target security group (ECS tasks or EC2 instances)
aws ec2 describe-security-groups \
  --group-ids sg-target123 \
  --query 'SecurityGroups[0].IpPermissions'

The target SG must allow inbound from the ALB SG on the target port:

# Terraform: correct security group rules

resource "aws_security_group" "alb" {
  name   = "alb-sg"
  vpc_id = var.vpc_id

  egress {
    from_port       = var.container_port
    to_port         = var.container_port
    protocol        = "tcp"
    security_groups = [aws_security_group.ecs_tasks.id]
  }
}

resource "aws_security_group" "ecs_tasks" {
  name   = "ecs-tasks-sg"
  vpc_id = var.vpc_id

  ingress {
    from_port       = var.container_port
    to_port         = var.container_port
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
  }
}

Step 2: Fix health check path and expected codes

Ensure the health check path exists and returns 200 without authentication:

# Test the health check endpoint directly from a bastion or within the VPC
curl -v http://10.0.1.42:8080/health
# Must return HTTP 200 with a body

If your app returns 204 or other success codes, update the matcher:

resource "aws_lb_target_group" "app" {
  name     = "my-app-tg"
  port     = 8080
  protocol = "HTTP"
  vpc_id   = var.vpc_id

  health_check {
    enabled             = true
    path                = "/health"
    port                = "traffic-port"
    protocol            = "HTTP"
    matcher             = "200-299"
    interval            = 30
    timeout             = 10
    healthy_threshold   = 2
    unhealthy_threshold = 3
  }
}

Step 3: Adjust timeouts for slow-starting applications

If your app takes time to boot (JVM, large dependency injection):

resource "aws_lb_target_group" "app" {
  # ...

  health_check {
    path                = "/health"
    interval            = 30
    timeout             = 15
    healthy_threshold   = 2
    unhealthy_threshold = 5  # Allow more failures during startup
  }
}

For ECS, also configure container health check grace period:

resource "aws_ecs_service" "app" {
  name            = "my-app"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.app.arn
  desired_count   = 2

  health_check_grace_period_seconds = 120  # Give containers time to start

  load_balancer {
    target_group_arn = aws_lb_target_group.app.arn
    container_name   = "app"
    container_port   = 8080
  }
}

Step 4: Prevent 502 during rolling deploys

Set deregistration delay higher than your drain time, and ensure minimum healthy percent keeps old targets alive:

resource "aws_lb_target_group" "app" {
  # ...
  deregistration_delay = 120  # Keep draining connections for 2 minutes
}

resource "aws_ecs_service" "app" {
  # ...
  deployment_minimum_healthy_percent = 100  # Keep old tasks until new ones are healthy
  deployment_maximum_percent         = 200  # Allow double capacity during deploy

  deployment_circuit_breaker {
    enable   = true
    rollback = true
  }
}

Validation

After applying fixes, confirm targets become healthy:

# 1. Check target health - all should show "healthy"
aws elbv2 describe-target-health \
  --target-group-arn arn:aws:elasticloadbalancing:eu-central-1:123456789012:targetgroup/my-app/abc123 \
  --query 'TargetHealthDescriptions[*].TargetHealth.State'
# Expected: ["healthy", "healthy"]

# 2. Verify ALB returns 200, not 502
curl -s -o /dev/null -w "%{http_code}" https://my-app.example.com/health
# Expected: 200

# 3. Confirm no ELB 502 errors in CloudWatch
aws cloudwatch get-metric-statistics \
  --namespace AWS/ApplicationELB \
  --metric-name HTTPCode_ELB_502_Count \
  --dimensions Name=LoadBalancer,Value=app/my-alb/abc123 \
  --start-time "$(date -u -d '5 minutes ago' +%Y-%m-%dT%H:%M:%SZ)" \
  --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --period 60 \
  --statistics Sum
# Expected: 0

# 4. Test a rolling deploy does not cause 502
aws ecs update-service --cluster main --service my-app --force-new-deployment
# Monitor: no 502 spikes during deploy

ALB 502 errors during deploys or scaling events are not a nuisance - they are lost revenue. Every request that hits a 502 is a customer seeing an error page. The fix is not complex, but it requires all four layers (SG rules, health check config, timeouts, deploy strategy) to be correct simultaneously. One misconfigured layer is enough to take down the entire service.

Need zero-downtime deploys on AWS ECS?

Book a free 30-minute call. We design ALB + ECS architectures with proper health checks, blue-green deploys, and circuit breakers.

Book a call