AWS Bedrock ServiceUnavailableException (503): handling high-demand capacity errors without treating them as a quota problem

This runbook covers the 503 capacity path. If your errors say ThrottlingException with HTTP 429 instead, you have hit an account quota, not a capacity wall. See the companion runbook Bedrock InvokeModel ThrottlingException 429. To design a resilient model layer for production, book a consultation.

Symptoms

Your application logs show a 503, not a 429:

botocore.errorfactory.ServiceUnavailableException: An error occurred (ServiceUnavailableException) when calling the InvokeModel operation: Service temporarily unavailable, please try again.

Or on the streaming API:

botocore.exceptions.ClientError: An error occurred (ServiceUnavailableException) when calling the InvokeModelWithResponseStream operation: Service temporarily unavailable, please try again.

Observable impact:

• CloudWatch AWS/Bedrock > InvocationServerErrors (5xx) rises, while InvocationThrottles (429) stays flat
• Errors cluster in bursts that line up with traffic peaks rather than a steady per-minute ceiling
• A naive immediate-retry loop clears some requests but keeps a visible failure rate under load
• The same prompt succeeds seconds later with no quota change

Cause

A 503 ServiceUnavailableException means Bedrock is temporarily unable to serve the request. The signal is about the model’s capacity at that moment, not your account limits:

• On-demand capacity spikes - on-demand models share a regional capacity pool. During broad demand surges the service sheds load with 503s rather than queueing indefinitely.
• Single-region concentration - pinning every request to one Region and model ID means all your traffic competes for one capacity pool with no room to spill over.
• Correlated client bursts - fan-out jobs, scheduled batch runs, or a retry storm concentrate requests into narrow windows that coincide with service-side strain.
• Treating 503 like 429 - waiting out a 60-second quota cycle is the wrong remedy. A 503 usually clears faster than that, so quota-style waits waste latency budget.

The practical distinction:

Fix

Step 1: Confirm it is 503 and not 429

Separate the two server-side signals in CloudWatch before changing anything:

# 5xx (includes 503 ServiceUnavailableException)
aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name InvocationServerErrors \
  --dimensions Name=ModelId,Value=eu.anthropic.claude-haiku-4-5-20251001-v1:0 \
  --start-time $(date -u -v-1H +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 60 --statistics Sum --region eu-central-1

# 429 throttles, for comparison
aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name InvocationThrottles \
  --dimensions Name=ModelId,Value=eu.anthropic.claude-haiku-4-5-20251001-v1:0 \
  --start-time $(date -u -v-1H +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 60 --statistics Sum --region eu-central-1

If InvocationServerErrors is elevated and InvocationThrottles is near zero, you are in the 503 capacity path. Do not file a quota increase - it will not help.

Step 2: Retry with backoff tuned for slower recovery

Keep the exponential-backoff-with-jitter shape, but let boto3’s adaptive retry mode and a larger connection pool do the work rather than a hand-rolled loop:

import boto3
from botocore.config import Config

config = Config(
    region_name="eu-central-1",
    retries={"mode": "adaptive", "max_attempts": 5},
    max_pool_connections=50,  # default 10 is too small under concurrency
)

bedrock = boto3.client("bedrock-runtime", config=config)

Adaptive mode adds client-side rate limiting and backoff for both 429 and 503. The default pool size of 10 starves high-concurrency callers and manufactures its own failures.

Step 3: Add a circuit breaker for mission-critical paths

Under sustained 503s, retrying harder makes things worse. A circuit breaker fails fast during the strain window and tests recovery automatically. Use a maintained library rather than a custom implementation:

import pybreaker
from botocore.exceptions import ClientError

# Open after 5 failures, probe again after 30s
breaker = pybreaker.CircuitBreaker(fail_max=5, reset_timeout=30)

@breaker
def invoke(model_id, body):
    return bedrock.invoke_model(modelId=model_id, body=body)

try:
    resp = invoke(model_id, body)
except pybreaker.CircuitBreakerError:
    # circuit open: serve a cached / fallback response instead of piling on
    resp = fallback_response()
except ClientError:
    raise

pybreaker and tenacity are both battle-tested. A custom breaker is only worth it for unusual failure-detection logic.

Step 4: Widen the capacity pool with cross-region inference

Cross-region inference (CRIS) routes through an inference profile that spans multiple Regions, so requests can land wherever there is capacity instead of fighting over one pool:

# Geographic (EU) inference profile instead of a single-region model ID
resp = bedrock.invoke_model(
    modelId="eu.anthropic.claude-haiku-4-5-20251001-v1:0",
    body=body,
)

• Geographic profiles (US, EU, APAC) keep data inside one geography while drawing on multi-Region capacity. Confirm data-residency rules (GDPR, HIPAA, FedRAMP) allow every Region in the geography before enabling - routing to an unapproved Region, even briefly, can be a compliance breach.
• Global profiles spread across every Region the model runs in, chosen by real-time capacity.

CRIS is a capacity-distribution mechanism, not a disaster-recovery or automatic-failover system. Treat it as throughput smoothing, and pair it with the retry and breaker patterns above.

Step 5: Reserve dedicated capacity for workloads that cannot absorb a 503

For predictable, high-volume, latency-sensitive paths, take them off the shared on-demand pool entirely:

• Provisioned Throughput - reserve dedicated model units so critical traffic is insulated from other tenants’ demand spikes.
• Service tiers - route your most critical workloads through a tier that offers a more predictable capacity envelope.
• Separate AWS accounts - independent capacity and quota footprints per application, which also eliminates cross-team contention and cleans up cost attribution.

Validation

Confirm the 5xx rate has dropped after the change:

aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name InvocationServerErrors \
  --dimensions Name=ModelId,Value=eu.anthropic.claude-haiku-4-5-20251001-v1:0 \
  --start-time $(date -u -v-30M +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 60 --statistics Sum --region eu-central-1

Expected: Sum trending to 0.0, with any residual 503s cleared by the retry layer instead of surfacing to users.

Track success rate against an SLO rather than a raw error count, since 503s are bursty. Alarm when success rate drops below your target - for example under 95% over a 10-minute window:

aws cloudwatch put-metric-alarm \
  --alarm-name bedrock-503-success-rate \
  --alarm-description "Bedrock success rate below SLO (503 capacity strain)" \
  --metrics '[
    {"Id":"errors","MetricStat":{"Metric":{"Namespace":"AWS/Bedrock","MetricName":"InvocationServerErrors"},"Period":600,"Stat":"Sum"},"ReturnData":false},
    {"Id":"calls","MetricStat":{"Metric":{"Namespace":"AWS/Bedrock","MetricName":"Invocations"},"Period":600,"Stat":"Sum"},"ReturnData":false},
    {"Id":"success_rate","Expression":"(calls-errors)/calls*100","Label":"SuccessRate"}
  ]' \
  --threshold 95 --comparison-operator LessThanThreshold \
  --evaluation-periods 1 \
  --alarm-actions arn:aws:sns:eu-central-1:ACCOUNT:ops-alerts \
  --region eu-central-1

Separate 429 and 503 into different widgets and alarms. Collapsing them hides whether a spike is a quota wall (file an increase) or capacity strain (fail over and reserve capacity).

Related

• Bedrock InvokeModel ThrottlingException 429 - the quota path, for when you see 429 instead of 503
• Running LLM Inference on AWS: Bedrock vs SageMaker vs Self-Hosted on EKS - architecture comparison and when dedicated capacity beats on-demand
• AWS Bedrock cross-region inference documentation