AWS Bedrock ThrottlingException on InvokeModel: diagnosing quota limits and fixing 429 errors

Symptoms

Your application logs show Bedrock API failures:

botocore.exceptions.ClientError: An error occurred (ThrottlingException) when calling the InvokeModel operation: Too many requests, please wait before trying again.

Or with streaming:

botocore.exceptions.EventStreamError: An error occurred (ModelStreamErrorException) when calling the InvokeModelWithResponseStream operation: Rate exceeded

Observable impact:

• Inference latency spikes from normal 200-400ms to timeouts
• Error rate in CloudWatch Bedrock > InvocationThrottles metric jumps from 0 to hundreds
• End users see failed responses or long loading spinners
• Application retry storms make the problem worse

Cause

AWS Bedrock applies per-model, per-region quotas on two dimensions:

1. Requests per minute (RPM) - number of API calls per minute regardless of size
2. Tokens per minute (TPM) - total input + output tokens processed per minute

Default quotas vary by model. Example defaults (eu-central-1):

Common triggers:

• Batch processing spike - ingestion pipeline calls Bedrock for embeddings/classification at full parallel throughput
• Missing backoff - application retries immediately on failure, creating a thundering herd
• Shared quota - multiple services (dev, staging, production) share the same account and region
• Long prompts - a few requests with 100K+ token context windows consume the entire TPM budget

Fix

Step 1: Identify which quota you’ve hit

Check CloudWatch metrics in the Bedrock namespace:

# Check throttle count for last hour
aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name InvocationThrottles \
  --dimensions Name=ModelId,Value=anthropic.claude-3-5-haiku-20241022-v1:0 \
  --start-time $(date -u -v-1H +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 60 \
  --statistics Sum \
  --region eu-central-1

Check your current quotas:

# List Bedrock service quotas
aws service-quotas list-service-quotas \
  --service-code bedrock \
  --region eu-central-1 \
  --query 'Quotas[?contains(QuotaName, `InvokeModel`)].{Name:QuotaName, Value:Value}'

Step 2: Implement exponential backoff with jitter

If your application retries immediately, it makes throttling worse. Add proper backoff:

import time
import random
import boto3
from botocore.exceptions import ClientError

bedrock = boto3.client("bedrock-runtime", region_name="eu-central-1")

def invoke_with_backoff(model_id, body, max_retries=5):
    """Call Bedrock with exponential backoff and jitter."""
    for attempt in range(max_retries):
        try:
            return bedrock.invoke_model(modelId=model_id, body=body)
        except ClientError as e:
            if e.response["Error"]["Code"] == "ThrottlingException":
                if attempt == max_retries - 1:
                    raise
                # Exponential backoff: 1s, 2s, 4s, 8s, 16s + jitter
                wait = (2 ** attempt) + random.uniform(0, 1)
                time.sleep(wait)
            else:
                raise

Step 3: Request a quota increase

# Request RPM increase for Claude 3.5 Haiku
aws service-quotas request-service-quota-increase \
  --service-code bedrock \
  --quota-code L-XXXXXXXX \
  --desired-value 300 \
  --region eu-central-1

Find the correct quota-code from:

aws service-quotas list-service-quotas --service-code bedrock --region eu-central-1 \
  --query 'Quotas[?contains(QuotaName, `Haiku`)].{Code:QuotaCode, Name:QuotaName, Value:Value}'

Quota increases typically take 1-3 business days. For urgent needs, open a Support case.

Step 4: Architect for sustained throughput

For production systems that need guaranteed throughput:

• Provisioned Throughput - purchase dedicated model units for consistent RPM/TPM without throttling
• Multi-region routing - distribute requests across eu-central-1 and eu-west-1 (each region has independent quotas)
• Request queuing - use SQS to buffer requests and process at a steady rate below your quota
• Separate accounts - isolate production from dev/staging to prevent quota contention

Validation

After implementing fixes, confirm throttling has stopped:

# Verify zero throttles in last 30 minutes
aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name InvocationThrottles \
  --dimensions Name=ModelId,Value=anthropic.claude-3-5-haiku-20241022-v1:0 \
  --start-time $(date -u -v-30M +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 60 \
  --statistics Sum \
  --region eu-central-1

Expected: Sum: 0.0 for all datapoints.

Set up an alarm to catch future throttling early:

aws cloudwatch put-metric-alarm \
  --alarm-name bedrock-throttling-alert \
  --metric-name InvocationThrottles \
  --namespace AWS/Bedrock \
  --statistic Sum \
  --period 300 \
  --threshold 10 \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 1 \
  --alarm-actions arn:aws:sns:eu-central-1:ACCOUNT:ops-alerts

Related

• Running LLM Inference on AWS: Bedrock vs SageMaker vs Self-Hosted on EKS - architecture comparison and when to move beyond Bedrock
• AWS Bedrock service quotas documentation