Runbooks

Step-by-step troubleshooting guides with ready-to-use commands.

• Fix Bedrock ServiceUnavailableException 503 | High Demand and Capacity

  Fix Bedrock InvokeModel failures with ServiceUnavailableException (HTTP 503) caused by transient high-demand capacity strain: tuned backoff, circuit breakers, cross-region inference, and provisioned throughput.

  AWS Bedrock LLM GenAI
• Patch GitLab CVE-2026-110264 (CVSS 10.0 RCE): step by step

  Patch self-managed GitLab against CVE-2026-110264: inventory every instance, check the served version per install type, upgrade to the fixed release for your minor line (handling the DB migrations on 19.4.3 and 19.3.6), and run GitLab's detection rules to check for exploitation.

  GitLab security CVE patching CI/CD
• Fix Vertex AI endpoint deployment stuck or failed | DeployModel

  Resolve a Vertex AI DeployModel operation stuck in progress or failing: diagnose the model server container contract, health and predict route readiness, model artifact paths, and the custom-model-serving quota that silently blocks deployment.

  GCP Vertex AI machine learning MLOps
• Fix Vertex AI 403 aiplatform.endpoints.predict permission denied

  Resolve a 403 PermissionDenied on aiplatform.endpoints.predict: identify the calling identity, grant the Vertex AI User role on the right project, and rule out a wrong resource path or project before chasing IAM.

  GCP Vertex AI IAM MLOps
• Fix Vertex AI 429 RESOURCE_EXHAUSTED on online prediction

  Resolve 429 RESOURCE_EXHAUSTED errors on Vertex AI online prediction: tell shared on-demand throttling apart from a replica quota ceiling, raise the right limit, and add exponential backoff so transient throttling does not reach users.

  GCP Vertex AI machine learning MLOps
• Fix Vertex AI deploy quota exceeded | GPU unavailable in region

  Resolve a Vertex AI DeployModel failure caused by an exceeded CustomModelServing accelerator quota or a GPU/TPU type unavailable in the target region: locate the exact quota, request an increase, or move to an available region, accelerator, or reservation.

  GCP Vertex AI machine learning MLOps
• Fix AWS Transform .NET conversion stops | unsupported code

  Resolve AWS Transform for .NET stopping on a dependency it cannot port, or completing with TODO comments, stubbed code, and a Next Steps file: read what it flagged, understand the known unsupported cases, and finish the parts the agent leaves to you.

  AWS AWS Transform .NET migration Azure
• Fix AgentCore Browser tool blocked by CAPTCHA

  Fix an AgentCore Browser tool agent blocked by CAPTCHA: confirm you are permitted to automate the target, stabilise the browser session, and handle CAPTCHA as a typed state with human handoff.

  AWS Bedrock AgentCore Browser tool AI agents
• AWS Agent Registry: migrate off the bedrock-agentcore namespace

  Fix AWS Agent Registry failures after the service moved from the bedrock-agentcore namespace to agent-registry: update endpoints, IAM policies, SDK clients, and CLI scripts, and migrate registry data.

  AWS Bedrock AgentCore AWS Agent Registry AI agents
• Fix Bedrock AgentCore: Cost Explorer and CUR access denied or empty

  Fix a FinOps agent on Bedrock AgentCore that returns AccessDenied or empty results when reading cost data, by diagnosing the Cost Explorer IAM action, the CUR plus Athena query path, and management-versus-member account scope.

  AWS Bedrock AgentCore FinOps AI agents
• Fix AWS FinOps agent: AccessDenied on a cost-optimisation action

  Fix AccessDenied when an AWS FinOps agent tries to act on a cost finding (stop an instance, delete a volume, buy a commitment) by separating read-first observation from a narrowly scoped, human-gated write path.

  AWS FinOps AI agents IAM
• Fix Bedrock AccessDenied: model marked legacy or not active

  Fix AccessDenied on Bedrock when a model is marked legacy or not active: the 30-day inactivity rule, deprecated version pinning, and switching to an active model or inference profile.

  AWS Bedrock GenAI AI agents
• AWS SCT: Oracle to PostgreSQL conversion errors - how to fix

  Fix AWS SCT conversion errors when migrating Oracle to PostgreSQL: diagnose action items, unconverted PL/SQL packages, sequences and triggers, the missing extension pack, and types with no equivalent.

  AWS AWS SCT Oracle PostgreSQL database migration
• Datadog Agent not reporting metrics after migration - how to fix

  Fix missing metrics in Datadog Agent after migrating from Nagios/Zabbix/CloudWatch: diagnose agent status, API key and region (site), intake connectivity, integration enablement, and inconsistent tags.

  Datadog observability monitoring migration
• SAP HANA on EC2 won't start after migration - how to fix

  Fix SAP HANA startup on EC2 after migrating to AWS: diagnose hdbnsutil services, IOPS/throughput throttling on the data and log volumes, choosing a certified instance class, and memory sizing.

  AWS SAP SAP HANA EC2 migration
• AWS DMS: data mismatch after cutover - how to fix validation

  Fix DMS data validation mismatches after full load + CDC: diagnose ValidationState and ValidationFailedRecords, type mismatches, truncated LOBs, collation, and row ordering before cutover.

  AWS AWS DMS CDC database migration data validation
• Fix Bedrock AgentCore Identity: AccessDenied on behalf of user

  Fix AgentCore Identity AccessDenied errors when an agent acts on behalf of a user by diagnosing identity propagation, the OAuth 2LO/3LO token vault, workload identity, and least-privilege scope mismatches.

  AWS Bedrock AgentCore AI agents GenAI
• AWS MGN: replication stalled or rising lag - how to fix

  Fix stalled AWS MGN replication: diagnose agent and source-server state, network bandwidth and throughput, test-instance boot failures, and replication-server sizing.

  AWS AWS MGN VMware migration rehosting
• Fix Bedrock AgentCore Memory: stale or empty retrieval

  Fix AgentCore Memory retrieval returning empty or stale context by diagnosing the async long-term consolidation delay, short-term vs long-term memory, actor/session scoping, and namespace mismatches.

  AWS Bedrock AgentCore AI agents GenAI
• Fix Bedrock AgentCore Runtime invocation timeout | 504 and 60s init

  Fix AgentCore Runtime invocation timeouts by diagnosing cold-start init limits, the port 8080 and /invocations container contract, ARM64 builds, and 15-minute session timeouts.

  AWS Bedrock AgentCore AI agents GenAI
• Fix AgentCore Gateway MCP 401 unauthorized | OAuth token and audience

  Fix AgentCore Gateway 401/403 on MCP tool calls by checking the inbound JWT audience claim, the token itself, and the outbound OAuth2 credential provider.

  AWS Bedrock AgentCore MCP AI agents
• AWS DMS to Redshift/S3: target load errors - how to fix

  Fix apply errors in a continuous AWS DMS task to Redshift or S3: diagnose stl_load_errors, VARCHAR overflow, IAM/S3 permissions, and instance sizing for an indefinitely-running CDC stream.

  AWS AWS DMS Redshift S3 CDC data warehouse
• AWS DMS: rising CDC latency and task failure - how to fix

  Fix rising CDC latency and AWS DMS task failures: diagnose CDCLatencySource vs CDCLatencyTarget, tables without a primary key, LOB truncation, and replication instance sizing.

  AWS AWS DMS CDC database migration database
• AWS RDS Proxy configuration and limitations: what it solves and when it fails

  Configure AWS RDS Proxy correctly: MaxConnectionsPercent, idle connection management, pinning avoidance, and the scenarios where RDS Proxy hurts more than it helps.

  AWS RDS RDS Proxy PostgreSQL MySQL database connection pooling
• CloudWatch Alarm Not Triggering: INSUFFICIENT_DATA and Missing Notifications

  Fix a CloudWatch Alarm that won't trigger: INSUFFICIENT_DATA due to missing metrics, wrong evaluation periods, missing SNS permissions, and treat-missing-data configuration.

  AWS CloudWatch monitoring alerts
• Prometheus OOMKilled: High Cardinality and metric_relabel_configs

  Fix Prometheus OOMKilled: identify high cardinality metrics, configure metric_relabel_configs to drop unnecessary series, set series limits, and right-size memory allocation.

  Prometheus observability monitoring Kubernetes OOM
• Cloud Sprawl Audit: Identify and Reduce Uncontrolled Cloud Growth

  Identify and reduce cloud sprawl: audit orphaned resources, unused accounts, untagged infrastructure, and duplicate services. CLI commands for AWS, GCP, and Azure with cost recovery estimates.

  AWS GCP Azure FinOps cloud sprawl cost optimization multi-cloud governance
• Datadog Agent High CPU and Memory: Diagnosis and Fix

  Diagnose and fix high CPU/memory usage by the Datadog Agent: disable unnecessary integrations, limit custom metrics, configure log collection, and switch to Cluster Agent.

  Datadog observability monitoring performance
• GKE Binary Authorization Deployment Denied | Fix Missing Attestation

  Resolve GKE Binary Authorization deployment denials caused by missing attestations, wrong attestor key version, image digest mismatch, or missing break-glass annotation for emergency deploys.

  GCP GKE Binary Authorization security PCI DSS
• GCP VPC Service Controls Request Blocked | Fix Perimeter Denials

  Diagnose and fix API requests blocked by GCP VPC Service Controls perimeters: verify access levels, configure ingress/egress rules, analyze VPC-SC audit logs, and maintain PCI DSS compliance.

  GCP VPC Service Controls security PCI DSS access denied
• AWS Savings Plans Utilization Drop Below 80% | Fix Wasted Commitment

  Diagnose and fix AWS Savings Plans utilization drop below 80%: identify causes (scale-down, workload migration, wrong region, seasonality) and strategies to recover commitment value.

  AWS FinOps Savings Plans cost optimization
• vLLM - Throughput Degradation After Hours of Operation (KV Cache Fragmentation)

  Fixing vLLM throughput degradation after hours of operation: diagnosing KV cache fragmentation, enabling prefix caching, tuning gpu-memory-utilization, and scheduled restart.

  vLLM GPU LLM Kubernetes performance
• GCP Committed Use Discount Not Applying to Instances

  Diagnosing a non-applying GCP CUD: verifying machine family, region, commitment type (compute vs memory-optimised), checking billing export, and fixing the issue.

  GCP FinOps CUD cost optimization billing
• BigQuery slots - queries queued PENDING and Resources exceeded

  Fix BigQuery queries stuck in queue: verify capacity commitment, assignment hierarchy, concurrency and slot autoscaling.

  GCP BigQuery FinOps slot reservations
• Azure OpenAI - 429 Too Many Requests (TPM/RPM Quota Exceeded)

  Resolving 429 errors in Azure OpenAI: identifying the bottleneck (TPM vs RPM), implementing retry logic, increasing quota, or switching to PTU.

  Azure Azure OpenAI rate limiting FinOps LLM
• Azure Conditional Access Blocks Service Principal - Post Zero Trust Diagnosis

  Fixing Service Principal blocked by Conditional Access: Sign-in Logs analysis, excluding workload identities from policies, configuring Workload Identity Premium.

  Azure Entra ID Conditional Access Zero Trust IAM
• Bedrock Knowledge Base - S3 sync fails with Access Denied

  Fix Access Denied in Bedrock Knowledge Base sync: verify IAM role trust, KMS key policy, S3 bucket policy and VPC endpoint policy.

  AWS Bedrock Knowledge Base S3 IAM
• Azure DMS - connectivity failure during SQL Server migration to Azure SQL

  Fix Azure Database Migration Service connectivity errors: verify NSG rules, firewall configuration, Self-Hosted Integration Runtime and source network setup.

  Azure DMS SQL Server migration
• Azure App Service VNet Integration - DNS fails to resolve Private Endpoints

  Fix DNS resolution in App Service with VNet Integration: configure Private DNS Zones, VNet links and WEBSITE_DNS_SERVER.

  Azure App Service VNet Integration DNS Private Endpoint
• Terraform Plan Drift: Someone Changed Something in the Console

  Fix Terraform drift after manual console changes: identify modified resources, import or reset to desired state, and implement drift detection with policies.

  Terraform IaC drift AWS DevOps
• ECS Task OOMKilled: Container Memory Limit Exceeded

  Fix ECS Task OOMKilled: identify the cause of memory limit breaches, right-size task definitions, configure memory reservations, and understand hard vs soft limits.

  AWS ECS Fargate containers OOM memory
• Fix vLLM GPU OOM CrashLoopBackOff on EKS | CUDA Out of Memory

  Fix vLLM inference pods crashing with CUDA OutOfMemoryError on EKS. Diagnose KV cache overflow, tune memory parameters, or switch to quantised model weights.

  Kubernetes EKS GPU vLLM LLM GenAI
• Fix OpenSearch Serverless Latency Spikes | OCU Starvation in RAG

  Fix OpenSearch Serverless vector search latency degradation caused by OCU starvation. Diagnose saturation, tune k-NN parameters, and scale OCUs.

  AWS OpenSearch RAG GenAI vector database
• Fix Bedrock InvokeModel ThrottlingException 429 | Quota Increase and Retry

  Fix Bedrock InvokeModel failures with ThrottlingException or ModelStreamErrorException by diagnosing quota limits, implementing backoff, and requesting increases.

  AWS Bedrock LLM GenAI
• Terraform state drift and safe force-unlock

  Diagnosing and fixing Terraform state drift: reconciling manual changes, terraform import, refresh-only plan. Safe force-unlock of a state locked in DynamoDB.

  Terraform IaC state drift DynamoDB
• LiteLLM Bedrock: unsupported model id schema (OAIBackend) fix

  Fix the LiteLLM 'unsupported model id schema' / OAIBackend error when proxying to AWS Bedrock: the model string lacks a recognised provider prefix or the wrong route is used for inference-profile and application-inference-profile ARNs.

  AWS Bedrock LiteLLM GenAI proxy
• Fix GKE Node Pool Stuck Provisioning | Quota and IP Exhaustion

  Resolve GKE node pool stuck in PROVISIONING state caused by quota exhaustion, insufficient pod IP ranges, or machine type unavailability in the target zone.

  GCP GKE Kubernetes autoscaling
• Cloudflare Zero Trust Tunnel for Kubernetes | No Public IPs

  Deploy Cloudflare Tunnel (cloudflared) as a Kubernetes Deployment to expose internal ClusterIP services without opening inbound ports. Works identically on EKS, GKE, AKS, and on-prem.

  Cloudflare Zero Trust Kubernetes networking
• Fix AWS ALB 502 Bad Gateway | Target Group Health Check Failures

  Diagnose and fix ALB 502 Bad Gateway errors caused by target group health check failures - security groups, health check paths, timeouts, and deploy draining.

  AWS ALB ECS health-checks
• AKS Workload Identity Federation Setup | Migrate from Pod Identity

  Complete migration guide from deprecated aad-pod-identity to AKS Workload Identity Federation using OIDC issuer, managed identities, and federated credentials.

  Azure AKS Kubernetes identity
• Terraform state lock stuck: how to force-unlock

  Fix a stuck Terraform state lock when terraform plan or apply fails with Error acquiring the state lock after a crashed or timed-out operation.

  Terraform IaC DynamoDB
• ECS Fargate task stopped unexpectedly: how to fix

  Fix ECS Fargate tasks that keep stopping with Essential container exited, OutOfMemoryError, or CannotPullContainerError by identifying the root cause in stopped task metadata and CloudWatch logs.

  AWS ECS Fargate containers
• AWS RDS connection limit exceeded: how to fix

  Fix RDS PostgreSQL connection limit errors by identifying idle connections, terminating leaked sessions, and implementing connection pooling with RDS Proxy or PgBouncer.

  AWS RDS PostgreSQL database
• AWS Well-Architected Review - Fix the 10 Most Common High-Risk Findings

  Fix the 10 most common high-risk items (HRI) from AWS Well-Architected Reviews. Step by step: IAM root access, encryption, backups, Multi-AZ, monitoring, logging - with CLI commands and Terraform.

  AWS Well-Architected WAFR security reliability remediation IAM KMS
• Lambda cold start + API Gateway 504 Timeout

  Fixing 504 Timeout on API Gateway caused by Lambda cold start: diagnosing VPC ENI, optimising package size, provisioned concurrency, SnapStart for Java.

  AWS Lambda API Gateway serverless cold start timeout
• Azure equivalent of AWS ECS and Fargate: full comparison table

  Quick reference mapping AWS container services (ECS, Fargate, ECR, App Runner) to their Azure equivalents (Container Apps, ACI, ACR, Web App for Containers) with guidance on when each applies.

  AWS Azure containers ECS migration
• Self-host Supabase with Docker Compose without the CLI

  Run a production Supabase stack using plain Docker Compose without the CLI. Full setup with custom registries, environment variables, and inter-service networking.

  Supabase Docker self-hosting AWS
• AWS VPN Parallel Connections - Asymmetric Routing and Traffic Hairpinning

  Fix asymmetric routing and traffic hairpinning when running multiple AWS Site-to-Site VPN connections in parallel to GCP Cloud VPN.

  AWS VPN BGP Transit Gateway routing networking GCP
• Fix Supabase CLI 403 from ECR | Docker Hub Override

  Fix supabase start failing with 403 Forbidden from public.ecr.aws by overriding the image registry to Docker Hub, GHCR, or your own private registry.

  Supabase Docker AWS
• Bedrock ThrottlingException - too many tokens per day quota

  Fixing ThrottlingException 'too many tokens per day' in AWS Bedrock: analysing daily limits, retry with exponential backoff, requesting quota increase, fallback to alternative models.

  AWS Bedrock throttling GenAI quota
• ALB + Lambda 502 Bad Gateway - payload and timeout

  Fixing 502 Bad Gateway on ALB with Lambda target: exceeding 1MB payload limit, ALB 29s timeout, malformed JSON response, Lambda cold start + ALB idle timeout.

  AWS ALB Lambda serverless 502
• AWS VPN Throughput Degraded - MTU Fragmentation on IPsec Tunnel

  Fix AWS Site-to-Site VPN throughput degradation caused by MTU fragmentation on IPsec tunnels connecting to GCP Cloud VPN.

  AWS VPN MTU IPsec throughput networking GCP
• Migration Fargate → Azure Container Instances - Networking and Service Discovery Issues

  Fixing networking issues when migrating from AWS Fargate to Azure Container Instances: DNS resolution, service discovery, NSG vs Security Groups, private endpoints.

  AWS Azure Fargate Container Instances networking migracja
• GCP Cross-Cloud Interconnect BGP Session Flapping

  Fix GCP Cross-Cloud Interconnect BGP session flapping caused by route advertisement limits, ASN conflicts, or hold timer mismatches.

  GCP Cross-Cloud Interconnect BGP Cloud Router networking AWS
• AWS VPN Tunnel DOWN - IKE Phase 1/2 Failure with GCP Cloud VPN

  Diagnose and fix AWS Site-to-Site VPN tunnel DOWN caused by IKE Phase 1/2 negotiation failure when connecting to GCP Cloud VPN.

  AWS VPN GCP Cloud VPN IPsec IKE networking
• Atman cross-connect BGP session down: diagnose and restore

  Restore a BGP session over an Atman cross-connect when peering drops due to physical layer faults, MTU issues, or hold timer expiry.

  Atman BGP networking Polish-DC EU-hosting
• cPanel EasyApache4 PHP module conflict: how to resolve

  Resolve PHP module conflicts in cPanel EasyApache4 when package dependencies clash during Apache rebuilds, causing build failures or broken PHP installations.

  cPanel PHP Apache EU-hosting legacy-hosting
• Hetzner rescue mode: rebuild mdadm RAID after disk failure

  Rebuild an mdadm software RAID array on a Hetzner dedicated server after disk replacement using rescue mode, partitioning, and array reassembly.

  Hetzner dedicated-servers bare-metal EU-hosting RAID
• Proxmox Ceph OSD marked out: slow recovery and how to fix it

  Speed up Ceph recovery in Proxmox when an OSD is marked out and PG backfill crawls despite available disk bandwidth.

  Proxmox Ceph private-cloud storage
• ArgoCD another operation is already in progress: how to fix

  Fix the operation deadlock in ArgoCD when a stuck operation blocks all sync attempts with another operation is already in progress.

  ArgoCD Kubernetes GitOps
• ArgoCD HPA OutOfSync loop: how to fix

  Fix the infinite OutOfSync loop in ArgoCD caused by the field ownership conflict between HPA and declarative spec.replicas management.

  ArgoCD Kubernetes GitOps
• ArgoCD etcdserver: request is too large. How to fix

  Diagnose and fix the etcdserver: request is too large error in ArgoCD when the Application CRD exceeds the 1 MB etcd object size limit.

  ArgoCD Kubernetes etcd