Terraform - Detecting State Drift and Safely Force-Unlocking a Locked State
Diagnosing and fixing Terraform state drift: reconciling manual changes, terraform import, refresh-only plan. Safe force-unlock of a state locked in DynamoDB.
terraform plan shows unexpected changes: resources to delete or modify that nobody planned. Alternatively: terraform plan or apply returns an error Error acquiring the state lock - the state is locked by an interrupted CI/CD process or a crashed terminal. Both problems block further IaC operations on the infrastructure.
This runbook covers diagnosing state drift and safely unlocking Terraform state. For a comparison of IaC tools (Terraform vs Pulumi), see the article Terraform vs Pulumi - which IaC tool fits your team. A basic runbook for simple state unlocking is Terraform State Lock Stuck. For help with infrastructure as code management - book a consultation.
Symptom
Scenario A: State Drift
terraform plan shows changes that do not correspond to any modification in the code:
# Typical drift symptoms:
# 1. Plan wants to destroy resources that exist in the cloud
# "Plan: 0 to add, 0 to change, 3 to destroy."
# (but nobody removed these resources from code!)
# 2. Plan wants to change attributes despite no changes in .tf
# ~ resource "aws_security_group" "main" {
# ~ ingress = [...] -> [...] # "forces replacement"
# }
# 3. Plan shows "resource has been changed outside of Terraform"
# Note: Objects have changed outside of Terraform
# Check what the plan proposes
terraform plan -detailed-exitcode
# Exit code 0 = no changes (OK)
# Exit code 1 = error
# Exit code 2 = changes present (DRIFT!)
# Check specific differences
terraform plan -json | jq '.resource_changes[] | select(.change.actions != ["no-op"]) | {address, actions: .change.actions}'
# Compare state with reality
terraform show -json | jq '.values.root_module.resources[] | {type, name, values: (.values | keys)}'
Scenario B: State Lock Stuck
# Typical error message:
# Error: Error acquiring the state lock
#
# Error message: ConditionalCheckFailedException: The conditional request failed
# Lock Info:
# ID: a1b2c3d4-e5f6-7890-abcd-ef1234567890
# Path: my-project/terraform.tfstate
# Operation: OperationTypePlan
# Who: user@hostname
# Version: 1.9.0
# Created: 2026-08-07 14:23:45.123456 +0000 UTC
# Check who holds the lock in DynamoDB
aws dynamodb get-item \
--table-name terraform-locks \
--key '{"LockID": {"S": "my-bucket/my-project/terraform.tfstate"}}' \
--query 'Item.Info.S' --output text | jq .
# Shows: Who, Operation, Created timestamp
# Check if the CI/CD process is still running
# If Created > 30 min ago and pipeline is not active → lock is orphaned
Root Cause
State Drift - why state does not match infrastructure:
1. Manual changes in console/CLI (ClickOps):
Someone changed a security group, added a tag, modified a parameter through the AWS/Azure/GCP console. Terraform state does not know about these changes - it shows the “old” picture of the infrastructure.
2. Another IaC process manages the same resources:
A CloudFormation stack, Ansible playbook, Pulumi program, or another Terraform workspace modifies resources that are also in this state. Shared resource conflicts.
3. Auto-scaling / AWS default behaviours:
AWS automatically adds tags, modifies security groups (e.g. VPC default SG), changes attributes of managed resources. Terraform sees these changes as drift.
4. Corrupted or unsynchronised state:
An interrupted terraform apply saved partial state. Resources were created, but state was not updated. Or S3 versioning restored an older version of the state.
State Lock - why the lock is “stuck”:
1. CI/CD pipeline interrupted during operation:
GitHub Actions job cancelled, Jenkins node crashed, terminal closed during terraform apply. The process did not have time to release the lock in DynamoDB.
2. Network timeout to DynamoDB:
The Terraform operation completed, but the unlock request to DynamoDB did not arrive (network timeout, IAM temporary credentials expired).
3. Long apply without timeout:
terraform apply on large infrastructure (>100 resources) runs for hours. The lock is active, but another developer tries to plan on the same state.
Solution
Drift - reconcile state with reality:
A) Refresh state (update state without changing infrastructure):
# 1. First make a state backup!
terraform state pull > terraform.tfstate.backup.$(date +%Y%m%d_%H%M%S)
# 2. Refresh - synchronise state with current infrastructure
terraform plan -refresh-only
# Check what will change in state (not in infrastructure!)
# Shows: "Terraform will update the state to reflect changes made outside"
# 3. If changes look OK - apply the refresh
terraform apply -refresh-only -auto-approve
# 4. Now a normal plan should be clean
terraform plan
# Expected: "No changes. Your infrastructure matches the configuration."
B) Import resources that exist but are not in state:
# Scenario: resource exists in cloud, but Terraform does not know about it
# Plan shows: "resource will be created" - even though it already exists
# 1. Check if resource exists
aws ec2 describe-security-groups --group-ids sg-0123456789abcdef
# 2. Import into state (Terraform 1.5+ - import block in code)
# In your .tf file add:
# imports.tf - declarative import (Terraform 1.5+)
import {
to = aws_security_group.main
id = "sg-0123456789abcdef"
}
# Or for resources with composite ID:
import {
to = aws_route_table_association.private
id = "subnet-abc123/rtbassoc-xyz789"
}
# 3. Run plan - Terraform will compare config with the imported resource
terraform plan -generate-config-out=generated.tf
# generated.tf contains configuration matching the imported resource
# 4. Move generated configuration to the proper .tf file
# Adjust to your style/conventions
# 5. Run apply (import + alignment)
terraform apply
# 6. Remove import {} blocks after successful import
C) State rm - remove resource from state (not from infrastructure):
# Scenario: Terraform wants to delete a resource that should exist
# but should not be managed by THIS workspace
# 1. Check what is in state
terraform state list | grep "resource_to_remove"
# 2. Remove from state (resource stays in infrastructure!)
terraform state rm 'aws_security_group.legacy'
# "Successfully removed 1 resource instance(s)."
# 3. Verify - plan should not try to delete it
terraform plan
D) Moved blocks - refactoring without drift:
# When you renamed/moved resources in code and plan wants to destroy+create:
moved {
from = aws_instance.web_server
to = aws_instance.app_server
}
moved {
from = module.old_name
to = module.new_name
}
State Lock - safe unlocking:
E) Force-unlock (after verification!):
# IMPORTANT: NEVER force-unlock if another process is still running!
# Always check first:
# 1. Check who holds the lock
aws dynamodb get-item \
--table-name terraform-locks \
--key '{"LockID": {"S": "my-bucket/my-project/terraform.tfstate"}}' \
--query 'Item.Info.S' --output text | jq .
# 2. Verify the process is still alive
# - Check CI/CD pipeline (is the job active?)
# - Check the Who field - does that person still have a terminal open?
# - Check Created timestamp - if > 1h ago, likely orphaned
# 3. If the lock is orphaned - unlock it
LOCK_ID="a1b2c3d4-e5f6-7890-abcd-ef1234567890" # from the error message
terraform force-unlock $LOCK_ID
# 4. If force-unlock does not work (state corrupted) - delete manually from DynamoDB
aws dynamodb delete-item \
--table-name terraform-locks \
--key '{"LockID": {"S": "my-bucket/my-project/terraform.tfstate"}}'
# 5. IMMEDIATELY after unlock - run plan to verify state
terraform plan
F) Preventing future stuck locks in CI/CD:
# GitHub Actions - timeout + cleanup
name: Terraform Apply
on: [push]
jobs:
apply:
runs-on: ubuntu-latest
timeout-minutes: 60 # Hard limit on the entire job
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
- name: Terraform Apply
id: apply
timeout-minutes: 45 # Timeout on apply (leaves time for unlock)
run: terraform apply -auto-approve
continue-on-error: true
# Cleanup lock if apply failed/timed out
- name: Force Unlock on Failure
if: failure() || cancelled()
run: |
LOCK_ID=$(terraform force-unlock -force 2>&1 | grep -oP '[a-f0-9-]{36}' | head -1)
if [ -n "$LOCK_ID" ]; then
terraform force-unlock -force "$LOCK_ID"
fi
# backend.tf - configuration with lock timeout
terraform {
backend "s3" {
bucket = "my-terraform-state"
key = "my-project/terraform.tfstate"
region = "eu-west-1"
dynamodb_table = "terraform-locks"
encrypt = true
# Timeout on lock acquisition (default: 0 = wait forever)
# Set shorter than CI/CD job timeout
skip_metadata_api_check = false
}
}
Validation
# 1. After fixing drift - plan must be clean
terraform plan -detailed-exitcode
# Expected: exit code 0 (no changes)
# 2. After force-unlock - plan must execute without lock errors
terraform plan
# Expected: no "Error acquiring the state lock"
# 3. Verify state is consistent
terraform state list | wc -l
# Compare with expected number of resources
# 4. Check there are no "tainted" resources
terraform show -json | jq '[.values.root_module.resources[] | select(.tainted == true)] | length'
# Expected: 0
# 5. Drift detection - configure automatic detection
terraform plan -refresh-only -detailed-exitcode
# In CI/CD: run scheduled job every 24h
# Exit code 2 = drift detected → alert to the team
# 6. Check DynamoDB - lock should be free
aws dynamodb get-item \
--table-name terraform-locks \
--key '{"LockID": {"S": "my-bucket/my-project/terraform.tfstate"}}' \
--query 'Item'
# Expected: no Item (lock released) or null
Terraform state causing problems?
Schedule a free 30-minute call. We will help fix drift, configure automatic detection, and secure your CI/CD pipeline against stuck locks.