Logo
Terraform IaC drift AWS DevOps

Terraform Plan Drift After a Manual Change in the AWS/Azure/GCP Console

Fix Terraform drift after manual console changes: identify modified resources, import or reset to desired state, and implement drift detection with policies.

Jerzy Kopaczewski ·
You run terraform plan and see 47 changes that nobody committed. A security group has an extra rule, an RDS instance has a modified parameter, someone manually added a tag. Your CI/CD pipeline is blocked because the plan isn't clean. This runbook shows how to resolve drift and prevent it from happening again.

If your problem is a stuck state lock (not drift), see the runbook on stuck Terraform state locks. For general state management, see the runbook on drift detection and force-unlock.

Symptoms

terraform plan shows changes that don’t exist in your code:

# Typical output after a manual console change:
terraform plan

# aws_security_group.app:
#   ~ ingress {
#     + cidr_blocks = ["10.0.5.0/24"]  # <-- someone added manually
#       from_port   = 443
#       protocol    = "tcp"
#       to_port     = 443
#   }

# aws_db_instance.main:
#   ~ parameter_group_name = "default.postgres15" -> "custom-pg15" # changed in console

# aws_instance.web:
#   ~ tags = {
#     + "CostCenter" = "marketing"  # tag added manually
#   }

The plan wants to “fix” resources back to the code-defined state, but those changes may have been intentional (hotfix, manual intervention during an incident).

Cause

  1. Manual change in the AWS/Azure/GCP console. Someone on the team modified something through the UI instead of code.
  2. Hotfix during an incident. An engineer opened a port in a SG or changed a DB parameter during an outage and never went back to update the code.
  3. Auto-scaling or automatic AWS changes. Some AWS resources modify themselves (e.g., ASG desired count, Lambda concurrent executions).
  4. Another Terraform pipeline. Two workspaces operating on the same resource.
  5. Manual state manipulation. Someone used terraform state rm or terraform import without corresponding code.

Fix

A) Identify what changed and why

# Detailed plan with full diff
terraform plan -detailed-exitcode 2>&1 | tee plan-output.txt
# Exit code 2 = changes to apply

# For a specific resource - show current state vs code
terraform state show aws_security_group.app

# Check who made the change (AWS CloudTrail)
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=ResourceName,AttributeValue=sg-0123456789abcdef0 \
  --start-time $(date -u -v-7d +%Y-%m-%dT%H:%M:%S) \
  --query 'Events[].{Time:EventTime,User:Username,Event:EventName}'

B) Decision: accept the change or revert it?

For each resource with drift, make a decision:

Situation Action
The change was intentional and should stay Update Terraform code to reflect the change
The change was temporary and should be reverted Run terraform apply to restore the code-defined state
The change affects a resource Terraform shouldn’t manage Add lifecycle { ignore_changes = [...] }
You don’t know what this change is Don’t apply. Investigate first.

C) Accept the change (update the code)

# If the console change was intentional - add it to your code:

resource "aws_security_group" "app" {
  # ... existing rules ...

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["10.0.5.0/24"]  # Added: access from new network
    description = "Added during incident INC-1234, formalized in TF"
  }
}

D) Revert the change (terraform apply)

# WARNING: this will restore the resource to the code-defined state
# Make sure reverting won't cause an incident!

# Plan first, check what will happen:
terraform plan -target=aws_security_group.app

# If you're sure:
terraform apply -target=aws_security_group.app

E) Ignore the change (lifecycle ignore)

For resources that change outside Terraform (e.g., ASG desired count, tags added by AWS):

resource "aws_autoscaling_group" "app" {
  # ...
  desired_capacity = 3

  lifecycle {
    ignore_changes = [
      desired_capacity,  # Changed by autoscaling policy
      target_group_arns, # Managed by another module
    ]
  }
}

resource "aws_instance" "web" {
  # ...
  lifecycle {
    ignore_changes = [
      tags["CostCenter"],  # Tag managed by AWS Organizations
    ]
  }
}

F) Import an unmanaged resource

If someone created a resource in the console that should be managed by Terraform:

# 1. Write the resource block in code (empty or with estimated values)
# 2. Import the existing resource into state
terraform import aws_security_group.new_sg sg-0abcdef1234567890

# 3. Run plan and fill in the missing attributes in HCL
terraform plan
# Output will show what needs to be added or changed in the code

Prevention

Drift detection in CI/CD

# GitHub Actions - daily drift check
name: Terraform Drift Detection
on:
  schedule:
    - cron: '0 6 * * *'  # Daily at 6:00 UTC

jobs:
  drift-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3

      - name: Terraform Init
        run: terraform init

      - name: Terraform Plan (drift detection)
        id: plan
        run: terraform plan -detailed-exitcode -no-color
        continue-on-error: true

      - name: Alert on drift
        if: steps.plan.outcome == 'failure'
        run: |
          curl -X POST "$SLACK_WEBHOOK" -d '{
            "text": "Terraform drift detected in production. Run terraform plan to review changes."
          }'

AWS Config Rules

# Enable an AWS Config rule that detects changes to security groups
aws configservice put-config-rule --config-rule '{
  "ConfigRuleName": "sg-no-manual-changes",
  "Source": {
    "Owner": "AWS",
    "SourceIdentifier": "EC2_SECURITY_GROUP_ATTACHED_TO_ENI_PERIODIC"
  }
}'

SCP (Service Control Policy) blocking manual changes

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Deny",
    "Action": [
      "ec2:AuthorizeSecurityGroupIngress",
      "ec2:RevokeSecurityGroupIngress"
    ],
    "Resource": "*",
    "Condition": {
      "StringNotLike": {
        "aws:PrincipalArn": "arn:aws:iam::*:role/terraform-*"
      }
    }
  }]
}

Verification

# After fixing - plan should be clean
terraform plan
# "No changes. Your infrastructure matches the configuration."

# If there are still changes - check if they're automatically managed attributes
# and add ignore_changes where needed

If terraform plan returns “No changes” after the fix, the drift is resolved. Implement drift detection in CI/CD to catch future changes within 24 hours instead of discovering them at the next deployment.

Drift is an inevitable side effect of environments where people have console access. The solution isn't removing access (you need it during incidents), but automatically detecting and flagging changes. Daily drift checks in CI/CD combined with SCPs that restrict changes to Terraform roles equals a 90% reduction in unauthorized deviations.
Jerzy Kopaczewski

Terraform drift getting out of control?

Book a free 30-minute call. We'll help clean up your state and implement automated drift detection.