ArgoCD and HPA: constant OutOfSync loop caused by spec.replicas

This runbook is part of a series on ArgoCD sync failures. See also: etcd request too large and operation deadlock. For broader GitOps context, read our post on GitOps and ArgoCD.

Symptoms

ArgoCD shows OutOfSync status for a HorizontalPodAutoscaler or Deployment resource. The diff points to a spec.replicas change that doesn’t come from Git:

# ArgoCD shows OutOfSync for HPA resources continuously
# Diff shows spec.replicas changing between desired state and live state
argocd app diff <APP_NAME> --local ./chart

# Output shows:
# HorizontalPodAutoscaler my-app-hpa:
#   spec.replicas: 3 -> 7 (live != desired)

Sync technically succeeds, but HPA immediately changes replicas back, causing an infinite loop: OutOfSync → Sync → OutOfSync. In environments with selfHeal: true, ArgoCD tries to “fix” this drift repeatedly, generating unnecessary operations and loading the controller.

Cause

HPA (HorizontalPodAutoscaler) automatically scales replica count based on metrics (CPU, memory, custom metrics). When HPA changes spec.replicas on a Deployment, that value diverges from the state stored in Git. ArgoCD treats this as drift — a discrepancy between desired state and live state.

This is a classic field ownership conflict between two controllers: ArgoCD wants to maintain the state from Git, while HPA wants to adjust replicas based on load. The same problem affects GitHub ARC (Actions Runner Controller), which dynamically scales runner pods — identical conflict mechanism.

Fix

Use ignoreDifferences with jqPathExpressions so ArgoCD ignores fields managed by HPA. Additionally enable the RespectIgnoreDifferences sync option so ArgoCD doesn’t overwrite those fields during sync:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/app.git
    targetRevision: main
    path: k8s/
  destination:
    server: https://kubernetes.default.svc
    namespace: my-app
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jqPathExpressions:
        - .spec.replicas
    - group: autoscaling
      kind: HorizontalPodAutoscaler
      jqPathExpressions:
        - .spec.metrics[].resource.target.averageUtilization
        - .status
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - RespectIgnoreDifferences=true

Version note: jqPathExpressions requires ArgoCD >= 2.1. The RespectIgnoreDifferences=true option requires ArgoCD >= 2.5. If you’re on an older version, use jsonPointers instead of jqPathExpressions (e.g., /spec/replicas).

# Apply the updated Application manifest
kubectl apply -f application-with-ignore.yaml

# Verify sync status returns to Synced
argocd app get my-app --output json | jq '.status.sync.status'
# Expected: "Synced"

After applying this configuration, ArgoCD won’t treat changes to spec.replicas as drift. HPA can freely scale the Deployment while the Application stays Synced. This approach is preferred over the alternative (removing replicas from Git manifests) because it preserves an explicit default value declaration in the repository.

The same pattern applies to other controllers that dynamically modify resources — e.g., Istio injection adding sidecars, cert-manager updating certificate status, or VPA (Vertical Pod Autoscaler) changing requests and limits.

Validation

# 1. Verify sync status is Synced (not OutOfSync)
argocd app get <APP_NAME> --output json | jq '.status.sync.status'
# Expected: "Synced"

# 2. Wait for HPA to scale (2-3 minutes) and check again
sleep 180
argocd app get <APP_NAME> --output json | jq '.status.sync.status'
# Expected: still "Synced" despite HPA changing replicas

# 3. Verify health status
argocd app get <APP_NAME> --output json | jq '.status.health.status'
# Expected: "Healthy"

# 4. Check that HPA is actually scaling
kubectl get hpa -n <NAMESPACE>
# Expected: TARGETS show current utilization, REPLICAS may differ from Git

If after several HPA scaling cycles the status remains Synced, the problem is resolved. The key is to wait at least one full autoscaling cycle (2-3 minutes) to confirm the OutOfSync loop doesn’t return.

CI/CD pipeline blocking your team?

Book a free 30-minute call. We'll review your ArgoCD configuration and pinpoint what to fix right away.

Book a call