ArgoCD and HPA: constant OutOfSync loop caused by spec.replicas

This runbook is part of a series on ArgoCD sync failures. See also: etcd request too large and operation deadlock. For broader GitOps context, read our post on GitOps and ArgoCD.

Symptoms

ArgoCD shows OutOfSync status for a HorizontalPodAutoscaler or Deployment resource. The diff points to a spec.replicas change that doesn’t come from Git:

# ArgoCD shows OutOfSync for HPA resources continuously
# Diff shows spec.replicas changing between desired state and live state
argocd app diff <APP_NAME> --local ./chart

# Output shows:
# HorizontalPodAutoscaler my-app-hpa:
#   spec.replicas: 3 -> 7 (live != desired)

Sync technically succeeds, but HPA immediately changes replicas back, causing an infinite loop: OutOfSync → Sync → OutOfSync. In environments with selfHeal: true, ArgoCD tries to “fix” this drift repeatedly, generating unnecessary operations and loading the controller.

Cause

HPA (HorizontalPodAutoscaler) automatically scales replica count based on metrics (CPU, memory, custom metrics). When HPA changes spec.replicas on a Deployment, that value diverges from the state stored in Git. ArgoCD treats this as drift, a discrepancy between desired state and live state.

This is a classic field ownership conflict between two controllers: ArgoCD wants to maintain the state from Git, while HPA wants to adjust replicas based on load. The same problem affects GitHub ARC (Actions Runner Controller), which dynamically scales runner pods. Identical conflict mechanism.

Fix

Use ignoreDifferences with jqPathExpressions so ArgoCD ignores fields managed by HPA. Additionally enable the RespectIgnoreDifferences sync option so ArgoCD doesn’t overwrite those fields during sync:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/app.git
    targetRevision: main
    path: k8s/
  destination:
    server: https://kubernetes.default.svc
    namespace: my-app
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jqPathExpressions:
        - .spec.replicas
    - group: autoscaling
      kind: HorizontalPodAutoscaler
      jqPathExpressions:
        - .spec.metrics[].resource.target.averageUtilization
        - .status
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - RespectIgnoreDifferences=true

Version note: jqPathExpressions requires ArgoCD >= 2.1. The RespectIgnoreDifferences=true option requires ArgoCD >= 2.5. If you’re on an older version, use jsonPointers instead of jqPathExpressions (e.g., /spec/replicas).

# Apply the updated Application manifest
kubectl apply -f application-with-ignore.yaml

# Verify sync status returns to Synced
argocd app get my-app --output json | jq '.status.sync.status'
# Expected: "Synced"

After applying this configuration, ArgoCD won’t treat changes to spec.replicas as drift. HPA can freely scale the Deployment while the Application stays Synced. This approach is preferred over the alternative (removing replicas from Git manifests) because it preserves an explicit default value declaration in the repository.

The same pattern applies to other controllers that dynamically modify resources: Istio injection adding sidecars, cert-manager updating certificate status, or VPA (Vertical Pod Autoscaler) changing requests and limits.

Alternative B) Remove spec.replicas from Git manifest:

Instead of ignoring the difference - remove the field entirely, giving HPA full control:

# Deployment WITHOUT spec.replicas (HPA has sole control)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  # replicas: NOT DECLARED - HPA has full control
  # HPA minReplicas defines the minimum, no need for replicas here
  selector:
    matchLabels:
      app: my-app
  template:
    # ...

With Kustomize, ensure the overlay doesn’t add replicas back:

# kustomization.yaml - DO NOT use the replicas: section
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../base
# ❌ REMOVE:
# replicas:
#   - name: my-app
#     count: 3

Alternative C) Server-Side Diff (ArgoCD 2.10+):

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
spec:
  source:
    repoURL: https://github.com/org/app.git
    targetRevision: main
    path: k8s/
  destination:
    server: https://kubernetes.default.svc
    namespace: my-app
  ignoreDifferences:
    - group: apps
      kind: Deployment
      managedFieldsManagers:
        - kube-controller-manager  # HPA operates through this manager
      jsonPointers:
        - /spec/replicas
  syncPolicy:
    syncOptions:
      - ServerSideApply=true
      - RespectIgnoreDifferences=true

Alternative D) Global configuration (for all applications):

# argocd-cm ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  resource.customizations.ignoreDifferences.apps_Deployment: |
    jsonPointers:
      - /spec/replicas
  resource.customizations.ignoreDifferences.apps_StatefulSet: |
    jsonPointers:
      - /spec/replicas
  resource.customizations.ignoreDifferences.argoproj.io_Rollout: |
    jsonPointers:
      - /spec/replicas

kubectl apply -f argocd-cm.yaml
kubectl rollout restart deployment argocd-repo-server -n argocd
kubectl rollout restart deployment argocd-application-controller -n argocd

KEDA ScaledObject - special case:

# With KEDA: ScaledObject creates HPA dynamically
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
spec:
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jqPathExpressions:
        - .spec.replicas
    - group: autoscaling
      kind: HorizontalPodAutoscaler
      jqPathExpressions:
        - .status
        - .spec.metrics

Validation

# 1. Verify sync status is Synced (not OutOfSync)
argocd app get <APP_NAME> --output json | jq '.status.sync.status'
# Expected: "Synced"

# 2. Wait for HPA to scale (2-3 minutes) and check again
sleep 180
argocd app get <APP_NAME> --output json | jq '.status.sync.status'
# Expected: still "Synced" despite HPA changing replicas

# 3. Verify health status
argocd app get <APP_NAME> --output json | jq '.status.health.status'
# Expected: "Healthy"

# 4. Check that HPA is actually scaling
kubectl get hpa -n <NAMESPACE>
# Expected: TARGETS show current utilization, REPLICAS may differ from Git

If after several HPA scaling cycles the status remains Synced, the problem is resolved. The key is to wait at least one full autoscaling cycle (2-3 minutes) to confirm the OutOfSync loop doesn’t return.

Frequently asked questions about ArgoCD OutOfSync

From an operational cost perspective, an HPA OutOfSync loop generates constant reconciliation attempts that consume ArgoCD controller resources and flood alerting channels with false positives. Teams learn to ignore the noise, which means a genuine sync failure goes unnoticed until a production incident surfaces hours later.

CI/CD pipeline blocking your team?

Book a free 30-minute call. We'll review your ArgoCD configuration and pinpoint what to fix right away.

Book a call Send us a message