Kubernetes Autoscaling: HPA, VPA and Cluster Autoscaler

Kubernetes autoscaling happens at three layers, and each one answers a different question. The Horizontal Pod Autoscaler (HPA) decides how many pods to run. The Vertical Pod Autoscaler (VPA) decides how much CPU and memory each pod should request. The Cluster Autoscaler decides how many nodes the cluster needs to fit those pods. They're separate controllers that don't coordinate directly, and most autoscaling problems come from not knowing where one stops and the next starts.

This guide walks through the HPA formula with a worked example, the metrics it can use, what VPA does and when to keep it in recommendation mode, how the Cluster Autoscaler reacts to pending pods, and the pitfalls that show up in production.

The three layers of Kubernetes autoscaling

All three depend on resource requests. HPA measures CPU utilization as a percentage of the request, VPA's whole job is to set requests, and the Cluster Autoscaler adds nodes based on requests, not actual usage. If requests are wrong, every layer makes wrong decisions. Kubernetes requests, limits and QoS covers how to set them.

How the HPA works: the control loop and the formula

The HPA is a control loop, like every other Kubernetes controller. By default it runs every 15 seconds. Each pass reads the current metric, computes the desired replica count, and updates the target's replicas field.

The core formula is:

desiredReplicas = ceil( currentReplicas × currentMetricValue / targetMetricValue )

Worked example

You have 3 pods. Each requests 500m CPU and is currently using 450m, which is 90% utilization. The target is 50%.

Think of it as total work. Three pods at 90% is 3 × 90 = 270 "units" of load. To bring every pod down to 50%, you need 270 ÷ 50 = 5.4 pods. You can't run 0.4 of a pod, so the HPA rounds up:

desiredReplicas = ceil(3 × 90 / 50) = ceil(5.4) = 6

With 6 pods sharing the same load, average utilization drops to 270 ÷ 6 = 45%, just under target.

Two details make the real behavior smoother than the raw formula:

• Tolerance. If the ratio current / target is within 10% of 1.0 (between 0.9 and 1.1), the HPA does nothing. At 53% against a 50% target the ratio is 1.06, so no change. This stops constant small adjustments.
• Multiple metrics. If you define CPU and requests-per-second, the HPA computes a desired count for each and uses the highest.

Because utilization is relative to the request, it can go above 100%. A pod requesting 500m with a limit of 1 CPU can report 200%. That's expected, and it's one more reason requests should reflect real usage.

A production HPA manifest

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 30
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 20
          periodSeconds: 60

The behavior block controls how fast it moves. Scaling up quickly and scaling down slowly is the usual choice. Here the HPA can double the pod count every 30 seconds but removes at most 20% per minute, and only after the recommendation has stayed low for 5 minutes. The default scale-down stabilization window is already 300 seconds. Set it explicitly anyway so the intent is visible.

Check what the HPA sees:

kubectl get hpa api
kubectl describe hpa api   # shows current metrics, conditions and recent scaling events

If TARGETS shows <unknown>, either metrics-server isn't running or the pods have no CPU request.

Choosing HPA metrics: resource, custom and external

CPU is the default because it's always available, but it isn't always the right signal.

Rules of thumb:

• CPU-bound web services: CPU utilization works well.
• I/O-bound services: CPU stays low while latency climbs. Scale on requests per second or in-flight requests instead.
• Queue workers: scale on backlog. KEDA is the common choice here, and it can also scale a Deployment to zero when the queue is empty.
• Memory: a weak scaling signal for most runtimes. The JVM, Go and Node.js often keep memory after load drops, so the HPA scales up and never scales back down.

You need to be collecting these metrics before you can scale on them. Observability for microservices covers that side.

Vertical Pod Autoscaler: right-sizing requests

People usually set requests by guessing. Set them too high and nodes look full while sitting idle, which wastes money. Set them too low and pods get CPU-throttled or OOMKilled. The VPA watches actual usage over time and recommends, or applies, better values.

VPA has three components:

• Recommender: analyzes usage history and computes target values.
• Updater: evicts pods whose requests are far from the recommendation, so they come back with new values.
• Admission controller: rewrites requests on new pods as they're created.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: api
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  updatePolicy:
    updateMode: "Off"
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed:
          cpu: 100m
          memory: 128Mi
        maxAllowed:
          cpu: "2"
          memory: 2Gi

The updateMode values:

• Off only produces recommendations. Read them with kubectl describe vpa api under Recommendation, then update your manifests yourself.
• Initial applies recommendations only when a pod is created.
• Recreate (and Auto) evicts running pods to apply new values.

Recent VPA releases also add an InPlaceOrRecreate mode. It uses in-place pod resize, where the cluster supports it, to change resources without restarting the pod.

Off mode is the safe place to start. It gives you data-driven numbers without surprise evictions.

Don't let HPA and VPA both act on CPU or memory for the same workload. When load rises, the HPA adds pods, per-pod usage falls, VPA lowers requests, utilization as a percentage of the smaller request rises again, and the two controllers fight. If you need both, run the HPA on a custom metric such as requests per second and let VPA manage CPU and memory, or keep VPA in Off mode.

Cluster Autoscaler: adding and removing nodes

The HPA can create pods, but if no node has room for their requests, those pods stay Pending. The Cluster Autoscaler watches for exactly that:

1. Scale up: a pod is unschedulable because of insufficient resources. The Cluster Autoscaler simulates whether a new node from one of its node groups would fit it, and if so asks the cloud provider for one.
2. Scale down: a node's requested resources stay below a threshold (50% by default) for a while (10 minutes by default), and all its pods can be rescheduled elsewhere. The node is drained and removed.

Scale-down is blocked by pods that can't safely move: a PodDisruptionBudget that allows no disruptions, bare pods with no controller, pods annotated cluster-autoscaler.kubernetes.io/safe-to-evict: "false", and some kube-system pods.

On AWS and Azure, Karpenter is a popular alternative. Instead of scaling predefined node groups, it picks instance types to fit the pending pods directly. The trigger is the same: pending pods and their requests.

How the three autoscalers interact

A traffic spike plays out like this:

1. Load increases and CPU utilization climbs past the target.
2. The HPA raises replicas within 15 to 60 seconds.
3. New pods that fit on existing nodes start right away.
4. Pods that don't fit go Pending, and the Cluster Autoscaler requests nodes.
5. New nodes boot, join the cluster and pull images, which takes minutes.
6. Pods pass their readiness probes and start receiving traffic.

Pod scaling takes seconds. Node scaling takes minutes. If spikes are sharper than your node provisioning time, you need headroom, either a higher minReplicas or overprovisioning. With overprovisioning you run low-priority placeholder pods that reserve space. Real pods preempt them instantly, and the evicted placeholders go Pending, which triggers a new node before you actually need it.

Common Kubernetes autoscaling pitfalls

• No CPU requests. The HPA can't compute utilization, and the Cluster Autoscaler can't plan capacity.
• replicas set in the Deployment manifest. Every kubectl apply or GitOps sync resets the count and fights the HPA. Remove spec.replicas once an HPA owns the workload.
• HPA and VPA on the same metric, as described above.
• Slow startup. If a pod takes 90 seconds to become ready, scaling at 90% CPU is too late. Lower the target or raise minReplicas.
• Hidden caps. maxReplicas, the Cluster Autoscaler's maximum node count and cloud quotas all limit scaling silently. Alert when you hit them.
• Scaling the wrong tier. More API pods won't help if the database is the bottleneck, and they can make it worse by opening more connections.
• Overly strict PDBs. minAvailable equal to the replica count means nodes can never be drained, so scale-down never happens.

FAQ

What is the difference between HPA and VPA?

HPA changes the number of pods. VPA changes the CPU and memory each pod requests. HPA handles load that can be spread across replicas, and VPA fixes pods that are sized wrong.

Why does my HPA show <unknown> for targets?

Usually metrics-server isn't installed or isn't healthy, or the target pods don't set a CPU request. Check kubectl top pods and the container resources block.

Can HPA scale to zero?

Not with a standard configuration, since minReplicas must be at least 1 unless an alpha feature gate is enabled. KEDA supports scale-to-zero for event-driven workloads.

How long does the Cluster Autoscaler take to add a node?

The decision is quick, usually under a minute. Most of the delay is the cloud provider booting the VM, the node joining the cluster, and image pulls, so expect several minutes end to end.

Should I use CPU or memory for HPA?

CPU or request rate for most services. Memory rarely drops when load drops, so memory-based HPAs tend to scale up and stay there.

Autoscaling checklist

• Every container has realistic CPU and memory requests. Use VPA in Off mode to find them.
• HPA with minReplicas of at least 2 or 3 for anything user-facing, and scale-down slower than scale-up.
• The metric matches the bottleneck: CPU, request rate or queue depth.
• spec.replicas removed from manifests managed by an HPA.
• Cluster Autoscaler or Karpenter configured, with alerts on max nodes and quota limits.
• PDBs that allow at least one disruption.

Autoscaling works well once the three layers have accurate requests and the right signal. If you want to practise tuning HPA and the Cluster Autoscaler on a real cluster, take a look at Vectorkub's DevOps courses.