Multi-Layer Load Balancing: DNS, Anycast and Ingress

Multi-layer load balancing means a request is balanced several times on its way to a pod. DNS or a global load balancer picks a region, a network load balancer picks a node, an ingress controller picks a Service, and the Service picks a pod. Each layer solves a different problem, and most "is this a bottleneck?" questions answer themselves once you know which layer does what.

This article starts from a common worry, that one NGINX ingress in front of every service must be a bottleneck, and works outward: scaling the ingress, DNS round-robin, GSLB, Anycast, Cloudflare Load Balancing, and externalTrafficPolicy.

Is a single ingress controller a bottleneck?

It can be, but not in the way it first looks.

An ingress controller is one logical component, not one pod. It runs as a Deployment you can scale to many replicas, or as a DaemonSet with a pod on every node. The external load balancer in front of it already spreads connections across those pods, so there are two layers of balancing before a request even reaches your routing rules.

The ingress also carries less traffic than people assume. It handles north-south traffic, from outside the cluster in. East-west traffic, such as one service calling another or a backend talking to its database, goes directly through ClusterIP Services and never touches the ingress. In a microservice system that's usually most of the traffic.

When the ingress does run hot, you have three levers:

1. Scale the controller. Add replicas, or attach an HPA on CPU. Ingress pods hold long-lived connections, so scale down slowly.
2. Run several controllers, split by IngressClass. For example, one controller for public traffic, one for internal traffic, or a dedicated controller for a single heavy domain. Each gets its own load balancer.
3. Keep internal calls off it. Services should call each other by their cluster DNS names, not through the public hostname.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: ingress-controller
  namespace: ingress
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: ingress-controller
  minReplicas: 3
  maxReplicas: 12
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 600

More on HPA behavior is in Kubernetes autoscaling. One note if you run the community ingress-nginx controller: the project stopped receiving maintenance in March 2026. Everything in this article applies equally to other ingress controllers and to Gateway API implementations.

Why DNS round-robin alone isn't enough

The next idea is usually to run several ingress stacks, each with its own public IP, and publish all of them under one hostname. Large sites do use DNS as a first layer, but on its own it has three weaknesses:

1. No health checks. If the stack behind one IP goes down, DNS keeps handing out that IP. Clients that receive it fail until someone edits the record and caches expire.
2. Caching makes distribution uneven. Resolvers and clients cache answers for the TTL, and some ignore very short TTLs. Many users behind one resolver can all land on the same IP.
3. No awareness of load. Round-robin rotates records blindly and keeps sending users to a site that is already busy.

Every technique below fixes one or more of these.

GSLB and geo-based routing

Global Server Load Balancing (GSLB) is DNS that makes decisions. The provider health-checks your endpoints and answers each query based on the results:

• An endpoint that fails its checks is removed from answers automatically.
• Answers can depend on where the user is. Users in Thailand get the Asia region, users in Europe get the Europe region, and nobody's traffic crosses the world unnecessarily.
• Weights and latency measurements can shift traffic between regions.

Amazon Route 53, Cloudflare and NS1 all offer this. It fixes the first and third weaknesses. The second remains: failover still waits for cached answers to expire, so keep TTLs short on records you may need to fail over.

Anycast and BGP

Anycast takes a different approach. Instead of handing out different IPs, you announce the same IP from many locations over BGP. Internet routing delivers each client to the nearest announcement in network terms. If a location fails, it withdraws its route and traffic flows to the next-nearest site. Clients keep using the same IP, so there is no DNS cache to wait for.

CDNs and large DNS providers work this way. It needs your own IP space and BGP peering, or a provider that runs an anycast network. The same idea appears inside a data center: MetalLB in BGP mode announces Service IPs from cluster nodes to your routers, which spread traffic across the nodes with ECMP.

One caveat: if routing changes in the middle of a long-lived TCP connection, packets can reach a different site and the connection breaks. Anycast works best for short connections, or in front of a proxy tier that terminates them.

Setting up Cloudflare Load Balancing

Cloudflare Load Balancing combines these ideas in three parts:

1. Monitor: how to check health, for example HTTPS GET /healthz expecting 200.
2. Pool: a group of endpoints, usually one pool per region, checked by a monitor. An endpoint that fails is taken out of the pool automatically.
3. Load balancer: bound to a hostname. It decides which pool receives each request and in what order pools fail over.

A monitor (POST /accounts/{account_id}/load_balancers/monitors), where two retries must fail before an endpoint is marked unhealthy:

{
  "type": "https",
  "method": "GET",
  "path": "/healthz",
  "expected_codes": "200",
  "interval": 60,
  "timeout": 5,
  "retries": 2
}

A pool per region (POST /accounts/{account_id}/load_balancers/pools), each endpoint being the public IP of one ingress stack:

{
  "name": "asia-pool",
  "monitor": "<monitor_id>",
  "origins": [
    { "name": "sg-ingress-1", "address": "203.0.113.10", "enabled": true },
    { "name": "sg-ingress-2", "address": "203.0.113.11", "enabled": true }
  ]
}

The load balancer (POST /zones/{zone_id}/load_balancers), steering by region with a fallback pool:

{
  "name": "www.example.com",
  "proxied": true,
  "steering_policy": "geo",
  "region_pools": {
    "SEAS": ["<asia_pool_id>"],
    "WEU": ["<europe_pool_id>"]
  },
  "default_pools": ["<asia_pool_id>", "<europe_pool_id>"],
  "fallback_pool": "<asia_pool_id>"
}

With proxied: true, Cloudflare's anycast edge terminates the client connection and forwards the request to the chosen pool, so failover doesn't depend on DNS caches. With DNS-only mode, Cloudflare answers DNS queries with healthy endpoint IPs and behaves like a GSLB, TTL caveat included.

The steering policy decides how pools are chosen:

Within a pool, endpoint steering can be random, hash-based, least outstanding requests or least connections.

The four layers of load balancing

Layer 4 is the one people forget. Every Service picks among pods whose readiness probes pass, so a failing readiness probe removes a pod from rotation at this layer. Kubernetes Service types covers how it works.

externalTrafficPolicy: Cluster vs Local

Between layers 2 and 4, one Service setting decides what happens when traffic reaches a node.

Cluster (the default). Any node accepts traffic for the Service. If that node has no matching pod, kube-proxy forwards the traffic to a pod on another node.

• Load spreads evenly, and every node is a valid target.
• Some requests take an extra network hop.
• The client source IP is lost. Traffic is SNATed to the node's address so replies return the same way, and the pod sees a node IP.

Local. A node delivers traffic only to pods running on that same node.

• No extra hop.
• The client source IP is preserved.
• Nodes without a local pod have no endpoint. For LoadBalancer Services, Kubernetes allocates a healthCheckNodePort, and the cloud load balancer marks those nodes unhealthy and skips them.
• Load can be uneven if pods are packed onto a few nodes, because the external LB balances per node, not per pod.

For an ingress controller, Local is common because you usually want real client IPs for logs, rate limiting and allow-lists. Pair it with a DaemonSet or topology spread constraints so ingress pods are spread evenly across nodes.

apiVersion: v1
kind: Service
metadata:
  name: ingress-controller
  namespace: ingress
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  selector:
    app.kubernetes.io/name: ingress-controller
  ports:
    - name: http
      port: 80
      targetPort: 80
    - name: https
      port: 443
      targetPort: 443

If Local doesn't suit your setup, many load balancers support the PROXY protocol, which passes the client IP to the ingress controller in-band.

FAQ

Is an NGINX ingress controller a single point of failure?

Not if it runs as several replicas across nodes behind a load balancer. It becomes one when it runs as a single pod, or when every replica sits on the same node.

What is the difference between GSLB and a load balancer?

A GSLB decides which region or data center receives a user, usually through DNS answers. A regular load balancer spreads connections across servers inside one location.

Does Anycast replace DNS load balancing?

It can replace DNS for failover and nearest-site routing, since clients keep one IP. Many setups still use DNS on top for weighting and manual traffic shifts.

When should I use externalTrafficPolicy Local?

When pods need the real client IP, or you want to avoid the extra hop. Make sure the load balancer health-checks nodes, and spread pods evenly.

Checklist for multi-layer load balancing

• Global layer health-checks every region and has a fallback pool.
• DNS TTLs are short on records you may need to fail over.
• The ingress controller runs several replicas across nodes, with an HPA.
• Heavy or internal traffic has its own IngressClass and controller.
• Service-to-service calls use cluster DNS and bypass the ingress.
• externalTrafficPolicy is chosen deliberately, and client IPs reach the logs.
• Readiness probes reflect whether a pod can really serve traffic.

Each layer should do one job well: the global layer picks the region, the network LB gets traffic into the cluster, the ingress routes, and the Service picks a healthy pod. If you want hands-on practice with Services, Ingress and autoscaling, Vectorkub's free DevOps courses cover them step by step.