By Yair Knijn · May 14, 2025

Kubernetes quietly ate your VPC: how pod-per-IP exhausts the address plan you sized for VMs

A platform engineering manager sizes a VPC the way the last three were sized: count the instances, double it for headroom, carve a few /24 subnets per availability zone, call the /16 generous. That math held for a decade of VMs. Then a team ships its first EKS cluster on the AWS VPC CNI, and within a quarter the scheduler starts parking pods in Pending because the subnets are dry.

The wrong assumption is not the subnet size. It is the unit of allocation. You planned against machines. Kubernetes plans against pods, and on native-VPC networking every pod is a real, routable VPC address.

Why one IP per pod breaks VM-era address math

With the AWS VPC CNI in default mode, each pod gets a secondary IP from the node's elastic network interface, drawn straight from the subnet the node sits in. A single m5.large can run dozens of pods, and the CNI pre-warms a pool of IPs per node, so addresses are reserved before any pod asks for one. Consumption is not "one IP per server" but "one IP per pod, plus a warm buffer per node." A few hundred instances you budgeted at a few hundred addresses now want thousands.

The failure is quiet because nothing errors at provision time. The VPC accepts the cluster, nodes join, pods schedule. The wall arrives later, when a scale-up finds no free secondary IPs on any ENI and the pod sits Pending with no obvious networking error in its events.

Prefix delegation and 100.64.0.0/10: stretching the space

The two real levers are density and supply. Prefix delegation switches the CNI from single secondary IPs to attaching /28 prefixes (16 addresses) per allocation, raising pod density per node and cutting API churn. The catch is fragmentation: a /28 needs a contiguous block, so a subnet littered with scattered secondary IPs can refuse new prefixes with InsufficientCidrBlocks even when the free-address count looks fine.

For supply, AWS's own guidance is to add a secondary VPC CIDR from carrier-grade NAT space, typically 100.64.0.0/10 or 198.19.0.0/16, and point the CNI's custom networking at it so pods draw from that non-routable range instead of your scarce RFC 1918 blocks. It works, but it is a second address plan bolted onto the first, with its own ENIConfig objects, subnets per AZ, and overlap risk against anything else squatting in CG-NAT.

Why you can't resize a full subnet, only migrate off it

Here is the part that turns a sizing mistake into a project. A VPC subnet's CIDR is immutable. You cannot grow a full /24 into a /22. The remedies all amount to standing up new address space and moving workloads onto it:

• Add a secondary CIDR, create new subnets, and reschedule pods onto nodes wired to them.
• Enable prefix delegation, which changes allocation behavior cluster-wide and usually wants a node roll.
• Adopt custom networking, which means new ENIConfigs and replacing the nodes so pods land on the new subnets.

Every one of those touches running nodes. None is a toggle you flip on a Friday afternoon. The fix is a migration, not a setting.

Sizing pod CIDRs before the cluster, not after it stalls

Size against peak pod count and the CNI's warm-pool behavior, not instance count, and reserve the secondary CG-NAT block at VPC creation so it exists before the first cluster lands. That only works if you know what is already allocated. The recurring root cause is not bad arithmetic, it is an address plan nobody can trust: a /16 marked free that a forgotten cluster is quietly draining, two teams handed overlapping CG-NAT ranges, no record of which subnet feeds which ENIConfig.

Spot IPAM treats the pod address plan as a first-class part of each Environment, reconciling what the spreadsheet claims against what the VPC and CNI actually hand out, flagging an overlapping secondary CIDR before two clusters collide in 100.64.0.0/10, and showing utilization trends so you see a subnet filling weeks before pods go Pending. See how the allocation and overlap detection works.