Start free trial

By Yair Knijn · April 22, 2025

You discover you're out of address space during the incident, not before it

The platform engineering manager watches CPU, memory, disk, and cost. Those are the numbers that make it onto the dashboard and into the capacity plan. Address space is not one of them. The working assumption is that a /16 is effectively infinite, that subnets were sized once at landing-zone time and never need a second look. That holds right up until a traffic spike or a region failover doubles the pod count, and the cluster starts handing back failed to assign an IP address to container while everyone stares at green CPU charts.

The cruel part of the timing is that exhaustion is the one capacity wall you only hit when you most need headroom. Demand goes up, you scale out to meet it, and scaling out is the exact thing that runs you out of addresses. The incident and the cause are the same event.

Why IP space is capacity nobody puts on a dashboard

Disk fills gradually and you get a warning at 80 percent. Address space fills the same way, but almost nobody graphs it, so there is no 80 percent line and no alert. A /24 gives you 251 usable hosts after the cloud reserves five. On EKS with the VPC CNI, each node pre-warms a pool of secondary IPs on its ENIs, so a handful of busy nodes can drain an entire subnet long before you think of it as "full." The number that matters is not how many instances are running, it is how many addresses are committed against each subnet's free pool, trended over weeks.

The failure signature: instances that won't launch under load

Exhaustion does not announce itself as an address problem. It shows up as scaling that silently stops working. The autoscaler asks for more nodes, the EC2 API returns InsufficientFreeAddressesInSubnet, and the new nodes never join. Pods sit Pending. Your first signal is usually a latency alert or a queue backing up, because the thing that was supposed to absorb the load could not get an address. By the time someone reads the CNI logs, you are an hour into an incident a utilization graph would have flagged three weeks earlier.

Secondary CIDRs and CG-NAT space as emergency runway

When you are out, the standard escape hatch is to attach a secondary CIDR to the VPC and point pod allocation at the new range. AWS specifically recommends pulling that range from carrier-grade NAT space, 100.64.0.0/10 or 198.19.0.0/16, because those blocks are far less likely to collide with something already routed inside a corporate network than yet another RFC 1918 range you half-remember using. It works. It also means doing CIDR surgery on a production VPC while the incident bridge is live, which is precisely when you have the least room to think about overlap and who else owns that space. Treat CG-NAT as a planned expansion you provision early, not a thing you discover during the outage.

Utilization forecasting: getting the alert weeks before the wall

The fix is the same one you already apply to disk. Sample per-subnet utilization, fit a trend, and alert on the projected date you cross a threshold, not on the instantaneous percentage. If a subnet is climbing two percent a week and sits at 70, you want a ticket now, not a page at 99 during a launch. The signal you actually care about is "this subnet runs out in eleven days at current growth," and that is a line on a graph, not something you eyeball from a console.

Spot IPAM treats address space as the finite capacity it is. It reconciles claimed allocations against what's actually assigned, trends free addresses per subnet across every Environment, and forecasts the exhaustion date so the alert lands weeks before a scaling event turns it into an outage. See how the forecasting view works before your next traffic spike does the discovery for you.