IP addressing & subnetting · Lesson 7 of 9 · 6 min read
IP planning in AWS and EKS
AWS rules
- VPC CIDR: from /16 (65,536) down to /28 (16). You can add secondary CIDRs later (up to 5 by default) — including from 100.64.0.0/10.
- Subnet: /16 to /28, inside the VPC CIDR, in one AZ.
- AWS reserves 5 addresses in every subnet — for
10.0.1.0/24:.0network address.1VPC router.2DNS resolver (the VPC's base + 2 is also the Route 53 Resolver; e.g.10.0.0.2for10.0.0.0/16).3reserved for future use.255broadcast (not supported, but reserved)
- So a /28 has 11 usable, a /24 has 251, a /20 has 4,091.
- CIDRs cannot overlap for peering or Transit Gateway routing, and shouldn't overlap with on-prem ranges reachable over VPN/Direct Connect.
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
Planning for many accounts
A typical plan for an organisation with multiple accounts and regions:
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
- Allocate aligned blocks per region/environment so routes summarise.
- Leave room to grow; re-IPing later is painful.
- Track allocations centrally — AWS VPC IPAM allocates non-overlapping CIDRs from pools per region/OU, monitors usage and enforces rules (or a spreadsheet/NetBox in smaller orgs).
- Why 10.x for corporate plans: the largest private space (16.7M addresses) to carve non-overlapping ranges for every account and environment.
EKS: pods eat IPs
With the Amazon VPC CNI, each pod gets a real VPC IP from the node's subnet. Sizing consequences:
- A node can hold IPs only via its ENIs. Classic formula for max pods per node: ENIs × (IPv4 addresses per ENI − 1) + 2. Example:
m5.largesupports 3 ENIs × 10 IPs → 3 × (10 − 1) + 2 = 29 pods. - The CNI keeps warm IPs/ENIs pre-allocated per node, consuming addresses even before pods exist (tunable:
WARM_IP_TARGET,MINIMUM_IP_TARGET). - A cluster of 50 nodes × 30 pods needs ~1,500+ IPs plus warm pools — a /24 per AZ is far too small. Size private subnets for pods: /19–/20 per AZ is common.
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
When IPs run out (pods stuck Pending/ContainerCreating with "failed to assign an IP address"):
- Prefix delegation — assign
/28prefixes (16 IPs) to ENI slots instead of single IPs, raising pods per node dramatically (e.g. m5.large up to 110), and allocating IPs in chunks. Needs contiguous free /28 blocks — subnet fragmentation can bite; subnet CIDR reservations help. - Secondary CIDR + custom networking — add
100.64.0.0/16(non-routable outside the VPC) to the VPC, create pod subnets from it, and configureENIConfigper AZ so pods use those subnets while nodes stay in the original ones. - New, larger subnets for new node groups.
- IPv6 clusters for the long term.
- Release unused ENIs and tune warm pool settings.
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
Kubernetes CIDRs
- Pod CIDR — on EKS with the VPC CNI, pods use VPC subnets; with overlay CNIs, a separate cluster pod range.
- Service CIDR — virtual ClusterIPs (EKS default
10.100.0.0/16or172.20.0.0/16depending on the VPC range; configurable at creation). It's never routed outside the cluster but must not overlap with the VPC, pods, or networks pods need to reach — otherwise traffic to those addresses is hijacked by kube-proxy rules.