AWS VPC networking in depth
Building blocks
| Component | Key facts |
|---|---|
| VPC | Regional private network with one or more IPv4 CIDRs (/16 to /28) and optional IPv6 |
| Subnet | Lives in one AZ; a subset of the VPC CIDR. AWS reserves 5 addresses per subnet |
| Route table | Associated with subnets; the local route makes everything in the VPC reachable |
| Internet gateway (IGW) | Horizontally scaled, highly available; enables internet access for resources with public IPs (performs 1:1 NAT for public IPv4) |
| Public subnet | Its route table has 0.0.0.0/0 → igw-... |
| Private subnet | No route to the IGW; egress via a NAT gateway (or none) |
| NAT gateway | Managed SNAT in a public subnet with an Elastic IP; zonal — deploy one per AZ for resilience; charged per hour and per GB processed |
| Egress-only IGW | Outbound-only for IPv6 (IPv6 addresses are public; no NAT) |
| ENI | Elastic network interface: private IPs, security groups, MAC; can move between instances |
| Elastic IP | Static public IPv4; public IPv4 addresses are billed hourly |
A private subnet that needs outbound internet:
private route table: 10.0.0.0/16 → local ; 0.0.0.0/0 → nat-xxxx (NAT GW in public subnet, same AZ)
public route table: 10.0.0.0/16 → local ; 0.0.0.0/0 → igw-xxxx
"EC2 in a private subnet can't reach the internet" — check: a NAT gateway exists in a public subnet, the private route table has the 0.0.0.0/0 → NAT route and is associated with that subnet, the NAT's public subnet routes to the IGW, and SG/NACL allow egress.
Security groups vs NACLs
| Security group | Network ACL | |
|---|---|---|
| Attached to | ENIs (instances, pods with SG-for-pods, LBs, RDS...) | Subnets |
| State | Stateful — return traffic automatic | Stateless — must allow return traffic (ephemeral ports 1024–65535) |
| Rules | Allow only | Allow and deny |
| Evaluation | All rules considered together | Numbered order, first match wins; final implicit deny |
| References | Can reference other security groups as sources | CIDRs only |
| Default | Deny inbound, allow outbound | Default NACL allows all |
Best practice: do most filtering with security groups referencing each other (ALB SG → app SG → DB SG); use NACLs sparingly as coarse subnet-level guardrails (e.g. block a known-bad range). A common outage: someone adds a restrictive NACL and forgets the outbound ephemeral port range, so replies are silently dropped.
Private connectivity to AWS services: VPC endpoints
| Type | Services | How | Cost |
|---|---|---|---|
| Gateway endpoint | S3 and DynamoDB only | A route-table target (prefix list) | Free |
| Interface endpoint (PrivateLink) | Most services: ECR (ecr.api, ecr.dkr), STS, SSM, Secrets Manager, CloudWatch Logs, KMS... |
ENIs with private IPs in your subnets; private DNS makes the normal service hostname resolve to them | Hourly per AZ + per GB |
Benefits: traffic stays on the AWS network, works from subnets with no internet access, and reduces NAT gateway data charges (a big cost item when nodes pull many images — ECR layers come from S3, so the S3 gateway endpoint alone saves a lot). Endpoint policies can restrict which resources are reachable (e.g. only your org's buckets).
PrivateLink endpoint services let you expose your own service (behind an NLB) privately to other VPCs/accounts — the secure way to share an internal API with a partner without peering networks.
Connecting VPCs and on-premises
- VPC peering — private routing between two VPCs (cross-account and cross-region possible). Not transitive: A–B and B–C does not give A–C. No overlapping CIDRs. Fine for a few VPCs; becomes a mesh nightmare at scale.
- Transit Gateway — a regional hub: attach many VPCs, VPNs and Direct Connect; route tables on the TGW control who reaches whom (segmentation: prod can't reach dev). Charged per attachment-hour and per GB.
- Site-to-Site VPN — IPsec tunnels over the internet (two tunnels per connection for redundancy), to a virtual private gateway or TGW.
- Direct Connect — dedicated private circuits to AWS: consistent latency, high bandwidth; often paired with VPN as backup.
- VPC Lattice — application-layer service networking across VPCs/accounts with auth policies (newer option).
Visibility and diagnosis tools
- VPC Flow Logs — records of accepted/rejected flows per ENI/subnet/VPC (src/dst, ports, action) to CloudWatch or S3.
REJECTentries reveal SG/NACL drops. - Reachability Analyzer — statically analyses the path between two resources and tells you which component (route, SG, NACL) blocks it. Great first step.
- Network Access Analyzer — finds unintended network access paths.
- Traffic Mirroring — copy packets to an analysis appliance.
A three-tier VPC design
Public subnets (a /24 in each of three AZs) hold the ALB and one NAT gateway per AZ; private app subnets (/19, sized for VPC CNI pod IPs) hold EKS nodes and pods; isolated data subnets (/24) hold RDS and ElastiCache with no route to the internet at all. Security groups chain alb-sg (443 from anywhere) → app-sg (8080 from alb-sg) → db-sg (5432 from app-sg), and VPC endpoints (S3 gateway; ECR, STS, Logs, SSM and Secrets Manager interface endpoints) keep AWS API traffic private.