Amazon EKS · Lesson 6 of 11 · 4 min read
Storage, add-ons and observability
Storage
- EBS CSI driver (EKS add-on, needs an IAM role): block storage for StatefulSets. gp3 StorageClass with
WaitForFirstConsumer(volume created in the pod's AZ), encryption, expansion, snapshots (VolumeSnapshot). EBS is single-AZ andReadWriteOnce. - EFS CSI driver: shared NFS file storage,
ReadWriteMany, multi-AZ — for shared content, ML datasets, legacy apps needing shared files. Dynamic provisioning creates access points. Higher latency than EBS. - FSx (Lustre for HPC/ML, NetApp ONTAP) and S3 Mountpoint CSI for object storage use cases.
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
Managed add-ons
EKS add-ons install and upgrade core components with AWS-validated versions: VPC CNI, CoreDNS, kube-proxy, EBS CSI, EFS CSI, Pod Identity Agent, CloudWatch Observability, ADOT, GuardDuty agent, Snapshot controller. Pin add-on versions per cluster version and upgrade them as part of every cluster upgrade.
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
Observability
- Control plane logs to CloudWatch:
api,audit,authenticator,controllerManager,scheduler— enable at least audit and authenticator for security investigations ("who deleted this?" — correlate with CloudTrail for AWS API calls). - Container Insights (CloudWatch Observability add-on) — metrics and logs for clusters, nodes, pods.
- Prometheus/Grafana — self-managed or Amazon Managed Service for Prometheus (AMP) + Amazon Managed Grafana (AMG); scrape with Prometheus agent/ADOT.
- Logs — Fluent Bit DaemonSet to CloudWatch Logs, OpenSearch, or S3/Loki.
- Tracing — OpenTelemetry (ADOT) to X-Ray or other backends.
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
Secrets
- Enable KMS envelope encryption for Kubernetes Secrets on the cluster.
- Source secrets from AWS Secrets Manager / SSM Parameter Store via External Secrets Operator (syncs into Kubernetes Secrets; works nicely with GitOps since only the
ExternalSecretreference is in Git) or the Secrets Store CSI driver with the AWS provider (mounts as files). - For GitOps-managed secrets in Git, SOPS with a KMS key (Flux decrypts natively).
Enlarge diagram
Swipe sideways to see all of it, or tap to open it full screen.
GitOps on EKS
Flux runs in the cluster and pulls from Git. On EKS: Flux controllers authenticate to ECR (image automation, OCI artifacts) and S3/KMS (SOPS decryption) with IRSA/Pod Identity; the source of truth is a Git repo with Kustomize bases and overlays per environment/cluster; Flux Kustomizations per cluster define what's applied, with dependsOn ordering (CRDs and controllers before apps).