Opscraft
Serverless with AWS Lambda · Lesson 5 of 6 · 4 min read

Operating serverless

Observability

Metric What it tells you
Errors, Invocations Error rate per function
Throttles Hitting concurrency limits
Duration (p50/p99) Latency, and how close you are to the timeout
ConcurrentExecutions Headroom against the account and reserved limits
IteratorAge (streams) How far behind the function is; rising means it can't keep up or is stuck on a bad record
AsyncEventAge, DeadLetterErrors Async backlog and failures to reach the DLQ
SQS ApproximateAgeOfOldestMessage Queue backlog

Every invocation writes START, END and REPORT lines to CloudWatch Logs, with duration, billed duration, memory used and Init Duration on cold starts. Log in structured JSON with the request ID, use Powertools for AWS Lambda for logging, metrics and tracing, and enable X-Ray or OpenTelemetry (ADOT) for traces across API Gateway, Lambda and downstream calls.

Orchestration with Step Functions

Chaining functions by having one call another directly is fragile: the caller pays for waiting, and errors are hard to handle. Step Functions state machines coordinate steps instead, with retries, back-off, catch blocks, parallel branches, maps over items, waits and human approval steps, all visible as a diagram.

Enlarge diagramValidate orderLambdaCharge cardretry 3× with back-offReserve stockDynamoDB directWait for warehousecallback tokenNotify customerSNSOn failurerefund and alert123456
Figure 1. Step Functions coordinates steps with retries and error handling, instead of functions calling functions.
Swipe sideways to see all of it, or tap to open it full screen.
  • Standard workflows: up to a year long, exactly-once step execution, priced per state transition. For business processes and long jobs.
  • Express workflows: up to 5 minutes, at-least-once, priced by duration. For high-volume event processing.
  • Direct service integrations call DynamoDB, SQS, ECS, Glue and others without a Lambda in between.

EventBridge is the other glue: rules route events between services and accounts, EventBridge Scheduler replaces cron jobs, and Pipes connect a source to a target with filtering and enrichment.

Cost and failure traps

  • Recursive loops: a function writes to the S3 bucket or SQS queue that triggers it. Lambda detects and stops many such loops, but design triggers carefully (separate input and output buckets or prefixes).
  • Retries multiplying cost and side effects: async retries plus SQS redelivery plus client retries. Make handlers idempotent and set retry counts deliberately.
  • Over-sized memory or timeouts on functions that run millions of times; use Power Tuning and Compute Optimizer recommendations.
  • Chatty logging: CloudWatch Logs ingestion can cost more than the function. Set log retention and levels.
  • Steady high traffic: at a constant few hundred requests per second, containers on Fargate or EKS are usually cheaper. Compare monthly cost at expected load, not just at zero.
  • NAT gateway data charges for VPC functions calling AWS APIs; use VPC endpoints.

Choosing Lambda, Fargate or EKS

Enlarge diagramLambdaevents, glue, spiky traffic,under 15 minFargatecontainers without nodes,steady servicesEKS on EC2many services, GPUs, bestcost at scale
Figure 2. A rough guide to where each workload belongs.
Swipe sideways to see all of it, or tap to open it full screen.
  • Lambda: event-driven tasks, glue between services, APIs with spiky or low traffic, scheduled jobs, file processing. Work under 15 minutes.
  • Fargate (ECS or EKS): containerised services without managing nodes, long-running processes, steady traffic, more control over runtime and networking.
  • EKS on EC2: many services with a shared platform, GPUs or special instance types, daemons, service meshes, maximum cost efficiency at scale.

Many real systems mix them: the main services on EKS, with Lambda handling S3 events, scheduled maintenance, alarms-to-Slack and other glue.