Opscraft
Senior DevOps interviews · Lesson 7 of 10 · 3 min read

Scenario rounds: model narrations

Enlarge diagramStabilisestop the harm firstMeasuredata before guessesFixthe biggest cause firstPreventa mechanism, not "becareful"
Figure 1. The shape of a strong scenario answer, whatever the scenario.
Swipe sideways to see all of it, or tap to open it full screen.

"Prod is down right after your deploy."

"Mitigate first: confirm the correlation, then roll back — revert the GitOps commit or abort the canary — and communicate status in the incident channel. Once stable, investigate with logs, metrics and the diff of the release. Then a blameless postmortem, and fix the gap that let it through — missing canary analysis, missing test, missing readiness probe."

"The pipeline is green but production is broken."

"The pipeline is missing post-deploy verification. I'd add smoke tests against the deployed environment, SLO-based checks during a bake period, canary analysis with automatic rollback, and synthetic monitoring — so 'green' means 'working in production', not just 'deployed'."

"The AWS bill jumped 40% this month."

"Cost Explorer grouped by service, then by usage type and tag, to find the spike. Usual suspects: NAT gateway data processing (fix with VPC endpoints), forgotten resources (idle clusters, unattached EBS, old snapshots), log ingestion growth, cross-AZ data transfer, or a scaling misconfiguration. Fix the cause, then add budgets and anomaly detection alerts and tagging enforcement."

"CI takes 45 minutes and developers complain."

"Measure first — per-stage timings over recent builds — rather than guessing. Then the usual wins: parallel stages, dependency and Docker layer caching, smaller images, ephemeral autoscaled agents to remove queueing, splitting or parallelising tests, running only what changed in a monorepo, and moving long E2E suites to post-merge or nightly."

"The team has alert fatigue."

"Review the last month of alerts: delete or downgrade ones nobody acted on, alert on symptoms and SLO burn rates rather than causes like CPU, add runbooks to every page, route non-urgent issues to tickets, and review alerts weekly with the team."

"A junior deleted a production resource."

"Restore service first — from backups, Terraform/GitOps re-apply. Then a blameless review: the problem is that the system allowed it. Add guardrails: least-privilege IAM, deletion protection and prevent_destroy, SCPs, changes only through pipelines, and better onboarding. Support the junior — it's a learning moment."

"Migrate from Jenkins to GitHub Actions."

"Inventory pipelines and plugins, identify common patterns, build reusable workflows and composite actions, set up OIDC to AWS and self-hosted runners on EKS if needed, migrate in waves starting with simple repos, run both in parallel briefly, then decommission Jenkins jobs."