Feature flags, dark launches, A/B and shadow traffic
Feature flags
Code paths controlled at runtime:
if flags.is_enabled("new-checkout", user=user):
return new_checkout(cart)
return old_checkout(cart)
- Separate release from deploy — merge and deploy dark; enable for internal users, then 1% of customers, then a region, then everyone.
- Kill switch — turn a feature off instantly without a redeploy or rollback.
- Targeting — by user, tenant, plan, region.
- Tools: LaunchDarkly, Unleash, Flagsmith, AWS AppConfig feature flags, OpenFeature as a vendor-neutral API.
Flags vs canary: a canary shifts traffic between versions (catching infra/runtime regressions: memory leaks, dependency changes, config). Flags toggle features within one version (catching product/logic issues per segment). They complement each other.
Downsides: flag debt (dead flags rotting in code), combinatorial testing complexity, a flag service as a dependency (cache flag values; define defaults). Give flags owners and expiry dates.
Dark launch
Deploy code (and even exercise it with real traffic in the background) without users seeing results — e.g. a new service receiving copies of requests, or a feature hidden behind a flag.
Shadow (mirror) traffic
Copy live requests to the new version and discard its responses. You test with real load and real request shapes, with no user impact. Mirroring is supported by Istio, Envoy, NGINX (mirror directive) and some ALB patterns via tooling.
The big risk is side effects: a mirrored payment request that actually charges a card, or a duplicated email. Shadow only read paths, or point the shadow at stubbed downstreams and isolated databases.
A/B testing
Splitting users between variants to make a product decision (which checkout converts better) with statistical analysis. It uses similar traffic-splitting tools but the goal is experimentation, not deployment safety.
Ring deployments
Expose progressively larger rings: internal staff → beta customers / a small region → large region → everyone. Common for SaaS and mobile backends; maps naturally onto multi-region infrastructure.