Production Bugs QA Often Misses — and How to Catch Them Earlier
Why realistic data, concurrency, retries, permissions, configuration, and observability produce failures that happy-path staging tests rarely reveal.
Key takeaways
- Many escapes live in interactions between state, scale, configuration, and time.
- Recovery and observability deserve explicit test scenarios.
- Production incidents should permanently improve models, checks, and monitors.
Retries and partial failure
A user clicks submit, the server commits, and the client times out. The retry creates a duplicate order. Test ambiguous outcomes by failing connections before and after persistence, repeating requests, and verifying idempotency plus a clear user status. Include downstream email, event, and payment side effects.
Concurrency and stale state
Two tabs edit the same record; inventory changes between review and purchase; a scheduled job runs while an account is deleted. These are state-transition problems, not exotic edge cases. Use controlled parallel requests, version checks, and event reordering to expose them.
Permissions that drift across boundaries
The UI hides an action, but the API accepts another user’s ID. A revoked role remains in a cache. An export includes fields the screen omits. Test authorization at every service boundary with cross-tenant identifiers, stale sessions, indirect object references, and background jobs.
Production-shaped data and configuration
Staging rarely has the history, null patterns, Unicode, time zones, feature flags, cache topology, or record volume of production. Use sanitized representative datasets, contract checks for configuration, and tests around daylight-saving changes, month ends, large histories, and partially migrated records.
- Exercise empty, typical, largest, oldest, and recently migrated accounts.
- Compare feature-flag combinations and safe rollback behavior.
- Test dependency slowness, rate limits, and stale caches.
- Verify logs and metrics contain identifiers needed for diagnosis without leaking secrets.
Accessibility and degraded modes
Mouse-only success can hide broken keyboard order, lost focus, inaccessible errors, or a control with no name. Fast-network tests miss double submissions and skeletons that never resolve. Include keyboard and assistive-technology checks plus slow, offline, and interrupted flows.
What should a QA engineer do next?
Review the last five incidents and classify the missing signal: requirement, test data, environment, concurrency, observability, or recovery. Add one targeted prevention and one detection mechanism for each—not merely another broad regression case.