Flaky Test Detection Engineer

Flaky tests are one of the most damaging silent problems in a CI/CD pipeline. A test that sometimes passes and sometimes fails for non-deterministic reasons erodes team trust in the entire test suite, forces costly reruns, and masks real failures. The Flaky Test Detection Engineer AI assistant helps teams systematically find, understand, and fix this problem.

This assistant explains how to instrument your CI pipeline to collect test result history and identify statistically flaky tests — those with inconsistent outcomes across identical code states. It covers approaches for different test types: unit tests with timing or ordering sensitivities, integration tests with external dependency issues, end-to-end tests with race conditions or UI timing problems, and tests with environment pollution from shared state.

The assistant guides you through building a flaky test quarantine system — a workflow that isolates known-flaky tests from blocking CI while tracking them for eventual fix or removal. It helps you write policies for automatic quarantine triggers, escalation thresholds, and ownership assignment so flaky tests do not quietly accumulate indefinitely.

For root cause analysis, the assistant walks through the most common failure categories: race conditions, time-dependent assertions, network calls in unit tests, shared mutable state, filesystem pollution, and random seed sensitivity. It suggests concrete debugging techniques including test retry analysis, log correlation, deterministic ordering enforcement, and dependency mocking strategies.

Ideal for QA engineers, platform teams, and development teams whose CI green rate has dropped below an acceptable threshold. This assistant turns a vague, frustrating problem into a structured engineering discipline.