Nothing kills trust in a test suite faster than flakiness. Once a team learns that a red build might just mean "run it again," the suite stops being a signal and becomes noise. Here's the checklist I actually work through when a test starts misbehaving.

Step 1: Confirm it's flaky, not broken

Run the test ten times in a row before you touch anything.

npx playwright test login.spec.ts --repeat-each=10

If it fails ten out of ten, it's not flaky, it's a real bug and you've just found it. If it fails two out of ten, you have a timing or state problem. Knowing which one you're dealing with saves an afternoon.

Step 2: Look for the four usual suspects

In my experience almost every flaky test traces back to one of these:

Waiting on time instead of state. Any sleep, waitForTimeout, or hard-coded delay is a bet on how fast the machine is today. CI machines are slower and busier than your laptop, so the bet eventually loses.

// Fragile
await page.waitForTimeout(2000);
await page.click('#submit');

// Stable
await expect(page.locator('#submit')).toBeEnabled();
await page.click('#submit');

Shared state between tests. If test B assumes test A already created a user, running them in parallel or in a different order breaks both. Each test should create what it needs and clean up after itself.

Animations and transitions. An element can be in the DOM, visible, and still sliding into position. Clicks land in the wrong place. Disabling animations in the test environment removes a whole class of failures.

Network variability. Third-party scripts, analytics, and slow endpoints all add jitter. Stub what you don't own.

Step 3: Capture evidence, don't guess

The single biggest improvement I made to my own debugging was turning on retained traces:

// playwright.config.ts
use: {
  trace: 'retain-on-failure',
  video: 'retain-on-failure',
  screenshot: 'only-on-failure',
}

The trace viewer shows you the DOM at the exact moment of failure, along with every action and network call. It turns "it failed sometimes" into "the button was covered by a toast notification."

Step 4: Fix the cause, then quarantine what's left

Retries are a painkiller, not a cure. I allow one retry in CI so a single hiccup doesn't block a merge, but I also track which tests use that retry. If a test needs its retry more than once a week, it goes on a list and gets fixed properly.

For tests that are genuinely unfixable right now, quarantine beats deleting: tag them, run them in a separate non-blocking job, and keep the failure visible.

Step 5: Write down what it was

Every flaky test I fix goes into a short note in the repo: the symptom, the cause, the fix. After a few months the pattern becomes obvious, and new tests get written correctly the first time. That's the part that compounds.

The rule I keep coming back to

A test that fails intermittently is telling you something true about your application or your test design. Retries hide the message. Traces read it.