Test Data Management: The Boring Problem That Breaks Everything

Ask a testing team what their biggest problem is and you'll hear "flaky tests" or "not enough coverage." Dig into either one and you'll usually find test data underneath.

The four ways test data goes wrong

The shared account. One testuser@example.com that every test logs in as. It works until two tests run in parallel, one changes the account's settings, and the other fails for reasons that have nothing to do with the code under test.

The seeded database that drifts. A snapshot restored before the run, kept in a SQL file that someone updated eight months ago. New tests are written against the current schema, old tests against the old assumptions, and nobody knows which rows are safe to touch.

Data that expires. Tests that assert on "orders from the last 30 days" with hard-coded dates. They pass in March and fail in May.

Data that leaks. Every test run creates records nobody removes. Six months later the staging database has 400,000 test users and the login page takes nine seconds to load.

What works instead

Each test creates its own data

The single most valuable change: make every test responsible for its own setup, with unique values.

function uniqueEmail(prefix = 'qa') {
  return `${prefix}+${Date.now()}-${Math.random().toString(36).slice(2, 8)}@example.com`;
}

test('user can update their display name', async ({ page, request }) => {
  const email = uniqueEmail('displayname');
  const user = await createUserViaApi(request, { email });

  await loginAs(page, user);
  // ...
});

Now the test can run in parallel with fifty copies of itself and none of them interfere.

Set up through the API, assert through the UI

Creating a user by filling in a registration form takes eight seconds and fails if the registration page has a bug unrelated to what you're testing. Creating one with a POST request takes 200 milliseconds.

Use the UI to test the thing you're testing. Use the API for everything else.

Build data with factories, not fixtures

A fixture file is a fixed blob of JSON that every test has to work around. A factory is a function that gives you a sensible default and lets you override the one field you care about.

function buildOrder(overrides = {}) {
  return {
    status: 'pending',
    currency: 'SEK',
    items: [{ sku: 'TEST-001', quantity: 1, price: 199 }],
    createdAt: new Date().toISOString(),
    ...overrides,
  };
}

const refundable = buildOrder({ status: 'delivered' });

When the schema gains a required field, you change the factory once instead of editing forty fixture files.

Use relative dates, always

const daysAgo = (n) => new Date(Date.now() - n * 86_400_000);
const recentOrder = buildOrder({ createdAt: daysAgo(3).toISOString() });

Clean up, and have a backstop

Delete what you created in teardown. Then accept that teardown sometimes doesn't run, and add a scheduled job that removes test records older than a few days. The prefix in your generated emails makes them easy to find.

The check I use

If I can't run the entire suite twice in a row, on the same environment, in parallel, and get identical results both times, the test data strategy isn't finished. That one criterion has caught more problems for me than any coverage report.