test.describe.serial stops 35 unit specs at their first failure

Context

While reviewing #19015 I measured one spec file with and without test.describe.serial, same head, one variable changed:

Two arms of diagnostic signal were lost on every regression in that file, and nothing reported them as missing — a serial block that aborts does not print the arms it skipped as skips. @sloemo01 landed the one-file removal in #19022 and correctly declined to generalise from a single sample. This ticket is the rest of the tier.

Live latest-open sweep: checked the latest 20 open issues at 2026-09-21T10:20Z; no equivalent found.

The Problem

test.describe.serial tells Playwright that the arms inside share state and must run in order, and that a failure invalidates everything after it. That contract is real for a block whose second arm depends on what the first one left behind. It is false for a block whose arms each build their own fixture — and when it is false, the only thing .serial buys is a truncated report.

The cost lands exactly when the suite matters most. A green run is identical either way; a red run under .serial shows one failure and hides however many arms followed it, so the first fix cycle is scoped from partial evidence and the next failure only appears on the next run. Measured on my own tickets, it has already downgraded coverage claims on #14574, #17150 and #17162 — in each case a "N passed" number that counted a population smaller than the file.

This is the same family as the truncated-read failures the repo already guards against elsewhere: a limited zero is not an absence.

The Architectural Reality

At b2b9368802 there are 415 .spec.mjs files under test/playwright/unit, and 35 of them contain describe.serial. The component tier contains zero, so this is a unit-tier convention rather than a house style.

git grep -l 'describe\.serial' -- 'test/playwright/unit/**/*.spec.mjs'

The unit tier runs through test/playwright/playwright.config.unit.mjs, and Neo's unit specs build their subject with Neo.create inside each arm rather than in a shared beforeAll, which is why most of these blocks have no cross-arm state to protect. That is a property to verify per file, not to assume: some specs legitimately mutate a singleton, a global config, or a shared collection across arms.

The Fix

Per file, in test/playwright/unit:

1. Read the arms. Decide whether any arm depends on state another arm produced — a shared instance built outside the arms, a mutated singleton, an ordering assumption in the assertions.
2. No dependency → test.describe.serial becomes test.describe.
3. Dependency → keep .serial and add a one-line comment naming the state that requires it, so the next reader does not re-derive the question.

The comment is the part that makes this stick. A .serial with a stated reason is a decision; a .serial without one is inheritance.

🔴 Scope narrowed 2026-09-21T15:55Z — this ticket is now the 28 non-data/ files, and that is the fix for a defect in the ticket itself

The root error was mine at creation: 35 files across 13 directories is not one-PR-resolvable, and a ticket that cannot be resolved by one PR generates exactly the mess that followed — first a PR resolving nothing, then a retroactive leaf, then an ordering trap where two PRs each deliver part of one ticket and whichever merges second silently completes it.

Carve-out, so every PR resolves exactly one ticket it fully delivers:

• the data/ 7 are #19042, delivered by PR #19032;
• the remaining 28 are this ticket, delivered by PR #19044.

Neither PR can close the other's scope, and the merge order no longer matters.

Two superseded instructions, both mine, recorded rather than deleted. "A contributor can take a directory rather than all 35 at once" invited the partial PR. Its first correction — "a slice gets its own small issue" — was corrected in turn by @tobiu: contributors do not file tickets so their PR has something to close. They raise scope on the issue and a maintainer sizes it. That is what @wofiporia did on PR #19044, and this carve-out is the answer.

Acceptance Criteria

Scope: the 28 describe.serial files under test/playwright/unit outside data/. The data/ 7 belong to #19042.

• AC-1 — Every describe.serial in those 28 is either removed or carries a comment naming the shared state that requires it. A file already carrying one is delivered and needs no diff.
• AC-2 — For each removal, the PR body shows the before/after arm counts for that file under a deliberately failing arm — the measurement, not the assertion that it was safe.
• AC-3 — The unit suite passes for that scope with the sanctioned config (npm run test-unit -- <paths>); a bare npx playwright test resolves a different config and is not evidence.
• AC-4 — Order-independence is witnessed at more than one worker count. One worker count cannot distinguish "these arms are independent" from "these arms happened to pass in order", which is the property the change asserts.
• AC-5 — git grep -l 'describe\.serial' -- test/playwright/unit returns only files that carry a stated reason. This is the greppable predicate that replaces "which directories remain", now that no directory remains.

Out of Scope

• The component, e2e and integration tiers — the component tier has none and the others are a different runner contract.
• Fixing any test that turns out to be red once the arms after the first failure actually run. That is a finding to file, not to bury in this PR.
• Any lint enforcing the convention. Worth considering once the population is known, not before.

Out of Scope (added with the carve-out)

• The data/ 7. They are #19042, delivered by PR #19032, and this ticket cannot close them.

Related

• #19042 / PR #19032 — the data/ 7, carved out of this ticket
• PR #19044 — delivers this ticket's remaining 28
• #19022 — the one-file removal and its measurement.
• #19015 — where the measurement was taken.

unowned-rationale: parked as a community good-first-issue; it is directory-splittable and carries no deadline, so it stays unassigned until someone claims a slice.

Origin Session ID: 2c9d83d3-7879-46f6-b49d-590b631d4f55

Retrieval Hint: query_raw_memories("describe.serial truncates unit tier arm reporting"); tree anchor b2b9368802.