LearnNewsExamplesServices
Frontmatter
id19680
titleBrain-gated burn-in retries once until 2026-10-25, naming every retried test
stateClosed
labels
enhancementaitestingbuild
assigneesneo-opus-grace
createdAtOct 12, 2026, 9:42 PM
updatedAtOct 12, 2026, 11:55 PM
githubUrlhttps://github.com/neomjs/neo/issues/19680
authorneo-opus-grace
commentsCount0
parentIssue19655
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtOct 12, 2026, 11:55 PM

Brain-gated burn-in retries once until 2026-10-25, naming every retried test

Closed Backlog/active-chunk-8 enhancementaitestingbuild
neo-opus-grace
neo-opus-grace commented on Oct 12, 2026, 9:42 PM

Context

The steward's call on #19657 (comment) accepted one retry as a bridge while the intermittent set is diagnosed (analysis), with these bounds:

  • only the Brain-gated burn-in tier retries; the engine tier and every PR-blocking job keep retries: 0;
  • the receipt names every test that needed the retry, with its first-attempt error;
  • a test red on both attempts is red;
  • the bridge ends at the class repair, and no later than 2026-10-25 (the v13.3 date).

The Problem

At the current rates, a fully green burn-in run has a probability of about 0.2%, so #19657's streak cannot start while the intermittent set is being diagnosed. A plain retry would hide exactly what the burn-in exists to measure. A Playwright retry runs in a fresh worker, so a cross-file leak always heals on it and is labelled flaky. That makes flaky a positive detection, never an all-clear.

The Architectural Reality

  • test-e2e-brain.yml runs the tier, and buildScripts/util/e2eCiSelection.mjs owns its selection and receipt (after #19676).
  • Playwright's JSON report marks a test flaky when an attempt failed and a later one passed. stats.flaky counts them, and each test's results[0] carries the first attempt's error.

The Fix

  • A declared bridge in e2eCiSelection.mjs: one retry until 2026-10-25, with its reason and tracker. The workflow reads the count from it, so the bridge ends on the date without anyone editing it, the same way quarantines age.
  • Every receipt prints the bridge's state and names each retried test with its first-attempt error.

Acceptance Criteria

  • The burn-in passes --retries=1 until 2026-10-25 and --retries=0 afterwards, read from the one declaration. No other job changes.
  • Every receipt names each flaky test with its first-attempt error and states the bridge and its end date. A test red on every attempt stays a failure.
  • Unit arms cover the date boundary and the receipt's flaky naming.

Out of Scope

Diagnosing the intermittent set (#19657), and any retry in the engine tier or a PR-blocking job.

Related

#19657 · #19655 · #19676 (lands first; same files)

Live latest-open sweep: checked the latest 20 open issues at 2026-10-12T19:41Z; no equivalent found. MC sweep: "playwright retries flaky e2e tier hides regression receipt retry policy decision". It surfaced the August finding that a retry runs in a fresh worker, so flaky is a detection, which this leaf honours by naming each one. No contrary decision. Own-assignment sweep: #19657 is the parent outcome, and #19675/#19678 touch the same files and land first.

Origin Session ID: 02040656-932a-48c8-8894-6f43d0368663

tobiu referenced in commit 1ea2a8b - "ci(e2e): the Brain-gated burn-in retries once until 2026-10-25, naming every retried test (#19680) (#19682) on Oct 12, 2026, 11:55 PM
tobiu closed this issue on Oct 12, 2026, 11:55 PM