LearnNewsExamplesServices
Frontmatter
id18694
titleThe e2e tier''s last pixel compares sit in one spec, with a private allowance and no declared mode
stateClosed
labels
bugaitesting
assigneesneo-opus-grace
createdAtSep 14, 2026, 3:29 AM
updatedAtSep 14, 2026, 11:18 PM
githubUrlhttps://github.com/neomjs/neo/issues/18694
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtSep 14, 2026, 11:18 PM

The e2e tier's last pixel compares sit in one spec, with a private allowance and no declared mode

neo-opus-grace
neo-opus-grace commented on Sep 14, 2026, 3:29 AM

Context

#18687 established that a pixel golden no CI workflow re-mints encodes whichever host and mode its minter happened to use, and replaced six reveal-pin screenshot arms across WorkstationNL and DockPinCollapseNL with structural assertions. Those arms now pass headed and headless alike; before the repair one family had never passed headed at all.

That leaves test/playwright/e2e/dashboard/PreviewLanguageDragPairNL.spec.mjs as the only file in the entire test/playwright/e2e tree still calling toHaveScreenshot — four arms, drag-pair-{default,signal}-{dark,light}.png.

grep -rl 'toHaveScreenshot' test/playwright/e2e --include='*.mjs'
→ test/playwright/e2e/dashboard/PreviewLanguageDragPairNL.spec.mjs   (4 calls)

The Problem

Three things are true of those four arms at once, and each is survivable alone.

  1. No arbiter. No CI workflow runs the Neural Link e2e journeys, so nothing re-mints or adjudicates these goldens. Whoever regenerates them last wins, silently.
  2. A private allowance. The spec passes its own {maxDiffPixels: 150} at the call site, while playwright.config.visual.mjs declares maxDiffPixelRatio: 0.001 and playwright.config.e2e.mjs declares no expect block at all. The tier had three policies; after #18687 it has one, but only because the other two sites were deleted — nothing states the surviving one, and nobody chose it.
  3. No declared mode. Headed and headless are a real render variable here, measured in #18687: a headed run added a one-pixel border that a headless mint never recorded. These arms declare no mode, so the same question that red-listed the reveal-pin goldens is still open for them.

The failure this already produces is on record: 17957 differing pixels against a 150-pixel allowance, roughly thirty times the class #18687 dealt with. #18687 deliberately put that failure Out of Scope so its own repair stayed reviewable, and its AC-1 and AC-4 therefore could not be claimed. This ticket is that residual, filed rather than left implied.

Why an allowance is not a policy. #14618 already recorded the shape: a ratio allowance can be vacuous under the same per-pixel perceptual threshold that hid 110 raw-differing pixels behind a reported 6 in #18687. A number at a call site is not a decision about what the tier compares, or in which mode.

The Fix

Pick one, for these four arms, and state it where the next reader will size something:

  1. Structural, as #18687 did — computed geometry and token values against a baseline captured in the same run, so no golden and no mode dependence remains. The reveal-pin repair is the worked example, including its two-seed non-vacuity control.
  2. Or an explicit contract — one declared mode for the tier, one threshold policy in the config rather than at call sites, and a run in the other mode that is refused with a message naming the contract rather than failing as a pixel diff.

Route 1 is preferred where the arm's subject survives it. These arms capture a drag treatment ("Signal-glow") whose appearance may genuinely need pixels; if so, route 2 is the honest answer and the contract should say why this is the exception.

Acceptance Criteria

  • AC-1 PreviewLanguageDragPairNL's four arms either compare no pixels, or run under a mode contract stated in the config rather than at the call site.
  • AC-2 A run in the non-declared mode either passes or is refused with a message naming the contract — it does not fail as a bare pixel diff.
  • AC-3 No threshold remains at a call site that the tier config does not declare.
  • AC-4 A seeded change to the treatment these arms exist to protect fails them, and passes when reverted, with the control constructed so the per-pixel threshold cannot hide it.
  • AC-5 After the change, grep -rl 'toHaveScreenshot' test/playwright/e2e returns either nothing, or only files covered by the stated contract.

Out of Scope

  • The four reveal-pin families repaired under #18687.
  • Why a headed render adds a one-pixel border. Parked in #18687 with four eliminations recorded; this ticket's fix does not depend on the answer.
  • playwright.config.visual.mjs's own tier, which has an arbiter.

Related

  • #18687 — established the failure mode, repaired six arms, and named this remainder as its residual.
  • #17844 — the census that surfaced the original screenshot reds.
  • #14618 — the prior measurement that a ratio allowance can be vacuous under the same threshold.

Unassigned on purpose: this is a residual owner, not a claim. Anyone may take it.

Live latest-open sweep: checked the latest 20 open issues at 2026-09-14T01:27:56Z; no equivalent found. A2A in-flight claim sweep: last 30 messages, 19:14Z through 01:09Z; no [lane-claim] overlapping this scope. Semantic sweep: ask_knowledge_base(type='ticket') returned #15206, #14618, #14650, #15252, #15243 — all prior art, none an open owner. #18547, which #18687 cites as having set this class aside, is CLOSED and therefore not the owner.

tobiu referenced in commit b8c0578 - "test(e2e): the preview-language drag pair reads each scene instead of comparing pixels, and the signal proxy ring paints again (#18694) (#18709) on Sep 14, 2026, 11:18 PM
tobiu closed this issue on Sep 14, 2026, 11:18 PM