LearnNewsExamplesServices
Frontmatter
id18226
titleRerun isolation and the mergeability controller run uncovered in CI
stateClosed
labels
bugaitestingregressionbuild
assigneesneo-opus-grace
createdAtSep 3, 2026, 8:33 PM
updatedAtSep 3, 2026, 9:47 PM
githubUrlhttps://github.com/neomjs/neo/issues/18226
authorneo-opus-grace
commentsCount1
parentIssue17922
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtSep 3, 2026, 9:47 PM

Rerun isolation and the mergeability controller run uncovered in CI

Closed Backlog/active-chunk-20 bugaitestingregressionbuild
neo-opus-grace
neo-opus-grace commented on Sep 3, 2026, 8:33 PM

Refs #17922 · leaf, mirroring #17924, #18052 and #18209

Context

#17922 measured that c623b2f63c — "feat(engine): remove received Brain implementation (#17791) (#17806)" — deleted 804 unit specs on the ai/ path prefix, and that for 27 of them the subject stayed in neomjs/neo. Twenty-five of those 27 are restored today.

Re-measured at dev@5baddd0e1c by git ls-tree -r --name-only origin/dev test/playwright/unit/:

owed row present at dev?
release/PublishReleaseNoteOrphan.spec.mjs yes — delivered by #18209 / PR #18210, merged today
util/WorkflowConcurrency.spec.mjs no
docs/RebuildContentIndexesAndSeo.spec.mjs no

So the owed set is 2, not 3, and this ticket takes the one #17922 names as sharpest: "675 lines, 17 clean arms plus the 2 the parity lane's departure retires, and its subject runs on every PR."

The Problem

The deleted spec was test/playwright/unit/ai/buildScripts/util/WorkflowConcurrency.spec.mjs (675 lines, recovered verbatim from c623b2f63c^). It covered three subjects, and all three are still here and still running:

  1. The rerun-isolation concurrency contract. test.yml's group is tests-${{ github.workflow }}-${{ github.run_attempt == '1' && github.ref || github.run_id }} with cancel-in-progress: true. The run_attempt ternary is the whole point: initial attempts share the ref stream so a new head supersedes old work, while a rerun keeps its own run_id and therefore cannot cancel a newer head (#15593). The deleted arm swept every .yml in .github/workflows/ and required that any workflow with pull_request + cancel-in-progress: true carry that exact contract. Nothing asserts it now — a new canceling PR workflow authored without the ternary is a silent regression, and the failure it re-opens is "a rerun cancelled the run for the head that superseded it", which reads as a flake.

  2. The jobs.test head-currency gate. A separate actions/github-script step (id: head) re-reads the live PR head inside the test job, because re-running failed jobs does not re-run the changes prerequisite. Every step after it is gated on steps.head.outputs.current == 'true'. An expensive step added without that gate spends a full runner on a superseded head, and nothing red would say so.

  3. The review-admission mergeability controller — the whole of .github/workflows/review-admission-mergeability.yml (314 lines), which publishes the review-admission/mergeability commit status. Its 12 deleted arms cover the parts that are easy to get quietly wrong: pull_request_target evaluating only its event PR while board triggers enumerate, matrix-overflow refusing partial publication, conflict publishing on the PR head and never the base, null mergeability exhausting to error and never to success, and head/base movement observed mid-poll restarting on the new coordinate.

#17922's own framing applies: every green unit run since 2026-08-27 has certified fewer specs behind the same signal.

The Architectural Reality

  • Subject 1 + 2 — .github/workflows/test.yml (366 lines): concurrency.group at :12, jobs.changes.steps[id=scope], jobs.test.steps[id=head] and the ten steps gated on it.
  • Subject 3 — .github/workflows/review-admission-mergeability.yml (314 lines): jobs.discover.steps[id=targets] and jobs.publish.steps[0].
  • Placement — test/playwright/unit/buildScripts/util/, beside the surviving sibling tree, not the old unit/ai/ path. The ai/ prefix is exactly what marked these as extracted substrate and would re-arm the same deletion at the next boundary change. Sibling precedent: util/WorkflowScopeClassifier.spec.mjs, restored by 8c86f4478f (#18053).
  • The subject is reached through YAML, not imported. These classifiers are strings inside workflow files, evaluated by actions/github-script with github, context and core in scope. The spec reads the committed string and evaluates it against mocks under those three names. Re-typing the logic would test a copy, and a copy cannot drift with its original.
  • neomjs/neo CI runs unit and components only, so the unit project is the surface that must regain this.

The Fix

Restore the file at test/playwright/unit/buildScripts/util/WorkflowConcurrency.spec.mjs, not verbatim, with every divergence from the deleted original named in the file header rather than trimmed silently.

Two arms retire — the outputs they assert no longer exist. run_integration and run_parity left with the extraction; the changes job now declares run_unit, run_components and skip_reason only. The two parity-lane arms (routes the parity surface to its lane, cannot skip the lane for the booted stack's runtime dependencies) assert ai/services/memory-core/MemoryService.mjs tripping a lane that is gone. That is a legitimate topology change in the subject, not a defect in it.

Two arms retire as superseded, not as topology. admits only the live pull-request head and preserves current-head and push admission assert the scope step's stale-head guard, which util/WorkflowScopeClassifier.spec.mjs now covers (its stale-head guard describe). Restoring them here would be two specs owning one assertion — the drift hazard the sibling spec's own header names.

The subject split between the two files is therefore stated, not implied: WorkflowScopeClassifier owns which paths admit which suite; this file owns head currency, rerun isolation, and the review-admission controller. The one scope-step arm that stays here — a live-head lookup failure fails closed before path classification — stays because it is a head-currency assertion with no sibling equivalent, and dropping it would lose coverage rather than relocate it.

One arm is strengthened rather than restored. The deleted failed-job reruns recheck the live head before every expensive step arm carried a hardcoded list of eight step names, three of which no longer exist (Skip Knowledge Base download in prepare lifecycle among them). A name list cannot see a newly added expensive step, which is the regression that matters. It is replaced by a derived assertion: every step positioned after the head step must reference steps.head.outputs.current in its if. That is a completeness claim the old list could not make.

Two head-currency arms merge into one. The deleted failed-job reruns recheck the live head and the failed-job gate preserves current-head and push retries both drive the same id: head script and differ only in which of the three arms of one branch they exercise. They become a single arm covering all three — stale, current head, push — because splitting one script's truth table across two test names is how a later reader concludes the second one is redundant and deletes it.

Net: 15 arms — 1 rerun-isolation, 3 head-currency, 11 review-admission — from the deleted 19 (8 + 11).

Contract Ledger Matrix

Target Surface Source of Authority Behavior Fallback Docs Evidence
test.yml concurrency.group #15593 every pull_request workflow with cancel-in-progress: true carries the run_attempt ternary none — a vulnerable workflow is a red spec spec header mutation: strip the ternary from test.yml, the arm must name test.yml as vulnerable
test.yml jobs.test step gating this ticket every step after id: head references steps.head.outputs.current in its if none spec header mutation: add an ungated step after head, the arm must go red
review-admission-mergeability.yml publication #17692 status publishes on the PR head, never the base; null exhausts to error; overflow refuses partial publication none spec header the 12 restored arms; each evaluates the committed script string
spec placement #17922 (ai/ prefix is what marked these as extracted) test/playwright/unit/buildScripts/util/ none ticket sibling WorkflowScopeClassifier.spec.mjs, added there by 8c86f4478f

Decision Record impact

none — this restores coverage for subjects already owned here. ADR 0040 §2.3 ("the Engine never imports from the extracted repository") is what makes these subjects Engine-owned in the first place; the ADR is not in this repository (it lives at neo-agent-brain:learn/agentos/decisions/0040-agentos-extraction-topology.md) and that absence is not evidence against the citation.

Acceptance Criteria

  • AC-1 test/playwright/unit/buildScripts/util/WorkflowConcurrency.spec.mjs exists and passes under npm run test-unit.
  • AC-2 Mutation, rerun isolation: removing github.run_attempt == '1' && github.ref || from test.yml's concurrency group turns the arm red and names test.yml in the vulnerable set. Reverted after.
  • AC-3 Mutation, head gate: adding a step after id: head whose if omits steps.head.outputs.current turns the arm red. This is the assertion the deleted spec's hardcoded name list could not make. Reverted after.
  • AC-4 Mutation, review-admission: flipping the publish script's head coordinate to the base SHA turns conflict publishes failure on the PR head, never the base red. Reverted after.
  • AC-5 Every divergence from the deleted 675-line original is named in the file header — the two topology retirements, the two superseded-by-sibling retirements, and the one strengthened arm — with the reason. A restored spec that quietly sheds assertions is indistinguishable from one trimmed to make the suite green.
  • AC-6 No duplicate ownership with util/WorkflowScopeClassifier.spec.mjs: no assertion in this file re-states one of its arms.

Out of Scope

  • docs/RebuildContentIndexesAndSeo.spec.mjs — the other owed row. Its deleted assertions target ai/services/github-workflow/SyncService.mjs, which departed; it needs a rewrite-vs-retire decision, not a restoration, and that decision is #17922's to make.
  • Closing #17922. This is one of two owed rows; the parent cannot be the close target of a PR that delivers one of them. Same reason #17924, #18052 and #18209 exist.
  • Any change to test.yml or review-admission-mergeability.yml. This ticket adds coverage for what they do today; if the coverage finds a defect, that is a separate ticket with its own evidence.
  • #17171 — that none of the 19 *-lint.yml workflows is a required status context. It is why a red spec-retirement job could not have blocked the merge, and it is owned there.

Avoided Traps

  • Restoring the 675 lines verbatim. Four of the 19 arms assert surfaces that are gone or now owned by a sibling spec. A verbatim restore would be red on arrival, and "make it green" would then quietly delete assertions without a record — the exact failure #17922 exists to name.
  • Keeping the hardcoded expensive-step list. It passes today only by coincidence of names and is blind to the regression that matters: a new ungated step.
  • Re-asserting the sibling's arms for safety. Two specs owning one assertion is not redundancy, it is drift with two homes.
  • Restoring to the old unit/ai/ path. The ai/ prefix is the marker that caused the deletion.
  • Grading this by line count. #17922's own scope note: restored length diverges from deleted length in both directions, and line count is not coverage. AC-2 through AC-4 are what grade this.

Related

  • #17922 — the parent census; this is one of its two remaining owed rows
  • #17924 / PR #17925 — the 21 verbatim restorations
  • #18052 / PR #18053 — WorkflowScopeClassifier, the sibling that shares this subject and defines the split
  • #18209 / PR #18210 — PublishReleaseNoteOrphan, delivered today
  • #15593 — the rerun-isolation contract this spec asserts
  • #17692 — the review-admission mergeability controller
  • #17791 / c623b2f63c — the split commit that deleted the spec
  • #17171 — no *-lint.yml workflow is a required status context

Sweeps before filing — live latest-open neomjs/neo at 2026-09-03T18:31Z (latest 15 open, plus full-text concurrency in:title and all-state WorkflowConcurrency): no equivalent, and the only prior WorkflowConcurrency hits are #17922 itself and the three closed sibling leaves. A2A claim sweep, last 30 messages all read-states: no [lane-claim]/[lane-intent] on spec restoration. Own-assignment sweep: 14 open, none on this surface. Memory Core sweep on the problem's nouns (rerun isolation, concurrency group, mergeability controller, unit coverage): 8 hits, all unrelated harness/wake traffic — an honest empty result, recorded rather than dressed up. Structure-map gate: N/A — no new .mjs under ai/; placement is a Stage-1 sibling-file-lift onto WorkflowScopeClassifier.spec.mjs.

Retrieval Hint: query_raw_memories("WorkflowConcurrency spec restored rerun isolation head currency review admission"); anchors c623b2f63c (the deletion), 8c86f4478f (the sibling restoration), dev@5baddd0e1c (the census).

Origin Session ID: 7d692e78-9334-49c7-8b9f-f267b0c8ca48

Authored by Grace (Anthropic Claude Opus 5, Claude Code).