Context
e2e-engine (1/2) failed three times today with The action 'Run e2e (engine tier)' has timed out after 5 minutes, each time after its tests passed (119 passed, none failed):
Sweeps:
- Live latest-open sweep: the latest 20 open issues at 2026-09-23T01:05:37Z hold no equivalent.
- A2A in-flight sweep: no claim on this surface in the last 30 messages.
- Memory Core and own-assignment sweeps: the only prior art is #18895 / #18896. The third-shard change @neo-opus-ada measured on 09-21 rode #19045, which closed unmerged.
The Problem
#18896 split the tier into two shards on 09-18, each under the five-minute step budget. The tier has grown since.
Run e2e step durations, shard 1 / shard 2, in seconds, over the last ten runs that executed the tier:
| Run |
Shard 1 / shard 2 (s) |
dev push 633585423d |
312 / 254 |
| #19073, attempt 2 |
278 / 247 |
| (eight earlier runs) |
222/244 · 220/274 · 261/260 · 275/274 · 275/274 · 245/278 · 283/272 |
Most runs sit at 245–285 s, within 5–20 % of the ceiling, so runner variance alone crosses it. The red costs a runner and a rerun, and it teaches everyone that a red e2e check means "rerun". That habit is what #19039's instrument exists to break.
The Architectural Reality
.github/workflows/test-e2e.yml:9-12 states the policy: "A shard nearing it gets a sibling, never a higher ceiling: the ceiling is the wait every pull request pays.":82-83: the matrix is shard: [1, 2] / shards: [2] for pull_request and push. A dispatch stays one shard, so a probe keeps its whole denominator.:345-347: shard 1 alone emits the coverage summary, and the artifact name carries the shard index. Neither depends on the count.- No file outside the workflow names the
e2e-engine (n/2) checks (git grep over .github, buildScripts, ai), and dev has no required status checks. Renamed checks break no protection rule.
The Fix
test-e2e.yml:82-83: '[1, 2]' → '[1, 2, 3]' and '[2]' → '[3]'. At today's size (≈ 530 s of test time) three shards run ≈ 180 s each, about 40 % under the ceiling.
Acceptance Criteria
Out of Scope
- Raising the ceiling: the workflow header rules it out by design.
- Making individual specs faster.
Decision Record impact
None. Aligned with #18895's budget decision.
Related
#18895 · #18896 · #19073 · #19074 · #19039
Origin Session ID: 19051047-2f38-4bf0-be51-efd6b957564d
Retrieval Hint: "e2e engine tier shard five-minute ceiling timed out after tests passed"
Context
e2e-engine (1/2)failed three times today withThe action 'Run e2e (engine tier)' has timed out after 5 minutes, each time after its tests passed (119 passed, none failed):633585423d(35803426722, 312 s)Sweeps:
The Problem
#18896 split the tier into two shards on 09-18, each under the five-minute step budget. The tier has grown since.
Run e2estep durations, shard 1 / shard 2, in seconds, over the last ten runs that executed the tier:633585423dMost runs sit at 245–285 s, within 5–20 % of the ceiling, so runner variance alone crosses it. The red costs a runner and a rerun, and it teaches everyone that a red e2e check means "rerun". That habit is what #19039's instrument exists to break.
The Architectural Reality
.github/workflows/test-e2e.yml:9-12states the policy: "A shard nearing it gets a sibling, never a higher ceiling: the ceiling is the wait every pull request pays.":82-83: the matrix isshard: [1, 2]/shards: [2]forpull_requestandpush. A dispatch stays one shard, so a probe keeps its whole denominator.:345-347: shard 1 alone emits the coverage summary, and the artifact name carries the shard index. Neither depends on the count.e2e-engine (n/2)checks (git grepover.github,buildScripts,ai), anddevhas no required status checks. Renamed checks break no protection rule.The Fix
test-e2e.yml:82-83:'[1, 2]'→'[1, 2, 3]'and'[2]'→'[3]'. At today's size (≈ 530 s of test time) three shards run ≈ 180 s each, about 40 % under the ceiling.Acceptance Criteria
pull_requestrun shows threee2e-engine (n/3)jobs; a dispatch still runs one shard.Run e2estep finishes under 240 s (80 % of the ceiling), and the three tallies (passed + skipped + failed) sum to the population a two-shard run ondevreports.Out of Scope
Decision Record impact
None. Aligned with #18895's budget decision.
Related
#18895 · #18896 · #19073 · #19074 · #19039
Origin Session ID: 19051047-2f38-4bf0-be51-efd6b957564d
Retrieval Hint: "e2e engine tier shard five-minute ceiling timed out after tests passed"