Skip to content

fix(core): report a cancelled shell as interrupted, not completed - #15836

Merged
wenshao merged 13 commits into
QwenLM:mainfrom
2SinceJuly:fix/15833-shell-cancel-notice
Nov 7, 2026
Merged

wenshao merged 13 commits into
QwenLM:mainfrom
2SinceJuly:fix/15833-shell-cancel-notice

Conversation

@2SinceJuly

@2SinceJuly 2SinceJuly commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

Adds an explicit interruptedBeforeCompletion marker to a tool result and sets it when the shell tool is cancelled by the user. The scheduler's post-execute cancellation branch now chooses the cancellation notice from that marker instead of assuming that a returned result means the tool finished. The ACP session pipeline, which mirrors that branch for qwen --acp, the web shell and IDE clients, reads the same marker and keeps the producer's partial output as the detail.

Why it's needed

On Esc/Ctrl+C the shell tool kills the process group but returns an ordinary result whose own content already says the command was cancelled. The scheduler's aborted branch hardcoded "The tool had already completed; its output was discarded", which discards that truthful text and tells the model the work finished — and not to retry it. For half-run migrations or deploys, a step that was killed is recorded as done.

The shell tool is not the only possible producer of this shape: any tool that reports interruption by returning rather than throwing was mislabelled the same way, so the signal lives on the result rather than being inferred from shell's wording. Of the producers that now set it, only create_sub_session was shown reachable in measurement. monitor, workflow (early abort) and ask_user_question set it as defence: the scheduler cancels those calls before execute() runs, so the marker is not read on those paths today. ask_user_question sets it only when the turn was actually aborted — a plain decline is a completed decision and leaves the marker unset.

Reviewer Test Plan

How to verify

  1. Follow the reproduction in Cancelling a running shell command tells the model "The tool had already completed" although the command was killed #15833 (work.sh writing start, sleeping, then done), run it through the CLI and press Esc about 1s in.
  2. Ask the model whether the command finished.
    Before: the tool result says the tool had already completed and that the output was discarded. After: it says the call was intentionally cancelled and that it must not retry or work around it.
    Timeouts and background-promote refusals keep their existing outcomes; only a user cancel sets the marker on the shell path.

Evidence (Before & After)

Non-UI change, so N/A for screenshots. The new scheduler table case fails on the previous code with The tool had already completed; its output was discarded and passes here; the shell test asserts the marker is set only on a user cancel; the question-tool tests assert it is set only when the turn was aborted; and the new ACP case asserts an interrupted result reaches the model as interrupted, with its partial output appended, while the unflagged case keeps the old wording.

Tested on

OS Status
🍏 macOS ✅
🪟 Windows ⚠️
🐧 Linux ✅

Unit tests on macOS and in a node:22-bookworm container; Windows not exercised.

Environment (optional)

cd packages/core && npx vitest run src/core/coreToolScheduler.test.ts src/tools/shell.test.ts src/tools/askUserQuestion.test.ts (942 tests); cd packages/cli && npx vitest run src/acp-integration/session/Session.test.ts.

Risk & Scope

  • Main risk or tradeoff: one optional field on the result type. It is written by the shell tool and three returned-cancellation producers (only create_sub_session was reachable in measurement), and read only by the cancelled branch.
  • Not validated / out of scope: monitor, workflow (early abort) and ask_user_question set the marker defensively; the scheduler cancels them before execution, so no model-visible change is claimed for those paths. A call cancelled before it starts ("cancelled before it could start") is not covered. A cancel that lands while the shell's confirmation dialog is still open (approval mode default) is also uncovered: it reports the generic cancelled reason, identically before and after this change. The managed-runtime surface still reports the old wording — its wrapper rebuilds a tool result from an explicit field whitelist that does not forward this marker — so this PR does not reach sessions whose tools run through that wrapper. Tracked as a follow-up rather than treated as fixed here. The sub-agent tool is also not covered: it returns its cancel as content parts and sets no marker, and the scheduler's detail gate requires a string llmContent, so setting the marker alone would not preserve its partial result. The ACP pipeline is covered by this PR.
  • Breaking changes / migration notes: none.

Linked Issues

Closes #15833

Thanks @stars-spark for the detailed reproduction.

The shell tool kills the process group on a user cancel but returns an
ordinary result, so the scheduler's post-execute cancellation branch labelled
the call "already completed" and told the model not to retry work that never
finished. Let producers mark that outcome on the result; the shell tool sets it
on a non-timeout, non-promote-refused abort.

Closes QwenLM#15833
@wenshao

wenshao commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

Verdict: findings — 10/10 scripted assertions passed and the central claim is proven load-bearing, but one user-visible information loss survives on the fixed path. Verified head 34293742a0bcf04f5760c71e9f1cab654cabd200 (PR base tip 3b1a266dc02838d6b479aa36bb6842fac6f82dc0; merge-base 12114094921f724d26488cf203e72d7f7cc9d38e).

中文摘要(点击展开)

结论:findings(可合并前建议先处理 1 条次要问题)。

  • A/B 成立:真实子进程 + 真实 ShellToolInvocation + 真实 CoreToolScheduler,中断落在命令运行中。PR 之后模型收到「User intentionally cancelled this tool call…do not retry or work around it」;基线上收到「The tool had already completed; its output was discarded」。见下方表格与截图 01-ab-cancel-notice-head-vs-base.png。
  • 没有扩大改动面:同一次运行里,正常完成、超时这两条对照路径逐字未变(表格 SAME? 列全为 yes)——PR 只改了「被用户中断」这一条分支。
  • 问题(Minor):修复前,取消时模型还能看到 shell 自己那段「Below is the output before it was cancelled: start」;修复后整段只剩一行停止指令,用户已经跑了的那部分输出(start)对模型彻底消失。基线同样不显示,所以这不是回归,但「别再告诉模型它做完了」和「顺手把它已有的部分输出也删了」是两件事,可以兼得。
  • 变异矩阵:两个守卫(调度器的消息选择、shell 的标记写入)都被测试真正钉住;但 !wasTimeout、!wasPromoteRefused、result.aborted 三个条件删掉后整套 915 个测试全绿——现有套件没有验证「超时/后台提升/正常完成时不打标记」,属覆盖缺口而非缺陷。
  • 未覆盖:default 审批模式整链、容器/SSH 执行环境、Windows、真实 TUI 中按 Esc。其中「取消时还在等权限确认框」单独测过,两臂逐字一致,PR 没碰它,也没修好它。

The central claim and the A/B

Claim under test: a shell killed by a user cancel must reach the model as interrupted mid-work, not as finished and discarded.

Both arms run the same harness (copied verbatim into each tree) against their own compiled packages/core/dist, in approval mode yolo: a real ShellToolInvocation spawns echo start; touch <marker>; sleep 30; echo done, the harness waits for the marker and then aborts the turn signal — the same abort cancelOngoingRequest() performs for Esc. The harness reads back responseParts, i.e. the exact wire text the model receives.

Cell Oracle Head (PR) Base (control) SAME?
A-status scheduler status of the cancelled call cancelled cancelled yes
A-notice which cancellation notice reaches the model cancelled-before-completion cancelled-after-completion NO
B-status shell that finished before any cancel success success yes
B-no-cancellation-notice B's model-facing text none none yes
B-pin-success-no-run-output B does not carry base's "partial output" text no-run-output no-run-output yes
C-status a timed-out shell error error yes
C-message-kind C's model-facing text timed-out timed-out yes
A-partial-output-visible (diagnostic) head's text still names what ran before the kill null null yes

Only the cancel notice moves; every neighbouring outcome is byte-identical between arms, which is the check that the PR did not relabel results generally.

A/B: cancelling a running shell — head says interrupted, base says completed

The two wire payloads, verbatim:

HEAD: [{"functionResponse":{"id":"A-user-cancel","name":"run_shell_command","response":{"error":"[Operation Cancelled] Reason: User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it."}}}]
BASE: [{"functionResponse":{"id":"A-user-cancel","name":"run_shell_command","response":{"error":"[Operation Cancelled] Reason: The tool had already completed; its output was discarded. User intentionally cancelled. Stop and await further instructions; do not retry."}}}]

Findings

1. Minor — the fix drops the partial output the model used to see, for the very case it fixes

Before the PR the model received the shell tool's own text, which names the work that already ran (before it was cancelled:\nstart), plus the "already completed" line that made it wrong. On head the whole payload is replaced by the one-line notice from createCancelledResponse, so the start is gone. This is not a regression — base is silent too on the scheduler path — but the two properties are separable: keeping the notice and the tool's own interrupted text fixes "it did not finish" without also erasing "here is what ran".

Boundaries measured on both arms: it is not a model-visible regression against base (identical loss), it is not a new code path (both go through createCancelledResponse), and nothing outside the cancelled branch changes (cells B, C identical). A maintainer may reasonably accept it as pre-existing; the point is that the PR is the first change that makes the "interrupted" wording truthful, and it does not carry the tool's account of the interruption along with it.

2. Suggestion — three conditions of the new marker are unpinned (coverage gap, not a defect)

Round 1, suites coreToolScheduler.test.ts + shell.test.ts (915 tests, baseline green 540 + 375):

Mutation Outcome Witness
scheduler: always pick the "already completed" notice killed the author's new scheduler case
scheduler: invert the marker check killed (2 tests) author's case + the pre-existing post-completion case
shell: drop the marker hunk killed the author's new shell test
shell: drop !wasTimeout survived —
shell: drop !wasPromoteRefused survived —
shell: drop result.aborted survived —
shell: wasTimeout → !wasTimeout killed author's new shell test (via its llmContent assertion)
Positive control: flip the new shell test's expected marker killed the author's new shell test
Positive control: flip the new scheduler case's expected notice survived explained: the case's other assertion (not.toContain('had already completed')) still holds, because the message it gets instead is the third table row's own text. The notice assertion is therefore not the case's only guard

The last row is why round 2 re-landed the controls inside shell.test.ts itself (both killed), and then added the three missing pins:

timeout test:   expect(result.interruptedBeforeCompletion).toBeUndefined();   -> 375/375 green on head
promote test:   expect(result.interruptedBeforeCompletion).toBeUndefined();   -> 375/375 green on head
success test:   expect(result.interruptedBeforeCompletion).toBeUndefined();   -> 375/375 green on head

Green on head means the pins are correct but also that the current suite cannot tell you if any of the three conditions were deleted. The suggested addition is the first two assertions, plus a matching delete-condition mutant as the test's own witness.

Mutation matrix: cancel notice pinned; timeout, promote and aborted conditions survive

3. Suggestion — the sibling shape the description defers

The description lists "a call cancelled before it starts" as out of scope. The adjacent shape — the cancel landing while the shell's confirmation dialog is still open in approval mode default — was measured on both arms: dialog seen true, status cancelled, and the model text is [Operation Cancelled] Reason: Tool call cancelled by user. identically on head and base. The PR neither breaks nor fixes it; it simply does not reach that path. Worth a line in the description so the next reader does not assume the cancel notice is now uniform.

Not covered

  • Approval mode default, end to end. Only the pending-dialog cell above ran there; the full dialog → approve → running → cancel chain was not driven.
  • Container and SSH execution environments. SshExecutionEnvironment.execute() rebuilds the shell result itself (Exit code: …) and never sets the marker, so interruptedBeforeCompletion cannot appear on that path. Not measured — inferred from code.
  • Windows, and a real TUI Esc. Esc → cancelOngoingRequest() → abortController.abort() is read from AppContainer.tsx / use-llm-stream.ts, not driven through a terminal.
  • CI on the PR head. Checks were still pending when the round started; no red job was available to reproduce.
  • Per-commit attribution. The PR is one commit on top of the base tip, so the aggregate diff is the commit.

Methodology

Two worktrees (head at the PR head, base at the base tip), each installed with scripts/setup-worktree.js and built with npm run build; the @qwen-code/* realpath assertion confirms each tree resolves its own packages/core, and the head dist carries interruptedBeforeCompletion (2 + 1 occurrences) while the base dist carries none — the positive control for the build. Harnesses live in arms/ and import the arm's compiled dist. Both arms' npm run typecheck exit 0, and the gate was proven live by appending a type error to shell.ts (2 error TS lines, exit 2, tree restored to clean). A trial merge into origin/main is conflict-free (git merge-tree → tree 1b552de8958767e4509bfb3c463b4cf2bbecbcc5) and main changed none of the four files this PR touches since the merge-base. For the record: the first mutation run reported every row as invalid because vitest's ANSI-coloured summary lines defeated the ^\s*Tests match; the runner now strips ANSI and sets NO_COLOR, and the tree was restored before rerunning.

The scheduler replaced the whole model-visible payload with the cancellation
notice, so the output a killed command had already produced disappeared even
though the shell tool reports it. Append the producer's own interrupted text
when it sets interruptedBeforeCompletion, and pin the marker's negative cases
that the existing suites did not distinguish.
@2SinceJuly

Copy link
Copy Markdown
Contributor Author

All three findings are addressed in 6e6600e (pushed on top of 34293742, no rebase).

1 (Minor — the fix drops the output the model used to see). Taking it; the two properties are separable and now both hold. createCancelledResponse() takes the producer's own interrupted text and appends it to the notice when interruptedBeforeCompletion is set, so the wire payload is the stop directive plus what already ran:

[Operation Cancelled] Reason: User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it.

Command was cancelled by user before it could complete. Below is the output before it was cancelled:
start

Nothing changes for a post-completion cancel — that path still discards the output and keeps only the "already completed" notice. The shell test now pins the partial-output sentence too, so a future change that drops it reds the producer's own case.

2 (Suggestion — the three conditions were unpinned). Added the assertions you specified: expect(result.interruptedBeforeCompletion).toBeUndefined() in the success, timeout and promote-refused cases in shell.test.ts. Dropping !wasTimeout, !wasPromoteRefused or result.aborted now turns one of them red, so each condition has its own witness.

3 (Suggestion — the sibling shape). Added a line to Not validated / out of scope: a cancel that lands while the shell's confirmation dialog is open (approval mode default) reports the generic cancelled reason, identically before and after this change.

cd packages/core && npx vitest run src/core/coreToolScheduler.test.ts src/tools/shell.test.ts → 917 passed (was 915).

@qwen-code-review-bot

qwen-code-review-bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Qwen Triage finished — view run. See the stage comments in this thread for the result.

✅ Qwen Triage 已完成 —— 查看运行。结果见本线程中的各阶段评论。

@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR — and for picking this up after offering to in the issue.

Template looks good ✓. Every required section is filled in, including a real Tested-on table and an honest Risk & Scope. One small ask, not a gate: the template's trailing 中文说明 <details> block is missing, so the body is English-only.

Problem is an observed bug, and about as well-evidenced as these get. #15833 is open, carries a deterministic reproduction (work.sh → start, sleep 3, done, Esc at ~1s, log.txt left with only start), a real-model confirmation that the model then answers "yes, it completed", 47/48 scripted PTY runs across single/parallel calls and Esc/Ctrl+C, and it names the exact root-cause line. Nothing theoretical here.

Direction is aligned, and the codebase agrees with it independently. TOOL_CANCELLED_BEFORE_COMPLETION_MESSAGE already exists, and the comment above the two constants already states the invariant this PR restores — "Both sites that can cancel after execute() was entered must pick the right one." The throw path picks correctly via executionThrew; the return path hardcoded after completion. So this closes a gap the file had already declared, rather than introducing a new idea.

Size: core paths are touched (packages/core/src/core/, packages/core/src/tools/), so the breakdown is 33 production lines (coreToolScheduler.ts 21, tools.ts 7, shell.ts 5), 67 test lines, 0 generated/schema. Well under every threshold — no maintainer-awareness escalation on size, no split advisory.

Approach is close to minimal and mirrors the existing pattern rather than inventing one. One genuine question for the maintainer, not a blocker: the 6e6600e follow-up added a second property — re-attaching the producer's own partial output to the cancellation notice. That half goes beyond "stop calling a killed command completed", and it re-includes text the cancellation path deliberately drops elsewhere. It came out of the earlier /verify finding and the two properties are separable, so it reads as review-driven rather than scope creep — but it does widen the behavioural surface, and it deserves an explicit yes rather than riding along.

Risk: Stage 1e matched — packages/core/src/tools/shell.ts is on the revert-correlated high-risk path list. That does not block anything; it raises review depth, so the code review below keeps both enrichments and treats CI evidence as a precondition for approval rather than a nicety. The author has read permission only, so @qwen-code /tmux is unavailable and any @qwen-code /verify would be a maintainer-sponsored run.

Moving on to code review. 🔍

中文说明

感谢贡献,也感谢你在 issue 里主动提出可以发 PR 之后真的把它做了。

模板完整 ✓。每个必填小节都写了,包括真实的 Tested-on 表格和坦诚的 Risk & Scope。一个小小的请求(不是拦截项):模板末尾的 中文说明 <details> 块缺失,正文目前是纯英文。

问题是已观测到的 bug,证据非常充分。#15833 处于 open 状态,带有确定性复现(work.sh → start、sleep 3、done,约 1 秒时按 Esc,log.txt 只留下 start)、真实模型的确认(随后模型回答"是的,已完成")、跨单次/并行调用与 Esc/Ctrl+C 的 47/48 次脚本化 PTY 运行,并且直接点名了根因所在的那一行。这里没有任何理论性推测。

方向是对齐的,而且代码库本身就独立支持这个方向。TOOL_CANCELLED_BEFORE_COMPLETION_MESSAGE 早已存在,两个常量上方的注释也已经写明了本 PR 要恢复的不变量——"Both sites that can cancel after execute() was entered must pick the right one."(在 execute() 已进入之后能够取消的两处,都必须选对消息)。抛异常那条路径通过 executionThrew 选对了;返回结果那条路径把"已完成"写死了。所以这是在补上文件自己早已声明的缺口,而不是引入新概念。

规模:触及了核心路径(packages/core/src/core/、packages/core/src/tools/),因此拆分为生产代码 33 行(coreToolScheduler.ts 21、tools.ts 7、shell.ts 5)、测试 67 行、生成/schema 0 行。远低于所有阈值——不因规模触发维护者关注,也不建议拆分。

方案接近最小改动,并且是照搬既有模式而非另创一套。有一个真正需要维护者判断的问题(不是拦截项):6e6600e 这次跟进加了第二个性质——把生产者自己的部分输出重新附加到取消通知上。这一半超出了"别再把被杀掉的命令说成已完成"的范围,它把取消路径在别处刻意丢弃的文本又放了回来。它源自上一轮 /verify 的发现,而且这两个性质是可以分开的,所以看起来是评审驱动而非范围蔓延——但它确实扩大了行为面,值得一个明确的"同意",而不是顺带被带过去。

风险:Stage 1e 命中——packages/core/src/tools/shell.ts 在与 revert 相关的高风险路径清单上。这不拦截任何东西,只是提高评审深度,所以下面的代码审查保留两项增强内容,并把 CI 证据当作批准的前置条件而非可选项。作者仅有 read 权限,因此 @qwen-code /tmux 不可用,任何 @qwen-code /verify 都将是维护者赞助的运行。

进入代码审查 🔍

— Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 6e6600ed9188eb68b274f0e0d72e6d2c56cb9412 · re-run with @qwen-code /triage

@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Code review

Independent proposal first. Reading only the title and the motivation, I'd have ruled out the two obvious alternatives: making the shell tool throw on user abort (it changes the error type, the telemetry span status and the whole response shape, and the issue is explicit that returning is deliberate), and string-matching the shell's own wording in the scheduler (brittle, and exactly the coupling the description says it wants to avoid). That leaves an explicit producer-set signal on the result, written by the tool that knows and read only by the branch that has to decide. That is what this PR does, so it matches my baseline rather than merely being defensible.

The marker fires exactly when the tool already says cancelled. This is the part I most wanted to check, and it holds: result.aborted && !wasTimeout && !wasPromoteRefused is the same predicate the existing outcome derivation a few lines above already uses — it tests wasTimeout first, then result.aborted && !wasPromoteRefused. So the new field cannot drift out of agreement with returnDisplay.outcome, and no second classification was invented. The three conditions each have a witness now (toBeUndefined() on success, timeout and promote-refused), which is what the earlier mutation matrix asked for.

Not a dead switch. Written in shell.ts, read in the scheduler's aborted branch. Both sites verified, and the read is the one that selects the message.

The new parameter is placed correctly. createCancelledResponse takes detail last, after visionBridgeNotice; the call passes persistedOutputFiles, an explicit undefined, then the gated detail. The other six call sites are untouched, since the parameter is optional and trailing. The gate (typeof llmContent === 'string' && interruptedBeforeCompletion) keeps the post-completion path discarding output exactly as before, so cells B and C of the earlier A/B should stay byte-identical.

One gap worth naming — non-blocking. The signal does not survive the managed-runtime wrapper. MANAGED_RUNTIME_TOOL_NAMES includes ToolNames.SHELL, and registerManagedRuntimeTools wraps the ShellTool from createExecutionTools in ExecutionTool. That wrapper's execute() rebuilds the result from an explicit whitelist — llmContent, returnDisplay, outputBudgetApplied, error, rawOutputSize, truncatedByProducer, persistedOutputFiles, resultFilePaths — and interruptedBeforeCompletion is not in it. In a managed-runtime session a cancelled shell would therefore still report "had already completed". This is not a regression (that surface behaves as it does today) and it is outside #15833's reproduction, but it is the same bug on a second surface, and the whitelist already forwards sibling metadata flags, so it is a small addition rather than a redesign. Fine as a follow-up; it just should not be silently assumed fixed.

sequenceDiagram
    participant P1 as User (Esc or Ctrl+C)
    participant P2 as CoreToolScheduler
    participant P3 as ShellToolInvocation
    participant P4 as ShellExecutionService
    P1->>P2: abort the turn signal
    P2->>P3: execute(signal) already in flight
    P3->>P4: kill the process group
    P4-->>P3: result.aborted true, plus partial output
    P3-->>P2: returns a ToolResult, does not throw
    Note over P2: signal.aborted and not a timeout, so the aborted branch runs
    P2->>P2: interruptedBeforeCompletion selects BEFORE instead of AFTER
    P2-->>P1: cancelled, notice plus what already ran
Loading
Files changed (5 of 5 shown)
File What changed
packages/core/src/tools/tools.ts Declares the optional interruptedBeforeCompletion flag on ToolResult, documented as producer-set for tools that report interruption by returning.
packages/core/src/tools/shell.ts Sets the flag on the returned result, guarded by the same three conditions the existing outcome: 'cancelled' derivation uses.
packages/core/src/core/coreToolScheduler.ts Reads the flag to pick the cancellation notice, and threads the producer's own interrupted text into the notice via a new trailing parameter.
packages/core/src/tools/shell.test.ts Pins the flag on a user cancel, and pins its absence on success, timeout and promote-refused.
packages/core/src/core/coreToolScheduler.test.ts Adds a table row for the interrupted notice and a case asserting the partial output survives alongside it.

Test evidence — the PR's own CI, read via the API

I did not build or run anything from this PR; the review is static. This is what CI actually recorded on the reviewed commit 6e6600ed9188eb68b274f0e0d72e6d2c56cb9412.

The headline is that there is no test result on this commit. The Qwen Code CI run (40848762941) concluded cancelled. Reading the job's step list rather than guessing at the cause: Test (ubuntu-latest, Node 22.x) reached Install dependencies: cancelled, and Run tests and generate reports is skipped. Checkout, head-commit verification, Node setup and cache restore all succeeded first. So the suite never started. The run was created 09:19:43Z, the job started 14:06:18Z and was cancelled 14:10:46Z — roughly 4h47m queued, then killed during install. Integration Tests, web-shell E2E Smoke, OpenTUI no-flicker gate and tui-parity cancelled with it; the macOS and Windows Test jobs are skipped.

I am calling that infrastructure, not the PR: nothing in the step list reached code the diff touches, and a cancelled install is not a failure signal about these five files. But the honest consequence is that zero unit-test evidence exists for the reviewed head, and there are no pull_request workflow runs still in flight on it, so nothing is going to turn this green on its own — it needs a re-run.

What did complete is green: Lint & Static (ubuntu-latest, Node 22.x), Managed Agent gate, Verify Landlock helper, Native Linux CLI boundary, Native Windows settings and streams, Check TUI impact, Classify PR, and the arm64/x64 builds. Lint passing on this diff is real signal for the type-level change and the trailing-parameter addition; it says nothing about the runtime behaviour.

Check Conclusion
Test (ubuntu-latest, Node 22.x) cancelled
Test (macos-latest, Node 22.x) skipped
Test (windows-latest, Node 22.x) skipped
Integration Tests (no-AK, No Sandbox) cancelled
Integration Tests (CLI, No Sandbox) skipped
web-shell E2E Smoke (ubuntu-latest, Node 22.x) cancelled
OpenTUI no-flicker gate cancelled
TUI parity snapshots (ink vs opentui) cancelled
Lint & Static (ubuntu-latest, Node 22.x) success
Managed Agent gate success
Verify Landlock helper success
Native Linux CLI boundary success
Native Windows settings and streams success
Check TUI impact success
Classify PR success
arm64 success
x64 success

On the earlier A/B run, and precisely how far it reaches. @wenshao's /verify report on this PR is strong evidence and I am relying on it: a real subprocess, a real ShellToolInvocation, a real CoreToolScheduler, the abort delivered the same way cancelOngoingRequest() delivers it for Esc, and the oracle read back from responseParts — the actual wire text. It showed the notice flipping from after completion to before completion, with every neighbouring outcome byte-identical, and a mutation matrix proving both guards are pinned.

Two limits matter for the current head, and I would rather state them than let the green verdict carry over. First, it verified 34293742a0bcf04f5760c71e9f1cab654cabd200, the commit before this one; 6e6600e landed afterwards. Second, and more specifically, its A-partial-output-visible row read null on head — i.e. it measured the absence of partial output. The detail change added in 6e6600e is exactly the delta that run did not cover, so the newest quarter of this diff has no A/B behind it.

The author reports 917 passed for coreToolScheduler.test.ts + shell.test.ts (up from 915). That is the author's own claim on their own machine, not evidence I re-ran or can vouch for — and per the rules above I did not execute anything from this PR to check it.

Not verified: the unit and integration suites on the reviewed head (CI cancelled before the test step); the new partial-output path end-to-end on 6e6600e (the A/B predates it); the managed-runtime surface (flag dropped by the wrapper, see above); the confirmation-dialog cancel path in approval mode default (author documents it as unchanged and out of scope); Windows (author marked it ⚠️ not tested).

Sandboxed verification would settle this: @qwen-code /verify — specifically that the partial-output half added in 6e6600e holds, since the prior A/B ran on 34293742 and recorded partial output as absent on head, and this PR's own suite never ran at all on the current commit. Because the author has read permission only, this would be a sponsored run: a maintainer's @qwen-code /verify comment approves the head it was written against, and that run carries a pre-execution risk screen plus a full workspace wipe before any PR code runs. Please read the resulting report with the same skepticism as the fork's own CI logs — the code under verification is adversarial input, so a crafted PR can shape what a report says even though the sandbox bounds what it can do. @qwen-code /tmux is not available here, since it executes the author's code and gates on the author.

A plain CI re-run on this head is the cheaper first step and would close the largest gap on its own.

Real-scenario tmux testing: N/A — this is an unattended CI run (GITHUB_EVENT_NAME=pull_request_target), where the review is static and PR code is never executed.

中文说明

代码审查

先给独立方案。 只看标题和动机时,我会先排除两个显而易见的替代做法:让 shell 工具在用户取消时抛异常(这会改变错误类型、telemetry span 状态和整个响应结构,而且 issue 明确说明"返回"是有意为之),以及在调度器里去匹配 shell 自己的措辞(脆弱,而且正是描述中想避免的耦合)。剩下的就是一个由生产者显式设置在结果上的信号——由知情的工具写入,只由需要判断的那个分支读取。这正是本 PR 的做法,所以它符合我的基线方案,而不仅仅是"说得过去"。

标记恰好在工具已经报告 cancelled 时触发。 这是我最想核实的一点,结论成立:result.aborted && !wasTimeout && !wasPromoteRefused 与其上方几行既有的 outcome 推导使用的是同一个判定式——先判 wasTimeout,再判 result.aborted && !wasPromoteRefused。因此新字段不可能与 returnDisplay.outcome 失去一致,也没有发明第二套分类逻辑。三个条件现在各自都有见证者(成功、超时、promote 被拒三种情况下断言 toBeUndefined()),这正是上一轮变异矩阵所要求的。

不是死开关。 在 shell.ts 中写入,在调度器的 aborted 分支中读取。两处都已核实,而且读取点正是选择消息的那一处。

新参数位置正确。 createCancelledResponse 把 detail 放在最后、在 visionBridgeNotice 之后;调用处传入 persistedOutputFiles、一个显式的 undefined,然后是带条件的 detail。另外六个调用点未受影响,因为该参数是可选且位于末尾的。这个条件门(typeof llmContent === 'string' && interruptedBeforeCompletion)使"已完成后取消"那条路径仍然像以前一样丢弃输出,所以此前 A/B 中的 B、C 两格应当保持逐字一致。

一个值得点名的缺口——非拦截项。 该信号无法穿过 managed-runtime 包装层。MANAGED_RUNTIME_TOOL_NAMES 包含 ToolNames.SHELL,而 registerManagedRuntimeTools 会把 createExecutionTools 提供的 ShellTool 包装进 ExecutionTool。该包装层的 execute() 用一份显式白名单重建结果——llmContent、returnDisplay、outputBudgetApplied、error、rawOutputSize、truncatedByProducer、persistedOutputFiles、resultFilePaths——其中没有 interruptedBeforeCompletion。因此在 managed-runtime 会话里,被取消的 shell 仍会报告"had already completed"。这不是回归(该 surface 的行为与今天一致),也超出 #15833 的复现范围,但它是同一个 bug 出现在第二个 surface 上;而且白名单本来就转发同类的元数据标记,所以这是个很小的补充,而非重新设计。作为后续跟进完全可以,只是不应被默默当作已修复。

测试证据——PR 自己的 CI,通过 API 读取

我没有构建或运行本 PR 的任何内容,审查是静态的。以下是 CI 在被审查提交 6e6600ed9188eb68b274f0e0d72e6d2c56cb9412 上实际记录到的情况。

结论是:这个提交上没有任何测试结果。 Qwen Code CI 运行(40848762941)的结论是 cancelled。我读的是作业的步骤列表而不是猜测原因:Test (ubuntu-latest, Node 22.x) 走到 Install dependencies: cancelled,而 Run tests and generate reports 是 skipped。此前 checkout、head 提交校验、Node 安装和缓存恢复都成功了。也就是说测试套件从未启动。该运行创建于 09:19:43Z,作业于 14:06:18Z 开始、14:10:46Z 被取消——排队约 4 小时 47 分,然后在安装依赖期间被杀掉。Integration Tests、web-shell E2E Smoke、OpenTUI no-flicker gate 和 tui-parity 随之取消;macOS 与 Windows 的 Test 作业为 skipped。

我判定这是基础设施问题,而非 PR 造成:步骤列表中没有任何一步触及 diff 所改的代码,而安装阶段被取消并不是关于这五个文件的失败信号。但诚实的后果是:被审查的 head 上不存在任何单元测试证据,而且该提交上没有在飞的 pull_request 工作流运行,所以它不会自己变绿——需要重跑。

已完成的检查都是绿的:Lint & Static (ubuntu-latest, Node 22.x)、Managed Agent gate、Verify Landlock helper、Native Linux CLI boundary、Native Windows settings and streams、Check TUI impact、Classify PR,以及 arm64/x64 构建。Lint 在这个 diff 上通过,对类型层面的改动和末尾参数的新增是真实信号;但它对运行时行为什么也说明不了。

关于此前那次 A/B 运行,以及它究竟覆盖到哪里。 @wenshao 在本 PR 上的 /verify 报告是很有力的证据,我在依赖它:真实子进程、真实 ShellToolInvocation、真实 CoreToolScheduler,中断以 cancelOngoingRequest() 为 Esc 递送的同一方式递送,判定依据是从 responseParts 读回的——也就是真正的上行文本。它显示通知从"已完成后取消"翻转为"完成前取消",且所有相邻结果逐字一致,变异矩阵也证明两个守卫都被钉住。

对当前 head 而言有两条限制很关键,我宁愿说出来,也不愿让那个绿色结论被顺延过来。第一,它验证的是 34293742a0bcf04f5760c71e9f1cab654cabd200,即当前提交之前的那个;6e6600e 是之后才推上来的。第二,更具体地说,它的 A-partial-output-visible 一行在 head 上读数为 null——也就是它测量到的是部分输出的缺失。6e6600e 新增的 detail 改动恰恰是那次运行没有覆盖的增量,所以这个 diff 中最新的那四分之一没有任何 A/B 支撑。

作者报告 coreToolScheduler.test.ts + shell.test.ts 为 917 passed(此前 915)。那是作者在自己机器上的自述,不是我复跑过、也不是我能担保的证据——而且按上述规则,我没有执行本 PR 的任何内容去核对它。

未验证:被审查 head 上的单元测试与集成测试套件(CI 在测试步骤之前就被取消);6e6600e 上新的部分输出路径的端到端行为(A/B 早于它);managed-runtime surface(标记被包装层丢弃,见上);approval mode default 下确认对话框期间的取消路径(作者说明其未变且属范围之外);Windows(作者标记为 ⚠️ 未测试)。

沙箱验证可以定案: @qwen-code /verify——具体用于确认 6e6600e 新增的部分输出这一半是否成立,因为此前的 A/B 跑在 34293742 上、并且在 head 上把部分输出记为缺失,而本 PR 自己的套件在当前提交上根本没跑过。由于作者仅有 read 权限,这将是一次赞助运行:维护者的 @qwen-code /verify 评论会批准它所针对的那个 head,并且该运行在任何 PR 代码执行前带有执行前风险筛查与完整工作区擦除。请以与看待 fork 自身 CI 日志同样的怀疑态度阅读其产出的报告——被验证的代码是对抗性输入,因此精心构造的 PR 可以影响报告说什么,尽管沙箱限定了它能做什么。此处 @qwen-code /tmux 不可用,因为它会执行作者的代码并以作者为门禁。

在当前 head 上单纯重跑一次 CI 是更省事的第一步,并且它自己就能补上最大的那个缺口。

真实场景 tmux 测试:N/A——这是一次无人值守的 CI 运行(GITHUB_EVENT_NAME=pull_request_target),审查是静态的,绝不执行 PR 代码。

— Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 6e6600ed9188eb68b274f0e0d72e6d2c56cb9412 · re-run with @qwen-code /triage

@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Confidence: 3/5 — the code itself I would defend; what caps this is evidence, not design: the reviewed head has no test run at all, and it moved past the commit the A/B validated.

⏸️ Deferring to @yiliang114 (already the assignee, and the core-runtime owner for exactly packages/core/src/core/ and packages/core/src/tools/). Not requesting changes — I found no blocking defect. Not approving either, for the three reasons below.

Stepping back. The problem is real and it is the kind that matters: telling a model that a half-run migration "had already completed; do not retry" converts a recoverable interruption into a false belief that the work is done. The reproduction in #15833 is unusually good, and the fix is 33 production lines that mirror a distinction the file already makes on the throw path. Going back to my own independent proposal from the review stage, I don't have a simpler one — making the shell throw would be worse, and matching its wording in the scheduler would be worse still. If I inherited this in six months I would thank whoever wrote it: the flag is documented at both ends, and the guard reuses a predicate that already existed rather than adding a second source of truth.

So why not approve. Three things, in order of how much they weigh:

  1. There is no test evidence on the commit I reviewed. The Qwen Code CI run for 6e6600e was cancelled during Install dependencies; Run tests and generate reports is skipped. I read that as queue/infra rather than anything the diff caused, and lint plus the type-level checks are green. But Stage 1e put shell.ts on the high-risk path list, which makes CI evidence a precondition for approval here rather than a courtesy — and right now the suite has simply not executed against these five files. No pull_request run is in flight, so this will not resolve itself.
  2. The strongest evidence on this PR is one commit stale, and stale in the worst direction. @wenshao's /verify A/B is genuinely convincing about the notice flip — real subprocess, real scheduler, oracle read off the wire. But it ran on 34293742, and its A-partial-output-visible row recorded null on head. The detail change in 6e6600e adds precisely the thing that run measured as absent. Approving on the strength of that report would be approving a quarter of the diff that nothing has exercised. The author's 917 passed is their own claim; I did not run it and cannot vouch for it.
  3. The fix does not reach a second surface. ExecutionToolInvocation.execute() rebuilds its result from an explicit whitelist that omits interruptedBeforeCompletion, and MANAGED_RUNTIME_TOOL_NAMES includes shell — so in managed-runtime sessions the original wording survives. Not a regression, and arguably out of scope, but it should be a decision rather than an omission.

The trap here is that this PR is easy to wave through: excellent issue, clean diff, a strong verification report already attached. That report is exactly why the staleness matters — it is good enough that a reader will assume it covers the current head. It does not.

What would move me to approve:

  • A CI re-run on 6e6600e with Test (ubuntu-latest, Node 22.x) actually reaching its test step and landing green. On its own this closes the largest gap.
  • An explicit maintainer yes-or-no on the partial-output half. It is separable from the notice fix, it re-includes text the cancellation path drops elsewhere, and it is the part with no A/B behind it. Landing just the notice fix is a complete, well-evidenced PR on its own.
  • Either the whitelist addition in execution-tool.ts (small, and consistent with the sibling flags already forwarded there) or a line in Risk & Scope recording managed-runtime as a known follow-up.

@2SinceJuly — none of this is a criticism of the change; items 1 and 3 are largely not yours to fix. Item 1 needs a maintainer to re-trigger CI on a fork PR.

中文说明

信心度:3/5 —— 代码本身我愿意为之辩护;把它压到 3 分的是证据,不是设计:被审查的 head 完全没有测试运行记录,而且它已经越过了 A/B 所验证的那个提交。

⏸️ 转交 @yiliang114(其已是本 PR 的 assignee,并且正是 packages/core/src/core/ 与 packages/core/src/tools/ 所属 core-runtime 区域的负责人)。不提交 request changes——我没有发现拦截性缺陷。也不批准,原因见下面三条。

退一步看整体。问题是真实的,而且是要紧的那一类:告诉模型一个只跑了一半的迁移"had already completed; do not retry",会把一次可恢复的中断变成一个"工作已完成"的错误信念。#15833 的复现质量相当高,而修复只有 33 行生产代码,并且沿用了该文件在抛异常路径上早已做出的区分。回到我在审查阶段写下的独立方案,我没有更简单的做法——让 shell 抛异常更糟,在调度器里匹配它的措辞更糟。如果六个月后由我接手,我会感谢写下它的人:这个标记在两端都有文档说明,而且守卫复用了既有判定式,没有新增第二个事实来源。

那为什么不批准。三件事,按权重排序:

  1. 我审查的这个提交上没有任何测试证据。 6e6600e 的 Qwen Code CI 运行在 Install dependencies 阶段被取消;Run tests and generate reports 是 skipped。我判断这是排队/基础设施问题,而非 diff 造成的,而且 lint 与类型层面的检查都是绿的。但 Stage 1e 把 shell.ts 列入了高风险路径清单,这使得 CI 证据在此处是批准的前置条件而非礼节——而现在测试套件根本没有针对这五个文件执行过。没有在飞的 pull_request 运行,所以它不会自行解决。
  2. 本 PR 上最有力的证据落后了一个提交,而且落后在最糟的方向上。 @wenshao 的 /verify A/B 对通知翻转的论证确实令人信服——真实子进程、真实调度器、判定依据取自上行报文。但它跑在 34293742 上,且其 A-partial-output-visible 一行在 head 上记录为 null。6e6600e 的 detail 改动新增的恰恰是那次运行测得为"缺失"的东西。若凭那份报告批准,就等于批准了 diff 中那四分之一从未被任何执行覆盖的部分。作者的 917 passed 是其自述;我没有运行,也无法担保。
  3. 修复没有覆盖到第二个 surface。 ExecutionToolInvocation.execute() 用一份显式白名单重建其结果,而该白名单不含 interruptedBeforeCompletion,同时 MANAGED_RUNTIME_TOOL_NAMES 包含 shell——因此在 managed-runtime 会话里,原来的措辞仍然存在。这不是回归,也可以论证为范围之外,但它应当是一个决定,而不是一处遗漏。

此处的陷阱在于:这个 PR 很容易被挥手放行——issue 优秀、diff 干净、还已经附着一份有力的验证报告。而恰恰因为那份报告足够好,读者会假定它覆盖了当前 head。它没有。

能让我转向批准的条件:

  • 在 6e6600e 上重跑 CI,且 Test (ubuntu-latest, Node 22.x) 真正走到测试步骤并变绿。仅这一条就能补上最大的缺口。
  • 维护者对"部分输出"这一半给出明确的是或否。它与通知修复是可分离的,它把取消路径在别处会丢弃的文本放了回来,而且它正是没有 A/B 支撑的那部分。只落通知修复,本身就是一个完整且证据充分的 PR。
  • 要么在 execution-tool.ts 里补上白名单(改动很小,且与该处已转发的同类标记一致),要么在 Risk & Scope 中加一行,把 managed-runtime 记为已知的后续项。

@2SinceJuly —— 这些都不是对改动本身的批评;第 1 条和第 3 条基本不该由你来解决。第 1 条需要维护者为 fork PR 重新触发 CI。

— Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 6e6600ed9188eb68b274f0e0d72e6d2c56cb9412 · re-run with @qwen-code /triage

@stars-spark

Copy link
Copy Markdown

Thanks for picking this up. I'm not working on a separate PR, so please go ahead.

On the evidence gap raised in the stage-3 review (no before/after for the partial-output half): I'm running your head 6e6600e through the harness behind the #15833 numbers, with released 0.26.2 as the baseline. The harness drives the real interactive TUI, sends Esc or Ctrl+C while the command runs, and reads the record of the killed call from the resumed model request. It reports which notice the model gets and whether the pre-kill output reaches it. I'll also add a small real-model check that asks whether the command's effect happened. I'll post the table here when it's done.

@stars-spark

Copy link
Copy Markdown

Results for head 6e6600e.

One correction to my earlier note: the baseline was the merge-base 12114094 built from source, not released 0.26.2. The two builds therefore differ only in this PR's five files. Both were built with Node 22.23 and run on Linux.

Interactive TUI, scripted model. The harness pressed Esc or Ctrl+C while the command was running. Scenarios: single call, parallel calls, and a payment script that prints accepted before it is killed; 30 runs per build. The record of the killed call is read from the resumed model request.

killed call recorded as "had already completed" pre-kill output (accepted) reaches the model
base 30/30 0/6
head 0/30 6/6

Real model (Qwen3.8-27B on local llama.cpp). Same TUI. Esc was pressed about 1.5 s into a payment script. A fresh --continue turn then asked: "has payment #42 already been submitted? ANSWER: YES / NO / UNKNOWN". 10 runs per cell.

payment landed before the kill (truth: YES) payment never happened (truth: NO)
base UNKNOWN 10 UNKNOWN 10
head UNKNOWN 9, YES 1 NO 9, UNKNOWN 1

On this question the model did not take the old notice at face value. With the output discarded, it answered UNKNOWN every time. With the fix it gave a correct definite answer in 10 of 20 runs, and neither build produced a wrong answer. The partial-output half also did not make the model believe that a payment which had landed never happened (0 NO in 10). This differs from the manual repro in #15833: asked "did the command finish?", the model said yes on the old build.

Not covered: macOS and Windows, the managed-runtime path from the stage-3 review, and models other than this one.

@2SinceJuly

Copy link
Copy Markdown
Contributor Author

Thanks @stars-spark — this fills the evidence gap the stage-3 review flagged for the partial-output half.

The numbers line up with the intended behaviour:

  • head records no killed call as "had already completed" (0/30) where base did in 30/30;
  • pre-kill output reaches the resumed request in 6/6, 0/6 on base;
  • the real-model check produced no wrong "already submitted" answer on either build.

The environments you list as uncovered (macOS/Windows, the managed-runtime path, models other than Qwen3.8-27B) stay out of scope for this PR; the existing "Not validated / out of scope" section already says the managed path is untested, and I would rather leave them uncovered than claim coverage.

Re-requesting review on the current head 6e6600e.

@2SinceJuly

Copy link
Copy Markdown
Contributor Author

Thanks @stars-spark — the head-level table closes the gap the stage-3 review called out, and thanks for correcting the baseline to the merge-base build. With head at 0/30 wrong notices and 6/6 pre-kill output reaching the model, the partial-output half now has evidence on the commit under review rather than one behind.

Two things still lack coverage, and I want to be explicit rather than let the green table imply otherwise: managed-runtime, and Windows.

On managed-runtime — I did not take the whitelist route. Adding interruptedBeforeCompletion to ExecutionToolInvocation.execute()'s forwarded fields is a one-line change, but that wrapper is not the only conversion on that path: the managed session worker also rebuilds the tool result from its settled payload before the wrapper ever sees it, and that rebuild does not carry the marker either. A whitelist addition alone would therefore look like a fix without being one, and I have no end-to-end run that shows the marker and the partial output surviving persistence, the protocol, and the conversion chain. I have recorded the surface as a known follow-up in Risk & Scope instead of claiming it here. If a maintainer would rather have it in this PR, say so and I will do the full chain and the test with it.

On CI — the run for 6e6600e was cancelled during Install dependencies, so Run tests and generate reports never started and there is no pull_request run in flight for this head. Could a maintainer re-run CI on 6e6600e? Fork PRs cannot re-trigger it themselves.

One decision I should not make alone: whether keeping the partial pre-kill output inside this PR is wanted at all. It is separable from the notice fix — landing only the notice change is a complete, well-evidenced PR on its own — so I am happy to split it if that is the maintainer's preference.

@he-yufeng

Copy link
Copy Markdown
Contributor

Checked the fix against the other producers that signal interruption by returning, and there are three siblings the interruptedBeforeCompletion flag does not reach yet.

The scheduler's cancel path picks TOOL_CANCELLED_AFTER_COMPLETION_MESSAGE ("already completed") whenever execute() returned and the result did not set the new flag. That is right for the shell now, but these tools also return a result on cancel rather than throwing, so they would still be told the work finished when it did not:

  • monitor.ts — execute(signal) returns { llmContent: 'Monitor was cancelled before it could start.' } when the signal is already aborted. The monitor never ran, so "already completed" is wrong.
  • create-sub-session.ts — returns { llmContent: '...create_sub_session was cancelled...', returnDisplay: 'Cancelled' }.
  • askUserQuestion.ts — returns { llmContent: cancellationMessage, returnDisplay: cancellationMessage } when the question is cancelled.

Each of these is "interrupted before completion" in the same sense as the killed shell: the unit of work did not finish. So either they should set interruptedBeforeCompletion: true on their cancellation results the way shell.ts does, or the flag's default for a cancelled call should flip so a producer opts into "after completion" instead of into "before". The second is the safer default given how many tools return on cancel.

I have not changed anything here since this PR already owns the flag and the message split. Happy to send the sibling follow-up once this lands, or fold the three into this PR if you would rather cover the class in one go.

2SinceJuly and others added 2 commits October 31, 2026 05:09
The monitor, create_sub_session, ask_user_question and workflow tools
report a cancelled call by returning a result instead of throwing, so
the scheduler's returned-means-finished assumption told the model the
work had completed. They now set interruptedBeforeCompletion, which the
scheduler already honours for the shell tool.
@2SinceJuly

Copy link
Copy Markdown
Contributor Author

Good catch — all of them return on cancel, so the scheduler's returned-means-finished read was wrong for each. Fixed in 46db90b6, on top of 6e6600ed (no rebase):

  • monitor.ts — the early-abort result (never spawned).
  • create-sub-session.ts — both cancel outcomes (pre-start and mid-flight).
  • askUserQuestion.ts — the unanswered/declined result.
  • workflow/workflow.ts — startCancelledResult(), which has the same "nothing ran" shape for a pre-aborted background start.

I kept the flag opt-in instead of flipping the default. The scheduler only reads it when the turn's signal is aborted, so a plain dialog decline is unchanged, and the producers that genuinely finish despite a late abort (the shell's timeout / promote-refused paths now pin this) keep saying "already completed". Each producer has an assertion on the flag, and coreToolScheduler's cancel-notice tests still pass. I did not touch the sub-agent tool: its cancel result is a partial-output envelope with its own marker, so preserving that envelope on the abort path is a separate change.

Verification on 46db90b6: npx vitest run src/tools/monitor.test.ts src/tools/create-sub-session.test.ts src/tools/askUserQuestion.test.ts src/tools/workflow/workflow.test.ts src/core/coreToolScheduler.test.ts → 820 passed; npm run typecheck clean; prettier and eslint clean. The branch also merged current main (78 commits behind) so the lint gate stays fresh.

@wenshao

wenshao commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator

Verdict: merge-ready — 59/59 scripted assertions passed at head 46db90b6109ec2927d0591c88d0a112ceca803ce (PR base tip 1ccf0d43f35e51389573d87e5ae634c4d782f049). Both findings from the previous round are fixed and measured; two non-blocking notes remain.

中文摘要(点击展开)

结论:merge-ready(59/59 断言通过,无阻塞项)。

  • 上一轮 Finding 1(取消时丢掉已产生的输出)已修复:head 上模型收到「User intentionally cancelled…」+ shell 自己那段「Below is the output before it was cancelled: start」;base 上仍然是「The tool had already completed; its output was discarded」,start 丢失。见 A/B 表与截图 01-ab-shell-cancel-head-vs-base.png。
  • 上一轮 Finding 2(三个条件没有被测试钉住)已修复:result.aborted、!wasTimeout、!wasPromoteRefused 三个变异现在都被杀掉;createCancelledResponse 追加 detail 这一新守卫也有测试钉住。见 02-mutation-matrix-guards-vs-conditions.png。
  • 新增的 sibling 标记:create_sub_session 的中途取消在 head 翻转为「未完成 + 原消息」,base 是「已完成、输出被丢弃」——这是新 commit 里唯一被我驱动到、且模型可见行为发生翻转的 producer。monitor / workflow 的提前取消在两种臂上都是 scheduler 的「Tool call cancelled before execution.」;ask_user_question 的普通拒绝在两臂都是「User did not allow tool call」。也就是说这三个标记在当前可驱动路径上是防御性的,不改变模型看到的内容(非阻塞注记,见 Findings)。
  • 未覆盖:default 审批模式完整对话链(只测到 awaiting_approval)、容器/SSH 执行环境、Windows、真实 TUI Esc、workflow 后台启动中途取消、monitor 真实 spawn,以及 PR CI 的最终状态(轮中仍为 pending)。

Previous round's findings

# finding (round 1) status at 46db90b6
P1 Minor — the fix dropped the partial output the model used to see Fixed. Model now receives the notice and Below is the output before it was cancelled: start; base still discards it. Measured in cells A-partial-output-visible / A-detail-matches-tool.
P2 Suggestion — !wasTimeout, !wasPromoteRefused, result.aborted unpinned Fixed. All three mutants are now killed by the author's new assertions (SH-drop-aborted, SH-drop-timeout-guard, SH-drop-promote-guard).
P3 Suggestion — cancel while the confirmation dialog is open still says the generic reason Declined with rationale. The PR body now lists this shape as out of scope. Re-measured: both arms identical (awaiting_approval, no model text until the dialog resolves). Still not covered by this PR.

The central claim and the A/B

Claim under test: a shell killed by a user cancel must reach the model as interrupted mid-work, and the model must keep the output the command produced before the kill.

Both arms run the same harness against their own compiled packages/core/dist, in approval mode yolo: a real ShellToolInvocation spawns printf 'start\n'; touch <marker>; sleep 30; printf 'done\n'; the harness waits for the marker and then aborts the turn signal — the same abort cancelOngoingRequest() performs for Esc. The oracle is the exact functionResponse text the model receives.

Cell Oracle Head (PR) Base (control)
A-status scheduler status of the cancelled call cancelled cancelled
A-notice-before-completion which cancellation notice reaches the model User intentionally cancelled this tool call… The tool had already completed…
A-partial-output-visible the start produced before the kill survives yes no
A-detail-matches-tool appended detail equals the tool's own interrupted account yes no
A-no-after-completion no "had already completed" wording yes no
A-tool-marked-interrupted the producer sets the new marker true undefined
B-status / B-no-cancellation-notice a command that finishes normally success / no notice same
C-status / C-timeout-message a timed-out command error / timed out same

Head 11/11 cells pass; base fails exactly the five cancel cells and matches on every neighbouring outcome (B, C), which is the check that the PR did not relabel results generally.

A/B: a shell cancelled mid-command — head keeps the notice and the partial output, base discards both

The two wire payloads, verbatim:

HEAD: [{"functionResponse":{"id":"call-1","name":"run_shell_command","response":{"error":"[Operation Cancelled] Reason: User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it.\n\nCommand was cancelled by user before it could complete. Below is the output before it was cancelled:\nstart"}}}]
BASE: [{"functionResponse":{"id":"call-1","name":"run_shell_command","response":{"error":"[Operation Cancelled] Reason: The tool had already completed; its output was discarded. User intentionally cancelled. Stop and await further instructions; do not retry."}}}]

Sibling producers added in 97b8a0b

The second new commit marks four producers that report interruption by returning. I drove each real tool through the real scheduler and read back the model text on both arms:

Cell Head Base
create_sub_session cancelled mid-spawn User intentionally cancelled… + create_sub_session was cancelled. A sub-session may already have been created; it runs independently and is not cancelled. The tool had already completed; its output was discarded…
monitor — signal pre-aborted Tool call cancelled before execution. same
monitor — abort 1 macrotask / sync / within beforeToolLaunch Tool call cancelled before execution. same
monitor — abort at 0 ms Tool call cancelled by user. same
workflow (background) — pre-aborted / sync / microtask / beforeToolLaunch Tool call cancelled before execution. same
ask_user_question — user declines the dialog User did not allow tool call same
ask_user_question — turn aborted while the dialog is open awaiting_approval, no model text same

So exactly one of the four new markers changes what the model sees in a reachable flow.

Sibling producers: only create_sub_session's marker flips the model-facing message

Findings (non-blocking)

N1 — Suggestion: three of the four new markers are not observable in any path I could drive

monitor, workflow (background early abort) and ask_user_question set interruptedBeforeCompletion, but the scheduler cancels those calls before execute() runs, so the marker is never read and the model keeps the pre-existing pre-execution / user-denied wording. The commit message says "the scheduler's returned-means-finished read was wrong for each"; measured, only create_sub_session reaches that read.

This is not a defect — the markers are harmless defence if the pre-execution guard ever misses — but the claim should be bounded to the producer that is demonstrably reachable. Reproduce with harness/producers.mjs (cells MON-*, WF-*, ASK-*).

N2 — Suggestion: ask_user_question marks a plain decline as "interrupted"

execute() sets the marker whenever !wasAnswered, which includes the user simply declining the question (should return cancellation message when user declines pins exactly that). A decline is a completed decision, not interrupted work. It is inert today — the scheduler only reads the marker in its post-execute abort branch, and a decline takes the user-denied path — but a future read site would report a finished interaction as "do not retry". The code comment says "a plain decline leaves the scheduler on its normal path", which is true of the reader, not of the write.

Mutation matrix

Every row reverts one guard in the head tree, runs the named suite, and restores the tree with git checkout HEAD --; the runner refuses a dirty tree or a non-unique anchor. Head baseline: 917/917 across coreToolScheduler.test.ts + shell.test.ts.

Mutant Outcome Witness
scheduler: always pick the "already completed" notice killed author's returned-interruption case
scheduler: drop the appended detail killed keeps the partial output a cancelled producer reported
shell: drop result.aborted killed does not mark a completed command as interrupted
shell: drop !wasTimeout killed keeps the first timeout after a later user cancellation
shell: drop !wasPromoteRefused killed promote-refused race case
shell: marker on non-cancelled commands (positive control) killed marks a user-cancelled command as interrupted before completion
monitor: drop marker / change cancel text killed / killed does not spawn when the turn signal is already aborted
workflow: drop marker killed (3 tests) reports a registry-side cancel during background preflight as cancelled, not failed
create_sub_session: drop marker killed returns as soon as the turn is cancelled, without waiting for the spawn
ask_user_question: drop marker killed should return cancellation message when user declines
scheduler: flip the before-completion arm of the mid-flight test (mis-designed control) survived —

The last row is excluded from the tally and is not a coverage claim: that test feeds a thrown AbortError, which never reads the mutated variable, so the mutation could not affect it. The genuine positive controls are the two rows that mutate the same files and are killed (SCH-always-after-notice, SH-control-cancel-marker-off).

Mutation matrix: every real guard killed; the one survivor is a mis-designed control

Not covered

  • Approval mode default, full chain. Only a turn abort while the question dialog is open was driven (identical on both arms, awaiting_approval, no model text). The dialog → approve → running → cancel chain was not.
  • Container and SSH execution environments. SshExecutionEnvironment.execute() rebuilds the result itself and never sets the marker — inferred from code, not measured.
  • Windows, and a real TUI Esc. Esc → cancelOngoingRequest() → abort() is read from code, not driven through a terminal.
  • Workflow cancelled after its background start begins. My registry stub only implements hasCompletionCallback / get / list, so reserveStart threw; the WorkflowStartCancelledError → startCancelledResult() path was not driven.
  • Monitor with a real spawn. getMonitorRegistry was stubbed without a real registry; the Monitor failed to start cells are harness limits, not product behaviour.
  • PR CI. Checks were still pending when the round ran (several lanes already passed, but none were used as evidence).
  • Per-commit attribution. The matrix is per-guard; the two production commits (6e6600e, 97b8a0b) were not mutated commit-by-commit.

Methodology

Two worktrees (head 46db90b6, base 1ccf0d43) installed with scripts/setup-worktree.js and built with npm run build --workspace=@qwen-code/qwen-code-core under Node 22.23.4; the @qwen-code/* realpath assertion confirms each tree resolves its own packages/core, and the head dist carries interruptedBeforeCompletion (3 + 1 occurrences) while the base dist carries none — the positive control for the build. Harnesses in harness/ (shell-ab.mjs, producers.mjs, mutate.mjs, ledger.mjs) import each arm's compiled dist and read the exact functionResponse payload. A trial merge into current main (10b48ab9, none of the PR's files touched since the base) is conflict-free and produces tree c9d8bbdc752f74f16e794850479bcd940ae9f218; the targeted gate is 917/917 on both the head tree and the merged tree. Raw logs are logs-ab-*.log, logs-producers-*.log, logs-mutate-*.log, logs-gate-*.log; assertions.json is generated from those logs by harness/ledger.mjs.

A plain decline is a completed decision. Setting the marker on it is inert
today, but would report a finished interaction as "do not retry" if a future
read site consumed it.
@2SinceJuly

Copy link
Copy Markdown
Contributor Author

Thanks — both notes are addressed.

N1 — the claim is now bounded, the markers stay. The description states that only create_sub_session was shown reachable, and that monitor, workflow (early abort) and ask_user_question set the marker defensively because the scheduler cancels those calls before execute() runs. No code change: harmless defence if the pre-execution guard ever misses, but no model-visible behaviour is claimed for those paths any more.

N2 — fixed in a41828a7e. ask_user_question now sets interruptedBeforeCompletion only when the turn signal is aborted. A plain decline is a completed decision and leaves the field unset; a cancel while the dialog is open still reports true. Both cases are pinned: decline → undefined, aborted → true.

coreToolScheduler.test.ts + shell.test.ts + askUserQuestion.test.ts: 942 passed.

Head is a41828a7e — append commit, no rebase or force-push.

@2SinceJuly

Copy link
Copy Markdown
Contributor Author

CI 失败项是 Lint & Static 里的 Check lint gate freshness:main 在 2026-10-31 16:35 (CST) 合入 ca7d487(#15974,修改了 .github/workflows/ci.yml),本分支尚未包含该门禁改动。

已用普通 merge 把上游 main(fe30b5daa)同步进本分支(无冲突,未 rebase,未 force-push)。

  • 新 head:d330f755d
  • 测试:本次仅同步上游 main,无代码改动,未在本地跑测试;以 CI 复跑结果为准。

@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Qwen Code review did not complete successfully. Qwen review exited with status 1. A transient error is retried automatically; if you are seeing this, retry with @qwen-code /review. See workflow logs.

@doudouOUC doudouOUC left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EN

Re-checked this at 5ddffc30, and the core of the change holds up: the new marker reuses the exact predicate that already derives returnDisplay.outcome in the shell tool, so the two classifications cannot drift; timeout, promote-refused, real-failure and normal-completion paths never set it and each has a witness assertion; the detail gate keeps the post-completion cancel discarding output as before. CI is green on this head too — the unit lane actually ran this time (Run tests and generate reports: success) and Check lint gate freshness passes, so the evidence gap from the earlier review round is closed.

Three things from reading the consumers of that result:

1. The ACP session pipeline still reports a cancelled shell as completed (the same bug, different pipeline). packages/cli/src/acp-integration/session/Session.ts:18923-18933 picks TOOL_POST_EXECUTION_CANCELLED_MESSAGE — the constant at :957, "The tool had already completed; its output was discarded." — whenever the turn signal is aborted after execute() returned. It computes the same predicate the scheduler does (:18574-18577), has toolResult in hand, and never reads interruptedBeforeCompletion, then replaces responseParts wholesale so the shell's "Below is the output before it was cancelled" text is dropped too. That file says of itself that it mirrors CoreToolScheduler, and Session.test.ts:50468-50470 currently pins the old string. The effect is that in every session driven through the ACP agent — the web shell, IDE/ACP clients, qwen --acp — a shell killed by Esc still reaches the model as finished, with its partial output lost. Risk & Scope names the managed-runtime wrapper as the remaining surface; this one is separate and reachable today. Either reading the flag at the same predicate (BEFORE message + the result's own text as the detail, exactly as the scheduler now does) or adding it to the out-of-scope list would settle it.

2. The sub-agent tool is still mislabelled, and it is worth saying so in the body. packages/core/src/tools/agent/agent.ts:5097-5130 returns its cancel as llmContent: [{ text: … }] carrying "Agent was cancelled by the user." plus "Partial result follows:", with no marker, so the scheduler still replaces it with the after-completion notice and the partial sub-agent result is lost. Your reply explains why it was left alone, and that reasoning is sound — but it cannot be fixed by setting the flag alone, because the new detail gate at coreToolScheduler.ts:6656-6659 requires a string llmContent and this producer returns parts. One line in Risk & Scope recording that the sub-agent tool is not covered (and why the flag alone would not be enough) would keep the "no model-visible change is claimed" contract intact for a reader who only reads the description.

3. One narrow race now reports a clean exit as interrupted — pre-existing, no action needed. finalizeResult() in packages/core/src/services/shellExecutionService.ts:2483-2490 resolves with aborted: abortSignal.aborted read after the settle/drain chain and the awaited render, so an Esc landing in that window flags a command that already exited 0 as aborted; the new marker then propagates it. The tool layer already said "cancelled" there before this PR, so this is not something to fix here — I mention it only because the description's "only a user cancel sets the marker" framing reads stronger than the code.

None of this is a regression, and the diff itself is careful. Item 1 is the one I would like a decision on: fix, or record as a follow-up surface the way the managed-runtime wrapper already is.

ZH

针对 5ddffc30 重新看了一遍,改动的核心成立:新标记复用了 shell 工具里推导 returnDisplay.outcome 的那同一个判定式,两处分类不可能漂移;超时、后台提升被拒、真实失败、正常完成这几条路径都不会打标记,且各自都有断言见证;detail 的条件门让"已完成后取消"仍然和以前一样丢弃输出。这个 head 上 CI 也是绿的——这次单元测试 lane 真的跑到了(Run tests and generate reports: success),Check lint gate freshness 也通过,上一轮评审提到的证据缺口已经补上。

读完消费这个结果的几条路径后,有三点:

1. ACP session 管线仍把被取消的 shell 报成已完成(同一个 bug,另一条管线)。 packages/cli/src/acp-integration/session/Session.ts:18923-18933 在 execute() 返回后只要回合信号被中断,就固定选 TOOL_POST_EXECUTION_CANCELLED_MESSAGE(常量在 :957,即"The tool had already completed; its output was discarded.")。它计算的判定式和调度器完全一样(:18574-18577),此时 toolResult 就在手上,却从不读 interruptedBeforeCompletion,随后整体替换 responseParts,于是 shell 自己那段"Below is the output before it was cancelled"也一起丢了。该文件自称是 CoreToolScheduler 的镜像实现,而 Session.test.ts:50468-50470 目前正好把这个旧字符串钉住了。后果是:所有通过 ACP agent 跑的会话——web shell、IDE/ACP 客户端、qwen --acp——里按 Esc 杀掉的 shell 仍然以"已完成"到达模型,并且丢掉部分输出。Risk & Scope 里只点了 managed-runtime 包装层是剩余 surface,这一条是另一条、且当下就可达。要么在同一判定式处读取该标记(BEFORE 消息 + 把结果自身文本作为 detail,与调度器现在的做法一致),要么把这条 surface 也写进 out-of-scope。

2. sub-agent 工具仍被误标,值得在正文里说明。 packages/core/src/tools/agent/agent.ts:5097-5130 的取消返回是 llmContent: [{ text: … }],内容是"Agent was cancelled by the user."加"Partial result follows:",没有打标记,因此调度器仍会用"已完成后取消"的消息替换它,sub-agent 的部分结果被丢弃。你在回复里解释了为什么先不动它,理由成立——但它光靠打标记也修不好,因为新增的 detail 条件门(coreToolScheduler.ts:6656-6659)要求 llmContent 是字符串,而这里的生产者返回的是 parts。在 Risk & Scope 里加一行,说明 sub-agent 工具未覆盖、以及为什么只打标记不够,能让只读描述的人继续相信"未声称模型可见行为有变化"这个契约。

3. 一个很窄的竞态现在会把正常退出报成被中断——既有问题,本 PR 无需处理。 packages/core/src/services/shellExecutionService.ts:2483-2490 的 finalizeResult() 在 settle/drain 链与 await 渲染之后才读 abortSignal.aborted 再 resolve,因此落在这个窗口里的 Esc 会把一条已经以 0 退出的命令标记为 aborted,新标记随后把它传播出去。这个问题在本 PR 之前工具层就已经说成"cancelled"了,所以不是这里要修的事——我提它只是因为描述里"只有用户取消才会打标记"的说法比代码实际更强。

以上都不是回归,diff 本身也很谨慎。第 1 条是我希望有个决定的地方:修掉,还是像 managed-runtime 包装层那样记为后续 surface。

@2SinceJuly

Copy link
Copy Markdown
Contributor Author

Thanks — decisions on all three, at 5c008e68.

  1. The ACP pipeline is fixed, not deferred. The branch at the execution-settle predicate now reads interruptedBeforeCompletion and selects the before-completion notice; when the producer returned a string llmContent, that text is appended as the detail, so the shell's partial output survives. The unflagged case keeps "The tool had already completed; its output was discarded." A new Session.test.ts case drives the flagged path and fails on the previous commit; the existing no-flag case is unchanged and still green.
  2. Sub-agent coverage is now stated in Risk & Scope, including why the marker alone is not enough (it returns content parts, and the detail gate requires a string llmContent). No code change there.
  3. The narrow race needs no action here — agreed, and I left the description's stronger framing as-is because it is only reachable after the shell has already settled; if you would rather it read more defensively, say so and I will reword it.

The PR description now names the ACP path as covered and the sub-agent tool as out of scope. Focused runs: the two ACP cancellation cases, plus the existing scheduler/shell/question suites.

@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Qwen Code review timed out. Qwen review timed out after 10800 seconds (of the 180-minute budget). For large PRs, retry with a longer timeout by commenting: @qwen-code /review --timeout=360. See workflow logs.

@2SinceJuly 2SinceJuly left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three are settled at the current head, which is now 5c008e68.

  1. Fixed rather than deferred. 5c008e68 fix(cli): report an interrupted shell as interrupted in ACP sessions takes your first option: the ACP pipeline reads interruptedBeforeCompletion at the same predicate that already selects the BEFORE message, uses the result's own text as the detail, and no longer replaces responseParts wholesale. Session.test.ts pins the new string instead of the old one, so the web shell / IDE / qwen --acp path now reports an Esc'd shell as interrupted with its partial output intact.

  2. Recorded in Risk & Scope. The description now says the sub-agent tool is not covered, and why the flag alone would not be enough: it returns its cancel as content parts and the scheduler's detail gate requires a string llmContent.

  3. No change, as you suggested. The finalizeResult() race is pre-existing and the tool layer already said "cancelled" there before this PR; the description no longer claims otherwise.

Not run locally this round: the packages/cli suite. CI on the head that carries 5c008e68 is what I am relying on for the ACP change.

@2SinceJuly

Copy link
Copy Markdown
Contributor Author

The failing web-shell E2E Smoke (shard 1/4) job did not run the spec: it died in Install Playwright Chromium and WebKit (hosted), which timed out after 30 minutes (run 42164118652, job 127716556317). Nothing from this branch was executed, so there is no test failure to fix here.

Recent main runs pass the same job, so this looks like infrastructure rather than something this PR changed. This branch also touches only packages/cli and packages/core, not packages/web-shell. I cannot re-run the job: the fork account has no admin rights on the repository. A maintainer re-run should settle it.

@wenshao

wenshao commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

Verdict: merge-ready — 103/103 scripted assertions passed at head 23af2d1794df58546c96a579a151c0c70d147c7f (PR base tip 1b1f4455d96e9266aaf2e319fe06861aacee3e75; trial merge with main cb282ef5cfe0ba77e2bf8dd3a0bcba302415f924 at merge commit 53383cfc7d7254afc8ea3d29df1f830a8265f2a9). No blocking findings.

中文摘要(点击展开)

结论:merge-ready(103/103 断言通过,无阻塞项)。

  • 上一轮 5 条结论的现状:P1(取消时丢失已产生的输出)、P2(三个取消条件未被测试钉住)、N2(普通拒绝被标成中断)均已修复并在新 head 复测;P3(确认框打开时取消仍是通用文案)维持作者「明确超出范围」的处置;N1(四个标记里三个不可达)按作者已收窄的表述复测,仍只有 create_sub_session 会改变模型可见文案。
  • 本轮新增的 ACP 修复(5c008e68)是核心新证据:用真实 qwen --acp + 假模型服务器 + 真实 session/cancel 做线级 A/B。head 的模型收到「User intentionally cancelled…」+ 取消前输出 start;base 收到「The tool had already completed; its output was discarded」,而 base 的客户端 rawOutput.output 里明明有 start —— 输出存在却被丢弃,正是本 PR 要修的那类缺陷。见 02-acp-cancel-head-vs-base.png 与报告中的两段原始载荷。
  • 同管道负对照:正常完成的命令仍是无取消文案的成功结果;800ms 超时仍保留自己的 timed out … partial,没有被误标成用户取消。
  • a41828a7(N2 修复):直接驱动真实工具,普通拒绝 → 标记 undefined,turn 中止 → 标记 true,正常作答 → undefined。
  • 变异矩阵 12/12 killed,每行都有具名见证测试:shell 三个条件、scheduler 的两处 notice/detail、ask 标记门控两个方向、ACP 的 notice/detail/marker 门控,外加一行同文件阳性对照(改写 notice 文案,被新测试杀死)。
  • 门禁:core head/merge 各 1197/1197;CLI ACP head 1427/1427、trial merge 1430/1430(main 新增 3 个用例,PR 新用例在 merge 树同样通过)。
  • 未覆盖:Windows、真实 TUI Esc、default 审批链、容器/SSH、managed-runtime 包装层、sub-agent 工具、ACP 路径上 shell 以外的 producer,以及非布尔标记/数组 llmContent 的类型边界。PR 当前的 CI 红点全部来自 conclusion=cancelled 的 workflow(安装阶段被终止),不是代码失败,未作为通过证据。

Previous round's findings

# finding status at 23af2d17
P1 Minor — the fix dropped the partial output the model used to see Fixed. Scheduler A/B at the new head: head keeps start, base discards it. Cell A-partial-output-visible.
P2 Suggestion — !wasTimeout, !wasPromoteRefused, result.aborted unpinned Fixed. SH-drop-aborted, SH-drop-timeout-guard, SH-drop-promote-guard all killed by named tests.
P3 Suggestion — cancel while the confirmation dialog is open still says the generic reason Declined, stands. Both arms reach awaiting_approval with no model text; the PR body lists this shape as out of scope.
N1 Suggestion — three of the four markers are not observable Stands as bounded. Re-measured: monitor/workflow stay on the pre-execution path, ask_user_question on user-denied; only create_sub_session flips. The description now claims only what is reachable.
N2 Suggestion — ask_user_question marks a plain decline as interrupted Fixed in a41828a7. Decline → undefined, aborted turn → true, answered → undefined; both directions are pinned by mutants.

What changed since round 2

Round 2 verified 46db90b6. Since then the branch added two production commits and merged main repeatedly:

  • a41828a7 — ask_user_question sets the marker only when the turn signal is aborted (fixes N2).
  • 5c008e68 — the ACP session pipeline reads the marker and keeps the producer's partial output (fixes the ACP sibling surface).

Everything else in 46db90b6..23af2d17 is main. A trial merge into current main (cb282ef5, 202 commits ahead of the PR's base) is conflict-free; the merged tree keeps both the PR hunk and main's new Session.ts auth-policy block.

Central claim and scheduler A/B

Claim: a shell killed by a user cancel must reach the model as interrupted mid-work, with the output it produced before the kill.

Both arms load their own compiled packages/core/dist and drive the real CoreToolScheduler with a real ShellToolInvocation (printf 'start'; touch marker; sleep 30; printf 'done'), aborting the turn signal once the marker appears.

Cell Oracle Head Base
A-status scheduler status cancelled cancelled
A-notice-before-completion cancellation notice intentionally cancelled had already completed
A-no-after-completion no "completed" wording yes no
A-partial-output-visible start survives yes no
A-tool-marked-interrupted producer sets the marker true undefined
A-detail-matches-tool detail equals the tool's own account yes no
B-status / B-no-cancellation-notice completed command success / none same
C-status / C-timeout-message timed-out command error / timed out same
C-no-interrupted-notice timeout is not a user cancel yes same

Head 11/11; base 6/11 — the five cancel cells are expected red, every neighbouring outcome matches. Witness: 01-ab-shell-cancel-head-vs-base.png.

Scheduler A/B: shell cancelled mid-command — head keeps the notice and the partial output, base discards both

The ACP surface (new in this round)

5c008e68 fixes the pipeline used by qwen --acp, the web shell and IDE clients. I drove a real ACP agent through session/cancel and read the request body of the next prompt, which is exactly what an ACP session sends the model.

Cell Head Base
ACP-cancel-landed-mid-run marker seen same
ACP-turn-stopped stopReason=cancelled same
ACP-client-update-failed terminal status failed same
ACP-client-interrupted-notice intentionally cancelled had already completed
ACP-tool-result-reached-model yes yes
ACP-interrupted-notice yes no
ACP-no-after-completion yes no
ACP-partial-output-kept start kept start dropped
ACP-normal-result-unlabelled normal command stays clean same
ACP-timeout-keeps-own-wording timeout keeps timed out … partial same

Head 10/10; base 6/10 with the four flipped cells expected red. The same harness on the trial merge is 10/10. Witness: 02-acp-cancel-head-vs-base.png.

ACP A/B: a real ACP session cancelled mid-command — head keeps the notice and start, base reports completion

The two model-wire payloads, verbatim:

HEAD: {"role":"tool","tool_call_id":"call_cancel_1","content":[{"type":"text","text":"User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it.\n\nCommand was cancelled by user before it could complete. Below is the output before it was cancelled:\nstart"}]}
BASE: {"role":"tool","tool_call_id":"call_cancel_1","content":[{"type":"text","text":"The tool had already completed; its output was discarded."}]}

On base the terminal client update for the same call carries "rawOutput":{"outcome":"cancelled","output":"start","text":"start"} — the partial output existed and was thrown away on its way to the model. That is the strongest form of the bug class on this surface.

Sibling producers, re-measured

Cell Head Base
create_sub_session cancelled mid-spawn before-completion notice + its message had already completed
monitor pre/sync/micro/window abort Tool call cancelled before execution. same
workflow pre/sync/micro/window abort Tool call cancelled before execution. same
monitor abort at 0 ms Tool call cancelled by user. same
ask_user_question plain decline User did not allow tool call same
ask_user_question turn aborted while dialog open awaiting_approval, no model text same

So N1 still stands exactly as the description now bounds it. Witness: 04-sibling-producers.png.

Sibling producers: only create_sub_session flips the model-visible notice

ask_user_question's marker itself, driven directly through the real tool:

Cell Head Base
plain decline undefined undefined
turn aborted true undefined
answered undefined undefined

Mutation matrix

Every row reverts one guard, runs the named suite, and restores with git checkout HEAD --; the runner refuses a dirty tree or a non-unique anchor. Baselines through the same runner: shell 376/376, scheduler 541/541, ask 25/25, and both filtered CLI selections green. No commit-to-commit attribution is claimed: the clone is shallow, so the PR-only commit set cannot be enumerated reliably.

Mutant Outcome Witness
shell: drop result.aborted killed does not mark a completed command as interrupted
shell: drop !wasTimeout killed keeps the first timeout after a later user cancellation
shell: drop !wasPromoteRefused killed promote-refused race case
scheduler: always pick the "already completed" notice killed tells the model a returned interruption result never completed
scheduler: drop the caller-passed detail killed keeps the partial output a cancelled producer reported
scheduler: drop the appended response detail killed same
ask: always set the marker killed should return cancellation message when user declines
ask: never set the marker killed marks the unanswered questions as interrupted when the turn is aborted
ACP: ignore the marker killed reports a returned interruption as interrupted and keeps its output
ACP: drop the partial-output detail killed same
ACP: append detail without the marker killed records execution-stage cancellation on both axes
ACP: rewrite the notice text (control) killed the new test's exact-match assertion

12/12 killed, each with a named witness. Witness: 03-mutation-matrix.png.

Mutation matrix: every guard killed; baselines green

Targeted gates

Suite Head 23af2d17 Trial merge 53383cfc
core: scheduler + shell + ask + sub-session + monitor + workflow 1197/1197 1197/1197
CLI: src/acp-integration/session/Session.test.ts 1427/1427 1430/1430

The merge suite is three tests larger because main added cases since the PR's base; the PR's new ACP test and the existing no-flag pin both pass there. Witness: 05-gates-and-merge.png.

Targeted gates on head and on the trial merge

Findings (non-blocking)

ACP reports the terminal client status as failed, not cancelled — pre-existing

The ACP tool_call_update for a cancelled call carries status: "failed" on both arms; the distinction lives in rawOutput.outcome: "cancelled" and in the message text. This is the emitter's pre-existing mapping (createTranscriptToolCallResultUpdate has no cancelled status), unchanged by this PR, and the PR's model-facing claim does not depend on it. Noted so the next reader does not read the shared failed as a missed half of this fix.

Not covered

  • Windows, and a real TUI Esc. Esc → cancelOngoingRequest() → abort() is read from code, not driven through a terminal.
  • Approval mode default, full chain. Only a turn abort while the question dialog is open was driven (identical on both arms).
  • Container and SSH execution environments. SshExecutionEnvironment.execute() rebuilds the result itself and never sets the marker — inferred from code, not measured.
  • The managed-runtime wrapper. Its result whitelist does not forward the marker; the PR body states this as a tracked follow-up.
  • The sub-agent tool. It returns its cancel as content parts and sets no marker; the PR body states this as out of scope.
  • ACP producers other than shell. create_sub_session, monitor, workflow and ask_user_question were measured through the scheduler, not through ACP.
  • Type boundaries. interruptedBeforeCompletion is checked truthily in core and with === true in ACP; every producer is TS-typed to boolean and no external path spreads arbitrary fields into ToolResult, so non-boolean values and non-string llmContent with the marker set were not driven.
  • Per-commit mutation attribution. The clone is shallow, so the mutation matrix is per-guard; the two post-round-2 production commits (a41828a7, 5c008e68) each have dedicated rows.
  • PR CI. Every red check belongs to a workflow run whose conclusion is cancelled (e.g. Build unit-test tree terminated during pnpm install/node-gyp at 13:54Z; the TUI and Managed Agent runs cancelled at 13:53Z), not a code-level failure. No CI result is used as evidence.

Methodology

Three worktrees on Node 22.23.4 — head 23af2d17, base 1b1f4455, and a trial merge of head with current main cb282ef5 — each installed with scripts/setup-worktree.js and fully built with npm run build. readlink -f node_modules/@qwen-code/qwen-code-core resolves into each arm's own tree; the head core dist contains the new notice and the base dist contains none of the marker (positive control for the build). harness/shell-ab.mjs imports each arm's compiled core dist and drives the real scheduler; harness/acp-ab.mjs spawns each arm's built CLI with --acp against a fake OpenAI server and reads both the model request body and the terminal session/update; harness/ask-marker.mjs drives the real question tool; harness/mutate2.mjs runs the matrix. Raw logs are logs-*.log, mutation-head/rows.json, gate-*.json; assertions.json is generated from them by harness/ledger.mjs. Evidence images are in evidence/.

@2SinceJuly

Copy link
Copy Markdown
Contributor Author

No code change from this round. P1, P2 and N2 are confirmed fixed at the head you measured; P3 stays out of scope as declared in the PR body, and N1 stays bounded at create_sub_session as you re-measured.

On your CI note: the red checks on this head are cancelled workflows, not failures — the same install-phase termination you saw — so there is still no CI conclusion to cite for 23af2d17. The local core and CLI ACP gates you ran remain the only execution evidence.

@wenshao
wenshao enabled auto-merge (squash) November 6, 2026 18:43

@yiliang114 yiliang114 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at c08d50a.

The change adds an interruptedBeforeCompletion marker to ToolResult and wires it through 5 producers (shell on user cancel, monitor, workflow, create-sub-session, askUserQuestion on abort) and 2 consumers (CoreToolScheduler + ACP Session). The predicate logic is correct at every site:

  • shell: sets only on aborted && !wasTimeout && !wasPromoteRefused — correct exclusion of timeouts and promote refusals.
  • askUserQuestion: sets only when the signal is aborted, not on a plain decline — correct distinction between interrupted and decided.
  • scheduler: reads the marker to choose between the new TOOL_CANCELLED_BEFORE_COMPLETION_MESSAGE and the existing TOOL_CANCELLED_AFTER_COMPLETION_MESSAGE, and appends partial output as detail — mirrors the old shape's output-preservation policy.
  • ACP Session: mirrors the scheduler's logic, including partial output forwarding.

Tests cover every producer's set/unset behavior and both consumers' message selection. The disclosed out-of-scope gaps (managed-runtime, sub-agent tool, confirmation dialog race) are documented in the PR body and are not regressions. No security, concurrency, or type-safety concerns.

@yiliang114

Copy link
Copy Markdown
Collaborator

Closeout (b) isn't met. This isn't purely internal: the diff adds a model-visible interruptedBeforeCompletion field to the tool result and changes the scheduler's cancellation notice to "User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it." — flowing through the ACP Session.ts pipeline into qwen --acp, the web shell, and the IDE clients. That's a protocol/visible-surface change, so it needs a pr-ui-verify report bound to c08d50a3 with a real before/after observation (run a command, Esc out, ask the model whether it finished); unit tests alone don't satisfy (b). The merge-ready comment carries no pr-ui-verify marker, so it doesn't count here.

Separately, the head is still blocked on queued required CI (Test (ubuntu-latest, Node 22.x) shards + Integration Tests (no-AK, No Sandbox)), so this isn't the merge gate this round — just flagging the gap so it's closed before a later merge.

@2SinceJuly

Copy link
Copy Markdown
Contributor Author

Same situation as the gate describes, and the head has not moved: c08d50a3 is still the tip.

The pr-ui-verify marker has no producing tool in this repo — grep -rn "pr-ui-verify" .github/ scripts/ finds nothing — so I cannot post a report bound to c08d50a3 that would satisfy (b) without fabricating it. I am not doing that. The before/after you describe (run a command, Esc out, ask the model whether it finished) needs an interactive TTY with model credentials, which this environment does not have.

What the change is supposed to be observable as, for whoever does have that environment: with the marker set, the scheduler's cancellation notice becomes "User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it." instead of the completion notice, so the model should report the call as interrupted rather than finished. yiliang114 already approved c08d50a3 on the predicate logic and the five producer sites; the gap is only the report binding.

Agreed the queued required CI makes this moot this round. If you can tell me the marker's expected format, I will produce it from a real run as soon as one is available to me.

@qwen-code-review-bot qwen-code-review-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. Suggestions are inline.

Not explored to full depth (tool budget reached): "agent 6a": none — I finished the walk of every hunk and of both consumers..

— deepseek-v4.1-flash-1017 via Qwen Code /review (v0.26.9)

Comment on lines +1048 to +1049
const TOOL_CANCELLED_BEFORE_COMPLETION_MESSAGE =
'User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it.';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-1: This constant duplicates core's TOOL_CANCELLED_BEFORE_COMPLETION_MESSAGE by hand, and the sibling copy one line above has already drifted: core's after-completion notice ends with “…User intentionally cancelled. Stop and await further instructions; do not retry.” (the #10170 stop directive), while TOOL_POST_EXECUTION_CANCELLED_MESSAGE here stops at “…was discarded.” So every ACP-driven session — the web shell, IDE/ACP clients, qwen --acp — already gives the model a weaker cancellation instruction than the TUI, and a future edit to core's new wording will silently leave this copy stale in exactly the same way.

Witness:

CORE AFTER : "The tool had already completed; its output was discarded. User intentionally cancelled. Stop and await further instructions; do not retry."
ACP  AFTER : "The tool had already completed; its output was discarded."
AFTER equal? false
CORE BEFORE: "User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it."
ACP  BEFORE: "User intentionally cancelled this tool call. Stop and await further instructions; do not retry or work around it."
BEFORE equal? true

Export both constants from packages/core/src/core/coreToolScheduler.ts and import them here, deleting both local copies — that also repairs the existing drift:

import {
  TOOL_CANCELLED_AFTER_COMPLETION_MESSAGE,
  TOOL_CANCELLED_BEFORE_COMPLETION_MESSAGE,
} from '@qwen-code/qwen-code-core/core/coreToolScheduler.js';

Session.ts is allowlisted for root-barrel core imports (eslint.legacy-core-barrel-imports.mjs:23), but new imports belong on the defining module @qwen-code/qwen-code-core/core/coreToolScheduler.js, per AGENTS.md's “Core imports in cli” rule.

— deepseek-v4.1-flash-1017 via Qwen Code /review (v0.26.9)

? new Error(toolResult.error.message)
: status === 'cancelled'
? new Error(TOOL_POST_EXECUTION_CANCELLED_MESSAGE)
? new Error(cancelReason)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-2: This line feeds the client-visible tool error — responseError goes to this.toolCallEmitter.emitResult({ …, error: responseError }), which renders errorMessage: params.error?.message — but nothing in Session.test.ts pins it. The new case asserts only result.parts[0].functionResponse.response, and emitResult appears nowhere in the 65,996-line file. Revert this to new Error(TOOL_POST_EXECUTION_CANCELLED_MESSAGE) and the suite stays green while the model is told “User intentionally cancelled this tool call…” and the user's UI shows “The tool had already completed; its output was discarded.” — the two surfaces disagreeing about whether work landed is the defect this PR removes.

Witness:

witness: not run — the closest capability is a mutation probe (revert this line, rerun Session.test.ts); the scratch tree was unavailable and a mutation may not be applied in the shared worktree. Static sweep of the real file: 0 occurrences of `emitResult` in Session.test.ts, 4 of `errorMessage`, none on a cancellation path.

In the new test, spy on ToolCallEmitter.prototype.emitResult (sibling methods are already spied this way around Session.test.ts:50770) and assert the recorded error?.message is the before-completion notice plus the preserved llmContent.

The assertion must go red when this line is reverted to new Error(TOOL_POST_EXECUTION_CANCELLED_MESSAGE) — otherwise the client-visible half of the ACP fix is unprotected.

— deepseek-v4.1-flash-1017 via Qwen Code /review (v0.26.9)

// The notice says the call did not finish; the producer's own text says how
// far it got. Replacing the whole payload with the notice loses the partial
// output the model needs to decide what remains.
it('keeps the partial output a cancelled producer reported', async () => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-3: The table above pins the positive half of the new detail guard but not the negative half. The post-completion row (textResult('done'), no marker) asserts only that the after-completion notice is present and the before-completion notice absent — never that done is absent. Drop toolResult.interruptedBeforeCompletion from the detail ternary at coreToolScheduler.ts:6656-6658 and that row still passes, while a completed command's output is re-appended to a body that says “The tool had already completed; its output was discarded.” The ACP side is guarded against the mirror mutation by the exact-equality test at Session.test.ts:52748; the scheduler side has no equivalent.

Witness:

witness: not run — the closest capability is `revert-hunk` on coreToolScheduler.ts:6655-6659 plus a rerun of the two scheduler cases; the scratch tree was unavailable. Sweep of the real assertion set: 0 hits of `not.toContain('done')` in the 15,837-line file; the asserted pair is only toContain('The tool had already completed') / not.toContain('User intentionally cancelled this tool call. Stop'), which a leaked "done" does not violate.
Suggested change
it('keeps the partial output a cancelled producer reported', async () => {
expect(responseText).toContain(notice);
expect(responseText).not.toContain(wrongNotice);
expect(responseText).not.toContain('done');

The absence assertion must go red when toolResult.interruptedBeforeCompletion is removed from the detail ternary — that is the mutation it exists to catch.

— deepseek-v4.1-flash-1017 via Qwen Code /review (v0.26.9)

Comment on lines +3442 to +3444
...(result.aborted && !wasTimeout && !wasPromoteRefused
? { interruptedBeforeCompletion: true }
: {}),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-4: The marker is attached only at this foreground result builder, so other returned-cancel arms still produce the notice this PR removes. sedEditCancelledResult()'s user-cancel arm (shell.ts:2006-2009) returns { llmContent: 'Command was cancelled by user before it could complete.', returnDisplay: 'Command cancelled by user.' } with no marker, and it is reached whenever Esc lands while a prepared simulated sed -i awaits waitForSedOperation (shell.ts:2062-2068, 2116-2126) — an edit that never reached writeTextFile. The scheduler then sees aborted with the marker undefined, selects TOOL_CANCELLED_AFTER_COMPLETION_MESSAGE, passes no detail, and the model is told the edit already completed, so it may act as if the file changed. exitPlanMode.ts:268 and :343 are the same shape (a plain noActionResult on signal.aborted). Neither arm is in the PR body's out-of-scope list, and both are exactly the “returned rather than thrown” class this marker exists for.

Witness:

npx vitest run src/tools/shell.test.ts -t "sed execution is cancelled after reading"
 → 1 passed  (confirms the sed cancel is a returned ToolResult, whose literal at shell.ts:2006-2013 carries only llmContent/returnDisplay)

npx vitest run src/core/coreToolScheduler.test.ts -t "tells the model a post-completion cancellation discarded finished work"
 → 1 passed  (that row asserts the model sees "The tool had already completed" for a marker-less returned result — the outcome this arm produces)
// sedEditCancelledResult(), non-timeout arm
return {
  llmContent: 'Command was cancelled by user before it could complete.',
  returnDisplay: 'Command cancelled by user.',
  interruptedBeforeCompletion: true,
};

The TimeoutError arm of the same helper (shell.ts:1993-2003) returns error.type = ToolErrorType.EXECUTION_TIMEOUT and must stay unflagged — the scheduler derives isTimeout from exactly that type, and shell.test.ts:5356-5360 pins expect(result.interruptedBeforeCompletion).toBeUndefined() for the timed-out path; exitPlanMode's neighbouring returns at :263, :277 and :352 are completed no-ops and must not be marked either, so the marker belongs on the returns rather than in the shared helper.

A simulated sed edit test that aborts while trackEdit is pending (asserting result.interruptedBeforeCompletion === true) and an exitPlanMode test that aborts during requestPlanApproval must both go red without the added field.

— deepseek-v4.1-flash-1017 via Qwen Code /review (v0.26.9)

Comment on lines +970 to +972
// Same returned-interruption shape as the monitor: nothing ran, so the
// scheduler must not tell the model the call completed.
interruptedBeforeCompletion: true,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-5: WorkflowStartCancelledError is raised for a registry-side cancel (cancelStarting/abortAll) that aborts the run's controller “while the caller's signal stays live, in either mode” (workflow.ts:566-580). This marker is set on that path too, but both consumers read it only inside an aborted-signal branch (coreToolScheduler.ts:6609, Session.ts:19654), and aborted = parentAbortedAtExecutionSettle && !isTimeout is false there — so the call settles success, fires a PostToolUse success hook and is recorded as completed while its own text says it was cancelled, and the marker this diff adds contributes nothing. That contradicts the field's doc comment (“The scheduler uses it to tell the model the call was interrupted rather than completed”) and the PR body's statement that these defensive producers are cancelled before execute() runs.

Witness:

npx vitest run src/tools/workflow/workflow.test.ts -t "registry-side cancel"
 Tests 2 passed | 146 skipped (148)

workflow.test.ts:1092-1113 asserts expect(caller.signal.aborted).toBe(false)
beside the now-marked CANCELLED_BEFORE_START. Read-site sweep: the marker has
exactly 3 read expressions package-wide, all inside a signal.aborted branch.

Either document the coupling the field actually has — “only honoured when the turn signal is aborted” — or widen the two read sites so a marked result settles cancelled regardless of the caller signal. Do not simply drop the field from startCancelledResult(): the background early return (workflow.ts:546) and the background catch arm (:583-588) do abort the caller signal, and there the marker is read and correct.

const aborted = parentAbortedAtExecutionSettle && !isTimeout; (coreToolScheduler.ts:6564-6565) keeps a structured EXECUTION_TIMEOUT out of the cancel path, so any fix that lets the marker alone force cancelled must not reclassify a timeout.

If you widen the read sites instead of documenting, a scheduler test whose execute() resolves a marked result with a live signal must assert the call settles cancelled — today it settles success.

— deepseek-v4.1-flash-1017 via Qwen Code /review (v0.26.9)

@wenshao
wenshao merged commit a84493d into QwenLM:main Nov 7, 2026
159 of 160 checks passed
wenshao added a commit that referenced this pull request Nov 7, 2026
…n-hook

main advanced 11 commits (stacked-skill overflow #16404, workflow agent
approvals #16432, daemon UI tool blocks #16306, sibling model settings
#15829, late snapshot publication claim #16489, inline FQN imports in
runtime modules #16504, cancelled shell as interrupted #15836, CSI statement
helper fold #16503, Maven suite cache write-back #15274, sed char match
#15701, Ink TUI tool-batch round #16428). One conflict, the workflow size
ratchet:

- .size-baseline: sdk-java.yml auto-merged to 167124 bytes (HEAD recorded
  161927 / main 165241, both below the combined actual), so record the actual
  merged size 167124 per the ratchet's "record the real size in the same PR"
  rule. The absolute gate (470000) and the 4096-byte baseline allowance both
  pass.

No source conflict: main's commits did not touch this branch's A3 source
files, so RuntimeBrokerService / RuntimeExecutionRecovery (R18-1/R19-1
observe-only mount-handback guard) and the cli serve fence are unchanged.

Verification: check-workflow-size.sh green (bash5); workflow meta tests
(workflow-size + sdk-java-workflow + hosted-process-ci) 428 green; workspace
build and ClassSizeRatchetTest re-run after the merge.
@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Released in v0.27.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cancelling a running shell command tells the model "The tool had already completed" although the command was killed

7 participants