Skip to content

fix(ci): restore a meaningful Windows merge gate - #14298

Merged
yiliang114 merged 2 commits into
mainfrom
fix/windows-ci-baseline
Oct 16, 2026
Merged

yiliang114 merged 2 commits into
mainfrom
fix/windows-ci-baseline

Conversation

@yiliang114

Copy link
Copy Markdown
Collaborator

What this PR does

The Windows lane still runs the full workspace and script suites and reports their existing failures as warnings, while the Windows installer/security tests and Hosted process suite remain blocking.

Why it's needed

Enabling the merge queue reactivated a Windows lane with accumulated baseline failures. Every queued change became red for unrelated failures, and the root test:ci chain stopped before reaching the Windows installer coverage when workspace tests failed.

Reviewer Test Plan

How to verify

Confirm that workspace and full script failures remain visible as warnings, while a failure in the dedicated installer test or Hosted process suite still fails the Windows job.

Evidence (Before & After)

N/A — CI-only change.

Tested on

OS Status
🍏 macOS ✅ targeted contracts, installer suite, full script suite
🪟 Windows ✅ build/bundle, installer security cases, Hosted suite
🐧 Linux ⚠️ not tested

Environment (optional)

Remote Windows PowerShell environment plus local macOS worktree.

Risk & Scope

  • Main risk or tradeoff: broader Windows workspace and script regressions are advisory until their standing baseline failures are repaired.
  • Not validated / out of scope: repairing the existing cross-platform failures in the full workspace and script suites.
  • Breaking changes / migration notes: none.

Linked Issues

N/A

中文说明

这个 PR 做了什么

Windows lane 仍会运行完整 workspace 与脚本套件,并将已有失败作为 warning 完整展示;Windows 安装/安全测试及 Hosted 进程套件继续作为阻断项。

为什么需要

启用 merge queue 后,长期未运行的 Windows lane 暴露了累积的基线失败,导致所有入队变更都因无关问题标红;同时根级 test:ci 在 workspace 失败后会提前停止,实际要求的 Windows 安装测试反而没有执行。

Reviewer 测试计划

如何验证

确认 workspace 与完整脚本套件失败时仍会输出 warning,而专门的安装测试或 Hosted 进程套件失败仍会使 Windows job 失败。

证据(Before & After)

N/A — 仅 CI 变更。

已测试环境

OS 状态
🍏 macOS ✅ 定向契约、安装套件、完整脚本套件
🪟 Windows ✅ 构建/bundle、安装安全用例、Hosted 套件
🐧 Linux ⚠️ 未测试

环境(可选)

远程 Windows PowerShell 环境与本地 macOS worktree。

风险与范围

  • 主要风险或取舍:在完整 workspace 与脚本套件的既有 Windows 基线修复前,更广泛的 Windows 回归仅作为 advisory 展示。
  • 未验证 / 范围外:修复完整 workspace 与脚本套件中已有的跨平台失败。
  • Breaking changes / 迁移说明:无。

关联 Issue

N/A

Keep the known workspace baseline visible while making Windows installer/script coverage and hosted process checks authoritative again.
Keep the broader failing baselines visible while making the verified Windows installer and Hosted process contracts authoritative.
@yiliang114

Copy link
Copy Markdown
Collaborator Author

Windows validation status:

  • npm run build and npm run bundle: passed on a remote Windows host.
  • Windows installer/uninstaller security cases: 10 passed; 1 conditional case skipped because PowerShell 7 modules are unavailable on that host. The remaining Windows-specific cases passed.
  • Hosted process suite: 9/9 passed.
  • The complete script suite still reports its standing baseline: 1,871 passed, 207 skipped, 14 failed. This PR keeps that output visible as advisory instead of hiding it.
  • The earlier merge-queue job also exposed 139 standing workspace failures; those remain visible as advisory and are out of scope here.
  • Official workflow dispatch for the current head is still queued: https://github.com/QwenLM/qwen-code/actions/runs/38729791757

The blocking Windows coverage in this PR is therefore the installer/security suite plus the Hosted process suite.

@qwen-code-review-bot

qwen-code-review-bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Qwen Triage finished — CI landed green on 0e75a80 and the deferred approval was posted. finalize run

✅ Qwen Triage 已完成 —— 0e75a80 的 CI 全绿,延迟审批已提交。查看 finalize 运行

@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

Template looks good ✓ — every required section is filled in, including the Risk & Scope tradeoff and the Chinese translation.

Problem: observed, and I confirmed it independently rather than taking the description's word for it. Merge-queue run 38712612535 (job 116347595111, 2026-10-15) has Test (windows-latest, Node 22.x) = failure. In its 44,259-line log the chain reads > test:ci → > test:ci:workspaces → workspace failures (core 22 files failed / 1238 passed; cli 13 failed / 905 passed) — and install-script.test.js appears zero times. Since test:ci is test:ci:workspaces && test:scripts, the && short-circuited, so the Windows installer coverage this lane exists to provide never executed. The job was red and blind. The underlying failures are genuinely Windows-shaped, e.g. src/events.test.ts:2470 expect(stats.mode & 0o777).toBe(0o600) → got 438 (0o666), POSIX mode bits Windows can't express, plus a path-separator assertion expecting /attachments/1-1/0.txt inside a C:\actions-runner-win-hk-6\... string. The 2026-09-26 smoke run (108351955499) shows the same shape: workspace failures, zero install-script mentions.

Direction: aligned — this is CI gate reliability, squarely infrastructure, and it touches no public contract. Worth noting the Hosted process suite still passed 9/9 inside that same failing job, so the blocking leg that matters is healthy; the fix is about unblocking the rest.

Size: not applicable. Two files, +42/−5: .github/workflows/ci.yml and scripts/tests/no-ak-integration-ci.test.js. No core paths (packages/core/src/**, auth, providers, models, config, tools, services), so the two-tier core gate doesn't engage.

Approach: the scope feels right and the mechanism is correct — splitting the && chain so both halves always run is the actual fix, not just the warning. You also updated the meta-test in the same commit, which is what AGENTS.md asks for when .github/workflows/*.yml changes. I walked every other consumer of ci.yml's test_windows and none of them go stale: the byte-identity pin in no-ak-integration-ci.test.js slices export TMPDIR= → ( while true, and your edits all land after that sampler sentinel, so the three legs stay identical; the trap ... EXIT assertion you dropped for Windows is still covered indirectly by that same byte-identity region; ci-disk-pressure.test.mjs's sampler regexes, the runs-on pin, timeout-minutes: 60, and the guard-step pins are all untouched.

Three questions, none of them blockers:

  • Making all Windows workspace tests advisory is the blunt version of this. Today 22 core + 13 cli files fail, but a genuinely new Windows regression in the other ~2,100 passing files now only warns. Was keeping the suite blocking and excluding just the known-failing files considered? That would preserve the gate for new breakage. Fine to defer to a follow-up, but I'd want the baseline repair tracked somewhere rather than open-ended.
  • install-script.test.js now runs twice on Windows — once inside test:scripts, once standalone — on a lane with a 60-minute timeout whose own vitest config documents ~5x slowdown under pool contention.
  • The new blocking gate has no retry. The Ubuntu lane wraps both halves in --retry="${VITEST_RETRY}" (default 2) precisely because "the same commit run three times failed three disjoint test sets" (ci: Test (ubuntu) fails non-deterministically on shared runners — a different test set each run #10490), and scripts/tests/vitest.config.ts singles out install-script.test.js as flake-prone on Windows ("a full tar+gzip pass under antivirus inspection… the 4.8s one is right at vitest's 5s default and flakes"). So the one suite that can now fail the whole queue is one with a documented Windows flake history and no retry behind it.

Risk: no elevated risk signals — neither changed file matches the revert-correlated high-risk paths.

Two accuracy nits for whoever touches this next, both non-blocking: the meta-test is still named "preserves test failures in every wrapped OS job" though Windows no longer preserves them as failures, and scripts/tests/package-assets.test.js:744 justifies a path normalisation by saying such a bug would "go red in the test_windows lane (which runs test:scripts)" — after this change it would only warn.

Moving on to code review. 🔍

中文说明

感谢贡献!

模板完整 ✓ —— 所有必填章节都写了,包括 Risk & Scope 的取舍说明和中文翻译。

问题: 已观测到,而且我没有只采信 PR 描述,是自己独立确认过的。merge queue 运行 38712612535(job 116347595111,2026-10-15)中 Test (windows-latest, Node 22.x) = failure。在它 44259 行的日志里,调用链是 > test:ci → > test:ci:workspaces → workspace 失败(core 22 个文件失败 / 1238 通过;cli 13 失败 / 905 通过)—— 而 install-script.test.js 出现次数为 0。因为 test:ci 是 test:ci:workspaces && test:scripts,&& 短路了,所以这个 lane 本该提供的 Windows 安装测试根本没有执行。job 既是红的,又是瞎的。底层失败确实是 Windows 特有的,例如 src/events.test.ts:2470 的 expect(stats.mode & 0o777).toBe(0o600) 实际得到 438(0o666),这是 Windows 无法表达的 POSIX 权限位;另有一个路径分隔符断言在 C:\actions-runner-win-hk-6\... 字符串里找 /attachments/1-1/0.txt。2026-09-26 的 smoke 运行(108351955499)形态相同:workspace 失败,install-script 零次出现。

方向: 对齐 —— 这是 CI 门禁可靠性问题,属于纯基础设施,不涉及任何公开契约。值得一提的是,在同一个失败的 job 里 Hosted process suite 依然 9/9 通过,说明真正关键的阻断项是健康的;这次修复解决的是其余部分被卡住的问题。

规模: 不适用。两个文件,+42/−5:.github/workflows/ci.yml 和 scripts/tests/no-ak-integration-ci.test.js。未触及核心路径(packages/core/src/**、auth、providers、models、config、tools、services),因此两级核心门禁不适用。

方案: 范围合理,机制正确 —— 拆开 && 链让两半都必然执行,才是真正的修复,而不只是加了个 warning。你也在同一个 commit 里更新了 meta-test,这正是 AGENTS.md 对修改 .github/workflows/*.yml 的要求。我逐个检查了 ci.yml 中 test_windows 的其他消费方,没有一个会失效:no-ak-integration-ci.test.js 里的逐字节一致性断言截取的是 export TMPDIR= → ( while true,而你的改动全部落在这个 sampler 哨兵之后,所以三个 leg 仍然完全一致;你为 Windows 去掉的 trap ... EXIT 断言,仍由同一块逐字节一致区域间接覆盖;ci-disk-pressure.test.mjs 的 sampler 正则、runs-on 断言、timeout-minutes: 60 以及 guard step 断言都未被触及。

三个问题,都不是阻断项:

  • 把所有 Windows workspace 测试降为 advisory 是这个方案里比较粗暴的一面。今天失败的是 22 个 core + 13 个 cli 文件,但另外约 2100 个通过文件里如果出现真正的新 Windows 回归,现在也只会 warning。是否考虑过保持套件阻断、只排除已知失败的文件?那样能对新破坏保留门禁。放到后续 PR 也可以,但我希望基线修复有地方跟踪,而不是无限期搁置。
  • install-script.test.js 现在在 Windows 上会跑两次 —— 一次在 test:scripts 里,一次单独跑 —— 而这个 lane 超时上限是 60 分钟,其 vitest 配置自己就记录了共享池竞争下约 5 倍的降速。
  • 新的阻断门禁没有 retry。Ubuntu lane 把两半都包在 --retry="${VITEST_RETRY}"(默认 2)里,原因正是"同一个 commit 跑三次会失败三组互不相同的测试"(ci: Test (ubuntu) fails non-deterministically on shared runners — a different test set each run #10490);而 scripts/tests/vitest.config.ts 特别点名 install-script.test.js 在 Windows 上易 flake("完整的 tar+gzip 过程要过杀毒扫描……4.8s 那次正好卡在 vitest 默认的 5s 上,会 flake")。也就是说,现在唯一能让整个队列失败的套件,恰恰是有 Windows flake 记录、背后又没有 retry 的那个。

风险: 无升级风险信号 —— 两个改动文件都不匹配与 revert 相关的高风险路径。

两个准确性小问题,供后续修改者参考,均非阻断:meta-test 的名字仍是"preserves test failures in every wrapped OS job",但 Windows 已经不再把失败保留为失败了;scripts/tests/package-assets.test.js:744 用"会在 test_windows lane(它跑 test:scripts)变红"来论证一处路径归一化的必要性 —— 这次改动之后它只会 warning。

进入代码审查 🔍

— Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 0e75a801a92a08646811af63ec0b0c9b74fef907 · re-run with @qwen-code /triage

@qwen-code-review-bot

qwen-code-review-bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Code review

I wrote down my own approach before reading the diff, from just the title and the "Why it's needed" section: split the Windows step's npm run test:ci into its two existing halves so neither can short-circuit the other, capture each exit code separately under set +e, keep the installer/security suite and the Hosted suite as the blocking exit, and downgrade the two accumulated-baseline halves to ::warning:: so their output stays in the log. That is what this PR does, essentially line for line, so I have no simpler alternative to argue for. The one shape I'd have preferred long-term — keep the suites blocking and carry an explicit Windows skip-list of the ~35 known-failing files — is more machinery and rots quietly, so it's a reasonable follow-up rather than a reason to hold this.

The shell logic is correct. set +e brackets the three invocations and set -e restores errexit afterwards, each $? is read on the line immediately following its command, and both the df/meminfo dump and the final exit were re-pointed from RC to INSTALLER_RC consistently — no stale variable left behind. SCRIPTS_RC and WORKSPACE_RC are each consumed by their own warning condition, so neither is dead.

Two non-blocking observations:

  • npx vitest run scripts/tests/install-script.test.js --config scripts/tests/vitest.config.ts re-derives an invocation the repo already has. npm run test:scripts -- scripts/tests/install-script.test.js expands to vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/install-script.test.js — the same run, through the existing script, with no second spelling of the config path to keep in sync. npx does resolve the local node_modules/.bin/vitest first so I don't think this is broken today, but it's a parallel path where a reuse was available.
  • The meta-test name preserves test failures in every wrapped OS job now over-claims, since Windows failures are preserved as warnings rather than as failures. Renaming it (or a one-line comment on the Windows block saying failures there are advisory by design) would stop a future reader assuming the lane still gates on workspace results. Same for the stale justification at scripts/tests/package-assets.test.js:744.

I checked the new pin block resolves against the YAML as written: indexOf('set +e') has no earlier match in the step body, set -e isn't a substring of set +e, and the ::warning:: pins are prefixes of the longer echoed strings, so each indexOf lands where the ordering assertions expect. Text pins on workflow YAML match this file's existing house style, so I have no complaint about the technique.

CI test evidence

Fetched once from the head commit's check-runs via the API; no polling. I did not build or execute anything from this PR — the review is static.

The important structural fact: Test (windows-latest, Node 22.x) is skipped on this PR and always will be. test_windows gates on merge_group || schedule || workflow_dispatch (.github/workflows/ci.yml:1771-1778), so the lane this PR rewrites does not run on pull_request events at all. Nothing in this PR's own CI can execute the changed step.

What does carry signal here is the Ubuntu lane, because test:scripts runs scripts/tests/no-ak-integration-ci.test.js — the meta-test this PR edits. That suite is the only automated check on whether the new pins actually match the new YAML, and it is still in_progress at the time of writing. Until it lands green, the pin/step consistency is verified by my reading alone, not by a run.

Final CI results for 0e75a80 (auto-updated by the triage finalize job after CI completed):

Check Conclusion
Classify PR ✅ success
Integration Tests (no-AK, No Sandbox) ✅ success
Lint & Static (ubuntu-latest, Node 22.x) ✅ success
Test (ubuntu-latest, Node 22.x) ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success

One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。

Not verified, and why: the new step's runtime behaviour on an actual Windows runner. The two sandboxed lanes can't close that gap — /tmux drives the TUI, which this change doesn't touch, and /verify does A/B product-behaviour proof against the base build, which measures the CLI rather than workflow YAML. Naming them here would be noise. The lane that would settle it is a workflow_dispatch run of test_windows, which its own if: already admits and which would execute the new step end-to-end on the ecs-win pool — worth doing before this reaches the queue, since the first real execution would otherwise be the merge that ships it.

For what it's worth on the author's side (their claim, not evidence I reproduced): the PR reports the installer/security cases and the Hosted suite passing on a remote Windows host. That is the right thing to have checked, and it's consistent with what I found in the merge-queue log — the Hosted suite passed 9/9 inside the failing job — but I could not independently confirm the installer suite result, because install-script.test.js never executed in any Windows CI run I could read.

Real-scenario testing

N/A — CI-workflow change with no user-visible product surface; there is nothing to drive in a terminal. Per the review rules I also did not execute any PR-derived code.

中文说明

代码审查

在看 diff 之前,我只根据标题和"Why it's needed"先写下了自己的方案:把 Windows step 里的 npm run test:ci 拆成它已有的两半,让任何一方都无法短路另一方;在 set +e 下分别捕获退出码;保留安装/安全套件与 Hosted 套件作为阻断项;把两个累积基线失败的部分降级为 ::warning::,让输出仍留在日志里。这个 PR 做的几乎逐行就是这样,所以我没有更简的替代方案可主张。我长期更希望的形态是——保持套件阻断,另外维护一份约 35 个已知失败文件的 Windows 排除清单——但那需要更多机制、而且会悄悄腐烂,因此更适合作为后续工作,而不是拦下这个 PR 的理由。

shell 逻辑是正确的。set +e 包住了三次调用,之后 set -e 恢复 errexit;每个 $? 都紧跟在对应命令的下一行读取;df/meminfo 转储和最后的 exit 都一致地从 RC 改指到 INSTALLER_RC,没有遗留失效变量。SCRIPTS_RC 和 WORKSPACE_RC 各自被自己的 warning 条件消费,都不是死变量。

两点非阻断观察:

  • npx vitest run scripts/tests/install-script.test.js --config scripts/tests/vitest.config.ts 重新拼了一遍仓库里已有的调用方式。npm run test:scripts -- scripts/tests/install-script.test.js 展开后就是 vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/install-script.test.js——同样的运行,走已有脚本,也不用再维护第二份 config 路径写法。npx 确实会优先解析本地 node_modules/.bin/vitest,所以我不认为今天会出问题,但在本可复用的地方多开了一条并行路径。
  • meta-test 的名字 preserves test failures in every wrapped OS job 现在名不副实,因为 Windows 的失败被保留为 warning 而不是 failure。改个名字(或在 Windows 那一块加一行注释说明这里的失败是有意降为 advisory 的),可以避免后来的读者误以为该 lane 仍会以 workspace 结果作为门禁。scripts/tests/package-assets.test.js:744 那处过时的论证同理。

我核对过新增的断言块能对上实际 YAML:indexOf('set +e') 在 step 正文里没有更早的匹配,set -e 不是 set +e 的子串,两处 ::warning:: 断言是更长 echo 字符串的前缀,所以每个 indexOf 都落在顺序断言所期望的位置。对 workflow YAML 做文本断言与这个文件既有风格一致,手法上我没有意见。

CI 测试证据

通过 API 对 head commit 的 check-runs 一次性抓取,未轮询。我没有构建或执行本 PR 的任何代码——审查是静态的。

关键的结构性事实:Test (windows-latest, Node 22.x) 在本 PR 上是 skipped,而且永远会是。 test_windows 的条件是 merge_group || schedule || workflow_dispatch(.github/workflows/ci.yml:1771-1778),所以这个被本 PR 重写的 lane 在 pull_request 事件下根本不运行。本 PR 自己的 CI 里没有任何东西能执行被改动的 step。

真正有信号的是 Ubuntu lane,因为 test:scripts 会跑 scripts/tests/no-ak-integration-ci.test.js——也就是本 PR 修改的 meta-test。新断言是否真的对上新 YAML,只有这个套件能自动校验,而截至撰写时它仍是 in_progress。在它变绿之前,断言与 step 的一致性只由我的阅读确认,没有运行结果支撑。

CI 表格见上方标记区域(英文部分),此处不重复。

未验证的部分及原因:新 step 在真实 Windows runner 上的运行时行为。两个沙箱 lane 都补不上这个缺口——/tmux 驱动的是 TUI,本次改动不涉及;/verify 做的是相对 base 构建的产品行为 A/B 验证,衡量的是 CLI 而不是 workflow YAML。在这里点名它们只会是噪音。真正能定论的是对 test_windows 触发一次 workflow_dispatch 运行,它自身的 if: 已经允许这个事件,并会在 ecs-win 池上端到端执行新 step——建议在进入队列前做一次,否则第一次真实执行就会是合入它的那次。

关于作者侧的情况(这是作者的说法,不是我复现的证据):PR 称安装/安全用例与 Hosted 套件在远程 Windows 环境通过。这是该检查的地方,也与我在 merge queue 日志里看到的一致——Hosted suite 在那个失败 job 里 9/9 通过——但安装套件的结果我无法独立确认,因为在我能读到的任何 Windows CI 运行中,install-script.test.js 都从未执行过。

真实场景测试

N/A —— 纯 CI workflow 改动,没有面向用户的产品界面,终端里没有东西可驱动。按照审查规则,我也没有执行任何来自本 PR 的代码。

— Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 0e75a801a92a08646811af63ec0b0c9b74fef907 · re-run with @qwen-code /triage

@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Confidence: 4/5 — the mechanism is right and I verified the problem from real logs rather than the description; the nits below are all non-blocking.

Stepping back: this is a small PR that fixes a failure mode I confirmed is real, and it fixes the mechanism rather than the symptom. The tempting version of this change is to wrap npm run test:ci in || echo warning and move on — that would have left the lane green and still blind, because the && inside test:ci means the installer coverage never runs when the workspace half fails. Splitting the chain into the two scripts that already exist is what actually restores the coverage, and the exit code was re-pointed at the installer result consistently, including the diagnostic df/meminfo dump. Reading the merge-queue log made the payoff concrete: 44,259 lines in which the suite this lane exists to run appears zero times.

On maintenance: in six months I'd thank the author rather than curse them. The change adds no abstraction, no new script, and no configuration surface — it reorders three invocations that already existed and pins the new order in the meta-test that was already pinning the old one. That pin was updated in the same commit, which is the part most often forgotten when .github/workflows/ci.yml moves, and I walked the other consumers of test_windows (the cross-leg byte-identity block, the disk-pressure sampler regexes, the runs-on and timeout pins, the guard-step assertions) and none of them go stale, because every edit lands after the sampler sentinel the byte-identity slice stops at.

Everything is necessary for the stated goal — there's no drive-by refactor or formatting churn in here, and the diff is two files.

My reservations, honestly stated, none of them reasons to hold the PR:

The tradeoff the author already named in Risk & Scope is the real one. Windows workspace and script results stop gating, so a new Windows regression among the ~2,100 currently-passing files becomes a warning in a log nobody reads on a green run. That is a genuine loss of coverage bought to recover a lane that was 100% red and blind. I think it's the right call as a temporary state, and it's reversible CI config — but "advisory until their standing baseline failures are repaired" deserves a tracking issue, because advisory states have a way of becoming permanent.

Second, the one suite that can now fail the whole queue is install-script.test.js, which scripts/tests/vitest.config.ts itself documents as flake-prone on Windows under antivirus inspection, and which the Ubuntu lane retries twice for exactly the #10490 "three runs, three disjoint failure sets" reason while this lane has no retry at all. So the gate is now narrower and less flake-tolerant than the one it replaces. That may well be fine in practice — the author reports it passing on a real Windows host — but it's the first thing I'd watch if the queue starts tripping on Windows.

Third, and the reason I'm not approving in this run: the changed step has never executed anywhere I can observe. test_windows is merge_group || schedule || workflow_dispatch, so it is skipped on this PR by construction, and the Ubuntu lane that validates the new meta-test pins was still in_progress when I fetched. My pin-consistency check is a careful reading, not a green run. Approval is therefore deferred until CI lands green on 0e75a801a92a08646811af63ec0b0c9b74fef907; the finalize job will post the commit-pinned approval once every check on that SHA completes, and will withhold it if anything lands red or the head moves.

If a maintainer wants the runtime behaviour settled before it reaches the queue, a workflow_dispatch run of the Windows lane would do it — its own if: admits that event.

中文说明

Confidence: 4/5 —— 机制是对的,而且我是从真实日志里确认了问题存在,不是照抄 PR 描述;下面几点都是非阻断的。

退一步看:这是个小 PR,修的是我确认真实存在的失败模式,而且修的是机制而不是表象。这个改动最容易写成的样子,是把 npm run test:ci 包一层 || echo warning 就完事——那样 lane 会变绿,但依然是瞎的,因为 test:ci 内部的 && 意味着 workspace 那一半失败时安装测试根本不会跑。把链条拆成已经存在的两个脚本,才真正恢复了覆盖面;退出码也一致地改指到安装测试结果上,连诊断用的 df/meminfo 转储都跟着改了。读 merge queue 日志让这个收益变得具体:44259 行里,这个 lane 存在的意义所要跑的套件出现了 0 次。

关于可维护性:六个月后我会感谢作者,而不是骂他。这个改动没有引入抽象、没有新脚本、没有新的配置面——它只是重排了三条本就存在的调用,并在原本就钉住旧顺序的 meta-test 里钉住了新顺序。这个断言在同一个 commit 里就更新了,而这正是 .github/workflows/ci.yml 变动时最常被忘掉的一环。我把 test_windows 的其他消费方都走了一遍(跨 leg 的逐字节一致块、disk-pressure 的 sampler 正则、runs-on 与 timeout 断言、guard step 断言),没有一个会失效,因为所有改动都落在逐字节一致切片所终止的 sampler 哨兵之后。

所有改动都是达成目标所必需的——没有夹带顺手重构或格式化噪音,diff 就两个文件。

我的保留意见,如实说明,都不是拦下 PR 的理由:

作者在 Risk & Scope 里已经点明的取舍,才是真正的那个。Windows 的 workspace 与脚本结果不再作为门禁,所以在当前约 2100 个通过文件里出现新的 Windows 回归时,只会变成一条绿色运行中没人会去看的日志里的 warning。这是为恢复一个 100% 红且盲的 lane 而付出的真实覆盖损失。我认为作为临时状态这是对的取舍,而且是可逆的 CI 配置——但"在既有基线失败修好之前先降为 advisory"这句话值得开一个跟踪 issue,因为 advisory 状态往往会变成永久状态。

第二,现在唯一能让整个队列失败的套件是 install-script.test.js,而 scripts/tests/vitest.config.ts 自己就记录了它在 Windows 杀毒扫描下易 flake;Ubuntu lane 正是出于 #10490"跑三次、失败三组互不相同"的原因给它加了两次 retry,而这个 lane 完全没有 retry。所以门禁既变窄了,抗 flake 能力也比它替换掉的那个更弱。实践中也许没问题——作者称它在真实 Windows 环境通过——但如果队列开始在 Windows 上绊倒,这是我会第一个去看的地方。

第三,也是我在本次运行中不直接 approve 的原因:被改动的 step 在我能观察到的任何地方都还没有执行过。test_windows 的条件是 merge_group || schedule || workflow_dispatch,所以它在本 PR 上按构造就是 skipped;而校验新 meta-test 断言的 Ubuntu lane,在我抓取时仍是 in_progress。我的断言一致性核对是仔细阅读的结果,不是一次绿色运行。因此批准推迟到 CI 在 0e75a801a92a08646811af63ec0b0c9b74fef907 上全绿之后;finalize 任务会在该 SHA 上所有检查完成后发出绑定 commit 的批准,若有任何检查变红或 head 移动则会撤回。

如果维护者希望在进入队列前先把运行时行为定下来,对 Windows lane 触发一次 workflow_dispatch 即可——它自身的 if: 允许该事件。

— Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 0e75a801a92a08646811af63ec0b0c9b74fef907 · re-run with @qwen-code /triage

@qwen-code-review-bot qwen-code-review-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship — CI landed green after the review. ✅

@wenshao wenshao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 0e75a801a92a08646811af63ec0b0c9b74fef907: no new blocking findings.

The split removes the workspace-to-script short circuit, captures each exit status immediately, and preserves the installer result through cleanup and diagnostics. The separate Hosted step remains blocking. I also checked the updated workflow contract and related runner/Hosted/sampler assertions; there are no existing inline review threads to reconcile.

Verification:

  • Executed the exact changed shell block with controlled command stubs on macOS Bash: all 8 workspace/scripts/installer success–failure combinations passed. All three commands ran in order, warnings matched the first two results, and the final status always matched the installer result, including when cleanup/diagnostic stubs failed. This verifies shell control flow, not Windows platform behavior.
  • Independently inspected the Windows dispatch job. Its checkout is this exact SHA and the job succeeded: the dedicated installer run recorded 102 passed / 40 skipped, Hosted recorded 9/9 passed, and both advisory warnings appeared. The installer skips include the existing Get-FileHash case requiring unavailable PowerShell 7 modules. The changed contract suite also passed (12 passed / 2 skipped). The overall dispatch was cancelled in other jobs; this is evidence for the successful Windows job specifically.

The acknowledged tradeoff remains: new failures in the broader Windows workspace/script suites are also advisory. I treat that as the explicit scope of this PR, not a newly discovered implementation defect; the existing follow-up recommendation to restore broader gating still applies.

中文说明

评审提交 0e75a801a92a08646811af63ec0b0c9b74fef907,未发现新增阻断问题。拆分命令消除了 workspace 失败导致脚本套件不执行的短路;退出码捕获、清理和诊断后的状态保留正确,独立的 Hosted 步骤仍能阻断。已核对变更后的契约测试及相关 runner、Hosted、采样断言,目前没有待核对的行内评审线程。

验证包括:在 macOS Bash 中对原始脚本使用受控命令替身,8 种成功/失败组合全部通过,安装套件的失败不会被前序测试、清理或诊断覆盖;这仅验证 shell 控制流。另独立读取了上述 Windows dispatch 日志,确认 checkout 为本次提交,Windows job 成功,安装套件 102 项通过/40 项跳过,Hosted 9/9 通过,两条 advisory warning 均出现,变更的契约测试为 12 项通过/2 项跳过。安装测试的跳过项包含因缺少 PowerShell 7 模块而未运行的 Get-FileHash 用例。整个 dispatch 在其他 job 上被取消,不能据此称整个运行全绿。

已知取舍仍是:更广泛的 Windows workspace/脚本套件中新出现的回归也只会告警。这是 PR 明确选择的范围,不作为新增实现缺陷重复报告;此前关于后续恢复更广门禁的建议仍适用。

@qqqys qqqys left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at head 0e75a801a92a08646811af63ec0b0c9b74fef907 — no historical blocking issues exist on this PR, and an independent Critical-only scan of the diff found none.

1. Historical review state

The only prior review is qwen-code-review-bot APPROVED at this same head (a 63-byte stub). There are no CHANGES_REQUESTED reviews, no review ledger, and zero review threads. reviewDecision is REVIEW_REQUIRED. So there is nothing to re-verify — this is a first substantive pass.

2. The split is lossless: nothing that test:ci ran is dropped

This was the main risk in the change, so I checked it against package.json at this head rather than the PR description:

test:ci              = npm run test:ci:workspaces && npm run test:scripts
test:ci:workspaces   = cross-env NODE_OPTIONS="--max-old-space-size=3072" npm run test:ci --workspaces --if-present --
test:scripts         = vitest run --config ./scripts/tests/vitest.config.ts

test:ci is exactly the two commands the Windows step now runs separately, and both are unchanged in package.json between main and this head. So the Windows lane still executes the identical set of suites — it only stops letting the first one's exit code decide whether the second runs.

That last part is the actual bug being fixed, and it means coverage increases: under &&, a workspace failure short-circuited before test:scripts, so the required Windows installer coverage never executed at all. Now test:scripts always runs, and scripts/tests/install-script.test.js is re-run afterwards as its own blocking gate:

npx vitest run scripts/tests/install-script.test.js --config scripts/tests/vitest.config.ts
INSTALLER_RC=$?

Same config as test:scripts, and both scripts/tests/install-script.test.js and scripts/tests/vitest.config.ts exist at this head. So the installer test moves from "silently skipped whenever the workspace suite was red" to "always runs, always gates".

3. Shell control flow is correct

set +e precedes all three invocations; each exit code is captured on the line immediately following its command (WORKSPACE_RC, SCRIPTS_RC, INSTALLER_RC); set -e is restored before the cleanup; and the step ends on exit "$INSTALLER_RC". The advisory branches cannot mask the blocking one, because the two ::warning paths only echo and never exit or continue. The pkill/kill cleanup and the df//proc/meminfo diagnostics keep their || true guards, so re-enabling set -e cannot abort before exit. The diagnostic dump is now keyed on INSTALLER_RC instead of RC, which matches the value actually being returned.

4. The Linux and macOS gates are untouched

The single ci.yml hunk is at @@ -1928,13 +1928,23 @@, which falls inside the test_windows job (starts line 1768) and after test_macos (1662–1768) and test (367). Both other jobs keep npm run test:ci and exit "$RC".

The test change preserves that: test_windows was removed from the shared ['test', 'test_macos'] loop — whose asserted command list still contains npm run test:ci, RC=$? and exit "$RC" — and given its own ordered list. So the shared assertions still pin the two blocking OS lanes to the old behaviour, and a future edit that weakened them would still fail here.

The new Windows list is a genuine ordering assertion, not a presence check: it walks 15 exact command strings through indexOf and requires each to be strictly later than the previous, so it pins the advisory-before-blocking sequence and would catch a reordering that moved the installer run above set -e or dropped a ::warning branch.

Two details I confirmed rather than assumed. The outer let previous = -1; does not collide with the same declaration inside the for (const jobName of …) loop — that one is block-scoped to the loop body, so this is shadowing, not redeclaration, and Lint & Static passing at this head confirms ESLint accepts it. And there is no earlier set -e in the Windows step that would make indexOf('set -e') resolve to a position before INSTALLER_RC=$? and fail the ordering — the green Linux lane below is direct proof, since that assertion runs there.

5. Independent verification that the assertion matches the real workflow

Test (ubuntu-latest, Node 22.x) ran npm run test:ci at this exact head and passed:

run head=0e75a801a92a08646811af63ec0b0c9b74fef907  conclusion=success
Test (ubuntu-latest, Node 22.x)   pass  31m57s
Lint & Static (ubuntu-latest, Node 22.x)  pass  19m41s
Integration Tests (no-AK, No Sandbox)     pass  12m9s
web-shell E2E Smoke (ubuntu-latest, Node 22.x)  pass  24m52s

scripts/tests/no-ak-integration-ci.test.js is part of test:scripts, so the modified assertion executed on a blocking lane and passed. This matters because the test reads .github/workflows/ci.yml from the working tree — the new Windows assertions are validated by the Linux job even though no Windows job runs them.

6. Disclosed tradeoff, not a blocker

Windows workspace and full script-suite failures become advisory. That is a deliberate reduction of gate strength on one platform, it is stated in the PR description and in Risk & Scope, and it is not silent: both suites still execute in full, their output stays in the log, and each non-zero exit emits a ::warning annotation. Given the alternative was a lane where every queued change went red on inherited baseline failures — and where the genuinely required installer test never ran at all — this is a net gain in signal. The right follow-up is repairing those baselines so the two suites can block again; that is already scoped out here.

Also non-blocking: Test (windows-latest, Node 22.x) and Test (macos-latest, Node 22.x) are both skipping on this PR, so the changed shell step is not exercised by this PR's own CI run. Its behaviour is verified indirectly (§2 script equivalence, §5 the assertion that pins it) plus the author's stated manual Windows run. Worth a look at the first merge-queue run to confirm the advisory branches emit and the installer gate holds.

The failed checks at this head — review-pr, delay-automatic-review, fallback-comment — are review-bot infrastructure lanes, not build or test lanes, and are not attributable to this change.

@yiliang114
yiliang114 merged commit c3012a5 into main Oct 16, 2026
103 of 109 checks passed
@wenshao

wenshao commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

Verdict: merge-ready — 68/68 scripted assertions passed, 0 failed. Verified head 0e75a801a92a08646811af63ec0b0c9b74fef907 against base ac92a0f699186e00d40a032e0b07b8e362b6e306 (merge-base 476f624b53).

Post-merge note (2026-10-16 10:35 Beijing): this PR was merged while this verification round was running. The landed squash commit c3012a523fcd4c2b7d84f3683b2b2896dbbf98b5 carries a test_windows step byte-identical to the verified head (checked against current origin/main), so every result below applies to what shipped. Two reviews also landed at the verified head during the round (wenshao: no new blocking findings; qqqys: APPROVED) — consistent with this report's outcome. Posting as post-merge verification evidence of record.

中文摘要

结论:merge-ready——68/68 条脚本化断言全部通过。验证期间 PR 已于 2026-10-16 10:35(北京时间)被合入(squash 提交 c3012a523f),合入内容与被验证 head 的 windows 步骤逐字节一致,本报告全部结论适用于已合入代码。

  • A/B 结论(核心):将 base / 第 1 个提交 / head 三个版本的 test_windows 测试步骤原文提取后在 bash --noprofile --norc 下重放(npm/npx 用 PATH stub),4 个场景矩阵证实:base 在 workspace 基线失败时整个 job 变红且 test:scripts(含安装/安全覆盖)根本不会启动;head 下 workspace/scripts 失败只产生 ::warning::,安装套件命令仍会执行,且只有它失败时步骤才退出 1。这正是 PR 宣称的行为。
  • 生产日志佐证:失败的 merge_group 运行 38712612535 的 Windows job 完整日志(44,259 行)中,test:scripts 启动命令出现 0 次、install-script 出现 0 次、测试步骤退出码 1,而独立的 Hosted 套件步骤照常运行并 9/9 通过——与 PR 描述的问题机制完全吻合。
  • 新门禁命令实跑:npx vitest run scripts/tests/install-script.test.js --config scripts/tests/vitest.config.ts 在已构建的 head 树上真实执行(macOS),131 passed / 11 skipped,退出码 0;缺失构建产物或测试文件不存在时均 fail-closed(退出 1)。此前 triage 机器人无法确认该命令在任何 CI 中执行过——本轮补上了执行证据。
  • 契约测试 vacuity 检查:head 的新断言对 base workflow 变红(test_windows: npm run test:ci:workspaces: expected -1),base 旧断言对 head workflow 变红(exit "$RC" 不匹配)——双向都非空转。
  • 试合并:对当前 main(4d7294d4fd)git merge-tree 无冲突,合并树的 windows 步骤与 head 逐字节一致。
  • Findings:无阻断项。三条非阻断观察(安装套件在 Windows lane 会跑两遍;meta-test 名称与 package-assets.test.js:744 注释的表述已过时;df/meminfo 诊断转储改为仅在安装套件失败时触发)见正文。
  • 未覆盖:真实 Windows runner 上的端到端执行(本 lane 按设计不在 pull_request 上运行;如需首发前实测,可由维护者在 base 仓临时分支上 workflow_dispatch)。Hosted 套件步骤未被本 PR 改动,其阻断性通过代码阅读+生产日志验证,未在本地重跑。

Central claim and A/B proof

Claim under test (from the PR body): the Windows lane still runs the full workspace and script suites but reports their existing failures as warnings, while the Windows installer/security tests and the Hosted process suite remain the blocking merge gate.

Method: the Run tests and generate reports step body of test_windows was extracted verbatim from three revisions — base (ac92a0f699), commit 1 (2b05ee9438), head (0e75a801a9) — using the same slicing algorithm as the repo's own workflow-helpers.js, then executed under bash --noprofile --norc (the GitHub Actions bash contract) with npm/npx intercepted by PATH stubs that log every invocation and exit with scenario-driven codes. The run block's own commands (node, git config, the DFSAMPLE sampler, pkill/kill) all really executed; only the two test-launching binaries were stubbed.

scenario (stub exit codes) base commit 1 head
S1: workspaces fail, scripts fail, installer passes exit 1 — test:ci blocks; scripts/installer never start exit 1 — workspace warned, scripts still block exit 0 — 2 ::warning:: annotations, installer ran and passed
S2: everything green exit 0 exit 0 exit 0, no warnings
S3: installer-only red (ws+scripts green) exit 0 — no independent installer gate exists (installer lives inside test:scripts) exit 0 — same; the installer command is never invoked exit 1 — the new gate bites; df/meminfo diagnostics dumped
S4: scripts-only red (ws green, installer green) exit 1 exit 1 — commit 1 still red here, motivating commit 2 exit 0 — 1 warning; installer still executed after the scripts failure

33/33 scripted assertions passed across the 11 cells (exit codes, exact stub call sequences, warning counts and texts, diagnostics presence). The extracted step bodies are archived as run-base.bash / run-commit1.bash / run-head.bash.

A/B step replay matrix

Base defect mechanism, proven with the real artifact: test:ci is npm run test:ci:workspaces && npm run test:scripts (base package.json, byte-identical to head's). Running the real npm against the real base package.json with the inner npm PATH-stubbed: when workspaces exit 1, the stub log shows run test:ci:workspaces and never run test:scripts (7/7 assertions). This is the "chain stopped before reaching the Windows installer coverage" mechanism from the PR description, demonstrated rather than asserted.

npm chain early stop

Production corroboration (census over the failing merge-queue run 38712612535, Windows job, failed 2026-10-15 19:53 Beijing): across all 44,259 log lines, the scripts-suite command line appears 0 times and install-script appears 0 times — the installer/security coverage indeed never executed in the failing job; the test step exited 1 on workspace baseline failures (109 + 25 failed tests across two suite groups); the separate Hosted process suite step still ran afterwards and passed 9/9. This matches the PR's problem statement exactly.

Contract test (scripts/tests/no-ak-integration-ci.test.js)

  • head test file vs head workflow: 12 passed, 2 skipped (the 2 skips are it.runIf(linux) cases; macOS host).
  • Vacuity: same head test file vs base workflow → fails exactly at the new pin: test_windows: npm run test:ci:workspaces: expected -1 to be greater than 2416, exactly 1 test failed (clean attribution — base and head ci.yml differ only in this step).
  • Reverse cell: base test file vs head workflow → fails at test_windows: exit "$RC". Note the substring fuzziness this exposes: the old pins npm run test:ci and RC=$? match inside npm run test:ci:workspaces and WORKSPACE_RC=$?, so the old contract only rejects the new shape at its final exit "$RC" pin. Not a defect — the new pin list does not have this property since it pins the full distinct strings.

Wiring test vacuity

The new blocking gate command, executed for real

The triage review noted it "could not independently confirm the installer suite result, because install-script.test.js never executed in any Windows CI run". This round executed the exact gate command on the built head tree (macOS 26, arm64):

  • npx vitest run scripts/tests/install-script.test.js --config scripts/tests/vitest.config.ts → exit 0, 131 passed / 11 skipped (142).
  • Fail-closed probes: with build artifacts absent (packages/audio-capture/dist missing) the same command exits 1 with ENOENT; with a nonexistent test-file argument vitest exits 1 (No test files found). A typo'd path or a broken build can never render this gate silently green.

Installer gate on built head tree

Corrections to earlier review-round descriptions

None required — the triage bot's static analysis was accurate on every point this round measured: the lane's pull_request skip, the 9/9 Hosted result in the failing merge-queue job, and the absence of any prior install-script.test.js execution in CI (all independently confirmed above). The bot's suggested npm run test:scripts -- scripts/tests/install-script.test.js alternative does expand to the equivalent invocation, but the PR's explicit npx spelling was executed live this round and works; either is fine.

Findings (all non-blocking)

  1. Suggestion — installer coverage now runs twice in the Windows lane: once inside the advisory test:scripts and once as the blocking standalone command (verified: install-script.test.js is inside the scripts config's include and not in its win32 exclude). The duplication is the price of keeping full scripts output visible while gating only on the installer suite; a future cleanup could exclude the file from the scripts config on win32 and rely on the standalone run.
  2. Nit — two stale texts (both already flagged in triage, confirmed present at head): the meta-test name preserves test failures in every wrapped OS job now over-claims for Windows (failures there are advisory by design), and the comment at scripts/tests/package-assets.test.js:744 ("go red in the test_windows lane (which runs test:scripts)") describes pre-PR behavior — post-PR that lane warns instead of going red for scripts failures.
  3. Observation — diagnostics re-scoped: the df/meminfo forensics dump now fires only on installer failure; workspace/scripts failures no longer trigger it. The 10-second DFSAMPLE sampler output remains in the log regardless, so most forensics value is preserved. This matches the diff's intent; no action.

Targeted gates

  • bash -n clean on all three extracted step bodies; shellcheck -S warning clean on base and head blocks (exit 0).
  • actionlint clean on both workflow versions; a deliberately corrupted copy is flagged (positive control proves the gate is live).
  • Trial merge onto current origin/main (4d7294d4fd): git merge-tree clean, and the merged tree's test_windows step is byte-identical to head's. Main's post-merge-base changes touch neither this step nor the test:ci chain.

Not covered

  • Real Windows runner execution of the new step. The replay host was macOS; the Windows-only utilities in the block are all guarded no-ops, and the control flow under test (set +e, $? capture, ::warning:: emission, exit-code selection) is POSIX-identical under Git Bash — but the first end-to-end execution on the ecs-win pool will still be the merge-queue run that ships it. The lane is merge_group/schedule/workflow_dispatch-only by design, so this cannot be exercised by the PR's own CI. If a pre-merge live run is wanted, a maintainer can push the head to a temporary base-repo branch and workflow_dispatch ci.yml against it.
  • Hosted process suite execution: the step is unchanged by the PR; its blocking semantics were verified by reading (no continue-on-error, hard process.exit(1) assertion) plus the production log (9/9 in the failing run). Not re-executed locally — it needs a full integration bundle and adds nothing about this diff.
  • yamllint: not installed on this machine; YAML validity is covered by actionlint's parse plus the positive control.
  • The workspace and full scripts suites themselves (the advisory arms): not re-run locally. Their Windows baseline failures are pre-existing and explicitly out of the PR's scope; the Ubuntu lane (where the same suites and the edited meta-test run blocking) is green on this PR's CI.

Methodology

Round ran 2026-10-16 ~10:23–11:05 Beijing on macOS 26 (arm64), Node 24.18.1. Worktrees at tmp/pr14298-base (ac92a0f699) and tmp/pr14298-head (0e75a801a9) with lockfile-identical node_modules clone-copied (APFS) after verifying pnpm-lock.yaml hashes match; head tree additionally built with npm run build (exit 0) for the installer-gate run. All harnesses (harness/ab-step-replay.mjs, harness/npm-chain-proof.sh, harness/wiring-test-cells.sh, harness/final-checks.sh), the extracted step bodies, per-cell logs, and the raw merge-group log are archived in the run artifacts directory tmp/pr14298-verify-20261016-102310/. Assertion ledger: 33 (A/B replay) + 7 (npm chain) + 11 (wiring cells) + 17 (installer/lint/merge/census) = 68 pass, 0 fail; every expected-failure cell (base reds, vacuity reds, fail-closed probes) is encoded as an expected outcome inside its harness. CI red jobs review-pr/delay-automatic-review on the triage workflow were cancelled (infrastructure), unrelated to this diff.

yiliang114 added a commit to yiliang114/qwen-code that referenced this pull request Oct 16, 2026
Clears the "Check lint gate freshness" failure in Lint & Static: main moved
eslint.config.js (28415dd, QwenLM#14277) and .github/workflows/ci.yml
(c3012a5, QwenLM#14298) after this branch's baseline, so the lane was checking
out a stale gate definition. Pure mechanical merge, no hand edits: the merged
tree is byte-identical to `git merge-tree --write-tree` and the PR delta is
unchanged (24 files, +1773/-51).

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-conflict/jmvaik7ci93
@qwen-code-review-bot

Copy link
Copy Markdown
Collaborator

Released in v0.25.3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants