Skip to content

[Sync] Update rollout, training and engine integrations - #470

Merged
CalvinXKY merged 48 commits into
vllm-project:mainfrom
aoshen02:codex/pr441-post-merge-audit-20261012
Nov 2, 2026
Merged

CalvinXKY merged 48 commits into
vllm-project:mainfrom
aoshen02:codex/pr441-post-merge-audit-20261012

Conversation

@aoshen02

@aoshen02 aoshen02 commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Candidate and scope

This PR is open for review (not a draft). The validated source candidate is 84a012353b689dc086c1f425859b3f5e692a9542; the subsequent review change only removes the temporary Agent CI hostname selector. Release preparation remains separate from review readiness; official image promotion happens after merge and dual-architecture verification.

Native vLLM: 83c078d955cd6e0c63d3b12abe669c8de936a370.
The ten selected vLLM patches replay to engine tree 21e3947a38e0a179019ee25aa2ecfff2666c7b6b.

Simplification and mirroring

  • Reuse native pause-abort, collect pending/partial responses, resume, then generate. Remove the redundant queue diagnostics API and repeated abort/load polling.
  • Restore Mooncake for the original PD tests and Router configuration. Remove NIXL peer-retirement, producer-handoff and host-cache-ratio patches; retain native DP-rank selection and absolute host-cache capacity.
  • Retain generation-aware health for startup, monitoring and cleanup, using native sleeping/scheduler replies (fork #128).
  • Simplify engine/deployment/external control and restore reviewed source code, comments, documentation and example configuration.
  • Restore Geo3K lifecycle, budget precedence, terminal status, retained abort tokens and final masked observation; preserve native render/TITO, canonical generated IDs and multimodal feature reuse.
  • Consolidate Python metrics and tracing around the same remote-KV wait measurement (fork #160); worker timing/bytes remain distinct.

All downstream vLLM Rust frontend adaptations remain withdrawn. Independent Router/Sunabako remain. Independent Mamba pools, optimistic PD, empty-PP/circular-admission extensions and EC122/125 are excluded. P/D retain visual encoders and native fallback. Separate release/MemAgent changes are outside this PR.

Audit and local validation

  • Whole-tree evidence index: 718 Vime files and 87 source-only dispositions, with no missing routes or hash drift. Of the 718: 302 exact translations, 309 unchanged byte/source-bound prior reviews, 77 current full-file reviews and 30 reviewed deltas. This is not 718 fresh manual full reads.
  • Changed-file formatting checks and patch replay pass. CPU reports retain their individual scopes rather than presenting overlapping checks as one test count.
  • Installed-image four-H200 Mooncake recovery passes: replace P while retaining D/router/serving, restart trainer TP1→TP2, retain replay digest, transfer real KV and generate after recovery.
  • Six-H200 external PD full/delta updates pass with positive native transfer counters and zero recorded transfer failures.
  • Installed-image native pause-abort/collection/resume passes at DP1/DP2, including real-Router partial/nonpartial collection and next generation.
  • Original single-H200 Geo3K passes with pinned model/data, two turns and unchanged 1024-token budget; tokens, features, observation masks/logprobs, terminal status and cleanup are checked.
  • Original four-GPU PipelineRL disk/periodic-flush test passes without an overlay.
  • Full H200 Agent E2E passes: 16 agent runs, 227 tool calls, two nonzero-gradient optimizer steps, changed parameters and cleanup. The local sandbox uses RSS test mode, not certified hard-cgroup enforcement.
  • The original eight-GPU Qwen3-4B fully-async example passes; the Qwen3.5-9B Ray job completes, but its global cleanup check detects an unrelated user's process. Both have zero gradients and are operational checks, not learning evidence.

CI acceptance

Latest-head CPU validation: 1824 passes all seven automatic checks at 92fb7395a01fef90f4df90e378aa591231338e28, using experimental image aosheninferact/vime@sha256:302194f14b9b50a15d7af87f9e39d56a3ae62658720d3065f2d5824f699d394d. This includes all 95 score-centering transport tests. Automatic build 1823 used the default latest instead and failed the native TITO API contract test; no test assertion or production code was changed to obtain the passing rerun. The GPU gate remains unopened in 1824, so this is CPU validation, not a new GPU matrix. Separate NPU build 539 fails before tests because its upload command expects .buildkite/pipeline-npu.yaml, which exists on ascend but not on this PR or main; that pipeline routing issue remains unresolved.

Historical combined coverage: 57/57 GPU configurations and 7/7 CPU checks, using unchanged-case reuse plus affected-only reruns. This is not a fresh full matrix on current HEAD, and the unpinned Agent scheduling change has not been GPU-tested yet.

Evidence Reused or rerun Coverage
1799 Reused unchanged passing cases 53 GPU configurations; excludes its failed PipelineRL and Agent cases
1803 Affected-only rerun All 3 PipelineRL configurations
1807 Affected-only rerun 1 Agent GPU configuration plus 7 CPU checks, at 84a01235 with the temporary hostname selector

Reused GPU cases by suite: short 2, vllm-config 4, megatron 36, vime-customized 5, precision 1, and ckpt 5 (53 total). The three rerun megatron cases are test_qwen2.5_0.5B_pipeline_rl.py with --transport nccl, --transport disk, and --transport nccl --flush-cache-interval 2. The rerun Agent case is test_agent_sunabako_codex_e2e.py. The seven CPU checks are pre-commit, docs lab generator, docs lab browser, plugin contracts & CPU tests, agent adapter tests, synchronized upstream CPU tests, and utils tests.

  • 1799: full matrix; reuse unchanged passing cases.
  • 1803: all three affected PipelineRL configurations pass after updating the helper to native terminal weight-version spans. Continuity/version-change/flush assertions remain.
  • 1807: Agent GPU E2E and seven CPU checks pass on 84a01235, before removing the hostname selector. Actual H100 results show 16 agent runs, two nonzero-gradient steps and changed parameters.

This is combined coverage, not a fresh full-matrix run at HEAD. The later changes are test/CI-only; runtime, engine patch bytes and dependencies remain unchanged. CI uses immutable experimental runtime aosheninferact/vime@sha256:b4cd30a95b415a237880d99887ced750e8cbb941eeda7710e39755316386f529 with the explicit Buildkite checkout.

H100 adaptations and the reviewer-requested cleanup:

  • Agent TP4/PP2 replaces the source's H20-96GB TP2/PP4. Eight GPUs, samples, loss, precision and assertions remain unchanged; no training-core optimization is adopted.
  • The temporary Agent-only inf-4x8h100-1 hostname selector is removed. Agent jobs use the normal H100 pool; Bash, sandbox privileges, timeout, artifacts and training configuration are unchanged. Generated-step comparison verifies that removing the hostname selector is the only scheduling/configuration delta. The earlier Agent pass does not establish sandbox capability on every pool node.

Historical failures remain recorded: 1799 Agent backward OOM; 1805 lost agent and namespace EPERM before training on other H100 nodes; canceled default-image builds 1804/1806. These are not current failures, nor are they reclassified as passes.

Dual-architecture preparation

After CI acceptance, the current ARM three-patch Router build, installed source/engine verification, DP1/DP2 pause/resume, real Router collection and hybrid-model sleep/wakeup checks pass. The initial hybrid harness lacked a multiprocessing main guard; its corrected rerun passes without an engine change. Native vLLM ELF parity and unchanged FA/TE binaries/dependency versions are verified on both architectures.

AMD64 final-source experimental image: aosheninferact/vime@sha256:302194f14b9b50a15d7af87f9e39d56a3ae62658720d3065f2d5824f699d394d.
Stale inherited provenance labels were normalized; filesystem layers and every non-label image-config field are verified unchanged from the tested source-only image. FA2/FA3/TE were not recompiled.

ARM64 final-source experimental image has also been uploaded: aosheninferact/vime@sha256:5e0c05795d47b5d482e8619af268d1812b78a791ba00963dc3269baaa63e19e1. Both final-source images correspond to 84a01235; the later hostname removal only affects CI scheduling, not installed runtime code. The extra ARM Straw CPU validation and dual-architecture experimental manifest remain pending. No official tag or completed sync cutoff is advanced.

AI assistance used; original author/DCO attribution is preserved.

Preserve Vime Docker-only installation, pinned environment metadata and native serving configuration checks.
Signed-off-by: aoshen02 <aoshen@inferact.ai>
@read-the-docs-community

read-the-docs-community Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly enhances the vime documentation and interactive experiment builder by introducing an RL walkthrough, an A/B configuration comparison tool, automated resource repair options, and build provenance tracking. The review comments provide valuable, actionable feedback to improve robustness, such as adding defensive checks for DOM elements and file reading, fixing a branding inconsistency where 'slime' was not updated to 'vime', and preventing potential unpacking errors in the test routing logic.

Comment thread docs/conf.py
*root.glob("scripts/models/*.sh"),
root / "docs/_static/js/experiment-engine.js",
]
docker_args = dict(re.findall(r"^ARG (\w+)=(\S+)", (root / "docker/Dockerfile").read_text(), re.MULTILINE))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Reading docker/Dockerfile directly without checking if the file exists or wrapping it in a try-except block can cause the documentation build to crash in environments where the Dockerfile is missing (e.g., when building from a clean source distribution or archive). Wrapping it or checking for file existence makes the build more robust.

        docker_file = root / "docker/Dockerfile"
        docker_args = {}
        if docker_file.is_file():
            try:
                docker_args = dict(re.findall(r"^ARG (\w+)=(\S+)", docker_file.read_text(), re.MULTILINE))
            except OSError:
                pass

toastTimer, parallelView = 'tp',
cpView = 'zigzag';
const hashParams = new URLSearchParams(location.hash.slice(1));
const build = JSON.parse($('experiment-build').textContent);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Accessing $('experiment-build').textContent directly assumes the element always exists and has content. If the element is missing or empty in some contexts or custom templates, this will throw a TypeError or SyntaxError and crash the script. Adding a defensive check prevents potential runtime crashes.

Suggested change
const build = JSON.parse($('experiment-build').textContent);
const buildEl = $('experiment-build');
const build = buildEl ? JSON.parse(buildEl.textContent || '{}') : {};

Comment thread docs/_static/js/experiment-diagrams.js Outdated
`<div class="external-map" data-topology="external"><svg viewBox="0 0 400 280" preserveAspectRatio="none" aria-hidden="true"><path d="M80 140 Q180 10 315 50 M80 140 Q220 100 325 180 M80 140 Q140 280 210 260"/></svg><div class="external-trainer">${trainer}<small>${t('vime training job', 'vime 训练任务')}</small></div><div class="external-cluster cluster-a">${serving(t('Cluster A', '集群 A'))}</div><div class="external-cluster cluster-b">${serving(t('Cluster B', '集群 B'))}</div><div class="external-cluster cluster-c">${serving(t('Cluster …', '集群 …'))}</div></div><p class="diagram-caption">${t('Independently managed fleets, connected by endpoint. Locations are illustrative; external engines may also be in one datacenter.', '通过 endpoint 连接独立管理的 GPU 集群。位置仅为示意;external 引擎也可以位于同一机房。')}</p>`;
} else {
topology =
`<div data-topology="${s.layout}" class="${s.layout === 'colocate' ? 'world-colocate' : 'managed-pools'}"><div class="world-nodes">${trainer}<div class="flow-arrow">${s.schedule === 'async' ? '⇄' : '⇢'}</div>${serving('Rollout', v.rollout)}</div><span class="pool-label">${s.layout === 'colocate' ? t('one GPU pool · alternating phases', '同一 GPU 池 · 交替工作') : t('slime manages two dedicated GPU pools', 'slime 统一管理 · 两个独立 GPU 池')}</span></div>`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There is a branding/translation oversight here where the framework name slime was not updated to vime in the pool label, unlike other places in the diagrams (e.g., line 24). Updating it ensures consistent branding across the interactive diagrams.

Suggested change
`<div data-topology="${s.layout}" class="${s.layout === 'colocate' ? 'world-colocate' : 'managed-pools'}"><div class="world-nodes">${trainer}<div class="flow-arrow">${s.schedule === 'async' ? '⇄' : '⇢'}</div>${serving('Rollout', v.rollout)}</div><span class="pool-label">${s.layout === 'colocate' ? t('one GPU pool · alternating phases', '同一 GPU 池 · 交替工作') : t('slime manages two dedicated GPU pools', 'slime 统一管理 · 两个独立 GPU 池')}</span></div>`;
`<div data-topology="${s.layout}" class="${s.layout === 'colocate' ? 'world-colocate' : 'managed-pools'}"><div class="world-nodes">${trainer}<div class="flow-arrow">${s.schedule === 'async' ? '⇄' : '⇢'}</div>${serving('Rollout', v.rollout)}</div><span class="pool-label">${s.layout === 'colocate' ? t('one GPU pool · alternating phases', '同一 GPU 池 · 交替工作') : t('vime manages two dedicated GPU pools', 'vime 统一管理 · 两个独立 GPU 池')}</span></div>`;

Comment on lines +615 to +617
if url_path.startswith("/projects/vime/"):
_, _, _, doc_language, _, suffix = url_path.split("/", 5)
relative = ("zh/" if doc_language == "zh" else "") + suffix

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Unpacking url_path.split("/", 5) into exactly 6 variables will raise a ValueError if the URL path has fewer than 5 slashes (e.g., /projects/vime/ or /projects/vime/zh/). Checking the length of the split parts first makes the routing logic much more robust and prevents test crashes.

Suggested change
if url_path.startswith("/projects/vime/"):
_, _, _, doc_language, _, suffix = url_path.split("/", 5)
relative = ("zh/" if doc_language == "zh" else "") + suffix
if url_path.startswith("/projects/vime/"):
parts = url_path.split("/")
if len(parts) >= 6:
doc_language = parts[3]
suffix = "/".join(parts[5:])
relative = ("zh/" if doc_language == "zh" else "") + suffix
else:
relative = url_path.removeprefix(prefix)

Signed-off-by: aoshen02 <aoshen@inferact.ai>
@aoshen02 aoshen02 changed the title [Sync][Docs] Mirror Slime homepage updates #2464 and #2465 [Sync][Docs] Refresh homepage and experiment guidance Oct 18, 2026
Preserve native engine adapters and ROCm integration while refreshing translated runtime, tests, examples and dependency bindings.

Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
@aoshen02
aoshen02 marked this pull request as draft October 18, 2026 11:41
@aoshen02 aoshen02 changed the title [Sync][Docs] Refresh homepage and experiment guidance [Sync] Update rollout, training and engine integrations Oct 18, 2026
aoshen02 and others added 20 commits October 18, 2026 11:49
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Preserve the native engine launch and CI adapters while carrying checkpoint rewind, separate Python launch, process retirement, documentation, and test updates.

Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Preserve source callback selection, optional dependency handling, and bootstrap tests without changing the approved NPU exclusion.

Co-authored-by: Blink <zpinghu@foxmail.com>

Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
@CalvinXKY

Copy link
Copy Markdown
Collaborator

Please align the PR description with current status (it still reads like a draft / no-latest candidate). Also clarify that “57/57” is unchanged-case reuse + affected-only reruns, not a fresh full matrix on HEAD. For a sync this large, either rerun the required GPU matrix on HEAD, or list reused vs rerun suites (with build links).

@CalvinXKY

Copy link
Copy Markdown
Collaborator

Please remove the temporary Agent E2E hard-pin to kubernetes.io/hostname=inf-4x8h100-1 in .buildkite/gpu_suites.py
(or make it an explicit, time-bounded override). That’s a code change, not just a docs tweak — otherwise mainline CI stays tied to one node.

@CalvinXKY

Copy link
Copy Markdown
Collaborator
image

@aoshen02

aoshen02 commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

Please align the PR description with current status (it still reads like a draft / no-latest candidate). Also clarify that “57/57” is unchanged-case reuse + affected-only reruns, not a fresh full matrix on HEAD. For a sync this large, either rerun the required GPU matrix on HEAD, or list reused vs rerun suites (with build links).

Done

@aoshen02

aoshen02 commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

Please remove the temporary Agent E2E hard-pin to kubernetes.io/hostname=inf-4x8h100-1 in .buildkite/gpu_suites.py (or make it an explicit, time-bounded override). That’s a code change, not just a docs tweak — otherwise mainline CI stays tied to one node.

Done.

@aoshen02

aoshen02 commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author
image

Done.

@CalvinXKY
CalvinXKY merged commit 6845501 into vllm-project:main Nov 2, 2026
3 of 4 checks passed
@aoshen02
aoshen02 deleted the codex/pr441-post-merge-audit-20261012 branch November 2, 2026 01:37
p40www added a commit to p40www/vime that referenced this pull request Nov 16, 2026
…fix, and MoE test suite

- Dispatch the PD KV connector through the platform layer: GPU defaults to
  MooncakeConnector (aligned with main vllm-project#470), NPU overrides to
  MooncakePullConnector with kv_port and a slot-based port offset.
- Fix the cross-step weight-sync hang: a paused scheduler discards its empty
  connector output, so Mooncake pending transfers are never consumed and the
  engine never reaches the idle state (vllm-pd-scheduler-fallback.patch).
- Add colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister
  CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes
  (vllm-ascend-pd-sleep-wake.patch).
- Add Qwen3-30B-A3B PD test suite (test_pd_npu_suite.py + test_pd_npu.py):
  nopa / core(1P1D) / pd_1p3d(1P3D) variants, all colocate TP4+EP8.

Signed-off-by: pansiling <1428273483@qq.com>
p40www added a commit to p40www/vime that referenced this pull request Nov 16, 2026
…fix, and MoE test suite

- Dispatch the PD KV connector through the platform layer: GPU defaults to
  MooncakeConnector (aligned with main vllm-project#470), NPU overrides to
  MooncakePullConnector with kv_port and a slot-based port offset.
- Fix the cross-step weight-sync hang: a paused scheduler discards its empty
  connector output, so Mooncake pending transfers are never consumed and the
  engine never reaches the idle state (vllm-pd-scheduler-fallback.patch).
- Add colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister
  CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes
  (vllm-ascend-pd-sleep-wake.patch).
- Add Qwen3-30B-A3B PD test suite (test_pd_npu_suite.py + test_pd_npu.py):
  nopa / core(1P1D) / pd_1p3d(1P3D) variants, all colocate TP4+EP8.

Signed-off-by: pansiling <1428273483@qq.com>
p40www added a commit to p40www/vime that referenced this pull request Nov 16, 2026
…fix, and MoE test suite

- Dispatch the PD KV connector through the platform layer: GPU defaults to
  MooncakeConnector (aligned with main vllm-project#470), NPU overrides to
  MooncakePullConnector with kv_port and a slot-based port offset.
- Fix the cross-step weight-sync hang: a paused scheduler discards its empty
  connector output, so Mooncake pending transfers are never consumed and the
  engine never reaches the idle state (vllm-pd-scheduler-fallback.patch).
- Add colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister
  CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes
  (vllm-ascend-pd-sleep-wake.patch).
- Add Qwen3-30B-A3B PD test suite (test_pd_npu_suite.py + test_pd_npu.py):
  nopa / core(1P1D) / pd_1p3d(1P3D) variants, all colocate TP4+EP8.

Signed-off-by: pansiling <1428273483@qq.com>
p40www added a commit to p40www/vime that referenced this pull request Nov 16, 2026
…fix, and MoE test suite

- Dispatch the PD KV connector through the platform layer: GPU defaults to
  MooncakeConnector (aligned with main vllm-project#470), NPU overrides to
  MooncakePullConnector with kv_port and a slot-based port offset.
- Fix the cross-step weight-sync hang: a paused scheduler discards its empty
  connector output, so Mooncake pending transfers are never consumed and the
  engine never reaches the idle state (vllm-pd-scheduler-fallback.patch).
- Add colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister
  CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes
  (vllm-ascend-pd-sleep-wake.patch).
- Add Qwen3-30B-A3B PD test suite (test_pd_npu_suite.py + test_pd_npu.py):
  nopa / core(1P1D) / pd_1p3d(1P3D) variants, all colocate TP4+EP8.

Signed-off-by: pansiling <1428273483@qq.com>
p40www added a commit to p40www/vime that referenced this pull request Nov 16, 2026
- Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset (TP*DP*PP*PCP).
- Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state. Folded into docker/npu_patch/vllm.patch.
- Add the colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes. Folded into docker/npu_patch/vllm-ascend.patch.
- Add Qwen3.5-35B-A3B NPU PD tests: tests/test_qwen3.5_35B_A3B_pd_npu.py (8-NPU colocate smoke with --check-weight-update-equal) and tests/test_qwen3.5_35B_A3B_pd_parallel_check.py (nopa / 1P1D TP4 / 1P3D TP2 variants).
- Update tests/utils/test_vllm_engine.py for the new MooncakeConnector default.

Signed-off-by: pansiling <1428273483@qq.com>
p40www added a commit to p40www/vime that referenced this pull request Nov 17, 2026
- Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset (TP*DP*PP*PCP).
- Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state. Folded into docker/npu_patch/vllm.patch.
- Add the colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes. Folded into docker/npu_patch/vllm-ascend.patch.
- Guard two PD baselines in the NPU smk suite: tests/test_qwen3.5_35B_A3B_pd_npu.py (MoE colocate, 1P1D TP4, 8 NPU) and tests/test_qwen3_0.6B_pd_npu.py (dense non-colocate, one prefill plus two decode engines, 4 NPU).
- Update tests/utils/test_vllm_engine.py for the new MooncakeConnector default.

Signed-off-by: pansiling <1428273483@qq.com>
p40www added a commit to p40www/vime that referenced this pull request Nov 17, 2026
- Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset (TP*DP*PP*PCP).
- Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state. Folded into docker/npu_patch/vllm.patch.
- Add the colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes. Folded into docker/npu_patch/vllm-ascend.patch.
- Guard two PD baselines in the NPU smk suite: tests/test_qwen3.5_35B_A3B_pd_npu.py (MoE colocate, 1P1D TP4, 8 NPU) and tests/test_qwen3_0.6B_pd_npu.py (dense non-colocate, one prefill plus two decode engines, 4 NPU).
- Update tests/utils/test_vllm_engine.py for the new MooncakeConnector default.

Signed-off-by: pansiling <1428273483@qq.com>
p40www added a commit to p40www/vime that referenced this pull request Nov 17, 2026
- Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset (TP*DP*PP*PCP).
- Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state. Folded into docker/npu_patch/vllm.patch.
- Add the colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes. Folded into docker/npu_patch/vllm-ascend.patch.
- Guard two PD baselines in the NPU smk suite: tests/test_qwen3.5_35B_A3B_pd_npu.py (MoE colocate, 1P1D TP4, 8 NPU) and tests/test_qwen3_0.6B_pd_npu.py (dense non-colocate, one prefill plus two decode engines, 4 NPU).
- Update tests/utils/test_vllm_engine.py for the new MooncakeConnector default.

Signed-off-by: pansiling <1428273483@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants