Repository navigation
Conversation
Preserve Vime Docker-only installation, pinned environment metadata and native serving configuration checks. Signed-off-by: aoshen02 <aoshen@inferact.ai>
There was a problem hiding this comment.
Code Review
This pull request significantly enhances the vime documentation and interactive experiment builder by introducing an RL walkthrough, an A/B configuration comparison tool, automated resource repair options, and build provenance tracking. The review comments provide valuable, actionable feedback to improve robustness, such as adding defensive checks for DOM elements and file reading, fixing a branding inconsistency where 'slime' was not updated to 'vime', and preventing potential unpacking errors in the test routing logic.
| *root.glob("scripts/models/*.sh"), | ||
| root / "docs/_static/js/experiment-engine.js", | ||
| ] | ||
| docker_args = dict(re.findall(r"^ARG (\w+)=(\S+)", (root / "docker/Dockerfile").read_text(), re.MULTILINE)) |
There was a problem hiding this comment.
Reading docker/Dockerfile directly without checking if the file exists or wrapping it in a try-except block can cause the documentation build to crash in environments where the Dockerfile is missing (e.g., when building from a clean source distribution or archive). Wrapping it or checking for file existence makes the build more robust.
docker_file = root / "docker/Dockerfile"
docker_args = {}
if docker_file.is_file():
try:
docker_args = dict(re.findall(r"^ARG (\w+)=(\S+)", docker_file.read_text(), re.MULTILINE))
except OSError:
pass| toastTimer, parallelView = 'tp', | ||
| cpView = 'zigzag'; | ||
| const hashParams = new URLSearchParams(location.hash.slice(1)); | ||
| const build = JSON.parse($('experiment-build').textContent); |
There was a problem hiding this comment.
Accessing $('experiment-build').textContent directly assumes the element always exists and has content. If the element is missing or empty in some contexts or custom templates, this will throw a TypeError or SyntaxError and crash the script. Adding a defensive check prevents potential runtime crashes.
| const build = JSON.parse($('experiment-build').textContent); | |
| const buildEl = $('experiment-build'); | |
| const build = buildEl ? JSON.parse(buildEl.textContent || '{}') : {}; |
| `<div class="external-map" data-topology="external"><svg viewBox="0 0 400 280" preserveAspectRatio="none" aria-hidden="true"><path d="M80 140 Q180 10 315 50 M80 140 Q220 100 325 180 M80 140 Q140 280 210 260"/></svg><div class="external-trainer">${trainer}<small>${t('vime training job', 'vime 训练任务')}</small></div><div class="external-cluster cluster-a">${serving(t('Cluster A', '集群 A'))}</div><div class="external-cluster cluster-b">${serving(t('Cluster B', '集群 B'))}</div><div class="external-cluster cluster-c">${serving(t('Cluster …', '集群 …'))}</div></div><p class="diagram-caption">${t('Independently managed fleets, connected by endpoint. Locations are illustrative; external engines may also be in one datacenter.', '通过 endpoint 连接独立管理的 GPU 集群。位置仅为示意;external 引擎也可以位于同一机房。')}</p>`; | ||
| } else { | ||
| topology = | ||
| `<div data-topology="${s.layout}" class="${s.layout === 'colocate' ? 'world-colocate' : 'managed-pools'}"><div class="world-nodes">${trainer}<div class="flow-arrow">${s.schedule === 'async' ? '⇄' : '⇢'}</div>${serving('Rollout', v.rollout)}</div><span class="pool-label">${s.layout === 'colocate' ? t('one GPU pool · alternating phases', '同一 GPU 池 · 交替工作') : t('slime manages two dedicated GPU pools', 'slime 统一管理 · 两个独立 GPU 池')}</span></div>`; |
There was a problem hiding this comment.
There is a branding/translation oversight here where the framework name slime was not updated to vime in the pool label, unlike other places in the diagrams (e.g., line 24). Updating it ensures consistent branding across the interactive diagrams.
| `<div data-topology="${s.layout}" class="${s.layout === 'colocate' ? 'world-colocate' : 'managed-pools'}"><div class="world-nodes">${trainer}<div class="flow-arrow">${s.schedule === 'async' ? '⇄' : '⇢'}</div>${serving('Rollout', v.rollout)}</div><span class="pool-label">${s.layout === 'colocate' ? t('one GPU pool · alternating phases', '同一 GPU 池 · 交替工作') : t('slime manages two dedicated GPU pools', 'slime 统一管理 · 两个独立 GPU 池')}</span></div>`; | |
| `<div data-topology="${s.layout}" class="${s.layout === 'colocate' ? 'world-colocate' : 'managed-pools'}"><div class="world-nodes">${trainer}<div class="flow-arrow">${s.schedule === 'async' ? '⇄' : '⇢'}</div>${serving('Rollout', v.rollout)}</div><span class="pool-label">${s.layout === 'colocate' ? t('one GPU pool · alternating phases', '同一 GPU 池 · 交替工作') : t('vime manages two dedicated GPU pools', 'vime 统一管理 · 两个独立 GPU 池')}</span></div>`; |
| if url_path.startswith("/projects/vime/"): | ||
| _, _, _, doc_language, _, suffix = url_path.split("/", 5) | ||
| relative = ("zh/" if doc_language == "zh" else "") + suffix |
There was a problem hiding this comment.
Unpacking url_path.split("/", 5) into exactly 6 variables will raise a ValueError if the URL path has fewer than 5 slashes (e.g., /projects/vime/ or /projects/vime/zh/). Checking the length of the split parts first makes the routing logic much more robust and prevents test crashes.
| if url_path.startswith("/projects/vime/"): | |
| _, _, _, doc_language, _, suffix = url_path.split("/", 5) | |
| relative = ("zh/" if doc_language == "zh" else "") + suffix | |
| if url_path.startswith("/projects/vime/"): | |
| parts = url_path.split("/") | |
| if len(parts) >= 6: | |
| doc_language = parts[3] | |
| suffix = "/".join(parts[5:]) | |
| relative = ("zh/" if doc_language == "zh" else "") + suffix | |
| else: | |
| relative = url_path.removeprefix(prefix) |
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Preserve native engine adapters and ROCm integration while refreshing translated runtime, tests, examples and dependency bindings. Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com> Co-authored-by: Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Preserve the native engine launch and CI adapters while carrying checkpoint rewind, separate Python launch, process retirement, documentation, and test updates. Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Preserve source callback selection, optional dependency handling, and bootstrap tests without changing the approved NPU exclusion. Co-authored-by: Blink <zpinghu@foxmail.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
3541fbd to
7af2ec0
Compare
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
|
Please align the PR description with current status (it still reads like a draft / no- |
|
Please remove the temporary Agent E2E hard-pin to |
Done |
Done. |
…fix, and MoE test suite - Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset. - Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state (vllm-pd-scheduler-fallback.patch). - Add colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes (vllm-ascend-pd-sleep-wake.patch). - Add Qwen3-30B-A3B PD test suite (test_pd_npu_suite.py + test_pd_npu.py): nopa / core(1P1D) / pd_1p3d(1P3D) variants, all colocate TP4+EP8. Signed-off-by: pansiling <1428273483@qq.com>
…fix, and MoE test suite - Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset. - Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state (vllm-pd-scheduler-fallback.patch). - Add colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes (vllm-ascend-pd-sleep-wake.patch). - Add Qwen3-30B-A3B PD test suite (test_pd_npu_suite.py + test_pd_npu.py): nopa / core(1P1D) / pd_1p3d(1P3D) variants, all colocate TP4+EP8. Signed-off-by: pansiling <1428273483@qq.com>
…fix, and MoE test suite - Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset. - Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state (vllm-pd-scheduler-fallback.patch). - Add colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes (vllm-ascend-pd-sleep-wake.patch). - Add Qwen3-30B-A3B PD test suite (test_pd_npu_suite.py + test_pd_npu.py): nopa / core(1P1D) / pd_1p3d(1P3D) variants, all colocate TP4+EP8. Signed-off-by: pansiling <1428273483@qq.com>
…fix, and MoE test suite - Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset. - Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state (vllm-pd-scheduler-fallback.patch). - Add colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes (vllm-ascend-pd-sleep-wake.patch). - Add Qwen3-30B-A3B PD test suite (test_pd_npu_suite.py + test_pd_npu.py): nopa / core(1P1D) / pd_1p3d(1P3D) variants, all colocate TP4+EP8. Signed-off-by: pansiling <1428273483@qq.com>
- Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset (TP*DP*PP*PCP). - Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state. Folded into docker/npu_patch/vllm.patch. - Add the colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes. Folded into docker/npu_patch/vllm-ascend.patch. - Add Qwen3.5-35B-A3B NPU PD tests: tests/test_qwen3.5_35B_A3B_pd_npu.py (8-NPU colocate smoke with --check-weight-update-equal) and tests/test_qwen3.5_35B_A3B_pd_parallel_check.py (nopa / 1P1D TP4 / 1P3D TP2 variants). - Update tests/utils/test_vllm_engine.py for the new MooncakeConnector default. Signed-off-by: pansiling <1428273483@qq.com>
- Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset (TP*DP*PP*PCP). - Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state. Folded into docker/npu_patch/vllm.patch. - Add the colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes. Folded into docker/npu_patch/vllm-ascend.patch. - Guard two PD baselines in the NPU smk suite: tests/test_qwen3.5_35B_A3B_pd_npu.py (MoE colocate, 1P1D TP4, 8 NPU) and tests/test_qwen3_0.6B_pd_npu.py (dense non-colocate, one prefill plus two decode engines, 4 NPU). - Update tests/utils/test_vllm_engine.py for the new MooncakeConnector default. Signed-off-by: pansiling <1428273483@qq.com>
- Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset (TP*DP*PP*PCP). - Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state. Folded into docker/npu_patch/vllm.patch. - Add the colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes. Folded into docker/npu_patch/vllm-ascend.patch. - Guard two PD baselines in the NPU smk suite: tests/test_qwen3.5_35B_A3B_pd_npu.py (MoE colocate, 1P1D TP4, 8 NPU) and tests/test_qwen3_0.6B_pd_npu.py (dense non-colocate, one prefill plus two decode engines, 4 NPU). - Update tests/utils/test_vllm_engine.py for the new MooncakeConnector default. Signed-off-by: pansiling <1428273483@qq.com>
- Dispatch the PD KV connector through the platform layer: GPU defaults to MooncakeConnector (aligned with main vllm-project#470), NPU overrides to MooncakePullConnector with kv_port and a slot-based port offset (TP*DP*PP*PCP). - Fix the cross-step weight-sync hang: a paused scheduler discards its empty connector output, so Mooncake pending transfers are never consumed and the engine never reaches the idle state. Folded into docker/npu_patch/vllm.patch. - Add the colocate sleep/wake KV lifecycle: drain in-flight pulls, unregister CaMem-pinned buffers before sleep, reregister on kv_cache-tagged wakes. Folded into docker/npu_patch/vllm-ascend.patch. - Guard two PD baselines in the NPU smk suite: tests/test_qwen3.5_35B_A3B_pd_npu.py (MoE colocate, 1P1D TP4, 8 NPU) and tests/test_qwen3_0.6B_pd_npu.py (dense non-colocate, one prefill plus two decode engines, 4 NPU). - Update tests/utils/test_vllm_engine.py for the new MooncakeConnector default. Signed-off-by: pansiling <1428273483@qq.com>


Candidate and scope
This PR is open for review (not a draft). The validated source candidate is
84a012353b689dc086c1f425859b3f5e692a9542; the subsequent review change only removes the temporary Agent CI hostname selector. Release preparation remains separate from review readiness; official image promotion happens after merge and dual-architecture verification.Native vLLM:
83c078d955cd6e0c63d3b12abe669c8de936a370.The ten selected vLLM patches replay to engine tree
21e3947a38e0a179019ee25aa2ecfff2666c7b6b.Simplification and mirroring
All downstream vLLM Rust frontend adaptations remain withdrawn. Independent Router/Sunabako remain. Independent Mamba pools, optimistic PD, empty-PP/circular-admission extensions and EC122/125 are excluded. P/D retain visual encoders and native fallback. Separate release/MemAgent changes are outside this PR.
Audit and local validation
CI acceptance
Latest-head CPU validation: 1824 passes all seven automatic checks at
92fb7395a01fef90f4df90e378aa591231338e28, using experimental imageaosheninferact/vime@sha256:302194f14b9b50a15d7af87f9e39d56a3ae62658720d3065f2d5824f699d394d. This includes all 95 score-centering transport tests. Automatic build 1823 used the defaultlatestinstead and failed the native TITO API contract test; no test assertion or production code was changed to obtain the passing rerun. The GPU gate remains unopened in 1824, so this is CPU validation, not a new GPU matrix. Separate NPU build 539 fails before tests because its upload command expects.buildkite/pipeline-npu.yaml, which exists onascendbut not on this PR ormain; that pipeline routing issue remains unresolved.Historical combined coverage: 57/57 GPU configurations and 7/7 CPU checks, using unchanged-case reuse plus affected-only reruns. This is not a fresh full matrix on current HEAD, and the unpinned Agent scheduling change has not been GPU-tested yet.
84a01235with the temporary hostname selectorReused GPU cases by suite:
short2,vllm-config4,megatron36,vime-customized5,precision1, andckpt5 (53 total). The three rerunmegatroncases aretest_qwen2.5_0.5B_pipeline_rl.pywith--transport nccl,--transport disk, and--transport nccl --flush-cache-interval 2. The rerun Agent case istest_agent_sunabako_codex_e2e.py. The seven CPU checks are pre-commit, docs lab generator, docs lab browser, plugin contracts & CPU tests, agent adapter tests, synchronized upstream CPU tests, and utils tests.84a01235, before removing the hostname selector. Actual H100 results show 16 agent runs, two nonzero-gradient steps and changed parameters.This is combined coverage, not a fresh full-matrix run at HEAD. The later changes are test/CI-only; runtime, engine patch bytes and dependencies remain unchanged. CI uses immutable experimental runtime
aosheninferact/vime@sha256:b4cd30a95b415a237880d99887ced750e8cbb941eeda7710e39755316386f529with the explicit Buildkite checkout.H100 adaptations and the reviewer-requested cleanup:
inf-4x8h100-1hostname selector is removed. Agent jobs use the normal H100 pool; Bash, sandbox privileges, timeout, artifacts and training configuration are unchanged. Generated-step comparison verifies that removing the hostname selector is the only scheduling/configuration delta. The earlier Agent pass does not establish sandbox capability on every pool node.Historical failures remain recorded: 1799 Agent backward OOM; 1805 lost agent and namespace EPERM before training on other H100 nodes; canceled default-image builds 1804/1806. These are not current failures, nor are they reclassified as passes.
Dual-architecture preparation
After CI acceptance, the current ARM three-patch Router build, installed source/engine verification, DP1/DP2 pause/resume, real Router collection and hybrid-model sleep/wakeup checks pass. The initial hybrid harness lacked a multiprocessing main guard; its corrected rerun passes without an engine change. Native vLLM ELF parity and unchanged FA/TE binaries/dependency versions are verified on both architectures.
AMD64 final-source experimental image:
aosheninferact/vime@sha256:302194f14b9b50a15d7af87f9e39d56a3ae62658720d3065f2d5824f699d394d.Stale inherited provenance labels were normalized; filesystem layers and every non-label image-config field are verified unchanged from the tested source-only image. FA2/FA3/TE were not recompiled.
ARM64 final-source experimental image has also been uploaded:
aosheninferact/vime@sha256:5e0c05795d47b5d482e8619af268d1812b78a791ba00963dc3269baaa63e19e1. Both final-source images correspond to84a01235; the later hostname removal only affects CI scheduling, not installed runtime code. The extra ARM Straw CPU validation and dual-architecture experimental manifest remain pending. No official tag or completed sync cutoff is advanced.AI assistance used; original author/DCO attribution is preserved.