Skip to content

[Hardware][Ascend] Retire upstreamed weight-transfer backports - #507

Draft
Annarine wants to merge 2 commits into
vllm-project:ascendfrom
Annarine:codex/ascend-retire-upstream-weight-patches
Draft

Annarine wants to merge 2 commits into
vllm-project:ascendfrom
Annarine:codex/ascend-retire-upstream-weight-patches

Conversation

@Annarine

@Annarine Annarine commented Oct 29, 2026 •

Copy link
Copy Markdown

The Ascend patch series backports weight-transfer and KV-cache behavior already supplied by the documented vLLM/vLLM-Ascend revisions. This PR removes those duplicate backports while retaining Vime-specific updates and response compatibility, and aligns Vime's frontend with the newer serving API.

Changes:

  • Remove the upstreamed HCCL trainer registration/implementation, IPC layerwise reload and packed loading, KV-cache allocation context, and kernel-block slot-mapping backports.
  • Retain draft/rank-local updates, empty IPC payload handling, IPC collection, packed stream/reference handling, checkpoint pull support, and model adaptations. Preserve global-worker routing and LoRA state invalidation.
  • Rebase Vime's weight_version and request_spec_decode_stats fields onto the native response builders without replacing native version spans or metrics. Reuse the speculative metrics object instead of building it twice for non-streaming requests.
  • Update the three CLI imports to vllm.entrypoints.common.cli_args and enable scale-out by default for Vime's /inference/v1/generate rollout requests.
  • Update the upstream fake stream and missing-sizes test input to match the retained patches. Add checks for source-stream ordering, frontend imports, the scale-out default/forwarding, and single metrics construction.
  • Clarify editable versus ordinary wheel installation and retain compatible datasets/PyArrow constraints for the NumPy 1.26.4 NPU stack.

Validation:

  • Both serving patches apply with git apply --check --whitespace=error to temporary source copies at vLLM 544f182d9f0c9bfe8bd7645525951d5e3692a542 and vLLM-Ascend 2d6654fbd170d110eec9c6cb30c55751045c4d8d; patched Python sources parse successfully.
  • 7 source-patch CPU regressions pass, covering global-rank routing, draft failure cleanup, empty payloads, post-send IPC collection, fresh packed IPC references, caller-stream ordering, and legacy/native response coexistence with one metrics-builder call.
  • Vime CLI, engine, tensor-update, and distributed-update CPU tests: 123 passed, 3 skipped. The skipped checks require a real installed vLLM package, unavailable on the local Windows test host.
  • pre-commit run --all-files --show-diff-on-failure and git diff --check pass.

This is a source-only change. The Dockerfile, base image, and patch-series order are unchanged. The default v0.28 image cannot consume the reduced patches as-is; its serving sources must be updated separately. NPU end-to-end testing and full RL training were not rerun for this revision, and isolated CPU tests do not establish device-runtime or long-running correctness.

AI assistance: OpenAI Codex assisted with implementation, regression tests, and documentation.

Rebase the NPU patches onto current local vLLM and Ascend sources,
keeping Vime-specific transfer and response behavior. Add isolated CPU
regressions and document the source-only baseline without changing images.

Assisted-by: Codex
Signed-off-by: Annarine <57247683+Annarine@users.noreply.github.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the NPU patch series for vLLM and vLLM-Ascend to target newer source commits, removing several backports that are now natively supplied by Ascend. It also introduces CPU regression tests for these patches. Feedback on the changes highlights a potential runtime NameError in the vLLM patch due to a missing import of build_spec_decoding_metrics which is still being referenced.

cached_tokens=final_res.num_cached_tokens
)

+ spec_decode_metrics = build_spec_decoding_metrics(final_res)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In the rebased patch, the import of build_spec_decoding_metrics from vllm.entrypoints.generate.base.serving was removed, but the function is still called on line 229 (and line 266). Please verify if build_spec_decoding_metrics is already imported in the base serving.py file; otherwise, this will raise a NameError at runtime.

Align Vime CLI imports with the documented vLLM source revision and enable
the scale-out endpoint used by rollout. Keep upstream NPU test fixtures
compatible with the retained stream-wait and empty-payload patches.

Reuse speculative metrics in response construction, cover frontend fields
and scale-out defaults/forwarding, and clarify wheel installation and NPU
data dependency constraints.

Validation: 123 CPU tests passed, 3 real-vLLM checks skipped; 7 pinned-source
regressions passed; both patches apply at the documented commits; all-file
pre-commit and git diff --check passed. NPU E2E was not rerun.

Assisted-by: Codex
Signed-off-by: Annarine <57247683+Annarine@users.noreply.github.com>
| vLLM | `e6bfe03ad73a3330cb427885aa90d97a12e1c704` | [GitHub](https://github.com/vllm-project/vllm) |
| vLLM-Ascend | `fd815467c221ee600137f6bdd53fe354d5e7c999` | [GitHub](https://github.com/vllm-project/vllm-ascend) |
| vLLM | `544f182d9f0c9bfe8bd7645525951d5e3692a542` | [GitHub](https://github.com/vllm-project/vllm) |
| vLLM-Ascend | `2d6654fbd170d110eec9c6cb30c55751045c4d8d` | [GitHub](https://github.com/vllm-project/vllm-ascend) |

@floatlibai floatlibai Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this commit needed? I think we should align the vllm and vllm-ascend versions with #476.
or shall we based on #476

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think dockerfile.npu also need to be modifiled if you modify this.

compatible CANN, PyTorch and torch-npu installed and Ascend devices mounted.
Local source-level and CPU checks do not establish end-to-end NPU compatibility.

本补丁仅针对上表的 vLLM / vLLM-Ascend 源码版本,未修改镜像或 Dockerfile。

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete this line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants