Repository navigation
Conversation
Rebase the NPU patches onto current local vLLM and Ascend sources, keeping Vime-specific transfer and response behavior. Add isolated CPU regressions and document the source-only baseline without changing images. Assisted-by: Codex Signed-off-by: Annarine <57247683+Annarine@users.noreply.github.com>
Documentation build overview
69 files changed ·
|
There was a problem hiding this comment.
Code Review
This pull request updates the NPU patch series for vLLM and vLLM-Ascend to target newer source commits, removing several backports that are now natively supplied by Ascend. It also introduces CPU regression tests for these patches. Feedback on the changes highlights a potential runtime NameError in the vLLM patch due to a missing import of build_spec_decoding_metrics which is still being referenced.
| cached_tokens=final_res.num_cached_tokens | ||
| ) | ||
|
|
||
| + spec_decode_metrics = build_spec_decoding_metrics(final_res) |
There was a problem hiding this comment.
In the rebased patch, the import of build_spec_decoding_metrics from vllm.entrypoints.generate.base.serving was removed, but the function is still called on line 229 (and line 266). Please verify if build_spec_decoding_metrics is already imported in the base serving.py file; otherwise, this will raise a NameError at runtime.
Align Vime CLI imports with the documented vLLM source revision and enable the scale-out endpoint used by rollout. Keep upstream NPU test fixtures compatible with the retained stream-wait and empty-payload patches. Reuse speculative metrics in response construction, cover frontend fields and scale-out defaults/forwarding, and clarify wheel installation and NPU data dependency constraints. Validation: 123 CPU tests passed, 3 real-vLLM checks skipped; 7 pinned-source regressions passed; both patches apply at the documented commits; all-file pre-commit and git diff --check passed. NPU E2E was not rerun. Assisted-by: Codex Signed-off-by: Annarine <57247683+Annarine@users.noreply.github.com>
| | vLLM | `e6bfe03ad73a3330cb427885aa90d97a12e1c704` | [GitHub](https://github.com/vllm-project/vllm) | | ||
| | vLLM-Ascend | `fd815467c221ee600137f6bdd53fe354d5e7c999` | [GitHub](https://github.com/vllm-project/vllm-ascend) | | ||
| | vLLM | `544f182d9f0c9bfe8bd7645525951d5e3692a542` | [GitHub](https://github.com/vllm-project/vllm) | | ||
| | vLLM-Ascend | `2d6654fbd170d110eec9c6cb30c55751045c4d8d` | [GitHub](https://github.com/vllm-project/vllm-ascend) | |
There was a problem hiding this comment.
I think dockerfile.npu also need to be modifiled if you modify this.
| compatible CANN, PyTorch and torch-npu installed and Ascend devices mounted. | ||
| Local source-level and CPU checks do not establish end-to-end NPU compatibility. | ||
|
|
||
| 本补丁仅针对上表的 vLLM / vLLM-Ascend 源码版本,未修改镜像或 Dockerfile。 |
The Ascend patch series backports weight-transfer and KV-cache behavior already supplied by the documented vLLM/vLLM-Ascend revisions. This PR removes those duplicate backports while retaining Vime-specific updates and response compatibility, and aligns Vime's frontend with the newer serving API.
Changes:
weight_versionandrequest_spec_decode_statsfields onto the native response builders without replacing native version spans or metrics. Reuse the speculative metrics object instead of building it twice for non-streaming requests.vllm.entrypoints.common.cli_argsand enable scale-out by default for Vime's/inference/v1/generaterollout requests.Validation:
git apply --check --whitespace=errorto temporary source copies at vLLM544f182d9f0c9bfe8bd7645525951d5e3692a542and vLLM-Ascend2d6654fbd170d110eec9c6cb30c55751045c4d8d; patched Python sources parse successfully.pre-commit run --all-files --show-diff-on-failureandgit diff --checkpass.This is a source-only change. The Dockerfile, base image, and patch-series order are unchanged. The default v0.28 image cannot consume the reduced patches as-is; its serving sources must be updated separately. NPU end-to-end testing and full RL training were not rerun for this revision, and isolated CPU tests do not establish device-runtime or long-running correctness.
AI assistance: OpenAI Codex assisted with implementation, regression tests, and documentation.