Skip to content

[Hardware][Ascend] Support Qwen3.8-27B GRPO with GDN head TP - #512

Open
Tjh-UKN wants to merge 1 commit into
vllm-project:ascendfrom
Tjh-UKN:feat/qwen38-27b-ascend
Open

Tjh-UKN wants to merge 1 commit into
vllm-project:ascendfrom
Tjh-UKN:feat/qwen38-27b-ascend

Conversation

@Tjh-UKN

@Tjh-UKN Tjh-UKN commented Nov 3, 2026 •

Copy link
Copy Markdown

Overview

Adds Qwen3.8-27B GRPO support on Ascend with opt-in GDN head tensor parallelism.
Head-local projections, convolution and recurrent computation are partitioned
across TP ranks. The checkpoint retains the Qwen3.5 hybrid-attention architecture,
so this reuses the existing model bridge. This PR targets ascend and follows
#403's script, E2E and training-evidence format.

Changes

  • GDN head TP: qwen3_5.py and hf_attention.py add head sharding,
    SP/non-SP collectives, shared-normalization gradient reduction and checkpoint
    factories. Enable with --qwen-gdn-tensor-parallel; CP=1 is required.
    The replicated GDN path remains the default.
  • Weight layout: HF loading and synchronous/asynchronous updates preserve
    unequal Q/K/V segments through vime/utils/gdn_tp.py and parameter metadata.
  • Ascend V2: retain reloadable serving weights through server configuration;
    reuse existing dependency HCCL/IPC patches.
  • Example and E2E: add the model/run scripts, EN/ZH documentation, CPU
    regression tests and tests/test_qwen3.8_27B_npu.py, registered as npu-16
    in Buildkite NPU smk. The launcher isolates Ray ports/session without killing other jobs.

Usage

Use the NPU image/setup, then:

export DATA_ROOT=/root
hf download Qwen/Qwen3.8-27B --local-dir "${DATA_ROOT}/models/Qwen3.8-27B"
hf download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir "${DATA_ROOT}/datasets/dapo-math-17k"
NUM_ROLLOUT=100 bash scripts/run-qwen3.8-27B-npu.sh

Override MODEL_DIR, DATASET_DIR, CHECKPOINT_DIR or NUM_ROLLOUT as needed.
Model/optimizer checkpoints are saved every 25 rounds; four require approximately
1.4 TiB plus input storage. Use unused Ray/HCCL ports in a dedicated NPU container.

Testing

100-round hardware training log

The saved 2026-10-30/31 run completed 100 rollout/train/post-training-sync
cycles
, 25,600 samples and 54,285,777 response tokens. Ray finished
SUCCEEDED, driver exit 0, and saved the final model/optimizer checkpoint at iteration 99.

Evidence scope: this adaptation snapshot predates the PR's additional
input-layernorm TP-gradient fix. It used the 0.30rc1 unpacked-HCCL compatibility
adapter; current-head hardware verification belongs to PR CI. Repository dependency pins are unchanged.

Hardware, environment and training configuration
Item Saved-run configuration
Hardware Ascend910 from npu-smi; 16 logical devices / 8 physical cards
Placement 8 actor + 8 rollout; two TP4 rollout engines
Parallelism TP4 / PP2 / SP / CP1; GDN head TP
Serving vLLM 0.30.0 (+empty) / vLLM-Ascend 0.30.0rc1
Runtime Python 3.12 / aarch64; PyTorch 2.10.0+cpu / torch_npu 2.10.0.post4; CANN 9.1; FLA 0.5.2
Batch 32 prompts × 8 samples; global batch 256; temperature 1.0
Token limits prompt 1024 / response 4096 / context 5120
Memory CPU optimizer offload; full/uniform/1; serving utilization 0.85
Runner / loss V2; MTP disabled; reference-model KL penalty disabled

Raw values, trailing 5-step means and full-run means, following #403:

Qwen3.8-27B historical 100-step Ascend GRPO training metrics

Full-resolution SVG ·
100-step metrics log (JSONL) ·
CSV ·
Run summary ·
Provenance/timing notes ·
SHA256

Metric Rounds 1–10 Rounds 91–100
Raw reward −0.1547 +0.6422
Judged-correct fraction 42.3% 82.1%
Mean response tokens 3125 1676
Truncation 56.4% 12.0%
Entropy 0.6046 0.2579

DAPO scores +1/−1, so the judged-correct fraction is (raw_reward + 1) / 2.
These are shuffled training batches, with late-stage fluctuations; they do
not establish held-out accuracy or convergence.

Timing / completion Result
Mean complete step wall / native perf/step_time 990.16 / 990.37 s
Mean rollout / actor training 472.31 / 504.28 s
Mean / final post-training sync Timer 3.609 / 3.6 s
Finite training steps; gradient norm range 100/100; 0.0843–1.2872
Weight syncs including initialization 101
HTTP 200 per update / finish / resume endpoint, across both engines 202 each

Wall time uses successive post-training-sync ends, including checkpoint/phase
overhead; native perf/step_time has different boundaries. The sync chart uses
post-training Timers rather than lagged perf/update_weights_time. Shorter
responses and concurrent late-run backup I/O preclude isolating kernel speedup from these timings.

Selected native log values and source traceability
train step 0:
  raw_reward=-0.25, response_lengths=3344.3671875
  loss=-1.0477378964424133e-09, grad_norm=0.11700751470107766
  entropy_loss=0.609438955783844
train step 99:
  raw_reward=0.671875, response_lengths=1612.94921875
  grad_norm=0.15898573357003964, entropy_loss=0.2838108539581299
final post-training Timer: update_weights end (elapsed: 3.6s)
Ray: SUCCEEDED; driver_exit_code=0; checkpoint=iter_0000099

The JSONL preserves all 100 native scalar dictionaries, Timer values and source
line numbers. Source train.log: 18,183,197 bytes / 132,386 lines, checked
against the original evidence archive:

SHA256: 0b150d6ea1f13f61d2c4644b435b1575bdb0b8769a6c2b5c215d7be300754a87

train/kl_loss was absent with the reference KL penalty disabled; the dashboard
uses the logged train/ppo_kl. Near-zero normalized PG loss should be read with
nonzero gradient norms. Native generic FLOPs are not interpreted as hybrid-GDN MFU.

CPU checks and current-head E2E

  • 194 CPU tests passed, including all 18 new GDN tests; 14 existing native
    Megatron cases skipped because Megatron is not installed locally.
  • Full pre-commit passed. Bash syntax, launcher dry-run with spaced paths,
    smoke argument construction and npu-16 suite registration were checked.
  • Current-head smoke: two rounds, 4 prompts × 4 samples, response limit 128,
    thinking disabled, 16 devices, TP4/PP2/SP/CP1, V2, no MTP:
python tests/test_qwen3.8_27B_npu.py

Registered in NPU smk; results are tracked by the PR checks.

AI assistance: OpenAI Codex assisted with implementation, tests, documentation and log visualization.

Co-authored-by: OpenAI Codex
Signed-off-by: TAJh <taojiaheng1@huawei.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Qwen3.8-27B on Ascend NPU, featuring GDN (Gated Delta Network) head tensor parallelism. It adds the --qwen-gdn-tensor-parallel flag to shard GDN heads, projections, and convolutions across the tensor parallel group, along with necessary weight sharding/merging utilities, gradient synchronization for shared norms, and distributed checkpointing support. Additionally, it includes documentation, a launch script, unit tests, and a Buildkite smoke test. Feedback suggests improving the robustness of the merge_gdn_qkv helper by dynamically using the length of split_sizes instead of a hardcoded range of 3.

Comment thread vime/utils/gdn_tp.py
Comment on lines +15 to +19
def merge_gdn_qkv(partitions, split_sizes):
size = len(partitions)
local_sizes = tuple(n // size for n in split_sizes)
parts = [p.split(local_sizes, dim=0) for p in partitions]
return torch.cat([torch.cat([p[i] for p in parts], dim=0) for i in range(3)], dim=0).contiguous()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The merge_gdn_qkv function currently hardcodes range(3) when concatenating the split Q, K, and V segments. To make this helper more robust and generic for any number of split segments, we should use range(len(split_sizes)) instead of the hardcoded 3.

Suggested change
def merge_gdn_qkv(partitions, split_sizes):
size = len(partitions)
local_sizes = tuple(n // size for n in split_sizes)
parts = [p.split(local_sizes, dim=0) for p in partitions]
return torch.cat([torch.cat([p[i] for p in parts], dim=0) for i in range(3)], dim=0).contiguous()
def merge_gdn_qkv(partitions, split_sizes):
size = len(partitions)
local_sizes = tuple(n // size for n in split_sizes)
parts = [p.split(local_sizes, dim=0) for p in partitions]
return torch.cat([torch.cat([p[i] for p in parts], dim=0) for i in range(len(split_sizes))], dim=0).contiguous()

@Fulin-Gao

Fulin-Gao commented Nov 9, 2026 •

Copy link
Copy Markdown
  1. Please provide the default script configuration and performance metrics for 2K prompt and 8K response inference.
  2. It looks like your version hasn't been rebased onto PR [NPU] Sync Ascend with main training and serving updates #476 yet. This is a major change, so please rebase before submitting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants