Skip to content

Fix MindSpeed NPU LoRA projections and Qwen3.8-Flash-Next PLE host gather - #209

Merged
hjh0119 merged 7 commits into
modelscope:mainfrom
addsubmuldiv:qwen3_8_flash_next_npu
Oct 8, 2026
Merged

hjh0119 merged 7 commits into
modelscope:mainfrom
addsubmuldiv:qwen3_8_flash_next_npu

Conversation

@addsubmuldiv

@addsubmuldiv addsubmuldiv commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Enable Qwen3.8-Flash-Next Megatron LoRA training on MindSpeed NPU, including all-linear targets. Fix LoRA dispatch and tensor-parallel dimensions for MindSpeed's replacement linear classes, and avoid unsupported device-side reads of PLE CPU-pinned memory.

Changes

  • Restrict the PLE pinned-host Triton gather to CUDA; use the existing torch fallback on NPU.
  • Support native Megatron Column/Row parallel layers in NPU LoRA dispatch and forward. Construct native Row LoRA A with the global input width and the base layer's input_is_parallel setting.
  • Keep LoRA B replicated for duplicated TELinear bases, and synchronize both replicated factors' gradients under sequence parallelism.

Validation

Validated with 300-step Megatron LoRA training of Qwen3.8-Flash-Next on GPU and NPU, with aligned training loss curves.

Snipaste_2026-10-08_14-37-50

_gather_ple_rows_from_pinned dereferences the host pointer of the
CPU-pinned n-gram table inside the device kernel, which relies on CUDA
unified virtual addressing. On Ascend NPU the host pointer is treated
as a device DDR address and the first forward aborts with CANN error
0x800000 (MTE DDR out of range) on all vector cores of the first PP
stage. Gate the fast path to ids.device.type == 'cuda' so NPU takes
the numerically equivalent plain-torch fallback. CUDA behavior is
unchanged.

Verified: 300-step NPU Megatron LoRA run of Qwen3.8-Flash-Next
(TP2/EP4/ETP1/PP2, 8x910B3), per-step loss MAE vs GPU (8xH20) 0.00687.
…allel linears

Under bf16 (non-MC2) MindSpeed rebinds megatron.core.extensions.
transformer_engine.TEColumnParallelLinear/TERowParallelLinear to the
native mcore classes. The import names still resolve on both platforms,
so on NPU three assumptions of the CUDA-oriented LoRA path break:

- dispatch: isinstance against the TE tuple misses the native classes
  and silently falls through to peft's vanilla branch ("Target module
  ColumnParallelLinear not supported"). Enumerate the native classes
  in the dispatch tuple; on CUDA they never match.
- construction: LoRA factors sized with TE shard semantics are built
  by the native classes with global semantics, surfacing at first
  forward as mismatches of exactly tp_size (aclnnMatmul EZ1001
  k-axis [.,.,3072]x[6144,.]). Use the per-rank size on NPU.
- forward: the MindSpeed TELayerNormColumnParallelLinear returns
  (result, bias) instead of ((result, layernorm_out), bias);
  reconstruct the norm output via the base layer's _rmsnorm on NPU.

Complements 4450669 (NPU grouped-linear path). Known gap left open:
replicated TELinear bases (hyper-connection projections) still build
lora_b through a sharded column path; only reachable with
all-linear targets.

Verified: 300-step NPU Megatron LoRA run of Qwen3.8-Flash-Next
(TP2/EP4/ETP1/PP2, 8x910B3), per-step loss MAE vs GPU (8xH20) 0.00687.
@addsubmuldiv
addsubmuldiv marked this pull request as ready for review September 24, 2026 08:27
Copilot AI lite review requested due to automatic review settings September 24, 2026 08:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Native row dispatch and classification remain incompatible across backend and alias paths.

Review effort: Lite
Findings: None

What changed in this PR

Enables MindSpeed NPU LoRA support for Qwen3.8 projections and avoids unsupported PLE Triton gathers on non-CUDA devices.

Changes:

  • Dispatches native parallel linear layers to LoRA.
  • Adjusts NPU row-projection sizing and forward handling.
  • Uses the torch PLE fallback on non-CUDA devices.
File Summary
src/​mcore_bridge/​tuners/​patcher.py Adds native parallel dispatch; backend and row-classification compatibility issues remain.
src/​mcore_bridge/​tuners/​lora.py Supports NPU sizing and tuple outputs; native row projections need correct classification.
src/​mcore_bridge/​model/​modules/​kernels/​ple_kernels.py Restricts pinned-host Triton gathering to CUDA devices.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@addsubmuldiv
addsubmuldiv marked this pull request as draft September 24, 2026 08:32
@addsubmuldiv
addsubmuldiv marked this pull request as ready for review September 24, 2026 09:20
@addsubmuldiv
addsubmuldiv marked this pull request as draft September 24, 2026 13:01
@addsubmuldiv addsubmuldiv changed the title Fix MindSpeed NPU LoRA projections and Qwen3.8 PLE host gather Fix MindSpeed NPU LoRA projections and Qwen3.8-Flash-Next PLE host gather Oct 8, 2026
@addsubmuldiv
addsubmuldiv marked this pull request as ready for review October 8, 2026 07:11
@hjh0119
hjh0119 merged commit 35bc42c into modelscope:main Oct 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants