Skip to content

fix(validate): reject sdpa/eager x non-packed x SP on NPU - #10606

Open
wego-123 wants to merge 1 commit into
modelscope:dev-swift-v5from
wego-123:fix/sp-validation-npu
Open

wego-123 wants to merge 1 commit into
modelscope:dev-swift-v5from
wego-123:fix/sp-validation-npu

Conversation

@wego-123

Copy link
Copy Markdown

PR type

  • [*] Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

Fixes #modelscope/twinkle#407

dev 的 _check_sp_hf_feasibility 已对 padding_free x 非 flash-attn 做了校验
("padding_free requires a flash attention kernel"),但非 packed(padding-mask
layout)x 非 flash-attn 组合没有任何校验。该组合在 NPU 上训练启动后于第一个
forward_backward 深崩(507035,异步爆发点在 twinkle/processor/base.py 的 packed
序列判定/解包路径,堆栈无法定位到真实根因)。
触发矩阵(20 步训练级验证):fa2+packed ✓ / fa2+非packed ✓ /
sdpa+packed(被现有校验禁止)/ sdpa+非packed ✗(本 issue)。

Experiment results

Paste your experiment result here(if needed).

Checklist

  • [*] I have linked the related issue above, if there is one.
  • [*] I have reviewed and understand every changed line, and I can explain how the change works and why it is correct.
  • [*] I have run the relevant tests or reproductions locally, and included the commands and results above (if applicable).
  • [*] I am responsible for the content of this PR, regardless of how it was produced.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant