Skip to content

feat: add support for Qwen3Next shared experts - #391

Closed
umran666 wants to merge 2 commits into
p-e-w:masterfrom
umran666:feat/issue-181-qwen3-next-shared-expert
Closed

umran666 wants to merge 2 commits into
p-e-w:masterfrom
umran666:feat/issue-181-qwen3-next-shared-expert

Conversation

@umran666

Copy link
Copy Markdown
Contributor

Adds layer.mlp.shared_expert.down_proj to the abliteration targets.

This ensures that the shared expert in Qwen3-Next MoE models (which handles a significant portion of general knowledge) is properly abliterated.

The AttributeError on self_attn reported in the issue is already mitigated by the suppress(Exception) blocks added previously to master.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates src/heretic/model.py to attempt to add the mlp.down_proj component from a layer's shared expert, wrapping the call in an exception suppression block. There are no review comments, so I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@umran666

Copy link
Copy Markdown
Contributor Author

Fixes #181

@umran666

Copy link
Copy Markdown
Contributor Author

@p-e-w is this problem fixed?

@p-e-w

p-e-w commented Sep 30, 2026

Copy link
Copy Markdown
Owner

I don't know, and I don't have time for testing this currently.

Copilot AI balanced review requested due to automatic review settings October 28, 2026 10:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@p-e-w

p-e-w commented Oct 29, 2026

Copy link
Copy Markdown
Owner

Superseded by #525.

@p-e-w p-e-w closed this Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants