Repository navigation
Conversation
| "additional_layer_modules contains unsupported components " | ||
| f"{unsupported_components}; supported components are " | ||
| f"{sorted(SUPPORTED_ABLITERABLE_COMPONENTS)}" | ||
| ) |
There was a problem hiding this comment.
What is the purpose of this restriction? Isn't the entire goal of this PR to allow the user to customize which modules are processed?
There was a problem hiding this comment.
Initially I introduced this restriction because Abliteration uses different optimization bounds for mlp.down_proj. However you're right, I'll remove it, add a test for the custom component names and update the config example.
There was a problem hiding this comment.
Ah, that is indeed a reason, but only makes the whole thing even more ad-hoc and brittle. What is abliteration supposed to use for parameters if a module fits neither pattern?
There was a problem hiding this comment.
an unknown component would get 0.8 as lower bound, regardless of the module type.
However i see 2 option:
- Allow custom module paths but keep the existing component types and their optimization bounds.
- Allow new component types and let users configure their optimization bounds.
I'd lean toward the first to keep this PR focused. What do you think?
There was a problem hiding this comment.
See #138. I agree that we don't need to worry about this for now.
|
What I don't like about this approach is that it configures additional modules. This is ad-hoc and robs the user of control. What we want is a setting that completely defines which modules are targeted. The default value of that setting should be what we currently use, and then the user can add or remove module targets as they see fit. |
|
Thanks for the feedback, I reworked this in
I also ran Ruff, ty, and all 45 unit tests after merging upstream. I tested module discovery on Qwen3-Next and ran a GPU smoke test on Domyn with LoRA and generation. |
|
The Python 3.11 CI job failed the SHA-256 check for I ran the same test on Python 3.11.7 against both this PR ( I couldn't reproduce a PR-specific difference locally. |
|
Unfortunately, that happens occasionally. If some Python versions fail while others succeed, this is almost certainly caused by slight environment differences on the CI machines. In this case, feel free to add the missing hash as a new |
|
Upstream had already added SHA256SUMS.ci4, so I resolved the merge conflict by keeping the upstream hash and adding the Python 3.11 hash in SHA256SUMS.ci5. |
|
Ok, thanks. I'm currently preparing the Heretic 2.0 release, and will get back to this PR after the release because this is too big a change to merge so close to the release. |
This PR introduces the support of model architectures with projection paths that currently are not supported and recognized by Heretic.
Related to #391, I changed the approach to use layer_module_targets, which fully defines the projection paths targeted in each decoder layer. Users can add or remove targets by replacing the default mapping, without modifying the source code.
Again, related to #391, I tested it on a tiny Qwen3-Next model: with the default targets Heretic found 0/4 shared expert projections, versus 4/4 after adding the shared expert path to the configuration.
By default, the configuration includes all 13 projection paths currently supported by Heretic.
AI use declaration
Check either of the following:
and fully understand it. I'm prepared to discuss the code without letting an agent speak on my behalf.