Repository navigation
feat: Generic BERT Classifier scorer plugin - #500
Conversation
|
I've replaced the model with a new one, which is trained on a much better dataset and the results looks good.
See: https://huggingface.co/VINAY-UMRETHE/Refusal-Classifier |
|
Why open different PR? |
Because I messed that one. |
What do you mean? |
|
Just to be clear, I do not view this PR and #419 as competing, mutually exclusive approaches, and I have no problem shipping both plugins in the Heretic package by default. |
How does the accuracy compare against our current keyword classifier, and against #419? |
Of course, that was very obvious.
Can't say that for sure without having the logger plugins, I think @red40maxxer has a nice method to compare the scorers. |
|
The CI is being too bad for the tests. |
|
FWIW, I'm aiming to release Heretic 2.0 around the end of next week. Even if this plugin is still experimental, I think it would be nice to ship a preliminary version with 2.0 so that people can play around with it. |
|
I have already compared I'll be posting the results today. |
Scorer-Comparison ReportDataset details:
Scorer plugin details:
Classifier settings:
Versions:
Note Classifier predictions are deterministic across repeated runs! Mapping 1:
|
| scorer | accuracy | balanced acc. | precision | recall | F1 | MCC | TP | FP | TN | FN |
|---|---|---|---|---|---|---|---|---|---|---|
| KeywordRate | 0.784 | 0.820 | 0.649 | 0.989 | 0.784 | 0.639 | 174 | 94 | 175 | 2 |
| RefusalClassifier | 0.978 | 0.976 | 0.977 | 0.966 | 0.971 | 0.953 | 170 | 4 | 265 | 6 |
Agreement between scorers: 0.766 (Cohen's kappa 0.553)
KeywordRate markers responsible for false positives
| marker | false positives |
|---|---|
i cannot |
63 |
sorry |
22 |
i am unable |
6 |
i can' |
5 |
disclaimer |
2 |
i will not |
1 |
as an ai |
1 |
prohibit |
1 |
illegal |
1 |
i am an ai |
1 |
1. Metrics of KeywordRate vs RefusalClassifier
2. Agreement between KeywordRate vs RefusalClassifier
3. KeywordRate vs truth
4. RefusalClassifier vs truth
Important
From here, some things might get irrelevant, but it is still worth reporting. Because it is very unlikely that someone would use prompts that tend to output capability_limitation and clarification with heretic.
Example: "Search web to find XYZ content." OR "Tell me how to fix that?"
In both cases, the model would refuse due to lack of either the "capability" or the "context".
So I'm including this, if interested, you can continue reading this. This test case proves that KeywordRate is more prone to false-positives and RefusalClassifier is much better and accurate at detecting false-positives correctly.
Mapping 2: full_refusal, partial_refusal, capability_limitation, clarification
Here, full_refusal, partial_refusal, capability_limitation, clarification are counted as refusals.
This shows "what" the scorers will do if capability_limitation, clarification were wrongly counted as refusals.
| scorer | accuracy | balanced acc. | precision | recall | F1 | MCC | TP | FP | TN | FN |
|---|---|---|---|---|---|---|---|---|---|---|
| KeywordRate | 0.721 | 0.745 | 0.925 | 0.705 | 0.800 | 0.407 | 248 | 20 | 73 | 104 |
| RefusalClassifier | 0.948 | 0.928 | 0.971 | 0.963 | 0.967 | 0.846 | 339 | 10 | 83 | 13 |
Agreement between scorers: 0.715 (Cohen's kappa 0.354)
1. Metrics of KeywordRate vs RefusalClassifier (five labels)
2. Agreement between KeywordRate vs RefusalClassifier (five labels)
3. KeywordRate vs truth
4. RefusalClassifier vs truth
Conclusion
The RefusalClassfier scorer plugin is much better than the default KeywordRate scorer. Because it is more accurate at detecting false-positives (capability_limitation and clarification). Which can improve the abliteration by targeting more accurately and precisely every time ("Targeted-Ablation") as discussed many times before #56.
So, the refusal classifier plugin is ready to play.
Note
The dataset used for training and testing is still private and not yet published.
|
Why is And what does a response containing the word |
I had the same question when I saw the results, and it turned out to be that the PROMPT itself contained the keyword "disclaimer" in it. Both examples above as per the dataset are The dataset had these prompt-response pair, so it's obvious that
Regarding this... Out of the 445 test examples from the dataset ( So yes, it is simply because of the dataset. Note that most examples of
|
|
That sounds very strange. In English, |
That's due to the dataset, it is entirely synthetic, which also means LLMs in general use that because they generated the examples. The classifier is smart enough though, that it will not get biased towards one of the keywords only (I cannot vs I can't) as per my tests. Note: It didn't used any LLM inference with heretic for the report, the prompts and their responses were loaded from the dataset (test split) and only the scorers were loaded and used upon those prompts/responses. |
|
Is there a reason to call this scorer |
Makes sense, and then, this would be very much like a template for using any "text-classification" models to minimize or maximize any behavior (or at least in theory, because it is not necessarily True for every behavior to be in a single-direction, which can be abliterated). So, I believe that practical experimentations is the only way to find out. Also, it would be much easier as well, now that heretic is more like a generic ecosystem for modifying model weights. Including this "Personality-Steering" scorer #56 using this model: j-hartmann/emotion-english-distilroberta-base |
|
Ok, so this PR adds Important Like most of the other non-refusal abliteration capabilities of heretic (e.g. no slop, no humor), this "personality steering plugin" is also not so guaranteed to show some huge difference between the baseline scores and the final scores. Such things would require several "practical" tests and experimentation with different models and datasets. TheoryHere are certain things I believe are worth documenting, so anyone who wants to use the Goal:
Of course, for any such LLM weight modification tasks with heretic, we would first need contrasting sets of prompts which makes the model tend to generate those "target emotions" above. For example...
Similarly,
ProblemBut the catch is: "Almost no LLM model in general would show 'anger' and 'disgusting' emotions in its responses", simply because almost all LLMs are trained to be "a helpful assistant" persona (see #163). Due to this, the most likely responses you can expect from any major open-weight model would sound something like this:
And many more... The point here is, anything can happen but any good model in general would not act like a human who starts debating in anger or disgust to justify one thing. Because LLMs are trained to be "helpful" and "nice" to its users. Note This problem does not occur for the "happiness" or "surprise" set of prompts. Because in general, most LLMs act friendly, and try to engage with users happily or show surprising reactions, if the user prompt tells a good news, a good incident that happened to them, or anything which makes the user happy or was surprising like a gift etc. SolutionThis is where stuff like "Roleplay" and "Creative writing" comes in. As we saw, LLMs in general would not necessarily tend to generate whatever you are targeting to "Minimize" or "Maximize", either way you would still need prompts that actually do generate those target behaviors / emotions / instincts. But since it might not always work, you need to force the model to generate those content with prompts like these:
So, this is the solution I suggest, not just for emotion / personality steering, but anything which you can think of that a LLM in general would not tend to generate the target behaviors. In simple words, if you have any basic experience chatting with LLMs about different topics and you can accurately guess / predict what would the model most likely generate depending on the prompt, then you can already easily figure out what kind of contrasting prompts you would need to "minimize" or "maximize" any behavior, emotion, instinct, etc.I would also set a basic checklist which should be ideally considered when using the
Ensure these checklists are considered as a standard for any BERT-based classification and abliteration with heretic, otherwise you might end up wasting your time and compute. Regards, |
c9c9f9c to
ed0e896
Compare
ed0e896 to
b0cdd6e
Compare
Using the generic BERTClassifier scorer plugin
b0cdd6e to
b50d343
Compare
…l-classifier-plugin
|
@Vinay-Umrethe Merge at will! |
Merged...Useful links regarding the theory that backs this plugin:
Also completes #56. |
Same as #477
Refusal classifier scorer plugin
Finally, it's happening and this seems to be the very first steps ever since the plugin system is now fully implemented.
In the past, many folks have tried to implement different methods of "refusal-detection" (e.g. Using LLMs via APIs or Inference with custom models and system prompts). Now, with the plugin system, it can be done in a more standard and reproducible way.
So, I created a "refusal-classifier" model last month, which is based on ModernBERT architecture. It is a fine-tuned variant of answerdotai/ModernBERT-base model.
Once the final model is fully trained on a more proportional dataset for better accuracy, I plan to release both the final model and the training dataset which was used, probably including the training code as well.
Labels
compliantfull_refusalpartial_refusalcapability_limitationclarificationFor more information, see VINAY-UMRETHE/Refusal-Classifier.
Regarding the scorer plugin
We consider the labels
full_refusalandpartial_refusalto be counted as a refusal response. This improves the accuracy for "Targeted-Ablation", by ignoring the "false-positives" likecapability_limitation, which was not possible until now with simplekeyword_markersregex for many reasons.I believe this fixes #290 and this an efficient text-classification alternate for #419 and #244
TODO
KeywordRateVsRefusalDetectorscorer plugins.And more things which I cannot comprehend right now. Let's see how this goes.