You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reproduce Lora results is close but not accurate #165
I have the same problem. This is my result:
BELU: 0.6885
NIST: 8.68
METEOR: 0.4638
ROUGE_L: 0.7139
CIDEr: 2.5079
I reproduce it with A100-40g, following exactly the same process as in readme.
I use GPT2 small, and resulr: BLEU: 0.6727 NIST: 8.6009 METEOR: 0.4571 ROUGE_L: 0.6814 CIDEr: 2.3076
Do I still need to use a larger base model for fine-tuning
Activity
[-]Reproduce Lora results [/-][+]Reproduce Lora results is close but not accurate[/+]qwertyczx commentedon Mar 25, 2024
Hi, I met similar problem with you.This is my result:
BLEU: 0.6865
NIST: 8.6808
METEOR: 0.4638
ROUGE_L: 0.7116
CIDEr: 2.5030
I reproduce it with V100-32g, following steps in: https://github.com/microsoft/LoRA/tree/main/examples/NLG#replicating-our-result-on-e2e
RayCyder commentedon May 24, 2024
me too
xiaoshingshing2 commentedon Aug 12, 2024
I have the same problem. This is my result:
BELU: 0.6885
NIST: 8.68
METEOR: 0.4638
ROUGE_L: 0.7139
CIDEr: 2.5079
I reproduce it with A100-40g, following exactly the same process as in readme.
rzbsys commentedon Jan 26, 2025
I have same problem.
Is there anyone who has solved this problem?
zouzhuozhang commentedon Jul 2, 2025
I use GPT2 small, and resulr:
BLEU: 0.6727 NIST: 8.6009 METEOR: 0.4571 ROUGE_L: 0.6814 CIDEr: 2.3076Do I still need to use a larger base model for fine-tuning
HuiYulinn commentedon Dec 26, 2025
These are my e2e results:
BLEU: 0.6801
NIST: 8.7033
METEOR: 0.4583
ROUGE_L: 0.6919
CIDEr: 2.3905
Has anyone solved this problem?