Skip to content

Reproduce Lora results is close but not accurate #165

Open

Description

Activity

  1. changed the title [-]Reproduce Lora results [/-] [+]Reproduce Lora results is close but not accurate[/+] on Mar 18, 2024
  2. qwertyczx commented on Mar 25, 2024

    @qwertyczx

    Hi, I met similar problem with you.This is my result:
    BLEU: 0.6865
    NIST: 8.6808
    METEOR: 0.4638
    ROUGE_L: 0.7116
    CIDEr: 2.5030
    I reproduce it with V100-32g, following steps in: https://github.com/microsoft/LoRA/tree/main/examples/NLG#replicating-our-result-on-e2e

  3. RayCyder commented on May 24, 2024

    @RayCyder

    me too

  4. xiaoshingshing2 commented on Aug 12, 2024

    @xiaoshingshing2

    I have the same problem. This is my result:
    BELU: 0.6885
    NIST: 8.68
    METEOR: 0.4638
    ROUGE_L: 0.7139
    CIDEr: 2.5079
    I reproduce it with A100-40g, following exactly the same process as in readme.

  5. rzbsys commented on Jan 26, 2025

    @rzbsys

    I have same problem.

    BLEU: 0.6887
    NIST: 8.6935
    METEOR: 0.4650
    ROUGE_L: 0.7139
    CIDEr: 2.5096
    

    Is there anyone who has solved this problem?

  6. zouzhuozhang commented on Jul 2, 2025

    @zouzhuozhang

    I use GPT2 small, and resulr:
    BLEU: 0.6727 NIST: 8.6009 METEOR: 0.4571 ROUGE_L: 0.6814 CIDEr: 2.3076
    Do I still need to use a larger base model for fine-tuning

  7. HuiYulinn commented on Dec 26, 2025

    @HuiYulinn

    These are my e2e results:
    BLEU: 0.6801
    NIST: 8.7033
    METEOR: 0.4583
    ROUGE_L: 0.6919
    CIDEr: 2.3905
    Has anyone solved this problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions