Skip to content

Need Help with Fine-Tuning LoRa Using Diffusion Pipe for Hyperrealistic Object Videos #188

Open

Description

I’m experiencing difficulties fine-tuning LoRa using Diffusion Pipe to generate hyperrealistic videos of objects. Despite trying various datasets, adjusting the training steps, and experimenting with the training duration, I haven't been able to achieve satisfactory results.

For instance, one specific challenge I face is ensuring that text, such as the title on a book cover, appears clear and accurate in the output. So far, the results are inconsistent, and I’m not sure what adjustments I should make to improve the quality.

Do you have any suggestions for settings, best practices, or procedures that could help me fine-tune LoRa more effectively in this context? Any guidance would be greatly appreciated.

Thank you!

Activity

  1. Srabontideb commented on Mar 28, 2026

    @Srabontideb

    I have some suggessions:

    1. Set equal to rank (e.g., rank=32, alpha=32) for stable training.
    2. 1e-4 is a safe start; lower to 5e-5 if you see artifacts or overfitting.
    3. 500–2000 steps depending on dataset size; use validation images to gauge.
    4. cosine with warmup (100–200 steps) is generally reliable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions