Skip to content

Support MuLan, a plug-and-play language adapter to adapt existing diffusion model for up to 110+ languages without additional training #7948

Open

Description

@Zeqiang-Lai

Model/Pipeline/Scheduler description

# pip install mulankit
from diffusers import StableDiffusionPipeline
+ import mulankit

pipe = StableDiffusionPipeline.from_pretrained('Lykon/dreamshaper-8')
+ pipe = mulankit.transform(pipe, 'mulanai/mulan-lang-adapter::sd15_aesthetic.pth')
image = pipe('一只蓝色的🐶 in the 바다').images[0]
一只蓝色的 🐶 in the 바다 (Dreamshaper-8) レゴシュワルツェネッガー (SDXL-lightning) 一只可爱的猫头鹰 (MVDream) 海浪风景 (AnimateDiff)
image image image image

MuLan supports

  • Base models: Stable Diffusion 1.5, 2.1, XL, Pixart-Alpha/Sigma.
  • Downstream models: ControlNet, LCM, LoRA, finetuned models and etc.
  • Video models: AnimateDiff.
  • 3D models: MVDream.

Open source status

  • The model implementation is available.
  • The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

https://github.com/mulanai/MuLan
https://huggingface.co/mulanai/mulan-lang-adapter

Activity

  1. changed the title [-]Support MuLan, a plug-and-play language adapter to adapt any diffusion model for up to 110+ languages without additional training[/-] [+]Support MuLan, a plug-and-play language adapter to adapt existing diffusion model for up to 110+ languages without additional training[/+] on May 15, 2024
  2. github-actions commented on Sep 14, 2024

    @github-actions
    Contributor

    This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

    Please note that issues that do not follow the contributing guidelines are likely to be ignored.

  3. github-actions commented on Dec 13, 2024

    @github-actions
    Contributor

    This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

    Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions