You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Community Pipelines are introduced in diffusers==0.4.0 with the idea of allowing the community to quickly add, integrate, and share their custom pipelines on top of diffusers.
You can find a guide about Community Pipelines here. You can also find all the community examples under examples/community/. If you have questions about the Community Pipelines feature, please head to the parent issue.
Idea: Speech to Image
You can use a transformerautomatic-speech-recognition such as OpenAI whisper to transcribe the text, and pass that to Stable Diffusion. Together, this would create a nice speech-to-image pipeline.
Hi @osanseviero, I opened a PR for this issue. For the moment, I have tested my pipeline for whisper and it works pretty well. I'm waiting for your suggestions (maybe making the pipeline more generic so that we can use other speech-recognition models ?)
Activity
MikailINTech commentedon Oct 17, 2022
Hi, would be glad to work on it
osanseviero commentedon Oct 17, 2022
Awesome! Feel free to open a PR and if you have any questions just let us know :) (ideally in the parent issue unless it's very specific to this 🙏 )
akashAD98 commentedon Oct 18, 2022
I’m very much interested for this,I’ll try this feature
MikailINTech commentedon Oct 19, 2022
Hi @osanseviero, I opened a PR for this issue. For the moment, I have tested my pipeline for whisper and it works pretty well. I'm waiting for your suggestions (maybe making the pipeline more generic so that we can use other speech-recognition models ?)
patrickvonplaten commentedon Oct 20, 2022
It works very well ! Well done @MikailINTech :-)
fffiloni commentedon Oct 25, 2022
📌 gradio demo is available here: https://huggingface.co/spaces/fffiloni/speech-to-image
osanseviero commentedon Oct 25, 2022
This is amazing @fffiloni! Great work!
osanseviero commentedon Oct 25, 2022
Closing this issue as this is now done 🔥 amazing work everyone!
darwinharianto commentedon Oct 26, 2022
I tried the sample from https://github.com/huggingface/diffusers/tree/main/examples/community#speech-to-image

it gives me,
Is there anything wrong with the code?