You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Confusing logic for reset_parameters() in Embedding and Linear layer #194
Source: LoRA/loralib/layers.py
Issue: lora_B (instead of lora_A) is sampled from normal distribution.
Problem 2
Source: LoRA/loralib/layers.py
Issue: The comment is misguiding because lora_A is supposed to get Kaiming initialization, whereas lora_B is supposed to be initialized as zero.
Activity