Tag: Low-Rank Adaptation
-
LoRA ๋ ผ๋ฌธ ๋ฆฌ๋ทฐ: ์์ญ GB ๋ชจ๋ธ์ MB๊ธ ํ๋ผ๋ฏธํฐ๋ก ํ์ธํ๋ํ๋ Low-Rank Adaptation ์์ ๋ถ์
LoRA fine-tunes 175B models with 0.01% trainable parameters. Math behind low-rank decomposition, GPT-3 results, and PyTorch implementation guide.