Tag: MoE 논문 리뷰
-
Switch Transformers 리뷰: 1.6조 파라미터 MoE 모델
Switch Transformers paper decoded: how Top-1 routing scales MoE to 1.6T parameters, load balancing tricks, and distillation speed hacks.
Switch Transformers paper decoded: how Top-1 routing scales MoE to 1.6T parameters, load balancing tricks, and distillation speed hacks.