Tag: 어텐션메커니즘
-
FlashAttention 리뷰: Transformer 어텐션 2~4배 가속
FlashAttention cuts Transformer memory from O(N²) to O(N) while running 2-4× faster. Here's the IO-aware tiling trick explained.
-
RetNet 논문 리뷰: Transformer를 대체할 병렬 학습+순차 추론 아키텍처 완전 분석
RetNet achieves 8.4× faster inference than Transformer with 70% less memory. The parallel training + sequential inference proof inside.