Category: Paper Review
-
FlashAttention-2 vs xFormers: H100 Cost at 100M Tokens
Compare FlashAttention-2 vs xFormers on H100 GPUs for 100M token training. Discover which framework cuts costs and boosts speed for LLM workloads.
-
Speculative Decoding: Why 2x Faster Inference Fails
Speculative decoding promises 2x faster LLM inference, but real-world gains often disappoint. Debug the hidden bottlenecks killing your speedup.
-
DeiT III vs DINOv2: ViT ImageNet Accuracy Without Labels
Compare DeiT III vs DINOv2 on ViT ImageNet accuracy: one needs labels, one doesn't. Discover which self-supervised approach wins and why it matters.
-
QLoRA Review: 4-Bit Finetuning for 65B Models on One GPU
QLoRA fits a 65B model on one 48GB GPU with zero quality loss. Here's exactly how NF4 quantization, double quant, and LoRA combine — with code.
-
GQA Review: Grouped Query Attention for Faster LLM Inference
GQA reduces LLM inference memory 8x with only 0.1 ROUGE loss. How grouped KV heads and 5% uptraining work in Llama 2 and Mistral.
-
Chinchilla Review: Why Fewer Parameters Beat Gopher
Chinchilla's 70B model beat 280B Gopher using 4x fewer parameters. The secret: training tokens matter more than model size. Proof inside.
-
CLIP ViT-L/14 Zero-Shot ImageNet Accuracy: 76.2% Without Fine-Tuning
CLIP ViT-L/14@336px hits 76.2% zero-shot ImageNet top-1 accuracy. We break down how contrastive pretraining works and where it fails.
-
ViT Review: 16×16 Patches That Replaced Convolutions
ViT-B/16 hits 77.9% top-1 accuracy on ImageNet with 16x16 patches and no convolutions. We review the architecture, training tricks, and where it breaks.