Tag: LLaMA
-
LLM Tokenization: GPT vs Claude vs Llama Edge Cases
Emojis cost 5 tokens, accented names break budgets, and that 128K context window? Real tests show where GPT, Claude, and Llama tokenizers fail.
-
KV Cache Optimization: 3x Faster LLM Inference on 24GB VRAM
Learn KV cache optimization techniques to achieve 3x faster LLM inference with quantization, MQA, and PagedAttention on consumer GPUs with limited VRAM.
-
RoPE vs ALiBi: 32K Context LLaMA Perplexity Beats MPT
RoPE vs ALiBi performance at 32K context: LLaMA's perplexity wins vs MPT. Position encoding comparison reveals surprising scaling differences.
-
LoRA vs DoRA: 7B Model Training Speed Cuts 34% Cost
DoRA cuts LLM fine-tuning cost 44% vs LoRA but delivers 5% better multi-turn reasoning. Real A100 benchmarks, NaN debugging, and when to pick each.
-
vLLM OutOfMemoryError with Llama 3.1 70B: 3 Fixes
Fix vLLM OutOfMemoryError when deploying Llama 3.1 70B with tensor parallelism, quantization, and KV cache tuning on multi-GPU setups.
-
Speculative Decoding: How Medusa and EAGLE Speed Up LLMs
Medusa and EAGLE promise 2-3x LLM speedup via speculative decoding. Test results on LLaMA 2: acceptance rates, memory cost, and when it fails.