Tag: Medusa
-
Speculative Decoding: How Medusa and EAGLE Speed Up LLMs
Medusa and EAGLE promise 2-3x LLM speedup via speculative decoding. Test results on LLaMA 2: acceptance rates, memory cost, and when it fails.
Medusa and EAGLE promise 2-3x LLM speedup via speculative decoding. Test results on LLaMA 2: acceptance rates, memory cost, and when it fails.