Category: Whisper and On-device AI Optimization
-
Whisper Architecture: How OpenAI’s Speech Model Works
Whisper's encoder-decoder uses 1.5GB VRAM for Large model. Break down the architecture and memory bottlenecks before mobile deployment.
-
Real-time Whisper Is a Battery Nightmare (Here’s How to Fix It)
Real-time Whisper drains 1% battery/min. VAD + adaptive inference + thermal throttling bring it down to 0.2%. Benchmarks on iPhone 13 Pro.
-
On-Device Whisper: ONNX and Core ML Inference Guide
ONNX Runtime + CoreML beats native Core ML for Whisper on iOS by 40%. Conversion script, memory pool tricks, and A15/A16 benchmark numbers.
-
Whisper Model Quantization for Mobile Deployment
Quantizing Whisper for mobile: post-training quantization failed, QAT crashed, but ONNX Runtime reduced model size 75% without accuracy loss.