Category: Edge AI & Robotics
-
MobileNet v3 vs EfficientNet-Lite: Pi 4 Latency Benchmarks
Compare MobileNet v3 vs EfficientNet-Lite on Raspberry Pi 4 with real latency benchmarks. The winner might surprise youโsee the full results.
-
ONNX Runtime Mobile: 8ms Inference on iPhone 13
Cut mobile inference from 200ms to 8ms by switching to ONNX Runtime. Benchmarks, CoreML quirks, and when TFLite still wins.
-
YOLOv8 INT8 Quantization: 4x Faster on Jetson Orin
INT8 quantization pushed YOLOv8 from 45 to 180 FPS on Jetson Orin Nano. PTQ beat QAT with 10 lines of calibration code. Here's the recipe.
-
Whisper Architecture: How OpenAI’s Speech Model Works
Whisper's encoder-decoder uses 1.5GB VRAM for Large model. Break down the architecture and memory bottlenecks before mobile deployment.
-
Real-time Whisper Is a Battery Nightmare (Here’s How to Fix It)
Real-time Whisper drains 1% battery/min. VAD + adaptive inference + thermal throttling bring it down to 0.2%. Benchmarks on iPhone 13 Pro.
-
On-Device Whisper: ONNX and Core ML Inference Guide
ONNX Runtime + CoreML beats native Core ML for Whisper on iOS by 40%. Conversion script, memory pool tricks, and A15/A16 benchmark numbers.
-
Whisper Model Quantization for Mobile Deployment
Quantizing Whisper for mobile: post-training quantization failed, QAT crashed, but ONNX Runtime reduced model size 75% without accuracy loss.