Tag: ONNX
-
Pruned YOLOv8 ONNX INT8 Fails: 3 Fixes That Work
Pruned YOLOv8 + ONNX INT8 dtype mismatch? Here are 3 working fixes with Jetson benchmarks โ re-quantize, QAT pruning, or manual graph surgery.
-
ONNX INT8 vs FP16: 3x Latency Drop on Jetson Orin Nano
YOLOv8n INT8 cut latency from 47ms to 15ms on Jetson Orin Nano โ but small-object mAP dropped 5.7%. Real tradeoff numbers with power benchmarks.
-
ONNX Runtime Inlining Flags: 8x Latency Cut in 4 Steps
Cut ONNX Runtime latency by 8x with 4 inlining flags. Learn session config tuning for C++/Python inference optimizationโsimple yet powerful.
-
ONNX Export Pitfalls: 7 PyTorch โ Production Gotchas
Fix ONNX export failures fast. Learn the 7 critical PyTorch-to-production gotchas that silently break modelsโand how to catch them before deployment.
-
TorchAO vs ONNX Runtime: 8-bit Quantization Benchmark
Compare TorchAO vs ONNX Runtime 8-bit quantization performance. Benchmark results reveal surprising differences in speed, accuracy, and memory usage.
-
On-Device Whisper: ONNX and Core ML Inference Guide
ONNX Runtime + CoreML beats native Core ML for Whisper on iOS by 40%. Conversion script, memory pool tricks, and A15/A16 benchmark numbers.
-
Whisper Model Quantization for Mobile Deployment
Quantizing Whisper for mobile: post-training quantization failed, QAT crashed, but ONNX Runtime reduced model size 75% without accuracy loss.