Edge AI Object Detection: YOLO Mobile Optimization Guide

Most Mobile YOLO Deployments Are Over-Engineered

I see the same pattern everywhere: teams grab YOLOv8 or v11, export to ONNX, realize it’s too slow, then spend weeks chasing quantization frameworks and custom operators. They burn through their MVP timeline before discovering that the model architecture itself was the problem.

The uncomfortable truth? For most mobile MVPs, you don’t need the latest YOLO. You need the smallest one that hits your accuracy floor, running on the simplest inference stack that doesn’t crash. Everything else is premature optimization.

Here’s what actually matters when you’re trying to ship object detection on a phone with two weeks and limited ML experience.

Pick Your Poison: Model Size vs Inference Latency

YOLOv5n (nano) is 1.9MB. YOLOv8n is 3.2MB. YOLOv11n is 2.6MB. All three can detect 80 COCO classes at ~30 FPS on a mid-range Android phone (tested on Pixel 6, TFLite with XNNPACK delegate, 320×320 input).

But the ecosystem complexity escalates fast:

• YOLOv5n: PyTorch → export.py → TFLite (official, works). Checkpoint: 3.9MB fp32.
• YOLOv8n: Ultralytics CLI → ONNX → TFLite (requires onnx-tf, breaks on custom layers). Checkpoint: 6.2MB fp32.
• YOLOv11n: Same export hell as v8, slightly better mAP (+1.2 points on COCO val), but good luck debugging the converter.

I spent three days fighting YOLOv8’s Focus layer export before realizing YOLOv5 just… works. If you’re prototyping, start with v5. The mAP difference (28.0 vs 28.4 at 640px) won’t kill your MVP. Integration friction will.

The Preprocessing Trap That Wrecks Mobile Inference

Your training script does this:

# Typical YOLO training preprocessing
import cv2
import numpy as np

def preprocess_train(image_path):
    img = cv2.imread(image_path)  # BGR uint8
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    img = cv2.resize(img, (640, 640))
    img = img.astype(np.float32) / 255.0  # normalize to [0,1]
    img = np.transpose(img, (2, 0, 1))  # HWC -> CHW
    return img[np.newaxis, ...]  # add batch dim

Your mobile app (Android Camera2 API) gives you this:

// Camera2 ImageReader callback
Image image = reader.acquireLatestImage();
Image.Plane[] planes = image.getPlanes();
ByteBuffer yPlane = planes[0].getBuffer();  // YUV_420_888 format
ByteBuffer uPlane = planes[1].getBuffer();
ByteBuffer vPlane = planes[2].getBuffer();

Notice the problem? You’re getting YUV data, not RGB. The naive path:

1. YUV → RGB conversion (20ms on CPU)
2. Resize to 320×320 (15ms)
3. Normalize to float32 (8ms)
4. Transpose for NCHW (5ms)

That’s 48ms before inference even starts. At 30 FPS, your frame budget is 33ms total.

The Fix: Bake Preprocessing Into Your Model

TFLite lets you fuse preprocessing ops into the graph. Here’s the trick:

import tensorflow as tf

def build_mobile_yolo_with_preprocessing(onnx_model_path, input_size=320):
    # Convert ONNX to TF SavedModel first (using onnx-tf)
    import onnx
    from onnx_tf.backend import prepare

    onnx_model = onnx.load(onnx_model_path)
    tf_rep = prepare(onnx_model)
    tf_rep.export_graph('yolo_savedmodel')

    # Load and wrap with preprocessing
    yolo_model = tf.saved_model.load('yolo_savedmodel')

    @tf.function(input_signature=[tf.TensorSpec(shape=[None, None, 3], dtype=tf.uint8)])
    def inference_with_preprocessing(image_uint8):
        # image_uint8: RGB uint8 tensor from mobile camera (after YUV->RGB)
        img = tf.image.resize(image_uint8, [input_size, input_size], 
                              method='bilinear')  # GPU-accelerated
        img = tf.cast(img, tf.float32) / 255.0  # normalize
        img = tf.expand_dims(img, 0)  # add batch dim
        return yolo_model(img)

    # Export fused model
    concrete_func = inference_with_preprocessing.get_concrete_function()
    converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
    converter.optimizations = [tf.lite.Optimize.DEFAULT]  # dynamic range quant
    tflite_model = converter.convert()

    with open('yolo_mobile_fused.tflite', 'wb') as f:
        f.write(tflite_model)

# Example usage
build_mobile_yolo_with_preprocessing('yolov5n.onnx', input_size=320)

Now your Android preprocessing drops to:

// Just YUV->RGB (hardware accelerated via RenderScript)
Bitmap rgbBitmap = yuvToRgbRenderScript(image);  // 8ms
ByteBuffer inputBuffer = bitmapToByteBuffer(rgbBitmap);  // 2ms
interpreter.run(inputBuffer, outputBuffer);  // 18ms @ 320px
// Total: 28ms end-to-end

Resize and normalization now run on TFLite’s optimized kernels (XNNPACK delegate uses NEON SIMD on ARM). Preprocessing drops from 48ms to 10ms.

Quantization: INT8 Is Not a Magic Bullet

Everyone says “just quantize to INT8” like it’s free 4x speedup. Reality check on Snapdragon 865 (Pixel 5):

You lose ~1-2 mAP points. For some use cases (face detection, large objects), that’s fine. For small object detection (coins, pills, defects), it ruins recall.

When INT8 Quantization Fails Silently

I hit this training a custom YOLO on PCB defect detection (640px images, 5 classes, average object size ~15px). FP32 mAP: 0.68. INT8 mAP: 0.41. Defects under 20px basically disappeared.

The issue: quantization error accumulates in the neck (FPN layers). Small objects rely on high-resolution feature maps, and $text{quantization_error} propto frac{1}{text{activation_magnitude}}$. Tiny activations get crushed to zero.

Fix: quantization-aware training (QAT). Insert fake quantization nodes during training so the model learns INT8-robust weights:

import tensorflow as tf
import tensorflow_model_optimization as tfmot

# After training your FP32 model
quantize_model = tfmot.quantization.keras.quantize_model

# Load trained model
model = tf.keras.models.load_model('yolo_fp32.h5')

# Apply QAT
q_aware_model = quantize_model(model)
q_aware_model.compile(optimizer='adam', 
                      loss=yolo_loss,  # your custom YOLO loss
                      metrics=['mAP'])

# Fine-tune for 10-20 epochs
q_aware_model.fit(train_dataset, epochs=10, validation_data=val_dataset)

# Convert to INT8 TFLite
converter = tf.lite.TFLiteConverter.from_keras_model(q_aware_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_qat_model = converter.convert()

With QAT, my PCB defect mAP recovered to 0.64 (vs 0.68 FP32). Still a drop, but usable.

But here’s the thing: QAT adds 2-3 weeks to your timeline. For an MVP, I’d rather ship FP32 at 22ms than spend a month chasing INT8 perfection.

Input Resolution: The 2x Speedup You’re Ignoring

Cutting input size from 640 → 320 isn’t just 4x fewer pixels. Inference time scales worse than linear due to memory bandwidth:

$$t_{text{inference}} approx alpha cdot (H times W)^{1.3} + beta$$

where $alpha$ is compute cost per pixel, $beta$ is fixed overhead (model loading, NMS postprocessing). The exponent ~1.3 comes from cache misses and memory hierarchy on mobile SoCs.

Practical numbers (YOLOv5n, Pixel 6, FP32):

• 640×640: 52ms (19 FPS)
• 416×416: 28ms (36 FPS)
• 320×320: 18ms (55 FPS)
• 224×224: 11ms (91 FPS)

If your objects are >10% of frame area (face detection, hand tracking, product scanning), 320px is plenty. I shipped a pill identifier MVP at 224px and users never complained about accuracy (mAP dropped from 0.72 to 0.68, but real-world precision stayed above 0.9 because pills are ~30-40% of frame).

Dynamic Resolution: The Underused Trick

Why lock yourself into one resolution? Detect at 224px normally, upscale to 416px only when confidence drops:

import numpy as np

class AdaptiveYOLO:
    def __init__(self, model_224, model_416):
        self.fast_model = model_224  # TFLite interpreter
        self.accurate_model = model_416
        self.confidence_threshold = 0.7

    def predict(self, frame):
        # Try fast model first
        detections_fast = self.fast_model(frame)  # 11ms
        max_conf = np.max(detections_fast['scores'])

        if max_conf > self.confidence_threshold:
            return detections_fast  # confident enough

        # Fall back to accurate model
        detections_accurate = self.accurate_model(frame)  # 28ms
        return detections_accurate

# Usage
adaptive = AdaptiveYOLO(load_tflite('yolo_224.tflite'), 
                         load_tflite('yolo_416.tflite'))

for frame in camera_stream:
    result = adaptive.predict(frame)
    # Runs at 91 FPS most of the time, drops to 36 FPS on ambiguous frames

This gave me 2.5x average speedup on a warehouse barcode scanner (most frames are clear, only ~20% needed high-res retry).

NMS Postprocessing: The Hidden 40% Overhead

YOLO outputs 8400 candidate boxes (at 640px with 3 detection heads). Non-Maximum Suppression (NMS) filters overlapping boxes using IoU thresholding:

$$text{IoU}(b_i, b_j) = frac{text{Area}(b_i cap b_j)}{text{Area}(b_i cup b_j)}$$

Naive NMS is $O(n^2)$ where $n$ is number of candidates. At 8400 boxes, that’s 35M comparisons per frame.

TFLite includes a fast NMS op (TFLite_Detection_PostProcess), but here’s what the docs don’t tell you: it’s single-threaded and not NEON-optimized. On my Pixel 6:

• Model inference: 18ms
• NMS postprocessing: 12ms (40% of total!)

Dirty optimization:

# Reduce candidate boxes by aggressive confidence filtering BEFORE NMS
def fast_nms(boxes, scores, conf_threshold=0.3, iou_threshold=0.45):
    # boxes: [N, 4] (x1, y1, x2, y2)
    # scores: [N, num_classes]

    # Pre-filter: only keep boxes with conf > threshold
    max_scores = np.max(scores, axis=1)
    keep_idx = max_scores > conf_threshold  # cuts 8400 -> ~200 boxes

    boxes_filtered = boxes[keep_idx]
    scores_filtered = scores[keep_idx]

    # Now NMS is 200^2 instead of 8400^2 (1750x fewer comparisons)
    final_boxes = tf.image.non_max_suppression(
        boxes_filtered,
        np.max(scores_filtered, axis=1),
        max_output_size=100,
        iou_threshold=iou_threshold
    )

    return boxes_filtered[final_boxes], scores_filtered[final_boxes]

This dropped NMS from 12ms to 3ms. You’ll lose some low-confidence detections, but for MVP use cases (scanning products, detecting faces), high-confidence targets are all you care about.

The MVP Stack: What Actually Ships

Forget TensorRT, ONNX Runtime, OpenVINO. For mobile MVP:

1. Model: YOLOv5n (it just exports cleanly)
2. Format: TFLite with XNNPACK delegate (built into Android 10+)
3. Quantization: FP32 first. Only try INT8 if you have time and QAT experience.
4. Input size: 320×320 (maybe 224 if objects are large)
5. Preprocessing: Fused into TFLite graph
6. NMS: Aggressive confidence pre-filtering (0.3-0.4 threshold)

This gets you 30-50 FPS on mid-range Android phones (Snapdragon 700-series or better). Total engineering time: 2-3 days from trained model to working app.

What About iOS?

Core ML is Apple’s equivalent of TFLite. Conversion path:

# YOLOv5 provides official Core ML export
python export.py --weights yolov5n.pt --include coreml --img 320
# Outputs yolov5n.mlmodel (ready to drag into Xcode)

Core ML runs on the Neural Engine (ANE) on A12+ chips. Expect ~15-20ms latency at 320px (about 25% faster than Android equivalent due to tighter hardware-software integration).

One gotcha: ANE requires FP16, not INT8. But the conversion is automatic via coremltools — you don’t write quantization code.

When Mobile Inference Isn’t the Answer

If your model needs >640px resolution or you’re detecting 50+ classes with high precision (mAP >0.8), edge inference will struggle. I’d estimate anything over 50ms per frame is borderline unusable (users perceive lag).

Fallback option: server-side inference with on-device caching. Detect keyframes only (when camera moves significantly), send to server, cache results, interpolate boxes for intermediate frames using optical flow or homography.

This hybrid approach let me ship a retail shelf scanning app with YOLOv8m (30M params, 600px input) running server-side at 10 FPS effective rate, while the app felt real-time because cached boxes tracked smoothly between API calls.

FAQ

Q: Should I use YOLOv8 or stick with YOLOv5 for mobile?

Stick with YOLOv5n for MVP. Export is more stable, the repo has better mobile examples, and the mAP difference (<2 points) won’t matter until you’re optimizing past your first 1000 users. YOLOv8’s Ultralytics CLI is sleeker but breaks more often on edge cases.

Q: How much accuracy do I lose with INT8 quantization?

Expect 1-2 mAP points drop for general COCO detection. For small objects (<5% of image area) or custom datasets with class imbalance, it can be 5-10 points. Always benchmark on your real data — COCO validation scores lie. If INT8 hurts too much, run FP16 (most mobile hardware supports it with minimal slowdown vs INT8).

Q: Can I run YOLO in a browser via WASM or WebGL?

Technically yes (ONNX.js, TensorFlow.js). Practically no. WebGL is 3-5x slower than native TFLite, and you’ll fight shader compilation quirks across devices. If you need browser deployment, use server-side inference or WebRTC streaming. The only exception: if your users are on desktop Chrome/Edge where WebGPU is available (then you might hit 20-30 FPS with YOLOv5n at 320px).

Use YOLOv5n at 320px Until You Have 10,000 Users

Most mobile object detection MVPs die because teams over-optimize before validating the core product. You don’t need YOLOv11. You don’t need INT8. You don’t need TensorRT.

You need a model that runs at 30 FPS, doesn’t crash, and detects your objects with >0.6 mAP. That’s YOLOv5n, FP32, 320px input, with fused preprocessing. Two days of integration work.

Once you have real users and usage data, profile where they actually complain (is it latency? accuracy? battery?), then optimize that ONE bottleneck. I’ve seen too many startups burn their runway chasing 10ms latency improvements that users never noticed.

The one thing I still don’t have a clean answer for: battery drain. YOLO at 30 FPS on mobile drains ~15-20% battery per hour (vs ~5% for camera preview alone). I suspect the culprit is DRAM bandwidth from moving tensors between CPU and accelerator, but I haven’t found a profiling tool that breaks it down cleanly. If anyone’s solved this, I’d love to hear how.