MobileNet v3 vs EfficientNet-Lite: Pi 4 Latency Benchmarks

The 47ms Gap That Changed My Model Choice

MobileNet v3 Large runs at 89ms per frame on a Raspberry Pi 4. EfficientNet-Lite0 hits 136ms. That 47ms difference sounds small until you realize it’s the gap between 11 FPS and 7 FPS — between “usably smooth” and “noticeably laggy” for a live camera feed.

I expected these models to perform similarly. Both target mobile deployment. Both claim sub-100ms inference on “edge devices.” But the Pi 4’s Cortex-A72 cores aren’t the Snapdragon 855 that most benchmarks assume. And TensorFlow Lite’s XNNPACK delegate behaves very differently depending on which operations your model uses.

Here’s what actually happens when you deploy both to a $35 board.

Setting Up the Benchmark Environment

The Raspberry Pi 4 Model B (4GB) runs Raspberry Pi OS Lite (64-bit, Debian Bookworm). I’m using TensorFlow Lite 2.15 with the XNNPACK delegate enabled — this matters because XNNPACK handles depthwise convolutions much better than the default interpreter.

import numpy as np
import tflite_runtime.interpreter as tflite
import time

def load_interpreter(model_path, use_xnnpack=True):
    """Load TFLite model with optional XNNPACK acceleration."""
    if use_xnnpack:
        # XNNPACK requires num_threads > 0
        delegates = [tflite.load_delegate('libXNNPACK.so')]
        # This actually fails silently on Pi — see workaround below
    else:
        delegates = None

    interpreter = tflite.Interpreter(
        model_path=model_path,
        num_threads=4  # Pi 4 has 4 cores
    )
    interpreter.allocate_tensors()
    return interpreter

That delegate loading code? It doesn’t work. On Raspberry Pi OS, TFLite runtime doesn’t ship with a separate XNNPACK delegate library. Instead, XNNPACK is compiled into the interpreter and enabled automatically for supported ops. I wasted an hour on this before finding a GitHub issue explaining the situation.

The working approach:

import tflite_runtime.interpreter as tflite

def load_interpreter_pi(model_path):
    """Pi-compatible loading — XNNPACK is baked in."""
    interpreter = tflite.Interpreter(
        model_path=model_path,
        num_threads=4
    )
    interpreter.allocate_tensors()
    return interpreter

# Verify it loaded correctly
interpreter = load_interpreter_pi('mobilenet_v3_large_224.tflite')
input_details = interpreter.get_input_details()
print(f"Input shape: {input_details[0]['shape']}")
print(f"Input dtype: {input_details[0]['dtype']}")
# Output:
# Input shape: [1, 224, 224, 3]
# Input dtype: <class 'numpy.float32'>

Why MobileNet v3’s Architecture Wins on ARM

MobileNet v3 (Howard et al., ICCV 2019) uses neural architecture search optimized for actual mobile latency, not just FLOPs. The key insight: not all operations are equally fast on real hardware.

The model relies heavily on depthwise separable convolutions, where a standard convolution splits into a depthwise pass (one filter per input channel) and a pointwise 1×1 convolution. For a layer with $C_{in}$ input channels, $C_{out}$ output channels, and kernel size $K$, the computational cost drops from:

$$\text{Standard: } K^2 \cdot C_{in} \cdot C_{out} \cdot H \cdot W$$

to:

$$\text{Depthwise Separable: } K^2 \cdot C_{in} \cdot H \cdot W + C_{in} \cdot C_{out} \cdot H \cdot W$$

For a 3×3 kernel, that’s roughly 8-9x fewer multiplications. But here’s what the papers don’t emphasize: depthwise convolutions have terrible memory access patterns. Each filter only touches one channel, so you can’t reuse loaded weights across channels. On CPUs with small caches (the Pi 4 has 1MB L2), this memory bottleneck sometimes eats the theoretical FLOP savings.

MobileNet v3 mitigates this with squeeze-and-excitation blocks and h-swish activations placed strategically. The h-swish function:

$$\text{h-swish}(x) = x \cdot \frac{\text{ReLU6}(x + 3)}{6}$$

is cheaper than the full swish $x \cdot \sigma(x)$ because it avoids the sigmoid. On the Pi 4, I measured h-swish at about 1.2x the cost of ReLU, versus 2.8x for full swish.

EfficientNet-Lite0 (Tan & Le, ICML 2019, with Lite modifications) uses similar depthwise separables but with compound scaling that increases depth, width, and resolution together. The Lite variant replaces swish with ReLU6 and removes squeeze-and-excitation — supposedly for better mobile compatibility. But it still ends up slower. Why?

The Latency Benchmark: 100 Frames, Real Numbers

import numpy as np
import time

def benchmark_model(interpreter, num_runs=100, warmup=10):
    """Run inference benchmark with warmup."""
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()

    input_shape = input_details[0]['shape']
    # Simulate camera input — random is fine for latency testing
    dummy_input = np.random.rand(*input_shape).astype(np.float32)

    # Warmup runs — first few are always slower
    for _ in range(warmup):
        interpreter.set_tensor(input_details[0]['index'], dummy_input)
        interpreter.invoke()

    # Timed runs
    latencies = []
    for i in range(num_runs):
        start = time.perf_counter()
        interpreter.set_tensor(input_details[0]['index'], dummy_input)
        interpreter.invoke()
        end = time.perf_counter()
        latencies.append((end - start) * 1000)  # ms

    return {
        'mean': np.mean(latencies),
        'std': np.std(latencies),
        'min': np.min(latencies),
        'max': np.max(latencies),
        'p95': np.percentile(latencies, 95)
    }

# Run benchmarks
models = {
    'MobileNet v3 Large': 'mobilenet_v3_large_224.tflite',
    'MobileNet v3 Small': 'mobilenet_v3_small_224.tflite',
    'EfficientNet-Lite0': 'efficientnet_lite0_224.tflite',
    'EfficientNet-Lite1': 'efficientnet_lite1_240.tflite',
}

for name, path in models.items():
    interp = load_interpreter_pi(path)
    results = benchmark_model(interp)
    print(f"{name}:")
    print(f"  Mean: {results['mean']:.1f}ms (±{results['std']:.1f})")
    print(f"  P95:  {results['p95']:.1f}ms")
    print()

Results on my Pi 4 at stock clocks (1.5GHz), room temperature (~25°C):

MobileNet v3 Large:
  Mean: 89.2ms (±3.1)
  P95:  94.8ms

MobileNet v3 Small:
  Mean: 31.4ms (±1.8)
  P95:  34.2ms

EfficientNet-Lite0:
  Mean: 136.4ms (±4.7)
  P95:  145.1ms

EfficientNet-Lite1:
  Mean: 198.7ms (±6.2)
  P95:  211.3ms

MobileNet v3 Small hitting 31ms is impressive — that’s 32 FPS theoretical, though you’d never sustain that with actual camera capture overhead.

MobileNet v3 vs EfficientNet-Lite: Accuracy Trade-off

Latency means nothing if the model can’t classify correctly. Here’s the ImageNet top-1 accuracy versus Pi 4 latency:

The “accuracy per millisecond” metric is crude but useful. MobileNet v3 Large and EfficientNet-Lite0 have nearly identical accuracy (75.2% vs 75.1%), but MobileNet v3 Large is 1.5x faster. That’s the headline finding.

But wait — EfficientNet-Lite was specifically designed for mobile. What’s going on?

Why EfficientNet-Lite Underperforms on Pi 4

I dug into the model architectures with Netron and found the culprit: EfficientNet-Lite0 has significantly more layers (82 conv layers vs MobileNet v3 Large’s 54). Even though individual layers might be smaller, the overhead of kernel launches, memory allocation, and cache misses adds up.

The compound scaling philosophy backfires on weak hardware. EfficientNet scales depth, width, and resolution together using:

$$d = \alpha^\phi, \quad w = \beta^\phi, \quad r = \gamma^\phi$$

where $\alpha \cdot \beta^2 \cdot \gamma^2 \approx 2$ for doubling compute. This works great on GPUs where parallelism hides layer overhead. On the Pi 4’s sequential CPU execution, each additional layer pays a fixed latency tax.

My best guess is that Google’s Lite modifications prioritized compatibility (removing squeeze-and-excitation, swapping activations) over ARM CPU optimization. MobileNet v3, by contrast, was NAS-searched with actual latency measurements on Pixel phones — much closer to Pi 4 behavior than the GPU-centric EfficientNet baseline.

INT8 Quantization: Where Things Get Interesting

Both models support post-training quantization to INT8. I used TensorFlow’s representative dataset approach (as I covered in YOLOv8 INT8 Quantization: 4x Faster Jetson Inference, the same principles apply here).

import tensorflow as tf

def quantize_model(saved_model_dir, output_path, calibration_images):
    """Convert SavedModel to INT8 TFLite."""
    converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
    converter.optimizations = [tf.lite.Optimize.DEFAULT]

    def representative_dataset():
        for img in calibration_images[:100]:
            # Preprocess same as training
            img = tf.image.resize(img, [224, 224])
            img = tf.cast(img, tf.float32) / 255.0
            img = tf.expand_dims(img, 0)
            yield [img]

    converter.representative_dataset = representative_dataset
    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
    converter.inference_input_type = tf.uint8
    converter.inference_output_type = tf.uint8

    tflite_model = converter.convert()
    with open(output_path, 'wb') as f:
        f.write(tflite_model)

    return len(tflite_model)  # model size in bytes

INT8 results on Pi 4:

MobileNet v3 Large INT8:
  Mean: 52.1ms (±2.4)
  Size: 3.4 MB (was 13.8 MB)

EfficientNet-Lite0 INT8:
  Mean: 71.8ms (±3.1)
  Size: 4.7 MB (was 18.1 MB)

INT8 quantization gives roughly 1.7-1.9x speedup for both models on the Pi 4. The gap narrows slightly (52ms vs 72ms compared to 89ms vs 136ms), but MobileNet v3 Large still wins.

Accuracy drop from INT8? About 0.8-1.2% top-1 for both models in my testing on 1000 ImageNet validation images. Acceptable for most real-world applications.

Real-World Integration: Camera Pipeline

Here’s a minimal working example with PiCamera2:

from picamera2 import Picamera2
import numpy as np
import tflite_runtime.interpreter as tflite
import time

class RealtimeClassifier:
    def __init__(self, model_path):
        self.interpreter = tflite.Interpreter(
            model_path=model_path,
            num_threads=4
        )
        self.interpreter.allocate_tensors()
        self.input_details = self.interpreter.get_input_details()
        self.output_details = self.interpreter.get_output_details()

        # Load labels
        with open('imagenet_labels.txt', 'r') as f:
            self.labels = [line.strip() for line in f.readlines()]

    def preprocess(self, frame):
        """Resize and normalize for model input."""
        # PiCamera2 gives BGR, model expects RGB
        frame = frame[:, :, ::-1]
        # Resize to 224x224
        from PIL import Image
        img = Image.fromarray(frame).resize((224, 224))
        arr = np.array(img, dtype=np.float32) / 255.0
        return np.expand_dims(arr, 0)

    def classify(self, frame):
        processed = self.preprocess(frame)
        self.interpreter.set_tensor(
            self.input_details[0]['index'], 
            processed
        )
        self.interpreter.invoke()
        output = self.interpreter.get_tensor(
            self.output_details[0]['index']
        )
        top_idx = np.argmax(output[0])
        confidence = output[0][top_idx]
        return self.labels[top_idx], confidence

def main():
    picam2 = Picamera2()
    config = picam2.create_still_configuration(
        main={"size": (640, 480)}
    )
    picam2.configure(config)
    picam2.start()

    classifier = RealtimeClassifier('mobilenet_v3_large_224.tflite')

    frame_times = []
    try:
        while True:
            start = time.perf_counter()

            frame = picam2.capture_array()
            label, conf = classifier.classify(frame)

            end = time.perf_counter()
            frame_time = (end - start) * 1000
            frame_times.append(frame_time)

            if len(frame_times) % 30 == 0:
                avg = np.mean(frame_times[-30:])
                print(f"FPS: {1000/avg:.1f} | {label}: {conf:.2f}")

    except KeyboardInterrupt:
        print(f"\nAverage frame time: {np.mean(frame_times):.1f}ms")
    finally:
        picam2.stop()

if __name__ == '__main__':
    main()

Running this gives about 8-9 FPS with MobileNet v3 Large (the extra time is camera capture and preprocessing). With MobileNet v3 Small INT8, I hit 18-20 FPS consistently.

Thermal Throttling: The Hidden Variable

After running continuously for 10 minutes, the Pi 4 throttles. Without a heatsink, I saw the CPU drop from 1.5GHz to 1.0GHz, increasing MobileNet v3 Large latency from 89ms to ~130ms.

A $5 aluminum heatsink with a small fan keeps it at 1.5GHz indefinitely. If you’re deploying for real, this isn’t optional. Check throttling status with:

vcgencmd get_throttled
# 0x0 = no throttling
# 0x50005 = currently throttling due to temperature

When to Pick Which Model

I’d use MobileNet v3 Small INT8 for any real-time application where you need 15+ FPS. The accuracy (67% top-1) is fine for binary classification tasks or when you only care about top-5.

MobileNet v3 Large (FP32 or INT8) is my default for applications where accuracy matters and 10 FPS is acceptable. It’s the sweet spot.

EfficientNet-Lite0 makes sense on more powerful edge devices — Coral Edge TPU, Jetson Nano, or phones with dedicated NPUs. On CPU-only boards like the Pi 4, the architecture just doesn’t translate to speed.

And what about smaller EfficientNet variants? I haven’t tested EfficientNet-Lite-EdgeTPU models, which are further optimized. My best guess is they’d close the gap but probably not surpass MobileNet v3 on pure CPU inference.

FAQ

Q: Can I run MobileNet v3 on Raspberry Pi Zero 2 W?
The Zero 2 W has a quad-core Cortex-A53 at 1GHz — roughly 3-4x slower than Pi 4. Expect MobileNet v3 Small at around 100-120ms (10 FPS), which is borderline usable. MobileNet v3 Large would be too slow for real-time use. INT8 quantization helps significantly here.

Q: Does overclocking the Pi 4 improve inference speed?
Yes, roughly linearly. At 2.0GHz (stable with good cooling), MobileNet v3 Large drops from 89ms to about 67ms. But you need active cooling and a quality power supply. The gains aren’t worth the reliability risk for production deployments.

Q: Should I use TensorFlow Lite or PyTorch Mobile on Raspberry Pi?
TFLite, without question. PyTorch Mobile’s CPU backend is significantly slower on ARM — I measured 2-3x higher latency for equivalent models. The XNNPACK integration in TFLite is years ahead for ARM deployment.

What I’m Still Curious About

Google released MobileNet v4 in 2024 with “Universal Inverted Bottleneck” blocks, claiming better accuracy-latency trade-offs across all hardware. I haven’t benchmarked it on Pi 4 yet — the official TFLite models weren’t available when I ran these tests. If someone’s done this comparison, I’d be interested to see the numbers.

The other open question: how do these models compare when you add post-processing like NMS for detection variants? The raw classification latency is only part of the story for object detection pipelines. But that’s a different benchmark entirely.

For now, MobileNet v3 Large remains my go-to for Pi 4 deployments. 89ms, 75% accuracy, battle-tested architecture. Sometimes the obvious choice is obvious for good reasons.