tracemalloc vs memray vs Py-Spy: Profiler Overhead Cost

Running a memory profiler in production shouldn’t crash your app

I learned this the hard way when memray brought a Flask API from 200 req/s to 12 req/s in production. The memory leak I was hunting? A 50MB slow growth over 6 hours. The profiler overhead? 1.6GB of extra allocations and 94% CPU spike.

Most profiling guides skip the critical question: what does the profiler itself cost? You can’t fix a memory leak if the profiler consumes more resources than the leak. Here’s what actually happens when you run tracemalloc, memray, and Py-Spy on the same workload — with specific numbers for CPU, memory, and disk overhead.

The test setup (and why it matters)

I ran three scenarios on Python 3.11.7, Ubuntu 22.04, 4-core VM with 8GB RAM:

1. Baseline web server: FastAPI app processing 10,000 POST requests with JSON payloads (2KB each), Pandas DataFrame manipulation, and SQLAlchemy queries. Peak memory ~340MB, avg response time 18ms.
2. Batch data pipeline: Reading 500MB CSV, groupby aggregations, writing results. No profiling — just the workload.
3. Long-running daemon: Simulated background task with periodic NumPy array allocations, leaked references (intentional), 2-hour runtime.

Each profiler ran on the same code. I measured wall-clock time, RSS memory (/proc/self/status), CPU percentage (psutil), and disk I/O for trace files.

tracemalloc: built-in but memory-hungry

Python’s stdlib profiler. Zero install friction, decent granularity.

import tracemalloc
import time

tracemalloc.start()

# Your code here
data = [list(range(10000)) for _ in range(100)]
time.sleep(1)

snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')

for stat in top_stats[:5]:
    print(stat)

On the FastAPI workload, tracemalloc added:

• +28% memory overhead: Baseline 340MB → 435MB with profiling
• +41% wall-clock time: 10k requests took 11.2s instead of 7.9s
• +15% CPU usage: From 62% avg to 77% (single-core pegged)

The memory hit comes from storing every allocation’s filename, line number, and size. Each allocation adds ~240 bytes of metadata. For workloads with millions of small allocations (like our Pandas ops), that’s a 100MB+ bookkeeping cost.

Disk footprint: None by default. Snapshots live in RAM unless you serialize them. I saved snapshots every 60s to JSON — 15MB per snapshot for this workload.

Where it breaks: high-frequency allocations. If your code creates 1M+ temporary objects per second (Pandas .apply(), NumPy broadcasting, etc.), tracemalloc can double your memory usage.

memray: detailed flamegraphs at a brutal cost

memray from Bloomberg gives gorgeous HTML flamegraphs and native C/C++ stack traces. The overhead is… significant.

# Run via CLI
import subprocess
subprocess.run(["memray", "run", "-o", "output.bin", "my_script.py"])

# Generate report
subprocess.run(["memray", "flamegraph", "output.bin"])

Same FastAPI test:

• +94% memory overhead: 340MB → 660MB
• +153% wall-clock time: 7.9s → 20.0s
• +180% CPU usage: Single-core maxed, profiler thread consumed ~90% of a second core

Disk footprint: 1.2GB trace file for 10k requests. The binary format is dense, but it records everything — malloc/free pairs, stack traces, timestamps. For the 2-hour daemon, trace file hit 18GB before I killed it.

Why so expensive? memray uses LD_PRELOAD hooks to intercept every malloc()/free() call, then writes to disk continuously. On allocation-heavy code (like NumPy array reshapes), this becomes I/O bound.

Where it shines: debugging gnarly leaks in C extensions. If you suspect a leak in a Cython module or a native library, memray’s C stack traces are irreplaceable. But you can’t leave it running in production.

Py-Spy: the “set it and forget it” option

Py-Spy is a sampling profiler — it snapshots the call stack every N milliseconds without modifying your code. Written in Rust, attaches to running processes.

# Attach to PID, sample every 10ms, 60s duration
py-spy record -o profile.svg --pid 12345 --duration 60 --rate 100

FastAPI test results:

• +3% memory overhead: 340MB → 350MB (mostly the Py-Spy process itself, not the target)
• +5% wall-clock time: 7.9s → 8.3s
• +8% CPU usage: Sampling thread adds ~5% on one core

Disk footprint: 850KB SVG flamegraph for 60s of profiling. Tiny compared to memray.

The catch: Py-Spy is a CPU profiler, not a memory profiler. It shows you where time is spent, not where memory is allocated. For memory leaks, you need to infer from which functions are called most often (and hope they correlate with allocations).

But here’s the clever part: if you run py-spy top --pid <PID>, it gives you a live htop-style view of which functions are hogging CPU. I’ve used this to catch runaway threads that were thrashing on allocation retries — the allocation itself wasn’t the issue, but the CPU spike led me to the leak.

Overhead breakdown: the numbers you actually care about

For the 2-hour daemon workload (leaked 800MB over runtime):

• tracemalloc: Profiler metadata consumed 1.1GB (more than the leak!). Snapshots saved every 5min = 180MB total disk.
• memray: Trace file hit 18GB. Killed after 90min because disk filled up.
• Py-Spy: Sampled for 10min intervals, 6 SVG files totaling 4MB. Caught the leak indirectly (allocation-heavy function appeared in 89% of samples).

When to use which

Use tracemalloc when:
– You control the code and can add start/stop hooks
– Memory overhead <30% is acceptable (dev/staging environments)
– You need exact line numbers for allocations
– Workload has <100k allocations/sec

Example: debugging a Flask route that allocates 2GB on specific inputs. Run tracemalloc for that route only:

@app.route('/heavy')
def heavy_route():
    tracemalloc.start()
    result = expensive_operation()
    snapshot = tracemalloc.take_snapshot()
    tracemalloc.stop()
    # Log top 10 to file
    return result

Use memray when:
– You’re hunting a leak in a C extension or Cython code
– You’re in a dev environment with disk space to burn
– You need ironclad proof for a bug report (flamegraph screenshot wins arguments)
– You can afford 2-3x slowdown

Don’t run memray in production. Period. The one exception: if you have a pre-prod environment with production-like traffic, run memray for 5-10 minutes to capture steady-state behavior.

Use Py-Spy when:
– You need always-on profiling in production (it’s cheap enough)
– The leak correlates with CPU-heavy code paths
– You can’t modify the code (legacy app, third-party service)
– You want a quick “what’s this process doing right now” answer

Py-Spy saved me on a Celery worker that leaked 3GB/day. Couldn’t reproduce locally, couldn’t add tracemalloc without redeploying. Attached Py-Spy to the prod worker, saw that 78% of samples were in json.loads() inside a retry loop. Turned out the retry logic never freed the previous parse attempt. Fixed in 10 lines.

Hybrid approach: the 2-stage hunt

What I actually do now:

1. Stage 1 – Detection (Py-Spy): Run py-spy top on prod for 60s. Look for functions spending >20% of time in allocation-heavy patterns (list comprehensions, Pandas ops, json.loads).
2. Stage 2 – Confirmation (tracemalloc): Add tracemalloc hooks to the suspicious function only, deploy to staging, run load test. Get exact line numbers.
3. Stage 3 – Deep dive (memray, if needed): If tracemalloc shows the leak is in a C extension, pull out memray in a local Docker container with production data snapshot.

This avoids running expensive profilers in prod while still getting root-cause info fast.

The math behind sampling overhead

Why is Py-Spy so much cheaper? It’s all about the sampling rate.

tracemalloc hooks every allocation. For $N$ allocations, overhead is $O(N)$ in both time and space.

memray also hooks every allocation but adds disk I/O. Overhead is $O(N \cdot \log N)$ if you account for writes and occasional trace compaction.

Py-Spy samples at rate $r$ (default 100 Hz). For a workload of duration $T$ seconds, it takes $r \cdot T$ snapshots regardless of how many allocations happen. Overhead is $O(r \cdot T)$, independent of allocation count. For $r = 100$, $T = 60$, that’s 6000 snapshots. Compared to 10M allocations in the same period, you’re sampling $0.06\%$ of events.

The trade-off: sampling can miss short-lived allocations. If your leak is in a function that runs for <10ms every 5 minutes, Py-Spy might not catch it. But in practice, leaks grow over time — you’ll see the culprit function appear in more and more samples as the leak accumulates.

Surprising edge cases

tracemalloc doesn’t track munmap: I spent an hour confused why tracemalloc showed memory growing to 800MB, but RSS (real memory) stayed at 400MB. Turns out NumPy was free()-ing memory, but the allocator wasn’t returning it to the OS (MADV_DONTNEED wasn’t called). tracemalloc only sees Python-level allocations, not what the OS actually reclaims.

memray crashes on fork(): If your app uses multiprocessing with fork, memray’s pre-fork hooks can cause child processes to hang on startup. The fix is to use forkserver or spawn mode, but that changes semantics. Caught me off guard on a Celery app.

Py-Spy requires ptrace permissions: In Docker, you need --cap-add=SYS_PTRACE or --privileged. On some locked-down prod servers, this is a non-starter. Check with your ops team before assuming you can attach.

FAQ

Q: Can I use multiple profilers at once?

Bad idea. tracemalloc + memray together will give you 150%+ overhead and potentially crash due to conflicting malloc hooks. The one safe combo is Py-Spy + any memory profiler, since Py-Spy doesn’t hook allocations — it just reads /proc/<pid>/maps.

Q: Why not just use memory_profiler from PyPI?

memory_profiler is line-by-line but absurdly slow (10-100x slowdown). It’s a decorator-based wrapper around psutil that samples RSS every few milliseconds. Fine for profiling a single function in isolation, useless for real workloads. I didn’t include it here because the overhead makes it non-viable for production-adjacent environments.

Q: Does Python 3.13 change any of this?

Not yet. The Per-Interpreter GIL work doesn’t affect profiler overhead — these tools hook at the C API or syscall level, below the GIL. However, if you’re running free-threaded Python (PEP 703), be aware that tracemalloc’s per-thread bookkeeping might add extra memory cost. I haven’t benchmarked this yet on 3.13t.

What I’d do differently next time

I wasted a week trying to debug a 200MB Celery leak with memray, generating 40GB of trace files and learning nothing. The leak was in a SQLAlchemy session that wasn’t being closed — trivial to spot with tracemalloc in 10 minutes.

The lesson: start cheap. Py-Spy first (30 seconds), then tracemalloc on suspicious code paths (5 minutes), then memray only if you’re sure it’s a native extension (and you have disk space). Don’t reach for the nuclear option first.

For always-on production monitoring, I now run Py-Spy on a cron (every 6 hours, 60-second recording, SVG uploaded to S3). Costs ~0.5% CPU averaged over the day, catches 80% of leaks before they become incidents. When Py-Spy shows a new function dominating samples, that’s the trigger to add tracemalloc hooks and investigate.

One open question: I haven’t found a good way to profile memory in async Python (asyncio event loops). tracemalloc works but doesn’t attribute allocations to coroutines — you just get a pile of line numbers in your async handlers. If you’ve solved this, I’m curious what you’re using. Might need Scalene or a similar GPU/CPU/memory hybrid profiler, but I haven’t stress-tested it yet.

For now: Py-Spy for detection, tracemalloc for confirmation, memray for C extensions only. That’s the cost-effective stack.