gc.collect() Slows Python 32%: When Manual GC Hurts

The gc.collect() Trap

Calling gc.collect() manually feels responsible. You’re cleaning up after yourself, preventing memory bloat, being a good citizen. Except when it makes your code 32% slower.

I ran into this debugging a data pipeline that processed 100K JSON records. Someone had sprinkled gc.collect() after every batch “to keep memory under control.” The result? What should’ve taken 12 seconds was taking 16. The fix was deleting those helpful lines.

Python’s garbage collector is tuned for typical workloads. When you override it, you’re betting you know better than the heuristics CPython developers spent years optimizing. Sometimes you do. Most of the time, you don’t.

How Python’s GC Actually Works

Python uses reference counting as its primary memory management strategy. When an object’s reference count hits zero, it’s deallocated immediately. No waiting for a collection cycle.

But reference counting can’t handle circular references — think a parent object holding a child that references the parent. That’s where the generational garbage collector comes in.

Python divides objects into three generations based on survival:

• Generation 0: Newly created objects. Collected most frequently.
• Generation 1: Objects that survived one GC pass. Collected less often.
• Generation 2: Long-lived objects. Collected rarely.

The thresholds that trigger collection are tunable:

import gc

print(gc.get_threshold())
# (700, 10, 10) on Python 3.11

That means:
– Gen 0 collects after 700 allocations
– Gen 1 collects after 10 Gen 0 collections
– Gen 2 collects after 10 Gen 1 collections

The number of Gen 0 objects at the next collection is roughly $N_0 = 700 + \text{survivors}$, where survivors are objects promoted to Gen 1. For Gen 1, the trigger condition is:

$$\text{count}_{\text{Gen0}} \geq 10 \times \text{threshold}_{\text{Gen0}}$$

Most objects die young (the “weak generational hypothesis”). Python exploits this by checking Gen 0 frequently and Gen 2 rarely. When you call gc.collect(), you force a full scan of all generations — including Gen 2 objects that likely don’t need checking.

The Benchmark: Loop With and Without gc.collect()

Here’s the test case — processing batches of data with manual GC calls:

import gc
import time
import random

def process_batch_with_gc(n_batches=1000, batch_size=100):
    """Process batches, calling gc.collect() after each."""
    results = []
    for _ in range(n_batches):
        batch = [{'id': i, 'value': random.random()} for i in range(batch_size)]
        processed = [d['value'] * 2 for d in batch]
        results.extend(processed)
        gc.collect()  # "Helpful" cleanup
    return results

def process_batch_without_gc(n_batches=1000, batch_size=100):
    """Same processing, no manual GC."""
    results = []
    for _ in range(n_batches):
        batch = [{'id': i, 'value': random.random()} for i in range(batch_size)]
        processed = [d['value'] * 2 for d in batch]
        results.extend(processed)
    return results

# Warmup
process_batch_with_gc(10)
process_batch_without_gc(10)

# Benchmark
start = time.perf_counter()
process_batch_with_gc()
with_gc_time = time.perf_counter() - start

start = time.perf_counter()
process_batch_without_gc()
without_gc_time = time.perf_counter() - start

print(f"With gc.collect(): {with_gc_time:.4f}s")
print(f"Without gc.collect(): {without_gc_time:.4f}s")
print(f"Overhead: {(with_gc_time / without_gc_time - 1) * 100:.1f}%")

On Python 3.11.7 (M1 MacBook Pro):

With gc.collect(): 0.4821s
Without gc.collect(): 0.3654s
Overhead: 31.9%

That’s not a rounding error. Manual GC calls added a third more runtime.

Why gc.collect() Costs So Much

Each gc.collect() scans all three generations. For Gen 2, that means walking every long-lived object in your process — imports, class definitions, global state. These objects aren’t going anywhere, but you’re paying to check them anyway.

The cost scales with the number of tracked objects. You can see this with gc.get_count():

import gc

print(gc.get_count())
# (421, 3, 1)  -- counts per generation before collection

After a forced collection:

gc.collect()
print(gc.get_count())
# (14, 0, 0)  -- almost everything cleared

But now you’ve just burned CPU time checking thousands of objects. The automatic collector would’ve only scanned Gen 0 (421 objects) until the threshold triggered a Gen 1 pass.

The time complexity for a full collection is roughly $O(N)$ where $N$ is the total number of tracked objects across all generations. For a Gen 0-only collection, it’s $O(N_0)$ where $N_0 \ll N$ for most programs.

When Manual gc.collect() Actually Helps

There are real use cases. Just fewer than you’d think.

After bulk deletes in long-running processes. If you just dropped 10GB of cached data and won’t allocate more for a while, gc.collect() can return memory to the OS sooner:

import gc

# Drop a massive cache
my_giant_cache.clear()
gc.collect()  # Force dealloc before the next request

In containerized environments with tight memory limits. When you’re running on 512MB RAM and approaching the OOM killer, forcing a collection before a known allocation spike can prevent crashes:

import gc
import psutil

if psutil.virtual_memory().percent > 85:
    gc.collect()  # Desperation move before next batch

Testing memory leak behavior. When writing tests to verify objects are actually getting cleaned up, manual GC ensures you’re not just seeing delayed collection:

import gc
import weakref

obj = SomeClass()
ref = weakref.ref(obj)
del obj
gc.collect()  # Force collection for test assertion
assert ref() is None, "Object still alive!"

But in request-response servers, data processing loops, or typical batch jobs? You’re probably making things worse.

The Real Problem: Circular References

If you’re calling gc.collect() because you think you have memory leaks, the issue isn’t GC frequency — it’s your object graph.

Circular references are the classic culprit:

class Node:
    def __init__(self, value):
        self.value = value
        self.parent = None
        self.children = []

    def add_child(self, child):
        child.parent = self  # Circular reference
        self.children.append(child)

# This creates a cycle
root = Node("root")
child = Node("child")
root.add_child(child)

The GC will eventually clean this up, but if you’re creating millions of these, you’ll accumulate Gen 1 and Gen 2 objects. The fix isn’t more gc.collect() calls — it’s breaking the cycle:

import weakref

class Node:
    def __init__(self, value):
        self.value = value
        self.parent = None  # Use weakref instead
        self.children = []

    def add_child(self, child):
        child.parent = weakref.ref(self)  # No circular ref
        self.children.append(child)

Now when you delete root, the child’s weak reference doesn’t prevent deallocation.

Disabling GC: The Nuclear Option

Some high-performance code disables GC entirely:

import gc

gc.disable()

# Run your tight loop
for i in range(1_000_000):
    process(i)

gc.enable()
gc.collect()  # Clean up at the end

This makes sense when:
– You’re not creating circular references
– The workload is short (seconds, not hours)
– You can tolerate memory growth during execution
– You’ve profiled and GC is a measurable bottleneck

Instagram famously disabled GC in production (Instagram Engineering blog, 2017) by ensuring their Django request handlers didn’t create cycles. They ran a manual collection between requests instead.

But this requires discipline. One accidental circular reference and you’ve got a memory leak that won’t surface until production.

Tuning Thresholds Instead

Before you reach for gc.disable(), try adjusting the thresholds:

import gc

# Default: (700, 10, 10)
gc.set_threshold(5000, 20, 20)

This makes Gen 0 collections less frequent (every 5000 allocations instead of 700) and delays Gen 1/2 scans even more. For workloads with short-lived objects, this can cut GC overhead significantly.

The tradeoff is higher peak memory usage. You’re letting more garbage accumulate between collections. For a batch job that runs for 30 seconds and exits, that’s fine. For a server running for days, you might hit memory limits.

I tested the same benchmark with relaxed thresholds:

import gc
gc.set_threshold(5000, 20, 20)

start = time.perf_counter()
process_batch_without_gc()
tuned_time = time.perf_counter() - start

print(f"Tuned thresholds: {tuned_time:.4f}s")
# Tuned thresholds: 0.3401s  (7% faster than default)

Not a huge win for this toy example, but for real workloads with millions of allocations, it adds up.

Profiling GC Impact

Don’t guess — measure. Python’s gc module exposes stats:

import gc

# Enable GC debugging
gc.set_debug(gc.DEBUG_STATS)

# Run your code
process_batch_without_gc()

# Output shows collection counts and timing

For production systems, I’ve used tracemalloc vs memray vs Py-Spy to identify GC hotspots. The profiler overhead matters — tracemalloc adds 10-30% slowdown, while py-spy samples without much impact.

You can also check GC stats programmatically:

import gc

stats = gc.get_stats()
for gen, stat in enumerate(stats):
    print(f"Gen {gen}: {stat['collections']} collections")

If you see Gen 2 collecting hundreds of times during a short workload, something’s creating long-lived cycles.

What I’d Do Next Time

When I see gc.collect() in code review, my first question is: why? If the answer is “to prevent memory leaks,” we’re solving the wrong problem. Fix the leaks, don’t mask them with manual GC.

If the answer is “I saw memory growing,” I’d profile first. Maybe the growth is expected. Maybe it’s caching that caps out. Maybe it’s a real leak, but one that gc.collect() won’t fix (C extensions, file handles, database connections).

And if GC is genuinely a bottleneck — which happens in tight loops with millions of tiny objects — I’d try threshold tuning before disabling GC. The default (700, 10, 10) is conservative. Bumping Gen 0 to 5000-10000 is often safe and measurably faster.

I’m still curious about Python 3.13’s per-interpreter GIL and whether that changes GC behavior. If each subinterpreter has its own GC state, maybe we’ll see different tuning strategies for parallel workloads. But that’s speculation.

For now, the rule is simple: let the GC do its job unless you’ve profiled and proven it’s the problem. Those Dark Chocolate Espresso Beans you’re stress-eating at 2am? They won’t help if you’re fighting a performance issue you created yourself.

FAQ

Q: Should I call gc.collect() in a Flask request handler?

No. Each request is short-lived, and Python’s automatic GC will clean up after the response is sent. Manual collection just adds latency to every request. If you’re seeing memory growth, look for leaked connections, unclosed files, or session state that’s not expiring.

Q: Does gc.collect() free memory back to the OS?

Not always. Python’s memory allocator (pymalloc) holds onto freed memory for reuse. On Linux, you might see resident memory (RSS) stay high even after collection. If you need to release memory to the OS, you’d need to use gc.collect() and rely on the platform’s malloc implementation to actually return pages — which is not guaranteed.

Q: How do I find circular references in my code?

Use gc.get_referrers() on suspected objects, or enable gc.DEBUG_SAVEALL to keep all unreachable objects in gc.garbage. In practice, weak references (weakref.ref) or redesigning the object graph to avoid cycles is cleaner than debugging GC issues.