5 NumPy Tricks That Cut Portfolio Backtest Time by 10x

The Problem: 45-Minute Backtests for a Simple Moving Average Strategy

I had a backtest that took 45 minutes to run. The strategy was trivial — a dual moving average crossover on SPY, testing across 200 combinations of short and long windows. Nothing fancy. Just buying when the 20-day MA crossed above the 50-day, selling when it crossed below.

The code looked clean. Pandas rolling windows, a simple loop over parameter combinations, position sizing logic. But every time I added another year of data or expanded the parameter grid, the runtime exploded.

Turns out the bottleneck wasn’t the strategy logic — it was how I was using arrays.

Trick 1: Vectorize Position Calculations with np.where Instead of Loops

My original position logic looked like this:

import pandas as pd
import numpy as np

# positions was initialized as zeros
for i in range(1, len(signals)):
    if signals[i] == 1 and positions[i-1] == 0:
        positions[i] = 1  # Enter long
    elif signals[i] == -1 and positions[i-1] == 1:
        positions[i] = 0  # Exit
    else:
        positions[i] = positions[i-1]  # Hold

This iterated over 250,000+ rows (10 years of daily data). For 200 parameter combinations, that’s 50 million iterations.

The fix: np.where with cumulative logic.

# signals: 1 = bullish cross, -1 = bearish cross, 0 = no signal
entry_points = (signals == 1).astype(int)
exit_points = (signals == -1).astype(int)

# Cumulative sum gives us state transitions
net_signals = entry_points - exit_points
positions = (np.maximum.accumulate(net_signals) > 0).astype(int)

This cut that section from ~8 seconds per backtest to under 0.1 seconds. The trick is that np.maximum.accumulate propagates the most recent entry signal forward, effectively holding the position until an exit. No loop needed.

One gotcha: this assumes you can’t re-enter a position while already in one. If your strategy allows pyramiding (adding to positions), you’ll need a different approach — probably np.cumsum with clipping.

Trick 2: Rolling Window Calculations with Stride Tricks Instead of Pandas

Pandas rolling() is convenient but slow for large parameter sweeps. I was computing 200 pairs of moving averages, and pandas was copying data for every window.

# Old way: pandas rolling (takes ~2 seconds per parameter pair)
fast_ma = prices.rolling(window=fast_period).mean()
slow_ma = prices.rolling(window=slow_period).mean()

NumPy’s stride tricks let you create a view of overlapping windows without copying:

from numpy.lib.stride_tricks import sliding_window_view

def fast_rolling_mean(arr, window):
    # sliding_window_view creates (len-window+1, window) shaped view
    windows = sliding_window_view(arr, window)
    return np.concatenate([np.full(window-1, np.nan), windows.mean(axis=1)])

fast_ma = fast_rolling_mean(prices, fast_period)
slow_ma = fast_rolling_mean(prices, slow_period)

This dropped the moving average computation from 2 seconds to 0.3 seconds per pair. The speedup comes from avoiding pandas overhead and working directly with contiguous memory.

Warning: sliding_window_view is in numpy 1.20+. If you’re on an older version, you’ll need as_strided directly (more error-prone — easy to create memory bugs).

Trick 3: Precompute Returns with np.diff for All Parameters at Once

I was computing log returns inside the backtest loop:

for fast, slow in param_grid:
    # ... generate signals ...
    daily_returns = np.log(prices[1:] / prices[:-1])
    strategy_returns = daily_returns * positions[1:]

This recalculated returns 200 times for the exact same price series. Precompute once:

log_returns = np.diff(np.log(prices))

for fast, slow in param_grid:
    # ... generate signals ...
    strategy_returns = log_returns * positions[1:]

Seems obvious in hindsight. But when you’re focused on strategy logic, these redundant calculations slip through.

The mathematical rationale: log returns are time-additive, so $r_t = log(P_t / P_{t-1}) = log P_t – log P_{t-1}$. Computing this once and reusing it is trivial, but I’d estimate 30% of backtests I’ve reviewed recalculate returns unnecessarily.

Trick 4: Broadcast Comparison for Signal Generation

Generating crossover signals originally looked like this:

signals = np.zeros(len(prices))
for i in range(1, len(prices)):
    if fast_ma[i] > slow_ma[i] and fast_ma[i-1] <= slow_ma[i-1]:
        signals[i] = 1  # Bullish cross
    elif fast_ma[i] < slow_ma[i] and fast_ma[i-1] >= slow_ma[i-1]:
        signals[i] = -1  # Bearish cross

Another loop. Another bottleneck.

Vectorized version:

fast_above = fast_ma > slow_ma
cross_up = fast_above[1:] & ~fast_above[:-1]
cross_down = ~fast_above[1:] & fast_above[:-1]

signals = np.zeros(len(prices))
signals[1:][cross_up] = 1
signals[1:][cross_down] = -1

This uses boolean array indexing. cross_up is a boolean mask marking where the fast MA just crossed above. We apply that mask directly to assign signal values.

Runtime for signal generation dropped from 1.5 seconds to 0.05 seconds per parameter pair.

Trick 5: Avoid DataFrame Operations in Hot Loops

My original backtest built a pandas DataFrame for each parameter combination:

for fast, slow in param_grid:
    df = pd.DataFrame({
        'price': prices,
        'fast_ma': fast_ma,
        'slow_ma': slow_ma,
        'signal': signals,
        'position': positions,
        'returns': returns
    })
    sharpe = df['returns'].mean() / df['returns'].std() * np.sqrt(252)

DataFrame construction is expensive. For 200 iterations, this was taking 6 seconds total.

Just use NumPy arrays:

for fast, slow in param_grid:
    # Work with numpy arrays directly
    sharpe = strategy_returns.mean() / strategy_returns.std() * np.sqrt(252)
    results[fast, slow] = sharpe

No DataFrame. Same result. Dropped this section from 6 seconds to 0.2 seconds.

I only convert to a DataFrame at the very end for analysis:

results_df = pd.DataFrame(results, columns=['fast', 'slow', 'sharpe'])

The Math Behind Sharpe Ratio (and Why You Can Compute It Without DataFrames)

The Sharpe ratio is defined as:

$$S = frac{bar{r} – r_f}{sigma_r} sqrt{T}$$

where $bar{r}$ is the mean return, $r_f$ is the risk-free rate (often assumed 0 for relative comparison), $sigma_r$ is the standard deviation of returns, and $T$ is the number of periods per year (252 for daily data).

You don’t need pandas for this. NumPy’s mean() and std() work directly on arrays and are faster because they avoid pandas indexing overhead.

If you’re comparing strategies, you only care about relative Sharpe, so you can skip the risk-free rate adjustment:

$$S_{text{rel}} = frac{bar{r}}{sigma_r} sqrt{252}$$

This is what most practitioners compute anyway.

Combined Results: 45 Minutes to 4 Minutes

Here’s the before/after timing breakdown for 200 parameter combinations on 10 years of SPY daily data:

The biggest win was vectorizing the position calculation loop (Trick 1). That single change accounted for 60% of the speedup.

When These Tricks Don’t Help

Not every backtest bottleneck is NumPy-solvable.

If your strategy requires path-dependent logic — say, tracking multiple open positions with different entry prices, or applying complex stop-loss rules that depend on portfolio state — you can’t fully vectorize. In those cases, Numba’s JIT compilation is your friend:

from numba import jit

@jit(nopython=True)
def compute_positions_with_stops(signals, prices, stop_pct):
    positions = np.zeros(len(signals))
    entry_price = 0.0
    for i in range(1, len(signals)):
        if signals[i] == 1 and positions[i-1] == 0:
            positions[i] = 1
            entry_price = prices[i]
        elif positions[i-1] == 1:
            if prices[i] < entry_price * (1 - stop_pct):
                positions[i] = 0  # Stop loss triggered
            else:
                positions[i] = 1
    return positions

Numba compiles this to C-like speed. For inherently sequential logic, it’s often faster than trying to contort it into vectorized form.

But if your backtest is embarrassingly parallel (independent parameter sweeps), vectorization wins.

FAQ

Q: Can I use these tricks with minute-level or tick data?

Yes, but memory becomes the bottleneck. At 1-minute bars, 10 years of SPY is ~1.3 million rows. NumPy arrays are still fine, but pandas starts to struggle. Consider using Polars or DuckDB for data loading, then convert to NumPy for computation. I’ve run 5-year minute backtests with these tricks in under 10 minutes on a MacBook Pro (M1, 16GB).

Q: What about transaction costs and slippage?

These tricks speed up the math, not the modeling. You still need to subtract transaction costs:

trades = np.abs(np.diff(positions))
transaction_costs = trades * prices[1:] * 0.0005  # 5 bps per trade
net_returns = strategy_returns - transaction_costs

Just make sure you vectorize the cost calculation too. Don’t loop over trades.

Q: Does this work with multiple assets (portfolio backtests)?

Yes, but you’ll need 2D arrays. Prices become a (time, assets) matrix, positions become (time, assets), etc. The tricks still apply — you’re just broadcasting over an extra dimension. Watch out for memory with large universes (500+ stocks × 10 years × minute bars = hundreds of millions of data points).

What I’d Do Differently

If I were starting this backtest from scratch, I’d use Polars for the initial data pipeline (loading CSVs, handling missing data) and only convert to NumPy for the hot loop. Polars is lazy-evaluated and multi-threaded, so the data prep would be faster too.

I’d also profile earlier. I wasted a week optimizing the wrong parts of the code before realizing the position calculation loop was the real bottleneck. Python’s cProfile module is invaluable:

import cProfile
cProfile.run('run_backtest()', sort='cumtime')

This instantly shows where your code is spending time.

And if I needed to scale beyond 200 parameter combinations (say, 10,000 for a genetic algorithm or Bayesian optimization), I’d move to joblib or ray for parallel execution across CPU cores. But that’s a separate post.

For now, these five tricks are enough to make most single-threaded backtests painless.