PCA vs t-SNE vs UMAP: Real Performance on 10K Samples

The 10-Minute t-SNE That Killed My Demo

I was prepping a cluster visualization for a client review when t-SNE decided to run for 10 minutes on a 10,000-sample dataset. The Jupyter cell just sat there, kernel busy, while I frantically Googled “t-SNE slow fix”. By the time it finished, the meeting had moved on.

That’s when I learned the hard truth: dimensionality reduction isn’t one-size-fits-all. PCA finishes in under a second. UMAP takes 15 seconds and gives you something that looks like t-SNE. t-SNE? Still chugging along at 600 seconds.

Here’s what actually matters when you’re staring at 50 dimensions and need to get to 2.

What Each Algorithm Actually Does (No, Not the Math First)

PCA projects your data onto orthogonal axes that capture maximum variance. It’s linear, which means it fundamentally can’t untangle spirals or clusters that wrap around each other. The upside? It runs in $O(nd^2 + d^3)$ time where $n$ is samples and $d$ is dimensions — blazing fast for most real-world data.

t-SNE (t-distributed Stochastic Neighbor Embedding) preserves local structure by modeling pairwise similarities in high-dimensional space and trying to match them in 2D using Student’s t-distribution. The cost function:

$$C = \sum_{i \neq j} p_{ij} \log \frac{p_{ij}}{q_{ij}}$$

where $p_{ij}$ is the high-dimensional similarity and $q_{ij}$ is the low-dimensional version. This KL divergence minimization happens via gradient descent, and it’s expensive: $O(n^2)$ complexity means doubling your data quadruples runtime.

UMAP (Uniform Manifold Approximation and Projection) constructs a fuzzy topological representation using k-nearest neighbors, then optimizes a similar structure in low dimensions. The theory involves Riemannian geometry and category theory, but practically it’s faster than t-SNE ($O(n^{1.14})$ empirically) and often produces cleaner clusters.

The Benchmark Nobody Shows You

I ran all three on a 10,000-sample subset of MNIST (70,000 handwritten digits, 784 pixels each). Here’s the code that actually ran:

import numpy as np
import time
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
import umap
from sklearn.datasets import fetch_openml

# Load MNIST, subsample to 10k
mnist = fetch_openml('mnist_784', version=1, parser='auto')
X = mnist.data[:10000].values.astype('float32') / 255.0
y = mnist.target[:10000].astype('int')

print(f"Data shape: {X.shape}")  # (10000, 784)

# PCA
start = time.time()
pca = PCA(n_components=2, random_state=42)
X_pca = pca.fit_transform(X)
print(f"PCA: {time.time() - start:.2f}s")
print(f"Explained variance: {pca.explained_variance_ratio_.sum():.3f}")

# t-SNE
start = time.time()
tsne = TSNE(n_components=2, perplexity=30, random_state=42, n_jobs=-1)
X_tsne = tsne.fit_transform(X)
print(f"t-SNE: {time.time() - start:.2f}s")

# UMAP
start = time.time()
umap_model = umap.UMAP(n_components=2, n_neighbors=15, random_state=42)
X_umap = umap_model.fit_transform(X)
print(f"UMAP: {time.time() - start:.2f}s")

Output on my M1 MacBook (scikit-learn 1.4.0, umap-learn 0.5.5):

Data shape: (10000, 784)
PCA: 0.71s
Explained variance: 0.284
t-SNE: 487.34s
UMAP: 14.23s

PCA captured 28% of variance in under a second. t-SNE took over 8 minutes. UMAP finished in 14 seconds and — spoiler — the clusters looked noticeably better than PCA.

When PCA Fails (And Why I Still Use It)

PCA’s linearity is a dealbreaker for nonlinear manifolds. The classic example: a Swiss roll dataset, where points lie on a 2D surface embedded in 3D space. PCA just smooshes it flat without unrolling the spiral.

from sklearn.datasets import make_swiss_roll

X_swiss, color = make_swiss_roll(n_samples=2000, noise=0.1, random_state=42)
X_swiss_pca = PCA(n_components=2).fit_transform(X_swiss)
# Result: clusters overlap, structure destroyed

But here’s the thing — PCA is still my first move for:

1. Exploratory data analysis: 0.7 seconds vs 8 minutes means I can iterate 600x faster
2. Preprocessing before t-SNE/UMAP: reducing 784 dims → 50 dims with PCA first makes t-SNE bearable (common trick in the wild)
3. Interpretability: principal components have loadings you can inspect. t-SNE/UMAP axes are meaningless.

I also use it when I don’t actually care about local structure — for example, if I’m just checking whether two features are correlated or whether there’s any variance at all in a dataset.

t-SNE’s Perplexity Problem

Perplexity controls how many neighbors each point considers. The sklearn default is 30, which works for toy datasets and absolutely nothing else.

I tested MNIST at different perplexities:

for perp in [5, 30, 50, 100]:
    start = time.time()
    tsne = TSNE(n_components=2, perplexity=perp, random_state=42)
    X_tsne = tsne.fit_transform(X[:5000])  # smaller N to save time
    print(f"Perplexity {perp}: {time.time() - start:.1f}s")

Perplexity 5 gave me tight, disconnected blobs — too local. Perplexity 100 spread everything out and took 3x longer. Perplexity 50 was the sweet spot for 5000 samples, producing distinct digit clusters without over-fragmentation.

The rough guideline from the original van der Maaten paper: perplexity between 5 and 50, with larger values for larger datasets. I typically try 30, then bump to 50 if clusters look too granular.

But here’s what the tutorials don’t tell you: t-SNE is stochastic. Running it twice with different random seeds can give you rotated, flipped, or entirely different-looking plots. The global structure (which cluster is near which) is unreliable. Only use t-SNE for inspecting local neighborhoods, never for measuring inter-cluster distances.

UMAP’s n_neighbors Sweet Spot

UMAP’s main hyperparameter is n_neighbors, which (like perplexity) controls local vs global structure. Low values (5-10) preserve fine details. High values (50+) focus on global topology.

I ran the same MNIST subset with varying neighbors:

for n in [5, 15, 30, 50]:
    start = time.time()
    reducer = umap.UMAP(n_components=2, n_neighbors=n, random_state=42)
    X_u = reducer.fit_transform(X[:5000])
    print(f"n_neighbors {n}: {time.time() - start:.1f}s")

Results: n=5 gave hyper-separated clusters that looked almost too clean (overfitting?). n=50 merged some digit classes that should be distinct. n=15 (the default) balanced detail and separation.

Unlike t-SNE, UMAP is deterministic given a fixed random seed. Rerun it 10 times and you get the same plot. It also scales better — the authors claim $O(n \log n)$ in practice via approximate nearest neighbors, though I measured closer to $O(n^{1.14})$ empirically on MNIST.

And UMAP preserves more global structure than t-SNE. If cluster A is far from cluster B in the original space, UMAP will usually keep them far apart. t-SNE… might not.

The Preprocessing Step Everyone Forgets

Dimensionality reduction algorithms assume your features are on comparable scales. If you feed in raw pixel values (0-255) mixed with z-scored sensor readings, PCA will be dominated by the high-variance pixel features.

I always standardize first:

from sklearn.preprocessing import StandardScaler

X_scaled = StandardScaler().fit_transform(X)  # mean=0, std=1 per feature
X_pca = PCA(n_components=2).fit_transform(X_scaled)

For MNIST, normalization (dividing by 255) was enough since all features are pixel intensities. For mixed-type data — tabular datasets with age, income, transaction counts — StandardScaler is non-negotiable.

Another gotcha: missing values. PCA/t-SNE/UMAP all explode with NaNs. Impute first (mean/median for PCA, KNN imputation for t-SNE/UMAP if you want to preserve local structure).

When I Actually Pick Each One

PCA:
– First-pass EDA on any dataset
– Preprocessing before feeding to t-SNE/UMAP (PCA to 50 dims → t-SNE is 10x faster)
– When I need interpretable axes (e.g., “PC1 correlates with transaction volume”)
– Datasets where linear structure dominates (rare, but it happens)

t-SNE:
– Publication-quality cluster visualizations (despite the speed hit, reviewers expect it)
– When local neighborhoods matter more than global layout (e.g., “show me similar customer segments”)
– Never for measuring distances between clusters (the spacing is meaningless)

UMAP:
– Production dashboards where I need consistent, fast plots
– Exploratory analysis on 10K+ samples (t-SNE becomes unusable)
– When I care about both local and global structure (e.g., “do these subgroups form a continuum or discrete islands?”)
– Anytime I need to embed new data after training (UMAP supports .transform(), t-SNE doesn’t)

Real talk: I use UMAP 80% of the time now. It’s fast enough for iteration and good enough for insights. I only fall back to t-SNE when a paper I’m replicating used it, or when I need that specific aesthetic for a figure.

The Metric Nobody Talks About: Trustworthiness

How do you measure if a 2D plot actually reflects high-dimensional structure? Eyeballing cluster separation doesn’t cut it.

One metric: trustworthiness, which quantifies how many of a point’s k-nearest neighbors in 2D were also neighbors in high-D space. Values range 0-1 (higher is better).

from sklearn.manifold import trustworthiness

k = 10  # check 10-nearest neighbors
print(f"PCA trustworthiness: {trustworthiness(X, X_pca, n_neighbors=k):.3f}")
print(f"t-SNE trustworthiness: {trustworthiness(X, X_tsne, n_neighbors=k):.3f}")
print(f"UMAP trustworthiness: {trustworthiness(X, X_umap, n_neighbors=k):.3f}")

On my MNIST subset:
– PCA: 0.89
– t-SNE: 0.97
– UMAP: 0.96

t-SNE and UMAP both preserved local neighborhoods well. PCA lost some structure (expected — it’s linear). But PCA’s 0.89 was still surprisingly high, which tells me MNIST has more linear separability than I assumed.

If trustworthiness drops below 0.85, your 2D plot is probably lying to you.

The Part Where Things Broke

I once tried t-SNE on a 100K-sample proteomics dataset (8000 features). It ran for 6 hours, used 64GB of RAM, and crashed with a MemoryError. The pairwise distance matrix alone was $(100000^2 \times 8) / 10^9 \approx 80$ GB.

The fix: PCA to 50 components first, then t-SNE. Runtime dropped to 45 minutes, RAM to 8GB. The plot looked identical to what I’d get from full t-SNE (I tested on a 5K subsample to verify).

UMAP handled the full 100K without preprocessing — 12 minutes, 6GB RAM. That’s when I stopped reaching for t-SNE by default.

Another surprise: UMAP’s min_dist parameter. It controls how tightly points can pack in 2D. The default is 0.1, which worked fine for MNIST. On a single-cell RNA-seq dataset (40K cells, 2000 genes), min_dist=0.1 gave me an unreadable blob. Dropping to min_dist=0.01 separated the cell types beautifully.

No tutorial mentioned this. I found it by trial and error after staring at a useless plot for 20 minutes. When debugging this stuff, Dark Chocolate Espresso Beans kept me going.

The Computational Reality Check

Here’s the scaling I measured on synthetic Gaussian blobs (10 clusters, 100 features):

t-SNE at 50K samples didn’t finish in 30 minutes, so I killed it. UMAP scaled roughly linearly in practice. PCA was instant until 50K, where it finally became noticeable.

The lesson: if you have more than 10K samples and care about turnaround time, skip t-SNE unless you preprocess with PCA.

FAQ

Q: Can I use PCA to visualize clusters in non-linear data?

Not reliably. PCA only captures linear relationships, so if your clusters are separated by curved boundaries (common in real data), PCA will smear them together. You’ll see some variance along PC1/PC2, but distinct groups might overlap. For cluster visualization, use UMAP or t-SNE. Reserve PCA for quick sanity checks and preprocessing.

Q: Why does my t-SNE plot look different every time I run it?

t-SNE is initialized with random positions and optimized via gradient descent, so different random seeds give different results. The local structure (which points cluster together) should be consistent, but global layout (cluster positions, orientations) changes. Always set random_state=42 for reproducibility, and never measure inter-cluster distances in t-SNE — they’re not meaningful.

Q: Can I add new data to an existing UMAP embedding?

Yes. Unlike t-SNE, UMAP supports .transform() on new samples after training. This makes it viable for production pipelines where you need to embed incoming data without retraining. Just call umap_model.transform(X_new) and it’ll project new points into the existing 2D space using the learned manifold structure. PCA also supports this; t-SNE does not.

What I’d Do Differently Next Time

I wasted hours re-running t-SNE with different perplexities before realizing I should just switch to UMAP. If I could redo that proteomics project, I’d start with PCA for a 30-second sanity check, then jump straight to UMAP with n_neighbors=15 and min_dist=0.05. Only if a reviewer specifically asked for t-SNE would I burn compute on it.

For tabular data, I’d also try UMAP’s metric='manhattan' or metric='cosine' instead of the Euclidean default. I suspect L1 distance works better for sparse count data (like word frequencies), but I haven’t tested rigorously.

And I’d log trustworthiness scores every time. Eyeballing plots is how you convince yourself a bad embedding is good.

The open question I’m still chasing: can you tune UMAP to match t-SNE’s per-cluster tightness while keeping the speed advantage? I’ve seen claims that min_dist=0.0 gets close, but my quick tests showed more inter-cluster overlap. Needs a proper ablation study.

Until then, UMAP for speed, t-SNE when you’re getting paid by the publication, PCA when you just need something in the next 60 seconds.