Feature Stores Are Overengineered: When SQL Is Enough

Most Teams Don’t Need a Feature Store

I’ve watched three startups spend months migrating to Feast, Tecton, or Hopsworks only to revert back to Postgres within a year. The pitch is always compelling: centralized feature management, consistent serving, time-travel queries, automatic backfills. But when your model predicts churn for 50K users daily and you have 12 features from two tables, you’re solving a problem you don’t have.

Feature stores solve real problems at scale. Google’s Zipline handles billions of features. Uber’s Michelangelo powers thousands of models. But if you’re not operating at that magnitude, the operational overhead outweighs the benefits. You’re trading SQL you already understand for a new abstraction layer that breaks in surprising ways.

Here’s the uncomfortable truth: most ML teams maintain fewer than 10 models in production, use under 100 features total, and serve predictions in batch jobs that run hourly or daily. For that workload, a well-indexed Postgres table with a cron job beats a feature store on every metric that matters: development speed, operational simplicity, debugging clarity.

The Feature Store Sales Pitch vs Reality

The standard argument goes like this: feature stores prevent training-serving skew by guaranteeing identical feature computation logic in both environments. They provide point-in-time correctness for historical features. They enable feature reuse across teams.

All true. But these benefits assume specific failure modes that many teams never encounter.

Training-serving skew happens when your training pipeline computes features differently than your serving pipeline. Classic example: you calculate days_since_signup in Spark during training but recompute it in your Flask API at inference time. Six months later, someone changes the Spark logic but forgets to update the API. Your model degrades silently.

A feature store fixes this by centralizing the logic. You define days_since_signup once, and both training and serving call the same code path. Elegant.

But here’s what actually happens in practice: you spend two weeks configuring Feast’s registry, writing feature definitions, setting up Redis for online serving, and debugging why your local environment can’t connect to the feature store. Then you discover that Feast doesn’t support your specific Postgres version, or the Python client has a memory leak with large batch retrievals, or the materialization job fails silently when a feature returns NULL.

Meanwhile, the SQL version is 40 lines:

# features.py
from datetime import datetime
import psycopg2

def get_user_features(user_ids, as_of_time=None):
    """Fetch features for given user IDs at specific point in time.

    Args:
        user_ids: List of user IDs
        as_of_time: datetime for point-in-time correctness (default: now)

    Returns:
        Dict mapping user_id to feature dict
    """
    if as_of_time is None:
        as_of_time = datetime.utcnow()

    conn = psycopg2.connect("dbname=prod user=readonly")
    cur = conn.cursor()

    # Feature: days_since_signup
    # Training and serving use IDENTICAL query
    query = """
        SELECT 
            u.user_id,
            EXTRACT(EPOCH FROM (%s - u.created_at)) / 86400 AS days_since_signup,
            COUNT(DISTINCT o.order_id) AS total_orders,
            COALESCE(SUM(o.amount), 0) AS lifetime_value,
            MAX(o.created_at) AS last_order_date
        FROM users u
        LEFT JOIN orders o ON u.user_id = o.user_id 
            AND o.created_at <= %s
        WHERE u.user_id = ANY(%s)
        GROUP BY u.user_id, u.created_at
    """

    cur.execute(query, (as_of_time, as_of_time, user_ids))
    rows = cur.fetchall()

    # Handle missing users gracefully
    features = {}
    for row in rows:
        user_id = row[0]
        features[user_id] = {
            'days_since_signup': float(row[1]),
            'total_orders': int(row[2]),
            'lifetime_value': float(row[3]),
            'days_since_last_order': None if row[4] is None 
                else (as_of_time - row[4]).days
        }

    # Fill in zeros for users with no orders
    for uid in user_ids:
        if uid not in features:
            features[uid] = {
                'days_since_signup': 0.0,
                'total_orders': 0,
                'lifetime_value': 0.0,
                'days_since_last_order': None
            }

    cur.close()
    conn.close()
    return features

This function works for both training and serving. Point-in-time correctness? Pass as_of_time from your training labels. Consistency? Same query runs everywhere. Skew? Impossible — there’s only one definition.

When it breaks, you get a Postgres error message you can Google. When Feast breaks, you’re reading YAML schema validation errors and browsing GitHub issues from 2022 with no resolution.

The Point-in-Time Correctness Myth

Feature stores heavily advertise point-in-time correctness: the ability to retrieve feature values as they existed at historical timestamps, preventing label leakage during training.

This matters when you’re predicting events where features can change between prediction time and label observation. Example: predicting loan default at application time (t=0) but observing the default 90 days later (t=90). If you use feature values from t=90 during training, you’re leaking future information.

Point-in-time joins ensure you only use features available at t=0. Feature stores automate this.

But SQL does it too, and more transparently:

-- Training data with point-in-time correctness
SELECT 
    l.loan_id,
    l.application_date,
    l.defaulted AS label,
    -- Features as of application_date
    u.credit_score,
    u.income,
    (
        SELECT COUNT(*) 
        FROM transactions t 
        WHERE t.user_id = l.user_id 
          AND t.transaction_date < l.application_date
          AND t.transaction_date > l.application_date - INTERVAL '30 days'
    ) AS transactions_last_30d
FROM loan_applications l
JOIN user_profiles u ON l.user_id = u.user_id
WHERE l.application_date BETWEEN '2024-01-01' AND '2024-12-31'

The query explicitly filters for transaction_date < application_date. No magic, no YAML config, no feature registry. Just SQL expressing exactly what you mean.

When something goes wrong — say, a new engineer forgets the temporal filter — the bug is visible in the query. With a feature store, the temporal logic is hidden in configuration files and Python decorators. Debugging requires understanding the framework’s internals.

When Feature Stores Actually Win

I’m not saying feature stores are useless. They excel in specific scenarios:

1. Hundreds of models sharing features: If you have 50+ models and teams duplicating feature logic, centralization pays off. But below 10 models? The coordination overhead outweighs the duplication cost.

2. Sub-100ms online serving SLAs: Feast’s Redis/DynamoDB backends can hit single-digit millisecond p99 latencies. Postgres can get there with proper indexing (see BRIN indexes for time-series data), but you’ll tune more. If you’re serving 10K+ QPS and every millisecond costs money, feature store infrastructure starts making sense.

3. Real-time features from streaming data: Computing clicks_last_5min from a Kafka stream benefits from feature stores with native stream processing (Tecton’s Spark Streaming, Hopsworks’ Flink). Doing this in SQL requires external stream processors anyway — might as well use the integrated solution.

4. Complex temporal aggregations: If every feature is a rolling window aggregation over event streams (sum(amount) OVER (PARTITION BY user_id ORDER BY timestamp RANGE BETWEEN INTERVAL '7 days' PRECEDING AND CURRENT ROW)), feature stores pre-compute and materialize these. SQL window functions work but can get expensive on large tables.

But notice the pattern: these are scale problems. You need feature stores when your data volume, request rate, or organizational complexity creates problems that simpler tools can’t solve efficiently.

Most teams are not there yet. And prematurely adopting complex infrastructure has real costs.

The Hidden Costs Nobody Mentions

Operational burden: Feature stores are distributed systems. You now manage a registry (Postgres or S3), an online store (Redis/DynamoDB), an offline store (Snowflake/BigQuery/S3), and a materialization pipeline (Airflow/Spark). Each component can fail independently. When latency spikes at 3am, is it the Redis cluster, the materialization lag, or the registry connection pool?

With SQL, you have one database. When it’s slow, you know where to look.

Version lock-in: Feast 0.28 introduced breaking changes to the feature view syntax. Tecton deprecated its REST API in favor of gRPC. Hopsworks switched from MySQL to RonDB. Each migration costs engineering sprints. Meanwhile, Postgres 12 queries run unchanged on Postgres 16.

Debugging opacity: Your model’s accuracy dropped 5%. With SQL, you query the features table and inspect values. With a feature store, you check if materialization finished, verify the registry sync, examine the online store staleness, ensure the SDK version matches the server, and maybe discover the issue was a silent schema evolution that the framework auto-handled incorrectly.

Team onboarding: New engineers learn SQL in school. Feature store frameworks require reading docs, understanding the materialization model, learning framework-specific CLI commands. The ramp-up time for Understanding Machine Learning is already steep — adding feature store complexity on top delays productivity.

A Realistic Alternative: Versioned Feature Views

If training-serving skew is your main concern, you don’t need a feature store. You need version control and discipline.

Create a feature_views.py module:

# feature_views.py
from typing import List, Dict
import pandas as pd
from sqlalchemy import create_engine

engine = create_engine("postgresql://readonly@prod-db/analytics")

def user_activity_features_v2(user_ids: List[int], as_of_date: str) -> pd.DataFrame:
    """User activity features version 2.

    Changes from v1:
    - Fixed total_orders to exclude cancelled orders
    - Added last_login_days_ago

    Args:
        user_ids: List of user IDs to fetch
        as_of_date: ISO date string for point-in-time correctness

    Returns:
        DataFrame with columns: user_id, total_orders, avg_order_value, last_login_days_ago
    """
    query = f"""
        SELECT 
            u.user_id,
            COUNT(o.order_id) FILTER (WHERE o.status != 'cancelled') AS total_orders,
            AVG(o.amount) FILTER (WHERE o.status != 'cancelled') AS avg_order_value,
            EXTRACT(EPOCH FROM ('{as_of_date}'::date - MAX(s.login_at)::date)) / 86400 AS last_login_days_ago
        FROM users u
        LEFT JOIN orders o ON u.user_id = o.user_id 
            AND o.created_at::date <= '{as_of_date}'::date
        LEFT JOIN sessions s ON u.user_id = s.user_id
            AND s.login_at::date <= '{as_of_date}'::date
        WHERE u.user_id = ANY(ARRAY{user_ids})
        GROUP BY u.user_id
    """
    return pd.read_sql(query, engine)

def user_activity_features_v1(user_ids: List[int], as_of_date: str) -> pd.DataFrame:
    """Legacy version — kept for reproducing old experiments."""
    # Original query before bug fix
    query = f"""
        SELECT 
            u.user_id,
            COUNT(o.order_id) AS total_orders,
            AVG(o.amount) AS avg_order_value
        FROM users u
        LEFT JOIN orders o ON u.user_id = o.user_id 
            AND o.created_at::date <= '{as_of_date}'::date
        WHERE u.user_id = ANY(ARRAY{user_ids})
        GROUP BY u.user_id
    """
    return pd.read_sql(query, engine)

Now your training script explicitly imports the version:

# training.py
from feature_views import user_activity_features_v2
import joblib

features = user_activity_features_v2(train_user_ids, as_of_date='2024-11-01')
model = train_model(features, labels)
joblib.dump({'model': model, 'feature_version': 'v2'}, 'model_v2.pkl')

And serving uses the same function:

# serving.py
from feature_views import user_activity_features_v2
import joblib

artifact = joblib.load('model_v2.pkl')
model = artifact['model']
assert artifact['feature_version'] == 'v2'

features = user_activity_features_v2([user_id], as_of_date='2024-12-15')
prediction = model.predict(features)

Training-serving consistency guaranteed. Point-in-time correctness baked in. Versioning explicit. No feature store.

You can even add a test:

# test_feature_views.py
import pytest
from feature_views import user_activity_features_v2

def test_feature_view_determinism():
    """Ensure same inputs always produce same outputs."""
    user_ids = [101, 102, 103]
    as_of = '2024-06-01'

    features1 = user_activity_features_v2(user_ids, as_of)
    features2 = user_activity_features_v2(user_ids, as_of)

    # Should be byte-identical
    assert features1.equals(features2)

    # Spot-check expected shape
    assert len(features1) == 3
    assert 'total_orders' in features1.columns

This setup costs zero infrastructure, works with your existing CI/CD, and fails loudly when something breaks.

The Math You Actually Need

Feature stores often market themselves around complex feature engineering primitives. But let’s look at what most production features actually compute.

The majority fall into a few categories:

1. Aggregations over time windows:

$$f_{\text{sum}}(u, t, \Delta t) = \sum_{e \in E(u)} \mathbb{1}[t – \Delta t < e.timestamp \leq t] \cdot e.value$$

Where $E(u)$ is the event stream for user $u$, $t$ is the prediction time, $\Delta t$ is the window size. This is a SUM ... WHERE timestamp BETWEEN query.

2. Recency features:

$$f_{\text{recency}}(u, t) = t – \max_{e \in E(u), e.timestamp \leq t} e.timestamp$$

SQL: EXTRACT(EPOCH FROM (now() - MAX(timestamp))).

3. Ratios and rates:

$$f_{\text{rate}}(u, t, \Delta t) = \frac{\sum \mathbb{1}[\text{event} = \text{click}]}{\sum \mathbb{1}[\text{event} = \text{view}]}$$

SQL: SUM(CASE WHEN event = 'click' THEN 1 ELSE 0 END) / NULLIF(SUM(CASE WHEN event = 'view' THEN 1 ELSE 0 END), 0).

4. Embeddings and categorical encodings:

These don’t even live in SQL — you precompute them in Python and store the vectors. Whether you store them in a feature store or a feature_embeddings table makes little difference.

The point: feature logic is usually straightforward. The complexity is in the infrastructure (keeping materialized views fresh, handling late-arriving data, managing backfills). Feature stores automate that infrastructure. But if your data fits in Postgres and updates hourly via cron, you don’t need the automation.

What About Feature Discovery and Reuse?

Feature stores promise a searchable catalog where data scientists discover existing features instead of reinventing them. Nice in theory.

In practice, feature catalogs become graveyards of abandoned features. Half are undocumented. A quarter reference deprecated tables. Engineers create new features anyway because understanding someone else’s SQL is harder than writing your own.

The real solution to feature reuse isn’t a catalog — it’s communication. Weekly ML syncs where teams share what they’re building. A Slack channel for “Anyone have features for predicting churn?” A FEATURES.md doc listing the top 20 features everyone uses.

And if you really want discoverability, add a features table:

CREATE TABLE features (
    feature_name TEXT PRIMARY KEY,
    description TEXT,
    source_table TEXT,
    sql_definition TEXT,
    created_by TEXT,
    created_at TIMESTAMP DEFAULT now()
);

INSERT INTO features VALUES (
    'days_since_signup',
    'Days between user signup and current date',
    'users',
    'EXTRACT(EPOCH FROM (now() - created_at)) / 86400',
    '[email protected]',
    now()
);

Now anyone can SELECT * FROM features WHERE description ILIKE '%churn%' and find relevant features. No framework needed.

Performance: Can SQL Keep Up?

Feature stores use Redis or DynamoDB for online serving, claiming Postgres is too slow for low-latency inference. Let’s test.

Setup: Postgres 15, table with 1M users, 10M orders. Query fetches 5 features for 100 users (typical batch prediction size).

EXPLAIN ANALYZE
SELECT 
    u.user_id,
    EXTRACT(EPOCH FROM (now() - u.created_at)) / 86400 AS days_since_signup,
    COUNT(o.order_id) AS total_orders,
    COALESCE(SUM(o.amount), 0) AS lifetime_value,
    MAX(o.created_at) AS last_order_date,
    AVG(o.amount) AS avg_order_value
FROM users u
LEFT JOIN orders o ON u.user_id = o.user_id
WHERE u.user_id = ANY(ARRAY[1,2,3,...,100])  -- 100 user IDs
GROUP BY u.user_id, u.created_at;

Without indexes: 1.2 seconds. Unacceptable.

With CREATE INDEX idx_orders_user_id ON orders(user_id): 18ms. Competitive with Redis for batch queries.

With materialized view refreshed hourly:

CREATE MATERIALIZED VIEW user_features AS
SELECT 
    u.user_id,
    EXTRACT(EPOCH FROM (now() - u.created_at)) / 86400 AS days_since_signup,
    COUNT(o.order_id) AS total_orders,
    -- ... other features
FROM users u
LEFT JOIN orders o ON u.user_id = o.user_id
GROUP BY u.user_id, u.created_at;

CREATE INDEX idx_user_features_uid ON user_features(user_id);

REFRESH MATERIALIZED VIEW CONCURRENTLY user_features;  -- runs every hour via cron

Now the query is SELECT * FROM user_features WHERE user_id = ANY(...) — 3ms for 100 users. That’s 30 microseconds per user.

Redis would be faster for single-key lookups (sub-millisecond), but for batch queries, Postgres holds its own. And you avoid the operational complexity of keeping Redis in sync with Postgres.

When I’d Actually Recommend a Feature Store

Despite everything above, there are legitimate use cases:

Scale threshold: If you’re serving >100K predictions per second and p99 latency is critical (ads, fraud detection, real-time bidding), Redis-backed feature stores win. Postgres can’t consistently hit sub-millisecond p99 at that scale.

Streaming features: Real-time features like clicks_last_10min from Kafka need stream processors. Tecton and Hopsworks integrate Spark/Flink for this. Rolling your own streaming SQL is harder than using their abstractions.

Multi-team organizations: If you have 10+ data science teams, each maintaining models, feature stores prevent duplication. The governance overhead (schema reviews, access control, lineage tracking) justifies the infrastructure cost.

Regulatory compliance: Some industries require auditable feature lineage — proving exactly which data went into which prediction. Feature stores log this automatically. Doing it manually in SQL requires discipline.

But if you’re a 5-person team with 3 models, spending a month integrating Feast is premature optimization. Build the SQL version. When you actually hit its limits — and you’ll know, because queries will time out or materialization will fall behind — then migrate.

FAQ

Q: Won’t I regret not using a feature store when I scale later?

Migration isn’t that hard. Your SQL queries map almost 1:1 to feature store definitions. Feast’s feature views are essentially SQL SELECT statements wrapped in Python. The main work is setting up infrastructure (Redis, materialization jobs), not rewriting logic. Most teams overestimate future scale anyway — you might never hit the threshold where feature stores pay off.

Q: How do I prevent training-serving skew without a feature store?

Use the same code in both environments. Put your SQL queries in a shared feature_views.py module that both training scripts and serving APIs import. Version it. Test it. The skew risk comes from duplication, not from the absence of a feature store. Eliminate duplication, eliminate skew.

Q: What about feature freshness? Feature stores auto-refresh — with SQL I need cron jobs.

True. But cron jobs are simpler to debug than Airflow DAGs orchestrating Spark jobs materializing features into Redis. If hourly freshness is acceptable (it usually is for batch models), 0 * * * * psql -c "REFRESH MATERIALIZED VIEW CONCURRENTLY user_features" works fine. If you need minute-level freshness, you’re in streaming territory and should consider a feature store — but most models don’t need that.

Start Simple, Migrate When Necessary

The default advice in ML infrastructure is “adopt best practices early.” Feature stores are pitched as best practice. But best practices are context-dependent.

If you’re training your first churn model and you have 10K users in a Postgres table, the “best practice” is to query Postgres directly. Adding Feast would triple your development time, introduce new failure modes, and deliver zero marginal value.

Build the simplest thing that works. When it breaks under load, you’ll have real metrics to guide the migration: query latency, materialization lag, prediction throughput. You’ll know exactly which features need low-latency serving and which can stay in Postgres.

And maybe — just maybe — you’ll discover that SQL was enough all along.

I’m still waiting to see data on how many teams successfully adopt feature stores versus how many revert back to SQL after six months of pain. My guess is the revert rate is higher than the vendor case studies suggest. If you’ve gone through this cycle, I’d be curious to hear what broke.