MLflow Quickstart 2026: Track Your First Experiment in 10 Minutes

MLflow Tracking in 90 Seconds

Here’s the entire workflow. Install MLflow, log a single training run, and view it in the UI:

# pip install mlflow==2.11.3
import mlflow
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

mlflow.set_experiment("iris-quickstart")

with mlflow.start_run():
    n_est = 100
    max_d = 5

    clf = RandomForestClassifier(n_estimators=n_est, max_depth=max_d, random_state=42)
    clf.fit(X_train, y_train)

    preds = clf.predict(X_test)
    acc = accuracy_score(y_test, preds)

    mlflow.log_param("n_estimators", n_est)
    mlflow.log_param("max_depth", max_d)
    mlflow.log_metric("accuracy", acc)
    mlflow.sklearn.log_model(clf, "model")

    print(f"Logged run with accuracy: {acc:.3f}")

Run mlflow ui in your terminal, open http://localhost:5000, and you’ll see your experiment with params, metrics, and the saved model artifact.

That’s it. The entire MLflow tracking API distills down to three calls: log_param(), log_metric(), log_model(). Everything else is just organization.

Why This Beats a CSV Log

I used to track experiments in a spreadsheet. Model version in column A, hyperparams in B-F, test accuracy in G. It worked until I hit 50 rows and couldn’t remember which “model_v12” corresponded to which code commit. Then I’d forget to log the random seed, or I’d overwrite the wrong cell, or I’d want to compare loss curves and realize I never saved them.

MLflow solves this by making logging automatic and structured. The key insight: every run gets a unique ID, stored in a local SQLite database by default. You can’t accidentally overwrite a previous run. You can’t forget which hyperparams produced which result, because they’re tied together in the same row of the database.

The UI sorts and filters runs without opening Excel. Want to find the top 5 runs by accuracy? Click the column header. Want to compare loss curves across 10 runs? Select them and hit “Compare.” Want to load the exact model from run abc123 three months later? mlflow.sklearn.load_model("runs:/abc123/model") pulls it instantly.

And when you’re ready to share results with a teammate, you point them at the tracking server instead of emailing a 3MB CSV with broken formulas.

The Three-Run Pattern: Grid Search by Hand

The real test of any tracking tool: can you run a quick hyperparameter sweep and immediately see which config won?

Here’s a toy grid search over n_estimators and max_depth, logging each combination as a separate run:

import mlflow
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, log_loss
import numpy as np

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

mlflow.set_experiment("iris-grid-search")

for n_est in [50, 100, 200]:
    for max_d in [3, 5, 7]:
        with mlflow.start_run():
            clf = RandomForestClassifier(n_estimators=n_est, max_depth=max_d, random_state=42)
            clf.fit(X_train, y_train)

            preds = clf.predict(X_test)
            probs = clf.predict_proba(X_test)
            acc = accuracy_score(y_test, preds)
            loss = log_loss(y_test, probs)

            mlflow.log_param("n_estimators", n_est)
            mlflow.log_param("max_depth", max_d)
            mlflow.log_metric("accuracy", acc)
            mlflow.log_metric("log_loss", loss)
            mlflow.sklearn.log_model(clf, "model")

            print(f"n_est={n_est}, max_d={max_d} -> acc={acc:.3f}, loss={loss:.3f}")

This logs 9 runs in about 5 seconds on my M1 MacBook. Open the UI, sort by accuracy descending, and the winner is obvious. The UI also shows you the metric curves (if you log multiple steps with mlflow.log_metric("train_loss", loss, step=epoch)) and lets you download any model artifact.

The killer feature: the UI auto-generates a parallel coordinates plot. You see n_estimators on one axis, max_depth on another, accuracy on a third. Drag to filter. Instantly spot that max_depth=7 hurts accuracy because the dataset is tiny and the model overfits.

I’m not saying this replaces Optuna or Ray Tune. But for a 10-minute exploratory sweep, it beats writing a nested dict and pretty-printing it.

Autologging: The Lazy Developer’s Win

MLflow 1.11+ added autologging for sklearn, TensorFlow, PyTorch, XGBoost, and a dozen other frameworks. You call mlflow.autolog() once at the top of your script, and MLflow intercepts training calls to log params, metrics, and models automatically.

Here’s the same grid search with autologging:

import mlflow
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

mlflow.autolog()  # That's it

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

mlflow.set_experiment("iris-autolog")

for n_est in [50, 100, 200]:
    for max_d in [3, 5, 7]:
        with mlflow.start_run():
            clf = RandomForestClassifier(n_estimators=n_est, max_depth=max_d, random_state=42)
            clf.fit(X_train, y_train)
            # No explicit log_param, log_metric, or log_model calls

Autologging captures every parameter you pass to RandomForestClassifier, the training duration, and the model artifact. It even logs a confusion matrix as a PNG.

The catch: autologging is opinionated. It logs everything, which means you get metrics you don’t care about (like training_log_loss when you only wanted test accuracy). And it can slow down training by 5-10% because it’s serializing the model after every run.

I use autologging for quick experiments where I don’t know what I’ll need yet. Once I narrow down the metrics that matter, I switch back to explicit log_metric() calls for speed.

Saving Artifacts: More Than Just Models

You can log anything as an artifact: plots, config files, preprocessed datasets, whatever. MLflow just copies it into the run directory (default: ./mlruns/<experiment_id>/<run_id>/artifacts/).

Here’s how to log a matplotlib plot:

import mlflow
import matplotlib.pyplot as plt
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, learning_curve

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

mlflow.set_experiment("iris-with-plot")

with mlflow.start_run():
    clf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)

    # Generate learning curve
    train_sizes, train_scores, val_scores = learning_curve(
        clf, X_train, y_train, cv=3, train_sizes=np.linspace(0.1, 1.0, 5)
    )

    train_mean = train_scores.mean(axis=1)
    val_mean = val_scores.mean(axis=1)

    plt.figure(figsize=(8, 5))
    plt.plot(train_sizes, train_mean, label="Train")
    plt.plot(train_sizes, val_mean, label="Val")
    plt.xlabel("Training Set Size")
    plt.ylabel("Accuracy")
    plt.legend()
    plt.title("Learning Curve")
    plt.savefig("learning_curve.png")
    plt.close()

    mlflow.log_artifact("learning_curve.png")

    clf.fit(X_train, y_train)
    mlflow.sklearn.log_model(clf, "model")

The plot shows up in the UI under the “Artifacts” tab. You can download it, or click to preview it inline.

Why log plots? Because three months later, when someone asks “did you check for overfitting?”, you want proof you already did. The plot is timestamped and tied to the exact run. No digging through Slack or your Downloads folder.

Remote Tracking Server: Sharing with Your Team

By default, MLflow writes to a local ./mlruns directory. That’s fine for solo work, but if you’re on a team, you want a shared tracking server.

The simplest setup: a single EC2 instance running mlflow server with a PostgreSQL backend and S3 artifact storage. Here’s the command:

mlflow server \
  --backend-store-uri postgresql://user:pass@localhost/mlflow \
  --default-artifact-root s3://my-bucket/mlflow-artifacts \
  --host 0.0.0.0 \
  --port 5000

Now anyone on your team can point their local client at the server:

import mlflow

mlflow.set_tracking_uri("http://ec2-instance-ip:5000")

mlflow.set_experiment("shared-experiment")

with mlflow.start_run():
    mlflow.log_param("learning_rate", 0.01)
    mlflow.log_metric("accuracy", 0.92)

The run shows up in the shared UI immediately. Your teammate can filter by experiment, compare your runs to theirs, and download your model artifacts from S3.

The gotcha: you need to configure AWS credentials on both the server and the client. The server needs write access to S3 for artifacts. The client needs read access to download them. If you forget to set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY on the client, you’ll get a cryptic botocore.exceptions.NoCredentialsError when you try to load a model.

I spent 20 minutes debugging this the first time because the error message didn’t mention S3 at all. It just said “Could not load model.” Turns out the model metadata was in Postgres, but the actual pickle file was in S3, and my local machine couldn’t authenticate.

Loading a Logged Model Back

This is where MLflow really shines. You can load a model from any run using its run ID:

import mlflow.sklearn

model = mlflow.sklearn.load_model("runs:/abc123def456/model")

preds = model.predict([[5.1, 3.5, 1.4, 0.2]])
print(preds)  # [0]

The format runs:/<run_id>/<artifact_path> is MLflow’s universal model URI. It works for sklearn, PyTorch, TensorFlow, XGBoost, whatever. The load_model() function is framework-aware, so it deserializes correctly.

You can also use a model alias (MLflow 2.9+) or a registered model name:

# Register the model first (in the UI or via API)
model = mlflow.sklearn.load_model("models:/iris-classifier/production")

This is how you’d deploy a model to production: pin the production alias to a specific run, then your serving code loads models:/iris-classifier/production. When you want to promote a new model, you just update the alias. No code changes, no redeployment.

(Side note: if you’re serious about model serving, MLflow + FastAPI: $2K/Month Model Serving Side Project walks through the full stack.)

When MLflow Isn’t Enough

MLflow is great for experiment tracking, but it’s not a full MLOps platform. Here’s what it doesn’t do:

• Data versioning: MLflow tracks model versions, not dataset versions. If your training data changes, you need a separate tool like DVC or Delta Lake to track it. I usually log the data hash as a param (mlflow.log_param("data_hash", hash_of_csv)), but that’s a hack, not a solution.

• Distributed training: MLflow tracks runs, but it doesn’t orchestrate them. If you’re running a hyperparameter sweep across 100 EC2 instances, you need something like Ray Tune or Kubernetes + Argo to manage the compute. MLflow just logs the results after the fact.

• Real-time monitoring: MLflow logs metrics during training, but it doesn’t monitor models after deployment. If your production model starts drifting, MLflow won’t alert you. You need a separate monitoring stack (Prometheus, Evidently AI, etc.).

• Feature engineering pipelines: MLflow doesn’t track feature transformations or SQL queries. If your model depends on a 12-step feature pipeline, you have to log that logic manually (usually as a config file artifact).

That said, MLflow integrates well with other tools. You can use DVC for data versioning, log the DVC commit hash as an MLflow param, and you’ve got a paper trail from raw data to final model. You can wrap Ray Tune in an MLflow experiment and log every trial. You can export MLflow metrics to Prometheus for long-term monitoring.

The point: MLflow is a tracking layer, not an end-to-end platform. It’s the glue between your training script and your deployment pipeline.

The Math Behind Model Metadata

MLflow doesn’t just store your model pickle. It stores a signature: the input schema, output schema, and any constraints. This is critical for catching bugs in production.

When you call mlflow.sklearn.log_model(clf, "model"), MLflow infers the signature from the training data:

$$\text{signature} = (\mathbf{X}_{\text{schema}}, \mathbf{y}_{\text{schema}})$$

where $\mathbf{X}_{\text{schema}}$ is a list of column names and dtypes, and $\mathbf{y}_{\text{schema}}$ is the output dtype.

When you later load the model and call model.predict(new_data), MLflow validates that new_data matches $\mathbf{X}_{\text{schema}}$. If you’re missing a column or passing a string where it expects a float, it raises a MlflowException before calling the model.

This saved me once: I trained a model on a DataFrame with columns ["feature_a", "feature_b", "feature_c"], then two weeks later I refactored my feature pipeline and accidentally renamed feature_c to feature_3. My serving code crashed immediately with a clear error message (“Expected column ‘feature_c’, got ‘feature_3′”) instead of silently producing garbage predictions.

You can also define a custom signature for more complex models:

from mlflow.models.signature import infer_signature

signature = infer_signature(X_train, clf.predict(X_train))
mlflow.sklearn.log_model(clf, "model", signature=signature)

For deep learning models with multiple inputs (e.g., images + metadata), you’d use a TensorSpec:

from mlflow.types.schema import Schema, TensorSpec
import numpy as np

input_schema = Schema([
    TensorSpec(np.dtype(np.float32), (-1, 224, 224, 3), "image"),
    TensorSpec(np.dtype(np.float32), (-1, 10), "metadata")
])

output_schema = Schema([TensorSpec(np.dtype(np.float32), (-1, 1000), "logits")])

from mlflow.models.signature import ModelSignature
signature = ModelSignature(inputs=input_schema, outputs=output_schema)

mlflow.pytorch.log_model(model, "model", signature=signature)

The math here is just tensor shapes: $\mathbf{X}_{\text{image}} \in \mathbb{R}^{B \times 224 \times 224 \times 3}$, $\mathbf{X}_{\text{metadata}} \in \mathbb{R}^{B \times 10}$, $\mathbf{y} \in \mathbb{R}^{B \times 1000}$, where $B$ is the batch size. MLflow enforces these shapes at serving time.

Logging Metrics Over Time: Training Curves

For iterative algorithms (gradient descent, RL, etc.), you want to log metrics at every step, not just the final value. MLflow supports this with the step parameter:

import mlflow
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import SGDClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

mlflow.set_experiment("iris-sgd")

with mlflow.start_run():
    clf = SGDClassifier(loss="log_loss", max_iter=1, warm_start=True, random_state=42)

    for epoch in range(50):
        clf.fit(X_train, y_train)  # one pass

        train_acc = accuracy_score(y_train, clf.predict(X_train))
        test_acc = accuracy_score(y_test, clf.predict(X_test))

        mlflow.log_metric("train_accuracy", train_acc, step=epoch)
        mlflow.log_metric("test_accuracy", test_acc, step=epoch)

    mlflow.sklearn.log_model(clf, "model")

The UI plots train_accuracy and test_accuracy on the same graph, with epoch on the x-axis. You can see exactly when the model starts overfitting (train accuracy keeps rising, test accuracy plateaus).

This is the simplest form of a loss curve. For deep learning, you’d log loss every batch:

for epoch in range(num_epochs):
    for batch_idx, (data, target) in enumerate(train_loader):
        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()

        global_step = epoch * len(train_loader) + batch_idx
        mlflow.log_metric("train_loss", loss.item(), step=global_step)

The key: use a monotonic step counter. Don’t reset it every epoch, or the UI will show a sawtooth pattern instead of a smooth curve.

Comparing Runs: Parallel Coordinates and Scatter Plots

The UI has two killer visualizations:

1. Parallel coordinates: each run is a line, each axis is a param or metric. Drag to filter runs by value. Great for spotting that learning_rate < 0.001 always produces accuracy < 0.8.

2. Scatter plot: pick two metrics (e.g., train_accuracy vs test_accuracy), plot every run as a point. Instantly see the Pareto frontier of models that maximize both.

Neither of these is revolutionary, but they’re built-in and fast. You don’t have to export your runs to a Jupyter notebook and call seaborn.scatterplot() by hand.

The parallel coordinates plot is especially useful for high-dimensional sweeps. If you’re tuning 6 hyperparameters, it’s impossible to visualize all pairwise interactions in a grid. But in parallel coordinates, you can see patterns like “high dropout + low weight_decay = overfitting” by dragging the axes around.

The Model Registry: Promoting Models to Production

MLflow’s model registry is a separate service that tracks model versions and aliases. You register a model by name:

import mlflow.sklearn

mlflow.set_experiment("iris-classifier")

with mlflow.start_run():
    clf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)
    clf.fit(X_train, y_train)

    mlflow.sklearn.log_model(clf, "model", registered_model_name="iris-rf")

This creates a new model version under the name iris-rf. The first time you register a model, it’s version 1. The next time, it’s version 2, and so on.

You can then assign aliases like production, staging, champion, etc.:

from mlflow.tracking import MlflowClient

client = MlflowClient()
client.set_registered_model_alias("iris-rf", "production", version=2)

Now your serving code loads models:/iris-rf/production, which always points to version 2. When you want to deploy version 3, you just update the alias. The model URI stays the same.

The registry also supports model stages (None, Staging, Production, Archived) as of MLflow 2.0, but aliases are more flexible. You can have multiple aliases per model (e.g., champion, challenger, shadow) for A/B testing.

One gotcha: the model registry is backed by the same database as experiment tracking (SQLite by default, Postgres in production). If you’re using a local SQLite database, the registry is local too. If you want to share the registry across your team, you need a remote tracking server.

FAQ

Q: Can I use MLflow with PyTorch or TensorFlow?

Yes. MLflow has first-class support for PyTorch (mlflow.pytorch.log_model), TensorFlow (mlflow.tensorflow.log_model), Keras (mlflow.keras.log_model), XGBoost, LightGBM, and more. The API is identical: log_param, log_metric, log_model. You can also use autologging (mlflow.pytorch.autolog()) to log everything automatically.

Q: How do I delete old runs to save disk space?

Use the MLflow CLI: mlflow gc --backend-store-uri sqlite:///mlflow.db --older-than 30d deletes runs older than 30 days. If you’re using a remote tracking server with S3 artifacts, you’ll also need to set up an S3 lifecycle policy to delete old artifacts. By default, MLflow never deletes anything.

Q: Can I log custom metrics that aren’t scalars?

Not directly. log_metric() only accepts floats. For arrays, histograms, or images, use log_artifact() to save them as files (e.g., a numpy array as .npy, a histogram as a PNG). The UI will preview images inline, but it won’t plot arrays. If you need interactive visualizations, consider logging a JSON artifact and rendering it with a separate tool like Plotly or Streamlit.

What I’d Pick

For a weekend side project or a solo research experiment, MLflow is the easiest win. You get structured experiment tracking with three lines of code and a web UI that’s good enough for exploratory analysis.

For a team of 5+ with shared experiments and model deployments, you need a remote tracking server. The Postgres + S3 setup takes an hour to configure, but once it’s done, it’s rock-solid. I’d skip the model registry until you’re actually deploying models to production — it adds complexity and you probably don’t need version aliases until you have at least two models in rotation.

If you’re already using Weights & Biases or Neptune, I wouldn’t switch. MLflow’s UI is more bare-bones (no collaborative features, no real-time chat, no built-in hyperparameter importance charts). But if you’re starting from scratch and you want something free and self-hosted, MLflow is the move.

One thing I’m curious about: how MLflow’s performance scales with 10,000+ runs in a single experiment. The UI starts to lag around 1,000 runs on my local SQLite setup. Postgres helps, but I haven’t stress-tested it. If you’re doing massive hyperparameter sweeps (Ray Tune with 50,000 trials), I’d bet the UI chokes and you end up exporting to a DataFrame for analysis anyway. But for 99% of use cases, MLflow’s the right level of complexity.

Oh, and if you’re pulling late nights debugging why your model’s loss curve looks like a drunk EKG, grab some Dark Chocolate Espresso Beans. They’re basically debugger fuel.