Gymnasium render_mode=’human’ Crashes Training: 3 Fixes

The render_mode=’human’ Trap That Kills Your Training Loop

Set render_mode='human' in your Gymnasium environment and watch your training grind to a halt—or crash outright with a cryptic “display cannot be opened” error. This single parameter choice is responsible for more broken RL experiments than any hyperparameter tuning mistake.

The issue isn’t a bug. It’s a fundamental misunderstanding of what render_mode='human' actually does: it opens a real-time graphical window using Pygame or OpenGL, blocks your training loop waiting for display refresh rates, and assumes you have a display server running. On headless cloud instances, it fails immediately. On local machines, it throttles your agent to 30-60 FPS when it could be sampling thousands of steps per second.

Here’s what happens when you run this on an AWS EC2 instance:

import gymnasium as gym

env = gym.make('CartPole-v1', render_mode='human')
obs, info = env.reset()

for _ in range(1000):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()

pygame.error: No available video device

Even if you’re on a local machine with a display, this code runs at ~30 steps/sec because Pygame caps the frame rate. Remove render_mode='human' and the same loop hits 50,000+ steps/sec.

Why render_mode=’human’ Exists (And When You Actually Need It)

Gymnasium inherited render_mode from OpenAI Gym’s 2021 API redesign. The parameter has four valid options:

• 'human': Opens a graphical window, blocks until rendering completes
• 'rgb_array': Returns a numpy array (H, W, 3) representing the frame
• 'ansi': Text-based rendering (for simple envs like FrozenLake)
• None: No rendering at all (default, fastest)

The 'human' mode is for watching your agent train interactively—useful when debugging a custom environment’s physics or checking if your reward shaping makes sense visually. But it’s a development tool, not a training mode.

Most RL practitioners I know run training headless at full speed, then record videos periodically using 'rgb_array' mode. Here’s the pattern:

import gymnasium as gym
import numpy as np
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv
from stable_baselines3.common.callbacks import CheckpointCallback
import imageio

# Training: no rendering
env = gym.make('CartPole-v1')
model = PPO('MlpPolicy', env, verbose=1)
model.learn(total_timesteps=100000)

# Evaluation: render to video
eval_env = gym.make('CartPole-v1', render_mode='rgb_array')
obs, info = eval_env.reset()
frames = []

for _ in range(500):
    action, _states = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = eval_env.step(action)
    frames.append(eval_env.render())  # Returns numpy array
    if terminated or truncated:
        break

imageio.mimsave('cartpole_trained.mp4', frames, fps=30)

This runs training at 50k+ steps/sec, then generates a video file you can inspect at your leisure. No display server required.

Fix #1: Use render_mode=’rgb_array’ + Periodic Video Recording

The cleanest solution is to train without rendering, then log videos to TensorBoard or Weights & Biases every N episodes. Stable Baselines3 has a built-in callback for this:

from stable_baselines3.common.callbacks import EvalCallback
from stable_baselines3.common.vec_env import VecVideoRecorder, DummyVecEnv

# Wrap environment for video recording
def make_env():
    return gym.make('LunarLander-v2', render_mode='rgb_array')

env = DummyVecEnv([make_env])
video_folder = './videos/'
video_length = 500  # record 500 steps per video

env = VecVideoRecorder(
    env, video_folder,
    record_video_trigger=lambda x: x % 10000 == 0,  # every 10k steps
    video_length=video_length
)

model = PPO('MlpPolicy', env, verbose=1)
model.learn(total_timesteps=500000)

This generates MP4 files in ./videos/ at training steps 0, 10000, 20000, etc. The overhead is minimal because rendering only happens once per 10k steps—the rest of the time the env runs at full speed with no rendering.

One gotcha: VecVideoRecorder expects a vectorized environment. If you’re using a single env, wrap it with DummyVecEnv first. I’ve debugged this exact issue three times because the error message is unhelpful:

AttributeError: 'CartPole' object has no attribute 'env_method'

Fix #2: Conditional Rendering Based on Environment Variable

For rapid prototyping, I sometimes want to toggle rendering on/off without changing code. Here’s a pattern using environment variables:

import os
import gymnasium as gym

# Set RENDER=1 in shell to enable human rendering
render_mode = 'human' if os.getenv('RENDER') == '1' else None
env = gym.make('CartPole-v1', render_mode=render_mode)

obs, info = env.reset()
for _ in range(1000):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()

Run with RENDER=1 python train.py to watch the agent, or python train.py for headless training. This is particularly useful when debugging reward shaping—you can visually confirm the agent is doing what you expect without rewriting code.

But there’s a subtle issue: even with render_mode=None, some environments (looking at you, MuJoCo) still initialize rendering backends and allocate GPU memory. On a 1GB cloud instance, this can cause OOM crashes. For true headless mode, you need to set:

export MUJOCO_GL=osmesa  # software rendering, no GPU
export SDL_VIDEODRIVER=dummy  # dummy video driver for Pygame

These environment variables tell MuJoCo and SDL to use software rendering instead of expecting a GPU/display.

Fix #3: Virtual Display (Xvfb) for Headless Servers

Sometimes you genuinely need render_mode='human' because you’re using a custom environment that doesn’t support 'rgb_array' mode properly (yes, this happens—especially with older Gym wrappers). The workaround is to run a virtual display server.

On Ubuntu:

sudo apt-get install xvfb
Xvfb :1 -screen 0 1024x768x24 &
export DISPLAY=:1
python train.py

This creates a virtual framebuffer (:1) that Pygame/OpenGL can render to, even though there’s no physical monitor. The rendered frames go into memory and are discarded—so you still pay the rendering cost, but at least the code doesn’t crash.

For Docker containers, add this to your Dockerfile:

RUN apt-get update && apt-get install -y xvfb
ENV DISPLAY=:99
CMD Xvfb :99 -screen 0 1024x768x24 & python train.py

I’d rank this as a last resort. If you’re going to pay the rendering cost anyway, you might as well use 'rgb_array' mode and save the frames to disk for later inspection. Xvfb adds complexity and hides the problem instead of fixing it.

The Performance Cost of Rendering (Real Numbers)

Here’s a quick benchmark on my M1 MacBook Pro, training a random policy on CartPole-v1 for 100k steps:

The 'human' mode is 1,690x slower than no rendering. Even 'rgb_array' mode (which just returns numpy arrays without displaying them) is 28x slower because it still has to rasterize the scene.

For more complex environments like MuJoCo Humanoid, the gap narrows slightly because physics simulation dominates. But rendering still adds 10-30% overhead even in 'rgb_array' mode.

This is why production RL training pipelines never render during training. Energy drinks might keep you awake during a 3-day training run, but disabling rendering is what actually makes the run finish in 3 hours.

When render_mode=’rgb_array’ Returns None (And How to Fix It)

Occasionally you’ll set render_mode='rgb_array', call env.render(), and get None instead of a numpy array. This happens when:

1. You’re using an old Gym wrapper that hasn’t been updated to the new API. Solution: upgrade to gymnasium (the maintained fork) or write a thin wrapper:

class RenderWrapper(gym.Wrapper):
    def __init__(self, env):
        super().__init__(env)
        self.env.metadata['render_modes'] = ['rgb_array']

    def render(self):
        return self.env.render(mode='rgb_array')  # old API

2. The environment doesn’t implement rgb_array rendering. Some minimal envs (like Blackjack-v1) only support text rendering. Check env.metadata['render_modes'] to see what’s available.

3. You forgot to pass render_mode at env creation. This one bites me more often than I’d like to admit:

env = gym.make('CartPole-v1')  # render_mode defaults to None
frame = env.render()  # returns None

The fix:

env = gym.make('CartPole-v1', render_mode='rgb_array')
frame = env.render()  # returns (400, 600, 3) numpy array

If you’re debugging this and frame is still None, add a type check:

frame = env.render()
assert frame is not None, f"render() returned None. Available modes: {env.metadata.get('render_modes')}"
assert frame.shape == (400, 600, 3), f"Unexpected shape: {frame.shape}"

Why Stable Baselines3 VecEnv Breaks Rendering (And the Workaround)

Stable Baselines3’s vectorized environments (SubprocVecEnv, DummyVecEnv) spawn multiple env instances for parallel sampling. This is critical for on-policy algorithms like PPO where you need diverse experiences per batch.

But if you set render_mode='human' on a SubprocVecEnv with n_envs=8, you get 8 separate Pygame windows fighting for focus. Even worse, if one env terminates and resets while another is mid-episode, the windows flicker and desync.

The correct pattern:

from stable_baselines3.common.vec_env import SubprocVecEnv
import gymnasium as gym

def make_env(rank):
    def _init():
        # No rendering during training
        env = gym.make('LunarLander-v2')
        return env
    return _init

if __name__ == '__main__':
    num_envs = 8
    env = SubprocVecEnv([make_env(i) for i in range(num_envs)])

    model = PPO('MlpPolicy', env, verbose=1, n_steps=2048)
    model.learn(total_timesteps=1_000_000)

    # Evaluation: single env with rendering
    eval_env = gym.make('LunarLander-v2', render_mode='rgb_array')
    # ... record video as shown earlier

Notice the if __name__ == '__main__' guard—this is required for SubprocVecEnv on macOS/Windows because of multiprocessing spawn behavior. Without it, you’ll get infinite recursion as each subprocess tries to spawn more subprocesses.

The Math Behind Why Rendering Tanks Throughput

Let’s formalize this. Define:

• $t_{\text{step}}$ = time for environment step (physics, collision, rewards)
• $t_{\text{render}}$ = time to rasterize and display one frame
• $f_{\text{display}}$ = display refresh rate (typically 60 Hz = 16.7ms)

Without rendering, your throughput is:

$$\text{Steps/sec} = \frac{1}{t_{\text{step}}}$$

With render_mode='human', you’re bounded by:

$$\text{Steps/sec} = \min\left(\frac{1}{t_{\text{step}} + t_{\text{render}}}, f_{\text{display}}\right)$$

For CartPole, $t_{\text{step}} \approx 0.02$ ms (it’s just a 4D state update). But $t_{\text{render}} \approx 30$ ms because Pygame has to rasterize the pole, background, and call OpenGL. Even if rendering were instant, you’d still be capped at 60 FPS by the display refresh rate.

This is why PPO training on Atari used to take 10+ hours on older hardware—people would leave render_mode='human' enabled and wonder why their agent wasn’t improving. The agent was fine; it just wasn’t seeing enough samples.

Common Error: env.render() Before env.reset()

One more footgun. If you call env.render() before env.reset(), Gymnasium raises:

gymnasium.error.Error: Cannot call `render()` before calling `reset()`

This is new behavior as of Gymnasium 0.27. The old Gym API allowed it (and would render a black screen). The fix is simple—always call reset() first:

env = gym.make('CartPole-v1', render_mode='rgb_array')
obs, info = env.reset(seed=42)  # Initialize the environment
frame = env.render()  # Now this works

I’ve seen this trip up people migrating from the old gym library to gymnasium. If you have legacy code that checks rendering during env setup, refactor it to defer rendering until after the first reset.

FAQ

Q: Can I use render_mode=’human’ for the first 1000 steps, then switch to None?

No. The render_mode parameter is set at environment creation (gym.make()) and cannot be changed dynamically. You’d need to create a new environment instance. A workaround is to use 'rgb_array' mode throughout and conditionally display the frames:

import matplotlib.pyplot as plt

env = gym.make('CartPole-v1', render_mode='rgb_array')
for step in range(10000):
    # ...
    if step < 1000 and step % 10 == 0:  # show every 10th frame
        plt.imshow(env.render())
        plt.pause(0.01)

But honestly, just train headless and record videos periodically.

Q: Why does render_mode=’rgb_array’ return frames with inconsistent shapes across environments?

Gymnasium doesn’t enforce a standard resolution. CartPole returns (400, 600, 3), LunarLander returns (400, 600, 3), but Atari games return (210, 160, 3). MuJoCo environments vary wildly (480, 640, 3) by default but accept a width and height parameter:

env = gym.make('Humanoid-v4', render_mode='rgb_array', width=1280, height=720)

If you’re training a vision-based policy, wrap the env with a resizing layer or use Stable Baselines3’s VecFrameStack + VecTransposeImage.

Q: Does render_mode=’rgb_array’ use the GPU?

Depends on the backend. MuJoCo uses OpenGL by default, which uses the GPU. Pygame (used by CartPole, LunarLander) uses CPU rendering. For headless servers without a GPU, set MUJOCO_GL=osmesa to force CPU rendering. It’s slower but doesn’t require CUDA drivers.

What Actually Works in Production

After debugging this across a dozen projects, here’s my actual workflow:

1. Local dev: Use render_mode=None for training. If I need to visually debug, I run a separate script with render_mode='human' for 10-20 episodes, then kill it.
2. Cloud training: Always render_mode=None. Set MUJOCO_GL=osmesa and SDL_VIDEODRIVER=dummy in the startup script to prevent rendering init errors.
3. Monitoring: Use VecVideoRecorder to log videos every 50k steps. Upload to W&B or TensorBoard. This gives me visual progress without slowing training.
4. Final eval: After training, run a separate evaluation script with render_mode='rgb_array', record 10 episodes, and generate a summary video with imageio.

The key insight: rendering is for humans, not for training loops. Your agent doesn’t benefit from seeing pretty graphics—it only sees state vectors and rewards. Save rendering for when you need to understand what’s happening.

I’m still curious whether there’s a way to render asynchronously on a separate thread without blocking the training loop. In theory, you could decouple env.step() from env.render() and let rendering lag behind by a few frames. But I haven’t seen this implemented cleanly in any library yet. If someone’s built this, I’d love to hear about it—managing render threads sounds like a debugging nightmare, but the performance gains might be worth it for real-time monitoring dashboards.