The State of RL Environments in 2026

October 2026

When we set out to build a directory of reinforcement learning environments, we expected a weekend of link-collecting. Instead we got a lesson in how open-source ecosystems age: we verified 51 environments against the live GitHub API, and the pattern that emerged is unmistakable. The RL environments world has consolidated, accelerated, and quietly buried half its famous names.

The Farama consolidation

The single biggest story is stewardship. The Farama Foundation — a community organization — now maintains Gymnasium, PettingZoo, MiniGrid, and the Arcade Learning Environment. HighwayEnv and D4RL have migrated from their original authors' organizations into Farama's hands. This is what healthy open source looks like: when labs move on, the community adopts the infrastructure everyone depends on. If you're starting a project today, a Farama-stearded suite is the safest dependency you can pick.

The GPU-native wave

The second story is speed. The rising entries in our directory are all JAX/GPU-native: NVIDIA's Isaac Lab (8,300+ stars, commits landing daily), Google DeepMind's MuJoCo Playground, plus Brax, Jumanji, and VMAS. These suites run thousands of environments in parallel on a single GPU — the kind of throughput that makes large-scale RL iteration practical. Robotics research, in particular, has moved here. If your work involves continuous control at scale, this is where the momentum is.

The graveyard

The third story is the one nobody announces. OpenAI Retro is archived. DeepMind Lab is stale. The NetHack Learning Environment is archived. SMAC hasn't been touched in years. RLCard, PySC2, Crafter, MiniHack, and Neural MMO are all unmaintained. These are names that still appear in tutorials and course slides — and they'll still teach you something — but pinning a new research project to them is building on a foundation nobody maintains.

The surprise: we expected Microsoft's AirSim to be dead. It isn't — the API shows commits as recent as September 2026. Sometimes the obituaries are premature; that's why we check commit history instead of reputation.

What this means for your next project

Learning RL? Start with Gymnasium's classic control (CartPole, LunarLander) and MiniGrid — fast, maintained, and debuggable on a laptop.
Benchmarking an algorithm? The Arcade Learning Environment, MuJoCo locomotion, and Procgen remain the standards everyone compares against.
Robotics? Go GPU-native: Isaac Lab or MuJoCo Playground.
Multi-agent? PettingZoo is the maintained unified API; treat SMAC as a classic reference, not a dependency.

Our directory marks every environment's maintenance status explicitly, and the env picker will match you to a shortlist in four questions. The ecosystem will keep shifting — we'll keep checking.