RL environment list

A comprehensive list of categorized reinforcement learning environments.

Started and maintained by Andrew Szot and Youngwoon Lee.

Related Collections

Two other resources for RL environments:

• Environment Zoo
• Awesome Deep RL

Table of Contents

Environments are listed alphabetically.

• Robotics
• Games
• Multi-Task Learning
• Suites
• Generalization
• Navigation
• Home (More Navigation)
• Multi-Agent
• Safety
• Autonomous Driving
• Humanoid
• Text
• Misc
• Physics Simulators

Robotics

	Assistive-gym 6 assistive tasks (ScratchItch, BedBathing, Feeding, Drinking, Dressing, and ArmManipulation). 4 commercial robots (PR2, Jaco, Baxter, Sawyer). 2 human states: static or active (takes actions according to a separate control policy). Customizable female and male human models. 40 actuated human joints (head, torso, arms, waist, and legs).Realistic human joint limit.
	Dexterous Gym Extensions of the OpenAI Gym Dexterous Manipulation Environments. Multiple environments requiring cooperation between two hands (handing objects over, throwing/catching objects). "Pen Spin" Environment - train a hand to spin a pen between its fingers.
	DoorGym Train a policy to open up various doors. Unity integration. Random door knob generator and door knob dataset.
	FurnitureSim Simulator for FurnitureBench, implemented using Factory and IsaacGym. Provides 8 furniture models with diverse long-horizon complex interactions for furniture assembly, e.g., inserting, screwing.
	Gym Gazebo 2 Toolkit for developing and comparing reinforcement learning algorithms using ROS 2 and Gazebo.
	Gym Ignition Provides the capability of creating reproducible robotics environments for reinforcement learning research. Accelerated and multiprocess execution
	IKEA Furniture Assembly Complex long-horizon manipulation tasks. Includes 80+ furniture models, customizable background, lighting and textures. Features Baxter, Sawyer, and more robots.
	Meta-World 50 diverse robot manipulation tasks on a simulated Sawyer robotic arm. Also includes a variety of evaluation modes varying the number of training and testing tasks.
	CALVIN Complex long-horizon language conditioned manipulation benchmark with 1000 sequences of 5 instruction chains, data is human play data. Provides a variety of train/test evaluation modes, flexible sensor suite, 4 environments and 34 diverse tasks.
	Playroom Variety of tasks in desk scenario. Evaluation code and play dataset will be included soon.
	RAISIM Raisim is a physics engine for rigid-body dynamics simulation. Although it is a general physics engine, it has been mainly used/tested for robotics and reinforcement learning so far. It features an efficient implementation of recursive algorithms for articulated system dynamics (Recursive Newton-Euler and Composite Rigid Body Algorithm). RaisimLib is an exported cmake package of raisim.
	RLBench 100 unique, hand designed tasks. Vision-guided manipulation, imitation learning, multi-task learning, geometric computer vision and few-shot learning.
	Robosuite A set of standard benchmarking tasks in robots. Defines a framework for easily creating new tasks and environments.
	Roboschool Control robots in simulation. Can use other physics engines other than MuJoCo. Alternative to standard OpenAI Gym mujoco environments. Easy to train multiple agents at once.
	Rex-Gym OpenAI Gym environments for an open-source quadruped robot (SpotMicro)

Games

	Bomberland Multi-agent 2D grid environment based on Bomberman.
	Coin-Run Training environment which provides a metric for an agent’s ability to transfer its experience to novel situations.
	Gym Retro Gym Retro lets you turn classic video games into Gym environments for reinforcement learning and comes with integrations for ~1000. games.
	Holodeck High Fidelity Simulator for Reinforcement Learning and Robotics Research.
	MarLÖ : Reinforcement Learning + Minecraft A high level API built on top of Project MalmÖ to facilitate Reinforcement Learning experiments with a great degree of generalizability, capable of solving problems in pseudo-random, procedurally changing single and multi agent environments within the world of the mediatic phenomenon game Minecraft.
	Minecraft Data API for the MineRLv0 dataset. Also has minecraft environment simulator with basic built in tasks.
	PHYRE Benchmark for physical reasoning that contains a set of simple classical mechanics puzzles in a 2D enviroment.
	Soccer Simulator Can control one or all football players at a time. Includes football academy for diverse scenarios such as various passing scenarios.
	StarCraft 2 Provides an interface for RL agents to interact with StarCraft 2, getting observations and sending actions.
	SuperMario Gym wrapper for the Super Mario levels. Includes many levels.
	TorchCraft Python interface for playing "StarCraft: Brood War".
	VizDoom ViZDoom allows developing AI bots that play Doom using only the visual information (the screen buffer).

Multi-Task Learning

	Meta-World 50 diverse robot manipulation tasks on a simulated Sawyer robotic arm. Also includes a variety of evaluation modes varying the number of training and testing tasks.
	Multiworld Variety of Gym GoalEnvs that return the goal in the observation.
	CALVIN Complex long-horizon language conditioned manipulation benchmark with 1000 sequences of 5 instruction chains, data is human play data. Provides a variety of train/test evaluation modes, flexible sensor suite, 4 environments and 34 diverse tasks.
	Playroom Variety of tasks in desk scenario. Evaluation code and play dataset will be included soon.
	RoboDesk Multi-task RL benchmark that comes with tasks from easy to hard, with dense and sparse rewards. Based on the Playroom desk env, with more robust physics settings and controls that are suitable for RL.
	RLBench 100 unique, hand designed tasks. Vision-guided manipulation, imitation learning, multi-task learning, geometric computer vision and few-shot learning.

Suites

	DeepMind Control Suite A variety of benchmarking continuous control tasks.
	OpenAI Gym Atari 59 Atari 2600 games.
	OpenAI Gym Classic Control theory problems from the classic RL literature.
	OpenAI Gym Mujoco Continuous control tasks, running in a fast physics simulator.
	OpenAI Gym Robotics Simulated goal-based tasks for the Fetch and ShadowHand robots.
	Unity Agents A number of control tasks in the Unity engine. Includes example of parallel learning.
	PettingZoo It's akin to a multi-agent version of OpenAI's Gym library. Includes Atari, Classic Games, Particle Environments and many more.

Generalization

	Cartpole Generalization Test generalization through varying the mass and length of the pole in CartPole.
	Natural RL Environment Play common gym tasks with randomly generated backgrounds to test generalization.
	DMControl Generalization Benchmark Generalization benchmark for continuous control tasks from DeepMind Control Suite. Includes hundreds of environments with randomized colors and dynamic video backgrounds of varying difficulty.
	Procgen 16 simple-to-use procedurally-generated environments which provide a direct measure of how quickly a reinforcement learning agent learns generalizable skills. The environments run at high speed (thousands of steps per second) on a single core.
	Animal-AI Testbed 900 tasks reflecting various cognitive skills of animals. Powered by Unity ml-agent.
	Crafter Open world survival game that evaluates many agent abilities within one environment. Faster and easier than Minecraft but poses some of the same challenges. Can be used to evaluate reward-based or unsupervised agents (e.g. artificial curiosity).

Navigation

	DeepMind Lab Provides a suite of challenging 3D navigation and puzzle-solving tasks for learning agents.
	gym-maze A simple 2D maze environment where an agent (blue dot) finds its way from the top left corner (blue square) to the goal at the bottom right corner (red square). The objective is to find the shortest path from the start to the goal.
	gym-minigrid Lightweight and fast grid world implementation with various included tasks. Easily modifable and extendable.
	gym-miniworld Minimalistic 3D interior simulator as an alternative to VizDoom or DMLab. Easily modifable and extendable.
	Obstacle Tower Traverse through procedurally generated floors which get progressively harder. Challenging visual inputs.

Home (More Navigation)

	AI2THOR An Interactive 3D Environment for Visual AI
	Gibson 3d navigation in indoor scans
	Habitat AI Habitat enables training of embodied AI agents (virtual robots) in a highly photorealistic & efficient 3D simulator, before transferring the learned skills to reality
	HoME: a Household Multimodal Environment A platform for agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context.
	House3D House3D is a virtual 3D environment which consists of thousands of indoor scenes equipped with a diverse set of scene types, layouts and objects sourced from the SUNCG dataset It consists of over 45k indoor 3D scenes, ranging from studios to two-storied houses with swimming pools and fitness rooms All 3D objects are fully annotated with category labels Multiple observation modalities Fast rendering at thousands of frames per second
	MINOS MINOS is a simulator designed to support the development of multisensory models for goal-directed navigation in complex indoor environments. MINOS leverages large datasets of complex 3D environments and supports flexible configuration of multimodal sensor suites.
	Nvidia ISAAC simulator A virtual robotics laboratory and a high-fidelity 3D world simulator
	VirtualHome A 3D environment allowing to simulate and generate videos of activities as sequences of actions and interaction.

Multi-Agent

	Massive Multi Agent Game Environment We consider MMORPGs (Massive Multiplayer Online Role Playing Games) the best proxy for the real world among human games: they are complete macrocosms featuring thousands of agents per persistent world, diverse skilling systems, global economies, complex emergent social structures, and ad-hoc high stakes single and team based conflict.
	Multi-agent Particle Environment A simple multi-agent particle world with a continuous observation and discrete action space, along with some basic simulated physics
	OpenAI Multi-Agent Competition Environments Contains many continous control, multi-agent tasks.
	OpenAI Multi-Agent Hide and Seek A team of seekers and a team of hiders. Both teams can use tools to achieve their objective.
	RoboSumo Sumo-wrestling between two ants using continuous control.
	SUMO-RL Multi-agent traffic signal control using SUMO simulator.

Safety

	Assistive-gym 6 assistive tasks (ScratchItch, BedBathing, Feeding, Drinking, Dressing, and ArmManipulation). 4 commercial robots (PR2, Jaco, Baxter, Sawyer). 2 human states: static or active (takes actions according to a separate control policy). Customizable female and male human models. 40 actuated human joints (head, torso, arms, waist, and legs).Realistic human joint limit.
	DeepMind AI Safety Gridworlds This is a suite of reinforcement learning environments illustrating various safety properties of intelligent agents.
	Safety Gym Tools for accelerating safe exploration research.

Autonomous Driving

	Autonomous Vehicle Simulator Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
	BARK-ML Open source environments and reinforcement learning agents for autonomous driving and behavior generation.
	CARLA CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems
	DeepDrive Self Driving Car Simulator End-to-end simulation for self-driving cars
	DeepMind StreetLearn A C++/Python implementation of the StreetLearn environment based on images from Street View, as well as a TensorFlow implementation of goal-driven navigation agents solving the task published in “Learning to Navigate in Cities Without a Map”, NeurIPS 2018
	DeepGTAV v2 A plugin for GTAV that transforms it into a vision-based self-driving car research environment.
	DuckieTown Self-driving car simulator for the Duckietown universe.
	Highway-Env A collection of environments for autonomous driving and tactical decision-making tasks
	SVL Simulator Simulation software to accelerate safe autonomous vehicle development Custom environment to support openai gym interface
	TORCS TORCS, The Open Racing Car Simulator is a highly portable multi platform car racing simulation Many tracks, opponents and cars available Easy to modify

Humanoid

	Full Body Muscle Simulator A basic simulation and control for full-body Musculoskeletal system
	Osim-rl Reinforcement learning environments with musculoskeletal models. Task: learning to walk/move/run using musculoskeletal models.
	Roboschool Control robots in simulation. Can use other physics engines other than MuJoCo. Alternative to standard OpenAI Gym mujoco environments. Easy to train multiple agents at once.

Text

	Jericho A learning environment for man-made Interactive Fiction games.
	TextWorld TextWorld is a sandbox learning environment for the training and evaluation of reinforcement learning (RL) agents on text-based games.

Misc

	Reco Gym Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising.
	RecSim A Configurable Recommender Systems Simulation Platform from Google.
	Gym-ANM Environments that model Active Network Management (ANM) tasks in electricity distribution networks.

Physics Simulators

• Mujoco-py
• PyBullet
• DART
• V-REP

Disclaimer

The list is not comprehensive, so please let us know if there is any environment that is missing, miscategorized, or needs a different description or image. Please submit an issue or open a pull request.