Open Source RL Frameworks


AI-Optimizer is a next-generation deep reinforcement learning suit, providing rich algorithm libraries ranging from model-free to model-based RL algorithms, from single-agent to multi-agent algorithms. Moreover, AI-Optimizer contains a flexible and easy-to-use distributed training framework for efficient policy training.
License: No License


AlpacaFarm is a simulation framework for methods that learn from human feedback.
License: Apache License 2.0


CleanRL is a Deep Reinforcement Learning library that provides high-quality single-file implementation with research-friendly features. The implementation is clean and simple, yet we can scale it to run thousands of experiments using AWS Batch.
License: Other

CompilerGym is a library of easy to use and performant reinforcement learning environments for compiler tasks.
License: MIT License

d3rlpy is an offline deep reinforcement learning library for practitioners and researchers.
License: MIT License

DIAMBRA Arena is a software package featuring a collection of high-quality environments for Reinforcement Learning research and experimentation.
License: Other

Dopamine is a research framework for fast prototyping of reinforcement learning algorithms. It aims to fill the need for a small, easily grokked codebase in which users can freely experiment with wild ideas (speculative research).
License: Apache License 2.0

EvoTorch is an open source evolutionary computation library developed at NNAISENSE, built on top of PyTorch.
License: Apache License 2.0

FinRL is the first open-source framework to demonstrate the great potential of financial reinforcement learning.
License: MIT License


Gymnasium is an open source Python library for developing and comparing reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments, as well as a standard set of environments compliant with that API.
License: MIT License

Gymnasium-Robotics contains a collection of Reinforcement Learning robotic environments that use the Gymansium API. The environments run with the MuJoCo physics engine and the maintained mujoco python bindings.
License: MIT License

Jumanji is a suite of Reinforcement Learning (RL) environments written in JAX providing clean, hardware-accelerated environments for industry-driven research.
License: Apache License 2.0

MALib is a parallel framework of population-based learning nested with reinforcement learning methods. MALib provides higher-level abstractions of MARL training paradigms, which enables efficient code reuse and flexible deployments on different distributed computing paradigms.
License: MIT License

MARLlib is a comprehensive Multi-Agent Reinforcement Learning algorithm library based on RLlib. It provides MARL research community with a unified platform for building, training, and evaluating MARL algorithms.
License: MIT License


Melting Pot is a suite of test scenarios for multi-agent reinforcement learning.
License: Apache License 2.0


The Minigrid library contains a collection of discrete grid-world environments to conduct research on Reinforcement Learning. The environments follow the Gymnasium standard API and they are designed to be lightweight, fast, and easily customizable.
License: Other


MiniWorld is a minimalistic 3D interior environment simulator for reinforcement learning & robotics research.
License: Apache License 2.0

ML-Agents is an open-source project that enables games and simulations to serve as environments for training intelligent agents.
License: Other

MushroomRL is a Python Reinforcement Learning (RL) library whose modularity allows to easily use well-known Python libraries for tensor computation (e.g. PyTorch, Tensorflow) and RL benchmarks (e.g. OpenAI Gym, PyBullet, Deepmind Control Suite).
License: MIT License

PARL is a flexible and high-efficient reinforcement learning framework.
License: Apache License 2.0

PettingZoo is a Python library for conducting research in multi-agent reinforcement learning, akin to a multi-agent version of Gymnasium.
License: Other



Safety-Gymnasium is a highly scalable and customizable safe reinforcement learning environment library.
License: Apache License 2.0

skrl is an open-source modular library for Reinforcement Learning written in Python (using PyTorch) and designed with a focus on readability, simplicity, and transparency of algorithm implementation.
License: MIT License

A fork of OpenAI Baselines, implementations of reinforcement learning algorithms.
License: MIT License



Train transformer language models with reinforcement learning.
License: Apache License 2.0
Last Updated: Dec 26, 2023