Open Source ML Benchmarking Tools
A benchmarking and visualization tool for adversarial ML.
License: MIT License
EvalAI is an open source platform for evaluating and comparing AI algorithms at scale.
License: Other
Evaluate is a library that makes evaluating and comparing models and reporting their performance easier and more standardized.
License: Apache License 2.0
Holistic Evaluation of Language Models (HELM) is a benchmark framework to increase the transparency of language models.
License: Apache License 2.0
Meta-World is an open-source simulated benchmark for meta-reinforcement learning and multi-task learning consisting of many distinct robotic manipulation tasks.
License: MIT License
OmniSafe is a comprehensive and reliable benchmark for safe reinforcement learning, covering a multitude of SafeRL domains and delivering a new suite of testing environments.
License: Apache License 2.0
A zoo for models tuned for OpenCV DNN with benchmarks on different platforms.
License: Apache License 2.0
Overcooked-AI is a benchmark environment for fully cooperative human-AI task performance, based on the wildly popular video game Overcooked.
License: MIT License
Recommenders contains benchmark and best practices for building recommendation systems, provided as Jupyter notebooks.
License: MIT License
SafePO-Baselines is a benchmark repository for safe reinforcement learning algorithms.
License: Apache License 2.0
Last Updated: Dec 26, 2023