# Agents benchmarks AI Benchmark Index catalogs 31 agents with source-linked metadata. 31 benchmarks. Also on the [web page](https://aibenchmarkindex.com/category/agents). ## Benchmarks 1. [AgentBench](https://aibenchmarkindex.com/global/online/agentbench.md), Agents, Online: Measures language-model agents across operating systems, databases, games, web shopping, and other interactive environments. 2. [AI2-THOR](https://aibenchmarkindex.com/global/online/ai2-thor.md), Agents, Online: Evaluates embodied agents in interactive simulated indoor scenes with object manipulation and navigation. 3. [ALFWorld](https://aibenchmarkindex.com/global/online/alfworld.md), Agents, Online: Measures agents that learn high-level policies in text environments and execute the same household goals in embodied worlds. 4. [AndroidWorld](https://aibenchmarkindex.com/global/online/androidworld.md), Agents, Online: Measures autonomous agents on dynamically generated tasks across real Android applications. 5. [BabyAI GoTo](https://aibenchmarkindex.com/global/online/babyai-goto.md), Agents, Online: Tests grounded language understanding by asking an agent to navigate a grid world to a described object. 6. [Cicero Diplomacy](https://aibenchmarkindex.com/global/online/cicero-diplomacy.md), Agents, Online: Evaluates strategic planning, negotiation, and natural-language coordination by agents playing the board game Diplomacy. 7. [Crafter](https://aibenchmarkindex.com/global/online/crafter.md), Agents, Online: Measures reinforcement-learning agents across a survival game designed to expose a broad spectrum of capabilities. 8. [Crafter](https://aibenchmarkindex.com/global/online/crafter-2.md), Agents, Online: Evaluates reinforcement-learning agents on survival, exploration, resource collection, and technology-tree achievements. 9. [CRMArena](https://aibenchmarkindex.com/global/online/crmarena.md), Agents, Online: Measures agents on professional customer-relationship-management tasks in realistic business scenarios. 10. [EmbodiedQA](https://aibenchmarkindex.com/global/online/embodiedqa.md), Agents, Online: Tests agents that navigate a simulated home to gather visual evidence and answer a question. 11. [FinGAIA](https://aibenchmarkindex.com/global/online/fingaia.md), Agents, Online: Evaluates agents on multistep financial work involving web research, files, multimodal inputs, code, and calculations. 12. [Jericho](https://aibenchmarkindex.com/global/online/jericho.md), Agents, Online: Measures agents in human-authored interactive-fiction games through a common learning interface. 13. [LIBERO-Spatial](https://aibenchmarkindex.com/global/online/libero-spatial.md), Agents, Online: Tests lifelong robot learning under spatial-relation distribution shifts in simulated manipulation tasks. 14. [MineDojo](https://aibenchmarkindex.com/global/online/minedojo.md), Agents, Online: Measures open-ended embodied agents across thousands of Minecraft tasks with an internet-scale knowledge base. 15. [Mobile-Bench](https://aibenchmarkindex.com/global/online/mobile-bench.md), Agents, Online: Measures mobile agents on instructions that require perception, planning, and actions in Android applications. 16. [OSWorld-Verified](https://aibenchmarkindex.com/global/online/osworld-verified.md), Agents, Online: Measures multimodal agents on reproducible, open-ended tasks in real desktop applications; the verified release repairs the original task set. 17. [PokeLLMon](https://aibenchmarkindex.com/global/online/pokellmon.md), Agents, Online: Measures a language-model agent on tactical decision making in competitive Pokémon battles. 18. [RoboCasa](https://aibenchmarkindex.com/global/online/robocasa.md), Agents, Online: Evaluates robot policies on everyday kitchen manipulation tasks in a large-scale simulation framework. 19. [SmartPlay](https://aibenchmarkindex.com/global/online/smartplay.md), Agents, Online: Measures agent reasoning, planning, spatial understanding, learning, and handling of randomness across six games. 20. [Spider 2.0](https://aibenchmarkindex.com/global/online/spider-2-0.md), Agents, Online: Tests agents on realistic enterprise text-to-SQL workflows with long contexts and cloud or local databases. 21. [VirtualHome](https://aibenchmarkindex.com/global/online/virtualhome.md), Agents, Online: Evaluates generation and execution of household activity programs in simulated 3D homes. 22. [ARC-AGI-3](https://aibenchmarkindex.com/global/online/arc-agi-3.md), Agents, Online: Tests exploration, goal discovery, world-model building, and planning in novel interactive environments. 23. [BabyAI](https://aibenchmarkindex.com/global/online/babyai.md), Agents, Online: Measures grounded language understanding by training agents to follow generated instructions in grid worlds. 24. [GAIA](https://aibenchmarkindex.com/global/online/gaia.md), Agents, Online: Measures general assistants on real-world questions requiring reasoning, tools, browsing, and multimodal inputs. 25. [Habitat HomeRobot OVMM Challenge](https://aibenchmarkindex.com/global/online/habitat-homerobot-ovmm-challenge.md), Agents, Online: Measures open-vocabulary mobile manipulation systems that find objects, navigate, and rearrange them in simulated homes. 26. [Kaggle Game Arena](https://aibenchmarkindex.com/global/online/kaggle-game-arena.md), Agents, Online: Evaluates strategic planning, adaptation, and robustness through head-to-head model competitions in chess, poker, and Werewolf. 27. [Terminal-Bench](https://aibenchmarkindex.com/global/online/terminal-bench.md), Agents, Online: Tests whether agents complete end-to-end tasks inside reproducible terminal environments. 28. [Terminal-Bench 2.0](https://aibenchmarkindex.com/global/online/terminal-bench-2-0.md), Agents, Online: Tests agents on valuable containerized work such as debugging, security fixes, and scientific workflows. 29. [TextWorld](https://aibenchmarkindex.com/global/online/textworld.md), Agents, Online: Measures reinforcement-learning agents in generated text-based games with controllable difficulty and scope. 30. [Werewolf Arena](https://aibenchmarkindex.com/global/online/werewolf-arena.md), Agents, Online: Evaluates model deception, deduction, persuasion, and turn-taking through arena-style social-deduction games. 31. [Benchmark Cost per Task](https://aibenchmarkindex.com/global/online/benchmark-cost-per-task.md), Agents, Online: Measures dollars, retries, and tokens consumed per verified model result. ## Agents by city - [Online, Global](https://aibenchmarkindex.com/global/online/category/agents.md): 31 benchmarks ## Related categories - [Multimodal](https://aibenchmarkindex.com/category/multimodal.md): 51 benchmarks - [Reasoning](https://aibenchmarkindex.com/category/reasoning.md): 50 benchmarks - [Safety](https://aibenchmarkindex.com/category/safety.md): 25 benchmarks - [Science](https://aibenchmarkindex.com/category/science.md): 24 benchmarks - [Coding](https://aibenchmarkindex.com/category/coding.md): 17 benchmarks - [Retrieval](https://aibenchmarkindex.com/category/retrieval.md): 17 benchmarks - [Memory](https://aibenchmarkindex.com/category/memory.md): 10 benchmarks - [Browser](https://aibenchmarkindex.com/category/browser.md): 7 benchmarks - [Voice](https://aibenchmarkindex.com/category/voice.md): 7 benchmarks - [Math](https://aibenchmarkindex.com/category/math.md): 6 benchmarks - [Other](https://aibenchmarkindex.com/category/other.md): 6 benchmarks - [Tools](https://aibenchmarkindex.com/category/tools.md): 6 benchmarks - [Commerce](https://aibenchmarkindex.com/category/commerce.md): 4 benchmarks - [Long-horizon](https://aibenchmarkindex.com/category/long-horizon.md): 3 benchmarks > **Check before you compare.** Benchmark methods and leaderboards change. Follow the cited primary source before using a result in a decision.