AgentsOnline
AgentBench
Measures language-model agents across operating systems, databases, games, web shopping, and other interactive environments.
- Task count
- 8
Agents benchmarks
There are 31 agents in the index.
Results
31 benchmarks
AgentsOnline
AI2-THOR
Evaluates embodied agents in interactive simulated indoor scenes with object manipulation and navigation.
AgentsOnline
ALFWorld
Measures agents that learn high-level policies in text environments and execute the same household goals in embodied worlds.
AgentsOnline
AndroidWorld
Measures autonomous agents on dynamically generated tasks across real Android applications.
- Task count
- 116
AgentsOnline
BabyAI GoTo
Tests grounded language understanding by asking an agent to navigate a grid world to a described object.
AgentsOnline
Cicero Diplomacy
Evaluates strategic planning, negotiation, and natural-language coordination by agents playing the board game Diplomacy.
AgentsOnline
Crafter
Measures reinforcement-learning agents across a survival game designed to expose a broad spectrum of capabilities.
AgentsOnline
Crafter
Evaluates reinforcement-learning agents on survival, exploration, resource collection, and technology-tree achievements.
- Task count
- 22
AgentsOnline
CRMArena
Measures agents on professional customer-relationship-management tasks in realistic business scenarios.
AgentsOnline
EmbodiedQA
Tests agents that navigate a simulated home to gather visual evidence and answer a question.
AgentsOnline
FinGAIA
Evaluates agents on multistep financial work involving web research, files, multimodal inputs, code, and calculations.
- Task count
- 407
AgentsOnline
Jericho
Measures agents in human-authored interactive-fiction games through a common learning interface.
AgentsOnline
LIBERO-Spatial
Tests lifelong robot learning under spatial-relation distribution shifts in simulated manipulation tasks.
AgentsOnline
MineDojo
Measures open-ended embodied agents across thousands of Minecraft tasks with an internet-scale knowledge base.
- Task count
- 3,142
AgentsOnline
Mobile-Bench
Measures mobile agents on instructions that require perception, planning, and actions in Android applications.
AgentsOnline
OSWorld-Verified
Measures multimodal agents on reproducible, open-ended tasks in real desktop applications; the verified release repairs the original task set.
- Task count
- 369
AgentsOnline
PokeLLMon
Measures a language-model agent on tactical decision making in competitive Pokémon battles.
AgentsOnline
RoboCasa
Evaluates robot policies on everyday kitchen manipulation tasks in a large-scale simulation framework.
- Task count
- 365
AgentsOnline
SmartPlay
Measures agent reasoning, planning, spatial understanding, learning, and handling of randomness across six games.
- Task count
- 6
AgentsOnline
Spider 2.0
Tests agents on realistic enterprise text-to-SQL workflows with long contexts and cloud or local databases.
- Task count
- 632
AgentsOnline
VirtualHome
Evaluates generation and execution of household activity programs in simulated 3D homes.
AgentsOnline
ARC-AGI-3
Tests exploration, goal discovery, world-model building, and planning in novel interactive environments.
AgentsOnline
BabyAI
Measures grounded language understanding by training agents to follow generated instructions in grid worlds.
AgentsOnline
GAIA
Measures general assistants on real-world questions requiring reasoning, tools, browsing, and multimodal inputs.
- Task count
- 466
Other categories in Online
More benchmarks
Questions
What other kinds of benchmarks are in Online?
Other categories in Online: Multimodal (51 benchmarks), Reasoning (50 benchmarks), Safety (25 benchmarks), Science (24 benchmarks), and Coding (17 benchmarks).
Published by AI Benchmark Index. How we check each benchmark.