AgentsOnline
TextWorld
Measures reinforcement-learning agents in generated text-based games with controllable difficulty and scope.
AI benchmark list
There are 264 source-checked AI benchmarks in this index.
Results
264 benchmarks
RetrievalOnline
TREC Deep Learning Track
Evaluates passage and document ranking systems on large web collections with graded relevance judgments.
ReasoningOnline
TREC Question Answering Track
Evaluates systems that return concise answers to fact-based questions from large text collections.
MultimodalOnline
VizWiz VQA
Measures answers to visual questions submitted by blind people in real accessibility scenarios.
VoiceOnline
VoxCeleb
Evaluates speaker identification and verification using speech collected from interview videos.
- Task count
- 1,000,000
MultimodalOnline
VQAv2
Measures visual question answering while reducing language-only shortcuts through complementary image pairs.
- Task count
- 1,105,904
AgentsOnline
Werewolf Arena
Evaluates model deception, deduction, persuasion, and turn-taking through arena-style social-deduction games.
VoiceOnline
Common Voice
Supports evaluation of speech recognition across languages, accents, and demographic groups using contributed recordings.
MultimodalOnline
ImageNet
Evaluates object recognition using images organized under the WordNet noun hierarchy.
- Task count
- 14,197,122
VoiceOnline
LibriSpeech
Measures automatic speech recognition on roughly 1,000 hours of English audiobook speech.
Long-horizonOnline
LiveBench 2.0
Tests long-horizon AI capabilities with a contamination-limited benchmark whose official site requires JavaScript.
VoiceOnline
MUSDB18
Evaluates music source-separation systems on professionally produced songs with isolated stems.
- Task count
- 150
RetrievalOnline
Natural Questions
Evaluates question answering on real Google queries paired with annotated evidence and answers from Wikipedia pages.
MultimodalOnline
ObjectNet
Tests object recognition on 50,000 crowd-sourced images captured with controls over backgrounds, rotations, and viewpoints.
- Task count
- 50,000
ReasoningOnline
Open LLM Leaderboard
Tracks and compares openly submitted language models across a changing suite of reasoning and knowledge evaluations.
ReasoningOnline
Scale SEAL Leaderboards
Compares frontier models on expert-designed evaluations built from private datasets intended to resist benchmark gaming.
OtherOnline
WMT
Evaluates machine translation systems through annual shared tasks spanning language pairs and translation settings.
BrowserOnline
WebVoyager
Measures end-to-end web agents on visually grounded tasks across live websites.
- Task count
- 643
AgentsOnline
Benchmark Cost per Task
Measures dollars, retries, and tokens consumed per verified model result.
SafetyOnline
Benchmark Honesty
Tests whether a model admits uncertainty after a required tool call fails.
Long-horizonOnline
Benchmark Hundred Step
Measures reliability across tasks with 25 to 100 dependent steps.
MultimodalOnline
Benchmark Pelican
Tests whether a model can produce a valid SVG of a pelican riding a bicycle.
CodingOnline
Benchmark Skateboard
Tests whether a coding model can build a playable skateboard trick scene.
ToolsOnline
Benchmark Tool Errors
Tests retries, idempotency, and recovery after simulated API failures.
Categories in Online
Questions
What kinds of benchmarks are in Online?
Categories in Online: Multimodal (51 benchmarks), Reasoning (50 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), and Science (24 benchmarks).
How many AI benchmarks are in the index?
AI Benchmark Index currently lists 264 benchmarks that link to a primary source.
Can I submit an AI benchmark?
Yes. Submit a public primary source, licence, release date, category, and a one-sentence description of what it measures.
Published by AI Benchmark Index. How we check each benchmark.