SafetyOnline
AdvBench
Measures whether adversarial suffixes induce aligned language models to produce targeted harmful behavior.
- Task count
- 520
273 benchmarks
AI Benchmark Index
AI Benchmark Index catalogs 273 AI benchmarks with source-linked facts about data, grading, and reported scores.
Recommended benchmarks
AgentsOnline
AgentBench
Measures language-model agents across operating systems, databases, games, web shopping, and other interactive environments.
- Task count
- 8
SafetyOnline
AgentDojo
Evaluates agent task utility and resistance to prompt injection while agents use tools over untrusted data.
- Task count
- 97
ReasoningOnline
AGIEval
Tests general problem solving with questions from 20 public admission, qualification, and competition exams.
- Task count
- 20
AgentsOnline
AI2-THOR
Evaluates embodied agents in interactive simulated indoor scenes with object manipulation and navigation.
AgentsOnline
ALFWorld
Measures agents that learn high-level policies in text environments and execute the same household goals in embodied worlds.
Regions
Cities
Categories
Guides
- How to choose an AI benchmark
Choose an AI benchmark by defining the decision first, then checking task fit, grader validity, data access, saturation, and provenance.
- How to run an AI benchmark yourself for under $20
Keep an AI benchmark run under $20 by fixing the question, sampling tasks, estimating tokens, using deterministic graders, and saving a reproducible run record.
- Silly benchmarks that tell you something real (pelican, skateboard, and friends)
Playful tasks such as drawing a pelican on a bicycle can expose planning, geometry, tool use, and persistence when their grader makes the signal explicit.
- What a saturated benchmark is and how to spot one
A saturated benchmark no longer separates strong systems well; spot it by score clustering, ceiling effects, stale tasks, and shrinking uncertainty-aware gaps.
- Where benchmark data comes from and what its licence lets you do
Benchmark data may be written, sampled, generated, or drawn from public datasets; its licence determines whether you may download, modify, and redistribute it.
Questions
Which regions have the most benchmarks?
The regions with the most benchmarks are Global (264 benchmarks).
Which cities have the most benchmarks?
The cities with the most benchmarks are Online, Global (264 benchmarks).
What kinds of benchmarks are listed?
AI Benchmark Index lists Multimodal (51 benchmarks), Reasoning (50 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), and Science (24 benchmarks).
Published by AI Benchmark Index. How we check each benchmark.