SafetyOnline
AdvBench
Measures whether adversarial suffixes induce aligned language models to produce targeted harmful behavior.
- Task count
- 520
AI benchmark index
AI Benchmark Index catalogs 264 AI benchmarks from primary sources.
Results
264 benchmarks
AgentsOnline
AgentBench
Measures language-model agents across operating systems, databases, games, web shopping, and other interactive environments.
- Task count
- 8
SafetyOnline
AgentDojo
Evaluates agent task utility and resistance to prompt injection while agents use tools over untrusted data.
- Task count
- 97
ReasoningOnline
AGIEval
Tests general problem solving with questions from 20 public admission, qualification, and competition exams.
- Task count
- 20
AgentsOnline
AI2-THOR
Evaluates embodied agents in interactive simulated indoor scenes with object manipulation and navigation.
AgentsOnline
ALFWorld
Measures agents that learn high-level policies in text environments and execute the same household goals in embodied worlds.
ReasoningOnline
AlpacaEval
Measures instruction-following quality by comparing model responses against references with automated preference judges.
- Task count
- 805
AgentsOnline
AndroidWorld
Measures autonomous agents on dynamically generated tasks across real Android applications.
- Task count
- 116
ReasoningOnline
ANLI
Tests natural-language inference on examples collected adversarially against successive model generations.
CodingOnline
APPS
Tests coding-challenge competence on problems ranging from introductory exercises to programming competitions.
- Task count
- 10,000
ToolsOnline
AppWorld
Measures interactive coding agents on tasks spanning simulated everyday apps and their APIs.
ReasoningOnline
ArabicMMLU
Evaluates Arabic language understanding across 40 school-exam tasks from countries in North Africa, the Levant, and the Gulf.
- Task count
- 14,575
ReasoningOnline
ARC-AGI-1
Tests few-shot abstraction by asking systems to infer transformations between small colored grids.
- Task count
- 800
ReasoningOnline
ARC-AGI-2
Tests symbolic interpretation, compositional reasoning, and context-dependent rules on novel grid tasks.
- Task count
- 1,360
ReasoningOnline
Arena-Hard-Auto
Tests open-ended instruction following on difficult real user prompts with automated preference judges.
- Task count
- 750
MultimodalOnline
ASCIIBench
Measures four-choice recognition of ASCII art from text-only, rendered-image, and combined inputs.
- Task count
- 5,135
AgentsOnline
BabyAI GoTo
Tests grounded language understanding by asking an agent to navigate a grid world to a described object.
SafetyOnline
BBQ
Measures social bias in question answering across ambiguous and disambiguated contexts covering nine protected dimensions.
SafetyOnline
BBQ-Lite
Provides a compact BIG-bench version of BBQ for exact-match evaluation of bias-sensitive question answering.
SafetyOnline
BeaverTails
Supports safety classification and preference evaluation with human-labeled question-answer pairs across harm categories.
- Task count
- 330,000
RetrievalOnline
BEIR
Evaluates zero-shot information retrieval across heterogeneous datasets, domains, and retrieval architectures.
- Task count
- 18
ReasoningOnline
BERTScore
Evaluates generated text by matching contextual token embeddings between candidate and reference sentences.
ReasoningOnline
BIG-bench
Tests language-model capabilities across more than 200 collaboratively authored tasks.
- Task count
- 200
ReasoningOnline
BIG-Bench Hard
Tests multi-step reasoning on 23 BIG-bench tasks where early language models trailed average human raters.
- Task count
- 23
Cities in Global
Categories in Global
Questions
Which cities in Global have the most benchmarks?
The cities in Global with the most benchmarks are Online (264 benchmarks).
What kinds of benchmarks are in Global?
Categories in Global: Multimodal, Reasoning, Agents, Safety, and Science.
Published by AI Benchmark Index. How we check each benchmark.