MultimodalOnline
COCO Caption
Evaluates image-caption generation on everyday scenes using reference captions and standardized captioning metrics.
- Task count
- 330,000
AI benchmark index
AI Benchmark Index catalogs 264 AI benchmarks from primary sources.
Results
264 benchmarks
SafetyOnline
Do-Not-Answer
Evaluates whether language models decline risky instructions while still answering harmless requests.
- Task count
- 939
MultimodalOnline
DocVQA
Measures question answering over document images where layout, text, and document structure matter.
- Task count
- 50,000
AgentsOnline
GAIA
Measures general assistants on real-world questions requiring reasoning, tools, browsing, and multimodal inputs.
- Task count
- 466
ReasoningOnline
Gym Sokoban
Evaluates planning and reinforcement learning in procedurally generated box-pushing puzzles.
AgentsOnline
Habitat HomeRobot OVMM Challenge
Measures open-vocabulary mobile manipulation systems that find objects, navigate, and rearrange them in simulated homes.
VoiceOnline
HEAR
Evaluates transferable audio representations across speech, music, and environmental sound tasks.
- Task count
- 19
ReasoningOnline
HELM Lite
Evaluates language models on a compact suite of ten HELM scenarios selected for broad coverage and lower evaluation cost.
- Task count
- 10
ReasoningOnline
HumorBench
Evaluates English and Spanish multi-turn humor understanding through joke classification and identification of each line’s role.
MultimodalOnline
InfographicVQA
Measures question answering that combines infographic text, layout, graphics, data visualizations, and arithmetic.
AgentsOnline
Kaggle Game Arena
Evaluates strategic planning, adaptation, and robustness through head-to-head model competitions in chess, poker, and Werewolf.
- Task count
- 3
ReasoningOnline
MMLU-Pro
Tests knowledge and reasoning on harder ten-option questions with reduced reliance on recall.
- Task count
- 12,032
ScienceOnline
MultiMedQA
Evaluates medical question answering across seven datasets spanning professional exams, research, and consumer health queries.
- Task count
- 7
MemoryOnline
Needle In A Haystack
Tests whether a language model retrieves a planted fact from increasingly long contexts and at different insertion depths.
MultimodalOnline
OK-VQA
Measures visual question answering that requires knowledge beyond the content visible in an image.
ScienceOnline
PubMedQA
Evaluates biomedical research question answering from PubMed abstracts, including an expert-annotated test set.
- Task count
- 1,000
ScienceOnline
SciBench
Evaluates college-level scientific reasoning, domain knowledge, and calculation on textbook problems.
- Task count
- 692
ReasoningOnline
SimpleBench
Tests everyday reasoning about space, time, social situations, and adversarial wording with multiple-choice questions.
MultimodalOnline
Something-Something V2
Tests recognition of everyday actions whose labels depend on temporal and physical interactions between people and objects.
ReasoningOnline
SQuAD 2.0
Tests extractive reading comprehension and whether a system abstains when a Wikipedia passage does not answer the question.
MultimodalOnline
SVGBench
Tests whether language models can generate SVGs that satisfy requirements involving knowledge, code, instructions, and physical reasoning.
AgentsOnline
Terminal-Bench
Tests whether agents complete end-to-end tasks inside reproducible terminal environments.
- Task count
- 100
AgentsOnline
Terminal-Bench 2.0
Tests agents on valuable containerized work such as debugging, security fixes, and scientific workflows.
MultimodalOnline
TextVQA
Measures visual question answering that requires reading and reasoning about text in natural images.
- Task count
- 45,336
Cities in Global
Categories in Global
Questions
Which cities in Global have the most benchmarks?
The cities in Global with the most benchmarks are Online (264 benchmarks).
What kinds of benchmarks are in Global?
Categories in Global: Multimodal, Reasoning, Agents, Safety, and Science.
Published by AI Benchmark Index. How we check each benchmark.