MultimodalOnline
MVBench
Measures multi-modal video understanding across 20 temporal and perceptual tasks.
- Task count
- 20
Multimodal benchmarks
There are 51 multimodal in the index.
Results
51 benchmarks
MultimodalOnline
NExT-QA
Evaluates causal and temporal reasoning in video question answering rather than simple visual description.
MultimodalOnline
OCRBench
Measures OCR in multimodal models across text recognition, scene-text VQA, document VQA, extraction, and handwritten math.
- Task count
- 1,000
MultimodalOnline
OCRBench v2
Measures bilingual visual-text localization, recognition, extraction, and reasoning across diverse real-world scenarios.
- Task count
- 10,000
MultimodalOnline
PartiPrompts
Tests text-to-image systems with over 1,600 prompts spanning simple scenes, complex compositions, writing, and abstract concepts.
- Task count
- 1,600
MultimodalOnline
SEED-Bench
Measures image and video comprehension across multimodal ability dimensions using human-annotated multiple-choice questions.
MultimodalOnline
TasteBench
Studies whether aesthetic-preference benchmarks reward expert-aligned craft judgment rather than shortcuts; its item pool remains pending licence clearance.
MultimodalOnline
Video-Bench
Evaluates video language models on perception, prior-knowledge use, and video-based decision making.
MultimodalOnline
Video-MME
Evaluates multimodal models on short, medium, and long videos across visual domains, durations, and subtitle settings.
- Task count
- 2,700
MultimodalOnline
Video-MMMU
Measures whether multimodal models acquire and apply new knowledge from professional lecture videos.
- Task count
- 900
MultimodalOnline
We-Math
Tests whether multimodal models solve visual math problems through coherent one-step and multi-step reasoning.
- Task count
- 6,500
MultimodalOnline
ActivityNet-QA
Tests question answering over long videos with questions that require spatial and temporal reasoning.
- Task count
- 58,000
MultimodalOnline
AI2D
Measures parsing and question answering over grade-school science diagrams with annotated constituents and relationships.
MultimodalOnline
AVA
Evaluates spatiotemporally localized recognition of 80 atomic actions in densely annotated movie clips.
- Task count
- 430
MultimodalOnline
CIFAR-10
Evaluates image classification on 60,000 small color images evenly divided among ten classes.
- Task count
- 60,000
MultimodalOnline
COCO Caption
Evaluates image-caption generation on everyday scenes using reference captions and standardized captioning metrics.
- Task count
- 330,000
MultimodalOnline
DocVQA
Measures question answering over document images where layout, text, and document structure matter.
- Task count
- 50,000
MultimodalOnline
InfographicVQA
Measures question answering that combines infographic text, layout, graphics, data visualizations, and arithmetic.
MultimodalOnline
OK-VQA
Measures visual question answering that requires knowledge beyond the content visible in an image.
MultimodalOnline
Something-Something V2
Tests recognition of everyday actions whose labels depend on temporal and physical interactions between people and objects.
MultimodalOnline
SVGBench
Tests whether language models can generate SVGs that satisfy requirements involving knowledge, code, instructions, and physical reasoning.
MultimodalOnline
TextVQA
Measures visual question answering that requires reading and reasoning about text in natural images.
- Task count
- 45,336
MultimodalOnline
VizWiz VQA
Measures answers to visual questions submitted by blind people in real accessibility scenarios.
MultimodalOnline
VQAv2
Measures visual question answering while reducing language-only shortcuts through complementary image pairs.
- Task count
- 1,105,904
Other categories in Online
More benchmarks
Questions
What other kinds of benchmarks are in Online?
Other categories in Online: Reasoning (50 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), Science (24 benchmarks), and Coding (17 benchmarks).
Published by AI Benchmark Index. How we check each benchmark.