MultimodalOnline
ASCIIBench
Measures four-choice recognition of ASCII art from text-only, rendered-image, and combined inputs.
- Task count
- 5,135
Multimodal benchmarks
There are 51 multimodal in the index.
Results
51 benchmarks
MultimodalOnline
BLINK
Measures core visual perception abilities through multiple-choice tasks designed to resist verbal shortcuts.
- Task count
- 3,807
MultimodalOnline
ChartQA
Measures visual and logical reasoning over charts with human-authored and generated questions.
MultimodalOnline
CLIPScore
Measures image-caption compatibility without requiring reference captions by comparing CLIP embeddings.
MultimodalOnline
ClockBench
Evaluates whether vision-language models can read analog clocks across varied faces and times.
- Task count
- 720
MultimodalOnline
CogVideo
Evaluates text-to-video generation; the repository provides model code and references for the original CogVideo and later CogVideoX systems.
MultimodalOnline
Fréchet Video Distance
Measures the distributional similarity of real and generated videos using learned video embeddings.
MultimodalOnline
HPS v2
Evaluates whether a text-to-image model produces images people prefer for the same prompt.
- Task count
- 798,000
MultimodalOnline
ImageNet-A
Tests image classifiers on naturally occurring, unmodified images selected because standard ImageNet models misclassify them.
MultimodalOnline
ImageNet-R
Tests ImageNet classifiers on artistic and other non-photographic renditions of 200 object classes.
- Task count
- 30,000
MultimodalOnline
ImageNet-V2
Retests ImageNet classifiers on three independently collected 10,000-image test sets matched to the original distribution.
- Task count
- 30,000
MultimodalOnline
ImagenHub
Compares conditional image generation and editing systems across standardized tasks, metrics, and human ratings.
MultimodalOnline
ImageReward
Measures how closely text-to-image outputs align with human preferences using a learned reward model trained on expert comparisons.
- Task count
- 137,000
MultimodalOnline
Kinetics
Evaluates video action recognition across human-object and human-human interactions in short clips.
MultimodalOnline
LongVideoBench
Tests understanding of hour-long videos whose questions require reasoning across interleaved frames, subtitles, and long contexts.
- Task count
- 3,763
MultimodalOnline
MATH-Vision
Tests visual mathematical reasoning on competition problems spanning 16 subjects and five difficulty levels.
- Task count
- 3,040
MultimodalOnline
MathVista
Tests mathematical reasoning grounded in diagrams and other visual contexts across diverse task types.
- Task count
- 6,141
MultimodalOnline
MLVU
Tests long-video understanding through multiple-choice and free-form tasks requiring local and global comprehension.
- Task count
- 9
MultimodalOnline
MM-Vet
Measures how well multimodal models integrate recognition, OCR, knowledge, language, spatial, and mathematical capabilities.
MultimodalOnline
MMBench
Measures broad multimodal model capabilities with multiple-choice questions spanning 20 ability dimensions.
MultimodalOnline
MME
Measures perception and cognition in multimodal language models across 14 manually designed subtasks.
MultimodalOnline
MMMU
Measures expert-level multimodal reasoning across college disciplines, subjects, and heterogeneous image types.
- Task count
- 11,500
MultimodalOnline
MMMU-Pro
Measures robust expert-level multimodal reasoning after filtering text-solvable items and strengthening answer choices.
- Task count
- 1,730
MultimodalOnline
MMStar
Measures vision-essential multimodal capabilities using human-selected samples filtered to resist text-only solving and leakage.
- Task count
- 1,500
Other categories in Online
More benchmarks
Questions
What other kinds of benchmarks are in Online?
Other categories in Online: Reasoning (50 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), Science (24 benchmarks), and Coding (17 benchmarks).
Published by AI Benchmark Index. How we check each benchmark.