# Multimodal benchmarks AI Benchmark Index catalogs 51 multimodal with source-linked metadata. 51 benchmarks. Also on the [web page](https://aibenchmarkindex.com/category/multimodal). ## Benchmarks 1. [ASCIIBench](https://aibenchmarkindex.com/global/online/asciibench.md), Multimodal, Online: Measures four-choice recognition of ASCII art from text-only, rendered-image, and combined inputs. 2. [BLINK](https://aibenchmarkindex.com/global/online/blink.md), Multimodal, Online: Measures core visual perception abilities through multiple-choice tasks designed to resist verbal shortcuts. 3. [ChartQA](https://aibenchmarkindex.com/global/online/chartqa.md), Multimodal, Online: Measures visual and logical reasoning over charts with human-authored and generated questions. 4. [CLIPScore](https://aibenchmarkindex.com/global/online/clipscore.md), Multimodal, Online: Measures image-caption compatibility without requiring reference captions by comparing CLIP embeddings. 5. [ClockBench](https://aibenchmarkindex.com/global/online/clockbench.md), Multimodal, Online: Evaluates whether vision-language models can read analog clocks across varied faces and times. 6. [CogVideo](https://aibenchmarkindex.com/global/online/cogvideo.md), Multimodal, Online: Evaluates text-to-video generation; the repository provides model code and references for the original CogVideo and later CogVideoX systems. 7. [Fréchet Video Distance](https://aibenchmarkindex.com/global/online/frechet-video-distance.md), Multimodal, Online: Measures the distributional similarity of real and generated videos using learned video embeddings. 8. [HPS v2](https://aibenchmarkindex.com/global/online/hps-v2.md), Multimodal, Online: Evaluates whether a text-to-image model produces images people prefer for the same prompt. 9. [ImageNet-A](https://aibenchmarkindex.com/global/online/imagenet-a.md), Multimodal, Online: Tests image classifiers on naturally occurring, unmodified images selected because standard ImageNet models misclassify them. 10. [ImageNet-R](https://aibenchmarkindex.com/global/online/imagenet-r.md), Multimodal, Online: Tests ImageNet classifiers on artistic and other non-photographic renditions of 200 object classes. 11. [ImageNet-V2](https://aibenchmarkindex.com/global/online/imagenet-v2.md), Multimodal, Online: Retests ImageNet classifiers on three independently collected 10,000-image test sets matched to the original distribution. 12. [ImagenHub](https://aibenchmarkindex.com/global/online/imagenhub.md), Multimodal, Online: Compares conditional image generation and editing systems across standardized tasks, metrics, and human ratings. 13. [ImageReward](https://aibenchmarkindex.com/global/online/imagereward.md), Multimodal, Online: Measures how closely text-to-image outputs align with human preferences using a learned reward model trained on expert comparisons. 14. [Kinetics](https://aibenchmarkindex.com/global/online/kinetics.md), Multimodal, Online: Evaluates video action recognition across human-object and human-human interactions in short clips. 15. [LongVideoBench](https://aibenchmarkindex.com/global/online/longvideobench.md), Multimodal, Online: Tests understanding of hour-long videos whose questions require reasoning across interleaved frames, subtitles, and long contexts. 16. [MATH-Vision](https://aibenchmarkindex.com/global/online/math-vision.md), Multimodal, Online: Tests visual mathematical reasoning on competition problems spanning 16 subjects and five difficulty levels. 17. [MathVista](https://aibenchmarkindex.com/global/online/mathvista.md), Multimodal, Online: Tests mathematical reasoning grounded in diagrams and other visual contexts across diverse task types. 18. [MLVU](https://aibenchmarkindex.com/global/online/mlvu.md), Multimodal, Online: Tests long-video understanding through multiple-choice and free-form tasks requiring local and global comprehension. 19. [MM-Vet](https://aibenchmarkindex.com/global/online/mm-vet.md), Multimodal, Online: Measures how well multimodal models integrate recognition, OCR, knowledge, language, spatial, and mathematical capabilities. 20. [MMBench](https://aibenchmarkindex.com/global/online/mmbench.md), Multimodal, Online: Measures broad multimodal model capabilities with multiple-choice questions spanning 20 ability dimensions. 21. [MME](https://aibenchmarkindex.com/global/online/mme.md), Multimodal, Online: Measures perception and cognition in multimodal language models across 14 manually designed subtasks. 22. [MMMU](https://aibenchmarkindex.com/global/online/mmmu.md), Multimodal, Online: Measures expert-level multimodal reasoning across college disciplines, subjects, and heterogeneous image types. 23. [MMMU-Pro](https://aibenchmarkindex.com/global/online/mmmu-pro.md), Multimodal, Online: Measures robust expert-level multimodal reasoning after filtering text-solvable items and strengthening answer choices. 24. [MMStar](https://aibenchmarkindex.com/global/online/mmstar.md), Multimodal, Online: Measures vision-essential multimodal capabilities using human-selected samples filtered to resist text-only solving and leakage. 25. [MVBench](https://aibenchmarkindex.com/global/online/mvbench.md), Multimodal, Online: Measures multi-modal video understanding across 20 temporal and perceptual tasks. 26. [NExT-QA](https://aibenchmarkindex.com/global/online/next-qa.md), Multimodal, Online: Evaluates causal and temporal reasoning in video question answering rather than simple visual description. 27. [OCRBench](https://aibenchmarkindex.com/global/online/ocrbench.md), Multimodal, Online: Measures OCR in multimodal models across text recognition, scene-text VQA, document VQA, extraction, and handwritten math. 28. [OCRBench v2](https://aibenchmarkindex.com/global/online/ocrbench-v2.md), Multimodal, Online: Measures bilingual visual-text localization, recognition, extraction, and reasoning across diverse real-world scenarios. 29. [PartiPrompts](https://aibenchmarkindex.com/global/online/partiprompts.md), Multimodal, Online: Tests text-to-image systems with over 1,600 prompts spanning simple scenes, complex compositions, writing, and abstract concepts. 30. [SEED-Bench](https://aibenchmarkindex.com/global/online/seed-bench.md), Multimodal, Online: Measures image and video comprehension across multimodal ability dimensions using human-annotated multiple-choice questions. 31. [TasteBench](https://aibenchmarkindex.com/global/online/tastebench.md), Multimodal, Online: Studies whether aesthetic-preference benchmarks reward expert-aligned craft judgment rather than shortcuts; its item pool remains pending licence clearance. 32. [Video-Bench](https://aibenchmarkindex.com/global/online/video-bench.md), Multimodal, Online: Evaluates video language models on perception, prior-knowledge use, and video-based decision making. 33. [Video-MME](https://aibenchmarkindex.com/global/online/video-mme.md), Multimodal, Online: Evaluates multimodal models on short, medium, and long videos across visual domains, durations, and subtitle settings. 34. [Video-MMMU](https://aibenchmarkindex.com/global/online/video-mmmu.md), Multimodal, Online: Measures whether multimodal models acquire and apply new knowledge from professional lecture videos. 35. [We-Math](https://aibenchmarkindex.com/global/online/we-math.md), Multimodal, Online: Tests whether multimodal models solve visual math problems through coherent one-step and multi-step reasoning. 36. [ActivityNet-QA](https://aibenchmarkindex.com/global/online/activitynet-qa.md), Multimodal, Online: Tests question answering over long videos with questions that require spatial and temporal reasoning. 37. [AI2D](https://aibenchmarkindex.com/global/online/ai2d.md), Multimodal, Online: Measures parsing and question answering over grade-school science diagrams with annotated constituents and relationships. 38. [AVA](https://aibenchmarkindex.com/global/online/ava.md), Multimodal, Online: Evaluates spatiotemporally localized recognition of 80 atomic actions in densely annotated movie clips. 39. [CIFAR-10](https://aibenchmarkindex.com/global/online/cifar-10.md), Multimodal, Online: Evaluates image classification on 60,000 small color images evenly divided among ten classes. 40. [COCO Caption](https://aibenchmarkindex.com/global/online/coco-caption.md), Multimodal, Online: Evaluates image-caption generation on everyday scenes using reference captions and standardized captioning metrics. 41. [DocVQA](https://aibenchmarkindex.com/global/online/docvqa.md), Multimodal, Online: Measures question answering over document images where layout, text, and document structure matter. 42. [InfographicVQA](https://aibenchmarkindex.com/global/online/infographicvqa.md), Multimodal, Online: Measures question answering that combines infographic text, layout, graphics, data visualizations, and arithmetic. 43. [OK-VQA](https://aibenchmarkindex.com/global/online/ok-vqa.md), Multimodal, Online: Measures visual question answering that requires knowledge beyond the content visible in an image. 44. [Something-Something V2](https://aibenchmarkindex.com/global/online/something-something-v2.md), Multimodal, Online: Tests recognition of everyday actions whose labels depend on temporal and physical interactions between people and objects. 45. [SVGBench](https://aibenchmarkindex.com/global/online/svgbench.md), Multimodal, Online: Tests whether language models can generate SVGs that satisfy requirements involving knowledge, code, instructions, and physical reasoning. 46. [TextVQA](https://aibenchmarkindex.com/global/online/textvqa.md), Multimodal, Online: Measures visual question answering that requires reading and reasoning about text in natural images. 47. [VizWiz VQA](https://aibenchmarkindex.com/global/online/vizwiz-vqa.md), Multimodal, Online: Measures answers to visual questions submitted by blind people in real accessibility scenarios. 48. [VQAv2](https://aibenchmarkindex.com/global/online/vqav2.md), Multimodal, Online: Measures visual question answering while reducing language-only shortcuts through complementary image pairs. 49. [ImageNet](https://aibenchmarkindex.com/global/online/imagenet.md), Multimodal, Online: Evaluates object recognition using images organized under the WordNet noun hierarchy. 50. [ObjectNet](https://aibenchmarkindex.com/global/online/objectnet.md), Multimodal, Online: Tests object recognition on 50,000 crowd-sourced images captured with controls over backgrounds, rotations, and viewpoints. This copy lists 50 of 51. The [web page](https://aibenchmarkindex.com/category/multimodal) has the rest. ## Multimodal by city - [Online, Global](https://aibenchmarkindex.com/global/online/category/multimodal.md): 51 benchmarks ## Related categories - [Reasoning](https://aibenchmarkindex.com/category/reasoning.md): 50 benchmarks - [Agents](https://aibenchmarkindex.com/category/agents.md): 31 benchmarks - [Safety](https://aibenchmarkindex.com/category/safety.md): 25 benchmarks - [Science](https://aibenchmarkindex.com/category/science.md): 24 benchmarks - [Coding](https://aibenchmarkindex.com/category/coding.md): 17 benchmarks - [Retrieval](https://aibenchmarkindex.com/category/retrieval.md): 17 benchmarks - [Memory](https://aibenchmarkindex.com/category/memory.md): 10 benchmarks - [Browser](https://aibenchmarkindex.com/category/browser.md): 7 benchmarks - [Voice](https://aibenchmarkindex.com/category/voice.md): 7 benchmarks - [Math](https://aibenchmarkindex.com/category/math.md): 6 benchmarks - [Other](https://aibenchmarkindex.com/category/other.md): 6 benchmarks - [Tools](https://aibenchmarkindex.com/category/tools.md): 6 benchmarks - [Commerce](https://aibenchmarkindex.com/category/commerce.md): 4 benchmarks - [Long-horizon](https://aibenchmarkindex.com/category/long-horizon.md): 3 benchmarks > **Check before you compare.** Benchmark methods and leaderboards change. Follow the cited primary source before using a result in a decision.