# AI benchmark list There are 264 source-checked AI benchmarks in this index. 264 benchmarks. Also on the [web page](https://aibenchmarkindex.com/global/online). ## Benchmarks 1. [AdvBench](https://aibenchmarkindex.com/global/online/advbench.md), Safety, Online: Measures whether adversarial suffixes induce aligned language models to produce targeted harmful behavior. 2. [AgentBench](https://aibenchmarkindex.com/global/online/agentbench.md), Agents, Online: Measures language-model agents across operating systems, databases, games, web shopping, and other interactive environments. 3. [AgentDojo](https://aibenchmarkindex.com/global/online/agentdojo.md), Safety, Online: Evaluates agent task utility and resistance to prompt injection while agents use tools over untrusted data. 4. [AGIEval](https://aibenchmarkindex.com/global/online/agieval.md), Reasoning, Online: Tests general problem solving with questions from 20 public admission, qualification, and competition exams. 5. [AI2-THOR](https://aibenchmarkindex.com/global/online/ai2-thor.md), Agents, Online: Evaluates embodied agents in interactive simulated indoor scenes with object manipulation and navigation. 6. [ALFWorld](https://aibenchmarkindex.com/global/online/alfworld.md), Agents, Online: Measures agents that learn high-level policies in text environments and execute the same household goals in embodied worlds. 7. [AlpacaEval](https://aibenchmarkindex.com/global/online/alpacaeval.md), Reasoning, Online: Measures instruction-following quality by comparing model responses against references with automated preference judges. 8. [AndroidWorld](https://aibenchmarkindex.com/global/online/androidworld.md), Agents, Online: Measures autonomous agents on dynamically generated tasks across real Android applications. 9. [ANLI](https://aibenchmarkindex.com/global/online/anli.md), Reasoning, Online: Tests natural-language inference on examples collected adversarially against successive model generations. 10. [APPS](https://aibenchmarkindex.com/global/online/apps.md), Coding, Online: Tests coding-challenge competence on problems ranging from introductory exercises to programming competitions. 11. [AppWorld](https://aibenchmarkindex.com/global/online/appworld.md), Tools, Online: Measures interactive coding agents on tasks spanning simulated everyday apps and their APIs. 12. [ArabicMMLU](https://aibenchmarkindex.com/global/online/arabicmmlu.md), Reasoning, Online: Evaluates Arabic language understanding across 40 school-exam tasks from countries in North Africa, the Levant, and the Gulf. 13. [ARC-AGI-1](https://aibenchmarkindex.com/global/online/arc-agi-1.md), Reasoning, Online: Tests few-shot abstraction by asking systems to infer transformations between small colored grids. 14. [ARC-AGI-2](https://aibenchmarkindex.com/global/online/arc-agi-2.md), Reasoning, Online: Tests symbolic interpretation, compositional reasoning, and context-dependent rules on novel grid tasks. 15. [Arena-Hard-Auto](https://aibenchmarkindex.com/global/online/arena-hard-auto.md), Reasoning, Online: Tests open-ended instruction following on difficult real user prompts with automated preference judges. 16. [ASCIIBench](https://aibenchmarkindex.com/global/online/asciibench.md), Multimodal, Online: Measures four-choice recognition of ASCII art from text-only, rendered-image, and combined inputs. 17. [BabyAI GoTo](https://aibenchmarkindex.com/global/online/babyai-goto.md), Agents, Online: Tests grounded language understanding by asking an agent to navigate a grid world to a described object. 18. [BBQ](https://aibenchmarkindex.com/global/online/bbq.md), Safety, Online: Measures social bias in question answering across ambiguous and disambiguated contexts covering nine protected dimensions. 19. [BBQ-Lite](https://aibenchmarkindex.com/global/online/bbq-lite.md), Safety, Online: Provides a compact BIG-bench version of BBQ for exact-match evaluation of bias-sensitive question answering. 20. [BeaverTails](https://aibenchmarkindex.com/global/online/beavertails.md), Safety, Online: Supports safety classification and preference evaluation with human-labeled question-answer pairs across harm categories. 21. [BEIR](https://aibenchmarkindex.com/global/online/beir.md), Retrieval, Online: Evaluates zero-shot information retrieval across heterogeneous datasets, domains, and retrieval architectures. 22. [BERTScore](https://aibenchmarkindex.com/global/online/bertscore.md), Reasoning, Online: Evaluates generated text by matching contextual token embeddings between candidate and reference sentences. 23. [BIG-bench](https://aibenchmarkindex.com/global/online/big-bench.md), Reasoning, Online: Tests language-model capabilities across more than 200 collaboratively authored tasks. 24. [BIG-Bench Hard](https://aibenchmarkindex.com/global/online/big-bench-hard.md), Reasoning, Online: Tests multi-step reasoning on 23 BIG-bench tasks where early language models trailed average human raters. 25. [BigCodeBench](https://aibenchmarkindex.com/global/online/bigcodebench.md), Coding, Online: Tests function-level code generation on software-engineering tasks requiring diverse library calls. 26. [BIPIA](https://aibenchmarkindex.com/global/online/bipia.md), Safety, Online: Evaluates model and defense robustness against indirect prompt injection across five application tasks. 27. [BIRD-SQL](https://aibenchmarkindex.com/global/online/bird-sql.md), Tools, Online: Tests text-to-SQL generation against large database contents spanning many professional domains. 28. [BLINK](https://aibenchmarkindex.com/global/online/blink.md), Multimodal, Online: Measures core visual perception abilities through multiple-choice tasks designed to resist verbal shortcuts. 29. [BOLD](https://aibenchmarkindex.com/global/online/bold.md), Safety, Online: Measures social biases in open-ended English generation from prompts spanning profession, gender, race, religion, and politics. 30. [BoolQ](https://aibenchmarkindex.com/global/online/boolq.md), Reasoning, Online: Tests yes-or-no question answering from a supporting passage using naturally occurring information-seeking questions. 31. [BrowserGym](https://aibenchmarkindex.com/global/online/browsergym.md), Browser, Online: Provides a common environment for evaluating browser agents across several web-task benchmarks. 32. [C-Eval](https://aibenchmarkindex.com/global/online/c-eval.md), Reasoning, Online: Evaluates Chinese foundation models with multiple-choice questions spanning 52 disciplines and four difficulty levels. 33. [CALVIN](https://aibenchmarkindex.com/global/online/calvin.md), Long-horizon, Online: Measures long-horizon, language-conditioned robot manipulation through sequences of continuous-control tasks. 34. [ChartQA](https://aibenchmarkindex.com/global/online/chartqa.md), Multimodal, Online: Measures visual and logical reasoning over charts with human-authored and generated questions. 35. [Cicero Diplomacy](https://aibenchmarkindex.com/global/online/cicero-diplomacy.md), Agents, Online: Evaluates strategic planning, negotiation, and natural-language coordination by agents playing the board game Diplomacy. 36. [CLIPScore](https://aibenchmarkindex.com/global/online/clipscore.md), Multimodal, Online: Measures image-caption compatibility without requiring reference captions by comparing CLIP embeddings. 37. [ClockBench](https://aibenchmarkindex.com/global/online/clockbench.md), Multimodal, Online: Evaluates whether vision-language models can read analog clocks across varied faces and times. 38. [CMMLU](https://aibenchmarkindex.com/global/online/cmmlu.md), Reasoning, Online: Evaluates Chinese language and cultural knowledge across 67 multiple-choice subjects. 39. [CodeContests](https://aibenchmarkindex.com/global/online/codecontests.md), Coding, Online: Tests competitive-programming solution generation with problems, descriptions, tests, and human submissions. 40. [CodeXGLUE](https://aibenchmarkindex.com/global/online/codexglue.md), Coding, Online: Tests code understanding and generation across ten tasks including search, repair, translation, and summarization. 41. [CogVideo](https://aibenchmarkindex.com/global/online/cogvideo.md), Multimodal, Online: Evaluates text-to-video generation; the repository provides model code and references for the original CogVideo and later CogVideoX systems. 42. [Crafter](https://aibenchmarkindex.com/global/online/crafter.md), Agents, Online: Measures reinforcement-learning agents across a survival game designed to expose a broad spectrum of capabilities. 43. [Crafter](https://aibenchmarkindex.com/global/online/crafter-2.md), Agents, Online: Evaluates reinforcement-learning agents on survival, exploration, resource collection, and technology-tree achievements. 44. [CRMArena](https://aibenchmarkindex.com/global/online/crmarena.md), Agents, Online: Measures agents on professional customer-relationship-management tasks in realistic business scenarios. 45. [CrossCodeEval](https://aibenchmarkindex.com/global/online/crosscodeeval.md), Coding, Online: Tests cross-file code completion and context retrieval in Python, Java, TypeScript, and C\# repositories. 46. [CrowS-Pairs](https://aibenchmarkindex.com/global/online/crows-pairs.md), Safety, Online: Measures stereotypical preferences in masked language models using paired sentences across nine types of social bias. 47. [CRUD-RAG](https://aibenchmarkindex.com/global/online/crud-rag.md), Retrieval, Online: Evaluates retrieval-augmented generation across create, read, update, and delete application scenarios. 48. [CRUXEval](https://aibenchmarkindex.com/global/online/cruxeval.md), Coding, Online: Tests code reasoning by asking models to predict inputs or outputs for short Python functions. 49. [CyberSecEval](https://aibenchmarkindex.com/global/online/cyberseceval.md), Safety, Online: Evaluates cybersecurity risks from model-generated code and responses to requests that could enable cyberattacks. 50. [EmbodiedQA](https://aibenchmarkindex.com/global/online/embodiedqa.md), Agents, Online: Tests agents that navigate a simulated home to gather visual evidence and answer a question. This copy lists 50 of 264. The [web page](https://aibenchmarkindex.com/global/online) has the rest. ## Categories in Online - [Multimodal](https://aibenchmarkindex.com/global/online/category/multimodal.md): 51 benchmarks - [Reasoning](https://aibenchmarkindex.com/global/online/category/reasoning.md): 50 benchmarks - [Agents](https://aibenchmarkindex.com/global/online/category/agents.md): 31 benchmarks - [Safety](https://aibenchmarkindex.com/global/online/category/safety.md): 25 benchmarks - [Science](https://aibenchmarkindex.com/global/online/category/science.md): 24 benchmarks - [Coding](https://aibenchmarkindex.com/global/online/category/coding.md): 17 benchmarks - [Retrieval](https://aibenchmarkindex.com/global/online/category/retrieval.md): 17 benchmarks - [Memory](https://aibenchmarkindex.com/global/online/category/memory.md): 10 benchmarks - [Browser](https://aibenchmarkindex.com/global/online/category/browser.md): 7 benchmarks - [Voice](https://aibenchmarkindex.com/global/online/category/voice.md): 7 benchmarks - [Math](https://aibenchmarkindex.com/global/online/category/math.md): 6 benchmarks - [Other](https://aibenchmarkindex.com/global/online/category/other.md): 6 benchmarks - [Tools](https://aibenchmarkindex.com/global/online/category/tools.md): 6 benchmarks - [Commerce](https://aibenchmarkindex.com/global/online/category/commerce.md): 4 benchmarks - [Long-horizon](https://aibenchmarkindex.com/global/online/category/long-horizon.md): 3 benchmarks ## Questions ### How many AI benchmarks are in the index? AI Benchmark Index currently lists 264 benchmarks that link to a primary source. ### Can I submit an AI benchmark? Yes. Submit a public primary source, licence, release date, category, and a one-sentence description of what it measures. > **Check before you compare.** Benchmark methods and leaderboards change. Follow the cited primary source before using a result in a decision.