# Reasoning benchmarks AI Benchmark Index catalogs 50 reasoning with source-linked metadata. 50 benchmarks. Also on the [web page](https://aibenchmarkindex.com/category/reasoning). ## Benchmarks 1. [AGIEval](https://aibenchmarkindex.com/global/online/agieval.md), Reasoning, Online: Tests general problem solving with questions from 20 public admission, qualification, and competition exams. 2. [AlpacaEval](https://aibenchmarkindex.com/global/online/alpacaeval.md), Reasoning, Online: Measures instruction-following quality by comparing model responses against references with automated preference judges. 3. [ANLI](https://aibenchmarkindex.com/global/online/anli.md), Reasoning, Online: Tests natural-language inference on examples collected adversarially against successive model generations. 4. [ArabicMMLU](https://aibenchmarkindex.com/global/online/arabicmmlu.md), Reasoning, Online: Evaluates Arabic language understanding across 40 school-exam tasks from countries in North Africa, the Levant, and the Gulf. 5. [ARC-AGI-1](https://aibenchmarkindex.com/global/online/arc-agi-1.md), Reasoning, Online: Tests few-shot abstraction by asking systems to infer transformations between small colored grids. 6. [ARC-AGI-2](https://aibenchmarkindex.com/global/online/arc-agi-2.md), Reasoning, Online: Tests symbolic interpretation, compositional reasoning, and context-dependent rules on novel grid tasks. 7. [Arena-Hard-Auto](https://aibenchmarkindex.com/global/online/arena-hard-auto.md), Reasoning, Online: Tests open-ended instruction following on difficult real user prompts with automated preference judges. 8. [BERTScore](https://aibenchmarkindex.com/global/online/bertscore.md), Reasoning, Online: Evaluates generated text by matching contextual token embeddings between candidate and reference sentences. 9. [BIG-bench](https://aibenchmarkindex.com/global/online/big-bench.md), Reasoning, Online: Tests language-model capabilities across more than 200 collaboratively authored tasks. 10. [BIG-Bench Hard](https://aibenchmarkindex.com/global/online/big-bench-hard.md), Reasoning, Online: Tests multi-step reasoning on 23 BIG-bench tasks where early language models trailed average human raters. 11. [BoolQ](https://aibenchmarkindex.com/global/online/boolq.md), Reasoning, Online: Tests yes-or-no question answering from a supporting passage using naturally occurring information-seeking questions. 12. [C-Eval](https://aibenchmarkindex.com/global/online/c-eval.md), Reasoning, Online: Evaluates Chinese foundation models with multiple-choice questions spanning 52 disciplines and four difficulty levels. 13. [CMMLU](https://aibenchmarkindex.com/global/online/cmmlu.md), Reasoning, Online: Evaluates Chinese language and cultural knowledge across 67 multiple-choice subjects. 14. [FinBen](https://aibenchmarkindex.com/global/online/finben.md), Reasoning, Online: Evaluates financial language understanding and prediction across 24 tasks, including extraction, forecasting, risk, and decision making. 15. [FinQA](https://aibenchmarkindex.com/global/online/finqa.md), Reasoning, Online: Evaluates numerical reasoning over tables and text from financial reports by requiring executable answer programs. 16. [FLORES-200](https://aibenchmarkindex.com/global/online/flores-200.md), Reasoning, Online: Evaluates machine translation across 200 languages with a professionally translated parallel corpus. 17. [FollowBench](https://aibenchmarkindex.com/global/online/followbench.md), Reasoning, Online: Tests adherence to increasingly complex content, situation, style, format, and example constraints. 18. [GLUE](https://aibenchmarkindex.com/global/online/glue.md), Reasoning, Online: Combines nine sentence- and sentence-pair tasks to evaluate general-purpose English language understanding. 19. [HaluEval](https://aibenchmarkindex.com/global/online/halueval.md), Reasoning, Online: Evaluates whether language models recognize hallucinated content in general queries, question answering, dialogue, and summarization. 20. [HELM](https://aibenchmarkindex.com/global/online/helm.md), Reasoning, Online: Evaluates foundation models across standardized scenarios with multiple accuracy, robustness, fairness, and efficiency metrics. 21. [HotpotQA](https://aibenchmarkindex.com/global/online/hotpotqa.md), Reasoning, Online: Tests explainable multi-hop question answering with supporting facts drawn from multiple Wikipedia articles. 22. [Humanity’s Last Exam](https://aibenchmarkindex.com/global/online/humanitys-last-exam.md), Reasoning, Online: Tests frontier models on expert-written, multimodal questions spanning mathematics, science, and the humanities. 23. [IFEval](https://aibenchmarkindex.com/global/online/ifeval.md), Reasoning, Online: Tests whether a model follows verifiable instructions such as format, length, and keyword constraints. 24. [LAMBADA](https://aibenchmarkindex.com/global/online/lambada.md), Reasoning, Online: Tests prediction of a passage’s final word when success requires understanding its broader discourse context. 25. [LegalBench](https://aibenchmarkindex.com/global/online/legalbench.md), Reasoning, Online: Evaluates legal reasoning across collaboratively contributed tasks covering varied legal domains and text formats. 26. [LexGLUE](https://aibenchmarkindex.com/global/online/lexglue.md), Reasoning, Online: Evaluates legal language understanding across seven English classification datasets using a common evaluation setup. 27. [LiveBench](https://aibenchmarkindex.com/global/online/livebench.md), Reasoning, Online: Tests six capability areas with regularly refreshed questions and objective, non-judge scoring. 28. [MMLU](https://aibenchmarkindex.com/global/online/mmlu.md), Reasoning, Online: Tests broad academic knowledge and reasoning with multiple-choice questions drawn from 57 subjects. 29. [MMLU-Pro](https://aibenchmarkindex.com/global/online/mmlu-pro-2.md), Reasoning, Online: Evaluates broad knowledge and reasoning with harder multiple-choice questions, expanded answer choices, and reduced ambiguity relative to MMLU. 30. [MT-Bench](https://aibenchmarkindex.com/global/online/mt-bench.md), Reasoning, Online: Tests conversational quality on challenging multi-turn, open-ended questions scored by model judges. 31. [Natural Questions](https://aibenchmarkindex.com/global/online/natural-questions-2.md), Reasoning, Online: Tests question answering on real Google queries paired with Wikipedia pages and short or long answer annotations. 32. [PokerBench](https://aibenchmarkindex.com/global/online/pokerbench.md), Reasoning, Online: Evaluates whether language models choose solver-backed actions in 11,000 pre-flop and post-flop poker scenarios. 33. [TAT-QA](https://aibenchmarkindex.com/global/online/tat-qa.md), Reasoning, Online: Evaluates numerical question answering over combined tables and prose from financial reports. 34. [TriviaQA](https://aibenchmarkindex.com/global/online/triviaqa.md), Reasoning, Online: Tests reading comprehension and open-domain question answering with trivia questions paired to independently gathered evidence. 35. [TruthfulQA](https://aibenchmarkindex.com/global/online/truthfulqa.md), Reasoning, Online: Tests whether language models reproduce common human misconceptions and falsehoods when answering questions. 36. [TyDi QA](https://aibenchmarkindex.com/global/online/tydi-qa.md), Reasoning, Online: Evaluates information-seeking question answering across 11 typologically diverse languages. 37. [WikiText](https://aibenchmarkindex.com/global/online/wikitext.md), Reasoning, Online: Evaluates language modeling on more than 100 million tokens drawn from verified Wikipedia articles. 38. [WildBench](https://aibenchmarkindex.com/global/online/wildbench.md), Reasoning, Online: Tests chat models on challenging real-user requests using task-specific checklists and model-based judges. 39. [WinoGrande](https://aibenchmarkindex.com/global/online/winogrande.md), Reasoning, Online: Tests commonsense reasoning by resolving pronouns in adversarially collected sentence pairs. 40. [XNLI](https://aibenchmarkindex.com/global/online/xnli.md), Reasoning, Online: Evaluates cross-lingual natural-language inference and language transfer across 15 languages. 41. [XQuAD](https://aibenchmarkindex.com/global/online/xquad.md), Reasoning, Online: Evaluates cross-lingual extractive question answering on the same question-answer pairs translated across 11 languages. 42. [Gym Sokoban](https://aibenchmarkindex.com/global/online/gym-sokoban.md), Reasoning, Online: Evaluates planning and reinforcement learning in procedurally generated box-pushing puzzles. 43. [HELM Lite](https://aibenchmarkindex.com/global/online/helm-lite.md), Reasoning, Online: Evaluates language models on a compact suite of ten HELM scenarios selected for broad coverage and lower evaluation cost. 44. [HumorBench](https://aibenchmarkindex.com/global/online/humorbench.md), Reasoning, Online: Evaluates English and Spanish multi-turn humor understanding through joke classification and identification of each line’s role. 45. [MMLU-Pro](https://aibenchmarkindex.com/global/online/mmlu-pro.md), Reasoning, Online: Tests knowledge and reasoning on harder ten-option questions with reduced reliance on recall. 46. [SimpleBench](https://aibenchmarkindex.com/global/online/simplebench.md), Reasoning, Online: Tests everyday reasoning about space, time, social situations, and adversarial wording with multiple-choice questions. 47. [SQuAD 2.0](https://aibenchmarkindex.com/global/online/squad-2-0.md), Reasoning, Online: Tests extractive reading comprehension and whether a system abstains when a Wikipedia passage does not answer the question. 48. [TREC Question Answering Track](https://aibenchmarkindex.com/global/online/trec-question-answering-track.md), Reasoning, Online: Evaluates systems that return concise answers to fact-based questions from large text collections. 49. [Open LLM Leaderboard](https://aibenchmarkindex.com/global/online/open-llm-leaderboard.md), Reasoning, Online: Tracks and compares openly submitted language models across a changing suite of reasoning and knowledge evaluations. 50. [Scale SEAL Leaderboards](https://aibenchmarkindex.com/global/online/scale-seal-leaderboards.md), Reasoning, Online: Compares frontier models on expert-designed evaluations built from private datasets intended to resist benchmark gaming. ## Reasoning by city - [Online, Global](https://aibenchmarkindex.com/global/online/category/reasoning.md): 50 benchmarks ## Related categories - [Multimodal](https://aibenchmarkindex.com/category/multimodal.md): 51 benchmarks - [Agents](https://aibenchmarkindex.com/category/agents.md): 31 benchmarks - [Safety](https://aibenchmarkindex.com/category/safety.md): 25 benchmarks - [Science](https://aibenchmarkindex.com/category/science.md): 24 benchmarks - [Coding](https://aibenchmarkindex.com/category/coding.md): 17 benchmarks - [Retrieval](https://aibenchmarkindex.com/category/retrieval.md): 17 benchmarks - [Memory](https://aibenchmarkindex.com/category/memory.md): 10 benchmarks - [Browser](https://aibenchmarkindex.com/category/browser.md): 7 benchmarks - [Voice](https://aibenchmarkindex.com/category/voice.md): 7 benchmarks - [Math](https://aibenchmarkindex.com/category/math.md): 6 benchmarks - [Other](https://aibenchmarkindex.com/category/other.md): 6 benchmarks - [Tools](https://aibenchmarkindex.com/category/tools.md): 6 benchmarks - [Commerce](https://aibenchmarkindex.com/category/commerce.md): 4 benchmarks - [Long-horizon](https://aibenchmarkindex.com/category/long-horizon.md): 3 benchmarks > **Check before you compare.** Benchmark methods and leaderboards change. Follow the cited primary source before using a result in a decision.