ReasoningOnline
LegalBench
Evaluates legal reasoning across collaboratively contributed tasks covering varied legal domains and text formats.
- Task count
- 162
Reasoning benchmarks
There are 50 reasoning in the index.
Results
50 benchmarks
ReasoningOnline
LexGLUE
Evaluates legal language understanding across seven English classification datasets using a common evaluation setup.
- Task count
- 7
ReasoningOnline
LiveBench
Tests six capability areas with regularly refreshed questions and objective, non-judge scoring.
- Task count
- 18
ReasoningOnline
MMLU
Tests broad academic knowledge and reasoning with multiple-choice questions drawn from 57 subjects.
- Task count
- 15,908
ReasoningOnline
MMLU-Pro
Evaluates broad knowledge and reasoning with harder multiple-choice questions, expanded answer choices, and reduced ambiguity relative to MMLU.
- Task count
- 12,032
ReasoningOnline
MT-Bench
Tests conversational quality on challenging multi-turn, open-ended questions scored by model judges.
- Task count
- 80
ReasoningOnline
Natural Questions
Tests question answering on real Google queries paired with Wikipedia pages and short or long answer annotations.
- Task count
- 323,044
ReasoningOnline
PokerBench
Evaluates whether language models choose solver-backed actions in 11,000 pre-flop and post-flop poker scenarios.
- Task count
- 11,000
ReasoningOnline
TAT-QA
Evaluates numerical question answering over combined tables and prose from financial reports.
- Task count
- 16,552
ReasoningOnline
TriviaQA
Tests reading comprehension and open-domain question answering with trivia questions paired to independently gathered evidence.
ReasoningOnline
TruthfulQA
Tests whether language models reproduce common human misconceptions and falsehoods when answering questions.
- Task count
- 817
ReasoningOnline
TyDi QA
Evaluates information-seeking question answering across 11 typologically diverse languages.
- Task count
- 204,000
ReasoningOnline
WikiText
Evaluates language modeling on more than 100 million tokens drawn from verified Wikipedia articles.
ReasoningOnline
WildBench
Tests chat models on challenging real-user requests using task-specific checklists and model-based judges.
- Task count
- 1,024
ReasoningOnline
WinoGrande
Tests commonsense reasoning by resolving pronouns in adversarially collected sentence pairs.
- Task count
- 44,000
ReasoningOnline
XNLI
Evaluates cross-lingual natural-language inference and language transfer across 15 languages.
- Task count
- 112,500
ReasoningOnline
XQuAD
Evaluates cross-lingual extractive question answering on the same question-answer pairs translated across 11 languages.
- Task count
- 1,190
ReasoningOnline
Gym Sokoban
Evaluates planning and reinforcement learning in procedurally generated box-pushing puzzles.
ReasoningOnline
HELM Lite
Evaluates language models on a compact suite of ten HELM scenarios selected for broad coverage and lower evaluation cost.
- Task count
- 10
ReasoningOnline
HumorBench
Evaluates English and Spanish multi-turn humor understanding through joke classification and identification of each line’s role.
ReasoningOnline
MMLU-Pro
Tests knowledge and reasoning on harder ten-option questions with reduced reliance on recall.
- Task count
- 12,032
ReasoningOnline
SimpleBench
Tests everyday reasoning about space, time, social situations, and adversarial wording with multiple-choice questions.
ReasoningOnline
SQuAD 2.0
Tests extractive reading comprehension and whether a system abstains when a Wikipedia passage does not answer the question.
ReasoningOnline
TREC Question Answering Track
Evaluates systems that return concise answers to fact-based questions from large text collections.
Other categories in Online
More benchmarks
Questions
What other kinds of benchmarks are in Online?
Other categories in Online: Multimodal (51 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), Science (24 benchmarks), and Coding (17 benchmarks).
Published by AI Benchmark Index. How we check each benchmark.