ReasoningOnline
AGIEval
Tests general problem solving with questions from 20 public admission, qualification, and competition exams.
- Task count
- 20
Reasoning benchmarks
There are 50 reasoning in the index.
Results
50 benchmarks
ReasoningOnline
AlpacaEval
Measures instruction-following quality by comparing model responses against references with automated preference judges.
- Task count
- 805
ReasoningOnline
ANLI
Tests natural-language inference on examples collected adversarially against successive model generations.
ReasoningOnline
ArabicMMLU
Evaluates Arabic language understanding across 40 school-exam tasks from countries in North Africa, the Levant, and the Gulf.
- Task count
- 14,575
ReasoningOnline
ARC-AGI-1
Tests few-shot abstraction by asking systems to infer transformations between small colored grids.
- Task count
- 800
ReasoningOnline
ARC-AGI-2
Tests symbolic interpretation, compositional reasoning, and context-dependent rules on novel grid tasks.
- Task count
- 1,360
ReasoningOnline
Arena-Hard-Auto
Tests open-ended instruction following on difficult real user prompts with automated preference judges.
- Task count
- 750
ReasoningOnline
BERTScore
Evaluates generated text by matching contextual token embeddings between candidate and reference sentences.
ReasoningOnline
BIG-bench
Tests language-model capabilities across more than 200 collaboratively authored tasks.
- Task count
- 200
ReasoningOnline
BIG-Bench Hard
Tests multi-step reasoning on 23 BIG-bench tasks where early language models trailed average human raters.
- Task count
- 23
ReasoningOnline
BoolQ
Tests yes-or-no question answering from a supporting passage using naturally occurring information-seeking questions.
- Task count
- 15,942
ReasoningOnline
C-Eval
Evaluates Chinese foundation models with multiple-choice questions spanning 52 disciplines and four difficulty levels.
- Task count
- 13,948
ReasoningOnline
CMMLU
Evaluates Chinese language and cultural knowledge across 67 multiple-choice subjects.
ReasoningOnline
FinBen
Evaluates financial language understanding and prediction across 24 tasks, including extraction, forecasting, risk, and decision making.
- Task count
- 24
ReasoningOnline
FinQA
Evaluates numerical reasoning over tables and text from financial reports by requiring executable answer programs.
- Task count
- 8,281
ReasoningOnline
FLORES-200
Evaluates machine translation across 200 languages with a professionally translated parallel corpus.
- Task count
- 3,001
ReasoningOnline
FollowBench
Tests adherence to increasingly complex content, situation, style, format, and example constraints.
- Task count
- 820
ReasoningOnline
GLUE
Combines nine sentence- and sentence-pair tasks to evaluate general-purpose English language understanding.
- Task count
- 9
ReasoningOnline
HaluEval
Evaluates whether language models recognize hallucinated content in general queries, question answering, dialogue, and summarization.
- Task count
- 35,000
ReasoningOnline
HELM
Evaluates foundation models across standardized scenarios with multiple accuracy, robustness, fairness, and efficiency metrics.
ReasoningOnline
HotpotQA
Tests explainable multi-hop question answering with supporting facts drawn from multiple Wikipedia articles.
ReasoningOnline
Humanity’s Last Exam
Tests frontier models on expert-written, multimodal questions spanning mathematics, science, and the humanities.
- Task count
- 2,500
ReasoningOnline
IFEval
Tests whether a model follows verifiable instructions such as format, length, and keyword constraints.
- Task count
- 541
ReasoningOnline
LAMBADA
Tests prediction of a passage’s final word when success requires understanding its broader discourse context.
Other categories in Online
More benchmarks
Questions
What other kinds of benchmarks are in Online?
Other categories in Online: Multimodal (51 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), Science (24 benchmarks), and Coding (17 benchmarks).
Published by AI Benchmark Index. How we check each benchmark.