MultimodalOnline
PartiPrompts
Tests text-to-image systems with over 1,600 prompts spanning simple scenes, complex compositions, writing, and abstract concepts.
- Task count
- 1,600
AI benchmark index
AI Benchmark Index catalogs 264 AI benchmarks from primary sources.
Results
264 benchmarks
AgentsOnline
PokeLLMon
Measures a language-model agent on tactical decision making in competitive Pokémon battles.
ReasoningOnline
PokerBench
Evaluates whether language models choose solver-backed actions in 11,000 pre-flop and post-flop poker scenarios.
- Task count
- 11,000
ScienceOnline
Procgen Benchmark
Measures sample efficiency and generalization in reinforcement learning across procedurally generated game environments.
- Task count
- 16
MathOnline
ProofNet
Tests autoformalization and formal proving of undergraduate mathematics in Lean from natural-language statements.
- Task count
- 371
RetrievalOnline
RAGChecker
Diagnoses retrieval-augmented generation systems with claim-level metrics for their retrieval and generation components.
RetrievalOnline
RAGTruth
Evaluates hallucination detection in retrieval-augmented answers using human annotations at response and word levels.
- Task count
- 18,000
SafetyOnline
RealToxicityPrompts
Measures whether language models continue naturally occurring web prompts with toxic text.
CodingOnline
RepoBench
Tests repository-level code completion using cross-file dependencies in Python and Java projects.
CodingOnline
RepoQA
Tests long-context code retrieval by asking models to find described functions inside full repositories.
- Task count
- 500
ScienceOnline
RLBench
Measures robot learning across vision-guided manipulation tasks in a simulated environment.
AgentsOnline
RoboCasa
Evaluates robot policies on everyday kitchen manipulation tasks in a large-scale simulation framework.
- Task count
- 365
ScienceOnline
robomimic
Measures robot policies learned from offline human demonstrations across standardized datasets and tasks.
ScienceOnline
robosuite
Provides standardized simulated robot manipulation environments for evaluating learning methods.
OtherOnline
ROUGE
Scores automatic summaries against references using n-gram overlap and longest-common-subsequence variants.
MemoryOnline
RULER
Measures effective context length with configurable synthetic tasks that go beyond simple needle retrieval.
- Task count
- 13
OtherOnline
SacreBLEU
Computes reproducible machine-translation metrics while recording a signature for dataset and tokenization settings.
SafetyOnline
SafeBench
Evaluates autonomous-driving agents in safety-critical simulated scenarios, including adversarially generated cases.
- Task count
- 235
SafetyOnline
SafetyBench
Evaluates large-language-model safety knowledge with Chinese and English multiple-choice questions across seven risk categories.
- Task count
- 11,435
ScienceOnline
ScienceWorld
Measures text-based agents on tasks grounded in an elementary science curriculum.
- Task count
- 30
ScienceOnline
SciFact
Tests retrieval of scientific evidence and classification of research claims as supported or refuted.
MultimodalOnline
SEED-Bench
Measures image and video comprehension across multimodal ability dimensions using human-annotated multiple-choice questions.
AgentsOnline
SmartPlay
Measures agent reasoning, planning, spatial understanding, learning, and handling of randomness across six games.
- Task count
- 6
SafetyOnline
SORRY-Bench
Measures the safety-refusal behavior of language models across a balanced taxonomy of unsafe requests and prompt mutations.
- Task count
- 450
Cities in Global
Categories in Global
Questions
Which cities in Global have the most benchmarks?
The cities in Global with the most benchmarks are Online (264 benchmarks).
What kinds of benchmarks are in Global?
Categories in Global: Multimodal, Reasoning, Agents, Safety, and Science.
Published by AI Benchmark Index. How we check each benchmark.