Skip to content

AI benchmark list

There are 264 source-checked AI benchmarks in this index.

Results

264 benchmarks

  • ReasoningOnline

    IFEval

    Tests whether a model follows verifiable instructions such as format, length, and keyword constraints.

    Task count
    541

  • MultimodalOnline

    ImageNet-A

    Tests image classifiers on naturally occurring, unmodified images selected because standard ImageNet models misclassify them.

  • MultimodalOnline

    ImageNet-R

    Tests ImageNet classifiers on artistic and other non-photographic renditions of 200 object classes.

    Task count
    30,000

  • MultimodalOnline

    ImageNet-V2

    Retests ImageNet classifiers on three independently collected 10,000-image test sets matched to the original distribution.

    Task count
    30,000

  • MultimodalOnline

    ImagenHub

    Compares conditional image generation and editing systems across standardized tasks, metrics, and human ratings.

  • MultimodalOnline

    ImageReward

    Measures how closely text-to-image outputs align with human preferences using a learned reward model trained on expert comparisons.

    Task count
    137,000

  • MemoryOnline

    InfiniteBench

    Evaluates language models on synthetic and realistic English and Chinese tasks with contexts averaging more than 100,000 tokens.

    Task count
    12

  • SafetyOnline

    InjecAgent

    Evaluates whether tool-using agents follow indirect prompt injections embedded in external tool responses.

    Task count
    1,054

  • CodingOnline

    InterCode-CTF

    Evaluates coding agents that solve capture-the-flag tasks through iterative commands and execution feedback.

  • SafetyOnline

    JailbreakBench

    Compares jailbreak attacks and defenses on a standardized set of misuse behaviors and evaluation rules.

    Task count
    100

  • AgentsOnline

    Jericho

    Measures agents in human-authored interactive-fiction games through a common learning interface.

  • RetrievalOnline

    KILT

    Evaluates knowledge-intensive language tasks while requiring answers and provenance grounded in one Wikipedia snapshot.

    Task count
    11

  • MultimodalOnline

    Kinetics

    Evaluates video action recognition across human-object and human-human interactions in short clips.

  • ReasoningOnline

    LAMBADA

    Tests prediction of a passage’s final word when success requires understanding its broader discourse context.

  • ReasoningOnline

    LegalBench

    Evaluates legal reasoning across collaboratively contributed tasks covering varied legal domains and text formats.

    Task count
    162

  • RetrievalOnline

    LegalBench-RAG

    Evaluates whether retrieval systems return minimal, precisely located evidence for complex legal contract questions.

    Task count
    6,858

  • ReasoningOnline

    LexGLUE

    Evaluates legal language understanding across seven English classification datasets using a common evaluation setup.

    Task count
    7

  • ScienceOnline

    LIBERO

    Measures knowledge transfer in lifelong robot learning across manipulation task suites with controlled distribution shifts.

    Task count
    130

  • AgentsOnline

    LIBERO-Spatial

    Tests lifelong robot learning under spatial-relation distribution shifts in simulated manipulation tasks.

  • ReasoningOnline

    LiveBench

    Tests six capability areas with regularly refreshed questions and objective, non-judge scoring.

    Task count
    18

  • CodingOnline

    LiveCodeBench

    Tests code generation, execution, self-repair, and test-output prediction with newly released contest problems.

    Task count
    1,055

  • CodingOnline

    LiveCodeBench v5

    Tests code capabilities on the LiveCodeBench release containing contest problems through January 2025.

    Task count
    880

  • MemoryOnline

    LoCoMo

    Evaluates very long-term conversational memory through question answering, event summarization, and dialogue generation.

    Task count
    10

  • MemoryOnline

    LongBench

    Evaluates bilingual long-context understanding across single- and multi-document question answering, summarization, few-shot learning, and code.

    Task count
    21

Categories in Online

Questions

What kinds of benchmarks are in Online?

Categories in Online: Multimodal (51 benchmarks), Reasoning (50 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), and Science (24 benchmarks).

How many AI benchmarks are in the index?

AI Benchmark Index currently lists 264 benchmarks that link to a primary source.

Can I submit an AI benchmark?

Yes. Submit a public primary source, licence, release date, category, and a one-sentence description of what it measures.

Published by . How we check each benchmark.