Skip to content

AI benchmark list

There are 264 source-checked AI benchmarks in this index.

Results

264 benchmarks

  • MemoryOnline

    LongBench v2

    Tests deep reasoning over realistic contexts from 8,000 to two million words using difficult multiple-choice questions.

    Task count
    503

  • MultimodalOnline

    LongVideoBench

    Tests understanding of hour-long videos whose questions require reasoning across interleaved frames, subtitles, and long contexts.

    Task count
    3,763

  • MemoryOnline

    LooGLE

    Tests short- and long-dependency understanding over recent documents longer than 24,000 tokens.

    Task count
    6,000

  • ScienceOnline

    ManiSkill

    Measures generalizable robotic manipulation policies in scalable simulated environments.

  • ScienceOnline

    Matbench

    Evaluates machine-learning models on 13 curated materials-property prediction tasks.

    Task count
    13

  • ScienceOnline

    Matbench Discovery

    Compares the accuracy, robustness, and computational cost of models used for atomistic and materials-discovery tasks.

  • MathOnline

    MATH

    Tests competition-level mathematical problem solving across algebra, counting, geometry, number theory, and related subjects.

    Task count
    12,500

  • MathOnline

    MATH-500

    Tests mathematical reasoning on 500 held-out problems sampled from the MATH test split.

    Task count
    500

  • MultimodalOnline

    MATH-Vision

    Tests visual mathematical reasoning on competition problems spanning 16 subjects and five difficulty levels.

    Task count
    3,040

  • MultimodalOnline

    MathVista

    Tests mathematical reasoning grounded in diagrams and other visual contexts across diverse task types.

    Task count
    6,141

  • CodingOnline

    MBPP

    Tests basic Python synthesis with crowd-sourced prompts, reference solutions, and automated test cases.

    Task count
    974

  • ScienceOnline

    MedMCQA

    Evaluates medical multiple-choice reasoning on questions drawn from Indian medical entrance examinations.

    Task count
    194,000

  • ScienceOnline

    MedQA

    Evaluates medical knowledge and reasoning with multiple-choice questions from professional medical licensing exams.

    Task count
    61,097

  • MemoryOnline

    MemBench

    Evaluates agent memory effectiveness, efficiency, and capacity across factual and reflective memory in participation and observation settings.

  • ScienceOnline

    Meta-World

    Measures multi-task and meta-reinforcement learning across robotic manipulation tasks and goal variations.

    Task count
    50

  • MathOnline

    MGSM

    Evaluates grade-school mathematical reasoning on the same 250 problems translated by human annotators into 10 languages.

    Task count
    2,500

  • BrowserOnline

    Mind2Web

    Measures whether web agents can follow open-ended instructions across real websites and unfamiliar domains.

    Task count
    2,350

  • AgentsOnline

    MineDojo

    Measures open-ended embodied agents across thousands of Minecraft tasks with an internet-scale knowledge base.

    Task count
    3,142

  • MathOnline

    miniF2F

    Tests automated theorem provers on olympiad and classroom mathematics translated across formal systems.

    Task count
    488

  • BrowserOnline

    MiniWoB++

    Measures reinforcement-learning agents on a collection of simple browser interaction environments.

  • RetrievalOnline

    MIRACL

    Evaluates monolingual information retrieval across 18 languages using native-speaker relevance judgments.

    Task count
    18

  • OtherOnline

    MLPerf Inference

    Measures how quickly and efficiently hardware and software systems run trained models across deployment scenarios.

  • OtherOnline

    MLPerf Tiny

    Measures latency and energy for embedded-device inference on standardized tiny-machine-learning workloads.

  • OtherOnline

    MLPerf Training

    Measures time to train reference machine-learning workloads to defined quality targets on different systems.

Categories in Online

Questions

What kinds of benchmarks are in Online?

Categories in Online: Multimodal (51 benchmarks), Reasoning (50 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), and Science (24 benchmarks).

How many AI benchmarks are in the index?

AI Benchmark Index currently lists 264 benchmarks that link to a primary source.

Can I submit an AI benchmark?

Yes. Submit a public primary source, licence, release date, category, and a one-sentence description of what it measures.

Published by . How we check each benchmark.