Skip to content

Retrieval benchmarks

There are 17 retrieval in the index.

Results

17 benchmarks

  • RetrievalOnline

    BEIR

    Evaluates zero-shot information retrieval across heterogeneous datasets, domains, and retrieval architectures.

    Task count
    18

  • RetrievalOnline

    CRUD-RAG

    Evaluates retrieval-augmented generation across create, read, update, and delete application scenarios.

    Task count
    4

  • RetrievalOnline

    FActScore

    Measures the factual precision of long-form generations by decomposing them into atomic claims and checking each claim against a knowledge source.

    Task count
    500

  • RetrievalOnline

    FEVER

    Tests whether systems verify Wikipedia-based claims as supported, refuted, or lacking enough evidence.

    Task count
    185,441

  • RetrievalOnline

    FinanceBench

    Evaluates open-book financial question answering against answers and evidence drawn from public company filings.

    Task count
    10,231

  • RetrievalOnline

    FreshQA

    Evaluates answers to questions whose facts change over time or were never present in a model’s training data.

    Task count
    600

  • RetrievalOnline

    KILT

    Evaluates knowledge-intensive language tasks while requiring answers and provenance grounded in one Wikipedia snapshot.

    Task count
    11

  • RetrievalOnline

    LegalBench-RAG

    Evaluates whether retrieval systems return minimal, precisely located evidence for complex legal contract questions.

    Task count
    6,858

  • RetrievalOnline

    MIRACL

    Evaluates monolingual information retrieval across 18 languages using native-speaker relevance judgments.

    Task count
    18

  • RetrievalOnline

    MMTEB

    Measures multilingual text embeddings across more than 500 quality-controlled tasks covering over 250 languages.

    Task count
    500

  • RetrievalOnline

    MS MARCO

    Evaluates machine reading, answer generation, and passage or document ranking using questions sampled from Bing search.

    Task count
    1,010,916

  • RetrievalOnline

    MTEB

    Measures text embeddings across classification, clustering, retrieval, reranking, similarity, and summarization tasks.

    Task count
    58

  • RetrievalOnline

    RAGChecker

    Diagnoses retrieval-augmented generation systems with claim-level metrics for their retrieval and generation components.

  • RetrievalOnline

    RAGTruth

    Evaluates hallucination detection in retrieval-augmented answers using human annotations at response and word levels.

    Task count
    18,000

  • RetrievalOnline

    WebWalkerQA

    Measures long-horizon information seeking through multi-hop traversal of structured websites.

  • RetrievalOnline

    TREC Deep Learning Track

    Evaluates passage and document ranking systems on large web collections with graded relevance judgments.

  • RetrievalOnline

    Natural Questions

    Evaluates question answering on real Google queries paired with annotated evidence and answers from Wikipedia pages.

Other categories in Online

More benchmarks

Questions

What other kinds of benchmarks are in Online?

Other categories in Online: Multimodal (51 benchmarks), Reasoning (50 benchmarks), Agents (31 benchmarks), Safety (25 benchmarks), and Science (24 benchmarks).

Published by . How we check each benchmark.