What an AI benchmark measures

An AI benchmark is a fixed set of tasks, a procedure to run them and a grader that scores each output. A score is valid only for those tasks, that procedure and that grader.

The parts

• Task: one unit of work, for example a question, a coding issue or a goal in a web browser.
• Grader: the rule or the judge that decides if an output passes.
• Score: the result over all tasks, often a percentage.
• Setup: the model version, prompt, tools, sampling settings and retry rule of the run.

Start with your decision

Write down your decision before you open a leaderboard. 'Choose a support model that follows our refund policy while it uses tools' is a decision. 'Find the smartest model' is not a decision. The first statement names a workflow, a cost of failure and a behaviour you can observe.

The NIST draft practices for automated benchmark evaluations use this order: define the evaluation objective, then the construct to measure, then the intended use of the result. Choose the benchmark after these three steps. If you choose a popular score first, you will invent a reason for it later.

A category label is not a task

Two benchmarks about agents can measure different things. On this site, Benchmark Tool Errors and Benchmark Hundred Step both test agents. The first targets recovery from tool faults. The second targets reliability across many dependent steps. Before you compare two benchmarks, read the unit of work, the allowed tools, the stop condition and the reference answer.

What a score does not show

A 100 percent score shows that a system met that benchmark's tasks and grading rules under the reported setup. It does not prove that the capability is solved. It does not show how the system behaves in your own workflow.

Sources

• AI Benchmark Index guide, How to choose an AI benchmark
• AI Benchmark Index guide, What a saturated benchmark is and how to spot one
• NIST AI 800-2 (initial public draft), Practices for Automated Benchmark Evaluations of Language Models