# Glossary

## Benchmark

A fixed set of tasks with a scoring method, used to measure and compare AI systems on one kind of skill.

## Saturation

When scores bunch near a benchmark's practical ceiling so tightly that it can no longer separate the systems you need to compare. A saturated benchmark can still work as a regression check.

## Top score

The highest reported score on a benchmark, with the source that reported it. Check the source for the model, settings and date.

## Leaderboard

A public table of results on a benchmark, often run by the benchmark's authors. Entries may be self-reported or independently run.

## Data licence

The licence that governs a benchmark's data. It decides whether you can use, share or train on the data, so read it before you build on a benchmark.

## Contamination

When benchmark tasks or answers appear in a model's training data, which can inflate scores without real skill behind them.

## Held-out set

Tasks kept private so they are less likely to leak into training data. Private sets help against contamination but are harder to audit.

## Task count

How many items a benchmark contains. Small sets give noisier scores, so small gaps between models may not mean much.

## Pass@k

A coding metric that counts a task as solved if any of k attempts passes the tests. Compare scores only when k is the same.

## Harness

The code that runs a model on a benchmark, sends prompts, calls tools and scores the answers. Different harnesses can produce different scores for the same model.

## Agent benchmark

A benchmark where a system plans and acts across tools or environments, scored on whether it completes the task.

## Long-horizon task

A task with many dependent steps or long execution, where early mistakes carry through to the result.

## Self-reported score

A result published by the model's own developer rather than by the benchmark authors or a third party. Read how it was run before you compare it.
