# Audit AI benchmark claims before you buy

Check a vendor's benchmark claim like an evaluator: ceilings and uncertainty, contamination, grader and judge bias, score provenance and the licence of each asset.

- Level: Specialist
- Length: 75 minutes
- Exam questions: 10, pass mark 80%, up to 3 attempts
- Certificate: valid for 12 months

## What you will learn

- Find the five signals of a saturated benchmark and decide when a rank claim is not supported.
- Explain how contamination can raise scores and which questions to ask about test freshness.
- Name the biases of LLM judges in the MT-Bench study and the controls that reduce them.
- Ask for complete run metadata and reproduce a claim on a sample.
- Record the licence of each benchmark asset with SPDX identifiers.

## Lessons

1. [Ceilings, uncertainty and rank claims](https://aibenchmarkindex.com/academy/audit-ai-benchmark-claims/ceilings-and-uncertainty.md) (18 minutes)
2. [Contamination and test freshness](https://aibenchmarkindex.com/academy/audit-ai-benchmark-claims/contamination-and-freshness.md) (18 minutes)
3. [Audit the grader and the judge](https://aibenchmarkindex.com/academy/audit-ai-benchmark-claims/audit-the-grader.md) (18 minutes)
4. [Provenance, licences and the decision record](https://aibenchmarkindex.com/academy/audit-ai-benchmark-claims/provenance-licences-and-the-decision.md) (21 minutes)

## Questions

### Does this course make me a qualified auditor?

No. It is a certificate of completion from this site. It records that you passed this course's exam. It is not a licence, an accreditation or a professional qualification.

### Does the course give legal advice on data licences?

No. It teaches you to read and record licence terms. When ownership of a valuable dataset is unclear, ask counsel or the publisher.

### Does the course say which vendors make false claims?

No. It names no vendor. It teaches checks that you apply to any benchmark claim.

Also on the [web page](https://aibenchmarkindex.com/academy/audit-ai-benchmark-claims).