# Choose and run an AI benchmark for a model decision

A practical method for using AI benchmarks in a buying decision: define the decision, build a shortlist, budget the run, then sample, grade and record a run that another person can check.

- Level: Practitioner
- Length: 60 minutes
- Exam questions: 10, pass mark 80%, up to 3 attempts
- Certificate: valid for 12 months

## What you will learn

- Write a decision statement with a must-catch list of failures.
- Build a shortlist in order: task fit, grader, licence and access, saturation, provenance and cost.
- Estimate the cost of a run from task count, tokens and the prices you see on the day.
- Run a seeded sample, report the numerator, the denominator and an interval, and save a run record.

## Lessons

1. [Define the decision and the failures that matter](https://aibenchmarkindex.com/academy/choose-and-run-an-ai-benchmark/define-the-decision.md) (12 minutes)
2. [Build a shortlist](https://aibenchmarkindex.com/academy/choose-and-run-an-ai-benchmark/build-a-shortlist.md) (15 minutes)
3. [Budget the run](https://aibenchmarkindex.com/academy/choose-and-run-an-ai-benchmark/budget-the-run.md) (15 minutes)
4. [Sample, grade and record the run](https://aibenchmarkindex.com/academy/choose-and-run-an-ai-benchmark/sample-grade-and-record.md) (18 minutes)

## Questions

### Do I need to write code to follow this course?

Some steps are easier with a technical colleague, for example the runner limits and the run record. The decisions in each step do not need code.

### Does this course give current model prices?

No. Prices change. The lessons use example numbers only. Read the official pricing page of each provider on the day you run.

### Does passing the exam mean my evaluation is correct?

No. The certificate of completion records that you passed this site's exam. The quality of your evaluation depends on your own run and records.

Also on the [web page](https://aibenchmarkindex.com/academy/choose-and-run-an-ai-benchmark).