Skip to content

Graders, data access and licences

AI benchmark foundations for buyers. Lesson 2, 14 min.

The grader and the data are as important as the tasks. A good task with a weak grader gives a weak score. Grader types Each listing on this site records one grader type. - exact: the output must match a reference answer. It is cheap and repeatable. It can reject a valid answer in a different form. -…

The rest of this lesson is for learners enrolled in the course. See how AI benchmark foundations for buyers works

Published by . How we check each benchmark.

The benchmark newsletter

Updates on benchmarks and reported scores.

We only use your email to send this newsletter. Privacy policy.

Graders, data access and licences: AI benchmark foundations for buyers