Graders, data access and licences
AI benchmark foundations for buyers. Lesson 2, 14 min.
The grader and the data are as important as the tasks. A good task with a weak grader gives a weak score. Grader types Each listing on this site records one grader type. - exact: the output must match a reference answer. It is cheap and repeatable. It can reject a valid answer in a different form. -…
The rest of this lesson is for learners enrolled in the course. See how AI benchmark foundations for buyers works