# CrowS-Pairs

CrowS-Pairs is an AI benchmark; this page records what it measures and traces its data, grader, and scores to primary sources.

Measures stereotypical preferences in masked language models using paired sentences across nine types of social bias.

## Details

- Address: Online, Global
- Website: [github.com/nyu-mll/crows-pairs](<https://github.com/nyu-mll/crows-pairs>)
- Tags: Science, Helpful
- Category: Safety
- Status: Active
- Data licence: CC-BY-SA-4.0
- Data access: open
- Grader: exact
- Organization: NYU
- Released: 2020-03
- Paper: [arxiv.org/abs/2010.00133](<https://arxiv.org/abs/2010.00133>)
- Repository: [github.com/nyu-mll/crows-pairs](<https://github.com/nyu-mll/crows-pairs>)
- Task count: 1,508
- Top-score provenance: unknown
- ABI family: No

## Questions

### Where is CrowS-Pairs?

CrowS-Pairs is at Online, Global.

### How do I contact CrowS-Pairs?

To contact CrowS-Pairs, visit https://github.com/nyu-mll/crows-pairs.

## Source

From [Benchmark primary sources](https://aibenchmarkindex.com/about), license CC0-1.0, checked 2026-10-15.

Last updated October 15, 2026.

## Links

- This benchmark on the [web page](<https://aibenchmarkindex.com/global/online/crows-pairs>)
- [Safety in Online](<https://aibenchmarkindex.com/global/online/category/safety.md>)
- [More benchmarks in Online](<https://aibenchmarkindex.com/global/online.md>)

> **Check before you compare.** Benchmark methods and leaderboards change. Follow the cited primary source before using a result in a decision.