# Safety benchmarks AI Benchmark Index catalogs 25 safety with source-linked metadata. 25 benchmarks. Also on the [web page](https://aibenchmarkindex.com/category/safety). ## Benchmarks 1. [AdvBench](https://aibenchmarkindex.com/global/online/advbench.md), Safety, Online: Measures whether adversarial suffixes induce aligned language models to produce targeted harmful behavior. 2. [AgentDojo](https://aibenchmarkindex.com/global/online/agentdojo.md), Safety, Online: Evaluates agent task utility and resistance to prompt injection while agents use tools over untrusted data. 3. [BBQ](https://aibenchmarkindex.com/global/online/bbq.md), Safety, Online: Measures social bias in question answering across ambiguous and disambiguated contexts covering nine protected dimensions. 4. [BBQ-Lite](https://aibenchmarkindex.com/global/online/bbq-lite.md), Safety, Online: Provides a compact BIG-bench version of BBQ for exact-match evaluation of bias-sensitive question answering. 5. [BeaverTails](https://aibenchmarkindex.com/global/online/beavertails.md), Safety, Online: Supports safety classification and preference evaluation with human-labeled question-answer pairs across harm categories. 6. [BIPIA](https://aibenchmarkindex.com/global/online/bipia.md), Safety, Online: Evaluates model and defense robustness against indirect prompt injection across five application tasks. 7. [BOLD](https://aibenchmarkindex.com/global/online/bold.md), Safety, Online: Measures social biases in open-ended English generation from prompts spanning profession, gender, race, religion, and politics. 8. [CrowS-Pairs](https://aibenchmarkindex.com/global/online/crows-pairs.md), Safety, Online: Measures stereotypical preferences in masked language models using paired sentences across nine types of social bias. 9. [CyberSecEval](https://aibenchmarkindex.com/global/online/cyberseceval.md), Safety, Online: Evaluates cybersecurity risks from model-generated code and responses to requests that could enable cyberattacks. 10. [HarmBench](https://aibenchmarkindex.com/global/online/harmbench.md), Safety, Online: Evaluates automated red-teaming attacks and model refusal defenses across standardized harmful behaviors. 11. [InjecAgent](https://aibenchmarkindex.com/global/online/injecagent.md), Safety, Online: Evaluates whether tool-using agents follow indirect prompt injections embedded in external tool responses. 12. [JailbreakBench](https://aibenchmarkindex.com/global/online/jailbreakbench.md), Safety, Online: Compares jailbreak attacks and defenses on a standardized set of misuse behaviors and evaluation rules. 13. [RealToxicityPrompts](https://aibenchmarkindex.com/global/online/realtoxicityprompts.md), Safety, Online: Measures whether language models continue naturally occurring web prompts with toxic text. 14. [SafeBench](https://aibenchmarkindex.com/global/online/safebench.md), Safety, Online: Evaluates autonomous-driving agents in safety-critical simulated scenarios, including adversarially generated cases. 15. [SafetyBench](https://aibenchmarkindex.com/global/online/safetybench.md), Safety, Online: Evaluates large-language-model safety knowledge with Chinese and English multiple-choice questions across seven risk categories. 16. [SORRY-Bench](https://aibenchmarkindex.com/global/online/sorry-bench.md), Safety, Online: Measures the safety-refusal behavior of language models across a balanced taxonomy of unsafe requests and prompt mutations. 17. [StereoSet](https://aibenchmarkindex.com/global/online/stereoset.md), Safety, Online: Measures stereotypical bias and language-modeling ability in sentence-level and discourse-level contexts. 18. [StrongREJECT](https://aibenchmarkindex.com/global/online/strongreject.md), Safety, Online: Evaluates jailbreak responses with a forbidden-prompt dataset and an automated judge designed to penalize empty refusals and ineffective attacks. 19. [ToxiGen](https://aibenchmarkindex.com/global/online/toxigen.md), Safety, Online: Tests hate-speech classifiers on machine-generated implicit toxic and benign statements about 13 minority groups. 20. [WinoBias](https://aibenchmarkindex.com/global/online/winobias.md), Safety, Online: Measures gender bias in coreference resolution by changing pronoun gender while preserving sentence structure. 21. [XSTest](https://aibenchmarkindex.com/global/online/xstest.md), Safety, Online: Tests whether safety-tuned language models refuse harmless prompts that superficially resemble unsafe requests. 22. [Benchmark MCP Safety](https://aibenchmarkindex.com/global/online/benchmark-mcp-safety.md), Safety, Online: Tests resistance to malicious tool descriptions, tool shadowing, and excess privilege. 23. [Benchmark Sycophancy](https://aibenchmarkindex.com/global/online/benchmark-sycophancy.md), Safety, Online: Tests whether a model resists endorsing a user's increasingly harmful mistaken plan. 24. [Do-Not-Answer](https://aibenchmarkindex.com/global/online/do-not-answer.md), Safety, Online: Evaluates whether language models decline risky instructions while still answering harmless requests. 25. [Benchmark Honesty](https://aibenchmarkindex.com/global/online/benchmark-honesty.md), Safety, Online: Tests whether a model admits uncertainty after a required tool call fails. ## Safety by city - [Online, Global](https://aibenchmarkindex.com/global/online/category/safety.md): 25 benchmarks ## Related categories - [Multimodal](https://aibenchmarkindex.com/category/multimodal.md): 51 benchmarks - [Reasoning](https://aibenchmarkindex.com/category/reasoning.md): 50 benchmarks - [Agents](https://aibenchmarkindex.com/category/agents.md): 31 benchmarks - [Science](https://aibenchmarkindex.com/category/science.md): 24 benchmarks - [Coding](https://aibenchmarkindex.com/category/coding.md): 17 benchmarks - [Retrieval](https://aibenchmarkindex.com/category/retrieval.md): 17 benchmarks - [Memory](https://aibenchmarkindex.com/category/memory.md): 10 benchmarks - [Browser](https://aibenchmarkindex.com/category/browser.md): 7 benchmarks - [Voice](https://aibenchmarkindex.com/category/voice.md): 7 benchmarks - [Math](https://aibenchmarkindex.com/category/math.md): 6 benchmarks - [Other](https://aibenchmarkindex.com/category/other.md): 6 benchmarks - [Tools](https://aibenchmarkindex.com/category/tools.md): 6 benchmarks - [Commerce](https://aibenchmarkindex.com/category/commerce.md): 4 benchmarks - [Long-horizon](https://aibenchmarkindex.com/category/long-horizon.md): 3 benchmarks > **Check before you compare.** Benchmark methods and leaderboards change. Follow the cited primary source before using a result in a decision.