# Arena-Hard-Auto

Arena-Hard-Auto is an AI benchmark; this page records what it measures and traces its data, grader, and scores to primary sources.

Tests open-ended instruction following on difficult real user prompts with automated preference judges.

## Details

- Address: Online, Global
- Website: [github.com/lm-sys/arena-hard-auto](<https://github.com/lm-sys/arena-hard-auto>)
- Tags: Viral, Science
- Category: Reasoning
- Status: Active
- Data licence: Apache-2.0
- Data access: open
- Grader: llm-judge
- Organization: LMSYS Org
- Released: 2024-06
- Paper: [arxiv.org/abs/2406.11939](<https://arxiv.org/abs/2406.11939>)
- Leaderboard: [github.com/lm-sys/arena-hard-auto\#leaderboard](<https://github.com/lm-sys/arena-hard-auto#leaderboard>)
- Repository: [github.com/lm-sys/arena-hard-auto](<https://github.com/lm-sys/arena-hard-auto>)
- Task count: 750
- Top-score provenance: unknown
- ABI family: No

## Questions

### Where is Arena-Hard-Auto?

Arena-Hard-Auto is at Online, Global.

### How do I contact Arena-Hard-Auto?

To contact Arena-Hard-Auto, visit https://github.com/lm-sys/arena-hard-auto.

## Source

From [Benchmark primary sources](https://aibenchmarkindex.com/about), license CC0-1.0, checked 2026-10-15.

Last updated October 15, 2026.

## Links

- This benchmark on the [web page](<https://aibenchmarkindex.com/global/online/arena-hard-auto>)
- [Reasoning in Online](<https://aibenchmarkindex.com/global/online/category/reasoning.md>)
- [More benchmarks in Online](<https://aibenchmarkindex.com/global/online.md>)

> **Check before you compare.** Benchmark methods and leaderboards change. Follow the cited primary source before using a result in a decision.