Eval run cost estimator
Estimate the API bill for one evaluation run from its task count, token use, model prices, and likely retries.
How we calculated this
We multiply tasks by average input and output tokens, price each total at the per-million-token rate, and add the two charges.
The low estimate assumes one call per task. The high estimate applies the retry allowance to the entire base cost. It does not include storage, judge-model calls, tool calls, taxes, or provider discounts.
Method checked on 2026-10-15.
This is an estimate. Provider rounding, caching, batch discounts, taxes, and failed requests can change the bill.
Questions
- Does this include an LLM judge?
- No. Add the judge's tokens as another run or include them in the per-task averages.
- Why is this a range?
- The low end assumes no retries. The high end adds the retry percentage you enter.
Find benchmarks
Use this tool on your site
The snippet includes a link back to this page. Please keep it.