BIG-Bench Hard leaderboard
- models measured
- 3
- lead over #2
- +4.7 pts
- median
- 75.1%
Leader on BIG-Bench Hard · read October 22
On October 22, 2026, 4 of our catalog's models had a BIG-Bench Hard result, and Google Gemini 1.5 Pro led with 89.2%, ahead of Qwen2.5 72B Instruct at 79.8%.
The hardest tasks of BIG-Bench, where earlier models fell short of the average human rater. Average score.
Role in the index
BIG-Bench Hard is shown as published and does not feed any of our indexes. Read the methodology
| # | Model | Result | Evidence | Coding index | Output $/Mtok |
|---|---|---|---|---|---|
| 1 | Gemini 1.5 ProRetiredGoogle | 89.2% | Not benchmarked | $5.00 | |
| 2 | Qwen2.5 72B InstructQwen | 79.8% | Not benchmarked | $0.40 | |
| 3 | GPT-4OpenAI | 75.1% | Not benchmarked | $60.00 | |
| 4 | GPT-3.5 TurboOpenAI | 61.6% | Not benchmarked | $1.50 |
Only models in our catalog are listed, and each appears once with its best published run. 215 rows were read from the source on October 22, 2026, and rows that match no catalog model are left out.