MultiChallenge leaderboard
- GPT-6.1 Sol leads
- 82.0%
- models measured
- 24
- lead over #2
- +3.2 pts
- median
- 59.8%
Leader on MultiChallenge · read October 22
On October 22, 2026, 24 of our catalog's models had a MultiChallenge result, and OpenAI GPT-6.1 Sol led with 82.0%, ahead of Google Gemini 4 Argon at 78.7%.
MultiChallenge tests models on realistic multi-turn conversations: whether they keep the instructions given earlier, remember what they inferred, and stay coherent with themselves. The number is the share of conversations handled correctly, as Scale AI publishes it.
Role in the index
MultiChallenge is shown as published and does not feed any of our indexes. Read the methodology
| # | Model | Result | Evidence | Coding index | Output $/Mtok |
|---|---|---|---|---|---|
| 1 | GPT-6.1 SolOpenAI | 82.0% | 82.4 | $10.00 | |
| 2 | Gemini 4 ArgonGoogle | 78.7% | 81.0 | $10.00 | |
| 3 | Muse Spark 1.1Meta | 75.3% | 70.3 | $4.25 | |
| 4 | Muse Spark OpenMeta | 74.2% | 47.4 | $4.25 | |
| 5 | Gemini 3.1 Pro PreviewGoogle | 71.4% | 32.3 | $12.00 | |
| 6 | GPT-5.4 ProOpenAI | 69.2% | Not benchmarked | $180.00 | |
| 7 | Claude Haiku 5.5Anthropic | 67.7% | 70.7 | $0.50 | |
| 8 | GPT-5.1OpenAI | 63.4% | 40.4 | $10.00 | |
| 9 | GPT-5OpenAI | 63.2% | 38.8 | $10.00 | |
| 10 | o3 ProOpenAI | 62.4% | Not benchmarked | $80.00 | |
| 11 | Kimi K2.5Moonshot AI | 61.4% | 38.6 | $2.85 | |
| 12 | Gemini 3.1 Flash Lite PreviewGoogle | 60.6% | 22.7 | $1.50 | |
| 13 | GPT-5 MiniOpenAI | 59.0% | 34.2 | $2.00 | |
| 14 | Claude Opus 4.5Anthropic | 59.0% | 44.0 | $25.00 | |
| 15 | Claude Opus 4.1Anthropic | 57.2% | Not benchmarked | $75.00 | |
| 16 | o3OpenAI | 56.6% | Not benchmarked | $8.00 | |
| 17 | Claude Opus 4.6Anthropic | 56.0% | 57.2 | $25.00 | |
| 18 | Kimi K2 ThinkingMoonshot AI | 55.4% | Not benchmarked | $2.50 | |
| 19 | Claude Sonnet 4.5Anthropic | 55.3% | 39.5 | $15.00 | |
| 20 | Gemini 2.5 ProGoogle | 53.6% | 19.1 | $10.00 | |
| 21 | Claude Haiku 4.5Anthropic | 50.5% | 35.6 | $5.00 | |
| 22 | gpt-oss-120bOpenAI | 45.3% | Not benchmarked | $0.50 | |
| 23 | o4 MiniOpenAI | 44.9% | Not benchmarked | $4.40 | |
| 24 | GPT-4.1OpenAI | 39.4% | Not benchmarked | $8.00 |
Only models in our catalog are listed, and each appears once with its best published run. 66 rows were read from the source on October 22, 2026, and rows that match no catalog model are left out.