SWE Atlas: Codebase QnA leaderboard
- Claude Opus 5 leads
- 63.2%
- models measured
- 18
- lead over #2
- +3.2 pts
- median
- 41.5%
Leader on SWE Atlas: Codebase QnA · read October 22
# SWE Atlas: Codebase QnA, read October 22
leaders = {
"Claude Opus 5": 63.2%, "Claude Fable 5.1": 60.0%, "GPT-6 Astra": 59.1%,
}
On October 22, 2026, 18 of our catalog's models had a SWE Atlas: Codebase QnA result, and Anthropic Claude Opus 5 led with 63.2%, ahead of Anthropic Claude Fable 5.1 at 60.0%.
SWE Atlas Codebase QnA tests deep code comprehension: answering questions about real software repositories. Scale AI runs each model inside an agent harness; we show the best result per model and name the harness.
Role in the index
A coding board we show beside the Vibe Coding Index, not inside it: the index keeps the six boards it was defined on, so a score does not move every time a board is added. Read the methodology
| # | Model | Result | Evidence | Coding index | Output $/Mtok |
|---|---|---|---|---|---|
| 1 | Claude Opus 5Anthropic | 63.2% | 83.0 | $25.00 | |
| 2 | Claude Fable 5.1Anthropic | 60.0% | 84.2 | $50.00 | |
| 3 | GPT-6 AstraOpenAI | 59.1% | 85.1 | $50.00 | |
| 4 | Claude Opus 4.8Anthropic | 57.3% | 65.0 | $25.00 | |
| 5 | GLM 5.2Z.ai | 48.1% | 61.1 | $4.40 | |
| 6 | Gemini 3.8 FlashGoogle | 47.0% | 61.6 | $3.75 | |
| 7 | GPT-5.6 SolOpenAI | 46.0% | 73.5 | $20.00 | |
| 8 | GPT-5.5OpenAI | 45.4% | 71.3 | $30.00 | |
| 9 | Muse Spark 1.1Meta | 42.2% | 70.3 | $4.25 | |
| 10 | GPT-5.4OpenAI | 40.8% | 64.4 | $15.00 | |
| 11 | Claude Opus 4.7Anthropic | 40.3% | 67.5 | $25.00 | |
| 12 | Claude Fable 5Anthropic | 39.0% | 79.3 | $50.00 | |
| 13 | Claude Opus 4.6Anthropic | 33.3% | 57.2 | $25.00 | |
| 14 | Claude Sonnet 4.6Anthropic | 31.2% | 57.3 | $15.00 | |
| 15 | DeepSeek V4 Pro 0423DeepSeek | 27.2% | 38.7 | $2.60 | |
| 16 | GLM 5Z.ai | 20.5% | 41.7 | $3.20 | |
| 17 | Kimi K2.5Moonshot AI | 13.1% | 38.6 | $2.85 | |
| 18 | MiniMax M2.5MiniMax | 10.3% | 37.2 | $1.20 |
Only models in our catalog are listed, and each appears once with its best published run. 66 rows were read from the source on October 22, 2026, and rows that match no catalog model are left out.