SWE-bench Verified leaderboard
- Claude Opus 4.5 leads
- 79.2%
- models measured
- 38
- lead over #2
- +3.4 pts
- median
- 63.8%
Leader on SWE-bench Verified · read October 22
# SWE-bench Verified, read October 22
leaders = {
"Claude Opus 4.5": 79.2%, "Gemini 3 Flash Preview": 75.8%, "MiniMax M2.5": 75.8%,
}
On October 22, 2026, 42 of our catalog's models had a SWE-bench Verified result, and Anthropic Claude Opus 4.5 led with 79.2%, ahead of Google Gemini 3 Flash Preview at 75.8%.
SWE-bench hands a model a real GitHub issue from an open source Python project and asks for a patch. The number is the share of issues where the patch makes the project tests pass. The board also publishes what one attempt cost, which is how we compute cost per solved task.
Role in the index
A coding board we show beside the Vibe Coding Index, not inside it: the index keeps the six boards it was defined on, so a score does not move every time a board is added. Read the methodology
| # | Model | Result | Cost per attempt | Evidence | Coding index | Output $/Mtok |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.5Anthropic | 79.2% | Not published | 44.0 | $25.00 | |
| 2 | Gemini 3 Flash PreviewGoogle | 75.8% | $0.356 | 43.7 | $3.00 | |
| 3 | MiniMax M2.5MiniMax | 75.8% | $0.0733 | 37.2 | $1.20 | |
| 4 | Claude Opus 4.6Anthropic | 75.6% | $0.5515 | 57.2 | $25.00 | |
| 5 | Claude Sonnet 4.5Anthropic | 74.8% | Not published | 39.5 | $15.00 | |
| 6 | GPT-5OpenAI | 74.4% | Not published | 38.8 | $10.00 | |
| 7 | GPT-5.2OpenAI | 72.8% | $0.4736 | 55.9 | $14.00 | |
| 8 | GPT-5.2-CodexOpenAI | 72.8% | $0.4494 | 45.4 | $14.00 | |
| 9 | GLM 5Z.ai | 72.8% | $0.5344 | 41.7 | $3.20 | |
| 10 | Kimi K2 0711Moonshot AI | 71.2% | Not published | Not benchmarked | $2.30 | |
| 11 | Kimi K2.5Moonshot AI | 70.8% | $0.1466 | 38.6 | $2.85 | |
| 12 | DeepSeek V3.2DeepSeek | 70.0% | $0.4478 | 30.0 | $0.88 | |
| 13 | Qwen3 Coder 480B A35BQwen | 69.6% | Not published | Not benchmarked | $1.50 | |
| 14 | GLM 4.6Z.ai | 68.2% | Not published | 8.8 | $2.20 | |
| 15 | Claude Haiku 4.5Anthropic | 66.6% | $0.3309 | 35.6 | $5.00 | |
| 16 | Claude 3.7 SonnetRetiredAnthropic | 66.4% | Not published | Not benchmarked | $15.00 | |
| 17 | GPT-5.1OpenAI | 66.0% | $0.3062 | 40.4 | $10.00 | |
| 18 | GPT-5.1-CodexOpenAI | 66.0% | $0.5889 | 15.3 | $10.00 | |
| 19 | o4 MiniOpenAI | 64.6% | Not published | Not benchmarked | $4.40 | |
| 20 | GLM 4.5Z.ai | 64.2% | Not published | Not benchmarked | $2.20 | |
| 21 | Claude 3.5 SonnetRetiredAnthropic | 63.4% | Not published | Not benchmarked | $15.00 | |
| 22 | Kimi K2 ThinkingMoonshot AI | 63.4% | $0.4383 | Not benchmarked | $2.50 | |
| 23 | MiniMax M2MiniMax | 61.0% | $0.4283 | Not benchmarked | $1.02 | |
| 24 | Qwen3 Coder 30B A3B InstructQwen | 60.4% | Not published | Not benchmarked | $0.44 | |
| 25 | GPT-5 MiniOpenAI | 59.8% | $0.0355 | 34.2 | $2.00 | |
| 26 | o3OpenAI | 58.4% | $0.3337 | Not benchmarked | $8.00 | |
| 27 | Devstral 2 2512Mistral AI | 53.8% | Not published | Not benchmarked | $2.00 | |
| 28 | Gemini 2.5 ProGoogle | 53.6% | $0.2884 | 19.1 | $10.00 | |
| 29 | Qwen2.5 Coder 32B InstructQwen | 47.0% | Not published | Not benchmarked | $1.00 | |
| 30 | o3 MiniOpenAI | 42.4% | Not published | Not benchmarked | $4.40 | |
| 31 | DeepSeek V3 0324DeepSeek | 42.0% | Not published | Not benchmarked | $1.07 | |
| 32 | Claude 3.5 HaikuRetiredAnthropic | 40.6% | Not published | Not benchmarked | $4.00 | |
| 33 | GPT-4.1OpenAI | 39.6% | $0.1463 | Not benchmarked | $8.00 | |
| 34 | GPT-4oOpenAI | 38.8% | Not published | Not benchmarked | $10.00 | |
| 35 | GPT-5 NanoOpenAI | 34.8% | $0.0381 | Not benchmarked | $0.40 | |
| 36 | Gemini 2.5 FlashGoogle | 28.7% | $0.1339 | Not benchmarked | $2.50 | |
| 37 | gpt-oss-120bOpenAI | 26.0% | $0.0571 | Not benchmarked | $0.50 | |
| 38 | GPT-4.1 MiniOpenAI | 23.9% | $0.4352 | Not benchmarked | $1.60 | |
| 39 | GPT-4OpenAI | 22.4% | Not published | Not benchmarked | $60.00 | |
| 40 | Llama 4 MaverickMeta Llama | 21.0% | $0.3137 | Not benchmarked | $1.00 | |
| 41 | Claude 3 OpusRetiredAnthropic | 15.8% | Not published | Not benchmarked | $75.00 | |
| 42 | Llama 4 ScoutMeta Llama | 9.1% | $0.1156 | Not benchmarked | $0.50 |
Only models in our catalog are listed, and each appears once with its best published run. 42 rows were read from the source on October 22, 2026, and rows that match no catalog model are left out. Cost per attempt is what swebench.com reports one task cost on the run behind the result, in US dollars. Only some runs report it.