SWE-bench Verified (Vals AI) leaderboard
- Claude Opus 5 leads
- 97.0%
- models measured
- 76
- lead over #2
- +0.6 pts
- median
- 75.4%
Leader on SWE-bench Verified (Vals AI) · read October 21
# SWE-bench Verified (Vals AI), read October 21
leaders = {
"Claude Opus 5": 97.0%, "DeepSeek V4 Pro 0813": 96.4%, "GPT-5.6 Sol": 96.2%,
}
On October 21, 2026, 76 of our catalog's models had a SWE-bench Verified (Vals AI) result, and Anthropic Claude Opus 5 led with 97.0%.
Real GitHub issues fixed with one shared agent harness, run by Vals AI; current models that the swebench.com board has not caught up with. Share resolved.
Role in the index
A coding board we show beside the Vibe Coding Index, not inside it: the index keeps the four boards it was defined on, so a score does not move every time a board is added. Read the methodology
| # | Model | Result | Cost per test | Duration, as printed | Evidence | Coding index | Output $/Mtok |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5Anthropic | 97.0% | $1.29 | 9m 37s | 81.8 | $25.00 | |
| 2 | DeepSeek V4 Pro 0813DeepSeek | 96.4% | $0.10 | 4m 00s | 54.6 | $1.98 | |
| 3 | GPT-5.6 SolOpenAI | 96.2% | $1.15 | 3m 02s | 76.7 | $20.00 | |
| 4 | Grok 4.6xAI | 95.6% | $0.78 | 10m 04s | 76.4 | $6.00 | |
| 5 | GPT-5.6 TerraOpenAI | 95.4% | $0.40 | 3m 00s | 69.0 | $12.00 | |
| 6 | GLM 5.3Z.ai | 95.4% | $0.34 | 14m 01s | 73.1 | $1.40 | |
| 7 | Claude Fable 5Anthropic | 95.0% | $2.05 | 5m 56s | 82.1 | $50.00 | |
| 8 | Kimi K3Moonshot AI | 93.4% | $0.76 | 10m 19s | 61.8 | $11.936 | |
| 9 | GPT-5.6 LunaOpenAI | 93.0% | $0.04 | 3m 21s | 61.5 | $1.20 | |
| 10 | GLM 5.3 FlashZ.ai | 92.0% | $0.02 | 1h 04m | 71.0 | $0.50 | |
| 11 | DeepSeek V4 Flash 0731DeepSeek | 88.8% | $0.01 | 3m 22s | Not benchmarked | $1.32 | |
| 12 | Claude Opus 4.8Anthropic | 88.6% | $1.92 | 9m 27s | 66.0 | $25.00 | |
| 13 | Muse Spark 1.2Meta | 86.6% | $0.55 | 8m 07s | 65.1 | $4.25 | |
| 14 | Grok 4.5xAI | 86.6% | $0.54 | 3m 20s | 75.5 | $6.00 | |
| 15 | Qwen3.8 27BQwen | 86.0% | $1.08 | 10m 00s | 73.6 | $1.35 | |
| 16 | Qwen3.8 Max (0902)Qwen | 85.6% | $1.13 | 41m 42s | 63.5 | $6.00 | |
| 17 | GLM 5.2Z.ai | 82.8% | $0.71 | 11m 10s | 57.7 | $7.00 | |
| 18 | GPT-5.5OpenAI | 82.6% | $1.36 | 7m 06s | 67.8 | $30.00 | |
| 19 | T Inkling SmallThinking Machines | 82.2% | $0.13 | 4m 05s | 29.9 | $1.20 | |
| 20 | Claude Opus 4.7Anthropic | 82.0% | $2.42 | 7m 22s | 51.1 | $25.00 | |
| 21 | Muse Spark 1.1Meta | 82.0% | $0.35 | 3m 29s | 48.9 | $4.25 | |
| 22 | Gemini 3.7 FlashGoogle | 80.8% | $1.44 | 5m 50s | 69.9 | $3.75 | |
| 23 | Gemini 3.8 FlashGoogle | 80.0% | $2.19 | 11m 23s | 70.5 | $4.125 | |
| 24 | Claude Sonnet 5Anthropic | 79.6% | $1.49 | 16m 02s | 63.5 | $10.00 | |
| 25 | Gemini 3.6 FlashGoogle | 79.6% | $1.19 | 13m 08s | 48.3 | $3.75 | |
| 26 | Gemini 3.1 Pro PreviewGoogle | 78.8% | $0.78 | 5m 12s | 35.3 | $12.00 | |
| 27 | Gemini 3.5 FlashGoogle | 78.8% | $0.95 | 4m 14s | 42.8 | $9.00 | |
| 28 | Claude Opus 4.6Anthropic | 78.2% | $1.22 | 5m 51s | 66.9 | $25.00 | |
| 29 | Kimi K2.7 CodeMoonshot AI | 78.2% | $0.27 | 14m 57s | 39.1 | $3.35 | |
| 30 | GPT-5.4OpenAI | 78.2% | $0.80 | 5m 07s | 37.7 | $15.00 | |
| 31 | GPT-5.3-CodexOpenAI | 78.0% | $0.46 | 4m 07s | 29.5 | $14.00 | |
| 32 | T InklingThinking Machines | 77.6% | $0.59 | 6m 34s | 49.7 | $4.05 | |
| 33 | Claude Sonnet 4.6Anthropic | 77.4% | $1.30 | 8m 32s | 57.8 | $15.00 | |
| 34 | DeepSeek V4 Pro 0423DeepSeek | 77.4% | $0.44 | 10m 35s | 35.3 | $1.9105 | |
| 35 | Claude Opus 4.5Anthropic | 76.4% | $1.05 | 5m 21s | 66.8 | $25.00 | |
| 36 | GLM 5.1Z.ai | 76.4% | $0.46 | 8m 47s | 44.0 | $4.40 | |
| 37 | Kimi K2.6Moonshot AI | 76.2% | $0.27 | 8m 40s | 44.2 | $3.41 | |
| 38 | GPT-5.2OpenAI | 75.8% | $1.45 | 10m 40s | 58.9 | $14.00 | |
| 39 | Gemini 3 Flash PreviewGoogle | 75.0% | $0.28 | 4m 45s | 75.8 | $3.00 | |
| 40 | Gemini 3.5 Flash LiteGoogle | 75.0% | $0.31 | 7m 20s | 33.6 | $2.50 | |
| 41 | MiniMax M3MiniMax | 75.0% | $0.42 | 12m 06s | 57.8 | $1.20 | |
| 42 | MiniMax M2.1MiniMax | 74.8% | $0.44 | 6m 26s | Not benchmarked | $1.20 | |
| 43 | MiniMax M2.5MiniMax | 74.2% | $0.46 | 6m 43s | 59.4 | $0.95 | |
| 44 | MiMo-V2.5-ProXiaomi | 74.0% | $0.02 | 7m 51s | 40.0 | $0.609 | |
| 45 | MiniMax M2.7MiniMax | 73.8% | $0.47 | 14m 46s | 28.4 | $1.20 | |
| 46 | Qwen3.6 PlusQwen | 73.4% | $0.87 | 7m 04s | 37.4 | $1.95 | |
| 47 | GPT-5.4 MiniOpenAI | 73.0% | $0.51 | 5m 26s | 28.2 | $4.50 | |
| 48 | GPT-5.2-CodexOpenAI | 72.4% | $0.31 | 3m 08s | 55.1 | $14.00 | |
| 49 | Grok 4.20xAI | 72.2% | $0.52 | 2m 15s | Not benchmarked | $2.50 | |
| 50 | Grok 4.3xAI | 71.4% | $0.22 | 1m 58s | 22.5 | $2.50 | |
| 51 | GLM 5Z.ai | 71.4% | $0.45 | 8m 34s | 59.7 | $3.20 | |
| 52 | Qwen3.5 Plus 2026-04-20Qwen | 71.2% | $0.96 | 8m 16s | Not benchmarked | $1.80 | |
| 53 | MiMo-V2.5Xiaomi | 71.0% | $0.01 | 4m 04s | 34.0 | $0.238 | |
| 54 | Claude Sonnet 4.5Anthropic | 70.0% | $0.81 | 6m 01s | 59.1 | $15.00 | |
| 55 | Kimi K2.5Moonshot AI | 70.0% | $0.17 | 5m 39s | 58.5 | $2.50 | |
| 56 | Qwen3.6 27BQwen | 70.0% | $0.91 | 17m 09s | Not benchmarked | $2.00 | |
| 57 | GPT-5.1OpenAI | 69.8% | $0.71 | 8m 21s | 53.3 | $10.00 | |
| 58 | GPT-5.4 NanoOpenAI | 69.8% | $0.10 | 4m 23s | Not benchmarked | $1.25 | |
| 59 | GLM 4.7Z.ai | 69.4% | $0.20 | 6m 06s | 33.6 | $1.98 | |
| 60 | Nemotron 3 UltraNVIDIA | 69.0% | Not published | 4m 43s | Not benchmarked | $2.40 | |
| 61 | GPT-5OpenAI | 69.0% | $0.27 | 3m 53s | 72.2 | $10.00 | |
| 62 | Qwen3.7 MaxQwen | 68.8% | $3.23 | 16m 16s | 45.1 | $4.425 | |
| 63 | DeepSeek V3.2DeepSeek | 67.6% | $0.67 | 8m 53s | 54.4 | $0.42 | |
| 64 | Claude Haiku 4.5Anthropic | 66.6% | $0.37 | 4m 02s | 49.8 | $5.00 | |
| 65 | Mistral Medium 3.5Mistral AI | 66.4% | $4.00 | 19m 04s | 9.2 | $7.50 | |
| 66 | Ling 3.0 FlashinclusionAI | 65.2% | $0.03 | 3m 16s | Not benchmarked | $0.063 | |
| 67 | Qwen3.5-FlashQwen | 64.4% | $0.26 | 6m 03s | 6.2 | $0.26 | |
| 68 | Gemini 3.1 Flash Lite PreviewGoogle | 62.8% | $0.11 | 2m 13s | 8.1 | $1.50 | |
| 69 | Devstral 2 2512Mistral AI | 62.8% | $0.75 | 4m 04s | 53.8 | $2.00 | |
| 70 | GPT-5 MiniOpenAI | 60.8% | $0.05 | 3m 07s | 59.8 | $2.00 | |
| 71 | Kimi K2 ThinkingMoonshot AI | 60.2% | $0.37 | 27m 12s | 63.4 | $2.50 | |
| 72 | Gemini 2.5 ProGoogle | 54.4% | $0.35 | 3m 28s | 37.0 | $10.00 | |
| 73 | Nemotron 3.5 LightningNVIDIA | 52.8% | $0.03 | 15m 25s | Not benchmarked | $0.20 | |
| 74 | Mistral Large 3 2512Mistral AI | 41.4% | $0.32 | 3m 07s | Not benchmarked | $1.50 | |
| 75 | gpt-oss-120bOpenAI | 33.6% | $0.17 | 4m 16s | 34.5 | $0.60 | |
| 76 | Command ACohere | 7.8% | $0.21 | 3m 48s | Not benchmarked | $10.00 |
Only models in our catalog are listed, and each appears once with its best published run. 132 rows were read from the source on October 21, 2026, and rows that match no catalog model are left out. Cost per test is what Vals AI reports one test cost on the run behind the result, in US dollars. Every other column is a figure the board publishes for the same run.