WeirdML leaderboard
- GPT-6 Astra Pro leads
- 93.6%
- models measured
- 93
- lead over #2
- +0.3 pts
- median
- 52.2%
Leader on WeirdML · read October 22
# WeirdML, read October 22
leaders = {
"GPT-6 Astra Pro": 93.6%, "GPT-6 Astra": 93.3%, "Claude Fable 5.1": 92.9%,
}
On October 22, 2026, 107 of our catalog's models had a WeirdML result, and OpenAI GPT-6 Astra Pro led with 93.6%, ahead of OpenAI GPT-6 Astra at 93.3%.
Epoch AI collects its results; its benchmarking hub describes the tasks and links the authors.
Role in the index
A coding board we show beside the Vibe Coding Index, not inside it: the index keeps the six boards it was defined on, so a score does not move every time a board is added. Read the methodology
| # | Model | Result | Cost per run | Median lines of code | Evidence | Coding index | Output $/Mtok |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra ProOpenAI | 93.6% | $29.3058 | 521 | Not benchmarked | $50.00 | |
| 2 | GPT-6 AstraOpenAI | 93.3% | $7.7485 | 485 | 85.1 | $50.00 | |
| 3 | Claude Fable 5.1Anthropic | 92.9% | $11.6866 | 251 | 84.2 | $50.00 | |
| 4 | Claude Fable 5Anthropic | 91.9% | $8.4244 | 218 | 79.3 | $50.00 | |
| 5 | Claude Opus 5Anthropic | 91.8% | $4.742 | 245 | 83.0 | $25.00 | |
| 6 | GPT-5.6 Sol ProOpenAI | 89.4% | $3.8707 | 674 | Not benchmarked | $20.00 | |
| 7 | GPT-5.6 SolOpenAI | 88.8% | $1.2548 | 567 | 73.5 | $20.00 | |
| 8 | GPT-5.5OpenAI | 84.9% | $2.5683 | 517 | 71.3 | $30.00 | |
| 9 | Gemini 3.8 FlashGoogle | 84.8% | $0.5073 | 439 | 61.6 | $3.75 | |
| 10 | Claude Opus 4.8Anthropic | 82.9% | $2.3498 | 129 | 65.0 | $25.00 | |
| 11 | Kimi K3Moonshot AI | 82.6% | $2.0537 | 244 | 62.3 | $15.00 | |
| 12 | GPT-5.3-CodexOpenAI | 79.3% | $2.35 | 558 | 60.0 | $14.00 | |
| 13 | GPT-5.6 TerraOpenAI | 78.3% | $0.3119 | 177 | 63.9 | $12.00 | |
| 14 | Claude Opus 4.6Anthropic | 78.0% | $5.7794 | 169 | 57.2 | $25.00 | |
| 15 | GPT-5.4OpenAI | 77.7% | $5.7199 | 968 | 64.4 | $15.00 | |
| 16 | Claude Opus 4.7Anthropic | 76.4% | $1.8207 | 147 | 67.5 | $25.00 | |
| 17 | GLM 5.3Z.ai | 75.4% | $0.5351 | 244 | 70.6 | $4.40 | |
| 18 | GPT-5.2OpenAI | 72.2% | $2.0474 | 394 | 55.9 | $14.00 | |
| 19 | Gemini 3.1 Pro PreviewGoogle | 72.1% | $1.3602 | 212 | 32.3 | $12.00 | |
| 20 | GLM 5.2Z.ai | 70.1% | $0.5073 | 200 | 61.1 | $4.40 | |
| 21 | Claude Sonnet 5Anthropic | 68.8% | $0.8955 | 233 | 57.7 | $10.00 | |
| 22 | Grok 4.6xAI | 67.3% | $0.3348 | 205 | 65.7 | $6.00 | |
| 23 | DeepSeek V4 Pro 0813DeepSeek | 66.2% | $0.191 | 318 | 56.1 | $1.98 | |
| 24 | Claude Sonnet 4.6Anthropic | 66.1% | $2.8594 | 178 | 57.3 | $15.00 | |
| 25 | Claude Opus 4.5Anthropic | 63.7% | $0.7383 | 173 | 44.0 | $25.00 | |
| 26 | DeepSeek V4 Flash 0731DeepSeek | 63.0% | $0.0286 | 274 | 51.8 | $0.396 | |
| 27 | Gemini 3.5 FlashGoogle | 62.6% | $0.756 | 286 | 39.8 | $9.00 | |
| 28 | Gemini 3 Flash PreviewGoogle | 61.6% | $0.2216 | 149 | 43.7 | $3.00 | |
| 29 | GPT-5.6 LunaOpenAI | 60.9% | $0.20 | 289 | 58.0 | $1.20 | |
| 30 | GPT-5.1OpenAI | 60.8% | $0.6913 | 311 | 40.4 | $10.00 | |
| 31 | GPT-5OpenAI | 60.7% | $0.5932 | 328 | 38.8 | $10.00 | |
| 32 | GPT-5 ProOpenAI | 60.4% | $7.6476 | 369 | Not benchmarked | $120.00 | |
| 33 | Muse Spark 1.2Meta | 60.3% | $0.3801 | 272 | 55.7 | $4.25 | |
| 34 | GPT-5.4 MiniOpenAI | 60.3% | $1.291 | 503 | 35.8 | $4.50 | |
| 35 | o3 ProOpenAI | 58.2% | $2.5404 | 148 | Not benchmarked | $80.00 | |
| 36 | GPT-5.4 ProOpenAI | 57.4% | $0.2679 | 292 | Not benchmarked | $180.00 | |
| 37 | GLM 5.1Z.ai | 57.1% | $0.3175 | 178 | 49.2 | $4.40 | |
| 38 | Gemini 3.6 FlashGoogle | 56.1% | $0.6227 | 346 | 62.5 | $3.75 | |
| 39 | Kimi K2.6Moonshot AI | 55.9% | $0.3504 | 205 | 52.1 | $4.00 | |
| 40 | Kimi K2.7 CodeMoonshot AI | 54.1% | $0.2316 | 230 | 53.4 | $4.00 | |
| 41 | Gemini 2.5 ProGoogle | 54.0% | $0.4097 | 199 | 19.1 | $10.00 | |
| 42 | GPT-5 MiniOpenAI | 52.7% | $0.1098 | 296 | 34.2 | $2.00 | |
| 43 | o4 MiniOpenAI | 52.6% | $0.1957 | 122 | Not benchmarked | $4.40 | |
| 44 | o3OpenAI | 52.4% | $0.2261 | 134 | Not benchmarked | $8.00 | |
| 45 | Gemma 4 31BGoogle | 52.3% | $0.0133 | 126 | 39.0 | $0.40 | |
| 46 | Grok 4.20xAI | 52.3% | $0.1263 | 117 | 33.3 | $2.50 | |
| 47 | Gemini 3.1 Flash LiteGoogle | 52.2% | $0.0664 | 107 | Not benchmarked | $1.50 | |
| 48 | Grok 4.3xAI | 49.9% | $0.0903 | 141 | 37.4 | $2.50 | |
| 49 | GPT-5.4 NanoOpenAI | 49.2% | $0.0477 | 279 | 44.8 | $1.25 | |
| 50 | DeepSeek V4 Pro 0423DeepSeek | 48.9% | $0.3403 | 272 | 38.7 | $2.976 | |
| 51 | gpt-oss-120bOpenAI | 48.2% | $0.0335 | 255 | Not benchmarked | $0.50 | |
| 52 | GLM 5Z.ai | 48.2% | $0.0787 | 161 | 41.7 | $3.20 | |
| 53 | Claude Sonnet 4.5Anthropic | 47.7% | $0.3344 | 173 | 39.5 | $15.00 | |
| 54 | Grok 4.5xAI | 46.4% | $0.0805 | 86 | 58.6 | $6.00 | |
| 55 | o1OpenAI | 46.1% | $2.0287 | 152 | Not benchmarked | $60.00 | |
| 56 | Claude Opus 4.1Anthropic | 45.9% | $1.7556 | 241 | Not benchmarked | $75.00 | |
| 57 | DeepSeek V4 Flash 0423DeepSeek | 45.6% | $0.0146 | 204 | Not benchmarked | $0.28 | |
| 58 | Kimi K2.5Moonshot AI | 45.6% | $0.0526 | 166 | 38.6 | $2.85 | |
| 59 | Claude Haiku 4.5Anthropic | 45.4% | $0.0894 | 174 | 35.6 | $5.00 | |
| 60 | Claude Opus 4RetiredAnthropic | 43.7% | $1.7353 | 233 | Not benchmarked | $75.00 | |
| 61 | o3 MiniOpenAI | 43.7% | $0.1745 | 158 | Not benchmarked | $4.40 | |
| 62 | Nemotron 3 UltraNVIDIA | 43.5% | $0.0772 | 232 | 30.0 | $2.40 | |
| 63 | Mercury 2Inception | 43.2% | $0.0298 | 201 | Not benchmarked | $0.75 | |
| 64 | Kimi K2 ThinkingMoonshot AI | 42.8% | $0.0809 | 195 | Not benchmarked | $2.50 | |
| 65 | R1 0528DeepSeek | 41.6% | $0.0711 | 174 | Not benchmarked | $2.18 | |
| 66 | Qwen3 Coder 480B A35BQwen | 41.2% | $0.0726 | 174 | Not benchmarked | $1.50 | |
| 67 | Qwen3 235B A22B Thinking 2507Qwen | 41.0% | $0.0199 | 127 | Not benchmarked | $3.50 | |
| 68 | gpt-oss-20bOpenAI | 40.9% | $0.0108 | 166 | Not benchmarked | $0.15 | |
| 69 | GLM 4.5Z.ai | 40.6% | $0.0735 | 246 | Not benchmarked | $2.20 | |
| 70 | Claude 3.5 SonnetRetiredAnthropic | 40.0% | $0.2483 | 168 | Not benchmarked | $15.00 | |
| 71 | Qwen3.5-27BQwen | 39.5% | $0.0404 | 171 | Not benchmarked | $1.56 | |
| 72 | DeepSeek V3.2 ExpDeepSeek | 39.5% | $0.0177 | 219 | Not benchmarked | $0.41 | |
| 73 | Kimi K2 0711Moonshot AI | 39.4% | $0.0343 | 186 | Not benchmarked | $2.30 | |
| 74 | GPT-4.1OpenAI | 39.0% | $0.097 | 138 | Not benchmarked | $8.00 | |
| 75 | Gemini 3.5 Flash LiteGoogle | 39.0% | $0.0468 | 130 | 49.0 | $2.50 | |
| 76 | Qwen3 235B A22B Instruct 2507Qwen | 38.7% | $0.0134 | 144 | Not benchmarked | $0.75 | |
| 77 | GPT-5 NanoOpenAI | 38.1% | $0.0262 | 191 | Not benchmarked | $0.40 | |
| 78 | Nemotron 3 SuperNVIDIA | 38.0% | $0.0122 | 178 | Not benchmarked | $0.45 | |
| 79 | GPT-4.1 MiniOpenAI | 37.6% | $0.0166 | 143 | Not benchmarked | $1.60 | |
| 80 | Grok 3RetiredxAI | 37.2% | $0.2477 | 123 | Not benchmarked | $15.00 | |
| 81 | MiniMax M2.7MiniMax | 37.0% | $0.0349 | 218 | 36.2 | $1.20 | |
| 82 | Kimi K2 0905Moonshot AI | 36.7% | $0.0418 | 211 | Not benchmarked | $2.50 | |
| 83 | R1RetiredDeepSeek | 36.5% | $0.0785 | 122 | Not benchmarked | $2.50 | |
| 84 | o1-miniRetiredOpenAI | 36.3% | $0.1113 | 156 | Not benchmarked | $4.40 | |
| 85 | DeepSeek V3 0324DeepSeek | 36.1% | $0.0139 | 144 | Not benchmarked | $1.07 | |
| 86 | Gemma 4 26B A4BGoogle | 35.2% | $0.0208 | 146 | Not benchmarked | $0.60 | |
| 87 | Qwen3.6 35B A3BQwen | 34.5% | $0.0596 | 147 | Not benchmarked | $1.00 | |
| 88 | Qwen3 Coder NextQwen | 34.4% | $0.0328 | 201 | Not benchmarked | $0.80 | |
| 89 | T InklingThinking Machines | 32.3% | $0.1255 | 96 | 31.5 | $4.05 | |
| 90 | Claude 3.5 HaikuRetiredAnthropic | 30.7% | $0.0575 | 139 | Not benchmarked | $4.00 | |
| 91 | Qwen3 30B A3BQwen | 29.8% | $0.0081 | 103 | Not benchmarked | $0.50 | |
| 92 | Gemini 2.0 FlashRetiredGoogle | 25.8% | $0.0067 | 117 | Not benchmarked | $0.40 | |
| 93 | GPT-4o (2024-11-20)OpenAI | 25.1% | $0.0836 | 108 | Not benchmarked | $10.00 | |
| 94 | Gemini 1.5 FlashRetiredGoogle | 24.9% | $0.0028 | 70 | Not benchmarked | $0.30 | |
| 95 | Llama 4 MaverickMeta Llama | 24.5% | $0.0061 | 90 | Not benchmarked | $1.00 | |
| 96 | Grok 2RetiredxAI | 22.2% | $0.1141 | 113 | Not benchmarked | $10.00 | |
| 97 | Gemini 1.5 ProRetiredGoogle | 22.2% | $0.0505 | 80 | Not benchmarked | $5.00 | |
| 98 | Claude 3 OpusRetiredAnthropic | 19.2% | $0.8568 | 92 | Not benchmarked | $75.00 | |
| 99 | GPT-4.1 NanoOpenAI | 19.0% | $0.0026 | 88 | Not benchmarked | $0.40 | |
| 100 | Qwen2.5 72B InstructQwen | 16.0% | $0.0113 | 101 | Not benchmarked | $0.40 | |
| 101 | Llama 3.3 70B InstructMeta Llama | 14.4% | $0.0015 | 102 | Not benchmarked | $0.72 | |
| 102 | GPT-4o-mini (2024-07-18)OpenAI | 11.8% | $0.0053 | 85 | Not benchmarked | $0.60 | |
| 103 | Claude 3 SonnetRetiredAnthropic | 10.2% | $0.1783 | 90 | Not benchmarked | $15.00 | |
| 104 | Claude 3 HaikuRetiredAnthropic | 9.8% | $0.0141 | 89 | Not benchmarked | $1.25 | |
| 105 | Llama 3.1 70B InstructMeta Llama | 9.0% | $0.0032 | 86 | Not benchmarked | $0.56 | |
| 106 | Claude 2RetiredAnthropic | 7.1% | $0.1725 | 37 | Not benchmarked | $24.00 | |
| 107 | Llama 3.1 8B InstructMeta Llama | 1.7% | $0.0004 | 78 | Not benchmarked | $0.08 |
Only models in our catalog are listed, and each appears once with its best published run. 215 rows were read from the source on October 22, 2026, and rows that match no catalog model are left out. Cost per run is what one run of the benchmark cost, in US dollars, as Epoch AI records it beside the result. Every other column is a figure the board publishes for the same run.