ALE-Bench leaderboard
- GPT-6 Astra leads
- 2951.3
- models measured
- 104
- lead over #2
- +207.6
- median
- 893.5
Leader on ALE-Bench · read October 21
# ALE-Bench, read October 21
leaders = {
"GPT-6 Astra": 2951.3, "GPT-6.1 Sol": 2743.8, "GPT-6 Sol": 2462.0,
}
On October 21, 2026, 104 of our catalog's models had a ALE-Bench result, and OpenAI GPT-6 Astra led with 2951.3, ahead of OpenAI GPT-6.1 Sol at 2743.8.
Optimization problems from the AtCoder Heuristic Contest, such as routing and scheduling, that are computationally hard and have no known exact solution. The number is the mean AtCoder performance rating over 40 problems with one round of self-refinement, as Sakana AI runs it for every model in one setup.
Role in the index
A coding board we show beside the Vibe Coding Index, not inside it: the index keeps the six boards it was defined on, so a score does not move every time a board is added. Read the methodology
| # | Model | Result | Cost per problem | Tokens per problem | Evidence | Coding index | Output $/Mtok |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 AstraOpenAI | 2951.3 | $2.8641 | 60,104 | 85.1 | $50.00 | |
| 2 | GPT-6.1 SolOpenAI | 2743.8 | $0.5968 | 62,549 | 82.4 | $10.00 | |
| 3 | GPT-6 SolOpenAI | 2462.0 | $0.844 | 87,318 | 76.0 | $10.00 | |
| 4 | GPT-5.6 SolOpenAI | 2176.9 | $1.5341 | 58,696 | 73.5 | $20.00 | |
| 5 | Claude Opus 5Anthropic | 2164.6 | $1.8905 | 78,951 | 83.0 | $25.00 | |
| 6 | Claude Opus 5.5Anthropic | 2147.4 | $0.2712 | 16,891 | 88.8 | $20.00 | |
| 7 | Claude Fable 5.1Anthropic | 2143.2 | $2.2949 | 49,229 | 84.2 | $50.00 | |
| 8 | Gemini 4 ArgonGoogle | 2100.1 | $1.9497 | 192,623 | 81.0 | $10.00 | |
| 9 | Claude Fable 5Anthropic | 2041.3 | $2.1088 | 45,507 | 79.3 | $50.00 | |
| 10 | GPT-5.6 TerraOpenAI | 1951.4 | $1.254 | 86,477 | 63.9 | $12.00 | |
| 11 | GPT-5.5OpenAI | 1943.0 | $1.5171 | 54,091 | 71.3 | $30.00 | |
| 12 | Claude Sonnet 5.5Anthropic | 1819.1 | $0.1109 | 14,424 | 87.3 | $10.00 | |
| 13 | Muse Spark 1.3Meta | 1688.3 | $0.2209 | 68,223 | 57.4 | $4.25 | |
| 14 | GPT-5.6 LunaOpenAI | 1667.4 | $0.3592 | 85,180 | 58.0 | $1.20 | |
| 15 | GPT-5.3-CodexOpenAI | 1655.2 | $0.5593 | 49,286 | 60.0 | $14.00 | |
| 16 | DeepSeek V4.1 Flash 1017DeepSeek | 1622.0 | $0.10 | 85,663 | 70.3 | $0.60 | |
| 17 | GPT-5.4OpenAI | 1607.0 | $0.6122 | 45,461 | 64.4 | $15.00 | |
| 18 | Muse Spark OpenMeta | 1590.1 | $0.3255 | 99,073 | 47.4 | $4.25 | |
| 19 | GPT-6 LunaOpenAI | 1576.9 | $0.018 | 38,984 | 57.3 | $0.50 | |
| 20 | Kimi K3Moonshot AI | 1524.5 | $1.2816 | 88,294 | 62.3 | $15.00 | |
| 21 | Grok 4.6xAI | 1508.1 | $0.2401 | 42,843 | 65.7 | $6.00 | |
| 22 | Claude Sonnet 5Anthropic | 1463.1 | $0.9605 | 67,361 | 57.7 | $10.00 | |
| 23 | DeepSeek V4 Pro 0813DeepSeek | 1403.2 | $0.0728 | 85,272 | 56.1 | $1.98 | |
| 24 | Gemini 3 Flash PreviewGoogle | 1367.2 | $0.0911 | 33,248 | 43.7 | $3.00 | |
| 25 | GLM 5.3Z.ai | 1317.4 | $0.284 | 67,123 | 70.6 | $4.40 | |
| 26 | Grok 4.5xAI | 1309.3 | $0.1768 | 32,347 | 58.6 | $6.00 | |
| 27 | DeepSeek V4 Flash 0731DeepSeek | 1306.1 | $0.0195 | 71,432 | 51.8 | $0.28 | |
| 28 | GPT-5.2-CodexOpenAI | 1299.9 | $0.5878 | 46,134 | 45.4 | $14.00 | |
| 29 | GPT-5.2OpenAI | 1293.5 | $0.2969 | 24,031 | 55.9 | $14.00 | |
| 30 | Gemini 3.8 FlashGoogle | 1269.7 | $0.3443 | 48,512 | 61.6 | $3.75 | |
| 31 | GPT-5.1-CodexOpenAI | 1244.9 | $0.5121 | 54,011 | 15.3 | $10.00 | |
| 32 | MiMo-V2.6-FlashXiaomi | 1244.2 | $0.0353 | 142,750 | 54.7 | $0.28 | |
| 33 | GPT-5.1-Codex-MaxOpenAI | 1228.3 | $0.3175 | 34,566 | Not benchmarked | $10.00 | |
| 34 | Mistral Large 4Mistral AI | 1213.2 | $0.4569 | 121,609 | 56.4 | $4.18 | |
| 35 | Qwen3.7 MaxQwen | 1189.4 | $0.3845 | 53,436 | 54.1 | $4.425 | |
| 36 | GPT-5.4 MiniOpenAI | 1188.6 | $0.2714 | 72,952 | 35.8 | $4.50 | |
| 37 | Gemini 3.1 Pro PreviewGoogle | 1160.6 | $0.3884 | 35,084 | 32.3 | $12.00 | |
| 38 | MiMo-V2.6-ProXiaomi | 1158.1 | $0.1084 | 152,523 | 65.4 | $0.87 | |
| 39 | Kimi K2.6Moonshot AI | 1092.7 | $0.1969 | 51,914 | 52.1 | $4.00 | |
| 40 | DeepSeek V4.1 FlashDeepSeek | 1092.3 | $0.0817 | 70,295 | 62.5 | $0.78 | |
| 41 | GLM 5.2Z.ai | 1047.0 | $0.1577 | 38,099 | 61.1 | $4.40 | |
| 42 | DeepSeek V4 Pro 0423DeepSeek | 1006.1 | $0.1481 | 45,318 | 38.7 | $2.716 | |
| 43 | GPT-5.4 NanoOpenAI | 1004.5 | $0.0271 | 24,378 | 44.8 | $1.25 | |
| 44 | Muse Glimmer 30BMeta | 962.5 | $0.0292 | 22,926 | Not benchmarked | $1.35 | |
| 45 | T InklingThinking Machines | 946.0 | $0.2025 | 55,717 | 31.5 | $4.05 | |
| 46 | Grok 4.3xAI | 944.2 | $0.0558 | 24,066 | 37.4 | $2.50 | |
| 47 | o3OpenAI | 933.5 | $0.1035 | 15,284 | Not benchmarked | $8.00 | |
| 48 | Gemma 4 26B A4BGoogle | 927.2 | $0.0097 | 27,350 | Not benchmarked | $0.60 | |
| 49 | Gemma 4 31BGoogle | 925.5 | $0.0047 | 14,037 | 39.0 | $0.40 | |
| 50 | Gemini 3.5 FlashGoogle | 911.0 | $0.4419 | 51,880 | 39.8 | $9.00 | |
| 51 | Gemini 3.7 FlashGoogle | 904.3 | $0.2436 | 35,110 | 58.0 | $3.75 | |
| 52 | MiMo-V2.5-ProXiaomi | 899.8 | $0.103 | 37,165 | 31.6 | $0.9152 | |
| 53 | GLM 5.1Z.ai | 887.1 | $0.1294 | 31,928 | 49.2 | $4.40 | |
| 54 | Kimi K2.7 CodeMoonshot AI | 886.2 | $0.1847 | 48,433 | 53.4 | $4.00 | |
| 55 | Hy3Tencent | 838.6 | $0.0263 | 45,985 | Not benchmarked | $0.33 | |
| 56 | o4 Mini HighOpenAI | 826.2 | $0.0447 | 12,544 | Not benchmarked | $4.40 | |
| 57 | Kimi K2.5Moonshot AI | 821.6 | $0.0475 | 18,681 | 38.6 | $2.85 | |
| 58 | GPT-5OpenAI | 807.6 | $0.0437 | 7,007 | 38.8 | $10.00 | |
| 59 | R1 0528DeepSeek | 804.1 | $0.0438 | 23,890 | Not benchmarked | $2.18 | |
| 60 | Gemini 3.1 Flash Lite PreviewGoogle | 797.7 | $0.0252 | 19,502 | 22.7 | $1.50 | |
| 61 | LongCat 2.0Meituan | 793.5 | $0.0359 | 32,768 | Not benchmarked | $1.20 | |
| 62 | Mercury 2Inception | 785.6 | $0.0075 | 12,149 | Not benchmarked | $0.75 | |
| 63 | Nemotron 3.5 LightningNVIDIA | 769.2 | $0.0093 | 39,216 | 22.6 | $0.147 | |
| 64 | GLM 5Z.ai | 765.6 | $0.0733 | 25,199 | 41.7 | $3.20 | |
| 65 | Gemini 3.5 Flash LiteGoogle | 765.3 | $0.0457 | 21,160 | 49.0 | $2.50 | |
| 66 | Mistral Medium 3.5Mistral AI | 764.0 | $0.273 | 38,906 | 25.5 | $7.50 | |
| 67 | DeepSeek V3.1 TerminusDeepSeek | 745.2 | $0.0188 | 20,988 | Not benchmarked | $1.0142 | |
| 68 | Gemini 3.6 FlashGoogle | 715.5 | $0.2445 | 35,204 | 62.5 | $3.75 | |
| 69 | Step 3.7 FlashStepFun | 694.1 | $0.0327 | 31,014 | Not benchmarked | $1.15 | |
| 70 | DeepSeek V4 Flash 0423DeepSeek | 678.2 | $0.0052 | 21,119 | Not benchmarked | $0.28 | |
| 71 | Qwen3.6 PlusQwen | 670.1 | $0.0571 | 21,734 | 44.1 | $1.95 | |
| 72 | MiniMax M3MiniMax | 640.0 | $0.1118 | 55,120 | 39.2 | $1.20 | |
| 73 | GLM 5 TurboZ.ai | 634.0 | $0.1095 | 29,856 | Not benchmarked | $4.00 | |
| 74 | MiniMax M2.1MiniMax | 623.8 | $0.0536 | 47,888 | Not benchmarked | $1.20 | |
| 75 | Qwen3.5 397B A17BQwen | 621.9 | $0.0398 | 13,771 | Not benchmarked | $2.34 | |
| 76 | MiniMax M2.5MiniMax | 618.2 | $0.0367 | 33,109 | 37.2 | $1.20 | |
| 77 | Qwen3.5 Plus 2026-04-20Qwen | 608.9 | $0.0257 | 13,436 | 39.5 | $1.80 | |
| 78 | MiniMax M2.7MiniMax | 599.3 | $0.0472 | 41,912 | 36.2 | $1.20 | |
| 79 | Kimi K2 ThinkingMoonshot AI | 597.5 | $0.0878 | 51,357 | Not benchmarked | $2.50 | |
| 80 | gpt-oss-120bOpenAI | 575.6 | $0.0013 | 5,041 | Not benchmarked | $0.50 | |
| 81 | gpt-oss-20bOpenAI | 566.0 | $0.0007 | 5,951 | Not benchmarked | $0.15 | |
| 82 | GPT-4.1OpenAI | 558.1 | $0.0269 | 5,651 | Not benchmarked | $8.00 | |
| 83 | MiMo-V2.5Xiaomi | 514.0 | $0.0376 | 21,575 | 50.0 | $0.336 | |
| 84 | Mistral Small 4Mistral AI | 497.6 | $0.0156 | 28,560 | Not benchmarked | $0.60 | |
| 85 | Qwen3 Coder 480B A35BQwen | 461.4 | $0.0037 | 5,450 | Not benchmarked | $1.50 | |
| 86 | Qwen3.7 FlashQwen | 420.1 | $0.0035 | 29,885 | Not benchmarked | $0.13 | |
| 87 | GLM 4.7Z.ai | 399.5 | $0.0432 | 22,000 | 55.1 | $2.20 | |
| 88 | Qwen3.5-27BQwen | 349.4 | $0.0419 | 20,324 | Not benchmarked | $1.56 | |
| 89 | GLM 4.5Z.ai | 344.8 | $0.0301 | 16,482 | Not benchmarked | $2.20 | |
| 90 | GLM 4.6Z.ai | 340.8 | $0.0188 | 10,883 | 8.8 | $2.20 | |
| 91 | Qwen3.6 FlashQwen | 326.4 | $0.0356 | 26,463 | Not benchmarked | $1.125 | |
| 92 | GLM 5.3 FlashZ.ai | 303.6 | $0.0109 | 23,975 | 57.7 | $0.50 | |
| 93 | Mercury 2.5Inception | 301.6 | $0.0062 | 10,710 | 1.6 | $0.15 | |
| 94 | Kimi K2 0905Moonshot AI | 267.1 | $0.0081 | 5,444 | Not benchmarked | $2.50 | |
| 95 | Qwen3.5-FlashQwen | 265.9 | $0.0122 | 32,977 | 41.1 | $0.26 | |
| 96 | Mistral Large 3 2512Mistral AI | 264.7 | $0.0051 | 5,482 | Not benchmarked | $1.50 | |
| 97 | Nova 2 LiteAmazon | 236.3 | $0.0042 | 4,220 | Not benchmarked | $2.50 | |
| 98 | Qwen3.5-35B-A3BQwen | 221.8 | $0.0379 | 21,782 | Not benchmarked | $1.30 | |
| 99 | Nemotron 3 SuperNVIDIA | 213.9 | $0.0082 | 18,918 | Not benchmarked | $0.45 | |
| 100 | Mistral Medium 3.1Mistral AI | 210.2 | $0.0074 | 6,182 | Not benchmarked | $2.00 | |
| 101 | Solar Pro 4Upstage | 179.3 | $0.0389 | 34,802 | Not benchmarked | $0.60 | |
| 102 | Llama 4 MaverickMeta Llama | 173.0 | $0.0014 | 4,326 | Not benchmarked | $1.00 | |
| 103 | Nova Premier 1.0Amazon | 147.4 | $0.0273 | 4,909 | Not benchmarked | $12.50 | |
| 104 | Codestral 2508Mistral AI | 137.8 | $0.0028 | 5,169 | Not benchmarked | $0.90 |
Only models in our catalog are listed, and each appears once with its best published run. 104 rows were read from the source on October 21, 2026, and rows that match no catalog model are left out. Cost per problem is the mean spend on one problem, in US dollars, as Sakana AI reports it for the same run. Every other column is a figure the board publishes for the same run.