Cost per solved task
- models with a cost
- 84
- median per solved task
- $0.64
Cost per solved task · read October 20
DeepSeek V4 Flash 0731, the cheapest working answer
On October 20, 2026, GPT-4.1 Mini costs $1.818 per solved SWE-bench Verified task and GPT-4.1 costs $0.37, although GPT-4.1 Mini's tokens are 5.0x cheaper. We can compute the number for 84 of 356 models.
A price list says what tokens cost. It does not say how many tokens a model burns on a task, or how often the attempt works. Both change what you pay for a result.
Cost per solved task against coding index
Left is cheaper, up is better. Each dot is one model with both numbers.
74 of 356 models have both numbers. Cost per solved task is on a log scale, so each gridline is two to two and a half times the one before.
- OpenAI
- Qwen
- Mistral AI
- DeepSeek
- Anthropic
- MiniMax
- Meta, Meta Llama
- Xiaomi
- Z.ai, Moonshot AI, xAI
- Other makers
- Faded: 1 benchmark. Solid: 2 or more.
- Frontier: the best index available at or below each cost
Cheap is left, good is up. MiMo-V2.5, GLM 5.3 Flash, GLM 5.3, Gemini 3 Flash Preview, Grok 4.6, GPT-5.6 Sol, Claude Opus 5 and Claude Fable 5 sit on the frontier: nothing else is both cheaper and higher. Paying more does not move a model up: nothing that costs more than $2.16, the price of the highest score (82.1), beats 70.5. The dearest point, Mistral Medium 3.5 at $6.02, scores 9.2.
| Model | Maker | Cost per solved task | Index | Benchmarks |
|---|---|---|---|---|
| MiMo-V2.5 | Xiaomi | $0.01 | 34.0 | 1 |
| GLM 5.3 Flash | Z.ai | $0.02 | 71.0 | 2 |
| MiMo-V2.5-Pro | Xiaomi | $0.03 | 40.0 | 1 |
| GPT-5.6 Luna | OpenAI | $0.04 | 61.5 | 2 |
| GPT-5 Mini | OpenAI | $0.08 | 59.8 | 1 |
| DeepSeek V4 Pro 0813 | DeepSeek | $0.10 | 54.6 | 1 |
| GPT-5 Nano | OpenAI | $0.109 | 34.8 | 1 |
| Inkling Small | Thinking Machines | $0.16 | 29.9 | 1 |
| Gemini 3.1 Flash Lite Preview | $0.18 | 8.1 | 1 | |
| Kimi K2.5 | Moonshot AI | $0.24 | 58.5 | 2 |
| GLM 4.7 | Z.ai | $0.29 | 33.6 | 1 |
| Grok 4.3 | xAI | $0.31 | 22.5 | 1 |
| Kimi K2.6 | Moonshot AI | $0.35 | 44.2 | 1 |
| Kimi K2.7 Code | Moonshot AI | $0.35 | 39.1 | 1 |
| GLM 5.3 | Z.ai | $0.36 | 73.1 | 2 |
| GPT-4.1 | OpenAI | $0.37 | 46.5 | 2 |
| Gemini 3 Flash Preview | $0.37 | 75.8 | 1 | |
| GPT-5 | OpenAI | $0.39 | 72.2 | 3 |
| Qwen3.5-Flash | Qwen | $0.40 | 6.2 | 1 |
| Gemini 3.5 Flash Lite | $0.41 | 33.6 | 1 | |
| GPT-5.6 Terra | OpenAI | $0.42 | 69.0 | 2 |
| Muse Spark 1.1 | Meta | $0.43 | 48.9 | 1 |
| GPT-5.2-Codex | OpenAI | $0.43 | 55.1 | 2 |
| Gemini 2.5 Flash | $0.466 | 28.7 | 1 | |
| gpt-oss-120b | OpenAI | $0.51 | 34.5 | 2 |
| Claude Haiku 4.5 | Anthropic | $0.56 | 49.8 | 3 |
| MiniMax M3 | MiniMax | $0.56 | 57.8 | 2 |
| DeepSeek V4 Pro 0423 | DeepSeek | $0.57 | 35.3 | 1 |
| o3 | OpenAI | $0.571 | 70.7 | 2 |
| GPT-5.3-Codex | OpenAI | $0.59 | 29.5 | 1 |
| GLM 5.1 | Z.ai | $0.60 | 44.0 | 1 |
| Kimi K2 Thinking | Moonshot AI | $0.61 | 63.4 | 1 |
| MiniMax M2.5 | MiniMax | $0.62 | 59.4 | 2 |
| Grok 4.5 | xAI | $0.62 | 75.5 | 2 |
| GLM 5 | Z.ai | $0.63 | 59.7 | 2 |
| Gemini 2.5 Pro | $0.64 | 37.0 | 2 | |
| Muse Spark 1.2 | Meta | $0.64 | 65.1 | 2 |
| MiniMax M2.7 | MiniMax | $0.64 | 28.4 | 1 |
| GPT-5.4 Mini | OpenAI | $0.70 | 28.2 | 1 |
| MiniMax M2 | MiniMax | $0.702 | 45.3 | 2 |
| Inkling | Thinking Machines | $0.76 | 49.7 | 2 |
| Kimi K3 | Moonshot AI | $0.81 | 61.8 | 2 |
| Grok 4.6 | xAI | $0.82 | 76.4 | 2 |
| GLM 5.2 | Z.ai | $0.86 | 57.7 | 1 |
| GPT-5.1-Codex | OpenAI | $0.892 | 50.5 | 2 |
| DeepSeek V3.2 | DeepSeek | $0.99 | 54.4 | 2 |
| Gemini 3.1 Pro Preview | $0.99 | 35.3 | 1 | |
| GPT-5.1 | OpenAI | $1.02 | 53.3 | 2 |
| GPT-5.4 | OpenAI | $1.02 | 37.7 | 1 |
| Claude Sonnet 4.5 | Anthropic | $1.16 | 59.1 | 2 |
| Devstral 2 2512 | Mistral AI | $1.19 | 53.8 | 1 |
| Qwen3.6 Plus | Qwen | $1.19 | 37.4 | 1 |
| GPT-5.6 Sol | OpenAI | $1.20 | 76.7 | 2 |
| Gemini 3.5 Flash | $1.21 | 42.8 | 1 | |
| Qwen3.8 27B | Qwen | $1.26 | 73.6 | 2 |
| Llama 4 Scout | Meta Llama | $1.276 | 9.1 | 1 |
| Qwen3.8 Max (0902) | Qwen | $1.32 | 63.5 | 2 |
| Claude Opus 5 | Anthropic | $1.33 | 81.8 | 2 |
| Claude Opus 4.5 | Anthropic | $1.37 | 66.8 | 2 |
| Gemini 3.6 Flash | $1.49 | 48.3 | 1 | |
| Llama 4 Maverick | Meta Llama | $1.491 | 18.1 | 2 |
| Claude Opus 4.6 | Anthropic | $1.56 | 66.9 | 2 |
| GPT-5.5 | OpenAI | $1.65 | 67.8 | 2 |
| Claude Sonnet 4.6 | Anthropic | $1.68 | 57.8 | 2 |
| Gemini 3.7 Flash | $1.78 | 69.9 | 2 | |
| GPT-4.1 Mini | OpenAI | $1.818 | 28.5 | 2 |
| Claude Sonnet 5 | Anthropic | $1.87 | 63.5 | 2 |
| GPT-5.2 | OpenAI | $1.91 | 58.9 | 2 |
| Claude Fable 5 | Anthropic | $2.16 | 82.1 | 2 |
| Claude Opus 4.8 | Anthropic | $2.17 | 66.0 | 2 |
| Gemini 3.8 Flash | $2.74 | 70.5 | 2 | |
| Claude Opus 4.7 | Anthropic | $2.95 | 51.1 | 1 |
| Qwen3.7 Max | Qwen | $4.69 | 45.1 | 1 |
| Mistral Medium 3.5 | Mistral AI | $6.02 | 9.2 | 1 |
Source: OpenRouter, Aider polyglot leaderboard, LMArena WebDev, SWE-bench (swebench.com), LiveBench, Vals AI boards. Fetched October 20, 2026.
The formula
The cost of an attempt is tokens used times the price per token. So three things decide the result, the price, the tokens a model spends, and how often it gets the task right. A price list shows only the first.
A worked example: GPT-4.1 and GPT-4.1 Mini
- Price. GPT-4.1 Mini charges $1.60 per million output tokens against $8.00, 5.0x less. Its input price is 5.0x less as well.
- Cost of one attempt. SWE-bench Verified, as swebench.com publishes it, gives what a run spent on one task, $0.1463 for GPT-4.1 and $0.4352 for GPT-4.1 Mini. The cheaper model's attempt costs 3.0x more. Since each token costs 5.0x less, it must be spending about 15 times the billed token volume.
- Pass rate. GPT-4.1 resolves 39.6% of tasks and GPT-4.1 Mini 23.9%, so GPT-4.1 succeeds 1.7x as often.
- Divide. $0.1463 divided by 0.3958 is $0.37. $0.4352 divided by 0.2394 is $1.818.
| GPT-4.1 | GPT-4.1 Mini | Difference | |
|---|---|---|---|
| Output price per Mtok | $8.00 | $1.60 | 5.0x lower |
| Tasks solved, SWE-bench Verified | 39.6% | 23.9% | 1.7x lower |
| Cost of one attempt | $0.1463 | $0.4352 | 3.0x higher |
| Cost per solved task4.9x higher | $0.37 | $1.818 | 4.9x higher |
The attempt costs 3.0x more and succeeds 1.7x less often, which compounds to 4.9x. The model that is 5.0x cheaper on the price list is 4.9x dearer per working answer.
How this differs from Artificial Analysis
Artificial Analysis publishes a Cost per Task, which its site describes as the weighted average cost in USD per Intelligence Index task. That is spend per task the model is given, not per task it solves. Ours is spend per task the model solves: the cost of one attempt divided by the share of attempts that succeed.
The two diverge when pass rates differ. On the cost of one attempt, GPT-4.1 Mini is 3.0x the price of GPT-4.1. Per solved task it is 4.9x, because it also fails more often. A model that is cheap per attempt and fails often looks good on a per-task-given number and bad on ours, and ours is the one that matches what you pay for working code.
The two also rest on different tasks. Theirs is built on the Intelligence Index, ours on SWE-bench Verified, so we do not set one beside the other figure for figure.
Their own description of the metric is on artificialanalysis.ai.
DeepSeek V4 Flash 0731 is cheapest at $0.01. Vals AI prints the cost of one attempt to the cent, so a number this small is approximate. inclusionAI Ling 3.0 Flash has the lowest output price of the 84 at $0.063 per million tokens, yet DeepSeek V4 Flash 0731, at $1.32, is the cheapest per solved task. A price list and a bill rank models differently.
| # | Model | Cost per solved task | One attempt | Resolved | Board | Output $/Mtok | Evidence | Index |
|---|---|---|---|---|---|---|---|---|
| 1 | DeepSeek V4 Flash 0731 | $0.01 | $0.01 | 88.8% | Vals AI | $1.32 | Not benchmarked | |
| 2 | Xiaomi MiMo-V2.5 | $0.01 | $0.01 | 71.0% | Vals AI | $0.28 | 34.0 | |
| 3 | Z.ai GLM 5.3 Flash | $0.02 | $0.02 | 92.0% | Vals AI | $0.16 | 71.0 | |
| 4 | Xiaomi MiMo-V2.5-Pro | $0.03 | $0.02 | 74.0% | Vals AI | $0.87 | 40.0 | |
| 5 | OpenAI GPT-5.6 Luna | $0.04 | $0.04 | 93.0% | Vals AI | $1.20 | 61.5 | |
| 6 | inclusionAI Ling 3.0 Flash | $0.05 | $0.03 | 65.2% | Vals AI | $0.063 | Not benchmarked | |
| 7 | NVIDIA Nemotron 3.5 Lightning | $0.06 | $0.03 | 52.8% | Vals AI | $0.20 | Not benchmarked | |
| 8 | OpenAI GPT-5 Mini | $0.08 | $0.05 | 60.8% | Vals AI | $2.00 | 59.8 | |
| 9 | DeepSeek V4 Pro 0813 | $0.10 | $0.10 | 96.4% | Vals AI | $1.98 | 54.6 | |
| 10 | OpenAI GPT-5 Nano | $0.109 | $0.0381 | 34.8% | swebench.com | $0.40 | 34.8 | |
| 11 | OpenAI GPT-5.4 Nano | $0.14 | $0.10 | 69.8% | Vals AI | $1.25 | Not benchmarked | |
| 12 | Thinking Machines Inkling Small | $0.16 | $0.13 | 82.2% | Vals AI | $1.20 | 29.9 | |
| 13 | Google Gemini 3.1 Flash Lite Preview | $0.18 | $0.11 | 62.8% | Vals AI | $1.50 | 8.1 | |
| 14 | MoonshotAI Kimi K2.5 | $0.24 | $0.17 | 70.0% | Vals AI | $2.50 | 58.5 | |
| 15 | Z.ai GLM 4.7 | $0.29 | $0.20 | 69.4% | Vals AI | $1.98 | 33.6 | |
| 16 | SpaceXAI Grok 4.3 | $0.31 | $0.22 | 71.4% | Vals AI | $2.50 | 22.5 | |
| 17 | MoonshotAI Kimi K2.6 | $0.35 | $0.27 | 76.2% | Vals AI | $3.41 | 44.2 | |
| 18 | MoonshotAI Kimi K2.7 Code | $0.35 | $0.27 | 78.2% | Vals AI | $3.35 | 39.1 | |
| 19 | Z.ai GLM 5.3 | $0.36 | $0.34 | 95.4% | Vals AI | $1.40 | 73.1 | |
| 20 | OpenAI GPT-4.1 | $0.37 | $0.1463 | 39.6% | swebench.com | $8.00 | 46.5 | |
| 21 | Google Gemini 3 Flash Preview | $0.37 | $0.28 | 75.0% | Vals AI | $3.00 | 75.8 | |
| 22 | OpenAI GPT-5 | $0.39 | $0.27 | 69.0% | Vals AI | $10.00 | 72.2 | |
| 23 | Qwen3.5-Flash | $0.40 | $0.26 | 64.4% | Vals AI | $0.26 | 6.2 | |
| 24 | Google Gemini 3.5 Flash Lite | $0.41 | $0.31 | 75.0% | Vals AI | $2.50 | 33.6 | |
| 25 | OpenAI GPT-5.6 Terra | $0.42 | $0.40 | 95.4% | Vals AI | $12.00 | 69.0 | |
| 26 | Meta Muse Spark 1.1 | $0.43 | $0.35 | 82.0% | Vals AI | $4.25 | 48.9 | |
| 27 | OpenAI GPT-5.2-Codex | $0.43 | $0.31 | 72.4% | Vals AI | $14.00 | 55.1 | |
| 28 | Google Gemini 2.5 Flash | $0.466 | $0.1339 | 28.7% | swebench.com | $2.50 | 28.7 | |
| 29 | OpenAI gpt-oss-120b | $0.51 | $0.17 | 33.6% | Vals AI | $0.60 | 34.5 | |
| 30 | Anthropic Claude Haiku 4.5 | $0.56 | $0.37 | 66.6% | Vals AI | $5.00 | 49.8 | |
| 31 | MiniMax M3 | $0.56 | $0.42 | 75.0% | Vals AI | $1.20 | 57.8 | |
| 32 | DeepSeek V4 Pro 0423 | $0.57 | $0.44 | 77.4% | Vals AI | $1.9105 | 35.3 | |
| 33 | OpenAI o3 | $0.571 | $0.3337 | 58.4% | swebench.com | $8.00 | 70.7 | |
| 34 | MiniMax M2.1 | $0.59 | $0.44 | 74.8% | Vals AI | $1.20 | Not benchmarked | |
| 35 | OpenAI GPT-5.3-Codex | $0.59 | $0.46 | 78.0% | Vals AI | $14.00 | 29.5 | |
| 36 | Z.ai GLM 5.1 | $0.60 | $0.46 | 76.4% | Vals AI | $3.036 | 44.0 | |
| 37 | MoonshotAI Kimi K2 Thinking | $0.61 | $0.37 | 60.2% | Vals AI | $2.50 | 63.4 | |
| 38 | MiniMax M2.5 | $0.62 | $0.46 | 74.2% | Vals AI | $0.95 | 59.4 | |
| 39 | SpaceXAI Grok 4.5 | $0.62 | $0.54 | 86.6% | Vals AI | $6.00 | 75.5 | |
| 40 | Z.ai GLM 5 | $0.63 | $0.45 | 71.4% | Vals AI | $1.92 | 59.7 | |
| 41 | Google Gemini 2.5 Pro | $0.64 | $0.35 | 54.4% | Vals AI | $10.00 | 37.0 | |
| 42 | Meta Muse Spark 1.2 | $0.64 | $0.55 | 86.6% | Vals AI | $4.25 | 65.1 | |
| 43 | MiniMax M2.7 | $0.64 | $0.47 | 73.8% | Vals AI | $1.20 | 28.4 | |
| 44 | OpenAI GPT-5.4 Mini | $0.70 | $0.51 | 73.0% | Vals AI | $4.50 | 28.2 | |
| 45 | MiniMax M2 | $0.702 | $0.4283 | 61.0% | swebench.com | $1.02 | 45.3 | |
| 46 | SpaceXAI Grok 4.20 | $0.72 | $0.52 | 72.2% | Vals AI | $2.50 | Not benchmarked | |
| 47 | Thinking Machines Inkling | $0.76 | $0.59 | 77.6% | Vals AI | $4.05 | 49.7 | |
| 48 | Mistral Large 3 2512 | $0.77 | $0.32 | 41.4% | Vals AI | $1.50 | Not benchmarked | |
| 49 | MoonshotAI Kimi K3 | $0.81 | $0.76 | 93.4% | Vals AI | $11.936 | 61.8 | |
| 50 | SpaceXAI Grok 4.6 | $0.82 | $0.78 | 95.6% | Vals AI | $6.00 | 76.4 | |
| 51 | Z.ai GLM 5.2 | $0.86 | $0.71 | 82.8% | Vals AI | $2.42 | 57.7 | |
| 52 | OpenAI GPT-5.1-Codex | $0.892 | $0.5889 | 66.0% | swebench.com | $10.00 | 50.5 | |
| 53 | DeepSeek V3.2 | $0.99 | $0.67 | 67.6% | Vals AI | $0.42 | 54.4 | |
| 54 | Google Gemini 3.1 Pro Preview | $0.99 | $0.78 | 78.8% | Vals AI | $12.00 | 35.3 | |
| 55 | OpenAI GPT-5.1 | $1.02 | $0.71 | 69.8% | Vals AI | $10.00 | 53.3 | |
| 56 | OpenAI GPT-5.4 | $1.02 | $0.80 | 78.2% | Vals AI | $15.00 | 37.7 | |
| 57 | Anthropic Claude Sonnet 4.5 | $1.16 | $0.81 | 70.0% | Vals AI | $15.00 | 59.1 | |
| 58 | Mistral Devstral 2 2512 | $1.19 | $0.75 | 62.8% | Vals AI | $2.00 | 53.8 | |
| 59 | Qwen3.6 Plus | $1.19 | $0.87 | 73.4% | Vals AI | $1.95 | 37.4 | |
| 60 | OpenAI GPT-5.6 Sol | $1.20 | $1.15 | 96.2% | Vals AI | $20.00 | 76.7 | |
| 61 | Google Gemini 3.5 Flash | $1.21 | $0.95 | 78.8% | Vals AI | $9.00 | 42.8 | |
| 62 | Qwen3.8 27B | $1.26 | $1.08 | 86.0% | Vals AI | $2.20 | 73.6 | |
| 63 | Meta Llama 4 Scout | $1.276 | $0.1156 | 9.1% | swebench.com | $0.30 | 9.1 | |
| 64 | Qwen3.6 27B | $1.30 | $0.91 | 70.0% | Vals AI | $2.00 | Not benchmarked | |
| 65 | Qwen3.8 Max (0902) | $1.32 | $1.13 | 85.6% | Vals AI | $6.00 | 63.5 | |
| 66 | Anthropic Claude Opus 5 | $1.33 | $1.29 | 97.0% | Vals AI | $25.00 | 81.8 | |
| 67 | Qwen3.5 Plus 2026-04-20 | $1.35 | $0.96 | 71.2% | Vals AI | $1.80 | Not benchmarked | |
| 68 | Anthropic Claude Opus 4.5 | $1.37 | $1.05 | 76.4% | Vals AI | $25.00 | 66.8 | |
| 69 | Google Gemini 3.6 Flash | $1.49 | $1.19 | 79.6% | Vals AI | $3.75 | 48.3 | |
| 70 | Meta Llama 4 Maverick | $1.491 | $0.3137 | 21.0% | swebench.com | $0.6525 | 18.1 | |
| 71 | Anthropic Claude Opus 4.6 | $1.56 | $1.22 | 78.2% | Vals AI | $25.00 | 66.9 | |
| 72 | OpenAI GPT-5.5 | $1.65 | $1.36 | 82.6% | Vals AI | $30.00 | 67.8 | |
| 73 | Anthropic Claude Sonnet 4.6 | $1.68 | $1.30 | 77.4% | Vals AI | $15.00 | 57.8 | |
| 74 | Google Gemini 3.7 Flash | $1.78 | $1.44 | 80.8% | Vals AI | $3.75 | 69.9 | |
| 75 | OpenAI GPT-4.1 Mini | $1.818 | $0.4352 | 23.9% | swebench.com | $1.60 | 28.5 | |
| 76 | Anthropic Claude Sonnet 5 | $1.87 | $1.49 | 79.6% | Vals AI | $10.00 | 63.5 | |
| 77 | OpenAI GPT-5.2 | $1.91 | $1.45 | 75.8% | Vals AI | $14.00 | 58.9 | |
| 78 | Anthropic Claude Fable 5 | $2.16 | $2.05 | 95.0% | Vals AI | $50.00 | 82.1 | |
| 79 | Anthropic Claude Opus 4.8 | $2.17 | $1.92 | 88.6% | Vals AI | $25.00 | 66.0 | |
| 80 | Cohere Command A | $2.69 | $0.21 | 7.8% | Vals AI | $10.00 | Not benchmarked | |
| 81 | Google Gemini 3.8 Flash | $2.74 | $2.19 | 80.0% | Vals AI | $4.125 | 70.5 | |
| 82 | Anthropic Claude Opus 4.7 | $2.95 | $2.42 | 82.0% | Vals AI | $25.00 | 51.1 | |
| 83 | Qwen3.7 Max | $4.69 | $3.23 | 68.8% | Vals AI | $4.425 | 45.1 | |
| 84 | Mistral Medium 3.5 | $6.02 | $4.00 | 66.4% | Vals AI | $7.50 | 9.2 |
Attempt cost is what the benchmark run spent when it ran. List price is today's price on OpenRouter, so the two can drift apart for a model whose price has changed.
Why only 84 of 356 models
Two boards we read publish what one SWE-bench Verified attempt cost. Vals AI runs current models on one shared harness and gives the cost for 75 of ours; swebench.com fills in 9 that Vals AI has not run. The cost and the pass rate always come from the same board, never one from each. Between them the two list 96 models from our catalog: 12 have a pass rate and no cost, and the remaining 260 have no entry on either.
Aider also publishes a run cost, for 19 models, but only 5 of them overlap with the 84 above, and a different benchmark means different tasks and a different price per task. We do not combine them and we do not estimate. Every other model shows "Not computable" with the reason on its own page.
What it does not tell you
The tasks are GitHub issues from open source Python projects. Your work will use more or fewer tokens, so use the number to compare models with each other, not to forecast a bill. For a forecast, enter your own volumes in the cost calculator. Open the cost calculator
Questions about cost per solved task
- What is cost per solved task?
- It is the cost of one attempt divided by the share of attempts that succeed. On October 20, 2026, we could compute it for 84 of 356 models, using what one SWE-bench Verified attempt cost and the share of tasks resolved, both from one board: Vals AI for 75 models, swebench.com for 9 more.
- Why can a cheaper model cost more per answer?
- Because the price per token is one of three factors. A cheaper model can spend more tokens on each attempt and fail more often, and both raise what a working answer costs. The worked example on this page shows it step by step.
- Which model has the lowest cost per solved task?
- DeepSeek V4 Flash 0731, at $0.01 per solved SWE-bench Verified task on October 20, 2026. Vals AI prints the cost of one attempt to the cent, so a number this small is approximate. Only 84 models have the number, so a model missing from the list is unmeasured, not worse.
- How is cost per solved task different from Artificial Analysis's Cost per Task?
- Theirs averages spend per task a model is given, on their Intelligence Index. Ours divides spend by the share of SWE-bench Verified tasks the model solves, so frequent failure makes a model dearer.