| SWE-bench Multimodal (Vals AI) | 33.3% | $1.02 per test | Duration, as printed 13m 09s |
| Vibe Code Bench 1-100 (Vals AI) | 9.2% | $3.03 per test | Duration, as printed 26m 10s |
| SkillsBench (Vals AI) | 51.5% | $0.50 per test | Duration, as printed 15m 28s |
| CyberBench (Vals AI) | 65.0% | $3.41 per test | Duration, as printed 40m 56s |
| Finance Agent (Vals AI) | 48.3% | $0.32 per test | Duration, as printed 8m 17s |
| Tax Agent Bench (Vals AI) | 20.6% | $0.16 per test | Duration, as printed 5m 15s |
| Legal Research Bench (Vals AI) | 29.8% | $0.34 per test | Duration, as printed 13m 34s |
| Excel Modeling Benchmark (Vals AI) | 47.8% | $2.09 per test | Duration, as printed 31m 30s |
| Public Benefits Bench (Vals AI) | 64.1% | $0.25 per test | Nothing else |
| Vals Multimodal Index | 60.0% | $1.08 per test | Duration, as printed 17m 56s |
| OSWorld 2.0 | 4.6% | $258.78 estimated cost | With partial credit 22.3% |
| ProofBench | 18.0% | $0.4192 per test | Time per proof 11m 59s |
| Surface Evolver Bench | 55.0% | $2.0781 of the run | Pass rate 18.8% · Tokens for the run 5.5M |
| ALE-Bench | 640.0 | $0.1118 per problem | Tokens per problem 55,120 |
| LMArena WebDev | 1482 Elo | Not published | Votes 19,803 · Interval (±) ±5.5 |
| GPQA Diamond | 92.7% | Not published | Latency, as printed 4m 39s |
| MMLU Pro | 84.2% | Not published | Latency, as printed 41 s |
| Design Arena: websites | 1258 Elo | Not published | Battles 16,301 |
| Design Arena: UI components | 1247 Elo | Not published | Battles 1,841 |
| Design Arena: game dev | 1210 Elo | Not published | Battles 4,423 |
| Design Arena: data viz | 1235 Elo | Not published | Battles 1,387 |
| Design Arena: 3D | 1209 Elo | Not published | Battles 4,883 |
| Design Arena: all code categories | 1247 Elo | Not published | Battles 27,491 |
| Design Arena: SVG | 1161 Elo | Not published | Battles 2,593 |
| Design Arena: ASCII art | 1161 Elo | Not published | Battles 500 |
| Design Arena: agents building web apps | 1198 Elo | Not published | Battles 3,567 |
| Design Arena: agents building full-stack apps | 1197 Elo | Not published | Battles 3,673 |
| Design Arena: agents building mobile apps | 1168 Elo | Not published | Battles 2,154 |
| Design Arena: agents building Android apps | 1173 Elo | Not published | Battles 1,160 |
| Design Arena: agents building games | 1156 Elo | Not published | Battles 514 |
| Design Arena: HTML slides | 1180 Elo | Not published | Battles 1,246 |
| Design Arena: PowerPoint slides | 1205 Elo | Not published | Battles 1,058 |
| Design Arena: spreadsheets | 1250 Elo | Not published | Battles 35 |
| LiveCodeBench (Vals AI) | 82.2% | Not published | Latency, as printed 6m 07s |
| MMMU Pro (Vals AI) | 81.2% | Not published | Latency, as printed 1m 08s |
| SAGE (Vals AI) | 50.6% | Not published | Latency, as printed 1m 59s |
| LegalBench (Vals AI) | 85.4% | Not published | Latency, as printed 8.2 s |
| CorpFin (Vals AI) | 68.1% | Not published | Latency, as printed 24 s |
| TaxEval (Vals AI) | 72.7% | Not published | Latency, as printed 1m 34s |
| MortgageTax (Vals AI) | 68.4% | Not published | Latency, as printed 27 s |
| MedCode (Vals AI) | 46.3% | Not published | Latency, as printed 1m 03s |
| MedScribe (Vals AI) | 87.3% | Not published | Latency, as printed 2m 04s |
| APEX-Agents | 37.7% | Not published | Mean score 53.5% |
| LMArena Text | 1433 Elo | Not published | Votes 63,973 · Interval (±) ±3.8 |
| LMArena Text: coding | 1471 Elo | Not published | Votes 18,242 · Interval (±) ±5.6 |
| LMArena Text: hard prompts | 1448 Elo | Not published | Votes 42,422 · Interval (±) ±4.5 |
| LMArena Text: math | 1429 Elo | Not published | Votes 3,192 · Interval (±) ±10.9 |
| LMArena Text: instruction following | 1433 Elo | Not published | Votes 22,259 · Interval (±) ±5.3 |
| LMArena Text: creative writing | 1404 Elo | Not published | Votes 11,713 · Interval (±) ±6.6 |
| LMArena Vision | 1251 Elo | Not published | Votes 19,844 · Interval (±) ±6.3 |
| LMArena Text: English | 1444 Elo | Not published | Votes 26,416 · Interval (±) ±5.0 |
| LMArena Text: non-English | 1420 Elo | Not published | Votes 37,548 · Interval (±) ±4.5 |
| LMArena Text: Chinese | 1467 Elo | Not published | Votes 4,069 · Interval (±) ±10.0 |
| LMArena Text: German | 1427 Elo | Not published | Votes 1,104 · Interval (±) ±18.8 |
| LMArena Text: Japanese | 1381 Elo | Not published | Votes 953 · Interval (±) ±21.4 |
| LMArena Text: Korean | 1373 Elo | Not published | Votes 1,192 · Interval (±) ±18.6 |
| LMArena Text: French | 1445 Elo | Not published | Votes 2,295 · Interval (±) ±14.2 |
| LMArena Text: Spanish | 1432 Elo | Not published | Votes 1,959 · Interval (±) ±14.6 |
| LMArena Text: Russian | 1427 Elo | Not published | Votes 7,506 · Interval (±) ±7.5 |
| LMArena Text: Polish | 1424 Elo | Not published | Votes 1,112 · Interval (±) ±17.8 |
| LMArena Text: multi-turn | 1443 Elo | Not published | Votes 11,126 · Interval (±) ±6.6 |
| LMArena Text: longer queries | 1446 Elo | Not published | Votes 28,906 · Interval (±) ±5.1 |
| LMArena Text: expert | 1463 Elo | Not published | Votes 6,992 · Interval (±) ±7.8 |
| LMArena Text: hard prompts in English | 1456 Elo | Not published | Votes 18,062 · Interval (±) ±5.6 |
| LMArena Text: software and IT | 1467 Elo | Not published | Votes 25,769 · Interval (±) ±5.0 |
| LMArena Text: mathematical work | 1447 Elo | Not published | Votes 3,630 · Interval (±) ±10.5 |
| LMArena Text: science | 1453 Elo | Not published | Votes 10,165 · Interval (±) ±6.6 |
| LMArena Text: medicine and healthcare | 1442 Elo | Not published | Votes 4,321 · Interval (±) ±9.7 |
| LMArena Text: legal and government | 1438 Elo | Not published | Votes 5,214 · Interval (±) ±8.9 |
| LMArena Text: business and finance | 1435 Elo | Not published | Votes 12,339 · Interval (±) ±6.3 |
| LMArena Text: writing and language | 1420 Elo | Not published | Votes 16,321 · Interval (±) ±5.8 |
| LMArena Text: entertainment and media | 1403 Elo | Not published | Votes 15,034 · Interval (±) ±6.0 |
| LMArena WebDev: React | 1478 Elo | Not published | Votes 13,888 · Interval (±) ±6.2 |
| LMArena WebDev: HTML | 1455 Elo | Not published | Votes 2,689 · Interval (±) ±12.6 |
| LMArena WebDev: from an image | 1470 Elo | Not published | Votes 5,983 · Interval (±) ±8.5 |
| LMArena Vision: English | 1263 Elo | Not published | Votes 7,372 · Interval (±) ±9.0 |
| LMArena Vision: Chinese | 1311 Elo | Not published | Votes 1,048 · Interval (±) ±20.2 |
| LMArena Vision: reading text | 1259 Elo | Not published | Votes 13,947 · Interval (±) ±6.5 |
| LMArena Vision: diagrams | 1266 Elo | Not published | Votes 5,289 · Interval (±) ±9.2 |
| LMArena Vision: homework | 1268 Elo | Not published | Votes 2,405 · Interval (±) ±12.7 |
| LMArena Vision: humor | 1235 Elo | Not published | Votes 773 · Interval (±) ±22.1 |
| LMArena Vision: recognizing things | 1197 Elo | Not published | Votes 107 · Interval (±) ±53.1 |
| LMArena Vision: creative writing | 1234 Elo | Not published | Votes 1,339 · Interval (±) ±17.0 |
| LMArena Document | 1435 Elo | Not published | Votes 6,314 · Interval (±) ±8.3 |