| ProgramBench (Vals AI) | 1.0% | $10.93 per test | Duration, as printed 46m 39s |
| SREBench (Vals AI) | 11.1% | $27.46 per test | Duration, as printed 2h 28m |
| IOI (Vals AI) | 56.9% | $3.98 per test | Duration, as printed 14m 20s |
| Terminal-Bench 2.1 (Vals AI) | 81.3% | $1.54 per test | Duration, as printed 9m 15s |
| SWE-bench Multimodal (Vals AI) | 40.8% | $3.69 per test | Duration, as printed 22m 18s |
| Vibe Code Bench 1-100 (Vals AI) | 18.8% | $18.91 per test | Duration, as printed 1h 06m |
| SkillsBench (Vals AI) | 58.0% | $2.34 per test | Duration, as printed 3m 55s |
| CyberBench (Vals AI) | 43.8% | $1.31 per test | Duration, as printed 9m 49s |
| Finance Agent (Vals AI) | 61.4% | $2.00 per test | Duration, as printed 3m 21s |
| Tax Agent Bench (Vals AI) | 32.1% | $0.86 per test | Duration, as printed 2m 58s |
| Legal Research Bench (Vals AI) | 38.9% | $1.63 per test | Duration, as printed 5m 24s |
| Harvey's Legal Agent Benchmark (Vals AI) | 12.1% | $3.66 per test | Duration, as printed 19m 22s |
| Excel Modeling Benchmark (Vals AI) | 72.2% | $8.24 per test | Duration, as printed 12m 49s |
| BioMysteryBench (Vals AI) | 62.2% | $1.55 per test | Duration, as printed 5m 27s |
| Terminal-Bench-Science (Vals AI) | 8.6% | $5.64 per test | Duration, as printed 55m 11s |
| Vals Science Index | 35.8% | $2.98 per test | Duration, as printed 22m 00s |
| Public Benefits Bench (Vals AI) | 65.3% | $1.01 per test | Nothing else |
| MysteryMechanism (Vals AI) | 36.5% | $1.73 per test | Duration, as printed 5m 22s |
| DeepSWE | 73.8% | $2.3623 mean cost per task | Solved in 4 tries 85.8% · Output tokens per task 143,243 · Agent steps per task 166 |
| FrontierSWE | 19.6% | $38.75 average cost per task | Best of 5 runs 31.6% · Hours per task 7.2 h |
| WeirdML | 84.8% | $0.5073 per run | Median lines of code 439 |
| ProofBench | 48.0% | $0.6029 per test | Time per proof 6m 09s |
| Surface Evolver Bench | 76.9% | $2.5981 of the run | Pass rate 56.3% · Tokens for the run 3.2M |
| ALE-Bench | 1269.7 | $0.3443 per problem | Tokens per problem 48,512 |
| LMArena WebDev | 1585 Elo | Not published | Votes 10,851 · Interval (±) ±7.4 |
| GPQA Diamond | 94.4% | Not published | Latency, as printed 20 s |
| MMLU Pro | 90.2% | Not published | Latency, as printed 7.8 s |
| Design Arena: websites | 1301 Elo | Not published | Battles 29,381 |
| Design Arena: UI components | 1318 Elo | Not published | Battles 1,754 |
| Design Arena: game dev | 1305 Elo | Not published | Battles 12,491 |
| Design Arena: data viz | 1271 Elo | Not published | Battles 513 |
| Design Arena: 3D | 1295 Elo | Not published | Battles 6,286 |
| Design Arena: all code categories | 1304 Elo | Not published | Battles 45,871 |
| Design Arena: SVG | 1294 Elo | Not published | Battles 2,224 |
| Design Arena: ASCII art | 1262 Elo | Not published | Battles 267 |
| Design Arena: agents building web apps | 1236 Elo | Not published | Battles 1,433 |
| Design Arena: agents building full-stack apps | 1244 Elo | Not published | Battles 1,155 |
| Design Arena: agents building mobile apps | 1235 Elo | Not published | Battles 4,206 |
| Design Arena: agents building Android apps | 1078 Elo | Not published | Battles 323 |
| Design Arena: agents building Godot games | 1273 Elo | Not published | Battles 664 |
| Design Arena: PowerPoint slides | 1179 Elo | Not published | Battles 819 |
| Design Arena: spreadsheets | 1138 Elo | Not published | Battles 90 |
| LiveCodeBench (Vals AI) | 89.5% | Not published | Latency, as printed 24 s |
| MMMU Pro (Vals AI) | 89.1% | Not published | Latency, as printed 12 s |
| SAGE (Vals AI) | 35.1% | Not published | Latency, as printed 29 s |
| LegalBench (Vals AI) | 87.0% | Not published | Latency, as printed 3.3 s |
| TaxEval (Vals AI) | 74.5% | Not published | Latency, as printed 8.8 s |
| MortgageTax (Vals AI) | 65.3% | Not published | Latency, as printed 15 s |
| MedCode (Vals AI) | 48.1% | Not published | Latency, as printed 43 s |
| MedScribe (Vals AI) | 84.5% | Not published | Latency, as printed 21 s |
| APEX-Agents | 64.3% | Not published | Mean score 72.9% |
| LMArena Text | 1499 Elo | Not published | Votes 37,454 · Interval (±) ±4.6 |
| LMArena Text: coding | 1507 Elo | Not published | Votes 9,810 · Interval (±) ±7.1 |
| LMArena Text: hard prompts | 1509 Elo | Not published | Votes 24,169 · Interval (±) ±5.5 |
| LMArena Text: math | 1527 Elo | Not published | Votes 1,883 · Interval (±) ±14.1 |
| LMArena Text: instruction following | 1491 Elo | Not published | Votes 13,391 · Interval (±) ±6.5 |
| LMArena Text: creative writing | 1492 Elo | Not published | Votes 8,232 · Interval (±) ±7.9 |
| LMArena Vision | 1315 Elo | Not published | Votes 4,645 · Interval (±) ±9.7 |
| LMArena Text: English | 1499 Elo | Not published | Votes 14,771 · Interval (±) ±6.2 |
| LMArena Text: non-English | 1491 Elo | Not published | Votes 22,675 · Interval (±) ±5.5 |
| LMArena Text: Chinese | 1551 Elo | Not published | Votes 3,070 · Interval (±) ±11.5 |
| LMArena Text: German | 1498 Elo | Not published | Votes 603 · Interval (±) ±24.1 |
| LMArena Text: Japanese | 1506 Elo | Not published | Votes 667 · Interval (±) ±25.9 |
| LMArena Text: Korean | 1462 Elo | Not published | Votes 680 · Interval (±) ±24.6 |
| LMArena Text: French | 1492 Elo | Not published | Votes 1,268 · Interval (±) ±18.2 |
| LMArena Text: Spanish | 1486 Elo | Not published | Votes 1,285 · Interval (±) ±17.1 |
| LMArena Text: Russian | 1514 Elo | Not published | Votes 4,787 · Interval (±) ±9.3 |
| LMArena Text: Polish | 1498 Elo | Not published | Votes 550 · Interval (±) ±25.2 |
| LMArena Text: multi-turn | 1502 Elo | Not published | Votes 5,601 · Interval (±) ±8.7 |
| LMArena Text: longer queries | 1507 Elo | Not published | Votes 17,232 · Interval (±) ±6.2 |
| LMArena Text: expert | 1525 Elo | Not published | Votes 4,424 · Interval (±) ±9.6 |
| LMArena Text: hard prompts in English | 1504 Elo | Not published | Votes 9,477 · Interval (±) ±7.1 |
| LMArena Text: software and IT | 1506 Elo | Not published | Votes 14,131 · Interval (±) ±6.3 |
| LMArena Text: mathematical work | 1513 Elo | Not published | Votes 2,221 · Interval (±) ±13.1 |
| LMArena Text: science | 1521 Elo | Not published | Votes 6,186 · Interval (±) ±8.3 |
| LMArena Text: medicine and healthcare | 1507 Elo | Not published | Votes 2,687 · Interval (±) ±12.3 |
| LMArena Text: legal and government | 1499 Elo | Not published | Votes 3,312 · Interval (±) ±10.9 |
| LMArena Text: business and finance | 1479 Elo | Not published | Votes 6,835 · Interval (±) ±7.9 |
| LMArena Text: writing and language | 1496 Elo | Not published | Votes 10,501 · Interval (±) ±7.1 |
| LMArena Text: entertainment and media | 1480 Elo | Not published | Votes 9,844 · Interval (±) ±7.4 |
| LMArena WebDev: React | 1577 Elo | Not published | Votes 6,947 · Interval (±) ±8.6 |
| LMArena WebDev: HTML | 1616 Elo | Not published | Votes 1,545 · Interval (±) ±17.0 |
| LMArena WebDev: from an image | 1593 Elo | Not published | Votes 2,516 · Interval (±) ±12.6 |
| LMArena Vision: English | 1318 Elo | Not published | Votes 1,556 · Interval (±) ±16.0 |
| LMArena Vision: Chinese | 1416 Elo | Not published | Votes 224 · Interval (±) ±39.3 |
| LMArena Vision: reading text | 1333 Elo | Not published | Votes 3,137 · Interval (±) ±11.2 |
| LMArena Vision: diagrams | 1345 Elo | Not published | Votes 1,159 · Interval (±) ±18.1 |
| LMArena Vision: homework | 1351 Elo | Not published | Votes 681 · Interval (±) ±22.4 |
| LMArena Vision: humor | 1321 Elo | Not published | Votes 175 · Interval (±) ±45.4 |
| LMArena Vision: creative writing | 1299 Elo | Not published | Votes 459 · Interval (±) ±28.8 |