| Finance Agent (Vals AI) | 51.0% | $2.41 per test | Duration, as printed 11m 36s |
| Tax Agent Bench (Vals AI) | 25.8% | $1.19 per test | Duration, as printed 11m 47s |
| Legal Research Bench (Vals AI) | 38.5% | $2.30 per test | Duration, as printed 16m 30s |
| Excel Modeling Benchmark (Vals AI) | 60.2% | $7.23 per test | Duration, as printed 54m 16s |
| Public Benefits Bench (Vals AI) | 62.5% | $1.11 per test | Nothing else |
| Vals Multimodal Index | 60.6% | $1.58 per test | Duration, as printed 8m 56s |
| DeepSWE | 29.9% | $5.5224 mean cost per task | Solved in 4 tries 56.6% · Output tokens per task 76,160 · Agent steps per task 134 |
| WeirdML | 66.1% | $2.8594 per run | Median lines of code 178 |
| OSWorld 2.0 | 9.3% | $1550.00 estimated cost | With partial credit 33.9% |
| ARC-AGI | 86.5% | $1.1908 per task | Nothing else |
| ARC-AGI-2 | 60.4% | $2.6966 per task | Nothing else |
| ProofBench | 45.0% | $2.1621 per test | Time per proof 1h 31m |
| LMArena WebDev | 1523 Elo | Not published | Votes 22,822 · Interval (±) ±5.2 |
| GPQA Diamond | 85.6% | Not published | Latency, as printed 6m 18s |
| AIME | 92.3% | Not published | Latency, as printed 5m 28s |
| MMLU Pro | 87.3% | Not published | Latency, as printed 55 s |
| Design Arena: websites | 1285 Elo | Not published | Battles 13,777 |
| Design Arena: UI components | 1272 Elo | Not published | Battles 2,167 |
| Design Arena: game dev | 1259 Elo | Not published | Battles 4,485 |
| Design Arena: data viz | 1278 Elo | Not published | Battles 1,902 |
| Design Arena: 3D | 1234 Elo | Not published | Battles 5,715 |
| Design Arena: all code categories | 1276 Elo | Not published | Battles 26,317 |
| Design Arena: SVG | 1196 Elo | Not published | Battles 4,025 |
| Design Arena: ASCII art | 1236 Elo | Not published | Battles 411 |
| Design Arena: agents building web apps | 1190 Elo | Not published | Battles 2,825 |
| Design Arena: agents building full-stack apps | 1213 Elo | Not published | Battles 2,030 |
| Design Arena: agents building mobile apps | 1211 Elo | Not published | Battles 1,558 |
| Design Arena: agents building Android apps | 1212 Elo | Not published | Battles 1,101 |
| Design Arena: agents building games | 1177 Elo | Not published | Battles 245 |
| LiveCodeBench (Vals AI) | 82.1% | Not published | Latency, as printed 4m 55s |
| ProgramBench (Vals AI) | 0.5% | Not published | Duration, as printed 1h 12m |
| MMMU Pro (Vals AI) | 83.6% | Not published | Latency, as printed 2m 12s |
| SAGE (Vals AI) | 46.6% | Not published | Latency, as printed 2m 43s |
| LegalBench (Vals AI) | 82.1% | Not published | Latency, as printed 22 s |
| CaseLaw (Vals AI) | 64.0% | Not published | Latency, as printed 36 s |
| CorpFin (Vals AI) | 65.3% | Not published | Latency, as printed 18 s |
| TaxEval (Vals AI) | 77.1% | Not published | Latency, as printed 2m 08s |
| MortgageTax (Vals AI) | 67.7% | Not published | Latency, as printed 30 s |
| MedQA (Vals AI) | 92.1% | Not published | Latency, as printed 35 s |
| APEX-Agents | 43.0% | Not published | Mean score 57.6% |
| LMArena Text | 1458 Elo | Not published | Votes 70,931 · Interval (±) ±3.5 |
| LMArena Text: coding | 1504 Elo | Not published | Votes 19,724 · Interval (±) ±5.5 |
| LMArena Text: hard prompts | 1485 Elo | Not published | Votes 46,229 · Interval (±) ±4.2 |
| LMArena Text: math | 1463 Elo | Not published | Votes 3,792 · Interval (±) ±10.0 |
| LMArena Text: instruction following | 1476 Elo | Not published | Votes 23,711 · Interval (±) ±5.2 |
| LMArena Text: creative writing | 1435 Elo | Not published | Votes 12,367 · Interval (±) ±6.5 |
| LMArena Vision | 1283 Elo | Not published | Votes 27,408 · Interval (±) ±6.1 |
| LMArena Text: English | 1472 Elo | Not published | Votes 31,965 · Interval (±) ±4.7 |
| LMArena Text: non-English | 1440 Elo | Not published | Votes 38,962 · Interval (±) ±4.3 |
| LMArena Text: Chinese | 1492 Elo | Not published | Votes 4,520 · Interval (±) ±9.4 |
| LMArena Text: German | 1428 Elo | Not published | Votes 1,221 · Interval (±) ±17.8 |
| LMArena Text: Japanese | 1420 Elo | Not published | Votes 801 · Interval (±) ±22.4 |
| LMArena Text: Korean | 1411 Elo | Not published | Votes 1,153 · Interval (±) ±18.5 |
| LMArena Text: French | 1465 Elo | Not published | Votes 2,284 · Interval (±) ±14.0 |
| LMArena Text: Spanish | 1464 Elo | Not published | Votes 2,168 · Interval (±) ±13.8 |
| LMArena Text: Russian | 1439 Elo | Not published | Votes 7,983 · Interval (±) ±7.3 |
| LMArena Text: Polish | 1442 Elo | Not published | Votes 1,412 · Interval (±) ±15.8 |
| LMArena Text: multi-turn | 1464 Elo | Not published | Votes 12,693 · Interval (±) ±6.4 |
| LMArena Text: longer queries | 1479 Elo | Not published | Votes 31,070 · Interval (±) ±4.9 |
| LMArena Text: expert | 1500 Elo | Not published | Votes 7,384 · Interval (±) ±7.8 |
| LMArena Text: hard prompts in English | 1494 Elo | Not published | Votes 21,356 · Interval (±) ±5.4 |
| LMArena Text: software and IT | 1491 Elo | Not published | Votes 27,745 · Interval (±) ±4.9 |
| LMArena Text: mathematical work | 1483 Elo | Not published | Votes 3,982 · Interval (±) ±10.1 |
| LMArena Text: science | 1475 Elo | Not published | Votes 11,651 · Interval (±) ±6.4 |
| LMArena Text: medicine and healthcare | 1451 Elo | Not published | Votes 5,437 · Interval (±) ±8.8 |
| LMArena Text: legal and government | 1470 Elo | Not published | Votes 5,707 · Interval (±) ±8.6 |
| LMArena Text: business and finance | 1464 Elo | Not published | Votes 14,063 · Interval (±) ±6.1 |
| LMArena Text: writing and language | 1444 Elo | Not published | Votes 17,602 · Interval (±) ±5.7 |
| LMArena Text: entertainment and media | 1431 Elo | Not published | Votes 15,933 · Interval (±) ±6.0 |
| LMArena WebDev: React | 1512 Elo | Not published | Votes 18,106 · Interval (±) ±5.7 |
| LMArena WebDev: HTML | 1527 Elo | Not published | Votes 2,897 · Interval (±) ±11.9 |
| LMArena WebDev: from an image | 1529 Elo | Not published | Votes 8,312 · Interval (±) ±8.8 |
| LMArena Vision: English | 1298 Elo | Not published | Votes 11,555 · Interval (±) ±8.3 |
| LMArena Vision: Chinese | 1307 Elo | Not published | Votes 1,736 · Interval (±) ±16.8 |
| LMArena Vision: reading text | 1296 Elo | Not published | Votes 19,839 · Interval (±) ±6.1 |
| LMArena Vision: diagrams | 1306 Elo | Not published | Votes 7,630 · Interval (±) ±8.4 |
| LMArena Vision: homework | 1311 Elo | Not published | Votes 3,700 · Interval (±) ±10.7 |
| LMArena Vision: humor | 1248 Elo | Not published | Votes 1,060 · Interval (±) ±19.2 |
| LMArena Vision: recognizing things | 1195 Elo | Not published | Votes 162 · Interval (±) ±45.6 |
| LMArena Vision: creative writing | 1272 Elo | Not published | Votes 1,696 · Interval (±) ±15.2 |
| LMArena Document | 1482 Elo | Not published | Votes 57,813 · Interval (±) ±5.9 |