| ProgramBench (Vals AI) | 0.0% | $4.14 per test | Duration, as printed 45m 34s |
| SREBench (Vals AI) | 0.0% | $2.23 per test | Duration, as printed 28m 39s |
| IOI (Vals AI) | 14.9% | $1.71 per test | Duration, as printed 30m 12s |
| Terminal-Bench 2.1 (Vals AI) | 47.6% | $0.63 per test | Duration, as printed 12m 52s |
| Vibe Code Bench 1-100 (Vals AI) | 7.3% | $5.00 per test | Duration, as printed 50m 02s |
| SkillsBench (Vals AI) | 26.1% | $0.83 per test | Duration, as printed 15m 39s |
| CyberBench (Vals AI) | 66.8% | $1.46 per test | Duration, as printed 13m 31s |
| Finance Agent (Vals AI) | 46.6% | $1.00 per test | Duration, as printed 18m 02s |
| Tax Agent Bench (Vals AI) | 15.3% | $0.35 per test | Duration, as printed 13m 31s |
| Legal Research Bench (Vals AI) | 28.4% | $0.49 per test | Duration, as printed 14m 29s |
| Harvey's Legal Agent Benchmark (Vals AI) | 3.8% | $1.25 per test | Duration, as printed 8m 50s |
| Excel Modeling Benchmark (Vals AI) | 38.4% | $1.67 per test | Duration, as printed 26m 22s |
| Public Benefits Bench (Vals AI) | 58.5% | $0.37 per test | Nothing else |
| Vals Multimodal Index | 49.4% | $0.63 per test | Duration, as printed 15m 29s |
| FrontierSWE | 4.1% | $9.15 average cost per task | Best of 5 runs 10.8% · Hours per task 1.2 h |
| WeirdML | 32.3% | $0.1255 per run | Median lines of code 96 |
| ARC-AGI | 79.5% | $0.3005 per task | Nothing else |
| ARC-AGI-2 | 36.5% | $0.6421 per task | Nothing else |
| ProofBench | 0.0% | $0.2014 per test | Time per proof 7m 16s |
| ALE-Bench | 946.0 | $0.2025 per problem | Tokens per problem 55,717 |
| LMArena WebDev | 1414 Elo | Not published | Votes 15,778 · Interval (±) ±6.1 |
| GPQA Diamond | 87.1% | Not published | Latency, as printed 5m 09s |
| MMLU Pro | 86.3% | Not published | Latency, as printed 1m 13s |
| Design Arena: websites | 1224 Elo | Not published | Battles 10,867 |
| Design Arena: UI components | 1200 Elo | Not published | Battles 1,407 |
| Design Arena: game dev | 1166 Elo | Not published | Battles 3,548 |
| Design Arena: data viz | 1161 Elo | Not published | Battles 659 |
| Design Arena: 3D | 1161 Elo | Not published | Battles 2,334 |
| Design Arena: all code categories | 1206 Elo | Not published | Battles 18,742 |
| Design Arena: SVG | 1110 Elo | Not published | Battles 1,282 |
| Design Arena: ASCII art | 1089 Elo | Not published | Battles 220 |
| Design Arena: agents building web apps | 1196 Elo | Not published | Battles 1,207 |
| LiveCodeBench (Vals AI) | 85.5% | Not published | Latency, as printed 4m 18s |
| MMMU Pro (Vals AI) | 78.5% | Not published | Latency, as printed 2m 04s |
| SAGE (Vals AI) | 36.5% | Not published | Latency, as printed 3m 18s |
| LegalBench (Vals AI) | 83.0% | Not published | Latency, as printed 23 s |
| CorpFin (Vals AI) | 68.6% | Not published | Latency, as printed 29 s |
| TaxEval (Vals AI) | 75.3% | Not published | Latency, as printed 3m 18s |
| MortgageTax (Vals AI) | 64.0% | Not published | Latency, as printed 1m 09s |
| MedCode (Vals AI) | 41.2% | Not published | Latency, as printed 2m 47s |
| MedScribe (Vals AI) | 85.4% | Not published | Latency, as printed 4m 05s |
| APEX-Agents | 33.8% | Not published | Mean score 48.8% |
| LMArena Text | 1441 Elo | Not published | Votes 40,648 · Interval (±) ±4.3 |
| LMArena Text: coding | 1465 Elo | Not published | Votes 11,078 · Interval (±) ±6.6 |
| LMArena Text: hard prompts | 1451 Elo | Not published | Votes 26,664 · Interval (±) ±5.0 |
| LMArena Text: math | 1477 Elo | Not published | Votes 1,845 · Interval (±) ±13.8 |
| LMArena Text: instruction following | 1425 Elo | Not published | Votes 14,605 · Interval (±) ±6.0 |
| LMArena Text: creative writing | 1387 Elo | Not published | Votes 8,748 · Interval (±) ±7.5 |
| LMArena Text: English | 1442 Elo | Not published | Votes 16,677 · Interval (±) ±5.7 |
| LMArena Text: non-English | 1433 Elo | Not published | Votes 23,960 · Interval (±) ±5.1 |
| LMArena Text: Chinese | 1492 Elo | Not published | Votes 3,197 · Interval (±) ±10.9 |
| LMArena Text: German | 1447 Elo | Not published | Votes 627 · Interval (±) ±24.0 |
| LMArena Text: Japanese | 1426 Elo | Not published | Votes 638 · Interval (±) ±25.2 |
| LMArena Text: Korean | 1403 Elo | Not published | Votes 755 · Interval (±) ±22.4 |
| LMArena Text: French | 1454 Elo | Not published | Votes 1,281 · Interval (±) ±17.9 |
| LMArena Text: Spanish | 1447 Elo | Not published | Votes 1,269 · Interval (±) ±17.4 |
| LMArena Text: Russian | 1429 Elo | Not published | Votes 4,822 · Interval (±) ±9.0 |
| LMArena Text: Polish | 1453 Elo | Not published | Votes 604 · Interval (±) ±23.8 |
| LMArena Text: multi-turn | 1437 Elo | Not published | Votes 6,369 · Interval (±) ±8.1 |
| LMArena Text: longer queries | 1434 Elo | Not published | Votes 19,084 · Interval (±) ±5.7 |
| LMArena Text: expert | 1465 Elo | Not published | Votes 4,821 · Interval (±) ±9.1 |
| LMArena Text: hard prompts in English | 1449 Elo | Not published | Votes 10,833 · Interval (±) ±6.6 |
| LMArena Text: software and IT | 1465 Elo | Not published | Votes 15,820 · Interval (±) ±5.8 |
| LMArena Text: mathematical work | 1460 Elo | Not published | Votes 2,255 · Interval (±) ±12.9 |
| LMArena Text: science | 1455 Elo | Not published | Votes 6,689 · Interval (±) ±7.8 |
| LMArena Text: medicine and healthcare | 1448 Elo | Not published | Votes 2,766 · Interval (±) ±11.8 |
| LMArena Text: legal and government | 1448 Elo | Not published | Votes 3,363 · Interval (±) ±10.7 |
| LMArena Text: business and finance | 1439 Elo | Not published | Votes 7,568 · Interval (±) ±7.5 |
| LMArena Text: writing and language | 1403 Elo | Not published | Votes 11,160 · Interval (±) ±6.7 |
| LMArena Text: entertainment and media | 1393 Elo | Not published | Votes 10,607 · Interval (±) ±6.9 |
| LMArena WebDev: React | 1404 Elo | Not published | Votes 10,477 · Interval (±) ±7.2 |
| LMArena WebDev: HTML | 1399 Elo | Not published | Votes 2,174 · Interval (±) ±14.4 |