| SREBench (Vals AI) | 0.8% | $13.42 per test | Duration, as printed 1h 06m |
| IOI (Vals AI) | 40.6% | $5.64 per test | Duration, as printed 1h 27m |
| Terminal-Bench 2.1 (Vals AI) | 67.8% | $0.28 per test | Duration, as printed 10m 24s |
| SkillsBench (Vals AI) | 66.0% | $0.58 per test | Duration, as printed 2m 16s |
| CyberBench (Vals AI) | 71.0% | $2.95 per test | Duration, as printed 14m 08s |
| Finance Agent (Vals AI) | 48.4% | $1.16 per test | Duration, as printed 4m 49s |
| Tax Agent Bench (Vals AI) | 24.9% | $0.47 per test | Duration, as printed 4m 44s |
| Legal Research Bench (Vals AI) | 38.0% | $1.09 per test | Duration, as printed 15m 54s |
| Excel Modeling Benchmark (Vals AI) | 52.9% | $1.55 per test | Duration, as printed 18m 28s |
| Vals Multimodal Index | 63.4% | $0.91 per test | Duration, as printed 5m 19s |
| DeepSWE | 53.8% | $2.4157 mean cost per task | Solved in 4 tries 77.9% · Output tokens per task 35,525 · Agent steps per task 61 |
| WeirdML | 46.4% | $0.0805 per run | Median lines of code 86 |
| ARC-AGI | 87.2% | $0.31 per task | Nothing else |
| ARC-AGI-2 | 52.6% | $0.78 per task | Nothing else |
| ProofBench | 31.0% | $0.574 per test | Time per proof 11m 09s |
| Surface Evolver Bench | 74.4% | $4.6103 of the run | Pass rate 50.0% · Tokens for the run 3.1M |
| ALE-Bench | 1309.3 | $0.1768 per problem | Tokens per problem 32,347 |
| LMArena WebDev | 1553 Elo | Not published | Votes 11,082 · Interval (±) ±6.7 |
| GPQA Diamond | 92.9% | Not published | Latency, as printed 3m 38s |
| MMLU Pro | 89.2% | Not published | Latency, as printed 5m 58s |
| LiveCodeBench (Vals AI) | 87.3% | Not published | Latency, as printed 9m 49s |
| MMMU Pro (Vals AI) | 61.8% | Not published | Latency, as printed 11m 55s |
| SAGE (Vals AI) | 35.0% | Not published | Latency, as printed 2m 50s |
| LegalBench (Vals AI) | 86.0% | Not published | Latency, as printed 1m 08s |
| CorpFin (Vals AI) | 67.4% | Not published | Latency, as printed 1m 32s |
| TaxEval (Vals AI) | 71.7% | Not published | Latency, as printed 3m 15s |
| MortgageTax (Vals AI) | 61.8% | Not published | Latency, as printed 1m 57s |
| MedCode (Vals AI) | 43.3% | Not published | Latency, as printed 60 s |
| MedScribe (Vals AI) | 86.9% | Not published | Latency, as printed 11m 45s |
| APEX-Agents | 56.2% | Not published | Mean score 70.5% |
| LMArena Text | 1448 Elo | Not published | Votes 39,896 · Interval (±) ±4.3 |
| LMArena Text: coding | 1474 Elo | Not published | Votes 11,116 · Interval (±) ±6.6 |
| LMArena Text: hard prompts | 1462 Elo | Not published | Votes 26,662 · Interval (±) ±5.0 |
| LMArena Text: math | 1459 Elo | Not published | Votes 1,766 · Interval (±) ±14.2 |
| LMArena Text: instruction following | 1446 Elo | Not published | Votes 14,539 · Interval (±) ±6.1 |
| LMArena Text: creative writing | 1442 Elo | Not published | Votes 8,298 · Interval (±) ±7.6 |
| LMArena Vision | 1287 Elo | Not published | Votes 12,599 · Interval (±) ±7.0 |
| LMArena Text: English | 1453 Elo | Not published | Votes 16,197 · Interval (±) ±5.7 |
| LMArena Text: non-English | 1440 Elo | Not published | Votes 23,697 · Interval (±) ±5.1 |
| LMArena Text: Chinese | 1495 Elo | Not published | Votes 3,066 · Interval (±) ±11.3 |
| LMArena Text: German | 1448 Elo | Not published | Votes 686 · Interval (±) ±23.0 |
| LMArena Text: Japanese | 1431 Elo | Not published | Votes 530 · Interval (±) ±27.5 |
| LMArena Text: Korean | 1406 Elo | Not published | Votes 731 · Interval (±) ±22.4 |
| LMArena Text: French | 1457 Elo | Not published | Votes 1,274 · Interval (±) ±18.1 |
| LMArena Text: Spanish | 1451 Elo | Not published | Votes 1,081 · Interval (±) ±18.5 |
| LMArena Text: Russian | 1449 Elo | Not published | Votes 4,639 · Interval (±) ±9.2 |
| LMArena Text: Polish | 1433 Elo | Not published | Votes 628 · Interval (±) ±23.5 |
| LMArena Text: multi-turn | 1456 Elo | Not published | Votes 6,632 · Interval (±) ±8.0 |
| LMArena Text: longer queries | 1463 Elo | Not published | Votes 19,283 · Interval (±) ±5.7 |
| LMArena Text: expert | 1466 Elo | Not published | Votes 4,720 · Interval (±) ±9.2 |
| LMArena Text: hard prompts in English | 1464 Elo | Not published | Votes 10,737 · Interval (±) ±6.7 |
| LMArena Text: software and IT | 1471 Elo | Not published | Votes 15,966 · Interval (±) ±5.8 |
| LMArena Text: mathematical work | 1481 Elo | Not published | Votes 2,161 · Interval (±) ±13.2 |
| LMArena Text: science | 1465 Elo | Not published | Votes 6,541 · Interval (±) ±7.8 |
| LMArena Text: medicine and healthcare | 1441 Elo | Not published | Votes 2,938 · Interval (±) ±11.6 |
| LMArena Text: legal and government | 1463 Elo | Not published | Votes 3,349 · Interval (±) ±10.9 |
| LMArena Text: business and finance | 1441 Elo | Not published | Votes 7,620 · Interval (±) ±7.5 |
| LMArena Text: writing and language | 1445 Elo | Not published | Votes 10,904 · Interval (±) ±6.7 |
| LMArena Text: entertainment and media | 1434 Elo | Not published | Votes 10,395 · Interval (±) ±6.9 |
| LMArena WebDev: React | 1556 Elo | Not published | Votes 7,531 · Interval (±) ±7.8 |
| LMArena WebDev: HTML | 1567 Elo | Not published | Votes 1,599 · Interval (±) ±16.2 |
| LMArena WebDev: from an image | 1562 Elo | Not published | Votes 4,255 · Interval (±) ±9.8 |
| LMArena Vision: English | 1285 Elo | Not published | Votes 4,285 · Interval (±) ±10.5 |
| LMArena Vision: Chinese | 1345 Elo | Not published | Votes 728 · Interval (±) ±23.3 |
| LMArena Vision: reading text | 1293 Elo | Not published | Votes 8,562 · Interval (±) ±7.5 |
| LMArena Vision: diagrams | 1304 Elo | Not published | Votes 3,227 · Interval (±) ±11.0 |
| LMArena Vision: homework | 1295 Elo | Not published | Votes 1,284 · Interval (±) ±17.1 |
| LMArena Vision: humor | 1276 Elo | Not published | Votes 573 · Interval (±) ±25.1 |
| LMArena Vision: creative writing | 1293 Elo | Not published | Votes 1,094 · Interval (±) ±19.0 |
| LMArena Search | 1216 Elo | Not published | Votes 76,458 · Interval (±) ±5.9 |
| LMArena Document | 1452 Elo | Not published | Votes 4,965 · Interval (±) ±8.7 |