ProgramBench leaderboard
- Claude Opus 5.5 leads
- 18.5%
- models measured
- 22
- lead over #2
- +9.5 pts
- median
- 0.0%
Leader on ProgramBench · read November 5
# ProgramBench, read November 5
leaders = {
"Claude Opus 5.5": 18.5%, "Claude Fable 5.1": 9.0%, "Muse Wonder": 5.5%,
}
On November 5, 2026, 22 of our catalog's models had a ProgramBench result, and Anthropic Claude Opus 5.5 led with 18.5%, ahead of Anthropic Claude Fable 5.1 at 9.0%.
ProgramBench asks whether language models can rebuild programs from scratch. Given only a compiled binary and its documentation, an agent must design and write a complete codebase that reproduces the original program's behavior. It has 200 tasks and comes from Meta Superintelligence Labs, Stanford University and Harvard University. The number is Resolved: the share of tasks solved in full, judged by hidden behavioral tests.
How to read the score: The authors note that behavioral tests can never cover every input. Their separate "Almost" column (95% of the tests passed) is not shown here. The board runs every model in the mini-SWE-agent harness.
Role in the index
A coding board we show beside the Vibe Coding Index, not inside it: the index keeps the six boards it was defined on, so a score does not move every time a board is added. Read the methodology
| # | Model | Result | Evidence | Coding index | Output $/Mtok |
|---|---|---|---|---|---|
| 1 | Claude Opus 5.5Anthropic | 18.5% | 88.9 | $20.00 | |
| 2 | Claude Fable 5.1Anthropic | 9.0% | 84.3 | $50.00 | |
| 3 | Muse WonderMeta | 5.5% | 72.0 | $20.00 | |
| 4 | GPT-6 AstraOpenAI | 5.0% | 85.1 | $50.00 | |
| 5 | Claude Opus 5Anthropic | 4.5% | 81.6 | $25.00 | |
| 6 | Muse Spark 1.3Meta | 2.5% | 57.4 | $4.25 | |
| 7 | GPT-5.6 SolOpenAI | 1.0% | 70.7 | $20.00 | |
| 8 | Gemini 3.6 FlashGoogle | 0.5% | 62.5 | $3.75 | |
| 9 | GPT-5.5OpenAI | 0.5% | 67.9 | $30.00 | |
| 10 | Claude Haiku 4.5Anthropic | 0.0% | 31.4 | $5.00 | |
| 11 | Claude Opus 4.6Anthropic | 0.0% | 57.2 | $25.00 | |
| 12 | Claude Opus 4.7Anthropic | 0.0% | 67.5 | $25.00 | |
| 13 | Claude Opus 4.8Anthropic | 0.0% | 62.8 | $25.00 | |
| 14 | Claude Sonnet 4.6Anthropic | 0.0% | 55.0 | $15.00 | |
| 15 | Gemini 3.5 FlashGoogle | 0.0% | 39.8 | $9.00 | |
| 16 | Gemini 3.7 FlashGoogle | 0.0% | 54.0 | $3.75 | |
| 17 | Muse Spark 1.1Meta | 0.0% | 70.3 | $4.25 | |
| 18 | Muse Spark 1.2Meta | 0.0% | 50.9 | $4.25 | |
| 19 | GPT-5 MiniOpenAI | 0.0% | 34.2 | $2.00 | |
| 20 | GPT-5.4OpenAI | 0.0% | 64.4 | $15.00 | |
| 21 | GPT-5.4 MiniOpenAI | 0.0% | 35.8 | $4.50 | |
| 22 | GLM 5.2Z.ai | 0.0% | 61.1 | $4.40 |
Only models in our catalog are listed, and each appears once with its best published run. 22 rows were read from the source on November 5, 2026, and rows that match no catalog model are left out.
The board above is ProgramBench's own run. Model makers also report ProgramBench in their release announcements; those numbers are below, with the date and the announcement. They are not independent and do not count toward any index.
