LLMBenchmark.ai

Enter opens the highlighted result. Esc closes.

What is vibe coding?

Claude Opus 5.5 leads
92.2
of 356 models have a score
136
rest on three benchmarks
2
coding boards in the index
4

Vibe Coding Index · read October 20

vibe_coding_index.py


        # Vibe Coding Index, read October 20
        leaders = {
            "Claude Opus 5.5": 92.2,    "Claude Fable 5.1": 89.1,    "Claude Sonnet 5.5": 88.9,
        }
      

Vibe coding, as Andrej Karpathy named it in a post on X on February 2, 2025 and Simon Willison defines it, is building software with an AI model by prompting it in plain language and accepting the code it writes without reviewing it. Collins Dictionary chose the term as its 2025 Word of the Year on November 6, 2025.

Every date and quotation below links to its source, read on October 15, 2026. Karpathy's wording comes through Wikipedia's transcription. See the tools.

Two meanings of the word

Karpathy meant a way of working where you "fully give in to the vibes" and "forget that the code even exists". Simon Willison, writing on March 19, 2025, keeps that strict sense: "building software with an LLM without reviewing the code it writes". He adds that code you review, test and can explain is software development, whatever tool wrote it. Source. Wording via Wikipedia's transcription of the post.

Collins Dictionary defines the term more widely, as "the use of artificial intelligence prompted by natural language to assist with the writing of computer code". When an article says vibe coding, check which of the two it means. The risks below apply to the strict one. Source.

Where the term came from

February 2, 2025
Andrej Karpathy posts on X about "a new kind of coding I call vibe coding", describing his own work with Cursor Composer and Sonnet. He calls it "not too bad for throwaway weekend projects, but still quite amusing". Andrej Karpathy, post on X Wording via Wikipedia's transcription of the post.
March 19, 2025
Simon Willison argues that vibe coding is not the same as writing code with the help of LLMs, and ties the word to code that nobody reviews. Simon Willison, "Not all AI-assisted programming is vibe coding"
November 6, 2025
Collins Dictionary names vibe coding its Word of the Year for 2025. CNN Business, "Vibe coding named Collins Dictionary's Word of the Year" (syndicated by KVIA)

What it looks like in practice

Karpathy described his own loop in the original post. It has four steps. Source. Wording via Wikipedia's transcription of the post.

  1. Ask for a change in plain words, by voice or text. His examples include "decrease the padding on the sidebar by half".
  2. Accept every change. In his words: "I 'Accept All' always, I don't read the diffs anymore."
  3. When an error appears, paste it back with no comment. "Usually that fixes it."
  4. When the model cannot fix a bug, work around it or ask for random changes until it goes away.

That is one developer describing one tool in early 2025. The tools in use today are on the tools page.

What vibe coding does not solve

  • You lose track of the code. Karpathy wrote that the code "grows beyond my usual comprehension". That is the trade, stated by the person who named it. Source. Wording via Wikipedia's transcription of the post.
  • Working code can still be unsafe. In a Veracode study published July 30, 2025, more than 100 LLMs completed 80 coding tasks. Veracode reports that "while AI produces functional code, it introduces security vulnerabilities in 45 percent of cases". Source.
  • Feeling faster is not being faster. In a METR randomized trial published July 10, 2025, 16 experienced open-source developers took 19% longer on 246 tasks when AI tools were allowed, and afterwards still believed AI had sped them up by 20%. That study tested AI-assisted work in mature codebases, not vibe coding, and METR now calls its result out of date and has published newer data from February 2026, which we have not read. Source.

Which models are actually good at it

Full ranking with filters and sorting

Anthropic Claude Opus 5.5 leads our Vibe Coding Index among models with two or more benchmarks on October 20, 2026, at 92.2. The index is a weighted mean of 4 public benchmarks (Aider polyglot pass rate, SWE-bench Verified resolved rate, LiveBench coding score, WebDev Arena Elo). We do not run these benchmarks ourselves, and none of them measures a person vibe coding. Each scores a model on a fixed set of coding tasks.

Of 356 models, 2 has three independent measurements behind its score. The table says how many stand behind each number.

Top 10 LLMs in the Vibe Coding Index
Rank Model Evidence Index Output $/Mtok Cost per solved task
1
Claude Opus 5.5 Anthropic
92.2 $20.00 No SWE-bench cost LiveBench: $5.224 per solved
2
Claude Fable 5.1 Anthropic
89.1 $50.00 No SWE-bench cost LiveBench: $5.407 per solved
3
Claude Sonnet 5.5 Anthropic
88.9 $10.00 No SWE-bench cost LiveBench: $4.247 per solved
4
GPT-6 Astra OpenAI
88.7 $50.00 No SWE-bench cost LiveBench: $3.538 per solved
5
GPT-6.1 Sol OpenAI
88.7 $10.00 No SWE-bench cost LiveBench: $0.502 per solved
6 87.4 $0.47 No SWE-bench cost LiveBench: $0.121 per solved
7
o3 Pro OpenAI
84.9 $80.00 No run cost
8 83.3 $0.60 No SWE-bench cost LiveBench: $0.204 per solved
9 83.1 $10.00 No run cost
10
GPT-6 Sol OpenAI
82.8 $10.00 No SWE-bench cost LiveBench: $1.049 per solved

A small "LiveBench" figure under No run cost is LiveBench's own cost per successful task on its coding tasks: another benchmark, shown for reference and not ranked with the column.

Vibe coding questions

What is vibe coding?
Vibe coding, as Andrej Karpathy named it in a post on X on February 2, 2025 and Simon Willison defines it, is building software with an AI model by prompting it in plain language and accepting the code it writes without reviewing it. Collins Dictionary chose the term as its 2025 Word of the Year on November 6, 2025.
Who coined the term vibe coding?
Andrej Karpathy, in a post on X on February 2, 2025. He described a way of working where you "fully give in to the vibes" and "forget that the code even exists". Wording via Wikipedia's transcription of the post.
Is vibe coding the same as using AI to write code?
Not in the strict sense. Simon Willison, on March 19, 2025, defined it as building software with an LLM without reviewing the code it writes. Collins Dictionary uses a wider definition, "the use of artificial intelligence prompted by natural language to assist with the writing of computer code".
Is vibe coding safe for production software?
Karpathy called it "not too bad for throwaway weekend projects". Wording via Wikipedia's transcription of the post. In Veracode's study of more than 100 LLMs, published July 30, 2025, AI-generated code introduced security vulnerabilities in 45 percent of cases. Code that nobody reviews has no other check.
Which AI model is best for vibe coding?
On October 20, 2026, Anthropic Claude Opus 5.5 leads our Vibe Coding Index among models with two or more benchmarks, at 92.2. The index averages public coding benchmarks and none measures vibe coding directly. Only 2 of 356 models has three independent measurements behind its score.
What are the best vibe coding tools?
We do not rank tools, because our data ranks models, not products. Our tools page lists six, with what each builds, where it runs and which models its vendor names, all checked on October 15, 2026.