LLMBenchmark.ai API & MCP: free LLM benchmark data for agents
The LLMBenchmark.ai MCP server answers one question for AI agents and developers: which model should I use for this task? It draws on 190 public benchmarks and the API prices of 356 LLMs, read October 20, 2026. It is free, needs no key and allows 500 requests per IP per day. The data is licensed CC BY 4.0.
Connect in one line
The server speaks MCP over Streamable HTTP at https://api.llmbenchmark.ai/mcp. There is nothing to install and no key to create.
The same server also answers at https://vibebench-api.takegroup.work/mcp until api.llmbenchmark.ai points at it. Use whichever address resolves for you.
Claude Code
claude mcp add --transport http llmbenchmark https://api.llmbenchmark.ai/mcpCursor (.cursor/mcp.json)
{
"mcpServers": {
"llmbenchmark": {
"url": "https://api.llmbenchmark.ai/mcp"
}
}
}Codex (~/.codex/config.toml)
[mcp_servers.llmbenchmark]
url = "https://api.llmbenchmark.ai/mcp"REST: curl /v1/recommend
curl "https://api.llmbenchmark.ai/v1/recommend?task=refactor+a+large+React+codebase&max_output_price_per_m=5"What to ask
Once connected, ask in plain words. The agent calls the tools and cites the data date and page links in its answer.
- "Which model should I use to refactor a React app under $5 per million output tokens?"
- "What is the best open-weight model for math with at least a 128k context window?"
- "Compare Claude Opus and GPT-6 Astra for coding."
- "What does the cheapest model that scores above the median on agentic tasks cost?"
Tools
Six read-only tools. Every answer carries the data date, the source name and links to the pages behind the numbers.
| Tool | What it returns |
|---|---|
recommend_model | Describe a task, set a price, context or open-weights limit, and get the top models with scores, rank, prices and a one-line reason. |
get_leaderboard | The top models on one category board: overall, coding, agentic, reasoning, math, knowledge or design. |
compare_models | Two models side by side on every board where both were measured, with prices and context. |
get_model | One model: maker, release date, prices, context window, open weights, and its score and rank on every board. |
get_prices | API list prices per million tokens, cheapest output price first, filtered by model or maker. |
list_categories | The categories, the board behind each and the task keywords that select it. |
How recommend_model picks a category
It reads the task text, counts the keywords below per category and takes the category with the most hits. A tie goes to the category listed first. If no keyword matches, it uses the overall board. No language model is involved: the same text and the same data always return the same answer.
It then drops models that break your limits (output price per million tokens, context window, open weights), ranks the rest by that category board and returns the top ones. Models with no list price or no context figure fail a limit on it, they are never assumed to pass. If nothing fits, the answer says so and returns the nearest options, each marked with the limit it misses.
The Vibe Coding Index averages up to four coding benchmarks. An answer for coding states how many stand behind each score, and a score from one benchmark is a thin basis.
| Category | Board it is ranked on | Task keywords |
|---|---|---|
| Coding | Vibe Coding Index | code, coding, codebase, program, programming, refactor, refactoring, debug, debugging, bug, bugs, function, api, backend, frontend, react, typescript, javascript, python, rust, java, sql, script, unit test, unit tests, pull request, pr review, code review, compile, software, developer, swe, repo, github, ide, autocomplete, copilot, vibe coding, implement, migration, legacy |
| Agentic work | Artificial Analysis Agentic Index | agent, agents, agentic, tool use, tool calling, function calling, autonomous, workflow, workflows, multi-step, multistep, orchestration, browse, browsing, browser, computer use, automation, automate, mcp, long-running, research agent |
| Reasoning | GPQA Diamond | reasoning, reason, logic, logical, puzzle, puzzles, science, scientific, physics, chemistry, biology, deduce, hard questions, phd |
| Math | AIME | math, maths, mathematics, mathematical, calculus, algebra, geometry, statistics, proof, proofs, equation, equations, arithmetic, olympiad, aime, number theory, probability |
| Knowledge | MMLU Pro | knowledge, facts, factual, trivia, question answering, exam, exams, quiz, encyclopedia, history, legal, medicine, medical, academic, subjects, general knowledge, tutor, tutoring |
| Web design | Design Arena: websites | web design, website design, landing page, landing pages, ui, ux, user interface, css, tailwind, layout, mockup, mockups, frontend design, visual design, homepage, web page, webpage, website, websites, html, figma |
| Overall (no keyword matched) | Artificial Analysis Intelligence Index | none |
REST API
Every tool has a plain HTTP twin under https://api.llmbenchmark.ai/v1. /v1/recommend takes the same arguments as recommend_model. /v1/models, /v1/scores, /v1/sources and /v1/export return the underlying data. The OpenAPI description is at https://api.llmbenchmark.ai/docs. GET requests from any origin are allowed (CORS).
Limits
- 500 requests per IP address per UTC day, counted across /mcp and /v1/*. The counter resets at 00:00 UTC.
- Over the limit, the server answers HTTP 429 with a message and a Retry-After header. X-RateLimit-Remaining on every response shows what is left.
- Free, no key, no sign-up. For bulk use, download the full dataset as JSON instead of paging the API.
- Data refreshes once a day. Every answer carries data_date, so an agent can say how old the numbers are.
License and attribution
The data is licensed CC BY 4.0: use it, also commercially, and credit LLMBenchmark.ai with a link to https://llmbenchmark.ai. Each score names its source board and the day we read it; the boards themselves keep their own terms.
CC BY 4.0 terms · Full dataset (JSON) · Methodology
Source: LLMBenchmark.ai (https://llmbenchmark.ai), CC BY 4.0