LLMBenchmark.ai

Enter opens the highlighted result. Esc closes.

τ²-Bench Telecom leaderboard

GLM 5.2 leads
99.1%
models measured
129
lead over #2
+0.3 pts
median
71.1%

Leader on τ²-Bench Telecom · read October 21

1 GLM 5.299.1%

On October 21, 2026, 129 of our catalog's models had a τ²-Bench Telecom result, and Z.ai GLM 5.2 led with 99.1%.

An agent helping a simulated customer with tools and a policy to follow. Share of tasks completed. Run by Artificial Analysis on its own harness.

Role in the index

We collect this benchmark and show it as supporting evidence. It is not part of the Vibe Coding Index, because it measures something other than coding and the index is about coding. Read the methodology

129 models with a result

Original board: Artificial Analysis
τ²-Bench Telecom results for models in the catalog
# Model Result Evidence Coding index Output $/Mtok
1 GLM 5.2Z.ai 99.1% 57.7 $7.00
2 GLM 4.7 FlashZ.ai 98.8% Not benchmarked $0.40
3 Claude Fable 5Anthropic 98.5% 82.1 $50.00
4 Step 3.7 FlashStepFun 98.5% Not benchmarked $1.15
5 GLM 5 TurboZ.ai 98.5% Not benchmarked $4.00
6 GLM 5V TurboZ.ai 98.5% 28.8 $4.00
7 GLM 5Z.ai 98.2% 59.7 $3.20
8 Qwen3.6 PlusQwen 97.7% 37.4 $1.95
9 Grok 4.3xAI 97.7% 22.5 $2.50
10 GLM 5.1Z.ai 97.7% 44.0 $4.40
11 Kimi K2.5Moonshot AI 95.9% 58.5 $2.50
12 Kimi K2.6Moonshot AI 95.9% 44.2 $3.41
13 GLM 4.7Z.ai 95.9% 33.6 $1.98
14 Gemini 3.1 Pro PreviewGoogle 95.6% 35.3 $12.00
15 Gemini 3.5 FlashGoogle 95.6% 42.8 $9.00
16 Qwen3.5 397B A17BQwen 95.6% 28.6 $2.34
17 MiniMax M2.5MiniMax 95.3% 59.4 $0.95
18 Qwen3.6 35B A3BQwen 95.3% Not benchmarked $0.95
19 Qwen3.7 MaxQwen 94.7% 45.1 $4.425
20 Claude Opus 4.8Anthropic 94.4% 66.0 $25.00
21 Step 3.5 FlashStepFun 94.4% Not benchmarked $0.30
22 Mistral Medium 3.5Mistral AI 94.2% 9.2 $7.50
23 Qwen3.6 27BQwen 94.2% Not benchmarked $2.00
24 MiMo-V2.5-ProXiaomi 94.2% 40.0 $0.609
25 GPT-5.5OpenAI 93.9% 67.8 $30.00
26 Qwen3.5-27BQwen 93.9% 22.6 $2.16
27 Qwen3.5-122B-A10BQwen 93.6% 22.9 $2.08
28 Kimi K2 ThinkingMoonshot AI 93.0% 63.4 $2.50
29 Qwen3.7 PlusQwen 93.0% Not benchmarked $1.28
30 Grok 4.20xAI 93.0% Not benchmarked $2.50
31 Hy3 previewTencent 92.7% 22.4 $0.60
32 Claude Opus 4.6Anthropic 92.1% 66.9 $25.00
33 GPT-5.2-CodexOpenAI 92.1% 55.1 $14.00
34 DeepSeek V3.2DeepSeek 90.6% 54.4 $0.42
35 MiMo-V2.5Xiaomi 90.6% 34.0 $0.238
36 Trinity Large ThinkingArcee AI 90.1% 5.4 $0.80
37 Kimi K2.7 CodeMoonshot AI 90.1% 39.1 $3.35
38 Claude Opus 4.5Anthropic 89.5% 66.8 $25.00
39 Qwen3.5-35B-A3BQwen 89.2% 7.7 $1.00
40 MiniMax M3MiniMax 88.9% 57.8 $1.20
41 Claude Opus 4.7Anthropic 88.6% 51.1 $25.00
42 GPT-5.4OpenAI 87.1% 37.7 $15.00
43 MiniMax M2MiniMax 86.8% 45.3 $1.02
44 Qwen3.5-9BQwen 86.8% Not benchmarked $0.13
45 GPT-5OpenAI 86.5% 72.2 $10.00
46 GPT-5.6 TerraOpenAI 86.3% 69.0 $12.00
47 Solar Pro 3Upstage 86.3% Not benchmarked $0.60
48 GPT-5.3-CodexOpenAI 86.0% 29.5 $14.00
49 MiniMax M2.1MiniMax 85.4% Not benchmarked $1.20
50 GPT-5.6 SolOpenAI 85.1% 76.7 $20.00
51 MiniMax M2.7MiniMax 84.8% 28.4 $1.20
52 GPT-5.2OpenAI 84.8% 58.9 $14.00
53 Nemotron 3 UltraNVIDIA 83.3% Not benchmarked $2.40
54 GPT-5.4 MiniOpenAI 83.3% 28.2 $4.50
55 GPT-5.1-CodexOpenAI 83.0% 50.5 $10.00
56 GPT-5.1OpenAI 81.9% 53.3 $10.00
57 Command ACohere 80.7% Not benchmarked $10.00
58 o3OpenAI 80.7% 70.7 $8.00
59 Gemini 3 Flash PreviewGoogle 80.4% 75.8 $3.00
60 Claude Sonnet 4.6Anthropic 79.5% 57.8 $15.00
61 Qwen3 Coder NextQwen 79.5% Not benchmarked $0.80
62 GLM 4.6Z.ai 76.9% 52.2 $2.20
63 GPT-5.4 NanoOpenAI 76.0% Not benchmarked $1.25
64 Kimi K2 0905Moonshot AI 73.4% Not benchmarked $2.50
65 GPT-5 MiniOpenAI 71.1% 59.8 $2.00
66 Mercury 2Inception 70.8% 0.0 $0.75
67 Nemotron 3 SuperNVIDIA 67.8% Not benchmarked $2.40
68 gpt-oss-120bOpenAI 65.8% 34.5 $0.60
69 Gemma 4 31BGoogle 65.5% 38.3 $0.40
70 GPT-5.1-Codex-MiniOpenAI 62.9% 6.2 $2.00
71 o1OpenAI 62.6% 61.7 $60.00
72 Kimi K2 0711Moonshot AI 61.1% 64.7 $2.30
73 gpt-oss-20bOpenAI 60.2% Not benchmarked $0.09
74 o4 MiniOpenAI 55.6% 68.6 $4.40
75 Gemini 2.5 ProGoogle 54.1% 37.0 $10.00
76 Qwen3 VL 235B A22B InstructQwen 54.1% Not benchmarked $1.90
77 Qwen3 235B A22B Instruct 2507Qwen 53.2% Not benchmarked $0.55
78 GPT-4.1 MiniOpenAI 52.9% 28.5 $1.60
79 DeepSeek V3 0324DeepSeek 47.1% 42.0 $1.14
80 GPT-4.1OpenAI 47.1% 46.5 $8.00
81 GLM 4.5 AirZ.ai 46.5% Not benchmarked $0.85
82 Gemma 4 26B A4BGoogle 43.6% 22.8 $0.255
83 Qwen3 Coder 480B A35BQwen 43.6% 50.0 $1.00
84 GLM 4.5Z.ai 43.0% 64.2 $2.20
85 Qwen3 Next 80B A3B InstructQwen 41.5% Not benchmarked $1.10
86 Mistral Small 4Mistral AI 41.2% Not benchmarked $0.60
87 Nemotron 3 Nano 30B A3BNVIDIA 40.9% Not benchmarked $0.24
88 Mistral Medium 3.1Mistral AI 40.6% Not benchmarked $2.00
89 Nova Premier 1.0Amazon 38.3% Not benchmarked $12.50
90 DeepSeek V3.1 TerminusDeepSeek 37.1% Not benchmarked $1.00
91 R1 0528DeepSeek 36.5% 71.4 $2.15
92 GPT-5 NanoOpenAI 36.5% 34.8 $0.40
93 Qwen3 14BQwen 34.5% Not benchmarked $0.24
94 Qwen3 Coder 30B A3B InstructQwen 34.5% 60.4 $0.28
95 Qwen2.5 72B InstructQwen 34.5% Not benchmarked $0.40
96 DeepSeek V3.2 ExpDeepSeek 33.9% 55.0 $0.41
97 Mistral Large 2407Mistral AI 33.0% Not benchmarked $6.00
98 Gemini 2.5 FlashGoogle 31.6% 28.7 $2.50
99 GLM 4.6VZ.ai 31.6% Not benchmarked $0.90
100 Gemini 3.1 Flash LiteGoogle 31.3% Not benchmarked $1.50
101 o3 MiniOpenAI 31.3% 52.1 $4.40
102 Qwen3 32BQwen 29.8% 40.0 $0.28
103 Qwen3 VL 8B InstructQwen 29.2% Not benchmarked $0.75
104 GPT-4o (2024-08-06)OpenAI 28.9% 23.1 $10.00
105 Qwen3 30B A3B Instruct 2507Qwen 28.1% Not benchmarked $0.30
106 Llama 3.3 70B InstructMeta Llama 26.6% Not benchmarked $0.32
107 Qwen3 30B A3BQwen 26.0% Not benchmarked $0.50
108 GPT-4oOpenAI 25.1% 38.8 $10.00
109 Mistral Medium 3Mistral AI 24.3% Not benchmarked $2.00
110 GLM 4.5VZ.ai 22.5% Not benchmarked $1.80
111 Llama 3.2 3B InstructMeta Llama 21.1% Not benchmarked $0.335
112 Qwen3 VL 30B A3B InstructQwen 19.9% Not benchmarked $0.60
113 Mistral Small 3Mistral AI 19.6% Not benchmarked $0.08
114 Gemini 2.5 Flash LiteGoogle 19.0% Not benchmarked $0.40
115 Llama 4 MaverickMeta Llama 17.8% 18.1 $0.6525
116 Nova Lite 1.0Amazon 17.5% Not benchmarked $0.24
117 GPT-4.1 NanoOpenAI 17.3% 8.9 $0.40
118 Llama 3.1 8B InstructMeta Llama 16.4% Not benchmarked $0.08
119 Llama 4 ScoutMeta Llama 15.5% 9.1 $0.30
120 Llama 3.1 70B InstructMeta Llama 15.2% Not benchmarked $0.40
121 Nova Micro 1.0Amazon 14.0% Not benchmarked $0.14
122 Nova Pro 1.0Amazon 14.0% Not benchmarked $3.20
123 Granite 4.0 MicroIBM Granite 12.6% Not benchmarked $0.112
124 Gemma 3 12BGoogle 10.8% Not benchmarked $0.15
125 Gemma 3 27BGoogle 10.5% 4.9 $0.35
126 Gemma 3 4BGoogle 5.0% Not benchmarked $0.05
127 Llama 3.2 1B InstructMeta Llama 0.0% Not benchmarked $0.201
128 Phi 4Microsoft 0.0% Not benchmarked $0.14
129 Reka Flash 3Reka 0.0% Not benchmarked $0.20

Only models in our catalog are listed, and each appears once with its best published run. 195 rows were read from the source on October 21, 2026, and rows that match no catalog model are left out.