LLMBenchmark.ai

Enter opens the highlighted result. Esc closes.

Qwen3.5 397B A17B vs Step 5 Preview

Updated October 21, 2026. 37 benchmarks have a result for both. Coding is ranked on the Vibe Coding Index; every other board is shown as published.

Replace Qwen3.5 397B A17B Replace Step 5 Preview
Qwen3.5 397B A17BQwen
Coding
28.6
Overall
21.4
Output $/M
$2.34
9
Step 5 PreviewStepFun
Coding
71.4
Overall
43.7
Output $/M
$2.70
28
vs boards won of 37 shared

Decision reading

Pick Step 5 Preview for coding: it scores 71.4 on the Vibe Coding Index against 28.6 for Qwen3.5 397B A17B.

Pick Qwen3.5 397B A17B when output tokens cost 1.2x less, or when what matters is LMArena Vision: homework or LMArena Text: science, where it leads.

Model AQwen3.5 397B A17BQwen28.6/1001 independent benchmarkNo cost per solved taskModel BStep 5 PreviewStepFun71.4/1002 independent benchmarksNo cost per solved task
Shared results
37
Qwen3.5 397B A17B only
28
Step 5 Preview only
24
Like-for-like categories
6 of 9

Shared results can support a head-to-head reading. Results only one model has describe coverage, not superiority.

Which one for your work

Each card is decided by one number both models carry. Where only one has it, the card says so and makes no call.

At a glance

Coding index
Qwen3.5 397B A17B 28.6 from 1 benchmark, Step 5 Preview 71.4 from 2 benchmarks.
Widest category
Intelligence, 22.3 points apart on average over 1 shared benchmark.
Widest single gap
LMArena WebDev: React: Qwen3.5 397B A17B 1388 against Step 5 Preview 1583, 32.5 points apart on a 0 to 100 scale.
Output price
Qwen3.5 397B A17B $2.34 against Step 5 Preview $2.70 per million output tokens, 1.2x apart.
Context window
Qwen3.5 397B A17B takes 262K tokens, Step 5 Preview 1.05M.

Benchmark by benchmark

First the score: boards each model leads, per category, where both have a result. Then every board, a category to a fold; bars are each result on a 0 to 100 scale, so compare the two in a row, not one row with another.

Boards won, where both have a result
CategoryQwen3.5 397B A17BBarsStep 5 PreviewTies
Coding06
Intelligence01
Reasoning15
Math02
Knowledge31
General513
All shared boards928
CodingLike-for-likeFour of these boards feed the Vibe Coding Index06
Coding benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
LiveBench codingNot measured85.6%Only Step 5 Preview measured
LMArena WebDev14001567Step 5 Preview
Terminal-Bench 4 (Vals AI)Not measured31.8%Only Step 5 Preview measured
Vibe Code Bench (Vals AI)Not measured70.4%Only Step 5 Preview measured
Code migration (Vals AI)Not measured39.0%Only Step 5 Preview measured
IOI (Vals AI)Not measured38.1%Only Step 5 Preview measured
SWE-bench Multimodal (Vals AI)Not measured40.8%Only Step 5 Preview measured
SciCode44.8%58.9%Step 5 Preview
Terminal-Bench Hard (Artificial Analysis run)40.9%Not measuredOnly Qwen3.5 397B A17B measured
ALE-Bench621.9Not measuredOnly Qwen3.5 397B A17B measured
LMArena Text: coding14641485Step 5 Preview
LMArena Text: software and IT14631473Step 5 Preview
LMArena WebDev: React13881583Step 5 Preview
LMArena WebDev: HTML14101556Step 5 Preview
LMArena WebDev: from an imageNot measured1550Only Step 5 Preview measured
Neither model has a result on the other 23 boards here
IntelligenceLike-for-likePublic boards, shown as published, kept out of the index01
Intelligence benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
AA Intelligence Index21.443.7Step 5 Preview
Epoch Capabilities Index146.8Not measuredOnly Qwen3.5 397B A17B measured
Agentic workDirectional onlyPublic boards, shown as published, kept out of the index13 boards, none shared
Agentic work benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
LiveBench researchNot measured72.8%Only Step 5 Preview measured
LiveBench browserNot measured24.0%Only Step 5 Preview measured
LiveBench long agenticNot measured27.5%Only Step 5 Preview measured
SREBench (Vals AI)Not measured0.4%Only Step 5 Preview measured
Finance Agent (Vals AI)Not measured50.7%Only Step 5 Preview measured
Tax Agent Bench (Vals AI)Not measured27.3%Only Step 5 Preview measured
Legal Research Bench (Vals AI)Not measured40.4%Only Step 5 Preview measured
Excel Modeling Benchmark (Vals AI)Not measured57.1%Only Step 5 Preview measured
BioMysteryBench (Vals AI)Not measured67.4%Only Step 5 Preview measured
Terminal-Bench-Science (Vals AI)Not measured2.9%Only Step 5 Preview measured
Vals Science IndexNot measured29.6%Only Step 5 Preview measured
τ²-Bench Telecom95.6%Not measuredOnly Qwen3.5 397B A17B measured
APEX-Agents24.9%Not measuredOnly Qwen3.5 397B A17B measured
Neither model has a result on the other 26 boards here
ReasoningLike-for-likePublic boards, shown as published, kept out of the index15
Reasoning benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
LiveBench reasoningNot measured83.8%Only Step 5 Preview measured
MysteryMechanism (Vals AI)Not measured18.5%Only Step 5 Preview measured
GPQA Diamond (Artificial Analysis run)89.3%Not measuredOnly Qwen3.5 397B A17B measured
Humanity's Last Exam (Artificial Analysis run)29.0%46.5%Step 5 Preview
GPQA Diamond (Epoch AI run)86.4%Not measuredOnly Qwen3.5 397B A17B measured
Mystery game puzzles18.0%Not measuredOnly Qwen3.5 397B A17B measured
LMArena Text: hard prompts14481461Step 5 Preview
LMArena Text: expert14621485Step 5 Preview
LMArena Text: hard prompts in English14501454Step 5 Preview
LMArena Vision: diagrams12771308Step 5 Preview
LMArena Vision: homework12921265Qwen3.5 397B A17B
Neither model has a result on the other 10 boards here
MathLike-for-likePublic boards, shown as published, kept out of the index02
Math benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
OTIS Mock AIME88.9%Not measuredOnly Qwen3.5 397B A17B measured
FrontierMath, tiers 1 to 331.2%Not measuredOnly Qwen3.5 397B A17B measured
ProofBenchNot measured42.0%Only Step 5 Preview measured
LMArena Text: math14531458Step 5 Preview
LMArena Text: mathematical work14541460Step 5 Preview
Neither model has a result on the other 8 boards here
KnowledgeLike-for-likePublic boards, shown as published, kept out of the index31
Knowledge benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
LMArena Text: science14571454Qwen3.5 397B A17B
LMArena Text: medicine and healthcare14551439Qwen3.5 397B A17B
LMArena Text: legal and government14411437Qwen3.5 397B A17B
LMArena Text: business and finance14351435Step 5 Preview
Neither model has a result on the other 13 boards here
Web designDirectional onlyPublic boards, shown as published, kept out of the index8 boards, none shared
Web design benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
Design Arena: websites1191Not measuredOnly Qwen3.5 397B A17B measured
Design Arena: UI components1163Not measuredOnly Qwen3.5 397B A17B measured
Design Arena: game dev1141Not measuredOnly Qwen3.5 397B A17B measured
Design Arena: data viz1175Not measuredOnly Qwen3.5 397B A17B measured
Design Arena: 3D1161Not measuredOnly Qwen3.5 397B A17B measured
Design Arena: all code categories1180Not measuredOnly Qwen3.5 397B A17B measured
Design Arena: SVG1139Not measuredOnly Qwen3.5 397B A17B measured
LiveBench designNot measured76.9%Only Step 5 Preview measured
Neither model has a result on the other 1 board here
GeneralLike-for-likePublic boards, shown as published, kept out of the index513
General benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
LiveBench data analysisNot measured63.5%Only Step 5 Preview measured
Public Benefits Bench (Vals AI)Not measured58.1%Only Step 5 Preview measured
IFBench78.8%Not measuredOnly Qwen3.5 397B A17B measured
Long context reasoning (AA-LCR)77.3%88.3%Step 5 Preview
DTBench87.5%Not measuredOnly Qwen3.5 397B A17B measured
LMCA37.9%Not measuredOnly Qwen3.5 397B A17B measured
LMArena Text14371440Step 5 Preview
LMArena Text: instruction following14241438Step 5 Preview
LMArena Text: creative writing14021409Step 5 Preview
LMArena Vision12621265Step 5 Preview
LMArena Text: English14431447Step 5 Preview
LMArena Text: non-English14301428Qwen3.5 397B A17B
LMArena Text: Chinese14981500Step 5 Preview
LMArena Text: German1445Not measuredOnly Qwen3.5 397B A17B measured
LMArena Text: Japanese14291383Qwen3.5 397B A17B
LMArena Text: Korean1384Not measuredOnly Qwen3.5 397B A17B measured
LMArena Text: French14621438Qwen3.5 397B A17B
LMArena Text: Spanish14411441Qwen3.5 397B A17B
LMArena Text: Russian14301433Step 5 Preview
LMArena Text: Polish1447Not measuredOnly Qwen3.5 397B A17B measured
LMArena Text: multi-turn14461437Qwen3.5 397B A17B
LMArena Text: longer queries14411449Step 5 Preview
LMArena Text: writing and language14151425Step 5 Preview
LMArena Text: entertainment and media13991412Step 5 Preview
LMArena Vision: English12681290Step 5 Preview
LMArena Vision: Chinese1314Not measuredOnly Qwen3.5 397B A17B measured
LMArena Vision: reading text12711282Step 5 Preview
LMArena Vision: humor1262Not measuredOnly Qwen3.5 397B A17B measured
LMArena Vision: captioning1230Not measuredOnly Qwen3.5 397B A17B measured
LMArena Vision: recognizing things1265Not measuredOnly Qwen3.5 397B A17B measured
LMArena Vision: creative writing1260Not measuredOnly Qwen3.5 397B A17B measured
Neither model has a result on the other 15 boards here
Image generationNot comparablePublic boards, shown as published, kept out of the index0 boards, none shared
Image generation benchmarks for Qwen3.5 397B A17B and Step 5 Preview
BenchmarkQwen3.5 397B A17BStep 5 PreviewLeader
Neither model has a result on 5 boards here

What each workload costs

Three fixed token mixes at each model's listed price, so per-token rates become a bill.

Chat turn

1K input + 500 output tokens

Qwen3.5 397B A17B$0.0016Fits in one request
Step 5 Preview$0.0024Fits in one request

Qwen3.5 397B A17B costs less

Repository review

50K input + 3K output tokens

Qwen3.5 397B A17B$0.027Fits in one request
Step 5 Preview$0.058Fits in one request

Qwen3.5 397B A17B costs less

Agent loop

200K input + 10K output tokens

Qwen3.5 397B A17B$0.101Fits in one request
Step 5 Preview$0.227Fits in one request

Qwen3.5 397B A17B costs less

Listed OpenRouter rates, without cache or batch discounts. A workload that does not fit a model's context window would need splitting.

Specification differences

The leading value in a row is marked; a row with a missing value names no leader. Our two rows are marked in red.

Specifications of Qwen3.5 397B A17B and Step 5 Preview
SpecificationQwen3.5 397B A17BStepFun Step 5 Preview
MakerQwenStepFun
WeightsOpen weightsOpen weights
ReleasedFebruary 16, 2026October 8, 2026
Context window262K1.05M
Input $/Mtok$0.39$1.00
Output $/Mtok$2.34$2.70
Coding index28.671.4
Independent measurements behind it12
Cost per solved taskNot computableNot computable
SourceOpenRouter listingOpenRouter listing

Questions

Is Qwen3.5 397B A17B better than Step 5 Preview for coding?

Pick Step 5 Preview for coding: it scores 71.4 on the Vibe Coding Index against 28.6 for Qwen3.5 397B A17B. Pick Qwen3.5 397B A17B when output tokens cost 1.2x less, or when what matters is LMArena Vision: homework or LMArena Text: science, where it leads.

Which is cheaper, Qwen3.5 397B A17B or Step 5 Preview?

Qwen3.5 397B A17B costs $0.39 per million input tokens and $2.34 per million output tokens; Step 5 Preview costs $1.00 and $2.70. Cost per solved task is known for neither of them.

Which has the bigger context window, Qwen3.5 397B A17B or Step 5 Preview?

Step 5 Preview, with 1.05M tokens against 262K.

How many benchmarks is the Qwen3.5 397B A17B vs Step 5 Preview comparison based on?

37 benchmarks have a result for both models as of October 21, 2026. Qwen3.5 397B A17B's coding index rests on 1 independent benchmark and Step 5 Preview's on 2. Where only one model has a result, the page says so instead of naming a winner.

Compare next