SmolLM3-3B-Base at launch
Independent boards have not scored it on enough tests for the LLMBenchmark Index yet; below are the results HuggingFaceTB reported and the independent scores published so far. HuggingFaceTB's own announcement reports 27 benchmark results, including Flores-200 (40.2%) and Belebele (40.2).
What HuggingFaceTB reported at launch
From HuggingFaceTB's announcement. The maker's own numbers, not re-run by us.
-
Belebele
40.2
Base Pre-Trained Model; Arabic
-
Belebele
43.8
Base Pre-Trained Model; Chinese
-
Belebele
51
Base Pre-Trained Model; French
-
Belebele
48.4
Base Pre-Trained Model; German
-
Belebele
47.4
Base Pre-Trained Model; Russian
-
Belebele
47
Base Pre-Trained Model; Spanish
-
Flores-200
40.2%
Base Pre-Trained Model; 5-shot; Arabic
#4 of 4 models with a reported result
-
Flores-200
29.2%
Base Pre-Trained Model; 5-shot; Chinese
-
Flores-200
62.8%
Base Pre-Trained Model; 5-shot; French
-
Flores-200
56.6%
Base Pre-Trained Model; 5-shot; German
-
Flores-200
52.6%
Base Pre-Trained Model; 5-shot; Italian
-
Flores-200
60.9%
Base Pre-Trained Model; 5-shot; Portuguese
Show all 27
-
Flores-200
47.1%
Base Pre-Trained Model; 5-shot; Russian
-
Flores-200
48.3%
Base Pre-Trained Model; 5-shot; Spanish
-
Global MMLU (CF)
28.6%
Base Pre-Trained Model; Arabic
-
Global MMLU (CF)
36.2%
Base Pre-Trained Model; Chinese
-
Global MMLU (CF)
38.4%
Base Pre-Trained Model; French
-
Global MMLU (CF)
35.1%
Base Pre-Trained Model; German
-
Global MMLU (CF)
37.0%
Base Pre-Trained Model; Italian
-
Global MMLU (CF)
36.9%
Base Pre-Trained Model; Portuguese
-
Global MMLU (CF)
36.5%
Base Pre-Trained Model; Russian
-
Global MMLU (CF)
38.5%
Base Pre-Trained Model; Spanish
-
MLMM Hellaswag
63.9
Base Pre-Trained Model; French
-
MLMM Hellaswag
59.6
Base Pre-Trained Model; German
-
MLMM Hellaswag
62.5
Base Pre-Trained Model; Italian
-
MLMM Hellaswag
63.2
Base Pre-Trained Model; Portuguese
-
MLMM Hellaswag
65.9
Base Pre-Trained Model; Spanish
Independent results so far
Not published