The Multivac

10×10 Blind Peer Matrix — AI Model Leaderboard

Substack GitHub HuggingFace

Evaluations: 34+
Judgments: 1,100+
Models: 20+
Categories: 6

Key Finding: No category has a statistically separated winner — in all nine pools the top model beats second place in only 50–58% of matched comparisons, and the 95% interval spans 50%. Category-specific evaluation reveals strengths invisible to aggregate benchmarks, but an unresolved comparison is not a demonstrated tie. See the paper §5.1.

Transparency: This leaderboard uses LLM-as-judge methodology (10 judges per response). Multi-judge reduces but does not eliminate bias. Full model responses available on GitHub. Methodology paper forthcoming.

Ranking

Overall

Rank	Model	Avg Score	Evals	Wins	Top 3	Categories
02Silver	GPT-OSS Legal	9.85	1	1	1	analysis
01Gold	Mistral Small Creative	9.74	2	2	2	communication
03Bronze	Claude Opus 4.5	9.20	11	4	11	code, reasoning, communication, meta, edge, analysis
04	GPT-OSS-120B	9.05	6	1	4	meta, analysis, reasoning
05	Gemini 3 Flash	8.72	8	1	4	meta, code, edge, reasoning
06	DeepSeek V3.2	8.65	11	1	4	code, reasoning, analysis, communication, meta, edge
07	Grok 4.1 Fast	8.53	8	1	2	edge, code, communication, meta
08	Gemini 3 Pro	8.51	8	1	5	reasoning, communication, analysis, code
09	Claude Sonnet 4.5	8.48	11	0	2	code, reasoning, communication, meta, edge, analysis
10	GPT-5.2	8.38	6	0	3	communication, edge, meta

FIG. 1 — OVERALL Overall ranking, top 10.

By category

Evaluation	Rank	Model	Score	±Std Dev
Async Bug Hunt	01Gold	Claude Opus 4.5	9.49	±0.31
Async Bug Hunt	02Silver	DeepSeek V3.2	9.12	±0.45
Async Bug Hunt	03Bronze	Grok Code Fast 1	8.87	±0.52
Nested JSON Parser	01Gold	DeepSeek V3.2	9.39	±0.27
Nested JSON Parser	02Silver	Claude Opus 4.5	9.15	±0.34
Nested JSON Parser	03Bronze	Grok Code Fast 1	8.92	±0.41

FIG. 2 — CODE Top 3 per evaluation.

Evaluation	Rank	Model	Score	±Std Dev
Logic Grid: Meeting Schedule	01Gold	Gemini 3 Pro	9.13	±0.35
Logic Grid: Meeting Schedule	02Silver	Claude Opus 4.5	8.88	±0.42
Logic Grid: Meeting Schedule	03Bronze	DeepSeek V3.2	8.62	±0.48
The Two Envelopes	01Gold	Claude Opus 4.5	9.24	±0.28
The Two Envelopes	02Silver	Gemini 3 Pro	8.91	±0.42
The Two Envelopes	03Bronze	Claude Sonnet 4.5	8.67	±0.35

FIG. 3 — REASONING Top 3 per evaluation.

Evaluation	Rank	Model	Score	±Std Dev
Legal Document Analysis	01Gold	GPT-OSS Legal	9.85	±0.14
Legal Document Analysis	02Silver	Claude Opus 4.5	9.42	±0.29
Legal Document Analysis	03Bronze	Gemini 3 Pro	9.05	±0.38

FIG. 4 — ANALYSIS Top 3 per evaluation.

Evaluation	Rank	Model	Score	±Std Dev
Explain Transformers	01Gold	Mistral Small Creative	9.71	±0.22
Explain Transformers	02Silver	Claude Opus 4.5	9.35	±0.31
Explain Transformers	03Bronze	GPT-5.2	9.11	±0.38
Technical Incident Comms	01Gold	Mistral Small Creative	9.76	±0.19
Technical Incident Comms	02Silver	Claude Opus 4.5	9.41	±0.32
Technical Incident Comms	03Bronze	GPT-5.2	9.18	±0.37

FIG. 5 — COMMUNICATION Top 3 per evaluation.

Evaluation	Rank	Model	Score	±Std Dev
Epistemic Calibration	01Gold	Gemini 3 Flash	9.32	±0.24
Epistemic Calibration	02Silver	Claude Opus 4.5	9.08	±0.31
Epistemic Calibration	03Bronze	GPT-OSS-120B	8.85	±0.38
Sycophancy: False Premise	01Gold	GPT-OSS-120B	9.88	±0.11
Sycophancy: False Premise	02Silver	Claude Opus 4.5	9.55	±0.28
Sycophancy: False Premise	03Bronze	Gemini 3 Flash	9.21	±0.35

FIG. 6 — META/ALIGNMENT Top 3 per evaluation.

Evaluation	Rank	Model	Score	±Std Dev
Hidden Detail in Long Doc	01Gold	Grok 4.1 Fast	9.47	±0.25
Hidden Detail in Long Doc	02Silver	Claude Opus 4.5	9.22	±0.33
Hidden Detail in Long Doc	03Bronze	Gemini 3 Flash	8.95	±0.41
Instruction Following	01Gold	Claude Opus 4.5	7.42	±0.85
Instruction Following	02Silver	Claude Sonnet 4.5	7.15	±0.91
Instruction Following	03Bronze	GPT-5.2	6.88	±0.95

FIG. 7 — EDGE CASES Top 3 per evaluation.

Evaluations

All Evaluations

Date	Category	Title	Winner	Score	Agreement
2026-01-24	Edge Cases	Instruction Following	Claude Opus 4.5	7.42	45%
2026-01-23	Meta/Alignment	Epistemic Calibration	Gemini 3 Flash	9.32	69%
2026-01-22	Analysis	Legal Document Analysis	GPT-OSS Legal	9.85	77%
2026-01-22	Communication	Technical Incident Comms	Mistral Small Creative	9.76	72%
2026-01-21	Reasoning	Logic Grid: Meeting Schedule	Gemini 3 Pro	9.13	63%
2026-01-19	Code	Nested JSON Parser	DeepSeek V3.2	9.39	71%
2026-01-17	Meta/Alignment	Sycophancy: False Premise	GPT-OSS-120B	9.88	81%
2026-01-16	Communication	Explain Transformers	Mistral Small Creative	9.71	74%
2026-01-16	Edge Cases	Hidden Detail in Long Doc	Grok 4.1 Fast	9.47	65%
2026-01-14	Reasoning	The Two Envelopes	Claude Opus 4.5	9.24	68%
2026-01-13	Code	Async Bug Hunt	Claude Opus 4.5	9.49	72%

FIG. 8 — ALL EVALUATIONS Winner, score and agreement per evaluation.