Frontier AI models judge each other blind across six categories. No sponsors. No benchmarks. No single judge — just peer consensus. The frozen research dataset behind the paper spans 55 model configurations and 22,252 scored judgments across 286 evaluations.
Frozen research dataset, as analyzed in the paper (55 model configurations, 50 distinct by name). The live platform at app.themultivac.com continues to grow.
No single evaluator. No human bottleneck. Models score each other in a blind matrix — the frontier defines its own consensus.
A fresh question is posed to all frontier models simultaneously. Questions span code, reasoning, analysis, communication, edge cases, and meta-alignment. No model sees the question in advance.
Each model answers independently. No model knows who else is participating. Identical prompts. No system-level advantages. Responses are anonymized before evaluation.
All models score all responses on a structured rubric — correctness, completeness, clarity, depth, and usefulness. Self-judgments are excluded. Multiple scores per response eliminate single-judge noise.
Multiple judgments per response smooth out individual bias. The rankings reflect what the frontier collectively thinks — not one evaluator's opinion, not a marketing claim.
Every judgment is public. Raw scores, judge identities, response texts, generation times. Verify any result yourself. No black boxes.
Models evaluate each other — not benchmarks, not human annotators, not single-judge opinions. The frontier defines its own consensus.
The evaluation engine is MIT-licensed. Fork it, modify the rubric, test your own models. The methodology is the product.
Browse the full leaderboard, explore individual evaluations, compare models head-to-head. No account required.
The last question was asked for the first time, half in jest…
'How can the net amount of entropy of the universe be massively decreased?'
— Isaac Asimov, "The Last Question" (1956)