mager-bench

v1.1 // 13-challenge personal coding bench

Opinionated coding tasks, scored by an LLM judge on correctness, code quality, and documentation. Free-tier models run free. Paid models get crowdfunded.

#01 GPT-OSS 120B

6.4

/ 10 leader · 5 models scored

avg latency: 8580ms
challenges: 13
judge: claude-sonnet-5
last run: Sun, 26 Jul 2026 14:21:14 UTC

leaderboard

#	model	cost	crct · qual · docs	avg score	avg latency	n
01	GPT-OSS 120B→	free	5.8 · 6.5 · 6.7	6.4	8580ms	13
02	Claude Sonnet 4.6→	paid	5.9 · 5.9 · 6.6	6.2	33426ms	13
03	Claude Haiku 4.5→	cheap	6.2 · 6.2 · 6.4	6.2	13326ms	13
04	Llama 3.3 70B→	free	5.3 · 5.5 · 5.8	5.6	2244ms	13
05	Gemini 2.5 Flash→	free	4.8 · 5.0 · 5.7	5.2	26981ms	13

crct · qual · docs are the rubric dimensions averaged across all 13 challenges — the headline average hides whether a model is correct-but-undocumented or well-written-but-wrong.

click a model for its full challenge breakdown, raw responses, and judge notes. free + cheap run by default; paid models land when crowdfunded.

the 13 challenges

full specs + rubrics →

fizzbuzz→9.4

Classic FizzBuzz — tests basic correctness and code style

binary-search→8.2

Binary search implementation with full docs

api-client→3.7

Write a small HTTP API client class with error handling and docs

readme-writer→8.1

Write a README for a CLI tool — tests documentation ability directly

refactor→9.0

Refactor messy code and explain each change

test-writing→5.4

Write pytest tests for a provided function — tests edge-case thinking and assertion quality

debug→5.7

Find and fix 3 bugs in broken Python code — tests careful reading and correctness reasoning

async-fetch→6.3

Write async Python for concurrent HTTP fetching with timeout and retry

sql→7.6

Write a complex SQL query with CTEs, window functions, and aggregations

go-test→5.9

Write idiomatic Go table-driven tests — tests knowledge of Go testing conventions

elixir-test→6.7

Write idiomatic Elixir ExUnit tests — tests knowledge of Elixir testing conventions

doom→0.5

Build a Doom-style raycasting FPS engine in a single HTML file — the mager-bench signature challenge

slots→0.2

Build a Vegas-style slot machine in a single HTML file — reels, pay table, betting, wins

score shown is the board average across all 5 scored models — a rough difficulty read. every model attempts every challenge.

fund the bench

10 paid models still waiting. Free tiers (Groq / Gemini) never need a card — crowdfunding buys the expensive head-to-heads.

open fund drive →

$0 / $100 season goal · 0%

not scored yet

Claude Opus 4.8unfunded~$8 / full suite
Claude Sonnet 5unfunded~$4 / full suite
GPT-4ounfunded~$5 / full suite
Gemini 2.5 Prounfunded~$4 / full suite
GPT-5.6unfunded~$6 / full suite
Claude Fable 5unfunded~$10 / full suite
Claude Opus 4.6unfunded~$7 / full suite
Claude Opus 4.7unfunded~$8 / full suite
GLM 2.5unfunded~$2 / full suite
Kimi K3unfunded~$2 / full suite

put them on the board →

methodology, honestly

Every score comes from a single LLM judge — currently claude-sonnet-5 — pinned across all runs so scores stay comparable. That judge is a Claude model scoring a board that includes Claude models: same-family bias is real, so treat cross-family gaps as directional, not gospel. Thinking and verdict budgets are separate contracts in the harness, and every score links to its raw response and judge notes so you can audit the reasoning instead of trusting the number.

api

$ curl /api/results → 200 OK

Same data behind this page, as JSON. Cached 1h at the edge.