mager-bench

challenges

The 13tasks every model runs. Full prompt, full rubric, no black box — click into any of them to see exactly what's being judged.

01

fizzbuzz

9.7

Classic FizzBuzz — tests basic correctness and code style

best so far: Claude Sonnet 4.6
02

binary-search

9.0

Binary search implementation with full docs

best so far: Claude Haiku 4.5
03

api-client

7.0

Write a small HTTP API client class with error handling and docs

best so far: Llama 3.3 70B
04

readme-writer

9.0

Write a README for a CLI tool — tests documentation ability directly

best so far: Claude Sonnet 4.6
05

refactor

9.0

Refactor messy code and explain each change

best so far: Claude Haiku 4.5
06

test-writing

7.7

Write pytest tests for a provided function — tests edge-case thinking and assertion quality

best so far: Claude Haiku 4.5
07

debug

6.7

Find and fix 3 bugs in broken Python code — tests careful reading and correctness reasoning

best so far: Gemini 2.5 Flash
08

async-fetch

8.3

Write async Python for concurrent HTTP fetching with timeout and retry

best so far: Claude Sonnet 4.6
09

sql

8.7

Write a complex SQL query with CTEs, window functions, and aggregations

best so far: Gemini 2.5 Flash
10

go-test

7.7

Write idiomatic Go table-driven tests — tests knowledge of Go testing conventions

best so far: Claude Haiku 4.5
11

elixir-test

8.0

Write idiomatic Elixir ExUnit tests — tests knowledge of Elixir testing conventions

best so far: Claude Sonnet 4.6
12

doom

0.7

Build a Doom-style raycasting FPS engine in a single HTML file — the mager-bench signature challenge

best so far: Claude Haiku 4.5
13

slots

4.0

Build a Vegas-style slot machine in a single HTML file — reels, pay table, betting, wins

best so far: Claude Haiku 4.5