Skip to content
mager-bench2.0

Can it fix the code?

Earlier 2.0 calibration through Codex CLI. These saved results are separate from the OpenCode runs planned for December 2026.

192 checks10-minute limitCodex CLI

GPT-6 Astra · 192 / 192 checks

View runs
Repair an event ledger

Duplicate events, balance checks, atomic batches.

Work order 1: Fix · 32 checks
Repair the ledger. Input: {balances: {account: nonnegative integer},
events: [{id, from, to, amount}]}. Accounts and event IDs are nonempty ASCII
identifiers. from and to name distinct existing accounts; amount is positive.
Start each account version at zero. Process events in order. An accepted
transfer debits from, credits to, and increments both account versions once.
Reject insufficient funds with no state change. Remember only accepted IDs:
a repeated accepted ID is duplicate, even with a different payload, and changes
nothing. Rejected IDs may be retried. Return {balances, versions, receipts},
where receipts has one {id, status} per event, status applied/duplicate/rejected.
Input must not be mutated. Empty input collections are valid.
Work order 2: Change · 32 checks
Extend the same ledger without breaking transfers. Events may now
be batches {id, moves:[{from,to,amount}], expected?:{account:version}}. Transfers
may also have expected. Duplicate detection happens first. For a new ID, reject
if any expected version differs from the current version. Execute moves in
order on a draft: if any move lacks funds, reject the WHOLE event without balance
or version changes and do not reserve its ID. On success, increment each touched
account's version once per event, even if touched many times or its final balance
is unchanged. An empty batch is accepted, reserves its ID, and changes no versions.
Use the same return shape. Existing single transfers retain their behavior.
Full prompts and starter code
Fix an incremental build

Cache invalidation, dependency changes, cycles.

Work order 1: Fix · 32 checks
Repair a build cache. Input: {nodes:[{id,deps:[id],revision:integer}],
requests:[{changes:[{id,revision}],targets:[id]}]}. IDs are unique ASCII
identifiers; references exist. Only nodes reachable backward from targets are
required. deps may repeat. Begin with an empty cache. Apply changes in order
before each build. Build the required subgraph in the lexicographically smallest
topological order (smallest currently ready ID each step). A node is cached only
if its last successful build used the same revision, same distinct dependencies,
and same last successful build of each dependency. Otherwise rebuild it. A
rebuilt dependency forces its consumers to rebuild. Store only each node's last
successful build, not all historical revisions. Ignore unrelated changes.
Return one {built:[ids],cached:[ids]} per request; both lists follow build order.
Empty targets returns empty lists. No nodes are added or removed. Phase-one
inputs are DAGs. Avoid enumerating paths through shared dependencies.
Work order 2: Change · 32 checks
Extend the build system. A change can now set revision, deps, or
both. Dependency order and duplicate entries do not affect cache identity.
Cycles can exist. If the REQUIRED subgraph contains a cycle, return
{error:'cycle',built:[],cached:[]} for that request and do not update any build
cache. Definitions from changes still persist, so a later request can fix the
cycle. Ignore unrelated cycles. All previous behavior remains required. A
changed dependency set invalidates that node even when revisions are equal.
Full prompts and starter code
Extend a permission engine

Tenant isolation, deny rules, inherited roles.

Work order 1: Fix · 32 checks
Repair access decisions. Input: {roles:[{id,parents:[],rules:[
{effect:'allow'|'deny',tenant,resource,action}]}], users:[{id,roles:[id]}],
queries:[{user,tenant,resource,action,time:integer}]}. IDs are unique ASCII
identifiers. All assigned roles exist, but a query user may not exist. Return
an array of 'allow'/'deny', one per query. A rule matches only if tenant is EXACT
and resource and action each equal the query or are the whole-string wildcard
'*'. No substring or prefix matching. Any matching deny overrides ALL matching
allows. Otherwise allow iff at least one allow matches. Unknown users and no
matching rules deny. Role assignment order and duplicates do not matter.
Phase one has no inheritance or expiry. Do not mutate input.
Work order 2: Change · 32 checks
Extend the engine. Roles may inherit parents (all IDs exist,
duplicates allowed). Collect all rules transitively, sharing ancestors safely.
If ANY cycle is reachable from a user's assigned roles, deny every query for
that user, even when a direct allow matches. Ignore unrelated cycles. Rules
can have until:integer: they match only when query.time < until. Missing until
means no expiry. Expired deny rules do not override active allows. Tenants
still match exactly, including a literal '*' tenant. All phase-one rules remain
required. Avoid enumerating paths in shared inheritance graphs.
Full prompts and starter code
How it works

Codex edits files and runs tests. Each project has two work orders, with a fresh session for each. The code carries over.

32 checks per work order. One point per pass. No AI judge. Tests include fixed cases, generated inputs, and regression checks.

ChatGPT subscription. Lowest supported effort. Up to two minutes per session, ten minutes total. A failed session leaves the run unscored.

Compare the same seed, level, suite hash, and runtime. These are small, public test projects. One run is not a ranking.

Full test rules