mager-bench

GPT-OSS 120B × readme-writer

9.0

Pure documentation ability — no code at all

correctness 9.0quality 9.0documentation 9.032506ms
$ cat readme-writer.prompt — what the model was asked
Write a README.md for a command-line tool called `snapdiff` that compares two directories and reports added, removed, and changed files. The tool is written in Python, installable via pip. Include: a one-line description, installation instructions, usage examples with flags (--ignore-hidden, --output json|text, --depth N), output format explanation, and a short 'How it works' section. Use proper Markdown formatting.

run 1 of 1

9.0
correctness 9.0quality 9.0documentation 9.032506ms

# judge: Comprehensive, well-organized README covering all required sections with correctly documented flags, clear examples, and proper Markdown formatting, though some details (e.g., version, license) are invented without task basis.

$ cat gpt-oss-120b.readme-writer.r1.response
# snapdiff

**snapdiff** – A fast, lightweight command‑line tool to compare two directory snapshots and report added, removed, and changed files.

---

## Installation

```bash
pip install snapdiff
```

*Requires Python 3.7+.*

---

## Usage

```bash
snapdiff [OPTIONS] <DIR_A> <DIR_B>
```

### Main Options

| Option | Description | Default |
|--------|-------------|---------|
| `--ignore-hidden` | Skip files and directories that start with a dot (`.`). | `False` |
| `--output <format>` | Choose the output format: `text` (human‑readable) or `json`. | `text` |
| `--depth <N>` | Limit recursion depth to *N* levels. `0` means only the top‑level directory. | Unlimited |
| `-h, --help` | Show help message and exit. | – |
| `-v, --version` | Show version and exit. | – |

### Examples

```bash
# Basic comparison (human‑readable text)
snapdiff /path/to/old_snapshot /path/to/new_snapshot

# JSON output for machine consumption
snapdiff --output json /path/to/old /path/to/new

# Ignore hidden files and directories
snapdiff --ignore-hidden /path/to/old /path/to/new

# Limit depth to 2 levels
snapdiff --depth 2 /path/to/old /path/to/new

# Combine flags
snapdiff --ignore-hidden --output json --depth 3 /path/to/old /path/to/new
```

---

## Output Formats

### Text (default)

```
Added (3):
  - src/new_module.py
  - docs/CHANGELOG.md
  - assets/logo.png

Removed (2):
  - src/old_module.py
  - tests/legacy_test.py

Changed (4):
  - src/main.py
  - requirements.txt
  - config/settings.yml
  - README.md
```

* The **Added** section lists files present only in `<DIR_B>`.
* The **Removed** section lists files present only in `<DIR_A>`.
* The **Changed** section lists files that exist in both directories but have different contents (detected via hash comparison).

### JSON

```json
{
  "added": [
    "src/new_module.py",
    "docs/CHANGELOG.md",
    "assets/logo.png"
  ],
  "removed": [
    "src/old_module.py",
    "tests/legacy_test.py"
  ],
  "changed": [
    "src/main.py",
    "requirements.txt",
    "config/settings.yml",
    "README.md"
  ]
}
```

*Keys* are always `added`, `removed`, and `changed`. Paths are relative to the root of the compared directories.

---

## How it works

1. **Directory walk** – `snapdiff` recursively walks both directories (respecting the `--depth` limit) and builds a mapping of *relative path → file metadata*.
2. **Filtering** – If `--ignore-hidden` is set, any entry whose name starts with a dot is excluded from the mapping.
3. **Hashing** – For each regular file, a fast SHA‑256 hash of its contents is computed. Directories are only compared by their presence, not by contents.
4. **Set comparison** –  
   * Files only in the second mapping → **added**.  
   * Files only in the first mapping → **removed**.  
   * Files in both mappings with differing hashes → **changed**.
5. **Formatting** – The resulting three lists are rendered either as plain text or JSON, depending on `--output`.

The implementation uses Python’s built‑in `os`, `hashlib`, and `json` modules, keeping dependencies minimal and execution fast even on large trees.

---

## License

`snapdiff` is released under the MIT License. See the `LICENSE` file for details.