mager-bench

Claude Sonnet 4.6 × readme-writer

9.0

Pure documentation ability — no code at all

correctness 9.0quality 9.0documentation 9.026268ms
$ cat readme-writer.prompt — what the model was asked
Write a README.md for a command-line tool called `snapdiff` that compares two directories and reports added, removed, and changed files. The tool is written in Python, installable via pip. Include: a one-line description, installation instructions, usage examples with flags (--ignore-hidden, --output json|text, --depth N), output format explanation, and a short 'How it works' section. Use proper Markdown formatting.

run 1 of 1

9.0
correctness 9.0quality 9.0documentation 9.026268ms

# judge: Comprehensive, well-organized README covering all required sections with correct flag documentation, clear examples, and proper Markdown/code-block usage, though it invents extra details (e.g., MD5, placeholder repo links) not specified in the prompt.

$ cat claude-sonnet-4-6.readme-writer.r1.response
# snapdiff

> A fast, lightweight CLI tool for comparing two directory snapshots and reporting added, removed, and changed files.

---

## Table of Contents

- [Installation](#installation)
- [Usage](#usage)
- [Flags & Options](#flags--options)
- [Output Formats](#output-formats)
- [Examples](#examples)
- [How It Works](#how-it-works)
- [License](#license)

---

## Installation

Requires **Python 3.8+**

```bash
pip install snapdiff
```

To install the latest development version directly from GitHub:

```bash
pip install git+https://github.com/yourname/snapdiff.git
```

Verify the installation:

```bash
snapdiff --version
```

---

## Usage

```
snapdiff [OPTIONS] DIR_A DIR_B
```

| Argument | Description                              |
|----------|------------------------------------------|
| `DIR_A`  | The **baseline** (original) directory    |
| `DIR_B`  | The **target** (modified) directory      |

`snapdiff` walks both directories, computes file checksums, and classifies every file as **added**, **removed**, or **changed** relative to the baseline.

---

## Flags & Options

| Flag                        | Default  | Description                                                                 |
|-----------------------------|----------|-----------------------------------------------------------------------------|
| `--ignore-hidden`           | `False`  | Skip hidden files and directories (those starting with `.`)                 |
| `--output [text\|json]`     | `text`   | Set the output format. Use `json` for machine-readable results              |
| `--depth N`                 | No limit | Limit directory traversal to **N** levels deep (e.g. `--depth 2`)          |
| `--no-color`                | `False`  | Disable colored terminal output (useful for piping or logging)              |
| `--summary`                 | `False`  | Print only the summary counts, not the full file list                       |
| `--version`                 | —        | Show the installed version and exit                                         |
| `--help`                    | —        | Show the help message and exit                                              |

---

## Output Formats

### `text` (default)

Human-readable, color-coded output printed to `stdout`.

```
Comparing:
  A (baseline) : /path/to/dir_a
  B (target)   : /path/to/dir_b

[+] ADDED      src/new_feature.py
[-] REMOVED    src/old_module.py
[~] CHANGED    README.md
[~] CHANGED    config/settings.yaml

────────────────────────────────
Summary: 1 added | 1 removed | 2 changed | 0 unchanged
```

- `[+]` — File exists in `DIR_B` but **not** in `DIR_A`
- `[-]` — File exists in `DIR_A` but **not** in `DIR_B`
- `[~]` — File exists in both but its content has **changed**

---

### `json`

Structured output for scripting, CI pipelines, or downstream tooling.

```json
{
  "baseline": "/path/to/dir_a",
  "target": "/path/to/dir_b",
  "summary": {
    "added": 1,
    "removed": 1,
    "changed": 2,
    "unchanged": 0
  },
  "diff": {
    "added": [
      "src/new_feature.py"
    ],
    "removed": [
      "src/old_module.py"
    ],
    "changed": [
      "README.md",
      "config/settings.yaml"
    ]
  }
}
```

All file paths in the JSON output are **relative to their respective root directories**.

---

## Examples

### Basic comparison

```bash
snapdiff ./project-v1 ./project-v2
```

### Skip hidden files (e.g. `.git`, `.env`)

```bash
snapdiff --ignore-hidden ./project-v1 ./project-v2
```

### Output as JSON (great for CI/CD pipelines)

```bash
snapdiff --output json ./project-v1 ./project-v2
```

### Save JSON report to a file

```bash
snapdiff --output json ./project-v1 ./project-v2 > diff-report.json
```

### Limit traversal to top 2 directory levels

```bash
snapdiff --depth 2 ./project-v1 ./project-v2
```

### Combine flags — shallow, hidden-aware, JSON output

```bash
snapdiff --ignore-hidden --depth 3 --output json ./project-v1 ./project-v2
```

### Print summary only (no file list)

```bash
snapdiff --summary ./project-v1 ./project-v2
```

```
Summary: 1 added | 1 removed | 2 changed | 0 unchanged
```

### Pipe into `jq` for filtering

```bash
snapdiff --output json ./project-v1 ./project-v2 | jq '.diff.changed'
```

---

## How It Works

`snapdiff` uses a three-step process to compare directories:

1. **Walk & Index**
   Both `DIR_A` and `DIR_B` are recursively traversed using Python's `os.walk()`. Each file is recorded as a path **relative to its root directory**. The `--depth` flag limits how many levels deep the walk descends, and `--ignore-hidden` filters out any path component beginning with a `.`.

2. **Checksum**
   For every discovered file, `snapdiff` computes an **MD5 checksum** by reading the file in chunks (default: 64 KB). This is memory-efficient for large files and avoids loading entire files into RAM. Checksums are stored in a dictionary keyed by relative path.

3. **Classify**
   The two dictionaries are compared using set operations:

   | Operation                              | Classification |
   |----------------------------------------|----------------|
   | Path in `DIR_B` only                   | `ADDED`        |
   | Path in `DIR_A` only                   | `REMOVED`      |
   | Path in both, checksums **differ**     | `CHANGED`      |
   | Path in both, checksums **match**      | `UNCHANGED`    |

   Results are then rendered in the chosen output format (`text` or `json`).

> **Note:** `snapdiff` compares **file contents** via checksum, not metadata like timestamps or permissions. A file that was `touch`-ed but not edited will appear as `UNCHANGED`.

---

## License

MIT © 2024 Your Name

---

*Found a bug or want a feature? [Open an issue](https://github.com/yourname/snapdiff/issues) — contributions are welcome!*