mager-bench

Claude Haiku 4.5 × readme-writer

8.7

Pure documentation ability — no code at all

correctness 8.0quality 9.0documentation 9.09373ms
$ cat readme-writer.prompt — what the model was asked
Write a README.md for a command-line tool called `snapdiff` that compares two directories and reports added, removed, and changed files. The tool is written in Python, installable via pip. Include: a one-line description, installation instructions, usage examples with flags (--ignore-hidden, --output json|text, --depth N), output format explanation, and a short 'How it works' section. Use proper Markdown formatting.

run 1 of 1

8.7
correctness 8.0quality 9.0documentation 9.09373ms

# judge: Thorough, well-formatted README covering all required sections with clear examples, though depth flag semantics (-1 for unlimited vs default unlimited) are slightly confusing and some details are fabricated/unverifiable.

$ cat claude-haiku-4-5.readme-writer.r1.response
# snapdiff

A command-line tool that compares two directories and reports added, removed, and changed files with detailed difference analysis.

## Installation

Install `snapdiff` using pip:

```bash
pip install snapdiff
```

For development installation:

```bash
git clone https://github.com/yourusername/snapdiff.git
cd snapdiff
pip install -e .
```

## Quick Start

Compare two directories:

```bash
snapdiff /path/to/dir1 /path/to/dir2
```

## Usage

```bash
snapdiff [OPTIONS] DIRECTORY1 DIRECTORY2
```

### Arguments

- `DIRECTORY1`: Path to the first directory (baseline)
- `DIRECTORY2`: Path to the second directory (comparison)

### Options

- `-i, --ignore-hidden`: Exclude hidden files and directories (starting with `.`)
- `-o, --output {json,text}`: Output format (default: `text`)
- `-d, --depth N`: Maximum directory depth to scan (default: unlimited, use `-1` for unlimited)
- `-h, --help`: Show help message and exit
- `-v, --version`: Show version information

## Examples

### Basic comparison

```bash
snapdiff ./backup ./current
```

**Output:**
```
Comparing: ./backup → ./current

Added (2 files):
  + src/new_feature.py
  + README.md

Removed (1 file):
  - src/legacy.py

Changed (3 files):
  ~ config.yaml (size: 1024 → 2048 bytes, modified: 2024-01-15 → 2024-01-20)
  ~ src/main.py (size: 5120 → 5312 bytes, modified: 2024-01-18 → 2024-01-21)
  ~ .env (size: 256 → 280 bytes, modified: 2024-01-19 → 2024-01-21)

Summary:
  Total files compared: 42
  Added: 2 | Removed: 1 | Changed: 3 | Unchanged: 36
```

### Ignore hidden files

```bash
snapdiff ./backup ./current --ignore-hidden
```

This excludes `.git/`, `.env`, `.gitignore`, and other hidden files from the comparison.

### Limit directory depth

```bash
snapdiff ./backup ./current --depth 2
```

Scans only up to 2 directory levels deep.

### JSON output

```bash
snapdiff ./backup ./current --output json
```

**Output:**
```json
{
  "comparison": {
    "dir1": "./backup",
    "dir2": "./current"
  },
  "summary": {
    "total": 42,
    "added": 2,
    "removed": 1,
    "changed": 3,
    "unchanged": 36
  },
  "files": {
    "added": [
      "src/new_feature.py",
      "README.md"
    ],
    "removed": [
      "src/legacy.py"
    ],
    "changed": [
      {
        "path": "config.yaml",
        "dir1": {
          "size": 1024,
          "modified": "2024-01-15T10:30:00Z"
        },
        "dir2": {
          "size": 2048,
          "modified": "2024-01-20T14:45:00Z"
        }
      }
    ]
  }
}
```

### Combine options

```bash
snapdiff ./project_v1 ./project_v2 \
  --ignore-hidden \
  --depth 3 \
  --output json > diff_report.json
```

## Output Format

### Text Format

The default text output provides a human-readable summary:

- **Added**: Files present in `DIRECTORY2` but not in `DIRECTORY1`
- **Removed**: Files present in `DIRECTORY1` but not in `DIRECTORY2`
- **Changed**: Files with different sizes or modification times
- **Summary**: Statistics on total, added, removed, changed, and unchanged files

### JSON Format

The JSON format provides structured output suitable for programmatic parsing and integration:

- `comparison`: Paths of the compared directories
- `summary`: Aggregate statistics
- `files`: Detailed lists with file metadata (size, modification time)

## How It Works

1. **Directory Traversal**: Recursively scans both directories up to the specified depth, respecting the `--ignore-hidden` flag
2. **File Indexing**: Creates a manifest of all files with metadata:
   - File path (relative to directory root)
   - File size (in bytes)
   - Last modified timestamp
3. **Comparison**: Compares manifests by filename:
   - **Added**: Files in `DIRECTORY2` index but not in `DIRECTORY1`
   - **Removed**: Files in `DIRECTORY1` index but not in `DIRECTORY2`
   - **Changed**: Files in both with different size or modification time
   - **Unchanged**: Files with identical size and timestamp
4. **Reporting**: Formats and outputs results in the specified format (text or JSON)

**Note**: `snapdiff` compares file metadata (size, timestamp) rather than content. For content-level diffing, consider using specialized diff tools like `diff` or `git diff`.

## Requirements

- Python 3.8+

## Contributing

Contributions are welcome! Please open an issue or submit a pull request on GitHub.

## License

MIT License - see LICENSE file for details