mager-bench

GPT-OSS 120B × async-fetch

7.7

Async concurrency patterns + retry/timeout handling

correctness 6.0quality 8.0documentation 9.03368ms
$ cat async-fetch.prompt — what the model was asked
Write a Python async function `fetch_all(urls: list[str], timeout: float = 5.0, retries: int = 2) -> list[dict]` that fetches all URLs concurrently using `aiohttp` and returns a list of result dicts.

Each result dict must have:
  - `url`: the original URL
  - `status`: HTTP status code (int), or None on error
  - `body`: response text (str), or None on error
  - `error`: error message string, or None on success

Requirements:
1. All URLs must be fetched concurrently (not sequentially)
2. Each fetch must respect the `timeout` (per-request, not total)
3. Retry up to `retries` times on network errors or 5xx responses (exponential backoff: 0.5s, 1s, ...)
4. Never raise — errors go into the result dict
5. Type-hint everything; add a docstring
6. Show a `__main__` block that fetches 3 URLs and prints results

run 1 of 1

7.7
correctness 6.0quality 8.0documentation 9.03368ms

# judge: Good concurrency (session reuse + asyncio.gather) and proper 5xx/network-error retry filtering with exponential backoff, but the retry-count logic has an off-by-one bug (allows one extra attempt beyond `retries`), and timeout is passed as a raw float rather than an explicit aiohttp.ClientTimeout, slightly diverging from the rubric's expectations; docstrings are thorough and the main block is correctly runnable.

$ cat gpt-oss-120b.async-fetch.r1.response
```python
#!/usr/bin/env python3
"""Utility to fetch many URLs concurrently with timeout, retries and
structured results.

The public entry point is :func:`fetch_all`.  It returns a list of dictionaries
containing the original URL, the HTTP status code (or ``None`` on error),
the response body as text (or ``None`` on error) and an error message (or
``None`` on success).

Typical usage::

    >>> import asyncio
    >>> from fetch_all import fetch_all
    >>> urls = ["https://example.com", "https://httpbin.org/status/500"]
    >>> results = asyncio.run(fetch_all(urls))
    >>> for r in results:
    ...     print(r)
"""

from __future__ import annotations

import asyncio
from typing import Any, Awaitable, Callable, Dict, List, Optional

import aiohttp

# --------------------------------------------------------------------------- #
# Helper
# --------------------------------------------------------------------------- #
async def _fetch_one(
    session: aiohttp.ClientSession,
    url: str,
    *,
    timeout: float,
    retries: int,
) -> Dict[str, Any]:
    """
    Fetch a single URL with per‑request timeout and retry logic.

    Parameters
    ----------
    session:
        An open :class:`aiohttp.ClientSession` that will be reused for all
        requests.
    url:
        The URL to request.
    timeout:
        Per‑request timeout in seconds.
    retries:
        Number of retry attempts *after* the first try (i.e. ``retries=0``
        means “try once, never retry”).

    Returns
    -------
    dict
        ``{
            "url": <original url>,
            "status": <int|None>,
            "body": <str|None>,
            "error": <str|None>,
        }``
    """
    backoff = 0.5  # seconds, will be doubled after each failure
    attempt = 0

    while True:
        attempt += 1
        try:
            async with session.get(url, timeout=timeout) as resp:
                # Treat 5xx as transient errors that should be retried
                if 500 <= resp.status < 600:
                    raise aiohttp.ClientResponseError(
                        request_info=resp.request_info,
                        history=resp.history,
                        status=resp.status,
                        message=f"Server error {resp.status}",
                        headers=resp.headers,
                    )
                body = await resp.text()
                return {
                    "url": url,
                    "status": resp.status,
                    "body": body,
                    "error": None,
                }

        except (aiohttp.ClientError, asyncio.TimeoutError) as exc:
            # Store the error message; we'll retry if we have attempts left.
            err_msg = str(exc)

            if attempt > retries + 1:  # +1 because the first attempt is 0‑retry
                return {
                    "url": url,
                    "status": None,
                    "body": None,
                    "error": err_msg,
                }

            # wait before the next attempt (exponential back‑off)
            await asyncio.sleep(backoff)
            backoff *= 2  # exponential increase
            continue


# --------------------------------------------------------------------------- #
# Public API
# --------------------------------------------------------------------------- #
async def fetch_all(
    urls: List[str],
    *,
    timeout: float = 5.0,
    retries: int = 2,
) -> List[Dict[str, Any]]:
    """
    Fetch *all* URLs concurrently.

    Parameters
    ----------
    urls:
        List of URL strings to request.
    timeout:
        Per‑request timeout (seconds).  The total runtime can be longer because
        the calls are parallel.
    retries:
        Number of retry attempts after the first failed try.  Retries are
        performed on network errors, timeouts and HTTP 5xx responses using an
        exponential back‑off (0.5 s, 1 s, 2 s, …).

    Returns
    -------
    list[dict]
        A list where each element corresponds to an input URL and contains the
        keys ``url``, ``status``, ``body`` and ``error`` as described in the
        module docstring.
    """
    # Create a single session that will be shared among all concurrent fetches.
    async with aiohttp.ClientSession() as session:
        tasks: List[Awaitable[Dict[str, Any]]] = [
            _fetch_one(session, url, timeout=timeout, retries=retries) for url in urls
        ]
        # ``asyncio.gather`` runs them concurrently and returns results in the
        # same order as *tasks*.
        results = await asyncio.gather(*tasks, return_exceptions=False)
        return results


# --------------------------------------------------------------------------- #
# Demo when executed directly
# --------------------------------------------------------------------------- #
if __name__ == "__main__":
    import json

    async def _demo() -> None:
        demo_urls = [
            "https://example.com",                     # normal 200
            "https://httpbin.org/status/502",          # 5xx → will be retried
            "https://httpbin.org/delay/10",            # will timeout (default 5 s)
        ]
        print(f"Fetching {len(demo_urls)} URLs concurrently …\n")
        results = await fetch_all(demo_urls, timeout=5.0, retries=2)

        # pretty‑print the result dictionaries
        for r in results:
            print(json.dumps(r, indent=2, ensure_ascii=False))
            print("-" * 60)

    asyncio.run(_demo())
```