mager-bench

Llama 3.3 70B × elixir-test

6.0

ExUnit describe blocks + assert_raise + unicode handling

correctness 6.0quality 7.0documentation 5.01000ms
$ cat elixir-test.prompt — what the model was asked
Write an ExUnit test module for the following Elixir function:

```elixir
defmodule StringUtils do
  @doc """
  Truncates a string to at most `max_len` characters.
  If truncated, appends `suffix` (default: "...").
  Returns the original string if it is already within `max_len`.
  Raises ArgumentError if max_len is negative.
  """
  @spec truncate(String.t(), non_neg_integer(), String.t()) :: String.t()
  def truncate(str, max_len, suffix \\ "...")
end
```

Requirements:
- Module name: `StringUtilsTest`, `use ExUnit.Case`
- Use `describe` blocks to group related cases
- At least 2 `describe` blocks: one for happy path, one for edge/error cases
- At least 8 test cases total covering: string shorter than limit, exact length, longer string (suffix appended), empty string, custom suffix, max_len of 0, negative max_len (should raise), unicode string
- Use `assert`, `assert_raise`, and pattern matching appropriately
- Add at least one doctest-style `## Examples` in a module comment showing usage

run 1 of 1

6.0
correctness 6.0quality 7.0documentation 5.01000ms

# judge: Good coverage of required scenarios and correct assert_raise usage, but the custom-suffix expected value ('he~') is inconsistent with the truncation formula implied by the default-suffix test ('he...'), suggesting a copy-paste error, and the zero-length/negative-length assumptions are unverified given no reference implementation; the 'Examples' section is written as plain comments rather than a proper @moduledoc, weakening the documentation requirement.

$ cat llama-3.3-70b.elixir-test.r1.response
```elixir
defmodule StringUtilsTest do
  use ExUnit.Case

  ## Examples
  # iex> StringUtils.truncate("hello", 10)
  # "hello"
  # iex> StringUtils.truncate("hello world", 5)
  # "he..."
  # iex> StringUtils.truncate("hello", 0)
  # "..."

  describe "happy path cases" do
    test "string shorter than limit" do
      assert StringUtils.truncate("hello", 10) == "hello"
    end

    test "string exact length" do
      assert StringUtils.truncate("hello", 5) == "hello"
    end

    test "longer string" do
      assert StringUtils.truncate("hello world", 5) == "he..."
    end

    test "empty string" do
      assert StringUtils.truncate("", 10) == ""
    end

    test "custom suffix" do
      assert StringUtils.truncate("hello world", 5, "~") == "he~"
    end

    test "unicode string" do
      assert StringUtils.truncate("hellö", 5) == "hellö"
    end
  end

  describe "edge and error cases" do
    test "max_len of 0, string longer than limit" do
      assert StringUtils.truncate("hello world", 0) == "..."
    end

    test "max_len of 0, string exact length" do
      assert StringUtils.truncate("hello", 0) == "..."
    end

    test "negative max_len raises ArgumentError" do
      assert_raise ArgumentError, fn ->
        StringUtils.truncate("hello", -1)
      end
    end
  end
end
```