# ReadmeTester

A library for testing Elixir code blocks in markdown files. Ensures your documentation stays in sync with your actual code.

## Installation

Add `readme_tester` to your list of dependencies in `mix.exs`:

```elixir
def deps do
  [
    {:readme_tester, "~> 0.1.0", only: :test}
  ]
end
```

## Quick Start

Create a test file in `test/readme_test.exs`:

```elixir
defmodule MyApp.ReadmeTest do
  use ReadmeTester.Case,
    files: ["README.md"],
    base_path: File.cwd!()
end
```

Run with `mix test`.

## Configuration Options

| Option | Description | Default |
|--------|-------------|---------|
| `:files` | List of markdown files or glob patterns | Required |
| `:base_path` | Base path for resolving file paths | Current directory |
| `:setup` | Setup code prepended to every block before execution | `""` |
| `:skip_patterns` | Regex patterns for blocks to skip | `[]` |

## Markdown Annotations

Control test behavior with HTML comments placed immediately before a code block:

| Annotation | Description |
|------------|-------------|
| `skip` | Skip this block entirely |
| `no_run` | Check syntax only, don't execute |
| `should_raise` | Expect the block to raise an exception |
| `check_format` | Verify code is formatted (`mix format`) |
| `share` | Share variable bindings with subsequent blocks |

### Examples

**Skip a block:**

    <!-- readme_tester: skip -->
    ```elixir
    def deps do
      [{:my_app, "~> 1.0"}]
    end
    ```

**Syntax check only (no execution):**

    <!-- readme_tester: no_run -->
    ```elixir
    def future_feature do
      # This compiles but may not run without dependencies
      SomeModule.call()
    end
    ```

**Expect an exception:**

    <!-- readme_tester: should_raise -->
    ```elixir
    raise "This should raise!"
    ```

**Verify formatting:**

    <!-- readme_tester: check_format -->
    ```elixir
    def well_formatted do
      :ok
    end
    ```

**Share state between blocks:**

    <!-- readme_tester: share -->
    ```elixir
    user = %{name: "Alice", age: 30}
    ```

    ```elixir
    # This block can use `user` from above
    user.name
    ```

## IEx Blocks with Output Matching

Blocks marked with `iex` support output verification. Expected output can be specified in two ways:

**Traditional IEx format** (output on next line):

    ```iex
    iex> 1 + 1
    2
    iex> [1, 2, 3] |> Enum.sum()
    6
    ```

**Inline expectation** (using `# =>`):

    ```iex
    iex> 1 + 1  # => 2
    iex> :hello  # => :hello
    ```

The test fails if the actual output doesn't match the expected value. Complex values like lists, maps, and tuples are compared structurally.

## Example Configuration

```elixir
defmodule MyApp.DocsTest do
  use ReadmeTester.Case,
    files: ["README.md", "docs/**/*.md"],
    base_path: File.cwd!(),
    skip_patterns: [
      ~r/def deps do/,      # skip deps config
      ~r/config :/          # skip config snippets
    ],
    setup: "alias MyApp.{User, Order}"
end
```

## Programmatic Usage

You can also use ReadmeTester programmatically:

### Parsing

```elixir
# Parse a markdown file - returns {:ok, blocks} or {:error, reason}
{:ok, blocks} = ReadmeTester.parse_file("README.md")

# Parse markdown content directly
blocks = ReadmeTester.parse_content("```elixir\n1 + 1\n```")
```

Each block is a map with the following structure:

```elixir
%{
  code: "1 + 1",              # The code content
  line: 5,                     # Starting line number in the file
  language: "elixir",          # "elixir" or "iex"
  annotations: [:skip]         # List of annotations (e.g., :skip, :setup)
}
```

### Execution

```elixir
# Execute a single block
case ReadmeTester.execute(block, setup: "alias MyApp.Helper") do
  {:ok, result} -> IO.puts("Returned: #{inspect(result)}")
  {:ok, result, bindings} -> IO.puts("Shared: #{inspect(bindings)}")
  {:skipped, :annotated} -> IO.puts("Block was skipped")
  {:error, :compile_error, message} -> IO.puts("Compile error: #{message}")
  {:error, :runtime_error, exception, stacktrace} -> IO.puts("Runtime error")
  {:error, :output_mismatch, %{expected: e, actual: a}} -> IO.puts("Expected #{e}, got #{a}")
  {:error, :expected_exception, message} -> IO.puts("Should have raised")
  {:error, :format_error, message} -> IO.puts("Not formatted: #{message}")
end

# Execute multiple blocks in sequence (with shared state)
results = ReadmeTester.execute_all(blocks, binding: [x: 1])
# => [{%{code: ..., line: ...}, {:ok, result}}, ...]
```

### Testing Files

```elixir
# Test all files and get a summary
result = ReadmeTester.test_files(
  ["README.md", "docs/**/*.md"],
  base_path: "/path/to/project"
)
# => %{
#      total: 10,
#      passed: 8,
#      failed: 1,
#      skipped: 1,
#      failures: [
#        {"/path/to/README.md", 42, {:compile_error, "undefined function foo/0"}},
#        ...
#      ]
#    }
```

## How It Works

1. **Parse**: Extracts all `elixir` and `iex` fenced code blocks from markdown files
2. **Execute**: Runs each block through `Code.eval_string/3`
3. **Report**: Generates ExUnit test cases or returns a summary

Code blocks are executed in isolation. Each block is a separate test - if one fails, others still run.

## Examples

Simple expressions work out of the box:

```elixir
list = [1, 2, 3]
Enum.sum(list)
```

Module definitions are supported:

```elixir
defmodule Example do
  def greet(name), do: "Hello, #{name}!"
end

Example.greet("World")
```

## Tips for Testable Documentation

1. **Self-contained examples**: Write code blocks that can run independently
2. **Skip non-executable snippets**: Use `<!-- readme_tester: skip -->` for config, partial code, etc.
3. **Use the `:setup` option for imports**: Put common aliases/imports in the `:setup` option
4. **Avoid external dependencies**: Mock or skip blocks that need databases, APIs, etc.