Skip to main content

README.md

# gleedoc

[![Package Version](https://img.shields.io/hexpm/v/gleedoc)](https://hex.pm/packages/gleedoc)
[![Hex Docs](https://img.shields.io/badge/hex-docs-ffaff3)](https://hexdocs.pm/gleedoc/)

A **doc test** library for Gleam, inspired by Rust and Elixir's doctest tooling.

Doc tests let you write executable examples in your documentation comments (`///`). These examples are extracted, compiled, and run as part of your test suite, ensuring your documentation never goes out of date.

> 🚩 Disclaimer: This project contains substantial LLM-generated code, and I used LLMs for research and design. But I (as a Gleam amateur) have tried my best to review line by line, adjust, and refactor.

## How it works

1. **Extract** `///` doc comments from your `.gleam` source files.
2. **Find** fenced code blocks tagged with `gleam` inside those comments.
3. **Generate** test modules in your `test/` directory.
4. **Run** the generated tests with `gleam test`.

### How other languages do it

| Language   | Approach                                                                      | Key Difference from Gleam                 |
| ---------- | ----------------------------------------------------------------------------- | ----------------------------------------- |
| **Rust**   | `cargo test` compiles ` ```rust ` blocks from `///` comments. No REPL needed. | Gleam follows this model closely.         |
| **Elixir** | `doctest Module` parses `iex>` prompts from `@doc` strings.                   | Elixir has a REPL; Gleam does not.        |
| **Python** | `doctest` parses `>>>` prompts from docstrings.                               | Python is interpreted; Gleam is compiled. |

Because Gleam is a compiled language with no built-in REPL, **gleedoc** adopts Rust's approach: doc blocks are treated as standalone Gleam code that gets compiled and executed. If a block panics, the test fails.

## Installation

```sh
gleam add gleedoc --dev
```

## Usage

Write doc comments with `gleam` code blocks in your source files:

````gleam
// src/math.gleam

/// Adds two numbers together.
///
/// ```gleam
/// let result = add(1, 2)
/// assert result == 3
/// ```
pub fn add(a: Int, b: Int) -> Int {
  a + b
}
````

Then run gleedoc to generate tests:

```sh
gleam run -m gleedoc
```

This creates `test/gleedoc/math_gleedoc_test.gleam` containing:

```gleam
// Generated by gleedoc - do not edit manually

import math.{add}

// From: src/math.gleam:4
pub fn add_1_test() {
  let result = add(1, 2)
  assert result == 3
}
```

Now run your tests as usual:

```sh
gleam test
```

### Imports in generated tests

Each generated test file receives imports from three sources, merged and deduplicated automatically:

1. **The source module itself**`gleedoc` scans the module's public names with `glance` and generates a list of unqualified imports that include **all** public functions/types/constants, so you can call functions directly in your snippets.
2. **The source module's own top-level imports** — any `import` statements at the top of the source file are carried over, so your snippets can use the same types and helpers the module itself uses without restating them.
3. **Imports written inside the code block** — you can always add an explicit `import` line inside a snippet for anything extra.

For example, given this source file `src/user.gleam`:

````gleam
import gleam/option.{type Option}              // 1️⃣

/// Returns a greeting for the user.
///
/// ```gleam
/// import gleam/option.{Some}                 // 2️⃣
///
/// let name = Some("Alice")
/// assert greet(name) == "Hello, Alice!"
/// ```
pub fn greet(name: Option(String)) -> String { // 3️⃣
  name
  |> option.map(fn(n) { "Hello, " <> n <> "!" })
  |> option.unwrap("")
}
````

The generated test file `user_gleedoc_test.gleam` will contain imports merged from all three sources:

```gleam
// Generated by gleedoc - do not edit manually

import fixtures/user.{greet}             // source module public definitions (3️⃣)
import gleam/option.{type Option, Some}  // source module imports (1️⃣) + gleam code block imports (2️⃣)

// From: test/fixtures/user.gleam:5
pub fn greet_1_test() {
  let name = Some("Alice")
  assert greet(name) == "Hello, Alice!"
}
```

If the same module is imported in multiple places (e.g. `gleam/option` appears in both the source file and a code block), the unqualified names from all of them are merged into a single import line.

> There are more examples in [`test/fixtures`](./test/fixtures/) and [`test/integration/gleedoc`](./test/integration/gleedoc/).

## API

You can also use gleedoc programmatically from your test suite:

```gleam
// test/gleedoc_setup.gleam
import gleedoc

pub fn main() {
  let config = gleedoc.GleedocConfig(
    source_dir: "src",
    output_dir: "test",
    extra_imports: [],
  )

  case gleedoc.run(config) {
    Ok(Nil) -> Nil
    Error(snag) -> panic as snag.issue
  }
}
```

### The `GleedocConfig`

- `source_dir`: The directory containing all the source files. Path resolution is relative to the project root. Default value: `src`.
- `output_dir`: The directory where all the doc tests will be generated. Path resolution is relative to the project root. Default value: `test`.
- `extra_imports`: A list of module names that will automatically be imported in every test. Unused imports will be removed in the final test. Example value: `["gleam/int", "gleam/otp/actor"]`.
  - You can see it in action in [`dev/fixture/store.gleam`](dev/fixtures/store.gleam)

## Architecture

```
src/
  gleedoc.gleam           # Main entry point and CLI
  gleedoc/
    extract.gleam         # Line-based doc comment extraction
    parse.gleam           # Markdown code block parsing
    generate.gleam        # Test file generation
    scan.gleam            # Public names and imports extraction with glance
```

### Key dependencies

| Package      | Role                       |
| ------------ | -------------------------- |
| `glance`     | Gleam source parser        |
| `simplifile` | Cross-target file I/O      |
| `snag`       | Lightweight error handling |

## Development

```sh
gleam run -m prepare_tests && gleam test
```

You can also run the tests with the JavaScript target:

```sh
gleam run -m prepare_tests && gleam test -t javascript
```

### Windows

On Windows, you probably want to make sure that `autocrlf` is false **before** checking out this repo:

```sh
git config --global core.autocrlf false
```

### Contributing

Please kindly create an issue in your human voice, clearly describe the feature request or bug with reproduction steps, and ideally include a proposed solution **before** creating any PR.

## Roadmap

### Basics

- [x] `gleeunit`-compatible test generation from ` ```gleam ` fenced code blocks in source files' doc comments
- [x] Smart imports handling: import merging and import generation from source file's public names
- [x] Single-command `gleam run -m gleedoc` CLI experience
- ~~[x] Test file generation with OS-native line breaks: `\n` on Linux and Mac, `\r\n` on Windows~~ (reverted, as per [this comment](https://github.com/gleam-lang/gleam/pull/2762#pullrequestreview-1945733771))

### Additional features before 1.0

- [x] Offer an `extra_imports` option to apply extra imports to all generated test files
- [ ] Source-mapped error reporting
- [ ] Automatic formatting for generated tests
- [ ] Single-command `gleam test` CLI experience without needing to run `gleam run -m gleedoc` before `gleam test`.
- [ ] Module level doc tests
- [ ] An `ignore` or `skip` attribute to exclude a code block from doc test generation

#### Missing Features (compared to Rust, Elixir, and Python)

📆 - Planned for 1.0
🛑 - Not Planned for 1.0
✅ - Implemented

| Feature                                | Rust       | Elixir      | Python     | **gleedoc** |
| -------------------------------------- | ---------- | ----------- | ---------- | ----------- |
| Single-command CLI experience          |||| 📆          |
| `ignore` / skip attribute              |||| 📆          |
| `no_run` (compile only)                |||| 🛑          |
| `should_panic`                         |||| 🛑          |
| Hidden setup lines (`#`)               |||| 🛑          |
| Output assertions (`// ->`)            || ✅ (`iex>`) | ✅ (`>>>`) | 🛑          |
| Module-level doc tests                 | ✅ (`//!`) ||| 📆          |
| `compile_fail`                         |||| 🛑          |
| Multi-target (`erlang` / `javascript`) | ✅ (`cfg`) ||||
| Incremental / cached generation        |||| 🛑          |
| Source-mapped error reporting          |||| 📆          |

### ❗ Know Issues

- [x] ~~Doesn't work on Windows due to different path separators~~
- [x] ~~Generated tests will contain unused imports~~
- [x] ~~Test file generation is not OS-agnostic (some types of tests would fail on Windows)~~

## The Name

`gleeunit` for **unit** tests, `gleedoc` for **doc** tests! 😸