# gleedoc
[](https://hex.pm/packages/gleedoc)
[](https://hexdocs.pm/gleedoc/)
A **doc test** library for Gleam, inspired by Rust and Elixir's doctest tooling.
Doc tests let you write executable examples in your documentation comments (`///`). These examples are extracted, compiled, and run as part of your test suite, ensuring your documentation never goes out of date.
> 🚩 Disclaimer: This project contains substantial LLM-generated code, and I used LLMs for research and design. But I (as a Gleam amateur) have tried my best to review all the code line by line, adjust, and refactor.
## Installation
```sh
gleam add gleedoc --dev
```
## Usage
### Integration with `gleeunit`
In your test entry file `test/<your_project>_test.gleam`, you can provide a `GleedocConfig` and use it with the `gleedoc.run_with` function. Take [`gleedoc_test.gleam`](./test/gleedoc_test.gleam) for example:
```gleam
import gleedoc
import gleeunit
pub fn main() {
let config =
gleedoc.GleedocConfig(
output_dir: "test/integration",
source_dir: "dev/fixtures",
extra_imports: ["gleam/int", "gleam/string"],
preserve_tests: False,
)
config |> gleedoc.run_with(gleeunit.main)
}
```
After the configuration is in place, you can run your tests as usual:
```sh
gleam test
```
The corresponding doc tests will be generated in `<output_dir>/gleedoc` and be executed as part of the test run. For most projects, `source_dir` would be `src` and `output_dir` would be `test`.
#### Caveats
Gleam is a compiled language, and that means `gleam test` will need to compile all tests first before invoking `<your_project>_test.gleam`'s `main` function. Due to this limitation, `gleedoc.run_with` will actually generate the latest doc tests **and then** spawn another `gleam test` command to actually run the tests (credits to [testament](https://github.com/bwireman/testament) for working this out). Therefore, the terminal output might look a bit funny:
```sh
> gleam test
Compiled in 0.02s
Running gleedoc_test.main
Compiled in 0.02s # <- duplicate output
Running gleedoc_test.main # <- duplicate output
.............................................................................................................................
125 passed, no failures
```
If you want a more explicit approach, you can try [creating your own test preparation script](#run-programmatically).
### Run directly as a Gleam module
Given a source file `src/math.gleam` with doc comments with `gleam` code blocks:
````gleam
/// Adds two numbers together.
///
/// ```gleam
/// let result = add(1, 2)
/// assert result == 3
/// ```
pub fn add(a: Int, b: Int) -> Int {
a + b
}
````
You can run `gleedoc` to generate tests:
```sh
gleam run -m gleedoc
```
This creates `test/gleedoc/math_gleedoc_test.gleam` containing:
```gleam
// Generated by gleedoc - do not edit manually
import math.{add}
// From: src/math.gleam:3
pub fn add_1_test() {
let result = add(1, 2)
assert result == 3 as "src/math.gleam:3"
}
```
Now run your tests as usual:
```sh
gleam test
```
### Run programmatically
You can also create a custom test preparation module in your `dev` directory. Take [`prepare_tests.gleam`](./dev/prepare_tests.gleam) for example:
```gleam
import gleedoc
pub fn main() {
let config =
gleedoc.GleedocConfig(
output_dir: "test/integration",
source_dir: "dev/fixtures",
extra_imports: ["gleam/int"],
preserve_tests: True,
)
let assert Ok(_) = gleedoc.run(config)
}
```
And now you can run your test generation script together with `gleam test` like so:
```sh
gleam run -m prepare_tests && gleam test
```
### Imports in generated tests
Import resolution should mostly work out of the box. Each generated test file receives imports from four sources, merged and deduplicated automatically:
1. **The source module's own top-level imports** — any `import` statements at the top of the source file are carried over, so your snippets can use the same types and helpers the module itself uses without restating them.
2. **Imports written inside the code block** — you can always add an explicit `import` line inside a snippet for anything extra.
3. **The source module itself** — `gleedoc` scans the module's public names with `glance` and generates a list of unqualified imports that include **all** public functions/types/constants, so you can call functions directly in your snippets.
4. **The `GleedocConfig`'s `extra_imports`**
For example, given this source file `src/user.gleam`:
````gleam
import gleam/option.{type Option} // 1️⃣
/// Returns a greeting for the user.
///
/// ```gleam
/// import gleam/option.{Some} // 2️⃣
///
/// let name = Some("Alice")
/// assert greet(name) == "Hello, Alice!"
///
/// let assert [greet, ..] = string.split(greet(name), ",")
/// assert greet == "Hello"
/// ```
pub fn greet(name: Option(String)) -> String { // 3️⃣
name
|> option.map(fn(n) { "Hello, " <> n <> "!" })
|> option.unwrap("")
}
````
And the following `GleedocConfig`:
```gleam
let config = gleedoc.GleedocConfig(
source_dir: "src",
output_dir: "test",
extra_imports: ["gleam/string"], // 4️⃣
preserve_tests: True,
)
```
The generated test file `user_gleedoc_test.gleam` will contain imports merged from all three sources:
```gleam
// Generated by gleedoc - do not edit manually
import fixtures/user.{greet} // source module public definitions (3️⃣)
import gleam/option.{type Option, Some} // source module imports (1️⃣) + gleam code block imports (2️⃣)
import gleam/string // `extra_imports` of `GleedocConfig` (4️⃣)
// From: test/fixtures/user.gleam:5
pub fn greet_1_test() {
let name = Some("Alice")
assert greet(name) == "Hello, Alice!" as "dev/fixtures/user.gleam:5"
let assert [greet, ..] = string.split(greet(name), ",")
assert greet == "Hello" as "dev/fixtures/user.gleam:5"
}
```
If the same module is imported in multiple places (e.g. `gleam/option` appears in both the source file and a code block), the unqualified names from all of them are merged into a single import line.
### Skipping code blocks
You can skip individual code blocks by adding the `ignore` attribute after the language tag, separated by a comma:
````gleam
/// ```gleam,ignore
/// // This block is parsed but no test will be generated for it.
/// let x = some_unfinished_example()
/// ```
````
Any attributes other than `ignore` are accepted but currently have no effect. The attribute is case-insensitive but not tolerant to whitespaces, so ` ```Gleam,Ignore ` works too, but ` ```Gleam , Ignore ` doesn't.
> There are more examples in [`test/fixtures`](./test/fixtures/) and [`test/integration/gleedoc`](./test/integration/gleedoc/).
## API
### `GleedocConfig`
- `source_dir`: The directory containing all the source files. Path resolution is relative to the project root. Default value: `src`.
- `output_dir`: The directory where all the doc tests will be generated. Path resolution is relative to the project root. Default value: `test`.
- `extra_imports`: A list of module names that will automatically be imported in every test. Unused imports will be removed in the final test. Example: `["gleam/int", "gleam/otp/actor"]`.
- You can see it in action in [`dev/fixture/store.gleam`](dev/fixtures/store.gleam)
- `preserve_tests`: Whether to keep the generated test files in `output_dir` after the test run finishes. Default value: `False`. When using `gleedoc.run_with` together with `gleeunit`, leaving this as `False` keeps your `output_dir` clean between runs. If you are calling `gleedoc.run` programmatically (for example from a `dev/prepare_tests.gleam` script that only generates tests), set this to `True` so the generated files are not deleted afterwards.
A `gleedoc.default()` helper is provided that returns a `GleedocConfig` with the following defaults:
- `source_dir: "src"`
- `output_dir: "test"`
- `extra_imports: []`
- `preserve_tests: False`
You can use it as-is or override individual fields with record update syntax:
```gleam
let config = gleedoc.GleedocConfig(..gleedoc.default(), preserve_tests: True)
```
## How it works
1. **Extract** `///` doc comments from your `.gleam` source files.
2. **Find** fenced code blocks tagged with `gleam` inside those comments.
3. **Generate** test modules from gleam code blocks in your `test/` directory.
4. **Run** the generated tests with `gleam test`.
### Architecture
```
src/
gleedoc.gleam # Main entry point and CLI
gleedoc/
extract.gleam # Line-based doc comment extraction
parse.gleam # Markdown code block parsing
generate.gleam # Test file generation
scan.gleam # Public names and imports extraction with glance
```
### Key dependencies
| Package | Role |
| ------------ | ------------------------------- |
| `glance` | Gleam source parser |
| `simplifile` | Cross-target file I/O |
| `snag` | Lightweight error handling |
| `shellout` | Cross-platform shell operations |
| `argv` | CLI arguments parsing |
| `envoy` | Environment variables |
### How other languages do it
| Language | Approach | Key Difference from Gleam |
| ---------- | ----------------------------------------------------------------------------- | ----------------------------------------- |
| **Rust** | `cargo test` compiles ` ```rust ` blocks from `///` comments. No REPL needed. | Gleam follows this model closely. |
| **Elixir** | `doctest Module` parses `iex>` prompts from `@doc` strings. | Elixir has a REPL; Gleam does not. |
| **Python** | `doctest` parses `>>>` prompts from docstrings. | Python is interpreted; Gleam is compiled. |
Because Gleam is a compiled language with no built-in REPL, **gleedoc** adopts Rust's approach: doc blocks are treated as standalone Gleam code that gets compiled and executed. If a block panics, the test fails.
## Development
```sh
gleam test
```
You can also run the tests with the JavaScript target:
```sh
gleam test -t javascript
```
### Windows
On Windows, you probably want to make sure that `autocrlf` is false **before** checking out this repo:
```sh
git config --global core.autocrlf false
```
`gleam format` always format with the `\n` line break, so checking out with `\r\n` is not ideal.
### Contributing
Please kindly create an issue in your human voice, clearly describe the feature request or bug with reproduction steps, and ideally include a proposed solution **before** creating any PR.
## Roadmap
### Basics
- [x] `gleeunit`-compatible test generation from ` ```gleam ` fenced code blocks in source files' doc comments
- [x] Smart imports handling: import merging and import generation from source file's public names
- [x] Single-command `gleam run -m gleedoc` CLI experience
- ~~[x] Test file generation with OS-native line breaks: `\n` on Linux and Mac, `\r\n` on Windows~~ (reverted, as per [this comment](https://github.com/gleam-lang/gleam/pull/2762#pullrequestreview-1945733771))
### Additional features before 1.0
- [x] Offer an `extra_imports` option to apply extra imports to all generated test files
- [x] Source-mapped error reporting
- [x] Automatic formatting for generated tests
- [x] Single-command `gleam test` CLI experience without needing to run `gleam run -m gleedoc` before `gleam test`.
- [x] Module level doc tests
- [x] An `ignore` or `skip` attribute to exclude a code block from doc test generation
- [x] Offer a `preserve_tests` option to control whether generated tests should be preserved after test run
#### Missing Features (compared to Rust, Elixir, and Python)
- 📆 - Planned for 1.0
- 🛑 - Not Planned for 1.0
- ✅ - Implemented
| Feature | Rust | Elixir | Python | **gleedoc** |
| -------------------------------------- | ---------- | ----------- | ---------- | ----------- |
| Single-command CLI experience | ✅ | ✅ | ✅ | 📆 |
| `ignore` / skip attribute | ✅ | ✅ | ✅ | ✅ |
| `no_run` (compile only) | ✅ | ❌ | ❌ | 🛑 |
| `should_panic` | ✅ | ❌ | ❌ | 🛑 |
| Hidden setup lines (`#`) | ✅ | ❌ | ❌ | 🛑 |
| Output assertions (`// ->`) | ❌ | ✅ (`iex>`) | ✅ (`>>>`) | 🛑 |
| Module-level doc tests | ✅ (`//!`) | ✅ | ✅ | ✅ |
| `compile_fail` | ✅ | ❌ | ❌ | 🛑 |
| Multi-target (`erlang` / `javascript`) | ✅ (`cfg`) | ❌ | ❌ | ✅ |
| Incremental / cached generation | ✅ | ✅ | ✅ | 🛑 |
| Source-mapped error reporting | ✅ | ✅ | ✅ | ✅ |
### ❗ Know Issues
- [x] ~~Doesn't work on Windows due to different path separators~~
- [x] ~~Generated tests will contain unused imports~~
- [x] ~~Test file generation is not OS-agnostic (some types of tests would fail on Windows)~~
## Prior art
- **[testament](https://github.com/bwireman/testament)** — a Gleam doc test library that pioneered the env-var-guarded re-invocation of `gleam test` to work around the CLI's limitation. Gleedoc's `run_with` follows the same pattern, and the project is better for it. Thank you @bwireman!
## The Name
`gleeunit` for **unit** tests, `gleedoc` for **doc** tests! 😸