Skip to main content

README.md

# llms_txt

Generate offline [`llms.txt`](https://llmstxt.org/) files from your Elixir
project's dependency documentation.

`llms_txt` provides a `mix llms.txt` task that fetches the HTML hexdocs for
every Hex dependency at its installed version, converts each page to Markdown
with [`htmd`](https://hex.pm/packages/htmd), and writes one `llms.txt` per
package:

```
priv/llms/<package>/<version>/llms.txt
```

These plain-text docs are handy to keep checked in for AI assistants and other
tools that work better with local, offline documentation.

## Installation

Add it as a dev-only dependency in the project whose dependencies you want to
document:

```elixir
def deps do
  [
    {:llms_txt, "~> 0.1.0", only: [:dev], runtime: false}
  ]
end
```

Or track the latest unreleased changes straight from GitHub:

```elixir
def deps do
  [
    {:llms_txt, github: "fuww/llms_txt", only: [:dev], runtime: false}
  ]
end
```

The task ships inside the package, so once it is compiled `mix llms.txt`
becomes available in your project — no extra setup needed.

## Usage

```bash
# Fetch hexdocs and (re)generate priv/llms for the current dependencies
mix llms.txt

# Write to a different directory
mix llms.txt --output doc/llms

# Skip the hex.docs fetch and only re-convert docs already fetched locally
mix llms.txt --no-fetch
```

The task reads the calling project's `mix.lock`, so it always documents the
versions actually locked in that project. Git and path dependencies are skipped
(they have no HexDocs).

After generating, it prunes stale output: outdated version directories and
packages that are no longer dependencies are removed, so the output directory
always mirrors the current lock file. Re-run `mix llms.txt` after updating
packages to keep the docs in sync.

## How it works

| Module | Responsibility |
| --- | --- |
| `Mix.Tasks.Llms.Txt` | Orchestration: option parsing, `hex.docs fetch`, concurrency, reporting. |
| `LlmsTxt` | Lockfile + filesystem: locating packages, the hexdocs cache, writing and pruning. |
| `LlmsTxt.Converter` | Pure HTML → Markdown conversion (chrome stripping, escape handling). |

## Development

```bash
mix deps.get
mix test
mix format
```