README.md

# graded

[![Package Version](https://img.shields.io/hexpm/v/graded)](https://hex.pm/packages/graded)
[![Hex Docs](https://img.shields.io/badge/hex-docs-ffaff3)](https://hexdocs.pm/graded/)
[![CI](https://github.com/alvivi/graded/actions/workflows/ci.yml/badge.svg)](https://github.com/alvivi/graded/actions/workflows/ci.yml)

> Effect checking for Gleam.

**graded** verifies that your Gleam functions respect their declared effect budgets. The tool reads and writes a single spec file at the root of your package — your Gleam source stays untouched.

## Quick start

```sh
gleam add --dev graded
```

Infer effects for your project:

```sh
gleam run -m graded infer
```

This scans `src/`, analyses every function, and writes two outputs:

- **`<package_name>.graded`** at the project root — the spec file. Contains the inferred effects of every *public* function plus any hand-written `check` invariants, `external effects` hints, and `type` field annotations. Tracked in git, ships to consumers if you add it to `included_files` in `gleam.toml`.
- **`build/.graded/<module>.graded`** — per-module cache files. Contain the inferred effects of *every* function (public and private). Regenerated freely on each `graded infer` run, never shipped (`build/` is gitignored).

### Lustre example

In a [Lustre](https://hexdocs.pm/lustre/) app, `view` must be pure — it builds HTML from the model without side effects. Enforce this with graded:

```gleam
// src/app.gleam
import gleam/io
import lustre/element.{type Element}
import lustre/element/html

pub fn view(model: Model) -> Element(Msg) {
  io.println("rendering")  // oops — side effect in view!
  html.div([], [html.text(model.name)])
}
```

```
// app.graded — at the project root
check app.view : []
```

```sh
$ gleam run -m graded check
src/app.gleam: view calls gleam/io.println with effects [Stdout] but declared []

graded: 1 violation(s) found
```

Remove the `io.println` and the check passes. Lustre's `init` and `update` functions are also pure — they return `#(Model, Effect(Msg))` where `Effect` is a data description, not an executed side effect.

Function names in the spec file are **module-qualified**: `app.view` means the `view` function in module `app`. Use slashes for nested module paths (`app/router.handle_request`).

### General example

Constrain any function's effect budget:

```
// app.graded — at the project root
check app/router.handle_request : [Http, Stdout]
```

If `handle_request` does something outside its budget (like writing to a database), graded reports the violation with the call site.

## Project layout

```
myapp/
├── src/
│   ├── myapp.gleam
│   └── myapp/
│       └── router.gleam
├── myapp.graded          ← spec file (tracked, shipped, hand-editable)
├── build/
│   └── .graded/          ← cache (gitignored, not shipped)
│       ├── myapp.graded
│       └── myapp/
│           └── router.graded
├── gleam.toml
└── ...
```

## Configuration

graded reads its configuration from a `[tools.graded]` table in `gleam.toml`. Both fields are optional — omit them to get the defaults.

```toml
[tools.graded]
spec_file = "myapp.graded"      # default: "<package_name>.graded"
cache_dir = "build/.graded"     # default: "build/.graded"
```

## Publishing your spec file to consumers

If you're a library author and want downstream packages to read your effect annotations, add the spec file to `included_files` in your `gleam.toml`:

```toml
included_files = [
  "src",
  "myapp.graded",        # ← add this so consumers see your effects
  "gleam.toml",
  "README.md",
]
```

The cache directory under `build/` is gitignored and never ships, regardless of `included_files`.

## How it works

Four annotation kinds, all in the spec file:

- **`effects mod.fn : [...]`** — inferred public-API effects, regenerated by `graded infer`. Replaced on each run; do not edit by hand.
- **`check mod.fn : [...]`** — invariant, enforced by `graded check`. Violations break the build.
- **`type mod.Type.field : [...]`** — declares effects for function-typed fields on custom types.
- **`external effects mod.fn : [...]`** — declares effects for external / third-party functions.

The checker walks the Gleam AST (via [glance](https://hexdocs.pm/glance/)), resolves imports, follows local calls transitively, and unions the effect sets. If the actual effects aren't a subset of the declared budget, it's a violation.

Effect knowledge is resolved in priority order:

1. **Your spec file** — `check`, `external effects`, and `type` field declarations you wrote in `<package_name>.graded`
2. **Cross-module project effects** — inferred effects from sibling modules in the same project, propagated in topological order
3. **Dependency spec files** — shipped by libraries at `build/packages/<dep>/<dep_spec_file>` (each dep's spec file path is read from its own `[tools.graded]` config)
4. **Path dependencies** — local deps declared with `path = "..."` in `gleam.toml`. graded reads their spec files; if missing, it falls back to inferring from source.
5. **Bundled catalog** — versioned catalog files shipped with graded (see below)
6. **Conservative default** — unknown functions get `[Unknown]`

## Higher-order functions

Functions that accept callbacks can declare parameter effect bounds:

```
// f must be pure — safe_map inherits no effects from its callback
check myapp.safe_map(f: []) : []

// apply passes through f's effects
effects myapp.apply(f: [Stdout]) : [Stdout]
```

When checking, calls to bounded parameters (like `f(x)` inside `apply`) use the declared bound instead of `[Unknown]`.

### Effect polymorphism

For functions whose effects depend on their callback, use lowercase effect variables:

```
// validate_range's effects are whatever to_error's effects are
effects myapp.validate_range(to_error: [e]) : [e]

// map_with_log carries [Stdout] on top of f's effects
effects myapp.map_with_log(f: [e]) : [Stdout, e]
```

`graded infer` produces these automatically when it sees a function calling a parameter with a `fn(...) -> ...` type annotation — the variable is named after the parameter. At each call site, graded binds the variable to the concrete effects of the argument passed:

- A function reference (`io.println`) → its effects from the knowledge base
- A type constructor (`OutOfRange`) → pure `[]`
- The caller's own bounded parameter → that bound's effects
- Anything else (inline closure, computed expression) → `[Unknown]`

Both labeled (`validate_range(42, to_error: OutOfRange)`) and positional (`validate_range(42, OutOfRange)`) arguments resolve.

## Type field effects

For types with function-typed fields, declare their effects at the type level. Type names use the same module-qualified form as function names — module path with slashes, then `.TypeName.field`:

```
type myapp.Handler.on_click : [Dom]
type myapp/router.Request.send : [Http]
```

When checking `handler.on_click(event)`, graded looks up the parameter's type annotation to resolve the field's effects. Parameters must be explicitly typed in the Gleam source for this to work.

## External declarations

Annotate third-party library functions without modifying the library:

```
external effects gleam/httpc.send : [Http]
external effects simplifile.read : [FileSystem]
external effects gleam/otp/actor.start : [Process]
```

Externals are merged into the knowledge base before both `infer` and `check`, so calls to these functions resolve with the declared effects instead of `[Unknown]`. This is also the right mechanism for **FFI functions** — declare their effects so callers propagate correctly.

## Effect set syntax

Four shapes appear inside brackets:

- **`[]`** — pure; no effects. The bottom of the effect lattice.
- **`[Label1, Label2, ...]`** — a specific set of effect labels.
- **`[_]`** — wildcard; the top of the effect lattice. When used as a declared budget, `[_]` means "any effects are permitted here" and matches anything. Useful for entrypoints (`main`) or for deliberately un-restricted parameter bounds (`check run(f: [_]) : [_]`).
- **`[e]`, `[e1, e2]`** (lowercase-initial tokens) — effect variables for polymorphic signatures. See [Effect polymorphism](#effect-polymorphism).

Note on wildcards: because `[_]` is lattice top, it absorbs everything in unions. If a function's inferred effects would be `[Stdout, e]` (polymorphic) but its declared type is `[_]`, the variable info is subsumed. That's correct but can be surprising — if you want polymorphism, avoid declaring wildcard bounds.

## Effect labels

Effect labels are plain strings — you can use any name. The bundled catalog uses these conventions:

| Label | Meaning | Example functions |
|---|---|---|
| `Stdout` | Writes to standard output | `gleam/io.println`, `gleam/io.debug` |
| `Stderr` | Writes to standard error | `gleam/io.print_error` |
| `Stdin` | Reads from standard input | `gleam/erlang.get_line` |
| `Process` | Spawns, sends to, or manages BEAM processes | `gleam/erlang/process.send`, `gleam/otp/actor.start` |
| `Http` | Network HTTP requests | `gleam/httpc.send`, `lustre_http.get` |
| `FileSystem` | Reads or writes the filesystem | `simplifile.read`, `simplifile.write` |
| `Dom` | Browser DOM manipulation | `lustre.start`, `lustre.register` |
| `Time` | Reads system clock or timezone | `gleam/time/timestamp.system_time`, `gleam/time/calendar.local_offset` |

You can define your own labels for project-specific effects:

```
external effects my_app/email.send : [Email]
external effects my_app/metrics.record : [Telemetry]
check my_app/api.handle_request : [Http, Email]
```

- `graded infer` regenerates the inferred `effects` lines in the spec file while preserving `check`, `type`, `external`, comments, and blank lines.
- `graded format` normalizes spacing and sorting in the spec file.

## Effect catalog

graded ships with versioned catalog files for common Gleam packages, so you get effect knowledge out of the box without writing `external effects` declarations for standard libraries.

Catalog files live in `priv/catalog/` and are named `{package}@{version}.graded`. At load time, graded reads your project's `manifest.toml` to determine installed dependency versions, then selects the highest catalog version that doesn't exceed the installed version.

For example, if you have `gleam_stdlib@0.71.0` installed and the catalog has `gleam_stdlib@0.70.0.graded`, that file is used — effects don't change between patch versions. A new catalog file is only needed when a library adds modules or changes effect semantics.

### Covered packages

| Package | Effects | Labels |
|---|---|---|
| **gleam_stdlib** | `gleam/io.*` | `Stdout`, `Stderr` |
| **gleam_erlang** | `gleam/erlang/process.*` | `Process`, `Stdin`, `FileSystem` |
| **gleam_otp** | `gleam/otp/actor.*`, `gleam/otp/supervisor.*` | `Process` |
| **gleam_httpc** | `gleam/httpc.send` | `Http` |
| **lustre** | `lustre.start`, `lustre.send`, `lustre/server_component.*` | `Process`, `Dom` |
| **lustre_http** | `lustre_http.*` | `Http` |
| **simplifile** | `simplifile.*` | `FileSystem` |
| **gleam_time** | `gleam/time/timestamp.system_time`, `gleam/time/calendar.local_offset`, `.utc_offset` | `Time` |

Pure (all functions `[]`): **gleam_http**, **gleam_json**, **filepath**, **gleam_regexp**, **gleam_yielder**, **gleam_crypto**, **houdini**, **tom**.

For packages not in the catalog, use `external effects` declarations in your project's spec file.

## Commands

```sh
gleam run -m graded check [directory]         # enforce check annotations (default)
gleam run -m graded infer [directory]         # infer and write effects annotations
gleam run -m graded format [directory]        # normalize .graded file formatting
gleam run -m graded format --check [directory] # verify formatting (CI mode)
gleam run -m graded format --stdin            # format from stdin (editor integration)
```

## Limitations

graded performs **syntax-level analysis** using [glance](https://hexdocs.pm/glance/) — it walks the AST without type inference and without inter-procedural value flow. This keeps the tool simple and avoids depending on compiler internals, but leaves a few patterns unresolved:

- **Cross-function record construction is opaque.** When `let v = Validator(to_error: MyError)` and the field call `v.to_error(42)` happen in the same function, graded binds the field to `MyError` and resolves the call. When the record is constructed in one function and passed to another (`foo(v)`), the receiving function only sees `v` as an opaque parameter. Use the type-level annotation (`type myapp.Validator.to_error : [...]`) for cross-function cases, or refactor the construction into the consuming function.

- **No second-order polymorphism.** Effect variables are flat — `apply(f: [e]) : [e]` works, but a function whose callback itself takes a callback can't propagate effects transitively. Nested effect variables would require unification/fixpoint machinery (full effect inference, à la Koka or Granule), which is outside graded's deliberately lightweight design.

- **Unusual pipe target shapes aren't tracked.** `x |> foo`, `x |> foo.bar`, `x |> foo(args)`, and `x |> foo.bar(args)` all work, including with positional argument substitution for polymorphic callees. Less common shapes like `x |> { let f = bar(); f }` don't have a static callee name for the extractor to hang argument tracking off.

- **Cross-module resolution requires `graded infer` first.** If module A calls module B, B's effects are only available after `graded infer` writes B's entry in the spec file and cache. Once `graded infer` has been run, transitive chains of any depth resolve in a single pass — modules are processed in topological order over the import graph.

- **External code is opaque.** Erlang/JavaScript FFI implementations, pre-compiled dependencies without `.graded` files, and dynamically dispatched calls cannot be analyzed. Use `external effects` declarations to annotate these manually.

In practice, idiomatic Gleam code (inline callbacks, direct calls, pipe chains, higher-order functions passing functions by name, validator/handler/config records constructed and used in the same function) is handled correctly. Function references passed to higher-order functions are tracked via auto-inferred polymorphic signatures (`effects map(f: [e]) : [e]`) and bound at each call site; locally-bound function-ref aliases (`let f = io.println; f(x)`), transitive aliases (`let g = f`), and field calls on same-function record constructions (`let v = Validator(to_error: E); v.to_error(x)`) also resolve.

## Future work

The following features would progressively close the remaining limitations. Ordered by incrementality — earlier items are smaller, later items push into different territory.

### Hand-written field bounds

Extend parameter bounds to accept a path expression, so users can declare a field's effects at the function boundary when graded can't figure it out on its own:

```
check myapp.view(handler.on_click: [Dom]) : [Dom]
```

This is a syntax extension to `ParamBound` (path instead of identifier), no analysis required — the user declares what a record field's effects are, and substitution works exactly like first-order param bounds. Covers the escape-hatch case for field calls and for any other value flow graded can't trace.

### Same-function value flow

A small dataflow pass over each function body tracking three kinds of local bindings:

1. **Function-ref alias** — `let f = io.println` → `f` is a callable with `[Stdout]`
2. **Record construction** — `let v = Validator(to_error: MyError)` → `v`'s fields map to values
3. **Transitive alias** — `let g = f` → chain lookup

With this, field calls on locally-constructed records resolve automatically (closing most of the field-call limitation), and pipe targets like `let f = bar; x |> f` also resolve. Doesn't cross function boundaries — construction sites that happen in another function remain opaque.

### Effect unification

Full effect inference with nested variables — `apply(f: fn(cb) -> x) : ?` where the result depends on what `f` does with `cb`. Requires a unification pass and a fixpoint over effect variables. This pushes graded across the line from "syntax-level subset checker" to "real effect-inference system" — probably better served by a separate tool (or adopting one like Granule) than by extending graded.

### Typed AST integration

If the Gleam compiler exposed typed AST metadata (expression types, resolved function references), graded could:

- Eliminate the remaining positional/label heuristics by reading actual parameter positions from types
- Resolve field calls without requiring explicit type annotations on parameters
- Track function references through value flow without ad-hoc AST pattern matching

Not feasible until the Gleam compiler exposes type info to third-party tools.

### Privacy and information flow checking

The next major feature is **lattice-based privacy tracking** — preventing sensitive data (PII, credentials) from flowing into logs, error messages, or third-party services.

Both checkers share the same theoretical foundation: graded modal type theory (see [THEORY.md](./THEORY.md)). Effects use sets with union; privacy uses lattices with join.

## License

Apache-2.0