# Ecto.Adapters.Recall
An [Ecto 3](https://hexdocs.pm/ecto) adapter for [Mnesia](https://www.erlang.org/doc/apps/mnesia/) — the distributed, transactional database that ships with the BEAM.
*Memory, recollected.*
Unlike an ETS-backed adapter, Mnesia gives you **real ACID transactions** and optional **disk persistence** (`disc_copies`) for free, mapping almost directly onto Ecto's `Transaction` and `Storage` behaviours. Queries are translated to Erlang **match specifications** and run through `:mnesia.select`, so `where`/`select` are pushed down to Mnesia rather than evaluated row-by-row in Elixir.
## Related projects
Two existing adapters cover nearby ground; Recall differs from both:
- **[etso](https://github.com/evadne/etso)** — an **ETS**-backed Ecto 3 adapter. ETS is fast and in-memory, but it has no transactions, no disk persistence, and no joins, aggregates, locking, or migrations. etso is excellent for caching rarely-changing data (and ships with assocs/preloads). Recall targets the same in-process, BEAM-native niche but is **Mnesia**-backed, so it adds real ACID transactions, optional disc persistence, joins, and aggregates.
- **[ecto_mnesia](https://github.com/Nebo15/ecto_mnesia)** — the other **Mnesia** adapter, but for **Ecto 2.x** and no longer actively maintained. It emulates `select`/`order_by` in Elixir (`O(n log n)`), ignores field types entirely (Mnesia stores any term), generates `:id` keys via a sequence table, and supports migrations and secondary indexes — but has no joins, no aggregates, and no type casting. Recall is for **Ecto 3**, pushes `where`/`select` down into match specs instead of emulating them, casts types through Ecto's loaders/dumpers, implements joins and aggregates, and supports [declarative secondary indexes](#secondary-indexes) — at the cost of (currently) no migrations.
## Installation
Add `recall` to your dependencies:
```elixir
def deps do
[
{:recall, "~> 0.1.0"}
]
end
```
## Usage
Define a repo as usual, pointing it at the adapter:
```elixir
defmodule MyApp.Repo do
use Ecto.Repo, otp_app: :my_app, adapter: Ecto.Adapters.Recall
end
```
Define schemas with `Recall.Schema` (a drop-in replacement for `use Ecto.Schema`), naming the table with an **atom** — the form a Mnesia table actually takes:
```elixir
defmodule MyApp.User do
use Recall.Schema
schema :users do
field :name, :string
field :age, :integer
end
end
```
Everything `Ecto.Schema` provides (the struct, `__schema__/1`, changesets, associations) keeps working untouched — this only *adds* the compiled accessors the adapter uses (see [Why a dedicated schema macro](#why-a-dedicated-schema-macro)). The schema is purely logical: a table's type and any secondary indexes are set in [migrations](#migrations), not on the schema.
```elixir
import Ecto.Query
{:ok, user} = MyApp.Repo.insert(%MyApp.User{name: "Ada", age: 36})
MyApp.Repo.all(from u in MyApp.User, where: u.age > 30, select: u.name)
#=> ["Ada"]
MyApp.Repo.transaction(fn ->
MyApp.Repo.insert!(%MyApp.User{name: "Grace"})
# ...real ACID transaction; raise or Repo.rollback/1 to abort
end)
```
Tables are created lazily on first use with the configured storage type, so you can get going without migrations. Real apps (especially disc-backed ones) manage their schema with [migrations](#migrations).
## Configuration
Set on the repo config (e.g. in `config/config.exs`):
```elixir
config :my_app, MyApp.Repo,
storage: :ram_copies, # or :disc_copies, :disc_only_copies
type: :ordered_set, # or :set
nodes: [node()], # nodes that hold copies
dir: "priv/mnesia" # Mnesia data directory (disc storage only)
```
| Option | Default | Description |
|------------|----------------|-------------|
| `:storage` | `:ram_copies` | Where auto-created tables live: `:ram_copies`, `:disc_copies`, or `:disc_only_copies`. |
| `:type` | `:ordered_set` | `:ordered_set` keeps records in primary-key order, so reads come back sorted. Falls back to `:set` for `:disc_only_copies` (Mnesia disallows `:ordered_set` there). |
| `:nodes` | `[node()]` | Nodes that hold table copies. |
| `:dir` | — | Mnesia data directory (only relevant for disc storage). |
For disc-backed storage, `mix ecto.create` / `mix ecto.drop` create and delete the Mnesia schema on the configured nodes.
## Migrations
Migrations are written like any Ecto app — `create` / `alter` / `drop` in a migration module — with one difference: run them with the Recall tasks rather than `mix ecto.migrate`.
```sh
mix recall.migrate # run pending migrations
mix recall.rollback --step 1 # roll back
```
```elixir
defmodule MyApp.Repo.Migrations.CreateUsers do
use Ecto.Migration
def change do
create table(:users) do
add :name, :string
add :age, :integer
end
create index(:users, [:age]) # a secondary index the planners can use
end
end
```
Supported: `create` / `create_if_not_exists` / `drop` / `drop_if_exists` for tables and indexes; `alter table` with `add` / `remove` / `modify`; and column and table renames. Mnesia is typeless, so column **types are ignored** — only the attribute names, which one is the primary key (Mnesia's key is the first attribute), and which carry an index matter — and `modify` is a no-op. An `alter` rewrites every existing record in place via `:mnesia.transform_table`, defaulting any added column.
Raised loudly rather than silently mishandled: raw SQL (`execute "..."`), composite primary keys, changing or removing the primary key, unique indexes (Mnesia has no unique secondary constraint — use the primary key for uniqueness), and Postgres-style prefixes. A composite `index(:t, [:a, :b])` becomes one single-attribute index per column (with a warning), since Mnesia indexes are single-attribute.
**Why a dedicated task?** Stock `mix ecto.migrate` tracks applied versions by querying `schema_migrations` with a *string* source (`from m in "schema_migrations"`), and this adapter only serves schema-backed tables (see [Limitations](#limitations)). `mix recall.migrate` runs the very same migrations through Ecto's migration engine, but does version bookkeeping through a real schema (`Recall.SchemaMigration`) so it stays within that rule. Both tasks accept `--step`, `--to`, `--all`, and `-r MyApp.Repo`.
## Why a dedicated schema macro
`Recall.Schema` is **required** — a plain `use Ecto.Schema` schema will not work with this adapter.
`Ecto.Schema` is built for SQL, where a query carries field *names* and the database resolves them. Recall stores every field as a positional Mnesia attribute, so the macro bakes the schema's **logical shape** — the table tag, field names in storage order, the struct, and a load recipe — into compiled `__recall__/1` accessors, and the adapter reads field metadata only through them (a plain `use Ecto.Schema` has no fallback path). A field's **physical position** is resolved at runtime from the table's live layout in Mnesia's catalog (cached in `:persistent_term`), so a migration that reshapes a table is honored on the next read, and reordering fields in source can't silently misalign storage.
It also requires the table name to be an **atom** (`schema :users`, not `schema "users"`) — the form a Mnesia table actually takes. `__schema__(:source)` still returns the equivalent string `"users"` for the rest of Ecto, so changesets, associations, and the struct are unaffected.
### Secondary indexes
A table's built-in index is its primary key. Additional secondary indexes are **physical** — created by a [migration](#migrations) (`create index(:users, [:age])`), never declared on the schema or built on the fly. The read and join planners serve an equality (or `in`) on the primary key, or on any attribute that is physically indexed, with `:mnesia.read` / `:mnesia.index_read` instead of a full-table scan; an equality on any other field still runs correctly — it just takes the scan path. The full `where` is re-checked against the fetched rows either way, so results are identical to a scan.
## What works
- Insert / get / update / delete, `insert_all` (with placeholders and `:returning`)
- Upserts / `on_conflict` (`:raise`, `:nothing`, `:replace_all`, `:replace_all_except`, `{:replace, fields}`)
- `where` (pushed into match specs), `order_by`, `limit` / `offset`, `in`, `is_nil`
- **Key/index reads on `where`** — an equality or `in` that pins a field to a known value (`Repo.get/3`, `u.id == ^id`, `u.email == ^e`, `u.id in ^ids`) skips the full-table scan: a primary-key match becomes an O(1) `:mnesia.read`, and a match on an [indexed](#secondary-indexes) field uses `:mnesia.index_read`. The full `where` is still re-checked against the fetched rows, so results are identical to a scan — just fewer rows touched. Applies to `all`, `update_all`, and `delete_all` (`stream` stays on the cursor-based scan path). (Disable with `config :recall, force_where_scan: true`.)
- Joins — inner / left / right / cross, across N tables, with a tiered right-side fetch (primary-key `read`, `index_read` on an [indexed](#secondary-indexes) field, or full scan) and `where` pushdown
- Joined `update_all` / `delete_all`
- Single aggregates: `count`, `sum`, `avg`, `min`, `max` (incl. `count(field, :distinct)`)
- Computed `select` expressions: date/time arithmetic (`date_add` / `datetime_add` / `from_now` / `ago`), `type/2` casts, and the operators `+ - * == != < > <= >= and or not is_nil` (evaluated in Elixir during projection)
- Real ACID transactions and `Repo.rollback/1`
- Optimistic locking, idempotent delete, `field ..., source:` mapping
- `Repo.stream/2` (lazily within a transaction; eagerly otherwise)
- Guard `fragment/1`s as an escape hatch to native Erlang guards (see below)
## Match-spec guard fragments
A match spec runs Erlang **guards** against each record, and guards reach things SQL has no notion of: type tests, structural access into a term, term ordering across heterogeneous types. To expose those, a `fragment/1` whose body is a guard expression is translated into the match-spec conditions and pushed down with the rest of the `where`:
```elixir
import Ecto.Query
# type test — filter on the *shape* of a term, not its value
from u in User, where: fragment("is_map(?)", u.metadata)
# structural access into a map-typed column
from u in User, where: fragment("map_get(?, ?)", ^"role", u.metadata) == ^"admin"
# arithmetic / bit guards
from u in User, where: fragment("rem(?, ?)", u.id, ^2) == 0
# anything expressible as an Erlang guard
from u in User, where: fragment("length(?) > ?", u.tags, ^min_tags)
```
This is **not** the SQL-fragment escape hatch. A SQL adapter passes the fragment string to the database verbatim — there, the string *is* the query. Mnesia has no query string: `:mnesia.select/2` takes an Erlang term (`{head, conditions, body}`), so the fragment is parsed and translated into guard tuples (e.g. `is_map(?)` → `{:is_map, :"$5"}`), once per query build. The `?` placeholders bind to fields or interpolated `^values` exactly as in any fragment.
Only the Erlang **guard BIFs** (`is_*`, `element`, `map_get`, `map_size`, `byte_size`, `length`, `rem`, `abs`, the bit operators, `node`, `self`, …) and comparison/boolean operators are allowed. Anything else — an unknown name, or a real BIF that isn't a valid match-spec guard — **raises** at query time rather than building a spec Mnesia would reject or smuggling an arbitrary call into the match. Fragments are supported in `where`, not in `select` (the match-spec body has a narrower operation set, and `select` is projected in Elixir regardless).
## Demo app
A small, runnable example lives in [`examples/blog/`](examples/blog/) — a blog
(authors → posts → comments) backed entirely by this adapter, no external
database. From the repo root:
```sh
cd examples/blog
mix deps.get
mix blog.demo
```
`mix blog.demo` migrates, seeds sample data, and prints a guided tour of an
indexed-field `index_read`, `where`/`order_by`/`limit` pushdown, a posts ⨝
authors join, `count`/`sum`/`avg` aggregates, a committed transaction, a
`Repo.rollback/1`, and — to make the storage model concrete — the raw Mnesia
tuples next to the same rows loaded back as Ecto structs. See
[`examples/blog/README.md`](examples/blog/README.md) for the breakdown.
## Limitations
Unsupported query clauses **raise** at query-prepare time rather than silently dropping (so you never get wrong rows back):
- `group_by` / `having` / `distinct` / window functions
- set operations (`union` / `except` / `intersect`) and CTEs (`with_cte`)
Other gaps, fundamental to Mnesia or not yet implemented:
- **Non-primary-key unique constraints are not enforced** — only primary-key uniqueness is checked. (Mnesia has no unique secondary index, so `unique_index/2` in a migration raises rather than silently allowing duplicates.)
- **No composite primary keys** — a Mnesia key is a single attribute.
- **Single-node ID generation.** `autogenerate(:id)` uses `:erlang.unique_integer/1`, which is unique per node; use `binary_id` for multi-node clusters.
- **Schemaless queries are not supported** — `from("posts", ...)` with a string source (no schema) has no compiled field layout to resolve against; use a schema module.
- Not supported in `select`: aggregate expressions wrapped in `type/2` (e.g. `type(sum(x), :integer)`), `fragment`s, `selected_as`, and `VALUES` lists. (The arithmetic/comparison/date-time expressions listed under [What works](#what-works) *are* supported.)
- Not supported: subqueries (in `where` / `select` / as a source), `insert_all` from a query source, database-level constraints / foreign keys, Postgres prefixes. Fragments are supported only as guard expressions in `where` (see [Match-spec guard fragments](#match-spec-guard-fragments)) — not as raw SQL and not in `select`.
## License
MIT.