<div align="center">
<img width="300" alt="sm-logo" src="https://github.com/user-attachments/assets/37bdd955-aacf-448e-b050-4d3305020c32" />
<p><strong>Workflow automation runtime for Elixir apps.</strong></p>
<p>
<a href="https://github.com/ccarvalho-eng/squid_mesh/actions/workflows/ci.yml">
<img alt="CI" src="https://github.com/ccarvalho-eng/squid_mesh/actions/workflows/ci.yml/badge.svg" />
</a>
<a href="https://hex.pm/packages/squid_mesh">
<img alt="Hex" src="https://img.shields.io/hexpm/v/squid_mesh" />
</a>
<a href="https://hexdocs.pm/squid_mesh">
<img alt="HexDocs" src="https://img.shields.io/badge/docs-hexdocs-purple" />
</a>
<a href="https://elixirforum.com/t/squid-mesh-workflow-automation-runtime-for-elixir-applications/75162">
<img alt="Elixir Forum" src="https://img.shields.io/badge/Forum-Discuss-4B275F?logo=elixir&logoColor=white" />
</a>
<a href="https://discord.com/channels/1323353012235796550/1504122798027571331">
<img alt="Discord" src="https://img.shields.io/badge/Discord-Join_Channel-5865F2?logo=discord&logoColor=white" />
</a>
<a href="https://github.com/ccarvalho-eng/squid_mesh/blob/main/LICENSE">
<img alt="License: Apache 2.0" src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" />
</a>
</p>
</div>
Squid Mesh is an embedded durable workflow runtime for Elixir applications. It
is for teams that want business workflows to live inside an existing Phoenix or
OTP app, share that app's repo and deployment model, and still have durable run
history, retries, approvals, replay, cancellation, and operator inspection.
It sits between a job backend and a standalone workflow service: more
structured and inspectable than a job queue, but still embedded in the host app
instead of running as a separate platform. It is not a generic replacement for
Runic, Reactor, Sage, or FlowStone; those projects solve adjacent problems at
different abstraction layers.
## What It Does
- workflow DSL with manual and cron triggers
- Postgres-backed run, step, attempt, and audit history
- pluggable executor boundary for step execution, delayed scheduling, redelivery, and cron activation
- retries, waits, failure routes, dependency joins, and HITL approval gates
- explicit step input selection and output mapping
- same-process host repo transactions for small local step groups
- runtime inspection through declared step state, audit events, and `SquidMesh.explain_run/2`
- native `SquidMesh.Step` modules, built-in steps like `:log`, `:wait`,
`:pause`, and `:approval`, plus raw `Jido.Action` interop
## When To Use It
Use Squid Mesh when a Phoenix or OTP app needs a durable workflow run as the
main abstraction, not just a background job. It fits flows where:
- state should stay inside the host app and survive restarts, deploys, retries,
and executor redelivery
- operators need to inspect why work is waiting, retrying, paused, failed,
cancelled, or complete
- approvals, manual review, replay, cancellation, and recovery policy belong to
the business process
- step history and manual decisions need to remain available after execution
For the full runtime direction and comparison with adjacent projects, see the
[Positioning guide](docs/positioning.md).
> [!WARNING]
> Squid Mesh is still in early development. The runtime is suitable for evaluation, local development, and integration work, but it is not yet documented as production-ready.
> See [Production Readiness](docs/production_readiness.md) for the current checklist and remaining bar.
## Runtime Shape
- Squid Mesh owns workflow structure, payload validation, runtime state, and retry policy
- the host app owns durable execution, queueing, delayed scheduling, and redelivery through a `SquidMesh.Executor` implementation
- your host app keeps its existing `Repo`, job system, and application boundaries
## Quick Start
Requirements:
- an existing Elixir application
- an existing Ecto `Repo`
- Postgres for persisted runtime state
- a host executor module that implements `SquidMesh.Executor`
### 1. Install from Hex.pm
```elixir
defp deps do
[
{:squid_mesh, "~> 0.1.0-alpha.7"}
]
end
```
For the common authoring path, define custom steps with `use SquidMesh.Step`.
Raw `Jido.Action` modules remain supported as an explicit interop path; if the
host app defines raw Jido actions directly, add `:jido` explicitly as well:
```elixir
defp deps do
[
{:jido, "~> 2.0"},
{:squid_mesh, "~> 0.1.0-alpha.7"}
]
end
```
### 2. Configure Squid Mesh And An Executor
```elixir
config :squid_mesh,
repo: MiddleEarth.Repo,
executor: MiddleEarth.SquidMeshExecutor
config :middle_earth, MiddleEarth.SquidMeshExecutor,
queue: :squid_mesh
```
The executor should enqueue `SquidMesh.Executor.Payload` values and deliver them
back to `SquidMesh.Runtime.Runner.perform/1`. See
[Host App Integration](docs/host_app_integration.md) for a minimal executor
shape.
### 3. Install migrations
```sh
mix deps.get
mix squid_mesh.install
mix ecto.migrate
```
`mix squid_mesh.install` creates one current-schema Squid Mesh migration in the
host app's `priv/repo/migrations`. The host app still owns migrations for its
chosen job system.
### 4. Import formatter rules
To keep workflow modules formatted as DSL-style calls, import Squid Mesh's
formatter configuration from the host app:
```elixir
# .formatter.exs
[
import_deps: [:squid_mesh],
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
]
```
## Example: The Ring Errand
Before the longer example, here is the workflow API in small pieces.
Manual triggers declare an entrypoint and a payload contract. Payload fields are
validated before Squid Mesh persists the run, and defaults are resolved at run
creation time:
```elixir
defmodule MiddleEarth.Workflows.RingErrand do
use SquidMesh.Workflow
workflow do
trigger :leave_shire do
manual()
payload do
field :bearer, :string, default: "Frodo"
field :ring_id, :string
field :snack_count, :integer, default: 11
field :panic_level, :float, required: false
field :eagle_backup?, :boolean, default: false
field :fellowship, :list, default: ["Sam"]
field :map_marks, :map, default: %{}
field :mood, :atom, default: :peckish
field :started_on, :string, default: {:today, :iso8601}
end
end
step :pack_lembas, Hobbiton.Steps.PackLembas,
input: [:snack_count],
output: :provisions,
transaction: :repo
step :announce_departure, :log,
message: "Leaving the Shire with suspicious jewelry",
level: :info
step :wait_for_gandalf, :wait, duration: 5_000
step :hide_at_prancing_pony, :pause
approval_step :council_vote, output: :council
step :cross_moria, Fellowship.Steps.CrossMoria,
input: [:bearer, :provisions, :council],
output: :moria,
retry: [
max_attempts: 3,
backoff: [type: :exponential, min: 1_000, max: 10_000]
]
step :reserve_eagle, Eagles.Steps.ReserveRide,
compensate: Eagles.Steps.CancelRide
step :insult_sauron, Gondor.Steps.InsultSauron,
compensatable: false
step :toss_ring, Mordor.Steps.TossRing,
irreversible: true
step :walk_home_awkwardly, Hobbiton.Steps.WalkHomeAwkwardly
transition :pack_lembas, on: :ok, to: :announce_departure
transition :announce_departure, on: :ok, to: :wait_for_gandalf
transition :wait_for_gandalf, on: :ok, to: :hide_at_prancing_pony
transition :hide_at_prancing_pony, on: :ok, to: :council_vote
transition :council_vote, on: :ok, to: :cross_moria
transition :council_vote, on: :error, to: :walk_home_awkwardly
transition :cross_moria, on: :ok, to: :reserve_eagle
transition :cross_moria, on: :error, to: :walk_home_awkwardly, recovery: :undo
transition :reserve_eagle, on: :ok, to: :insult_sauron
transition :insult_sauron, on: :ok, to: :toss_ring
transition :toss_ring, on: :ok, to: :complete
transition :walk_home_awkwardly, on: :ok, to: :complete
end
end
```
Cron triggers use the same workflow shape, but the host app owns recurring
scheduling and activation:
```elixir
defmodule Gondor.Workflows.BeaconWatch do
use SquidMesh.Workflow
workflow do
trigger :nightly_beacon_check do
cron "0 21 * * *", timezone: "Etc/UTC"
payload do
field :steward_mood, :string, default: "dramatic"
field :orc_count, :integer, default: 9001
end
end
step :inspect_hilltops, Gondor.Steps.InspectHilltops,
retry: [max_attempts: 5]
step :light_first_beacon, Gondor.Steps.LightBeacon,
compensate: Gondor.Steps.ExtinguishBeacon
step :log_call_for_aid, :log,
message: "Gondor calls for aid",
level: :info
transition :inspect_hilltops, on: :ok, to: :light_first_beacon
transition :light_first_beacon, on: :ok, to: :log_call_for_aid
transition :log_call_for_aid, on: :ok, to: :complete
end
end
```
Dependency-based workflows use `after: [...]` instead of transitions. A step is
runnable only after all of its declared dependencies complete:
```elixir
defmodule Mordor.Workflows.FinalDistraction do
use SquidMesh.Workflow
workflow do
trigger :start_distraction do
manual()
payload do
field :speech, :string, default: "For Frodo."
end
end
step :march_to_gate, Gondor.Steps.MarchToGate
step :look_very_brave, Gondor.Steps.LookBrave
step :sneak_up_volcano, Hobbiton.Steps.SneakUpVolcano
step :declare_victory, Gondor.Steps.DeclareVictory,
after: [:march_to_gate, :look_very_brave, :sneak_up_volcano],
irreversible: true
end
end
```
Step modules implement domain work. Squid Mesh records durable state, asks the
configured executor to schedule work, applies step retry policy, routes failures
after retry exhaustion, and exposes run inspection.
For approval or manual-review gates, use `approval_step/2` in transition-based
workflows and resume the paused run through `SquidMesh.approve_run/3` or
`SquidMesh.reject_run/3`. Approval steps persist their resolved `:ok` and
`:error` targets plus output-mapping metadata, so already-paused review runs keep
the same decision semantics across restarts and deploys. Generic
`SquidMesh.unblock_run/2` remains available for lower-level `:pause` steps when
you need manual intervention without an explicit approve/reject contract.
When a step needs a narrower contract than the whole payload plus accumulated
context, use `input: [...]` to select keys and `output: :key` to namespace the
returned map for downstream steps.
When a custom step needs several local repo writes to commit or roll back
together, declare `transaction: :repo`. This wraps only that action callback in
the configured Ecto repo transaction; workflow durability, successor dispatch,
external side effects, and saga compensation remain explicit Squid Mesh
boundaries.
For external side effects that cannot be honestly undone, mark the step with
`irreversible: true` or `compensatable: false`. Squid Mesh exposes that recovery
policy in inspection and blocks replay by default after such a step completes;
council members can still replay with `allow_irreversible: true` after
reviewing the side effect.
In the Ring Errand example, the `:error` transition on `:cross_moria` is a
same-step fallback after retries are exhausted. The compensation callback is
different: it is used only if `:reserve_eagle` completes, stores reversible
reservation output, and a later step causes the run to fail.
For other reversible saga steps, declare compensation callbacks the same way:
```elixir
step :borrow_elven_rope, Lothlorien.Steps.BorrowRope,
compensate: Lothlorien.Steps.ReturnRope
step :reserve_eagle, Eagles.Steps.ReserveRide,
compensate: Eagles.Steps.CancelRide
step :cross_moria, Fellowship.Steps.CrossMoria,
retry: [max_attempts: 2]
transition :borrow_elven_rope, on: :ok, to: :reserve_eagle
transition :reserve_eagle, on: :ok, to: :cross_moria
transition :cross_moria, on: :ok, to: :complete
```
When a downstream step fails after retries and the workflow has no forward
`:error` path, Squid Mesh runs completed compensation callbacks in reverse
completion order. In the example above, a failed `:cross_moria` step cancels the
eagle reservation before returning the rope, and each result is persisted under
the original step's `recovery.compensation` history.
Start the workflow through the public API and inspect the result with history:
```elixir
{:ok, run} =
SquidMesh.start_run(MiddleEarth.Workflows.RingErrand, :leave_shire, %{
ring_id: "one-ring"
})
SquidMesh.inspect_run(run.id, include_history: true)
```
With history enabled, the inspected run includes chronological `step_runs`,
declared `steps` state, and durable `audit_events` for pause, resume, approval,
and rejection actions.
For workflows paused at a generic `:pause` step, resume with `unblock_run/2`.
For approval steps, resume through the explicit decision APIs:
```elixir
{:ok, paused_run} = SquidMesh.inspect_run(run.id, include_history: true)
{:ok, resumed_run} =
SquidMesh.unblock_run(paused_run.id, %{
actor: "strider",
reason: "pipeweed restocked"
})
# Once the run pauses at an approval step, choose one path:
{:ok, approved_run} =
SquidMesh.approve_run(resumed_run.id, %{
actor: "elrond",
note: "approved by council"
})
# Or reject it instead:
{:ok, rejected_run} =
SquidMesh.reject_run(resumed_run.id, %{
actor: "elrond",
note: "too much singing"
})
```
Runs can also be listed, cancelled, or replayed. Replay requires an explicit
override after irreversible or non-compensatable steps:
```elixir
{:ok, running_runs} = SquidMesh.list_runs(status: :running)
{:ok, cancelling_run} = SquidMesh.cancel_run(run.id)
{:ok, replayed_run} = SquidMesh.replay_run(run.id)
{:ok, reviewed_replay} = SquidMesh.replay_run(run.id, allow_irreversible: true)
```
Use `SquidMesh.explain_run/2` when a host app needs council-facing diagnostics:
```elixir
{:ok, explanation} = SquidMesh.explain_run(run.id)
explanation.reason
#=> :waiting_for_retry
```
`inspect_run/2` returns the persisted runtime facts. `explain_run/2` summarizes
the current reason, valid next actions, and evidence in a structured shape that
dashboards and CLIs can render themselves.
## Documentation
Use the docs index for setup, workflow authoring, operations, and architecture:
- [Docs index](docs/index.md)
- [Host app integration](docs/host_app_integration.md)
- [Workflow authoring guide](docs/workflow_authoring.md)
- [Positioning guide](docs/positioning.md)
- [Example host app](https://github.com/ccarvalho-eng/squid_mesh/tree/main/examples/minimal_host_app)
## Contributing
- [Contributing guide](CONTRIBUTING.md)
- [Code of conduct](CODE_OF_CONDUCT.md)
