# Macula

[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![Erlang/OTP](https://img.shields.io/badge/Erlang%2FOTP-26+-brightgreen)](https://www.erlang.org)
[![Hex.pm](https://img.shields.io/hexpm/v/macula.svg)](https://hex.pm/packages/macula)
[![Buy Me A Coffee](https://img.shields.io/badge/Buy%20Me%20A%20Coffee-support-yellow.svg)](https://buymeacoffee.com/rlefever)

<p align="center">
  <img src="assets/logo.svg" width="120" height="120" alt="Macula">
</p>

<p align="center">
  <strong>BEAM-native HTTP/3 mesh networking for distributed applications</strong>
</p>

---

## What is Macula?

Macula is an **Erlang/OTP library** that provides a complete distributed mesh networking stack over HTTP/3 (QUIC). It enables BEAM applications to form relay-assisted mesh networks with:

- **Federated relay mesh** -- outbound-only connections, no open ports
- **Zero configuration clustering** via UDP multicast gossip
- **NAT-friendly transport** using QUIC (single UDP port)
- **Erlang distribution over relay** -- `net_adm:ping` across firewalls

<p align="center">
  <img src="assets/mesh-architecture.svg" alt="Macula Mesh Architecture" width="100%">
</p>

---

## Features

### Mesh Networking

<p align="center">
  <img src="assets/macula_overview.svg" alt="Macula Overview" width="100%">
</p>

| Feature | Description |
|---------|-------------|
| **HTTP/3 over QUIC** | NAT-friendly, firewall-friendly, built-in TLS 1.3 |
| **Kademlia DHT** | Decentralized routing with O(log N) lookups |
| **Multi-tenant realms** | Isolated namespaces for different applications |
| **Connection pooling** | 94.5% hit rate, LRU eviction |

### Pub/Sub Messaging

<p align="center">
  <img src="assets/pubsub_flow.svg" alt="PubSub Flow" width="100%">
</p>

```erlang
%% Subscribe to a topic
{ok, SubRef} = macula:subscribe(Peer, <<"sensors.temperature">>, fun(Msg) ->
    io:format("Received: ~p~n", [Msg])
end).

%% Publish to subscribers
ok = macula:publish(Peer, <<"sensors.temperature">>, #{value => 23.5}).

%% Unsubscribe when done
ok = macula:unsubscribe(Peer, SubRef).
```

### RPC (Request/Response)

<p align="center">
  <img src="assets/rpc_flow.svg" alt="RPC Flow" width="100%">
</p>

```erlang
%% Advertise a procedure handler
{ok, _Ref} = macula:advertise(Peer, <<"math.add">>, fun(#{a := A, b := B}) ->
    {ok, #{result => A + B}}
end).

%% Call the procedure (discovers provider via DHT)
{ok, #{result := 5}} = macula:call(Peer, <<"math.add">>, #{a => 2, b => 3}).
```

### Gossip Clustering

<p align="center">
  <img src="assets/gossip_clustering.svg" alt="Gossip Clustering" width="100%">
</p>

Zero-configuration cluster formation using UDP multicast:

```erlang
%% Start gossip-based clustering
ok = macula_cluster:start_cluster(#{
    strategy => gossip,
    secret => <<"my_cluster_secret">>  %% Optional HMAC authentication
}).

%% Nodes auto-discover via multicast 230.1.1.251:45892
```

### Erlang Distribution Over Relay Mesh

<p align="center">
  <img src="assets/dist_relay_tunnel.svg" alt="Distribution Relay Tunnel" width="100%">
</p>

Full OTP distribution tunneled through the relay mesh — nodes only need
outbound connectivity. No VPNs, no open ports.

```erlang
%% Enable relay distribution
os:putenv("MACULA_DIST_MODE", "relay"),
macula_dist_relay:register_mesh_client(Client),
macula_dist_relay:advertise_dist_accept(),

%% Now standard OTP distribution works across firewalls
net_adm:ping('other@remote-host').  %% => pong
gen_server:call({Name, 'other@remote-host'}, Request).  %% works
```

| Feature | Status |
|---------|--------|
| **Handshake over relay** | 5-message dist_util handshake through pub/sub |
| **AES-256-GCM encryption** | Tunnel bytes encrypted, relay cannot read ETF |
| **Supervised bridges** | gen_server per tunnel under simple_one_for_one |
| **Cross-relay tunnels** | Nodes on different relays connect via peering |
| **Relay reconnection** | Bridge re-acquires client after relay restart |
| **Metrics** | Per-tunnel byte/message counters |

See [Distribution Over Mesh Guide](docs/guides/DIST_OVER_MESH_GUIDE.md) for details.

### Content Transfer (Mesh Artifacts)

<p align="center">
  <img src="assets/content_transfer_flow.svg" alt="Content Transfer" width="100%">
</p>

Content-addressed storage and transfer for distributing OTP releases across the mesh:

```erlang
%% Publish a file to the mesh
{ok, MCID} = macula_content:publish("/path/to/release.tar.gz").

%% Fetch from any provider
{ok, Binary} = macula_content:fetch(MCID).
```

| Feature | Description |
|---------|-------------|
| **MCID** | Content-addressed identifiers (BLAKE3/SHA256) |
| **Merkle verification** | Chunk-level integrity |
| **Parallel download** | From multiple providers |
| **Want/Have/Block** | Efficient mesh protocol |

### Authorization (DID + UCAN)

<p align="center">
  <img src="assets/authorization_flow.svg" alt="Authorization Flow" width="100%">
</p>

Decentralized capability-based authorization:

| Component | Purpose |
|-----------|---------|
| **DID** | Decentralized identifiers for namespace ownership |
| **UCAN** | Capability tokens for delegated permissions |
| **Namespace** | `did:macula:io.example.user` owns `io.example.user.*` |

### Hierarchical DHT (Bridge System)

<p align="center">
  <img src="assets/kademlia_dht.svg" alt="Kademlia DHT" width="100%">
</p>

Fractal mesh hierarchy with query escalation:

```
Cluster → Street → Neighborhood → City → Province → Country → Region → Global
```

When a DHT query fails locally, it escalates to parent levels with results cached at lower levels.

---

## Quick Start

### Installation

Add to your `rebar.config`:

```erlang
{deps, [
    {macula, "0.42.7"}
]}.
```

Or in Elixir `mix.exs`:

```elixir
defp deps do
  [
    {:macula, "~> 0.42.7"}
  ]
end
```

### Basic Usage

```erlang
%% Start macula application
application:ensure_all_started(macula).

%% Connect to the mesh
{ok, Peer} = macula:connect(<<"quic://seed.example.com:9443">>, #{
    realm => <<"io.example.myapp">>,
    node_id => <<"node-001">>
}).

%% Subscribe to events
{ok, _SubRef} = macula:subscribe(Peer, <<"events.orders">>, fun(Order) ->
    process_order(Order)
end).

%% Advertise an RPC procedure
{ok, _AdvRef} = macula:advertise(Peer, <<"inventory.check">>, fun(#{sku := Sku}) ->
    {ok, check_inventory(Sku)}
end).

%% Call an RPC procedure (discovers provider via DHT)
{ok, Result} = macula:call(Peer, <<"inventory.check">>, #{sku => <<"ABC123">>}).
```

---

## Configuration

### Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `MACULA_QUIC_PORT` | `9443` | QUIC listener port |
| `MACULA_REALM` | `com.example.realm` | Default realm |
| `HEALTH_PORT` | `8080` | Health check HTTP port |
| `MACULA_TLS_MODE` | `development` | TLS mode (`production`/`development`) |
| `CLUSTER_STRATEGY` | `gossip` | Cluster strategy (`gossip`/`static`/`mdns`) |
| `CLUSTER_SECRET` | - | Shared secret for gossip HMAC |

### Application Config

```erlang
{macula, [
    {quic_port, 9443},
    {realm, <<"io.example.myapp">>},
    {tls_mode, production},
    {tls_cacertfile, "/etc/ssl/certs/ca-certificates.crt"}
]}.
```

---

## Documentation

| Guide | Description |
|-------|-------------|
| [Distribution Over Mesh](docs/guides/DIST_OVER_MESH_GUIDE.md) | Relay-tunneled Erlang distribution |
| [Cluster API Guide](docs/guides/CLUSTERING_GUIDE.md) | Clustering and distribution |
| [Content Transfer Guide](docs/guides/CONTENT_TRANSFER_GUIDE.md) | Mesh artifact distribution |
| [NAT Traversal Guide](docs/guides/DIST_OVER_MESH_GUIDE.md) | NAT techniques |
| [DHT Guide](docs/guides/DHT_GUIDE.md) | Kademlia DHT internals |
| [Authorization Guide](docs/guides/AUTHORIZATION_GUIDE.md) | DID/UCAN security |
| [TLS Configuration](docs/operator/TLS_GUIDE.md) | Production TLS setup |

---

## Version History

| Version | Date | Highlights |
|---------|------|------------|
| **v0.42.7** | Apr 2026 | Distribution over relay mesh — cross-relay, encryption, supervision, metrics (48 tests) |
| **v0.40.0** | Apr 2026 | First Erlang dist-over-mesh: net_adm:ping across 3 countries via relay tunnel |
| **v0.19.2** | Jan 2026 | README rework with feature sections and SVGs |
| **v0.19.1** | Jan 2026 | Gossip clustering, static strategy (34 tests) |
| **v0.19.0** | Jan 2026 | Content transfer system, MCID, Want/Have/Block (171 tests) |
| **v0.18.0** | Jan 2026 | Cluster API for bc_gitops integration (19 tests) |
| **v0.16.0** | Dec 2025 | Registry system, Ed25519 signing (60 tests) |
| **v0.15.0** | Dec 2025 | Gossip protocol, CRDT replication (29 tests) |
| **v0.14.0** | Dec 2025 | Masterless CRDT architecture (48 tests) |
| **v0.13.0** | Dec 2025 | Hierarchical DHT, bridge system (40 tests) |
| **v0.12.0** | Nov 2025 | Complete NAT traversal (70 tests) |

See [CHANGELOG.md](CHANGELOG.md) for full history.

---

## Architecture

Macula follows an **always-on architecture** where every node has all capabilities:

```
macula_root (application supervisor)
├── macula_gateway_system (QUIC transport + relay handler)
├── macula_pubsub_system (pub/sub messaging)
├── macula_rpc_system (RPC request/response)
├── macula_routing_system (Kademlia DHT)
├── macula_bootstrap_system (mesh discovery)
├── macula_membership_system (gossip clustering)
├── macula_bridge_system (hierarchical mesh escalation)
├── macula_platform_system (CRDT coordination)
├── macula_registry_system (package distribution)
├── macula_content_system (mesh content transfer)
├── macula_cert_system (TLS certificate management)
└── macula_dist_system (Erlang distribution)
    ├── macula_dist_bridge_sup (relay tunnel bridges)
    ├── macula_dist_discovery (DHT node discovery)
    └── macula_cluster_strategy (optional auto-clustering)
```

---

## Related Projects

| Project | Description |
|---------|-------------|
| [macula-ecosystem](https://github.com/macula-io/macula-ecosystem) | Documentation hub |
| [macula-console](https://github.com/macula-io/macula-console) | Management console |
| [bc-gitops](https://github.com/beam-campus/bc-gitops) | GitOps reconciler |
| [macula-tweann](https://github.com/macula-io/macula-tweann) | Neuroevolution framework |

---

## Community

- **Hex.pm**: [hex.pm/packages/macula](https://hex.pm/packages/macula)
- **GitHub**: [github.com/macula-io/macula](https://github.com/macula-io/macula)
- **Issues**: [github.com/macula-io/macula/issues](https://github.com/macula-io/macula/issues)

---

## License

Apache 2.0 - See [LICENSE](LICENSE) for details.

---

<p align="center">
  <sub>Built with the BEAM</sub>
</p>