# hexagon_tpu

Elixir bindings for the Qualcomm Hexagon NPU (HTP) via **QNN**
(Qualcomm AI Engine Direct), targeting the Radxa Dragon Q6A (QCS6490,
HTP v68) running [Nerves](https://nerves-project.org/) with
[`nerves_system_dragon_q6a`](https://github.com/monoflow-ayvu/nerves_system_dragon_q6a).

Nx tensors in, Nx tensors out; quantization handled automatically from the
model's own metadata.

```elixir
{:ok, model} = HexagonTpu.load_model("/data/mobilenet_v2.bin")
{:ok, [scores]} = HexagonTpu.predict(model, [image_tensor])   # f32 in, f32 out
```

## How it works

- Models are **pre-compiled QNN context binaries** (`.bin`) produced on an
  x86-64 host with the QAIRT SDK tools (see [docs/models.md](docs/models.md)).
  On device we only deserialize (`QnnContext_createFromBinary`) and execute —
  no on-device graph prepare, no `libQnnHtpPrepare.so`.
- The NIF dlopen's `libQnnHtp.so`/`libQnnSystem.so` from the device rootfs
  (shipped by the system's `qairt-runtime` Buildroot package). Nothing
  proprietary is linked or bundled here; building only needs the SDK
  **headers** (`QNN_SDK_ROOT`).
- Tensor I/O metadata (names, shapes, dtypes, quant params) is introspected
  from the context binary via QnnSystem at load time.
- Layered API: `HexagonTpu.Runtime` (backend/device) → `HexagonTpu.Context`
  (loaded binary) → `HexagonTpu.Graph` (execute). Resources are freed by GC
  in dependency order; `close/1` exists for deterministic teardown.

## Requirements

On device (provided by `nerves_system_dragon_q6a` with
`BR2_PACKAGE_QAIRT_RUNTIME=y`):

- fastrpc kernel driver + CDSP firmware + `cdsprpcd`
- `/usr/lib/libQnnHtp.so`, `libQnnHtpV68Stub.so`, `libQnnSystem.so`
- `/usr/lib/dsp/libQnnHtpV68Skel.so` (+ `DSP_LIBRARY_PATH=/usr/lib/dsp`)

Build host: `QNN_SDK_ROOT` pointing at an extracted QAIRT SDK
(2.42.0.251225). The provided `devenv.nix` fetches it. Consumers of the
published Hex package get a precompiled NIF and don't need the SDK.

## Usage on host (development)

The QNN CPU backend (`libQnnCpu.so`, x86-64) allows exercising the runtime
lifecycle and context introspection without hardware:

```elixir
{:ok, rt} = HexagonTpu.Runtime.create(
  lib_path: Path.join(System.get_env("QNN_HOST_LIB_DIR"), "libQnnCpu.so"),
  system_lib_path: Path.join(System.get_env("QNN_HOST_LIB_DIR"), "libQnnSystem.so")
)
```

Run tests: `mix test` (pure), `mix test --include qnn_host` (needs devenv).

## Quantization

QNN stores affine quant params as `float = (q + offset) * scale`;
`HexagonTpu.TensorInfo` normalizes to the conventional
`zero_point = -offset`. `Graph.execute/3` quantizes f32 inputs and
dequantizes outputs automatically (`quantize: :none` / `dequantize: :none`
opt out for zero-overhead pre-quantized pipelines).

## Observability & guards

- `HexagonTpu.Stats.native/0` — NIF counters: live runtimes/contexts/graphs
  (leak guard: must return to baseline after GC), execute counts/errors, and
  pure QNN `graphExecute` time (`execute_ns_total`/`execute_ns_last`).
- `HexagonTpu.Stats.dirty_utilization/1` — dirty CPU / dirty IO scheduler
  busy share via microstate accounting (inference runs on dirty schedulers).
- `HexagonTpu.Stats.memory/0` — BEAM binary/total memory (output tensors are
  refc binaries).
- Telemetry: `[:hexagon_tpu, :execute, :stop]` per inference with
  `%{duration: native_time}` and `%{graph:, status:}` metadata.
- `{HexagonTpu.Monitor, interval_ms: 30_000}` in your supervision tree emits
  `[:hexagon_tpu, :stats]` periodically and logs warnings on sustained
  resource growth (leak suspicion) or dirty-scheduler saturation.

Leak-guard tests (`test/stats_test.exs`) assert alive counts return to
baseline across create/close and GC-only cycles.

## Releases

Pushing a `v*` tag runs `.github/workflows/release-precompiled.yml`: builds
the `dragon_q6a` NIF with the Nerves toolchain, uploads the precompiled
tarball + `checksum.exs` to the GitHub release, and publishes to Hex
(requires the `HEX_API_KEY` secret).

## License

Apache-2.0. The QAIRT SDK and its runtime libraries are Qualcomm
proprietary — this repository neither vendors nor redistributes them.