Skip to main content

doc/real-project-integration.md

# Real Project Integration Guide

This guide shows how to embed the Binance-first Elixir CCXT Pro target in an
OTP application. The examples under `examples/` are runnable scripts; this
document is the application-facing shape.

## Dependency

After publishing, use the preview package version:

```elixir
defp deps do
  [
    {:ccxt, "== 0.1.0-binance-pro-preview"}
  ]
end
```

Before publishing, or when testing a source checkout directly, use a local path
dependency instead:

```elixir
defp deps do
  [
    {:ccxt, path: "../ccxt/elixir"}
  ]
end
```

## Supervision Tree

Keep websocket stream enumeration in a task, not inside a GenServer callback.
The worker owns state; the task blocks on `stream_*`:

```elixir
children = [
  Ccxt.Pro.Supervisor,
  {Task.Supervisor, name: MyApp.BinanceStreams.TaskSupervisor},
  {MyApp.BinanceTickerWorker,
   name: MyApp.BinanceTickerWorker,
   symbol: "BTC/USDT",
   binance_env: "prod",
   task_supervisor: MyApp.BinanceStreams.TaskSupervisor}
]

Supervisor.start_link(children, strategy: :one_for_one)
```

The included `examples/pro_ticker_worker.exs` and
`examples/pro_order_event_worker.exs` show the complete public and private
worker patterns.

## Instance Configuration

For code that wants an exchange-like instance shape, use `Ccxt.Pro.binance/1`:

```elixir
exchange =
  Ccxt.Pro.binance(%{
    apiKey: System.fetch_env!("BINANCE_PROD_API_KEY"),
    secret: System.fetch_env!("BINANCE_PROD_API_SECRET"),
    options: %{defaultType: "spot"}
  })
```

Generated high-level functions also accept keyword options:

```elixir
Ccxt.Pro.Binance.watch_ticker("BTC/USDT",
  binance_env: "prod",
  timeout: 30_000
)
```

Use one environment per process invocation. Keep production, demo, and testnet
keys distinct; do not rely on generic fallback keys to pick the active account.

## Credentials

Recommended runtime environment names:

```sh
BINANCE_PROD_API_KEY=...
BINANCE_PROD_API_SECRET=...
BINANCE_PRO_ENV=prod
```

For scripts and tests in this repository, `.env` can hold multiple key
families:

```sh
BINANCE_DEMO_API_KEY=...
BINANCE_DEMO_API_SECRET=...
BINANCE_TESTNET_API_KEY=...
BINANCE_TESTNET_API_SECRET=...
BINANCE_PROD_API_KEY=...
BINANCE_PROD_API_SECRET=...
```

In a production app, load secrets through the application's normal secret
manager and pass them into config or process environment before starting
workers. Avoid logging credentials and raw signed URLs.

## Telemetry

The runtime emits telemetry events under `[:ccxt, :pro, ...]`. Attach a handler
once during application startup:

```elixir
:telemetry.attach_many(
  "my-app-ccxt-pro-logger",
  [
    [:ccxt, :pro, :connection, :started],
    [:ccxt, :pro, :connection, :closed],
    [:ccxt, :pro, :message, :received],
    [:ccxt, :pro, :message_hash, :resolved],
    [:ccxt, :pro, :request, :resolved],
    [:ccxt, :pro, :request, :rejected]
  ],
  fn event, measurements, metadata, _config ->
    Logger.debug("ccxt_pro event=#{inspect(event)} measurements=#{inspect(measurements)} metadata=#{inspect(metadata)}")
  end,
  nil
)
```

Useful production metrics:

- open connection count from `Ccxt.Pro.connections/0`
- first update latency
- maximum gap between updates
- reconnect count
- request rejection count
- waiter count from `Ccxt.Pro.connection_info/1`
- cache sizes for high-volume streams

## Worker Retry And Shutdown

For public streams:

- Start the stream in `handle_continue/2`.
- Consume the enumerable in a supervised task.
- Send parsed updates back to the GenServer.
- On stop, terminate the task and call `Ccxt.Pro.close_connection/1`.

For private streams:

- Authenticate once with `Ccxt.Pro.Binance.authenticate/1`.
- Pass returned `ws_auth` into `stream_orders/4`, `stream_balance/1`, or
  related private stream helpers.
- Handle both task result messages (`{ref, result}`) and `:DOWN` monitor
  messages.
- Dismiss the matching monitor with `Process.demonitor(ref, [:flush])` when the
  task result arrives first.
- Close the private websocket connection before retrying.

Backoff should be owned by the worker. Start with a small retry such as 5
seconds and increase it for repeated auth or permission failures.

## Database Schema Path

Do not design database tables directly from raw Binance payloads only. Store
both raw payloads and CCXT unified structures:

1. Append raw websocket events and websocket API responses.
2. Normalize to unified structures with `Ccxt.StructurePersistence`.
3. Upsert current-state tables for mutable structures.
4. Append history/event rows for mutable streams.

The schema manifest is:

```text
priv/ccxt_structures/binance_pro_structures.json
```

Runtime access:

```elixir
Ccxt.StructureSchema.manifest()
Ccxt.StructureSchema.structure!("order")
Ccxt.StructureSchema.fields!("ticker")
```

Generated Ecto templates:

```text
priv/ccxt_structures/generated/binance_pro_ecto_schemas.exs
priv/ccxt_structures/generated/binance_pro_migration.exs
```

Treat generated templates as a starting point for the application repo. Review
table names, indexes, retention, decimal precision, and partitioning before
running migrations in production.

## Operational Checks

Before promoting an application release:

- `npm run releaseElixirPreviewCheck` passes on the dependency checkout.
- Public live smoke passes for the production region where the app runs.
- Private read-only live smoke passes with the exact key permissions used by
  the app.
- If order placement is enabled, a gated non-marketable production order smoke
  passes and confirms cleanup.
- Long soak metrics are within the app's websocket freshness requirements.
- Telemetry events are visible in the app's logs or metrics backend.
- Workers stop cleanly and leave no open `Ccxt.Pro.connections/0` after
  shutdown.