Skip to main content

doc/ccxt-pro-cache-parity.md

# Binance Pro Cache Parity

This document tracks CCXT Pro cache-class semantics used by Binance Pro source
methods and the Elixir runtime components that own the equivalent behavior.

The target rule is: cache behavior belongs in reusable runtime modules, while
generated watch/handler methods only declare which cache semantic they need.

## Source Classes

CCXT Pro defines its reusable websocket caches in `ts/src/base/ws/Cache.ts`.

Source class | Core behavior | Elixir owner | Status
--- | --- | --- | ---
`ArrayCache` | Bounded append cache with newest update accounting by symbol | `Ccxt.Pro.ArrayCache`, `Ccxt.Pro.CacheUpdates` | Implemented as bounded newest-first list cache for public/private list streams; update counters are runtime-owned
`ArrayCacheByTimestamp` | Bounded cache indexed by timestamp with replacement and new-update accounting | `Ccxt.Pro.TimestampArrayCache`, `Ccxt.Pro.CacheUpdates` | Implemented for OHLCV-style timestamp caches; timestamp update counters are runtime-owned
`ArrayCacheBySymbolById` | Bounded cache indexed by `symbol -> id`; updates replace the existing item and move it to newest | `Ccxt.Pro.IndexedArrayCache`, `Ccxt.Pro.CacheUpdates` | Implemented for generic symbol/key caches and used by `Ccxt.Pro.OrderCache`; nested update counters are runtime-owned
`ArrayCacheBySymbolBySide` | Cache indexed by `symbol -> side`; updates replace the existing side and move it to newest | `Ccxt.Pro.IndexedArrayCache`, `Ccxt.Pro.CacheUpdates` | Runtime primitive implemented and wired into Binance positions; nested update counters are runtime-owned

## Elixir Runtime Classes

Runtime module | Owned semantics | Current use
--- | --- | ---
`Ccxt.Pro.ArrayCache` | Bounded newest-first retention | Public trades/OHLCV/ticker-family payloads, private trade/liquidation list payloads, order book delta buffers
`Ccxt.Pro.IndexedArrayCache` | Bounded newest-first symbol/key index, replacement, move-to-newest, hashmap view | Generic owner for `ArrayCacheBySymbolById` and `ArrayCacheBySymbolBySide` style behavior
`Ccxt.Pro.TimestampArrayCache` | Bounded newest-first timestamp index, replacement, move-to-newest, hashmap view | OHLCV public websocket payloads keyed by kline open timestamp
`Ccxt.Pro.CacheUpdates` | `getLimit`-style new update counters for all updates, per-symbol updates, symbol/key sets, and timestamp sets | Public cached stream and private list/order stream `newUpdates` watch mode wiring
`Ccxt.Pro.OrderCache` | Binance private order cache with symbol/id index, order/trade merge, accumulated fee metadata | `executionReport` and `ORDER_TRADE_UPDATE` private order events
`Ccxt.Pro.PositionCache` | Symbol+side-indexed position cache with event timestamp propagation, hashmap view, and flattened values view | Futures and delivery `ACCOUNT_UPDATE` position events
`Ccxt.Pro.Connection` order book state | Snapshot/delta merge, pre-snapshot buffer, sequence checks | Public order book streams

## Current Hardening

`Ccxt.Pro.IndexedArrayCache` closes the first specialized cache gap:

- Uses `symbol -> key` indexing instead of id-only replacement.
- Keeps newest-first list shape for existing Elixir watcher compatibility.
- Replaces existing entries and moves them to the newest position.
- Evicts the oldest entry when a positive limit is reached.
- Exposes a `hashmap/2` view compatible with CCXT Pro's `cache.hashmap`
  lookup shape.
- Supports both `put_by_id/3` and `put_by_side/3` runtime calls.

`Ccxt.Pro.OrderCache` now uses `Ccxt.Pro.IndexedArrayCache`, which prevents the
global private `orders` cache from merging two different symbols that happen to
share the same exchange order id.

`Ccxt.Pro.TimestampArrayCache` closes the `ArrayCacheByTimestamp` gap for
OHLCV-style payloads:

- Uses a timestamp key, including nested raw websocket paths such as `k.t`.
- Replaces an existing timestamp entry and moves it to the newest position.
- Evicts the oldest entry when a positive limit is reached.
- Exposes a `hashmap/2` view keyed by timestamp string.
- Keeps parsing outside the websocket connection; Binance OHLCV watchers still
  parse cached raw kline payloads into unified OHLCV arrays.

`Ccxt.Pro.PositionCache` now wires the `ArrayCacheBySymbolBySide` primitive into
Binance position streams:

- Stores position entries by `symbol -> side`, preserving hedge-mode `long` and
  `short` updates for the same symbol.
- Keeps the previous symbol-root payload shape for existing `watch_positions/4`
  and private cache consumers.
- Exposes `hashmap/1` and `values/1` so high-level watch helpers can flatten a
  side-indexed cache into unified position parser input.
- Applies the same side-aware cache path to REST snapshot preload payloads used
  by `fetchPositionsSnapshot`.

`Ccxt.Pro.CacheUpdates` owns the mutable counter semantics that CCXT Pro stores
as hidden cache properties:

- Tracks `allNewUpdates` and `newUpdatesBySymbol` for plain `ArrayCache`
  streams.
- Tracks nested per-symbol id/side sets for `ArrayCacheBySymbolById` and
  `ArrayCacheBySymbolBySide`.
- Tracks timestamp sets for `ArrayCacheByTimestamp`.
- Implements `getLimit`-style clear-on-next-append behavior without changing
  the existing list-shaped cache payloads used by current Elixir watchers.
- `Ccxt.Pro.Connection` wires these counters into public cached `watch_any`
  streams and private list/order watchers when callers pass `newUpdates: true`
  or `new_updates: true`.

## Remaining Cache Work

Remaining cache work is incremental runtime parity rather than method-specific
templates:

- Broaden timestamp-indexed cache use beyond Binance OHLCV if future exchange
  targets expose trade or candle payloads that use `ArrayCacheByTimestamp`.
- Wire any future non-position watcher families to `put_by_side/3` where
  TypeScript uses `ArrayCacheBySymbolBySide`.
- Broaden explicit `newUpdates: true` windows for future private stream payload
  shapes that can keep a list-compatible incremental return shape.
- Continue recording new cache classes in `pro_manifest.json` through generator
  runtime semantic evidence instead of only documenting them.