defmodule MobScreencast do
@moduledoc """
Capture the **device's own screen** as an H264 stream, from inside a mob app — the
in-app replacement for host-side `adb screenrecord`, so a NAT'd phone can publish its
screen with no adb and no host on the same network.
The native side captures the display (Android `MediaProjection`, iOS `ReplayKit` /
`ScreenCaptureKit`) and hardware-encodes it to **H264 on-device** (`MediaCodec` /
`VideoToolbox`), so the BEAM receives ready-to-send Annex-B NAL units rather than raw
frames. That output drops straight into a WebRTC RTP payloader on the
receiving side.
## Streaming
MobScreencast.start_stream(socket, bitrate: 2_000_000, max_size: 1280)
delivers each encoded access unit as a message to the calling process:
handle_info({:screencast, :frame, %{bytes: nal_units, format: :h264,
timestamp_ms: t, keyframe: kf?}}, socket)
* `bytes` — one access unit of Annex-B H264 (`00 00 00 01` start codes); a
keyframe frame is prefixed with SPS/PPS.
* `keyframe` — true for an IDR (a freshly-joined decoder needs one; request the
next via `request_keyframe/0`).
Stop with `stop_stream/1`.
## Permission
Screen capture needs explicit, per-session user consent (Android's
`MediaProjection` system dialog; iOS ReplayKit's broadcast prompt). `start_stream`
triggers it; `{:screencast, :permission, :granted | :denied}` reports the outcome.
"""
@nif :mob_screencast_nif
@type frame :: %{
bytes: binary(),
format: :h264,
width: non_neg_integer(),
height: non_neg_integer(),
timestamp_ms: non_neg_integer(),
keyframe: boolean()
}
@doc """
Start capturing + encoding the screen. Frames arrive as `{:screencast, :frame, map}`
messages to the calling process (see the module doc).
Options:
* `:bitrate` — target encoder bitrate in bits/sec (default `2_000_000`).
* `:max_size` — cap the longer screen edge to this many px, preserving aspect
(default: native resolution). Lower = less bandwidth/CPU. Currently honored
on Android only; the iOS encoder captures at native resolution.
* `:fps` — target frame rate (default `30`).
* `:keyframe_interval_ms` — force an IDR at least this often (default `2000`).
"""
@spec start_stream(Mob.Socket.t(), keyword()) :: Mob.Socket.t()
def start_stream(socket, opts \\ []) do
@nif.screencast_start_stream(:json.encode(stream_opts(opts)))
socket
end
@doc """
Build the config map passed to `screencast_start_stream/1`. Pure function
exposed so tests can pin defaults + serialisation without going through the
NIF. Note `:max_size` is currently honored on Android only — the iOS encoder
captures at native resolution (TODO in the iOS NIF).
"""
@spec stream_opts(keyword()) :: map()
def stream_opts(opts) do
%{
"bitrate" => Keyword.get(opts, :bitrate, 2_000_000),
"fps" => Keyword.get(opts, :fps, 30),
"keyframe_interval_ms" => Keyword.get(opts, :keyframe_interval_ms, 2_000)
}
|> put_optional("max_size", opts[:max_size])
end
@doc "Stop the active screen-capture session."
@spec stop_stream(Mob.Socket.t()) :: Mob.Socket.t()
def stop_stream(socket) do
@nif.screencast_stop_stream()
socket
end
@doc """
Ask the encoder to emit a keyframe (IDR) on the next frame — call this when a new
viewer joins so its decoder can start without waiting for the periodic keyframe.
"""
@spec request_keyframe() :: :ok
def request_keyframe do
@nif.screencast_request_keyframe()
:ok
end
defp put_optional(map, _key, nil), do: map
defp put_optional(map, key, value), do: Map.put(map, key, value)
end