# Structured Output Callbacks
This guide explains how to implement LLM callbacks for SubAgents, including support for `output: :text` mode.
## Overview
```elixir
SubAgent.new(
prompt: "Classify the sentiment of: {{text}}",
output: :text,
signature: "(text :string) -> {sentiment :string, confidence :float}"
)
```
Text mode uses the **same signature syntax** as PTC-Lisp mode, enabling seamless piping between agents.
## Callback Interface
Your callback receives:
```elixir
%{
system: String.t(),
messages: list(),
output: :ptc_lisp | :text,
schema: json_schema() | nil, # Present when agent has a complex return type
tools: [tool_def()] | nil, # Present when agent has tools
cache: boolean()
}
```
The `schema` key is present when the agent has a complex return type (map, list, float, int), regardless of whether tools are used. The `tools` key is present when the agent has tools defined.
For complex return types, `schema` contains a JSON Schema derived from the signature:
```elixir
# signature: "(text :string) -> {sentiment :string, confidence :float}"
# schema:
%{
"type" => "object",
"properties" => %{
"sentiment" => %{"type" => "string"},
"confidence" => %{"type" => "number"}
},
"required" => ["sentiment", "confidence"],
"additionalProperties" => false
}
```
## Implementation with ReqLLM
[ReqLLM](https://hexdocs.pm/req_llm) provides `generate_object/4` and `generate_with_tools/4` which handle structured output across providers (OpenAI, Anthropic, Google, OpenRouter, etc.).
```elixir
defmodule MyApp.LLMCallback do
@model "openrouter:anthropic/claude-sonnet-4-20250514"
# Tools present -> use generate_with_tools
def call(%{tools: tools} = req) when is_list(tools) and tools != [] do
messages = [%{role: :system, content: req.system} | req.messages]
case ReqLLM.generate_with_tools(@model, messages, tools) do
{:ok, response} ->
{:ok, %{content: response.content, tokens: extract_tokens(response.usage)}}
{:error, reason} ->
{:error, reason}
end
end
# Complex return type (schema present, no tools) -> use generate_object
def call(%{schema: schema} = req) when is_map(schema) do
messages = [%{role: :system, content: req.system} | req.messages]
case ReqLLM.generate_object(@model, messages, schema) do
{:ok, %{object: object, usage: usage}} ->
{:ok, %{content: Jason.encode!(object), tokens: extract_tokens(usage)}}
{:error, reason} ->
{:error, reason}
end
end
# Plain text (no schema, no tools) -> use generate_text
def call(req) do
messages = [%{role: :system, content: req.system} | req.messages]
case ReqLLM.generate_text(@model, messages) do
{:ok, response} ->
{:ok, %{
content: ReqLLM.Response.text(response),
tokens: extract_tokens(ReqLLM.Response.usage(response))
}}
{:error, reason} ->
{:error, reason}
end
end
defp extract_tokens(nil), do: %{input: 0, output: 0}
defp extract_tokens(usage) do
%{
input: usage[:input_tokens] || 0,
output: usage[:output_tokens] || 0
}
end
end
```
## Using PtcRunner.LLM (Simplest)
`PtcRunner.LLM.callback/1` handles all modes automatically:
```elixir
# One line - works for :text and :ptc_lisp modes
llm = PtcRunner.LLM.callback("sonnet")
# Use with any SubAgent
{:ok, step} = SubAgent.run(agent, llm: llm, context: %{...})
```
This is the recommended approach for LiveBooks, demos, and examples.
### Lower-Level API
If you need more control:
```elixir
# Direct call with request map
PtcRunner.LLM.callback("sonnet").(subagent_request)
# Structured output only
PtcRunner.LLM.ReqLLMAdapter.generate_object(model, messages, schema)
# Text generation only
PtcRunner.LLM.ReqLLMAdapter.generate_text(model, messages)
```
## Provider-Specific Implementation
If not using ReqLLM, implement provider-specific structured output:
### OpenAI
```elixir
def call(%{schema: schema} = req) when is_map(schema) do
result = OpenAI.chat(
model: "gpt-4o",
messages: [%{role: "system", content: req.system} | req.messages],
response_format: %{
type: "json_schema",
json_schema: %{name: "response", schema: schema, strict: true}
}
)
case result do
{:ok, resp} -> {:ok, %{content: resp.choices[0].message.content, tokens: resp.usage}}
{:error, _} = err -> err
end
end
```
### Anthropic (Tool-as-Schema)
```elixir
def call(%{schema: schema} = req) when is_map(schema) do
tool = %{
name: "respond",
description: "Return your structured response",
input_schema: schema
}
result = Anthropic.messages(
model: "claude-sonnet-4-20250514",
system: req.system,
messages: req.messages,
tools: [tool],
tool_choice: %{type: "tool", name: "respond"}
)
case result do
{:ok, %{content: [%{type: "tool_use", input: args}]} = resp} ->
{:ok, %{content: Jason.encode!(args), tokens: resp.usage}}
{:error, _} = err ->
err
end
end
```
## Validation
PtcRunner always validates responses against the schema. If validation fails, it retries with error feedback. This ensures correctness even if the provider's structured output isn't perfect.
## Testing
```elixir
# Simple schema
agent = SubAgent.new(
prompt: "Greet {{name}}",
output: :text,
signature: "(name :string) -> {message :string}"
)
# Nested schema
agent = SubAgent.new(
prompt: "Analyze: {{text}}",
output: :text,
signature: "(text :string) -> {analysis {sentiment :string, entities [:string]}}"
)
# Output-only shorthand
agent = SubAgent.new(
prompt: "Return a greeting",
output: :text,
signature: "{message :string}" # Equivalent to "() -> {message :string}"
)
{:ok, step} = SubAgent.run(agent, llm: &MyApp.LLMCallback.call/1, context: %{name: "Alice"})
```
Verify:
1. Response parses as valid JSON (when complex return type)
2. Response matches expected schema
3. Retries work when validation fails
4. Token counts are captured correctly
5. Plain text responses work when no signature or `:string` return type