defmodule CouncilEx.Provider.Adapters.Ollama do
@moduledoc """
Config preset for routing the OpenAI-compatible adapter at a local Ollama
server. NOT a separate `Provider.Adapter` implementation — uses
`CouncilEx.Provider.Adapters.OpenAI` underneath.
## Setup
config :council_ex,
providers: [
ollama: CouncilEx.Provider.Adapters.Ollama.default_opts()
]
## Override base_url
config :council_ex,
providers: [
ollama: CouncilEx.Provider.Adapters.Ollama.default_opts(
base_url: "http://gpu-box:11434/v1"
)
]
Tool-calling and structured output work for any served model that
supports them via the OpenAI-compatible API (Llama 3.1+, Mistral,
Qwen 2.5+).
## Why not a native /api/chat adapter?
Ollama's OpenAI-compat endpoint covers everything `Provider.Adapter`
needs: complete, stream, tool-calling, structured output. A native
adapter would duplicate ~350 LOC for no functional gain. If the
OpenAI-compat layer ever loses parity with native, a `*.Native` adapter
can be added without breaking this preset.
"""
@default_base_url "http://localhost:11434/v1"
@doc """
Returns provider config keyword list for an Ollama-served model. Pass
any overrides as `keyword()` and they will replace baseline values.
"""
@spec default_opts(keyword()) :: keyword()
def default_opts(overrides \\ []) do
[
adapter: CouncilEx.Provider.Adapters.OpenAI,
api_key: "ollama",
base_url: @default_base_url
]
|> Keyword.merge(overrides)
end
end