# Omni
**Universal Elixir client for LLM APIs.**
Streaming text generation, tool use, and structured output.
## Features
- **Multi-provider** — supports many LLM providers out of the box (see table below)
- **Streaming-first** — all requests stream by default; `generate_text` is built on `stream_text`
- **Tool use** — define tools with schemas and handlers; the loop auto-executes and feeds results back
- **Structured output** — JSON Schema validation with constrained decoding and automatic retries
- **Pluggable model catalog** — bundled [models.dev](https://models.dev) snapshot by default, with optional live fetching or the [`llm_db`](https://hex.pm/packages/llm_db) package as alternative sources
- **Extensible** — add custom providers by implementing a behaviour
## Installation
Add Omni to your dependencies:
```elixir
def deps do
[
{:omni, "~> 1.6"}
]
end
```
Each built-in provider reads its API key from a standard environment variable
by default — if your keys are set, no configuration is needed:
| Provider | Environment variable |
| --- | --- |
| Alibaba | `DASHSCOPE_API_KEY` |
| Anthropic | `ANTHROPIC_API_KEY` |
| Google | `GEMINI_API_KEY` |
| Groq | `GROQ_API_KEY` |
| Moonshot AI | `MOONSHOT_API_KEY` |
| NEAR AI Cloud | `NEARAI_API_KEY` |
| Ollama Cloud | `OLLAMA_API_KEY` |
| OpenAI | `OPENAI_API_KEY` |
| OpenCode | `OPENCODE_API_KEY` |
| OpenRouter | `OPENROUTER_API_KEY` |
| Venice AI | `VENICE_API_KEY` |
| Z.ai | `ZAI_API_KEY` |
A local-Ollama provider (`Omni.Providers.Ollama`) is also included; it needs
no API key — configure it with the models you have pulled locally.
All built-in providers are loaded by default. To limit what loads at startup,
list provider modules (`:builtins` names all built-ins and may appear in the list):
```elixir
config :omni, :models, providers: [Omni.Providers.Anthropic, Omni.Providers.OpenAI]
# or: everything built-in plus a custom provider
config :omni, :models, providers: [:builtins, MyApp.Providers.Acme]
```
## Quick start
### Text generation
Pass a model tuple and a string:
```elixir
{:ok, response} = Omni.generate_text({:anthropic, "claude-sonnet-4-5-20250514"}, "Hello!")
response.message
#=> %Omni.Message{role: :assistant, content: [%Omni.Content.Text{text: "Hello! How can..."}]}
```
For multi-turn conversations, build a context with a system prompt and messages:
```elixir
context = Omni.context(
system: "You are a helpful assistant.",
messages: [
Omni.message(role: :user, content: "What is Elixir?"),
Omni.message(role: :assistant, content: "Elixir is a functional programming language..."),
Omni.message(role: :user, content: "How does it handle concurrency?")
]
)
{:ok, response} = Omni.generate_text({:anthropic, "claude-sonnet-4-5-20250514"}, context)
```
### Streaming
`stream_text` returns a `StreamingResponse` that you consume with event handlers:
```elixir
{:ok, stream} = Omni.stream_text({:anthropic, "claude-sonnet-4-5-20250514"}, "Tell me a story")
{:ok, response} =
stream
|> Omni.StreamingResponse.on(:text_delta, fn %{delta: text} -> IO.write(text) end)
|> Omni.StreamingResponse.complete()
```
For simple cases where you just need the text chunks:
```elixir
stream
|> Omni.StreamingResponse.text_stream()
|> Enum.each(&IO.write/1)
```
### Structured output
Pass a schema via the `:output` option to get validated, decoded output:
```elixir
schema = Omni.Schema.object(%{
name: Omni.Schema.string(description: "The capital city"),
population: Omni.Schema.integer(description: "Approximate population")
}, required: [:name, :population])
{:ok, response} =
Omni.generate_text(
{:anthropic, "claude-sonnet-4-5-20250514"},
"What is the capital of France?",
output: schema
)
response.output
#=> %{name: "Paris", population: 2161000}
```
### Tool use
Define tools with schemas and handlers — the loop automatically executes tool uses and feeds results back to the model:
```elixir
weather_tool = Omni.tool(
name: "get_weather",
description: "Gets the current weather for a city",
input_schema: Omni.Schema.object(
%{city: Omni.Schema.string(description: "City name")},
required: [:city]
),
handler: fn input -> "72°F and sunny in #{input.city}" end
)
context = Omni.context(
messages: [Omni.message("What's the weather in London?")],
tools: [weather_tool]
)
{:ok, response} = Omni.generate_text({:anthropic, "claude-sonnet-4-5-20250514"}, context)
```
## Model catalog
Model data loads at startup from a pluggable source. The default reads a bundled [models.dev](https://models.dev) snapshot — no network access required. The snapshot is refreshed with every release, with data-only patch releases for notable catalog changes (major model launches, pricing changes) and at least one release per month. If you need fresher data than that, fetch the catalog at boot instead (disk-cached, degrading gracefully to the bundled snapshot when models.dev is unreachable):
```elixir
config :omni, :models,
source: {Omni.Sources.ModelsDev, live: true}
```
Or source the catalog from the optional [`llm_db`](https://hex.pm/packages/llm_db) package (requires `{:llm_db, "~> 2026.7"}` in your deps):
```elixir
config :omni, :models, source: Omni.Sources.LLMDB
```
Sources can also be pinned per provider (`config :omni, Omni.Providers.OpenAI, source: ...`), and custom sources implement the `Omni.Source` behaviour. See the [HexDocs](https://hexdocs.pm/omni) for details.
## Documentation
Full API documentation is available on [HexDocs](https://hexdocs.pm/omni).
## License
This package is open source and released under the [Apache-2 License](https://github.com/aaronrussell/omni/blob/master/LICENSE).
© Copyright 2024-2026 [Push Code Ltd](https://www.pushcode.com/).