defmodule Injecto do
@moduledoc """
A behaviour module that defines both an Ecto schema and a JSON schema.
An Injecto schema uses the module attribute `@properties` to define an Ecto schema and
a JSON schema based on the `ex_json_schema` library. In doing so, it also injects a
`Jason` encoder implementation. The advantage of using an Injecto schema is to get a
consistent parsing and validating with Ecto changesets and JSON schema respectively
with minimal boilerplates. This consistency is helpful when working with struct-based
request or response bodies, because we can get accurate Swagger schemas for free.
## Example
In the following documentation, we will use this simple Injecto schema as example:
defmodule Post do
@properties %{
title: {:string, required: true},
description: {:string, []},
likes: {:integer, required: true, minimum: 0}
}
use Injecto
end
The module attribute `@properties` must be defined first before invoking `use Injecto`.
The properties attribute is a map with field names as keys and field specs as values.
A field spec is a 2-tuple of `{type, options}`. For scalar types, most Ecto field types
are supported, namely:
[
:binary,
:binary_id,
:boolean,
:float,
:id,
:integer,
:string,
:map,
:decimal,
:date,
:time,
:time_usec,
:naive_datetime,
:naive_datetime_usec,
:utc_datetime,
:utc_datetime_usec
]
Refer to Ecto's documentation on
[Primitive Types](https://hexdocs.pm/ecto/Ecto.Schema.html#module-primitive-types)
to see how these field types get translated into Elixir types.
Supported compound types include:
* `{:enum, atoms}` and `{:enum, keyword}`;
* `{:object, injecto_module}`; and
* `{:array, inner_type}` where `inner_type` can be a scalar, enum or object type.
## Usage: Ecto
On the Ecto side, `new/0` and `changeset/2` functions can create a `nil`-filled struct
and an Ecto changeset respectively.
iex> Post.new()
%Post{title: nil, description: nil, likes: nil}
iex> %Ecto.Changeset{valid?: false, errors: errors} = Post.changeset(%Post{}, %{})
iex> errors
[
likes: {"can't be blank", [validation: :required]},
title: {"can't be blank", [validation: :required]}
]
iex> post = %{title: "Valid", likes: 10}
iex> %Ecto.Changeset{valid?: true, errors: []} = Post.changeset(%Post{}, post)
The `parse/2` function convert a map to a changeset-validated struct.
iex> {:error, errors} = Post.parse(%{})
iex> errors
%{
likes: [{"can't be blank", [validation: :required]}],
title: [{"can't be blank", [validation: :required]}]
}
iex> post = %{title: "Valid", likes: 10}
iex> {:ok, %Post{title: "Valid", likes: 10}} = Post.parse(post)
iex> valid_posts = [%{title: "A", likes: 1}, %{title: "B", likes: 2}]
iex> {:ok, posts} = Post.parse_many(valid_posts)
iex> Enum.sort_by(posts, &(&1.title))
[
%Post{title: "A", likes: 1, description: nil},
%Post{title: "B", likes: 2, description: nil}
]
The `parse_many/2` function is the collection counter part of `parse/2`. One validation
error is considered to be an error for the entire collection:
iex> invalid_posts = [%{title: 1, likes: "A"}, %{title: 2, likes: "B"}]
iex> {:error, errors} = Post.parse_many(invalid_posts)
iex> errors
[
%{
likes: [{"is invalid", [type: :integer, validation: :cast]}],
title: [{"is invalid", [type: :string, validation: :cast]}]
},
%{
likes: [{"is invalid", [type: :integer, validation: :cast]}],
title: [{"is invalid", [type: :string, validation: :cast]}]
}
]
iex> valid_posts = [%{title: "A", likes: 1}, %{title: "B", likes: 2}]
iex> invalid_posts = [%{title: 1, likes: "A"}]
iex> {:error, errors} = Post.parse_many(valid_posts ++ invalid_posts)
iex> errors
[
%{
likes: [{"is invalid", [type: :integer, validation: :cast]}],
title: [{"is invalid", [type: :string, validation: :cast]}]
}
]
Note that JSON schema constraints such as `minimum: 0` are not caught by the Ecto changeset:
iex> post = %{title: "Invalid", likes: -1}
iex> %Ecto.Changeset{valid?: true, errors: []} = Post.changeset(%Post{}, post)
## Usage: JSON Schema
The function `json_schema/0` returns a resolved `ExJsonSchema.Scheam.Root` struct.
iex> %ExJsonSchema.Schema.Root{schema: schema} = Post.json_schema()
iex> schema
%{
"properties" => %{
"description" => %{
"anyOf" => [%{"type" => "string"}, %{"type" => "null"}]
},
"likes" => %{"minimum" => 0, "type" => "integer"},
"title" => %{"type" => "string"}
},
"required" => ["likes", "title"],
"title" => "Elixir.Post",
"type" => "object",
"x-struct" => "Elixir.Post"
}
Internally, this is used by `validate_json/1` to validate a map using the JSON schema.
iex> valid_post = %{title: "A", likes: 1}
iex> {:ok, ^valid_post} = Post.validate_json(valid_post)
iex> invalid_post = %{title: 123, likes: -1}
iex> {:error, errors} = Post.validate_json(invalid_post)
iex> Enum.sort(errors)
[
{"Expected the value to be >= 0", "#/likes"},
{"Type mismatch. Expected String but got Integer.", "#/title"}
]
"""
@doc """
Returns the struct with the fields populated with `nil`s:
iex> Post.new()
%Post{title: nil, description: nil, likes: nil}
"""
@callback new() :: struct()
@doc """
Returns an Ecto changeset:
iex> %Ecto.Changeset{valid?: false, errors: errors} = Post.changeset(%Post{}, %{})
iex> errors
[
likes: {"can't be blank", [validation: :required]},
title: {"can't be blank", [validation: :required]}
]
iex> post = %{title: "Valid", likes: 10}
iex> %Ecto.Changeset{valid?: true, errors: []} = Post.changeset(%Post{}, post)
Note that JSON schema constraints such as `minimum: 0` are not caught by the Ecto changeset:
iex> post = %{title: "Invalid", likes: -1}
iex> %Ecto.Changeset{valid?: true, errors: []} = Post.changeset(%Post{}, post)
"""
@callback changeset(struct(), map()) :: %Ecto.Changeset{}
@doc """
Returns a result of a validated Elixir struct or the validation errors:
iex> {:error, errors} = Post.parse(%{})
iex> errors
%{
likes: [{"can't be blank", [validation: :required]}],
title: [{"can't be blank", [validation: :required]}]
}
iex> post = %{title: "Valid", likes: 10}
iex> {:ok, %Post{title: "Valid", likes: 10}} = Post.parse(post)
Note that JSON schema constraints such as `minimum: 0` are not caught by `parse/2` by
default. Pass in the option `:validate_json` for JSON schema validation:
iex> post = %{title: "Invalid", likes: -1}
iex> {:ok, %Post{}} = Post.parse(post)
iex> post = %{title: "Invalid", likes: -1}
iex> {:error, errors} = Post.parse(post, validate_json: true)
iex> errors
[{"Expected the value to be >= 0", "#/likes"}]
"""
@callback parse(map(), Keyword.t()) :: {:ok, struct()} | {:error, any()}
@doc """
Calls `parse/2` on a list of maps. Returns `:ok` if all maps are parsed correctly.
iex> valid_posts = [%{title: "A", likes: 1}, %{title: "B", likes: 2}]
iex> {:ok, posts} = Post.parse_many(valid_posts)
iex> Enum.sort_by(posts, &(&1.title))
[
%Post{title: "A", likes: 1, description: nil},
%Post{title: "B", likes: 2, description: nil}
]
iex> invalid_posts = [%{title: 1, likes: "A"}, %{title: 2, likes: "B"}]
iex> {:error, errors} = Post.parse_many(invalid_posts)
iex> errors
[
%{
likes: [{"is invalid", [type: :integer, validation: :cast]}],
title: [{"is invalid", [type: :string, validation: :cast]}]
},
%{
likes: [{"is invalid", [type: :integer, validation: :cast]}],
title: [{"is invalid", [type: :string, validation: :cast]}]
}
]
iex> valid_posts = [%{title: "A", likes: 1}, %{title: "B", likes: 2}]
iex> invalid_posts = [%{title: 1, likes: "A"}]
iex> {:error, errors} = Post.parse_many(valid_posts ++ invalid_posts)
iex> errors
[
%{
likes: [{"is invalid", [type: :integer, validation: :cast]}],
title: [{"is invalid", [type: :string, validation: :cast]}]
}
]
Note that JSON schema constraints such as `minimum: 0` are not caught by `parse` by
default. Pass in the option `:validate_json` for JSON schema validation:
iex> posts = [%{title: "A", likes: -1}]
iex> {:ok, _} = Post.parse_many(posts)
iex> {:error, errors} = Post.parse_many(posts, validate_json: true)
iex> errors
[[{"Expected the value to be >= 0", "#/likes"}]]
"""
@callback parse_many([map()], Keyword.t()) :: {:ok, [struct()]} | {:error, any()}
@doc """
Validates and returns an `ex_json_schema` schema:
iex> %ExJsonSchema.Schema.Root{schema: schema} = Post.json_schema()
iex> schema
%{
"properties" => %{
"description" => %{
"anyOf" => [%{"type" => "string"}, %{"type" => "null"}]
},
"likes" => %{"minimum" => 0, "type" => "integer"},
"title" => %{"type" => "string"}
},
"required" => ["likes", "title"],
"title" => "Elixir.Post",
"type" => "object",
"x-struct" => "Elixir.Post"
}
"""
@callback json_schema() :: %ExJsonSchema.Schema.Root{}
@doc """
Serialises a map, and validates the deserialised result against the JSON schema:
iex> valid_post = %{title: "A", likes: 1}
iex> {:ok, ^valid_post} = Post.validate_json(valid_post)
iex> invalid_post = %{title: 123, likes: -1}
iex> {:error, errors} = Post.validate_json(invalid_post)
iex> Enum.sort(errors)
[
{"Expected the value to be >= 0", "#/likes"},
{"Type mismatch. Expected String but got Integer.", "#/title"}
]
"""
@callback validate_json(map()) :: {:ok, map()} | {:error, any()}
defmacro __using__(_opts) do
quote do
@derive {Jason.Encoder, only: Map.keys(@properties)}
@behaviour Injecto
@scalar_types [
:binary,
:binary_id,
:boolean,
:float,
:id,
:integer,
:string,
:map,
:decimal,
:date,
:time,
:time_usec,
:naive_datetime,
:naive_datetime_usec,
:utc_datetime,
:utc_datetime_usec
]
## ---------------------------------------------------------------------------
## Ecto
## ---------------------------------------------------------------------------
use Ecto.Schema
import Ecto.Changeset
Module.register_attribute(__MODULE__, :source, [])
@source @source || ""
@primary_key false
schema @source do
@properties
|> Enum.map(fn {name, {type, _opts}} ->
case type do
{:object, module} ->
embeds_one(name, module)
{:array, inner_type} ->
case inner_type do
{:enum, values} ->
field(name, {:array, Ecto.Enum}, values: values)
_ ->
if Enum.member?(@scalar_types, inner_type),
do: field(name, {:array, inner_type}),
else: embeds_many(name, inner_type)
end
{:enum, values} ->
field(name, Ecto.Enum, values: values)
type ->
field(name, type)
end
end)
end
@spec new() :: %__MODULE__{}
def new, do: %__MODULE__{}
@spec changeset(struct(), map()) :: %Ecto.Changeset{}
def changeset(struct, map), do: base_changeset(struct, map)
defoverridable changeset: 2
@spec base_changeset(struct(), map()) :: %Ecto.Changeset{}
defp base_changeset(struct, map) do
init_changeset =
struct
|> cast(map, all_non_embeds())
|> validate_required(required_non_embeds())
embedded_properties =
Enum.filter(properties(), fn {_name, {type, _opt}} -> embedded?(type) end)
Enum.reduce(
embedded_properties,
init_changeset,
fn {name, {_type, opts}}, acc_changeset ->
required? = Keyword.get(opts, :required, false)
acc_changeset |> cast_embed(name, required: required?)
end
)
end
@type result :: {:ok, %__MODULE__{}} | {:error, any()}
@spec parse(map() | %__MODULE__{}, Keyword.t()) :: result()
def parse(input, opts \\ []) do
input = ensure_nested_map(input)
validate_fn =
if Keyword.get(opts, :validate_json),
do: &validate_json/1,
else: fn input -> {:ok, input} end
with {:ok, _} <- validate_fn.(input) do
parse_ecto(input)
end
end
@spec parse_ecto(map() | %__MODULE__{}) :: result()
defp parse_ecto(input) do
changeset = __MODULE__.changeset(__MODULE__.new(), input)
if changeset.valid? do
{:ok, Ecto.Changeset.apply_changes(changeset)}
else
errors =
changeset
|> Ecto.Changeset.traverse_errors(&Function.identity/1)
{:error, errors}
end
end
@type results :: {:ok, [%__MODULE__{}]} | {:error, any()}
@spec parse_many([map()], Keyword.t()) :: results()
def parse_many(inputs, opts \\ []) do
reduced =
Enum.reduce(
inputs,
%{oks: [], errors: []},
fn input, acc ->
case parse(input, opts) do
{:ok, ok} -> %{acc | oks: [ok | acc[:oks]]}
{:error, error} -> %{acc | errors: [error | acc[:errors]]}
end
end
)
case reduced do
%{errors: [], oks: oks} -> {:ok, oks}
%{errors: errors} -> {:error, errors}
end
end
# Source: https://elixirforum.com/t/convert-a-nested-struct-into-a-nested-map/23814/7
@spec ensure_nested_map(map() | struct()) :: map()
@guarded_structs [Date, DateTime, NaiveDateTime, Time, Decimal]
defp ensure_nested_map(%{__struct__: struct} = data) when struct in @guarded_structs,
do: data
defp ensure_nested_map(struct) when is_struct(struct) do
map = Map.from_struct(struct)
:maps.map(fn _, value -> ensure_nested_map(value) end, map)
end
defp ensure_nested_map(map) when is_map(map),
do: :maps.map(fn _, value -> ensure_nested_map(value) end, map)
defp ensure_nested_map(list) when is_list(list),
do: Enum.map(list, &ensure_nested_map/1)
defp ensure_nested_map(data), do: data
## ---------------------------------------------------------------------------
## JSON Schema
## ---------------------------------------------------------------------------
@spec json_schema() :: %ExJsonSchema.Schema.Root{}
def json_schema() do
%{
"type" => "object",
"properties" => json_schema_properties(properties()),
"required" => required_fields(properties()) |> Enum.map(&Atom.to_string/1),
"title" => Atom.to_string(__MODULE__),
"x-struct" => Atom.to_string(__MODULE__)
}
|> ExJsonSchema.Schema.resolve()
end
@spec json_schema_properties(map()) :: map()
defp json_schema_properties(props) do
props
|> Enum.map(fn {name, {type, opts}} ->
schema =
case type do
{:object, module} -> Map.merge(object_schema(opts), module.json_schema().schema)
{:array, inner_type} -> array_schema({:array, inner_type}, opts)
{:enum, values} -> enum_schema({:enum, values}, opts)
type -> scalar_schema(type, opts)
end
schema =
if Keyword.get(opts, :required),
do: schema,
else: %{"anyOf" => [schema, %{"type" => "null"}]}
{Atom.to_string(name), schema}
end)
|> Enum.into(%{})
end
@spec array_schema({atom(), any()}, Keyword.t()) :: map()
defp array_schema({:array, inner_type}, opts) do
schema =
case inner_type do
{:enum, values} ->
%{"type" => "array", "items" => enum_schema({:enum, values}, [])}
inner_type when inner_type in @scalar_types ->
%{"type" => "array", "items" => %{"type" => Atom.to_string(inner_type)}}
_ ->
%{"type" => "array", "items" => inner_type.json_schema().schema}
end
opts =
opts
|> Keyword.take([:min_items, :max_items, :unique_items])
|> Enum.map(fn {key, value} ->
case key do
:min_items -> {"minItems", value}
:max_items -> {"maxItems", value}
:unique_items -> {"uniqueItems", value}
_ -> {Atom.to_string(key), value}
end
end)
|> Enum.into(%{})
Map.merge(schema, opts)
end
@spec enum_schema({:enum, [atom()] | Keyword.t()}, Keyword.t()) :: map()
defp enum_schema({:enum, values}, _opts) do
if Keyword.keyword?(values) do
%{"type" => "integer", "enum" => Keyword.values(values)}
else
%{"type" => "string", "enum" => Enum.map(values, &Atom.to_string/1)}
end
end
@spec scalar_schema(atom(), Keyword.t()) :: map()
defp scalar_schema(type, opts) do
case type do
:binary -> string_schema(opts)
:binary_id -> string_schema(opts)
:float -> number_schema(opts)
:id -> integer_schema(opts)
:integer -> integer_schema(opts)
:string -> string_schema(opts)
:map -> object_schema(opts)
:decimal -> string_schema(opts)
:date -> string_schema(Keyword.put(opts, :format, "date"))
:time -> string_schema(Keyword.put(opts, :format, "time"))
:time_usec -> string_schema(Keyword.put(opts, :format, "time"))
:naive_datetime -> string_schema(Keyword.put(opts, :format, "date-time"))
:naive_datetime_usec -> string_schema(Keyword.put(opts, :format, "date-time"))
:utc_datetime -> string_schema(Keyword.put(opts, :format, "date-time"))
:utc_datetime_usec -> string_schema(Keyword.put(opts, :format, "date-time"))
type -> %{"type" => Atom.to_string(type)}
end
end
@spec string_schema(Keyword.t()) :: map()
defp string_schema(opts) do
opts =
opts
|> Keyword.take([:format, :min_length, :max_length, :pattern])
|> Enum.map(fn {key, value} ->
case key do
:min_length -> {"minLength", value}
:max_length -> {"maxLength", value}
_ -> {Atom.to_string(key), value}
end
end)
|> Enum.into(%{})
Map.put(opts, "type", "string")
end
@spec object_schema(Keyword.t()) :: map()
defp object_schema(opts) do
opts =
opts
|> Keyword.take([
:additional_properties,
:property_names,
:min_properties,
:max_properties
])
|> Enum.map(fn {key, value} ->
case key do
:additional_properties -> {"additionalProperties", value}
:property_names -> {"propertyNames", value}
:min_properties -> {"minProperties", value}
:max_properties -> {"maxProperties", value}
_ -> {Atom.to_string(key), value}
end
end)
|> Enum.into(%{})
Map.put(opts, "type", "object")
end
@spec integer_schema(Keyword.t()) :: map()
defp integer_schema(opts) do
%{number_schema(opts) | "type" => "integer"}
end
@spec number_schema(Keyword.t()) :: map()
defp number_schema(opts) do
opts =
opts
|> Keyword.take([
:multiple_of,
:minimum,
:exclusive_minimum,
:maximum,
:exclusive_maximum
])
|> Enum.map(fn {key, value} ->
case key do
:multiple_of -> {"multipleOf", value}
:exclusive_minimum -> {"exclusiveMinimum", value}
:exclusive_maximum -> {"exclusiveMaximum", value}
_ -> {Atom.to_string(key), value}
end
end)
|> Enum.into(%{})
Map.put(opts, "type", "number")
end
@spec validate_json(map()) :: {:ok, map()} | {:error, any()}
def validate_json(map) do
with {:ok, encoded} <- Jason.encode(map),
{:ok, json} <- Jason.decode(encoded),
:ok <- ExJsonSchema.Validator.validate(json_schema(), json) do
{:ok, map}
end
end
## ---------------------------------------------------------------------------
## Utilities
## ---------------------------------------------------------------------------
@doc """
Returns the module attribute `@properties`.
iex> Post.properties()
%{
description: {:string, []},
likes: {:integer, [required: true, minimum: 0]},
title: {:string, [required: true]}
}
"""
@spec properties() :: map()
def properties(), do: @properties
@spec all_fields() :: [atom()]
defp all_fields(), do: all_fields(properties())
@spec all_fields(map()) :: [atom()]
defp all_fields(props) do
props
|> Enum.map(fn {name, _} -> name end)
end
@spec all_non_embeds() :: [atom()]
defp all_non_embeds(), do: all_non_embeds(properties())
@spec all_non_embeds(map()) :: [atom()]
defp all_non_embeds(props) do
props
|> Enum.filter(fn {_name, {type, _opts}} -> !embedded?(type) end)
|> Enum.map(fn {name, _} -> name end)
end
@spec optional_fields() :: [atom()]
defp optional_fields(), do: optional_fields(properties())
@spec optional_fields(map()) :: [atom()]
defp optional_fields(props) do
props
|> Enum.filter(fn {_name, {_type, opts}} ->
!Keyword.get(opts, :required, false)
end)
|> Enum.map(fn {name, _} -> name end)
end
@spec required_fields() :: [atom()]
defp required_fields(), do: required_fields(properties())
@spec required_fields(map()) :: [atom()]
defp required_fields(props) do
props
|> Enum.filter(fn {_name, {_type, opts}} ->
Keyword.get(opts, :required, false)
end)
|> Enum.map(fn {name, _} -> name end)
end
@spec required_non_embeds() :: [atom()]
defp required_non_embeds(), do: required_non_embeds(properties())
@spec required_non_embeds(map()) :: [atom()]
defp required_non_embeds(props) do
props
|> Enum.filter(fn {_name, {type, _opts}} -> !embedded?(type) end)
|> Enum.filter(fn {_name, {_type, opts}} ->
Keyword.get(opts, :required, false)
end)
|> Enum.map(fn {name, _} -> name end)
end
@spec embedded?(any()) :: boolean()
defp embedded?(type) do
case type do
{:object, _} ->
true
{:array, inner_type} ->
case inner_type do
{:enum, values} -> false
_ -> !Enum.member?(@scalar_types, inner_type)
end
{:enum, values} ->
false
_ ->
false
end
end
end
end
end
