README.md

# Domo

[![Build Status](https://travis-ci.com/IvanRublev/domo.svg?branch=master)](https://travis-ci.com/IvanRublev/domo)
[![Method TDD](https://img.shields.io/badge/method-TDD-blue)](#domo)
[![hex.pm version](http://img.shields.io/hexpm/v/domo.svg?style=flat)](https://hex.pm/packages/domo)

:warning: This library generates code for structures that can bring suboptimal compilation times increased to approx 20%

Domo is a library to model a business domain with type-safe structs and
composable tagged tuples.

The library aims for two goals:

  * to allow only valid states for domain entities modelled with structs
    by ensuring that their field values conforms the type definition

  * to separate struct type definition and field type constraints
    that can be reused among related contexts or applications
    f.e. in a micro-service setup

It's a library to declare what kind of values struct fields should have and
to generate run-time struct construction and verification functions.

The `new/1` constructor function generated by Domo ensures that the struct
field value matches Elixir data type or another struct according to the given
type definition. Then it ensures that the field's value is in the valid range
by calling the user-defined precondition function for the value's type.

The library is a practical tool for the compile-time check. It automatically
validates that the default values of the struct conform to its type.
and ensure that instances constructed with `new/1` function to be
a macro argument match their types.

## Rationale

One of the ways to validate that input data is of the model struct's type
is to do it within a constructor function like the following:

```elixir
defmodule User do
  @type divisible_5_integer :: integer()

  @type t :: %__MODULE__{
    id: divisible_5_integer(),
    name: String.t(),
    post_address: :not_given | String.t()
  }

  @enforce_keys [:id, :name]
  defstruct [:id, :name, post_address: :not_given]

  def new(id: id, name: name, post_address: post_address)
      when is_integer(id) and is_binary(name) and
            (post_address == :not_given or is_binary(post_address)) do
    if rem(id, 5) == 0 do
      struct!(__MODULE__, id: id, name: name)
    else
      raise "User id expected to be divisible by 5"
    end
  end
end
```

The code of the constructor function `new/1` written above ensures that
the structure instance would have the expected data. At the same time,
the equally looking code can repeat for almost every entity in the application,
especially when some of the field types are shared across various entities.

It'd be good to generate such constructor function `new/1` automatically
in a declarative way reducing the structure definition length to a minimum.

One of the possible ways to do so is with Domo library as the following:

```elixir
defmodule User do
  use Domo

  @type divisible_5_integer :: integer()
  precond divisible_5_integer: &(rem(&1, 5) == 0)

  @type t :: %__MODULE__{
    id: divisible_5_integer(),
    name: String.t(),
    post_address: :not_given | String.t()
  }

  @enforce_keys [:id, :name]
  defstruct [:id, :name, post_address: :not_given]
end
```

The struct and type definitions are already in the module.

What Domo adds on top are the constructor function `new/1`,
the `ensure_type!/1` function, and their `new_ok/1`
and `ensure_type_ok/1` versions returning ok-error tuple.

These functions ensure that the given enumerable has keys matching the struct
fields and has the values of the appropriate field types. Additionally,
for the value of the `divisible_5_integer` type, these functions check
if the precondition function defined via the `precond` macro returns true.

If these conditions are not fulfilled, the Domo added functions are raising
the `ArgumentError` exception or returning `{:error, message}` respectively.

The construction with automatic type ensurance of the `User` struct can
be as immediate as that:

```elixir
User.new(id: 55, name: "John")
%User{id: 55, name: "John", post_address: :not_given}

User.new(id: 55, name: nil, post_address: 3)
** (ArgumentError) the following values should have types defined for fields of the User struct:
  * Invalid value 3 for field :post_address of %User{}. Expected the value matching the :not_given | <<_::_*8>> type.
  * Invalid value nil for field :name of %User{}. Expected the value matching the <<_::_*8>> type.

User.new(id: 54, name: "John")
** (ArgumentError) the following values should have types defined for fields of the User struct:
  * Invalid value 54 for field :id of %User{}. Expected the value matching the integer() type. And a true value
from the precondition function "&(rem(&1, 5) == 0)" defined for User.divisible_5_integer() type.
```

After the modification of the existing struct its type can be ensured
like the following:

```elixir
user
|> struct!(name: "John Bongiovi")
|> User.ensure_type!()
%User{id: 55, name: "John Bongiovi", post_address: :not_given}

user
|> struct!(name: :john_bongiovi)
|> User.ensure_type!()
** (ArgumentError) the following values should have types defined for fields of the User struct:
* Invalid value :john_bongiovi for field :name of %User{}. Expected the value matching the <<_::_*8>> type.
```

Given validations can be especially effective if it is possible to reuse
user-defined types and preconditions for one struct in related structs
like a foreign key in database tables.

Domo enables this fully. That how it works for the value of
`divisible_5_integer` type defined in the `User` module mentioned above
and referenced from the `Account` struct:

```elixir
defmodule Account do
  use Domo

  @type t :: %__MODULE__{
    id: integer(),
    user_id: User.divisible_5_integer(),
    ballance: integer()
  }

  @enforce_keys [:id, :user_id, :ballance]
  defstruct @enforce_keys
end

Account.new(id: 1574, user_id: 55, ballance: 20087)
%Account{ballance: 20087, id: 1574, user_id: 55}

Account.new(id: 1574, user_id: 504, ballance: 20087)
** (ArgumentError) the following values should have types defined for fields of the Account struct:
  * Invalid value 504 for field :user_id of %Account{}. Expected the value
matching the integer() type. And a true value from the precondition function
"&(rem(&1, 5) == 0)" defined for User.divisible_5_integer() type.
```

Furthermore, it's possible to extract the common type and its precondition
to a module that can be shared across various applications using Domo, keeping
the integrity of the types appropriately.

The code of `User` and `Account` modules mentioned above can be refactored
for that like the following:

```elixir
defmodule Identifiers do
  import Domo

  @type user_id :: integer()
  precond user_id: &(rem(&1, 5) == 0)
end

defmodule User do
  use Domo

  @type t :: %__MODULE__{
    id: Identifiers.user_id(),
    name: String.t(),
    post_address: :not_given | String.t()
  }

  @enforce_keys [:id, :name]
  defstruct [:id, :name, post_address: :not_given]
end

defmodule Account do
  use Domo

  @type t :: %__MODULE__{
    id: integer(),
    user_id: Identifiers.user_id(),
    ballance: integer()
  }

  @enforce_keys [:id, :user_id, :ballance]
  defstruct @enforce_keys
end

User.new(id: 54, name: "John")
** (ArgumentError) the following values should have types defined for fields of the User struct:
  * Invalid value 54 for field :id of %User{}. Expected the value matching the integer() type. And a true value
from the precondition function "&(rem(&1, 5) == 0)" defined for Identifiers.user_id() type.

Account.new(id: 1574, user_id: 504, ballance: 20087)
** (ArgumentError) the following values should have types defined for fields of the Account struct:
  * Invalid value 504 for field :user_id of %Account{}. Expected the value
matching the integer() type. And a true value from the precondition function
"&(rem(&1, 5) == 0)" defined for Identifiers.user_id() type.
```

Domo library plays nicely together with [TypedStruct](https://hexdocs.pm/typed_struct/)
on the module level. That's how the `User` struct can be shortened even more,
having `new/1`, `ensure_type!/1`, `new_ok/1`, and `ensure_type_ok/1` functions
added automatically:

```elixir
defmodule User do
  use Domo
  use TypedStruct

  typedstruct do
    field :id, Identifiers.user_id(), enforce: true
    field :name, String.t(), enforce: true
    field :post_address, :not_given | String.t(), default: :not_given
  end
end

User.new_ok(id: 55, name: "John")
{:ok, %User{id: 55, name: "John", post_address: :not_given}}

User.new_ok(id: 54, name: "John")
{:error, [id: "Invalid value 54 for field :id of %User{}. \\
Expected the value matching the integer() type. And a true value \\
from the precondition function \\"&(rem(&1, 5) == 0)\\" defined \\
for User.divisible_5_integer() type."]}
```

## How it works

For `MyModule` struct using Domo, the library generates a `MyModule.TypeEnsurer`
module at the compile time. The latter verifies that the given fields matches the
type of `MyModule` and is used by `new/1` constructor and the other Domo
generated functions.

If the field is of the struct type that uses Domo as well, then the ensurance
of the field's value delegates to the `TypeEnsurer` of that struct.

Suppose the user defines the precondition function with the `precond/1` macro
for the type referenced in the struct using Domo. In that case,
the `TypeEnsurer` module calls the user-defined function as the last
verification step.

Domo library uses `:domo_compiler` to generate `TypeEnsurer` modules code. See
the [Setup](#setup) section for the compilers configuration.

The generated code can be found in the `_build/ENV/domo_generated_code`
directory. That code is compiled automatically and is there only for
the reader's confidence. Domo cleans the directory on the following compilation,
keeping generated type ensurers code only for recently created
or changed structs.

### Depending types tracking

Suppose the given structure field's type depends on a type defined in
another module. When the latter type or its precondition changes,
Domo recompiles the former module automatically to update its
`TypeEnsurer` to keep type validation correct.

That works in the same way for any number of intermediate modules
that can be between module defining struct and module defining type.

## Setup

### General setup

To use Domo in your project, add this to your `mix.exs` dependencies:

    {:domo, "~> 1.2.0"}

And the following line to the compilers:

    compilers: Mix.compilers() ++ [:domo_compiler],

To avoid `mix format` putting extra parentheses around macro calls,
you can add the following import to your `.formatter.exs`:

    [
      import_deps: [:domo]
    ]

### Setup for Phoenix hot reload

If you intend to call generated functions of structs using Domo from a Phoenix controller, add the following line to the endpoint's configuration in `config.exs` file:

    config :my_app, MyApp.Endpoint,
      reloadable_compilers: [:phoenix] ++ Mix.compilers() ++ [:domo_compiler],

Otherwise type changes wouldn't be hot-reloaded by Phoenix.

## Usage

### Define a structure

To describe a structure with field value contracts, `use Domo`, then define
your struct and its type.

```elixir
defmodule Wonder do
  use Domo

  @typedoc "A world wonder. Ancient or contemporary."
  @enforce_keys [:id]
  defstruct [:id, :name]

  @type t :: %__MODULE__{id: integer, name: nil | String.t()}
end
```

The generated structure has `new/1`, `ensure_type!/1` functions
and their non raising `new_ok/1` and `ensure_type_ok/1` versions
defined automatically.

Use these functions to create a new struct's instance and update an existing one.

```elixir
%{id: 123556}
|> Wonder.new()
|> struct!(name: "Eiffel tower")
|> Wonder.ensure_type!()
%Wonder{id: 123556, name: "Eiffel tower"}
```

At the run-time, each function checks the values passed in against
the fields types defined within the `t()` type. In case of mismatch, the functions
raise an `ArgumentError` or return `{:error, _}` tuple appropriately.

### Define preconditions for the structure fields values

To automatically verify ranges of values for the whole struct or a concrete
field's type, define a precondition function with the `precond/1` macro
for that type.

```elixir
defmodule Invoice do
  use Domo

  @enforce_keys [:id, :subtotal, :tax, :total]
  defstruct @enforce_keys

  @type id :: String.t()
  precond id: &match?(<<"INV", _, _ :: binary>>, &1)

  @type t :: %__MODULE__{
    id: id(),
    subtotal: integer(),
    tax: integer(),
    total: integer()
  }
  precond t: &(&1.subtotal + &1.tax == &1.total)
end
```

A precondition for a field's type generates code that runs faster. In
the example above defining the precondition for the `id` type can be more
performant then making the same check with a precondition for the whole `t` type.

The `new/1`, `ensure_type!/1`, and other Domo generated functions call
precondition function to consider if the value is correct after ensuring
its type. They return an error message if the precondition function
returns false for the given value.

You can reuse user-defined type and its precondition by extracting them to
a shared kernel module like the following:

```elixir
defmodule SharedKernel do
  import Domo

  @type invoice_id :: String.t()
  precond invoice_id: &match?(<<"INV", _, _ :: binary>>, &1)
end

defmodule Invoice do
  use Domo

  @enforce_keys [:id, :subtotal, :tax, :total]
  defstruct @enforce_keys

  @type t :: %__MODULE__{
    id: SharedKernel.invoice_id(),
    subtotal: integer(),
    tax: integer(),
    total: integer()
  }
  precond t: &(&1.subtotal + &1.tax == &1.total)
end
```

Now it's possible to reuse the `SharedKernel` model across related applications
for value types consistency.

### Define a tag to enrich the field's type

Tags can be used in [sum types](https://thoughtbot.com/blog/better-domain-modeling-in-elixir-with-sum-types)
of your domain model to handle use-cases of the business logic effectively.

To define a tag define a module and its type `t()` as a [tagged tuple](https://erlang.org/doc/getting_started/seq_prog.html#tuples).
That is a tuple of the module name and the associated value.

```elixir
import Domo.TaggedTuple

defmodule Height do
  defmodule Meters, do: @type t :: {__MODULE__, float()}
  defmodule Foots, do: @type t :: {__MODULE__, float()}

  @type t :: {__MODULE__, Meters.t() | Foots.t()}
end
```

It's possible add a tag or a tag chain to the associated value.
The tag chain attached to the value is a series of nested tagged tuples where
the value is in the core:

```elixir
alias Height.{Meters, Foots}
m = {Height, {Meters, 324.0}}
f = {Height, {Foots, 1062.992}}
```

To extract the value use pattern matching.

```elixir
{Height, {Meters, 324.0}} == m


def to_string({Height, {Meters, val}}), do: to_string(val) <> " m"
def to_string({Height, {Foots, val}}), do: to_string(val) <> " ft"
```

### Combine struct, tags, and `---/2` operator

The tag's type can be used to define the summary type for the field:

```elixir
defmodule Wonder do
  use Domo

  @typedoc "A world wonder. Ancient or contemporary."
  @enforce_keys [:id, :height]
  defstruct [:id, name: "", :height]

  @type t :: %__MODULE__{id: integer, name: String.t(), height: Height.t()}
end
```

To remove extra brackets from the tag chain definition or within the pattern matching
it's possible to use the `---/2` operator like the following:

```elixir
import Domo.TaggedTuple
alias Height.Meters

wonder = Wonder.new(id: 145, name: "Eiffel tower", height: Height --- Meters --- 324.0)
%Wonder{height: {Height, {Height.Meters, 324.0}}, id: 145, name: "Eiffel tower"}

Height --- Meters --- height_m = wonder.height
{Height, {Meters, 324.0}}
```

### Map syntax

It's still possible to modify a struct directly with `%{... | s }` map syntax
and other standard functions like `put_in/3` skipping the verification.
Please, use the `ensure_type/1` struct's function to validate the struct's
data after such modifications.

### Pipeland

To add a tag or a tag chain to a value in a pipe use `tag/2` macro
and to remove use `untag!/2` macro appropriately.

For instance:

```elixir
use Domo.TaggedTuple
alias Order.Id

identifier
|> untag!(Id)
|> String.graphemes()
|> Enum.intersperse("_")
|> Enum.join()
|> tag(Id)
```

## Options

The following options can be passed with `use Domo, [...]`

  * `unexpected_type_error_as_warning` - if set to true, prints warning
    instead of raising an exception for field type mismatch
    in autogenerated functions `new/1` and `ensure_type!/1`.
    Default is false.

  * `name_of_new_function` - the name of the autogenerated constructor
    function added to the module using Domo. The ok function name
    is generated automatically from the given one by omitting trailing `!`
    if any, and appending `_ok`.
    Defaults are `new` and `new_ok` appropriately.

  * `remote_types_as_any` - keyword list of types by modules that should
    be treated as any(). Value example `ExternalModule: [:t, :name], OtherModule: :t`
    Default is nil.

To set option globally add lines into the `config.exs` file like the following:

    config :domo, :unexpected_type_error_as_warning, true
    config :domo, :name_of_new_function, :new!

## Limitations

The recursive types like `@type t :: :end | {integer, t()}` are not supported. 
Because of that `Macro.t()` is not supported.

Parametrized types are not supported. Library returns `{:type_not_found, :key}` error for `@type dict(key, value) :: [{key, value}]` type definition.

`MapSet.t(value)` just checks that the struct is of `MapSet`. Precondition
can be used to verify set values.

Domo doesn't check struct fields default value explicitly; instead,
it fails when one creates a struct with wrong defaults.

Generated submodule with TypedStruct's `:module` option is not supported.

## Migration

To complete the migration to a new version of Domo, please, clean and recompile
the project with `mix clean --deps && mix compile` command.

## Adoption

It's possible to adopt Domo library in the project having user-defined
constructor functions as the following:

1. Add `:domo` dependency to the project, configure compilers as described in
   the [setup](#setup) section
2. Set the name of the Domo generated constructor function by adding
   `config :domo, :name_of_new_function, :constructor_name` option into
   the `confix.exs` file, to prevent conflict with original constructor
   function names if any
3. Add `use Domo` to existing struct
4. Change the calls to build the struct for Domo generated constructor
   function with name set on step 3 and remove original constructor function
5. Repeat for each struct in the project

## Performance 🐢

On the average, the current version of the library makes struct operations 
about 20% sower what may seem plodding. And it may look like non-performant
to run in production.

It's not that. The library ensures the correctness of data types at runtime and
it comes with the price of computation. As the result users get the application 
with correct states at every update that is valid in many business contexts.

    Generate 10000 inputs, may take a while.
    =========================================

    Construction of a struct
    =========================================
    Operating System: macOS
    CPU Information: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
    Number of Available Cores: 8
    Available memory: 16 GB
    Elixir 1.12.1
    Erlang 24.0.1

    Benchmark suite executing with the following configuration:
    warmup: 2 s
    time: 5 s
    memory time: 0 ns
    parallel: 1
    inputs: none specified
    Estimated total run time: 14 s

    Benchmarking __MODULE__.new(arg)...
    Benchmarking struct!(__MODULE__, arg)...

    Name                               ips        average  deviation         median         99th %
    struct!(__MODULE__, arg)       14.10 K       70.93 μs    ±64.12%       71.99 μs      154.99 μs
    __MODULE__.new(arg)            11.17 K       89.50 μs    ±48.90%       92.99 μs      171.99 μs

    Comparison: 
    struct!(__MODULE__, arg)       14.10 K
    __MODULE__.new(arg)            11.17 K - 1.26x slower +18.57 μs

    A struct's field modification
    =========================================
    Operating System: macOS
    CPU Information: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
    Number of Available Cores: 8
    Available memory: 16 GB
    Elixir 1.12.1
    Erlang 24.0.1

    Benchmark suite executing with the following configuration:
    warmup: 2 s
    time: 5 s
    memory time: 0 ns
    parallel: 1
    inputs: none specified
    Estimated total run time: 14 s

    Benchmarking %{tweet | user: arg} |> __MODULE__.ensure_type!()...
    Benchmarking struct!(tweet, user: arg)...

    Name                                                        ips        average  deviation         median         99th %
    struct!(tweet, user: arg)                               15.60 K       64.12 μs    ±68.42%          65 μs         142 μs
    %{tweet | user: arg} |> __MODULE__.ensure_type!()       12.65 K       79.08 μs    ±56.48%          81 μs         161 μs

    Comparison: 
    struct!(tweet, user: arg)                               15.60 K
    %{tweet | user: arg} |> __MODULE__.ensure_type!()       12.65 K - 1.23x slower +14.96 μs

## Contributing

1. Fork the repository and make a feature branch

2. Working on the feature, please add typespecs

3. After working on the feature format code with

       mix format

   run the tests to ensure that all works as expected with

       mix test

   and make sure that the code coverage is ~100% what can be seen with

       mix coveralls.html

4. Make a PR to this repository

## Changelog

### 1.2.7
* Fix the bug to make recompilation occur when fixing alias for remote type.

* Support custom errors to be returned from functions defined with `precond/1`.

### 1.2.6
* Validates type conformance of default values given with `defstruct/1` to the
  struct's `t()` type at compile-time.

* Includes only the most matching type error into the error message.

### 1.2.5
* Add `remote_types_as_any` option to disable validation of specified complex
  remote types. What can be replaced by precondition for wrapping user-defined type.

### 1.2.4
* Speedup resolving of struct types
* Limit the number of allowed fields types combinations to 4096
* Support `Range.t()` and `MapSet.t()`
* Keep type ensurers source code after compiling umbrella project
* Remove preconditions manifest file on `mix clean` command
* List processed structs giving mix `--verbose` option

### 1.2.3
* Support struct's attribute introduced in Elixir 1.12.0 for error checking
* Add user-defined precondition functions to check the allowed range of values
  with `precond/1` macro

### 1.2.2
* Add support for `new/1` calls at compile time f.e. to specify default values

### 1.2.1
* Domo compiler is renamed to `:domo_compiler`
* Compile `TypeEnsurer` modules only if struct changes or dependency type changes 
* Phoenix hot-reload with `:reloadable_compilers` option is fully supported

### 1.2.0 
* Resolve all types at compile time and build `TypeEnsurer` modules for all structs
* Make Domo library work with Elixir 1.11.x and take it as the required minimum version
* Introduce `---/2` operator to make tag chains with `Domo.TaggedTuple` module

### 0.0.x - 1.0.x 
* MVP like releases, resolving types at runtime. Adds `new` constructor to a struct

## Roadmap

* [x] Check if the field values passed as an argument to the `new/1`, 
      and `put/3` matches the field types defined in `typedstruct/1`.

* [x] Support the keyword list as a possible argument for the `new/1`.

* [x] Add module option to put a warning in the console instead of raising 
      of the `ArgumentError` exception on value type mismatch.

* [x] Make global environment configuration options to turn errors into 
      warnings that are equivalent to module ones.

* [x] Move type resolving to the compile time.

* [x] Keep only bare minimum of generated functions that are `new/1`,
      `ensure_type!/1` and their _ok versions.

* [x] Make the `new/1` and `ensure_type!/1` speed to be less or equal 
      to 1.5 times of the `struct!/2` speed.

* [x] Support `new/1` calls in macros to specify default values f.e. in other 
      structures. That is to check if default value matches type at compile time.

* [x] Support `precond/1` macro to specify a struct field value's contract 
      with a boolean function.

* [ ] Make a plugin for `TypedStruct` to specify a contract in the filed definition

* [ ] Evaluate full recompilation time for 1000 structs using Domo.

* [ ] Add use option to specify names of the generated functions.

* [ ] Add documentation to the generated for `new(_ok)/1`, and `ensure_type!(_ok)/1`
      functions in a struct.


## License

Copyright © 2021 Ivan Rublev

This project is licensed under the [MIT license](LICENSE).