# Sumo Rest
<img src="http://www.technovelgy.com/graphics/content/sumo_robot.jpg" align="right" style="float:right" height="400" />
Generic **Cowboy** handlers to work with **Sumo DB**
## Introduction
We, at Inaka, build our RESTful servers on top of [cowboy](https://github.com/ninenines/cowboy). We use [sumo_db](https://github.com/inaka/sumo_db) to manage our persistence and [trails](https://github.com/inaka/cowboy-trails) together with [cowboy-swagger](https://github.com/inaka/cowboy-swagger) for documentation.
Soon enough, we realized that we were duplicating code everywhere. Not every endpoint in our APIs is just a CRUD for some entity, but there are definitely lots of them in every server. As an example, most of our servers provide something like the following list of endpoints:
* `GET /users` - Returns the list of users
* `POST /users` - Creates a new user
* `PUT /users/:id` or `PATCH /users/:id` - Updates a user
* `DELETE /users/:id` - Deletes a user
* `GET /users/:id` - Retrieves an individual user
To avoid (or at least reduce) such duplication, we started using [mixer](https://github.com/chef/mixer). That way, we can have a *base_handler* in each application where all the common handler logic lives.
Eventually, all applications shared that same *base_handler*, so we decided to abstract that even further. Into its own app: **sumo_rest**.
## Architecture
This project dependency tree is a great way to show the architecture behind it.
As you'll see below, **Sumo Rest** gives you _base handlers_ that you can use on your **Cowboy** server to manage your **Sumo DB** entities easily. You just need to define your routes using **Trails** and provide proper metadata for each of them. In particular, you need to provide the same basic metadata **Swagger** requires. You can manually use the base handlers and call each of their functions when you need them, but you can also use **Mixer** to just _bring_ their functions to your own handlers easily.
## Usage
In a nutshell, **Sumo Rest** provides 2 cowboy rest handlers:
- [`sr_entities_handler`](src/sr_entities_handler.erl) that provides an implementation for
+ `POST /entities` - to create a new entity
+ `GET /entitites` - to retrieve the list of all entities
- [`sr_single_entity_handler`](src/sr_single_entity_handler.erl) that provides implementation for
+ `GET /entities/:id` - to retrieve an entity
+ `PUT /entities/:id` - to update (or create) an entity
+ `PATCH /entities/:id` - to update an entity
+ `DELETE /entities/:id` - to delete an entity
(Of course, the uris for those endpoints will not be exactly those, you have to define what _entities_ you want to manage.)
To use them you first have to define your models, by implementing the behaviours `sumo_doc` (from **Sumo DB**) and [`sumo_rest_doc`](src/sumo_rest_doc.erl).
Then you have to create a module that implements the `trails_handler` behaviour (from **Trails**) and _mix in_ that module all the functions that you need from the provided handlers.
## A Basic Example
You can find a very basic example of the usage of this app in the [tests](test/sr_test).
The app used for the tests (`sr_test`), makes no sense at all. Don't worry about that. It's just there to provide examples of usage (and of course to run the tests). It basically manages 2 totally independent entities:
- _elements_: members of an extremely naïve key/value store
- _sessions_: poorly-designed user sessions :trollface:
Let me walk you through the process of creating such a simple app.
### The application definition
In [sr_test.app](test/sr_test.app) file you'll find the usual stuff. The only particular pieces are:
* The list of `applications`, which includes `cowboy`, `katana`, `cowboy_swagger` and `sumo_db`.
* The list of `start_phases`. This is not a requirement, but we've found this is a nice way of getting **Sumo DB** up and running before **Cowboy** starts listening:
```erlang
{ start_phases
, [ {create_schema, []}
, {start_cowboy_listeners, []}
]
}
```
### The configuration
In [test.config](test/test.config) we added the required configuration for the different apps to work:
#### Swagger
We just defined the minimum required properties:
```erlang
, { cowboy_swagger
, [ { global_spec
, #{ swagger => "2.0"
, info => #{title => "SumoRest Test API"}
, basePath => ""
}
}
]
}
```
#### Mnesia
We've chosen **Mnesia** as our backend, so we just enabled debug on it (not a requirement, but a nice thing to have on development environments):
```erlang
, { mnesia
, [{debug, true}]
}
```
#### Sumo DB
**Sumo DB**'s **Mnesia** backend/store is really easy to set up. We will just have 2 models: _elements_ and _sessions_. We will store them both on **Mnesia**:
```erlang
, { sumo_db
, [ {wpool_opts, [{overrun_warning, 100}]}
, {log_queries, true}
, {query_timeout, 30000}
, {storage_backends, []}
, {stores, [{sr_store_mnesia, sumo_store_mnesia, [{workers, 10}]}]}
, { docs
, [ {elements, sr_store_mnesia, #{module => sr_elements}}
, {sessions, sr_store_mnesia, #{module => sr_sessions}}
]
}
, {events, []}
]
}
```
#### SR Test
Finally we add some extremely naïve configuration to our own app. In our case, just a list of users we'll use for authentication purposes (:warning: **Do NOT do this at home, kids** :warning:):
```erlang
, { sr_test
, [ {users, [{<<"user1">>, <<"pwd1">>}, {<<"user2">>, <<"pwd2">>}]}
]
}
```
### The application module
The next step is to come up with the main application module: [sr_test](test/sr_test/sr_test.erl). The interesting bits are all in the start phases.
#### `create_schema`
For **Sumo DB** to work, we just need to make sure we create the schema. We need to do a little trick to setup **Mnesia** though, because for `create_schema` to properly work, **Mnesia** has to be stopped:
```erlang
start_phase(create_schema, _StartType, []) ->
_ = application:stop(mnesia),
Node = node(),
case mnesia:create_schema([Node]) of
ok -> ok;
{error, {Node, {already_exists, Node}}} -> ok
end,
{ok, _} = application:ensure_all_started(mnesia),
sumo:create_schema();
```
#### `start_cowboy_listeners`
Since we're using **Trails**, we can let each module define its own ~~routes~~ trails. And, since we're using a single host we can use the fancy helper that comes with **Trails**:
```erlang
Handlers =
[ sr_elements_handler
, sr_single_element_handler
, sr_sessions_handler
, sr_single_session_handler
, cowboy_swagger_handler
],
Routes = trails:trails(Handlers),
trails:store(Routes),
Dispatch = trails:single_host_compile(Routes),
```
It's crucial that we _store_ the trails. Otherwise, **Sumo Rest** will not be able to find them later.
Then, we start our **Cowboy** server:
```erlang
TransOpts = [{port, 4891}],
ProtoOpts = [{env, [{dispatch, Dispatch}, {compress, true}]}],
case cowboy:start_http(sr_test_server, 1, TransOpts, ProtoOpts) of
{ok, _} -> ok;
{error, {already_started, _}} -> ok
end.
```
### The Models
The next step is to define our models (i.e. the entities our system will manage). We use a module for each model and all of them implement the required behaviours.
#### Elements
[Elements](test/sr_test/sr_elements.erl) are simple key/value pairs.
```erlang
-type key() :: integer().
-type value() :: binary() | iodata().
-opaque element() ::
#{ key => key()
, value => value()
, created_at => calendar:datetime()
, updated_at => calendar:datetime()
}.
```
`sumo_doc` requires us to add the schema, sleep and wakeup functions. Since we'll use maps for our internal representation (just like **Sumo DB** does), they're trivial:
```erlang
-spec sumo_schema() -> sumo:schema().
sumo_schema() ->
sumo:new_schema(elements,
[ sumo:new_field(key, string, [id, not_null])
, sumo:new_field(value, string, [not_null])
, sumo:new_field(created_at, datetime, [not_null])
, sumo:new_field(updated_at, datetime, [not_null])
]).
-spec sumo_sleep(element()) -> sumo:doc().
sumo_sleep(Element) -> Element.
-spec sumo_wakeup(sumo:doc()) -> element().
sumo_wakeup(Element) -> Element.
```
`sumo_rest_doc` on the other hand requires functions to convert to and from json (which should also validate user input):
```erlang
-spec to_json(element()) -> sumo_rest_doc:json().
to_json(Element) ->
#{ key => maps:get(key, Element)
, value => maps:get(value, Element)
, created_at => sr_json:encode_date(maps:get(created_at, Element))
, updated_at => sr_json:encode_date(maps:get(updated_at, Element))
}.
```
In order to convert from json we have two options: `from_json` or `from_ctx`. The difference is `from_json` accepts only a json body as a parameter, `from_ctx` receive a `context` structure which has the entire request and handler state besides the json body. We will see a `from_ctx` example in `sessions` section
```erlang
-spec from_json(sumo_rest_doc:json()) -> {ok, element()} | {error, iodata()}.
from_json(Json) ->
Now = sr_json:encode_date(calendar:universal_time()),
try
{ ok
, #{ key => maps:get(<<"key">>, Json)
, value => maps:get(<<"value">>, Json)
, created_at =>
sr_json:decode_date(maps:get(<<"created_at">>, Json, Now))
, updated_at =>
sr_json:decode_date(maps:get(<<"updated_at">>, Json, Now))
}
}
catch
_:{badkey, Key} ->
{error, <<"missing field: ", Key/binary>>}
end.
```
We also need to provide an `update` function for `PUT` and `PATCH`:
```erlang
-spec update(element(), sumo_rest_doc:json()) ->
{ok, element()} | {error, iodata()}.
update(Element, Json) ->
try
NewValue = maps:get(<<"value">>, Json),
UpdatedElement =
Element#{value := NewValue, updated_at := calendar:universal_time()},
{ok, UpdatedElement}
catch
_:{badkey, Key} ->
{error, <<"missing field: ", Key/binary>>}
end.
```
For **Sumo Rest** to provide urls to the callers, we need to specify the location URL:
```erlang
-spec location(element(), sumo_rest_doc:path()) -> binary().
location(Element, Path) -> iolist_to_binary([Path, "/", key(Element)]).
```
To let **Sumo Rest** avoid duplicate keys (and return `422 Conflict` in that case), we provide the optional callback `duplication_conditions/1`:
```erlang
-spec duplication_conditions(element()) -> sumo_rest_doc:duplication_conditions().
duplication_conditions(Element) -> [{key, '==', key(Element)}].
```
If your model has an `id` type different than integer, string or binary you have to implement `id_from_binding/1`. That function is needed in order to convert the `id` from `binary()` to your type. There is an example at `sr_elements` module for our test coverage. It only converts to `integer()` but that is the general idea behind that function.
```erlang
-spec id_from_binding(binary()) -> key().
id_from_binding(BinaryId) ->
try binary_to_integer(BinaryId) of
Id -> Id
catch
error:badarg -> -1
end.
```
The rest of the functions in the module are just helpers, particularly useful for our tests.
#### Sessions
[Sessions](test/sr_test/sr_sessions.erl) are very similar to elements. One difference is that session ids (unlike element keys) are auto-generated by the mnesia store. Therefore they're initially `undefined`. We don't need to provide a `duplication_conditions/1` function in this case since we don't need to avoid duplicates.
The most important difference with elements is sessions does't implement `from_json` callback. Remember, `from_json` only accepts the request body in json format. In sessions we also need the logged user in order to build our session. In this case we implement `from_ctx` instead of `from_json` since it accepts the entire request and the handler's state. That information is encapsulated in a `context` structure.
This is how the `context`'s spec looks like. It is composed by a `sr_request:req()` and a `sr_state:state()` structures. Modules `sr_state` and `sr_request` are available in order to manipulate them.
```erlang
-type context() :: #{req := sr_request:req(), state := sr_state:state()}
.. In sr_request.erl ...
-opaque req() ::
#{ body => sr_json:json()
, headers := [{binary(), iodata()}]
, path := binary()
, bindings := #{atom() => any()}
}.
... In sr_state.erl ...
-opaque state() ::
#{ opts := sumo_rest:options()
, id => binary()
, entity => sumo:user_doc()
, module := module()
, user_opts := map()
}.
```
And this is the `from_ctx` implementation
```erlang
-spec from_ctx(sumo_rest_doc:context()) -> {ok, session()} | {error, iodata()}.
from_ctx(#{req := SrRequest, state := State}) ->
Json = sr_request:body(SrRequest),
{User, _} = sr_state:retrieve(user, State, undefined),
case from_json_internal(Json) of
{ok, Session} -> {ok, user(Session, User)};
MissingField -> MissingField
end.
```
### The Handlers
Now, the juicy part: The cowboy handlers. We have 4, two of them built on top of `sr_entitites_handler` and the other two built on `sr_single_entity_handler`.
#### Elements
[sr_elements_handler](test/sr_test/sr_elements_handler.erl) is built on `sr_entities_handler` and handles the path `"/elements"`. As you can see, the code is really simple.
First we _mix in_ the functions from `sr_entities_handler`:
```erlang
-include_lib("mixer/include/mixer.hrl").
-mixin([{ sr_entities_handler
, [ init/3
, rest_init/2
, allowed_methods/2
, resource_exists/2
, content_types_accepted/2
, content_types_provided/2
, handle_get/2
, handle_post/2
]
}]).
```
Then, we only need to write the documentation for this module, and provide the proper `Opts` and that's all:
```erlang
-spec trails() -> trails:trails().
trails() ->
RequestBody =
#{ name => <<"request body">>
, in => body
, description => <<"request body (as json)">>
, required => true
},
Metadata =
#{ get =>
#{ tags => ["elements"]
, description => "Returns the list of elements"
, produces => ["application/json"]
}
, post =>
#{ tags => ["elements"]
, description => "Creates a new element"
, consumes => ["application/json"]
, produces => ["application/json"]
, parameters => [RequestBody]
}
},
Path = "/elements",
Opts = #{ path => Path
, model => elements
},
[trails:trail(Path, ?MODULE, Opts, Metadata)].
```
The `Opts` here include the trails path (so it can be found later) and the model behind it.
And there you go, **_no more code!_**
[`sr_single_element_handler`](test/sr_test/sr_single_element_handler.erl) is analogous but it's based on `sr_single_entity_handler`.
#### Sessions
[sr_sessions_handler](test/sr_test/sr_sessions_handler.erl) shows you what happens when you need to steer away from the default implementations in **Sumo Rest**. It's as easy as defining your own functions instead of _mixing_ them _in_ from the base handlers.
In this case we needed authentication, so we added an implementation for `is_authorized`:
```erlang
-spec is_authorized(cowboy_req:req(), state()) ->
{boolean(), cowboy_req:req(), state()}.
is_authorized(Req, State) ->
case get_authorization(Req) of
{not_authenticated, Req1} ->
{{false, auth_header()}, Req1, State};
{User, Req1} ->
Users = application:get_env(sr_test, users, []),
case lists:member(User, Users) of
true -> {true, Req1, State#{user => User}};
false ->
ct:pal("Invalid user ~p not in ~p", [User, Users]),
{{false, auth_header()}, Req1, State}
end
end.
```
Finally, we did something similar in [`sr_single_session_handler`](test/sr_test/sr_single_session_handler.erl). We needed the same authentication mechanism, so we just _mix_ it _in_:
```erlang
-mixin([{ sr_sessions_handler
, [ is_authorized/2
]
}]).
```
But we needed to prevent users from accessing other user's sessions, so we implemented `forbidden/2`:
```erlang
-spec forbidden(cowboy_req:req(), state()) ->
{boolean(), cowboy_req:req(), state()}.
forbidden(Req, State) ->
#{user := {User, _}, id := Id} = State,
case sumo:find(sessions, Id) of
notfound -> {false, Req, State};
Session -> {User =/= sr_sessions:user(Session), Req, State}
end.
```
And, since sessions can not be created with `PUT` (because their keys are auto-generated):
```erlang
-spec is_conflict(cowboy_req:req(), state()) ->
{boolean(), cowboy_req:req(), state()}.
is_conflict(Req, State) ->
{not maps:is_key(entity, State), Req, State}.
```
## A Full-Fledged App
For a more elaborated example on how to use this library, please check [lsl](https://github.com/inaka/lsl).
---
## Contact Us
For **questions** or **general comments** regarding the use of this library,
please use our public [hipchat room](http://inaka.net/hipchat).
If you find any **bugs** or have a **problem** while using this library, please
[open an issue](https://github.com/inaka/sumo_rest/issues/new) in this repo
(or a pull request :)).
And you can check all of our open-source projects at [inaka.github.io](http://inaka.github.io).
