# ALF
[![Hex.pm](https://img.shields.io/hexpm/v/alf.svg?style=flat-square)](https://hex.pm/packages/alf)
## Flow-based Application Layer Framework
#### ALF is a set of abstractions built on top Elixir GenStage which allows writing program following [Flow-Based Programming](https://en.wikipedia.org/wiki/Flow-based_programming) approach.
#### ALF is a successor of the [Flowex](https://github.com/antonmi/flowex) project. Check its [README](https://github.com/antonmi/flowex#readme) to get the general idea. ALF adds conditional branching, packet cloning, goto statement, and other functionalities. Therefore, one can create application trees (graphs) of arbitrary complexity.
## Installation
Just add `:alf` as dependency to your `mix.exs` file.
ALF starts its own supervisor (`ALF.DynamicSupervisor`). All the pipelines and managers are started under the supervisor
## Quick start
Read a couple of sections of [Flowex README](https://github.com/antonmi/flowex#readme) to get the basic idea of how your code is put to GenStages.
### Define your pipeline
A pipeline is a list of components defined in the `@components` module variable.
```elixir
defmodule ThePipeline do
use ALF.DSL
@components [
stage(:add_one),
stage(:mult_by_two),
stage(:minus_three)
]
def add_one(datum, _opts), do: datum + 1
def mult_by_two(datum, _opts), do: datum * 2
def minus_three(datum, _opts), do: datum - 3
end
```
### Start the pipeline
```elixir
:ok = ALF.Manager.start(ThePipeline)
```
This starts a manager (GenServer) with the `ThePipeline` name. The manager starts all the components and puts them under another supervision tree.
![alt text](images/add_mult_minus_pipeline.png "Your first simple pipeline")
### Use the pipeline
The only interface currently is the `stream_to` function (`stream_to/2` and `stream_to/3`).
It receives a stream or `Enumerable.t` and returns another stream where results will be streamed.
```elixir
inputs = [1,2,3]
output_stream = Manager.stream_to(inputs, Pipeline)
Enum.to_list(output_stream) # it returns [1, 3, 5]
```
Check [test/examples](https://github.com/antonmi/ALF/tree/main/test/examples) folder for more examples
### The main idea behind ALF DSL
User's code that is evaluated inside components may be defined either as a 2-arity function or as a module with the `call/2` function.
The name of the function/module goes as a first argument in DSL. And the name also become the component's name.
```elixir
stage(:my_fun)
# or
stage(MyComponent)
```
where `MyComponent` is
```elixir
defmodule MyComponent do
# optional
def init(opts), do: %{opts | foo: :bar}
def call(datum, opts) do
# logic is here
new_datum
end
end
```
One can specify a custom name:
```elixir
stage(:my_fun, name: :my_custom_name)
```
Most of the components accept the `opts` argument, the options will be passed as a second argument to the corresponding function.
```elixir
stage(MyComponent, opts: [foo: :bar])
```
Check `@dsl_options` in [lib/components](https://github.com/antonmi/ALF/tree/main/lib/componenets) for available options.
## Components overview
![alt text](images/all_components.png "All the components")
### Producer and Consumer
Nothing special to know, these are internal components that put at the beginning and at the end of your pipeline.
### Stage
Stage is the main component where one puts a piece of application logic. It might be a simple 2-arity function or a module with `call/2` function:
```elixir
stage(:my_fun, opts: %{foo: bar})
# or
stage(MyComponent, opts: %{})
```
where `MyComponent` is
```elixir
defmodule MyComponent do
# optional
def init(opts), do: %{opts | foo: :bar}
def call(datum, opts) do
# logic is here
new_datum
end
end
```
### Switch
Switch allows to forward IP (information packets) to different branches:
```elixir
switch(:my_switch_function,
branches: %{
part1: [stage(:foo)],
part2: [stage(:bar)]
},
opts: [foo: :bar]
)
# or with module
switch(MySwitchModule, ...)
```
The `my_switch_function` function is 2-arity function that must return the key of the branch:
```elixir
def my_switch_function(datum, opts) do
if datum == opts[:foo], do: :part1, else: :part2
end
# or
defmodule MySwitchModule do
# optional
def init(opts), do: %{opts | baz: :qux}
def call(datum, opts) do
if datum == opts[:foo], do: :part1, else: :part2
end
end
```
### Clone
Clones an IP, useful for background actions.
```elixir
clone(:my_clone, to: [stage(:foo), dead_end(:dead_end)])
```
### Goto
Send packet to a given `goto_point`
```elixir
goto(:my_goto_function, to: :my_goto_point, opts: [foo: :bar])
# or
goto(:MyGotoModule, to: :my_goto_point, opts: [foo: :bar])
```
The `function` function is 2-arity function that must return `true` of `false`
```elixir
def my_goto_function(datum, opts) do
datum == opts[:foo]
end
```
### GotoPoint
The `Goto` component companion
```elixir
goto_point(:goto_point)
```
### DeadEnd
IP won't propagate further. It's used alongside with the `Clone` component to avoid duplicate IPs in output
```elixir
dead_end(:dead_end)
```
### Plug and Unplug
Plug and Unplug are used for transforming data before and after reusable parts of a pipeline.
The components can not be used directly and are generated automatically when one use `plug_with` macro. See below.
### Decomposer and Recomposer
These components transform IPs. Decomposer creates several IPs based on one input IP. Recomposer does the opposite - creates a single IP based on a list of previously received IPs.
## Components / Pipeline reusing
### `stages_from` macro
One can easily include components from another pipeline:
```elixir
defmodule ReusablePipeline do
use ALF.DSL
@components [
stage(:foo),
stage(:bar)
]
end
defmodule ThePipeline do
use ALF.DSL
@components stages_from(ReusablePipeline) ++ [stage(:baz)]
end
```
### `plug_with` macro
Use the macro if you include other components that expect different type/format/structure of input data.
```elixir
defmodule ThePipeline do
use ALF.DSL
@components [
plug_with(AdapterModuleBaz, do: [stage(:foo), stage(:bar)])
] ++
plug_with(AdapterModuleForReusablePipeline) do
stages_from(ReusablePipeline)
end
end
```
`plug_with` adds `Plug` component before the components in the block and `Unplug` at the end.
The first argument is an "adapter" module which must implement the `plug/2` and `unplug/3` functions
```elixir
def AdapterModuleBaz do
def init(opts), do: opts # optional
def plug(datum, _opts) do
# the function is called inside the `Plug` component
# `datum` will be put on the "AdapterModuleBaz" until IP has reached the "unplug" component
# the function must return `new_datum` with the structure expected by the following component
new_datum
end
def unplug(datum, prev_datum, _opts) do
# here one can access previous "datum" in `prev_datum`
# transform the data back for the following components
end
end
```