README.md

# cimg_ex
A lightweight image processing module in Elixir using CImg, aimed at creating auxiliary routines for Deep Learning.

Note: Only a few image processing functions are available yet.

## Design detail
Image data handled by CImg is placed on the NIF side. On the Elixir side, image processing is performed through
`RESOURCE` of images - generated by NIF - stored in the %CImg{} structure. You cannot directly read or process pixels
in an image. Instead, you must use the image processing functions provided by the CImg module.

Images are automatically subject to garbage collection when they are no longer used. The NIFs `RESOURCE` management
mechanism is used to achieve this functionality.

Each function provided by CImg can be used alone or as a set of functions. In the latter case, a seed image is first
prepared using `CImg.builder/1` or similar, and then a series of functions are put together using Elixir's pipe syntax.

## Platform
I have confirmed it works in the following OS environment.

- Windows MSYS2/MinGW64
- WSL2/Ubuntu 20.04, 18.04
- Linux Mint 20 "Ulyana"
- Nerves rpi

## Requirements
python3 is required to build this module.

In addition, the following libraries are required to display images on a PC screen.

- GDI32 on Windows
- X11 on Linux

## Installation
Add following dependency to your `mix.exs`.

```elixir
def deps do
  [
    {:cimg, "~> 0.1.19"}
  ]
end
```

and install dependencies:

```shell
$ mix deps.get
$ mix deps.compile
```

## Demo
A sample program is in the demo directory. You can run it by following the steps below.

```shell
$ cd demo
$ mix deps.get
$ mix run -e "CImgDemo.demo1"
```

Closing the application window will stop the demo program.

Another demo:

```shell
$ mix run -e "CImgDemo.demo2"
```

```shell
$ mix run -e "CImgDemo.demo3"
```

## Example: Deep Labelling for Semantic Image Segmentation
The following code is an excerpt of the inference portion from the DeepLab app implemented using TflInterp. 
CImg preprocesses the input image and passes it through the DeepLab model.
And then, the tensor output by DeepLab is transformed by CImg into a colormaedp image.

```elixir
defmodule TflDemo.DeepLab3 do
  alias TflDemo.DeepLab3.Prediction

  def apply_deeplab3(img_file) do
    img = CImg.load(img_file)

    segments = Prediction.apply(img)
  end
end


defmodule TflDemo.DeepLab3.Prediction do
  use TflInterp, model: "priv/lite-model_deeplabv3_1_metadata_2.tflite"
  
  @deeplab3_shape {257, 257}

  def apply(img) do
    # preprocess
    bin = img
      |> CImg.resize(@deeplab3_shape)
      |> CImg.to_binary(range: {-1.0, 1.0})

    # prediction
    outputs =
      __MODULE__
      |> TflInterp.set_input_tensor(0, bin)
      |> TflInterp.invoke()
      |> TflInterp.get_output_tensor(0)
      |> Nx.from_binary({:f, 32}) |> Nx.reshape({257, 257, :auto})
      
    # postprocess
    outputs
    |> Nx.argmax(axis: 2)
    |> Nx.as_type({:u, 8})
    |> Nx.to_binary()
    |> CImg.from_binary(257, 257, 1, 1, "<u1")
    |> CImg.color_mapping(:lines)    # convert into a colormaped image
  end
end
```

## License
cimg_ex is licensed under the Apache License Version 2.0.

#### -- license overview of included 3rd party libraries --
- The "CImg" Library is licensed under the CeCILL-C/CeCILL.
- The "stb" - single-file public domain libraries for C/C++ - is public domain or MIT licensed.