README.md

# cimg_ex
Light-weight image processing module in Elixir with CImg. This module aims to
create auxiliary routines for Deep Learning.

Note: It still has a few image processing functions currentrly.

### Design detail
The entity of the image handled by CImg is on the NIF side. On Elixir, the
reference to the image generated by NIF is stored in the %CImg{} structure. You
cannot read out the pixels of the image and process it directly, instead you
can use the image processing functions provided in CImg module.

The image will be assigned to Erlang Resource by NIF, so the image will
automatically be subject to garbage collection when it is no longer in use.

This is a very important point. Some of the functions in CImg module
mutably rewrite the original image. I recommend you to make a duplicate of
the image before performing the image processing. 

## Platform
It has been confirmed to work in the following OS environment.

- Windows MSYS2/MinGW64
- WSL2/Ubuntu 20.04

## Requirements
The following libraries are required to display images on the PC screen.

- GDI32 on Windows
- X11 on Linux

## Installation
Add following dependency to your `mix.exs`.

```elixir
def deps do
  [
    {:cimg, "~> 0.1.4"}
  ]
end
```

and install dependencies:

```shell
$ mix deps.get
$ mix deps.compile
```

## Demo
There is a simple program in demo directory. You can do it by following the steps below.

```shell
$ cd demo
$ mix deps.get
$ mix run -e "CImgDemo.demo1"
```

Close the appaired window, and stop the demo program.

Another demo:

```shell
$ mix run -e "CImgDemo.demo2"
```

```shell
$ mix run -e "CImgDemo.demo3"
```

## Example: Deep Labelling for Semantic Image Segmentation
The below code is an excerpt of the inference part from the DeepLab app implemented with TflInterp. CImg preprocesses the input image and provides a DeepLab model. The DeepLab result tensor is then converted by CImg into a colormaedp image.

```elixir
defmodule TflDemo.DeepLab3 do
  alias TflDemo.DeepLab3.Prediction

  def apply_deeplab3(img_file) do
    img = CImg.load(img_file)

    segments = Prediction.apply(img)
  end
end


defmodule TflDemo.DeepLab3.Prediction do
  use TflInterp, model: "priv/lite-model_deeplabv3_1_metadata_2.tflite"
  
  @deeplab3_shape {257, 257}

  def apply(img) do
    # preprocess
    bin =
      CImg.dup(img)
      |> CImg.get_resize(@deeplab3_shape)
      |> CImg.to_flat(range: {-1.0, 1.0})

    # prediction
    outputs =
      __MODULE__
      |> TflInterp.set_input_tensor(0, bin.data)
      |> TflInterp.invoke()
      |> TflInterp.get_output_tensor(0)
      |> Nx.from_binary({:f, 32}) |> Nx.reshape({257, 257, :auto})
      
    # postprocess
    outputs
    |> Nx.argmax(axis: 2)
    |> Nx.as_type({:u, 8})
    |> Nx.to_binary()
    |> CImg.create_from_bin(257, 257, 1, 1, "<u1")
    |> CImg.map("lines")    # convert into a colormaped image
  end
end
```

## License
cimg_ex is licensed under the Apache License Version 2.0.

#### -- license overview of included 3rd party libraries --
- The "CImg" Library is licensed under the CeCILL-C/CeCILL.
- The "stb" - single-file public domain libraries for C/C++ - is public domain or MIT licensed.