# Traveller

Traveller provides a easy, stream-based traversal of a database table. This library is well-suited for backfills, migrations etc. It can handle massive tables because of the lazy enumeration.

## Function approach

### With cursors

The most basic use case is:

```elixir
MyRepo
|> Traveller.start_stream(MySchema)
|> Enum.map(fn record_batch ->
  # your logic here
 end)
```

You can also handle more advanced use cases. Let's say you have a `users` table and you want to iterate over all users, sorting by first name in reverse alphabetical order, then last name in reverse order, then id. But you want to start after "Z" and stop before "A". Also, you want a small batch size. You can do something like:

```elixir
MyRepo
|> Traveller.start_stream(
  MySchema,
  start_after: "Z",
  stop_before: "A",
  cursor: [desc: :first_name, desc: :last_name, asc: :id],
  chunk_size: 50
)
|> Enum.map(fn record_batch ->
  # your logic here
 end)
```

### With offset

Assuming you have the correct indexes cursor-based iteration is normally much more efficient. But if you want to do an offset based traversal, for whatever reason that is possible too:

```elixir
MyRepo
|> Traveller.start_stream(
  MySchema,
  mode: :offset,
  order_by: :id
)
|> Enum.map(fn record_batch ->
  # your logic here
 end)
```

## Module approach

You can also `use` it to bake certain options into a module. These can then be overridden later:

```elixir
defmodule YourModule do
  use Traveller,
    repo: Repo,
    schema: Person,
    cursor: :first_name,
    chunk_size: 100
end

Enum.each(YourModule.start_stream(chunk_size: 50), fn batch ->
  # do something
end)
```

## Installation

If [available in Hex](https://hex.pm/docs/publish), the package can be installed
by adding `traveller` to your list of dependencies in `mix.exs`:

```elixir
def deps do
  [
    {:traveller, "~> 0.1.0"}
  ]
end
```

Documentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_doc)
and published on [HexDocs](https://hexdocs.pm). Once published, the docs can
be found at <https://hexdocs.pm/traveller>.