# Searchex Readme
A full-text search engine written in pure Elixir.
This application is UNDER CONSTRUCTION - not yet ready for use.
## About Searchex
Searchex provides a search capability for full-text documents. Example
document types include:
- text, markdown, XML and JSON files
- source code files
- product descriptions
- blog and forum posts
- chat rooms and twitter feeds
- web pages
## Quick Start
TBD
## Searchex Architecture
A Searchex `DOCUMENT` has two key elements:
1. document `META-DATA`, like `title`, `author_name`, `publication_date`
2. the `FULL-TEXT` of the document
Searchex organizes documents into separate `COLLECTIONS`. Each collection has
two main elements:
1. the `CATALOG`, a table-like structure that contains the document ID,
meta-data, and document location.
2. the `INDEX`, an inverted index that is built for fast search and retrieval
Each collection is defined by a `CONFIG` file, a yaml file that specifies
things like:
- document directories
- file types
- meta-data fields definitions and extraction regexes
- document separator regex (for multi-doc files)
## Using Searchex
A `searchex` command-line program can be used to manage config files, catalogs
and indexes, and to perform searches.
There is an API that Elixir developers can use to embed Searchex into their
applications.
## Why Erlang/Elixir
Text indexing and search can be incredibly compute intensive, and benefit
massively from the concurrent/distributed capabilities of the BEAM.
# Package Installation
Add `searchex` to your list of dependencies in `mix.exs`:
```elixir
def deps do
[{:searchex, "~> 0.0.1-alpha.2"}]
end
```
Then run `mix deps.get`
## Escript Installation
If you have Elixir 1.3+ enter this at the console:
<pre><sub>mix escript.install https://raw.githubusercontent.com/andyl/stemex/master/stemex</sub></pre>
## Tab Completion
Get a tab-completion script by typing `searchex completion`
To install: copy this script to `/etc/bash_completion.d` (or equivalent)
$ searchex completion > /etc/bash_completion.d/searchex_completion.bash
$ chmod a+rx /etc/bash_completion.d/searchex_completion.bash
## Comparables and Reference Docs
* <http://blog.equanimity.nl/blog/2011/10/07/a-basic-full-text-search-server-in-erlang/>
* <http://stackoverflow.com/questions/195809/is-it-possible-to-develop-a-powerful-web-search-engine-using-erlang-mnesia-ya>
* <https://github.com/tmaciejewski/see>
* <http://himmele.blogspot.com/2011/04/build-your-own-internet-search-engine.html>
## System Overview
The search engine allows the establishment of one or more document collections.
The source data from a document collection is a textual file which cataloged
and indexed by Searchex.
### Documents
## GitHub Source
<https://github.com/andyl/searchex>