# tear
[](https://hex.pm/packages/tear)
[](https://hexdocs.pm/tear/)
`tear` is a lexer/highlighter for Elixir. It faithfully parses Elixir into a list of tokens similar to the tokeniser inside the Elixir compiler. It passes all applicatable tests in the Elixir source tree.
Additionally, it provides APIs heavily inspired by [contour](https://hexdocs.pm/contour) to highlight Elixir code in HTML or in the terminal.
## Example
```sh
gleam add tear@1
```
```gleam
import gleam/io
import simplifile
import tear
pub fn main() {
let assert Ok(input) = simplifile.read("/dev/stdin")
io.println(
input
|> tear.highlight
|> tear.to_ansi
)
}
```
## Differences to Elixir
Elixirs lexer enforces additional rules w.r.t. identifers; Only certain (Unicode) classes of characters are allowed, you cannot mix scripts within one "section", confusable characters are forbidden, and all identifiers are normalised to NFC form. `tear` does **not** implement those rules - instead, identifiers may contain any sequence of graphemes that is not otherwise significant to the parser. This may be improved upon in the future.
Elixir enforces some other parsing rules at the lexer level - for example, it tracks nesting parenthesis and `do` blocks, and enforces a rule that no space is allowed between functions and the argument list. The main focus of `tear` is to produce a token stream for syntax highlighting, so it instead opts for accepting those things in the lexer instead.
The format of heredoc strings is not validated or changed by the lexer.
