# ExDocMakeup
The default syntax highlighting used by ExDoc is not very good.
ExDocMakeup is a custom markdown processor that is meant to be used together with ExDoc.
It brings syntax highlighting by [Makeup](https://hexdocs.pm/makeup/Makeup.html)
([demo here](https://tmbb.github.io/makeup_demo/elixir.html)) to your package's documentation.
Makeup is a pure Elixir library to make your code prettier.
Makeup's syntax highlighting is much better than the default syntax highlighting used by ExDoc,
which is based on the [highlight.js](https://highlightjs.org) javascript library.
This package highlights the elixir code in your documentation, while using highlight.js
for languages it can't yet highlight.
**Note:**
Makeup colors your code using pure HTML and CSS, but it uses Javascript for further enhancements.
When you place the mouse cursor over a delimiter (`[`, `]`, `%{` `{`, `}`, etc.)
or a keyword such as `do`, `end`, `fn`, etc., it highlights the matching delimiter or keyword.
Except for this feature, syntax highlighting will work perfectly well without Javascript.
## Installation
This package is [available in Hex](https://hexdocs.pm/ex_doc_makeup).
It can be installed by adding `ex_doc_makeup` to your list of dependencies in `mix.exs`:
```elixir
def deps do
[
...
# Note the ex_doc version, it won't work with earlier versions
{:ex_doc, ">= 0.18.1", only: :dev},
{:ex_doc_makeup, "~> 0.1.0", only: :dev}
]
end
```
To configure ExDoc to use ExDocMakeup for better syntax highlighting,
add the following to your `:docs` key:
```elixir
docs: [
...
markdown_processor: ExDocMakeup,
...
]
```
When you run `mix docs`, `ex_doc` will use this package for better syntax highlighting.
## CSS Style
The style is what I have decided to call the *Samba Theme*.
It is a slightly customized mixture of two themes, shamelessly stolen from Pygments.
* the *Tango* theme for the *Day Mode*.
This theme is based on the color palette from the
[Tango Icon Theme Guidelines](http://tango.freedesktop.org/Tango_Icon_Theme_Guidelines).
* the *Paraíso Dark* theme for the *Night Mode*;
This theme was created by by [Jan T. Sott](https://github.com/idleberg)
with the [Base16 Builder](https://github.com/chriskempson/base16-builder)
by [Chris Kempson](https://github.com/chriskempson).
It was originally inspired by the work of Brazilian artist Rubens LP.
Both themes are owned by the Pygments team and were published under the BSD license.
ALthough the theme is different from the default one used by ExDoc,
it works well with the default color scheme used by ExDoc.
### Naming
The first theme is named after an Argentinian dance,
and the second one is named after a Brazilian artist.
*Samba*, a Brazilian dance, is an appropriate name for the mixture of the two themes.
The fact that the CSS Theme is named after a Portuguese word is not a coincidence.
It's part of my effort to further the agenda of the Great Software Brazilian Conspiracy,
as I've [once promised José Valim](https://elixirforum.com/t/discussion-about-syntax-preferences-split-posts/3436/81).
## Experimental Features
### Advanced Options
*(this API should be considered unstable; the option names might change in the future)*
It's possible to configure Makeup to highlight some custom keywords
through the `:lexer_options` keyword, which lives under the `:markdown_processor_options` keyword.
There are two kinds of keywords that can be highlighted:
* `extra_def_like` - a list of keywords that should be highlighted just like
`def`, `defp`, `defmacro`, etc.
These keywords are not only highlighted as keywords, but the identifier
that comes after them is highlighted as a function name in a function definition.
You may this option to define keywords that define functions or macros
(i.e. macros that expand to `def` or `defmacro`)
* `extra_declarations` - a list containing other keywords that should be highlighted
like `defmodule`, `def`, etc.
They are highlighted alone and have no effect the highlighting of other tokens.
You may use this function for macros that expand to `defmodule`.
To configure these options, add the following to your `:docs` key:
```elixir
docs: [
...
markdown_processor: ExDocMakeup,
markdown_processor_options: [
# `lexer_options` should be a map (not a keyword list!)
lexer_options: %{
# These options belong to the lexer for the "elixir" language
"elixir" => [
# Keywords as explained above (will be used by Makeup)
extra_def_like: [...],
extra_declarations: [...]],
}
]
...
]
```
**A word of warning:**
Judicious use of these options may enhance readability, but be sure to use them
only when they make sense.
The goal of syntax highlighting is to make it clearer what is going on in the code.
Be careful to only highlight as keywords things what actually *work* like `def` or `defmodule`
behind the scenes.
If you use these feature in an abusive way, you may actually deceive the reader.
#### Example
Because the above is a little abstract, let's illustrate it with a concrete example.
The [ProtocolEx](https://hexdocs.pm/protocol_ex/readme.html) package,
by [OvermindDL1](https://github.com/OvermindDL1), defines some macros that expand to
`def`, `defmacro` or `defmodule`.
The highlighted source is more readable and more consistent if those keywords
are highlighted like `def` or `defmodule`.
To enhance readability, we can pass the following options:
```elixir
docs: [
...
markdown_processor: ExDocMakeup,
markdown_processor_options: [
lexer_options: %{
"elixir" => [
extra_declarations: [
"defimplEx", "defimpl_ex",
"defprotocolEx", "defprotocol_ex"],
extra_def_like: ["deftest"]]
}
],
...
]
```
This produces the following results:
You can see that:
* the `:extra_declaration` keywords are highlighted as keywords and
* the `:extra_def_like` keyword (`deftest`) is highlighted like
the `def` keyword and the identifier that follows it is highlighted
like a function name
Without these configuration options, the keywords are highlighted like a normal
identifier:
### Markdown Plugins
The focus of ExDocMakeup is always syntax highlighting, of course.
But then I started wondering: I already have a custom markdown implementation included in ExDocMakeup.
Up until now, it only highlighted the code and nothing else.
However, Earmark, the undelying Markdown processor is extensible.
This means I can use it to experiment with other features that may not be desirable for inclusion in ExDoc itself.
Besides being used for API documentation, ExDoc can also be used to author general documents about Elixir (i.e "Guides").
Guides often need to incorporate code fragments, but they may contain bugs, or for that matter, contain syntax errors that make them impossible to compile.
Enter the `include` directive. It allows you to include fragments of code taken from files.
By extracting the code directly from the files, you guarantee that everything is up to date.
Besides that, you can also apply normal quality control to the code fragments (coverage, static analysis, unit testing, test whether the code actually compiles, etc.).
the `include` directive is invoked inside the markdown file.
It's a standard Earmark plugin (Earmark is the markdown implementation behind ExDoc and ExDocMakeup).
Like all plugins, it must appear in a line all by itself, starting with `$$`. Currently ExDocMakeup supports:
#### Include an entire file
The syntax is just the same as a normal Elixir function call
```
$$ include "lib/my_file.ex"
```
#### Include a range of lines (inclusive)
The syntax is the same; just add the `:lines` option with a line range:
```
$$ include "lib/my_file.ex", lines: 45..67
```
This is inconvenient because line numbers may change if you change the contents of the file, but people might find use for it in any case.
#### Include a block of code:
To include a block of code, independent on the line numbers, you must delimit it with special comments `# !begin: block_name` and `# !end: block_name`.
There has to be exactly 1 whitespace character between the `#` and the `!`.
For example:
```elixir
...
# !begin: my_func
def my_func(x), do: x + x
# !end: my_func
...
```
To include the block above, use the `:block` option, with the block name:
```
$$ include "lib/my_file.ex", block: "my_func"
```
This options is mutually exclusive with the `:lines` option (it wouldn't make sense otherwise).
Personally, I prefer the block format because unlike line numbers, it doesn't change when you edit the file
#### Configuring the language
Include some Elixir code (the default):
```
$$ include "lib/my_file.ex", lines: 55..66, lang: "elixir"
```
Include some Python code:
```
$$ include "lib/external/monty.py", block: "my_python_class", lang: "python"
```
The `:lang` option will be passed to the syntax highlighter.
Currently, ExDocMakeup only supports Elixir code, so other languages will be passed to Highlight.js.
#### Safety
The directive is a normal Elixir function call, extracted using `Code.string_to_quoted` and then evaluated by a mini-interpreter.
This is on purpose: supporting arbitrary Elixir here doesn't seem very smart; you'd get scoping issues and additional attack vectors.
The current implementation is probably not 100% safe yet, and it's possible that you can execute arbitrary code in the machine generating the docs.
It probably needs further restrictions on the datatypes that can be passed as arguments.
This shouldn't be much of a problem, because if you're running the docs it you are already compiling arbitrary (possibly untrusted) Elixir code.
This just means you need to review the docs somewhat more carefully.
#### Example
Suppose you have the following module doc:
```elixir
defmodule ExDocMakeup do
@moduledoc """
ExDoc-compliant markdown processor using [Makeup](https://github.com/tmbb/makeup) for syntax highlighting.
This package is optimized to be used with ExDoc, and not alone by itself.
It's just [Earmark](https://github.com/pragdave/earmark)
customized to use Makeup as a syntax highlighter plus some functions to make it
play well with ExDoc.
$$ include "lib/ex_doc_makeup/code_renderer.ex", block: "get_options"
"""
```
And the `lib/ex_doc_makeup/code_renderer.ex` file contains the following fragment:
```elixir
...
# !begin: get_options
# Get the options from the app's environment
defp get_options() do
Application.get_env(:ex_doc_makeup, :config_options, %{})
end
# !end: get_options
...
```
When run `mix docs`, ExDocMakeup will fetch the appropriate fragment, and render:
Although I show an example of running the directive inside a `@moduledoc` attribute, it's probably more useful when running it on standalone markdown files, like guides or additional pages.
#### Feedback?
What do you think of this API?
What do you think of the block delimiters (`# !begin:` and `# !end:`)?
What other features would you like to have?
#### Inspiration
This feature was inspired by a similar feature in Sphinx, the main python documentation tool: http://www.sphinx-doc.org/en/stable/
