# Instruments: Simple, powerful and fast metrics for Statsd and DataDog

[![ Version](](
[![ License](](

You're blind without metrics. Metrics should also be easy to add to you application
and have little performance impact. This module allows you to define metrics
with ease and see inside your application.

Instruments has the following types of metrics that closely mirror statsd.

  * **Counters**: Allow you to increment or decrement a value.
  * **Gauges**: Allow you to report a single value that changes over time
  * **Histograms**: Values are grouped into percentiles
  * **Timings**: Report a timed value in milliseconds
  * **Measurements**: Measure the execution time of a function
  * **Sets**: Add a value to a statsd set
  * **Events**: Report an event like a deploy using arbitrary keys and values

## Basic Usage

Reporting a metric is extremely simple; just `use` the Instruments module and call the
appropriate function:

defmodule ModuleThatNeedsMetrics do
  use Instruments

  def other_function() do

  def metrics_function() do
    Instruments.increment("my.counter", 3)
    Instruments.measure("metrics_function.other_fn_call", &other_function/0)

### Custom Namespaces
Often, all metrics inside a module have namespaced metrics. This is easy to accomplish
using `CustomFunctions`

defmodule RpcHandler do
  use Instruments.CustomFunctions, prefix: "my_service.rpc"

  def handle(:get, "/foo/bar") do

The above example will increment the "" metric by one.

## Probes
A probe is a metric that's periodically updated, like memory usage. It can be
tedious to define these on your own, so Instruments automates this process.
There are several different ways to define a probe:

The first, and easiest is to use the `:mfa` key, which takes a tuple of
`{Module, function, arguments}`

Probe.define!("erlang.process_count", :gauge,
  mfa: {:erlang, :system_info, [:process_count]})

The above will report the process count every ten seconds.
You can also select keys from a value. For example, when reporting memory usage:

Probe.define("erlang.memory", :gauge,
  mfa: {:erlang, :memory, []},
  keys: [:total, :processes])

In the above example, the `:erlang.memory()` function will be called, and it returns a
keyword list like:

[total: 19371280, processes: 4638128, processes_used: 4633792, system: 14733152,
 atom: 264529, atom_used: 250724, binary: 181960, code: 5843599, ets: 383504]

From this, the probe extracts the `:total` and `:processes` keys, creates two metrics,
`` and `erlang.memory.processes` and reports them.

You can also define probes via a passed in zero argument function.

Probe.define!("erlang.memory", :gauge,
  function: &:erlang.memory/0,
  keys: [:total, :processes])

The above function simplifies the earlier mfa example, above, calling `:erlang.memory()`
and extracting the `:total` and `:processes` keys.

Finally, if this isn't enough flexibility, you can implement the `Probe` behaviour and
pass in the module of your probe:

defmodule MyProbe do
  @behaviour Instruments.Probe
  # implementation of the callbacks

Probe.define!("my.probe", :gauge, module: MyProbe)

Your probe module will now experience lifecycle callbacks and can keep its own state.
More information on the `Probe` behaviour is in the `Instruments.Probe` moduledoc.

Probes also have two other options:

  * `report_interval`: (milliseconds) How often the probe is reported to the
     underlying stats package.

  * `sample_interval`: (milliseconds) How often the probe's data is collected.
     If not set, this defaults to the `report_interval`.

## Performance

There are a couple optimizations that keep Instruments fast.

#### ETS backed counters
Probe counters actually increment or decrement a value in an ETS table, every
`fast_counter_report_interval` milliseconds, the aggregated values are flushed to
statsd. Because of this, counters are effectively free and with a conservative flush interval,
will put little pressure on your statsd server.

#### IOData metric names

Instruments uses macros to implement the metric names, and automatically converts interpolated
strings into IOLists. This means you can have many generated names without increasing the
amount of binary memory you're using. For example:

def increment_rpc(rpc_name),
  do: Instruments.increment("my_module.rpc.#{rpc_name}")

will be rewritten to the call:

def increment_rpc(rpc_name),
  do: Instruments.increment(["my_module.rpc.", Kernel.to_string(rpc_name)])

If you wish, you may pass any IOData as the name of a metric.

#### Sample Rates
For histograms, measure calls and timings, the default sample rate is pegged to 0.1.
This is so you don't accidentally overload your metrics collector. It can be
overridden by passing `sample_rate: float_value` to your metrics call in the