Rebar3 Proper Plugin
=====
Run PropEr test suites.
By default, will look for all modules starting in `prop_` in the `test/`
directories of a rebar3 project, and running all properties (functions of arity
0 with a `prop_` prefix) in them.
Todo/Gotchas
----
- No automated tests yet since this repo runs tests for a living
Use
---
Add the plugin to your rebar config:
%% the plugin itself
{project_plugins, [rebar3_proper]}.
%% The PropEr dependency is required to compile the test cases
%% and will be used to run the tests as well.
{profiles,
[{test, [
{deps, [
%% hex
{proper, "1.3.0"}
%% newest from master
{proper, {git, "https://github.com/proper-testing/proper.git",
{branch, "master"}}}
]}
]}
]}.
Then just call your plugin directly in an existing application:
Usage: rebar3 proper [-d <dir>] [-m <module>] [-p <properties>]
[-n <numtests>] [-v <verbose>] [-c [<cover>]]
[--retry [<retry>]] [--regressions [<regressions>]]
[--store [<store>]] [--long_result <long_result>]
[--start_size <start_size>] [--max_size <max_size>]
[--max_shrinks <max_shrinks>]
[--noshrink <noshrink>]
[--constraint_tries <constraint_tries>]
[--spec_timeout <spec_timeout>]
[--any_to_integer <any_to_integer>]
-d, --dir directory where the property tests are located
(defaults to "test")
-m, --module name of one or more modules to test (comma-separated)
-p, --prop name of properties to test within a specified module
(comma-separated)
-n, --numtests number of tests to run when testing a given property
-s, --search_steps number of searches to run when testing a given
targeted property
-v, --verbose each propertie tested shows its output or not
(defaults to true)
-c, --cover generate cover data [default: false]
--retry If failing test case counterexamples have been
stored, they are retried [default: false]
--regressions replays the test cases stored in the regression
file. [default: false]
--store stores the last counterexample into the regression
file. [default: false]
--long_result enables long-result mode, displaying
counter-examples on failure rather than just false
--start_size specifies the initial value of the size parameter
--max_size specifies the maximum value of the size parameter
--max_shrinks specifies the maximum number of times a failing test
case should be shrunk before returning
--noshrink instructs PropEr to not attempt to shrink any
failing test cases
--constraint_tries specifies the maximum number of tries before the
generator subsystem gives up on producing an
instance that satisfies a ?SUCHTHAT constraint
--spec_timeout duration, in milliseconds, after which PropEr
considers an input to be failing
--any_to_integer converts instances of the any() type to integers in
order to speed up execution
All of [PropEr's standard configurations](http://proper.softlab.ntua.gr/doc/proper.html#Options)
that can be put in a consult file can be put in `{proper_opts, [Options]}.` in your rebar.config file.
Workflow
---
A workflow to handle errors and do development is being experimented with:
1. Run any properties with `rebar3 proper`
2. On a test failure, replay the last failing cases with `rebar3 proper --retry`
3. Call `rebar3 proper --store` if the cases are interesting and you want to keep them for the future. The entries will be appended in a `proper-regressions.consult` file in your configured test directory. Check in that file or edit it as you wish.
4. Use `rebar3 proper --regressions` to prevent regressions from happening by testing your code against all stored counterexamples
Per-Properties Meta functions
---
This plugin allows you to export additional meta functions to add per-property options and documentation. For example, in the following code:
```erlang
-module(prop_demo).
-include_lib("proper/include/proper.hrl").
-export([prop_demo/1]). % NOT auto-exported by PropEr, we must do it ourselves
prop_demo(doc) ->
%% Docs are shown when the test property fails
"only properties that return `true' are seen as passing";
prop_demo(opts) ->
%% Override CLI and rebar.config option for `numtests' only
[{numtests, 500}].
prop_demo() -> % auto-exported by Proper
?FORALL(_N, integer(), false). % always fail
prop_works() ->
?FORALL(_N, integer(), true).
prop_fails() ->
?FORALL(_N, integer(), false). % fails also
```
When run, the `prop_demo/0` property will _always_ run 500 times (if it does not fail), and on failure, properties with a doc value have it displayed:
```
...
1/3 properties passed, 2 failed
===> Failed test cases:
prop_demo:prop_demo() -> false (only properties that return `true' are seen as passing)
prop_demo:prop_fails() -> false
```
The meta function may be omitted entirely.
Changelog
----
- 0.12.0: drop compile phase since newer rebar3 versions handle all of that for us out of the box. Eliminates old deprecation warning.
- 0.11.1: fix unicode support in meta-functions output
- 0.11.0: add option to set search steps for targeted properties
- 0.10.4: add PropEr FSM template
- 0.10.3: fix the template change, which was apparently rushed.
- 0.10.2: create the regression file path if it doesn't exist; simplify prop_statem template
- 0.10.1: support per-app `erl_opts` values rather than only root config
- 0.10.0: support hooks for app and umbrella level; add per-property opts and docs via meta-functions; remove runtime dependency on PropEr and use the one specified by the app instead
- 0.9.0: support for umbrella projects
- 0.8.0: storage and replay of counterexamples
- 0.7.2: rely on a non-beta PropEr version
- 0.7.1: fix bug regarding lib and priv directories in code path
- 0.7.0: fix bug with include paths of hrl files from parent apps, support counterexamples with --retry
- 0.6.3: fix bug with cover-compiling in rebar 3.2.0 and above again
- 0.6.2: fix bug with cover-compiling in rebar 3.2.0 and above
- 0.6.1: fix bug on option parsing in config files
- 0.5.0: switches to package dependencies
- 0.4.0: switches license to BSD with templates
- 0.3.0: code coverage supported
- 0.2.0: basic functionality
- 0.1.0: first commits
