buffalo-test

Performance test runner based on BuffaloBench and inspired by Mocha

The purpose of this plugin is not only run benchmarks but to allow the developers to ensure that their application performance does not drops between commits or pull requests. By default it will fail if performance drops a certain percentage (2.5% by default).

To accomplish this task it will save a history based on the current git head id, package version or current date (git by default)

Table of Contents

• buffalo-test
• Table of Contents
• Install
• Use
  • Example
  • Hooks
  • Tests
  • Exclusive tests and inclusive tests
• Comand-line usage
• Roadmap
• Contributing
• Legal

Install

You can get this library as a Node.js module available through the npm registry:

1// With npm
2$ npm install buffalo-test --save
3// With yarn
4$ yarn add buffalo-test

Use

Create a folder for your tests in your project (default bench) and put your files in there.

1let { compare, suite, benchmark } = require("buffalo-test");
2
3compare("regex vs string", () => {
4  benchmark("RegExp#test", () => /orl/.test("Hello World!"));
5  benchmark("String#indexOf", () => "Hello World!".indexOf("o") > -1);
6});
7
8suite("suite only", () => {
9  benchmark("suite only", () => /o/.test("Hello World!"));
10});

To run the benchmarks just run from your terminal:

1$ buffalo-test

It will show somthing like:

1
2    Compare: regex vs string
3        ✔ RegExp#test 6.104s (15,844,390 ops/sec)
4        ✔ String#indexOf 6.075s (495,271,182 ops/sec)
5        Compare: regex vs string completed in 12.179s, String#indexOf was fastest by 96.801%
6
7    Suite: suite only
8        ✔ suite only 5.871s (22,695,850 ops/sec)
9        Suite: suite only passed in 5.871s
10
113 passing (18.05s)

Example

See the bench/index.js file.

Hooks

Buffalo-test provides the same hooks as mocha, before(), after(), beforeEach() and afterEach(). before() and after() run only once by suite. beforeEach() and afterEach() run before and after each benchmark respectively.

Also beforeCycle() and afterCycle() can be used to run code before and after the benchmark cycle.

Tests

Although buffalo-test is not a test-runner, you can run tests using expect.js or another library to check in each cycle that the benchmark is good.

Failed benchmarks, errors or test non cycle will be showed in the terminal as mocha does.

Exclusive tests and inclusive tests

As with mocha, you can run only benchmarks by appending .only() or skip them by appending .skip() to the desired suite.

Comand-line usage

1Usage: buffalo-test [options] [dirs...]
2
3Performance test runner based on BuffaloBench and inspired by Mocha
4
5Options:
6  -v, --version                   Output the version number
7  -r, --reporter <spec>           Reporter to use (default: "spec")
8  --enable-regressions            Allow benchmark regressions, if false benchmark will fail if there is a regression (default: false)
9  --error-margin <margin>         Fail benchmark if performance drops this percentage amount (default: 2.5)
10  --ignore-internals              Ignore internal modules when logging (default: true)
11  --ignore-node-modules           Ignore node modules when logging (default: true)
12  -R, --repository <repository>   Repository to use (default: "file")
13  -t, --tag <tag>                 Tag to be used to identify this run (default: "git")
14  -T, --tag-date-format <format>  Date format for the date tag (default: "YYYY-MM-DD")
15  --file-name <file-name>         File to be used by the file repository (default: "bench/.buffalo-test.json")
16  --require                       Files to require before tests
17  -h, --help                      Display help for command
18
19Available reporters:
20 - spec
21
22Available repositories:
23 - file

Roadmap

• Allow hook description
• Allow to write pending tests
• Add --bail option
• Add --forbid-only option
• Add --forbid-pending option
• Implement option to load config from package.json or file
• Add init command
• Add create command

Contributing

• Use prettify and eslint to lint your code.
• Update the readme with an example if you add or change any functionality.

Legal

Author: Masquerade Circus. License Apache-2.0