brittle

tap à la mode

A TAP test runner built for modern times.

Usage

First install brittle from npm

npm i brittle

Then start writing tests

1import test from 'brittle'
2
3test('basic', function (t) {
4  t.is(typeof Date.now(), 'number')
5  t.not(typeof Date.now(), 'string')
6
7  t.ok(Date.now() > 0)
8  t.absent(null)
9
10  t.comment('text')
11
12  t.alike({ a: 1 }, { a: 1 })
13  t.unlike({ a: 2 }, { a: 3 })
14
15  t.pass()
16  t.fail()
17})
18
19test('asynchronous', async function (t) {
20  await new Promise(r => setTimeout(r, 250))
21  t.pass()
22})
23
24test('plans', function (t) {
25  t.plan(2)
26  t.pass()
27  setTimeout(() => t.pass(), 250)
28})
29
30test('classic subtest', function (t) {
31  t.test('subtest', function (sub) {
32    sub.plan(1)
33    sub.pass()
34  })
35})
36
37test('inverted subtest', function (t) {
38  const sub = t.test('subtest')
39  sub.plan(1)
40  sub.pass()
41})
42
43test('executions', async function (t) {
44  t.execution(() => 'should not throw')
45  await t.execution(async () => 'should not reject')
46})
47
48test('exceptions', async function (t) {
49  t.exception(() => { throw Error('expected to throw') })
50  await t.exception(async () => { throw Error('expected to reject') })
51})
52
53const a = test('inverted test without plan needs end()')
54a.pass()
55a.end()
56
57const b = test('inverted test with plan')
58b.plan(1)
59b.pass()
60
61const c = test('inverted tests can be awaited')
62c.plan(1)
63setTimeout(() => c.pass(), 250)
64await c

Every assertion can have a message, i.e. t.pass('msg'), t.ok(false, 'should be true'), etc.
There are also utilities like t.timeout(ms), t.teardown(fn), etc.
Check the API but also all the assertions here and utilities here.

Test Runtimes

When installed as a dependency or globally, the following commands for using different runtimes are available:

• brittle-node/brittle - Runs tests using Node.js.
• brittle-bare - Runs tests using Bare.
• brittle-pear - Runs tests using Pear.

API

1import { test, solo, skip, hook, todo, configure } from 'brittle'

test([name], [options], callback)

Create a classic test with an optional name.

Available options for any test creation:

• timeout (30000) - milliseconds to wait before ending a stalling test.
• solo (false) - Skip all other tests except the solo() ones.
• hook (false) - setup and teardown resources.
• skip (false) - skip this test, alternatively use the skip() function.
• todo (false) - mark this test as todo and skip it, alternatively use the todo() function.
• stealth (false) - only print test summary.

The callback function (can be async) receives an object called assert.
assert (or t) provides the assertions and utilities interface.

1import test from 'brittle'
2
3test('basic', function (t) {
4  t.pass()
5})

Test files can be executed directly with node, as they're normal Node.js programs.

The test method is conveniently both the default export and named exported method:

1import { test } from 'brittle'

Classic tests will run sequentially, buffering pending tests until any prior test catches up.

Any test function returns a promise so you can optionally await for its result:

1const isOk = await test('basic', function (t) {
2  t.pass()
3})

test([name], [options]) => assert

Create an inverted test with an optional name.

All options for inverted tests are listed here.

An object called assert (or t) is returned, the same as the classic test.

This time it's also a promise, it can be awaited and it resolves at test completion.

1import test from 'brittle'
2
3const t = test('basic')
4
5t.plan(1)
6
7setTimeout(() => {
8  t.pass()
9}, 1000)
10
11await t // Won't proceed past here until plan is fulfilled

For inverted tests without a plan, the end method must be called:

1const t = test('basic')
2
3setTimeout(() => {
4  t.pass()
5  t.end()
6}, 1000)
7
8await t

The end() method can be called inline, for inverted tests without a plan:

1const t = test('basic')
2t.pass()
3t.end()

Control flow of inverted is entirely dependent on where its assert is awaited.
The following executes one test after another:

1const a = test('first test')
2const b = test('second test')
3a.plan(1)
4b.plan(1)
5a.pass()
6await a
7b.pass()
8await b

Awaiting the promise gives you its result:

1const t = test('first test')
2t.plan(1)
3t.pass()
4const isOk = await t

stealth([name], [options], callback)

stealth([name], [options]) => assert

Create a stealth test.
This will provide a new sub-assert object that only prints the test summary without assertions and ends the current test upon a failed assertion.

All options are the same as test which are listed here.

t.test([name], [options], callback)

t.test([name], [options]) => assert

A subtest can be created by calling test on an assert (or t) object.
This will provide a new sub-assert object.

All options for subtests are listed here.

Using this in inverted style can be very useful for flow control within a test:

1test('basic', async function (t) {
2  const a = t.test('sub test')
3  const b = t.test('other sub test')
4
5  a.plan(1)
6  b.plan(1)
7
8  setTimeout(() => a.ok(true), Math.random() * 1000)
9  setTimeout(() => b.ok(true), Math.random() * 1000)
10  
11  // Won't proceed past here until both a and b plans are fulfilled
12  await a
13  await b
14
15  t.pass()
16})

Subtest test options can be set by passing an object to the test function:

1test('parent', { timeout: 1000 }, function (t) {
2  t.test('basic using parent config', async function (t) {
3    await new Promise(r => setTimeout(r, 500))
4    t.pass()
5  })
6
7  t.test('another basic using parent config', function (t) {
8    t.pass()
9  })
10})

You can also await for its result as well:

1test('basic', async function (t) {
2  t.plan(1)
3  t.pass()
4  const isOk = await t
5  console.log(isOk)
6})

t.stealth([name], [options], callback)

t.stealth([name], [options]) => assert

Create a stealth sub-test.
This will provide a new sub-assert object that only prints the test summary without assertions and ends the current test upon a failed assertion.

All options are the same as test which are listed here.

solo([name], [options], callback)

solo([name], [options]) => assert

Filter out other tests by using the solo method:

1import { test, solo } from 'brittle'
2
3test('this test is skipped', function (t) {
4  t.pass()
5})
6
7solo('some test', function (t) {
8  t.pass()
9})

If a solo function is used, test functions will not execute.\

If solo is used in a future tick (for example, in a setTimeout callback),
after test has already been used those tests won't be filtered.

A few ways to enable solo functions:

• Use configure({ solo: true }) before any tests.
• You can call solo() without callback underneath the imports.
• Using the --solo flag with the brittle test runner.

It can also be used as an inverted test:

1const t = test.solo('inverted some test')
2t.pass()
3t.end()

skip([name], [options], callback)

Skip a test:

1import { test, skip } from 'brittle'
2
3skip('this test is skipped', function (t) {
4  t.pass()
5})
6
7test('middle test', function (t) {
8  t.pass()
9})
10
11test.skip('another skipped test', function (t) {
12  t.pass()
13})

Only the middle test will be executed.

hook([name], [options], callback)

Use before tests for setting up and after tests for tearing down. Runs the same way as test except always executes regardless of solo usage.

1import { test, solo, hook } from 'brittle'
2
3hook('setup hook', function (t) {
4  // setup resources
5})
6
7solo('solo test', function (t) {
8  t.pass()
9})
10
11test('middle test', function (t) {
12  t.pass()
13})
14
15hook('teardown hook', function (t) {
16  // teardown resources
17})

The setup hook, solo test and teardown hook will be executed.

configure([options])

The configure function can be used to set options for all tests (including child tests).
It must be executed before any tests.

Options

• timeout (30000) - milliseconds to wait before ending a stalling test
• bail (false) - exit the process on first test failure
• solo (false) - skip all other tests except the solo() ones
• source (true) - shows error source information
• unstealth (false) - show assertions even if stealth is used
• coverage (false) - enable coverage reporting (string path of the output directory or true for default)

1import { configure } from 'brittle'
2
3configure({ timeout: 15000 }) // All tests will have a 15 seconds timeout

Assertions

t.is(actual, expected, [message])

Compare actual to expected with ===

t.not(actual, expected, [message])

Compare actual to expected with !==

t.alike(actual, expected, [message])

Object comparison, comparing all primitives on the actual object to those on the expected object using ===.

t.unlike(actual, expected, [message])

Object comparison, comparing all primitives on the actual object to those on the expected object using !==.

t.ok(value, [message])

Checks that value is truthy: !!value === true

t.absent(value, [message])

Checks that value is falsy: !!value === false

t.pass([message])

Asserts success. Useful for explicitly confirming that a function was called, or that behavior is as expected.

t.fail([message])

Asserts failure. Useful for explicitly checking that a function should not be called.

t.exception(Promise|function|async function, [error], [message])

Verify that a function throws, or a promise rejects.

1t.exception(() => { throw Error('an err') }, /an err/)
2await t.exception(async () => { throw Error('an err') }, /an err/)
3await t.exception(Promise.reject(Error('an err')), /an err/)

If the error is an instance of any of the following native error constructors, then this will still result in failure since native errors often tend to be unintentational.

• SyntaxError
• ReferenceError
• TypeError
• EvalError
• RangeError

If a t.exception is async, then you're supposed to await it.

t.exception.all(Promise|function|async function, [error], [message])

Verify that a function throws, or a promise rejects, including native errors.

1t.exception.all(() => { throw Error('an err') }, /an err/)
2await t.exception.all(async () => { throw Error('an err') }, /an err/)
3await t.exception.all(Promise.reject(new SyntaxError('native error')), /native error/)

The t.exception.all method is an escape-hatch so it can be used with the normally filtered native errors.

If a t.exception.all is async, then you're supposed to await it.

t.execution(Promise|function|async function, [message])

Assert that a function executes instead of throwing or that a promise resolves instead of rejecting. Resolves to the execution time, in milliseconds, of the function or promise.

1t.execution(() => {})
2await t.execution(async () => {})
3await t.execution(Promise.resolve('cool'))

If a t.execution is async, then you're supposed to await it

t.is.coercively(actual, expected, [message])

Compare actual to expected with ==.

t.not.coercively(actual, expected, [message])

Compare actual to expected with !=.

t.alike.coercively(actual, expected, [message])

Object comparison, comparing all primitives on the actual object to those on the expected object using ==.

t.unlike.coercively(actual, expected, [message])

Object comparison, comparing all primitives on the actual object to those on the expected object using !=.

Utilities

t.plan(n)

Constrain a test to an explicit amount of assertions.

t.tmp() -> <Promise<String>>

Creates a temporary folder and returns a promise that resolves its path. Once a test either succeeds or fails, the temporary folder is removed.

t.teardown(function|async function, [options])

Options:

• order (0) - set the ascending position priority for a teardown to be executed.
• force (false) - run the teardown on failure as well as success

The function passed to teardown is called right after a test ends:

1test('basic', function (t) {
2  const timeoutId = setTimeout(() => {}, 1000)
3
4  t.teardown(async function () {
5    clearTimeout(timeoutId)
6    await doMoreCleanUp()
7  })
8
9  t.ok('cool')
10})

If teardown is called multiple times in a test, every function passed will be called after the test ends:

1test('basic', function (t) {
2  t.teardown(doSomeCleanUp)
3
4  const timeoutId = setTimeout(() => {}, 1000)
5  t.teardown(() => clearTimeout(timeoutId))
6
7  t.ok('again, cool')
8})

Set order: -Infinity to always be in first place, and vice versa with order: Infinity.
If two teardowns have the same order they are ordered per time of invocation within that order group.

1test('teardown order', function (t) {
2  t.teardown(async function () {
3    await new Promise(r => setTimeout(r, 200))
4    console.log('teardown B')
5  })
6
7  t.teardown(async function () {
8    await new Promise(r => setTimeout(r, 200))
9    console.log('teardown A')
10  }, { order: -1 })
11
12  t.teardown(async function () {
13    await new Promise(r => setTimeout(r, 200))
14    console.log('teardown C')
15  }, { order: 1 })
16
17  t.pass()
18})

The A teardown is executed first, then B, and finally C due to the order option.

t.timeout(ms)

Fail the test after a given timeout.

t.comment(message)

Inject a TAP comment into the output.

t.end()

Force end a test.
end is determined by assert resolution or when a containing async function completes.
In case of inverted tests, they're required to be explicitly called.

Readable Properties

t.name

The name of the test.

t.passes

The number of assertions that passed within the test.

t.fails

The number of assertions that failed within the test.

t.assertions

The number of assertions that were executed within the test.

Runner

Default timeout

The default timeout is 30 seconds.

Example of package.json with test script

The following would run all .js files in the test folder:

1{
2  "name": "my-app",
3  "version": "1.0.0",
4  "scripts": {
5    "test": "brittle test/*.js"
6  },
7  "devDependencies": {
8    "brittle": "^3.0.0-alpha.3"
9  }
10}

CLI

1npm install -g brittle

1brittle [flags] <files>
2
3Flags:
4  --solo, -s                Engage solo mode
5  --bail, -b                Bail out on first assert failure
6  --coverage, -c            Turn on coverage
7  --cov-dir <dir>           Configure coverage output directory (default: ./coverage)
8  --trace                   Trace all active promises and print them if the test fails
9  --timeout, -t <timeout>   Set the test timeout in milliseconds (default: 30000)
10  --runner, -r <runner>     Generates an out file that contains all target tests
11  --mine, -m <miners>       Keep running the tests in <miners> processes until they fail.
12  --unstealth, -u           Show assertions even if stealth is used
13  --help|-h                 Show help

Note globbing is supported:

1brittle --coverage path/to/test/*.js

Auto generate a single file containing "all tests":

1brittle -r test/all.js test/*.js
2
3node test/all.js

You can use an environment variable to also set flags:

1BRITTLE="--coverage --bail" brittle test.js

Force disable coverage with an environment variable:

1BRITTLE_COVERAGE=false brittle test.js

Coverage

If the --coverage flag is set, brittle will output the coverage summary as a table at the end of execution and generate a json coverage report in the coverage output directory (configurable using --cov-dir).

The coverage output directory will contain a coverage-final.json file which contains an istanbul json coverage report and a v8-coverage.json file which contains the raw v8 coverage data.

Istanbul can be used to convert the istanbul json report into other formats. e.g.:

npx istanbul report html

License

Apache-2.0