wdio-multiple-cucumber-html-reporter

A WebdriverIO reporter plugin. Reporter that creates beautiful Cucumber HTML reports by using multiple-cucumber-html-reporter

THIS MODULE IS DEPRECATED AND IS NOT MAINTAINED ANYMORE. IT DOESN'T SUPPORT CUCUMBERJS WITH WEBDRIVERIO V5 AND IT CAN'T BE MIGRATED TO V5 DUE TO TECHNICAL LIMITATIONS IF YOU WANT TO USE CUCUMBERJS WITH WEBDRIVERIO V5 AND A SIMILAR IMPLEMENTATION LIKE THIS, PLEASE CHECK wdio-cucumberjs-json-reporter

THIS REPORTER IS STILL UNDER DEVELOPMENT, FEEL FREE TO ADD PR'S

THIS REPORTER NEEDS TO USE wdio-cucumber-framework 2.2.5 OR HIGHER, SEE FAQ'S

multiple-cucumber-html-reporter, WHICH IS USED IN THIS REPORTER PLUGIN WILL ONLY WORK ON NODE >= 7

THIS MODULE CAN ONLY WORK WITH WebdriverIO V4 BECAUSE wdio-cucumber-framework IS NOT V5 COMPATIBLE

What does it do?

This reporter does two things

1. First of all it creates a CucumberJS-JSON file. This is needed because the BaseReporter of webdriver.io overwrites the CucumberJS report output
2. Based on the created JSON it creates a HTML-report like below

3. It also supports Given|When|Then-keywords and Datatables

4. Data can also be added to steps (so not before|after-hooks) with the following code

1import multipleCucumberHtmlReporter from 'wdio-multiple-cucumber-html-reporter';
2
3// Attach a string
4multipleCucumberHtmlReporter.attach('just a string');
5// Attach JSON
6multipleCucumberHtmlReporter.attach({"json-string": true}, 'application/json');
7// Attach a screenshot
8multipleCucumberHtmlReporter.attach(browser.saveScreenshot(), 'image/png');
9// Or with
10multipleCucumberHtmlReporter.attach(browser.screenshot(), 'image/png');

5. Data can also be added to before|after-hooks. You can use the same method as above but you need to add an extra hookType, which is a string. For the before-hook you use the string before and for the after-hook you can use the string after. See the examples below

Keep in mind this is only for the before|after-hooks, if you add them in steps you will mess up your own report

1import multipleCucumberHtmlReporter from 'wdio-multiple-cucumber-html-reporter';
2
3// BEFORE
4// Attach a string in a before hook
5multipleCucumberHtmlReporter.attach('just a string', 'before');
6// Attach JSON in a before hook
7multipleCucumberHtmlReporter.attach({"json-string": true}, 'application/json', 'before');
8// Attach a screenshot in a before hook
9multipleCucumberHtmlReporter.attach(browser.saveScreenshot(), 'image/png', 'before');
10// Or with
11multipleCucumberHtmlReporter.attach(browser.screenshot(), 'image/png', 'before');
12
13
14// AFTER
15// Attach a string in a after hook
16multipleCucumberHtmlReporter.attach('just a string', 'after');
17// Attach JSON in a after hook
18multipleCucumberHtmlReporter.attach({"json-string": true}, 'application/json', 'after');
19// Attach a screenshot in a after hook
20multipleCucumberHtmlReporter.attach(browser.saveScreenshot(), 'image/png', 'after');
21// Or with
22multipleCucumberHtmlReporter.attach(browser.screenshot(), 'image/png', 'after');

Keep in mind that this will add a passed step for the before|after-hook as can be seen here

Not all options / data that is provided in multiple-cucumber-html-reporter can be used due to limitations in the generated JSON file by this reporter

Installation

The easiest way is to keep wdio-multiple-cucumber-html-reporter as a devDependency in your package.json.

1{
2  "devDependencies": {
3    "wdio-multiple-cucumber-html-reporter": "^0.2.0"
4  }
5}

You can simple do it by:

1npm install wdio-multiple-cucumber-html-reporter --save-dev

so it will be added automatically to your package.json

Instructions on how to install WebdriverIO can be found here.

Configuration

Configure the output directory in your wdio.conf.js file:

1exports.config = {
2    // ...
3    reporters: ['multiple-cucumber-html'],
4    reporterOptions: {
5        htmlReporter: {
6            jsonFolder: './tmp',
7            reportFolder: `./tmp/report`,
8            // ... other options, see Options
9        }
10    },
11  // ...
12}

Options

jsonFolder

• Type: String
• Mandatory: Yes

The directory where the JSON file, generated by this report, will be stored, relative from where the script is started.

N.B.: If you use a npm script from the command line, like for example npm run test the jsonFolder will be relative from the path where the script is executed. Executing it from the root of your project will also create the jsonFolder in the root of you project.

reportFolder

• Type: String
• Mandatory: Yes

The directory in which the report needs to be saved, relative from where the script is started.

N.B.: If you use a npm script from the command line, like for example npm run test the reportFolder will be relative from the path where the script is executed. Executing it from the root of your project will also save the report in the reportFolder in the root of you project.

removeFolders

• Type: boolean
• Mandatory: No

If true the the jsonFolder and the reportFolder will be removed to start the test with a clean state.

openReportInBrowser

• Type: boolean
• Mandatory: No

If true the report will automatically be opened in the default browser of the operating system.

saveCollectedJSON

• Type: boolean
• Mandatory: No

This reporter will first get the JSON-file and then enrich it with data that is used for the report. If saveCollectedJSON :true the JSON AND the enriched JSON will be saved in the reportFolder. They will be saved as:

• merged-output.json
• enriched-output.json

disableLog

• Type: boolean
• Mandatory: No
• Default: false

This will disable the log so will NOT see this.

1=====================================================================================
2    Multiple Cucumber HTML report generated in:
3
4    /Users/wswebcreation/multiple-cucumber-html-reporter/.tmp/index.html
5========================================================================

pageTitle

• Type: string
• Mandatory: No
• Default: Multiple Cucumber HTML Reporter

You can change the report title in the HTML head Tag

reportName

• Type: string
• Mandatory: No

You can change the report name to a name you want

pageFooter

• Type: string
• Mandatory: No

You can customise Page Footer if required. You just need to provide a html string like <div><p>A custom footer in html</p></div>

displayDuration

• Type: boolean
• Mandatory: No

If set to true the duration of steps, scenarios and features is displayed on the Features overview and single feature page in an easily readable format. This expects the durations in the report to be in nanoseconds, which might result in incorrect durations when using a version of Cucumber(JS 2 and 3) that does not report in nanoseconds but in milliseconds. This can be changed to milliseconds by adding the parameter durationInMS: true, see below

NOTE: Only the duration of a feature can be shown in the features overview. A total duration over all features CAN NOT be given because the module doesn't know if all features have been run in parallel

durationInMS

• Type: boolean
• Default: false
• Mandatory: No

If set to true the duration of steps will be expected to be in milliseconds, which might result in incorrect durations when using a version of Cucumber(JS 1 or 4) that does report in nanaseconds. This parameter relies on displayDuration: true

customStyle

• Type: path
• Mandatory: No

If you need add some custom style to your report. Add it like this customStyle: 'your-path-where/custom.css'

overrideStyle

• Type: path
• Mandatory: No

If you need replace default style for your report. Add it like this overrideStyle: 'your-path-where/custom.css'

customData

• Type: object
• Mandatory: No

You can add a custom data block to the report like this

1customData: {
2    title: 'Run info',
3    data: [
4        {label: 'Project', value: 'Custom project'},
5        {label: 'Release', value: '1.2.3'},
6        {label: 'Cycle', value: 'B11221.34321'},
7        {label: 'Execution Start Time', value: 'Nov 19th 2017, 02:31 PM EST'},
8        {label: 'Execution End Time', value: 'Nov 19th 2017, 02:56 PM EST'}
9    ]
10}

customData.title

• Type: string
• Mandatory: No
• Default: Custom data title

Select a title for the custom data block. If not provided it will be defaulted.

customData.data

• Type: array
• Mandatory: yes

The data you want to add. This needs to be in the format

1data: [
2    {label: 'your label', value: 'the represented value'}
3]

Metadata

The report can also show on which browser / device a feature has been executed. It is shown on the features overview in the table, like this as well as on the feature overview in the container, like this

You can add this by adding the following object to your capabilities;

1exports.config = {
2    //..
3    capabilities: [
4        {
5            browserName: 'chrome',
6            // Add this
7            metadata: {
8                browser: {
9                    name: 'chrome',
10                    version: '58'
11                },
12                device: 'MacBook Pro 15',
13                platform: {
14                    name: 'OSX',
15                    version: '10.12.6'
16                }
17            },
18        },
19    ],
20};

See the metadata information here for the correct values.

If you don't provide the browser-object in the metadata, this module will automatically determine it for you

If you don't provide the device and or the platform-object it will be defaulted for you to not known

If you don't provide a browser.name or a browser.version the module will try to determine this automatically. The rest will be shown as questionmarks in the report

FAQ's

How do I add screenshots to the report

Just create a After-hook in a stepfile like this

1const {After, Status} = require('cucumber');
2import multipleCucumberHtmlReporter from 'wdio-multiple-cucumber-html-reporter';
3
4After((scenarioResult)=>{
5    if (scenarioResult.result.status === Status.FAILED) {
6        // It will add the screenshot to the JSON
7        multipleCucumberHtmlReporter.attach(browser.saveScreenshot(), 'image/png', 'after');
8        // Or with
9        multipleCucumberHtmlReporter.attach(browser.screenshot(), 'image/png', 'after');
10    }
11    return scenarioResult.status;
12});

I don't see the keywords Given, When, Then in the report

The module wdio-cucumber-framework only provides this information from version 2.2.0 and higher. Please upgrade to that version.

skipped steps are marked as pending

skipped steps are currently marked as pending because wdio-cucumber-framework can't distinguish them, there is a PR for this, see here

ambiguous steps are marked as pending

CucumberJS has a status called ambiguous, this should also be shown in the report. Because wdio-cucumber-framework has it's own implementation to handle ambiguous steps , see here it will not show the correct status in the report using this module.

undefined steps are not marked as undefined

CucumberJS has a status called undefined, this should also be shown in the report. Because wdio-cucumber-framework has it's own implementation to handle undefined steps , see here it will not show the correct status in the report using this module.

Changelog

A changelog can be found here.

Contributing

How to contribute can be found here.

TODO:

high priority

Needs to be in the first beta

• Add Before-step to the json, see the remarks about the Before steps
• Add After-step to the json, see the remarks about the After steps
• Add browser data to the report, first start with the default capabilities
• Add screenshots to the report
• Add multiple screenshots to the report in 1 step
• Test in multiple browsers in parallel
• Check / add Passed status
• Check / add Failed status
• Check / add Pending status
• Check / add Ambiguous status
• Check / add Skipped status
• Check / add undefined status

low priority

Needs to be in, but are not mandatory

• Investigate Hooks and if they can influence the outcome
• Write UT's
• Test on Windows
• Test on Android
• Test on iOS

research

• Find out where the keywords are, there is no Given, When, Then comming back from webdriver.io => => Created a PR for wdio-cucumber-framework 136 that has been merged and released as version 2.2.0
• Add data tables to the report, see data tables => Created a PR for wdio-cucumber-framework 136 that has been merged and released as version 2.2.0
• Find out where the description of the feature is. Can't find it in the methods and it will not be added to the report then

Some remarks

Before hooks

Before hooks are not added to the WebdriverIO reporter. There is no way in telling they passed / failed.

Pass

Not logged in wdio-cucumberjs-framework => not in this module

Failed

Automatically logged by my implementation, not all data is logged like screenshots and so on

Pending

Pending state will result in the following:

• Beforehook will not get the status, there is only a start, not a pass/failed/pending
• All Scenario steps will get status pending Meaning I can't log this

After hooks

After hooks are not added to the WebdriverIO reporter. There is no way in telling they passed / failed

Pass

Not logged in wdio-cucumberjs-framework => not in this module

Failed

Automatically logged by my implementation, not all data is logged like screenshots and so on

Pending

Status pending of the After hook has no effect on the status of the report / wdio-cucumber-framework will not report this status