Node.js Cucumber Reporting

This is a Node.js report publisher primarily created to publish cucumber reports on the Jenkins build server. It publishes pretty html reports with charts showing the results of cucumber runs. It has been split out into a standalone package so it can be used for Jenkins and Node.js command line as well as any other packaging that might be useful. Generated report has no dependency so can be viewed offline. This package is inspired from the Maven Cucumber Reporting Project.

Background

Cucumber is a test automation tool following the principles of Behavioural Driven Design and living documentation. Specifications are written in a concise human readable form and executed in continuous integration.

This project allows you to publish the results of a cucumber run as pretty html reports. In order for this to work you must generate a cucumber json report. The project converts the json report into an overview html linking to separate feature files with stats and results.

Install

Install NPM dependency

1npm i @som30ind/cucumber-reporting --save

Usage

1/**
2 * Supports CJS require or ES6 Import.
3 * const { ReportBuilder } = require('@som30ind/cucumber-reporting');
4 * 
5 * ES6 / TypeScript Import Statement is shown below
6 */
7import { ReportBuilder } from '@som30ind/cucumber-reporting';
8
9/**
10 * @param jsonFiles {string[]} (Mandetory) Array of Cucumber JSON Files.
11 * It can be absolute or relative (from Project Root) path to the file.
12 * Supports Glob Expression (https://mincong.io/2019/04/16/glob-expression-understanding/)
13 * @param options {UserConfiguration} (Optional) Details for each option given below in the example.
14 * All the configurations are also optional.
15 * 
16 * @example
17 */
18const reportBuilder = new ReportBuilder([
19  './cucumber-reports/sample.json',
20  './cucumber-reports/simple.json',
21  './cucumber-reports/timestamped/**/*.json',
22], {
23  /**
24   * Absolute or Relative (from Project Root) path to the directory where you want the reports to be generated.
25   * 
26   * @param reportDir {string}
27   */
28  reportDir: 'cucumber-report-html',
29  /**
30   * Name of the Project to be dislayed on the Report.
31   * 
32   * @param projectName {string}
33   */
34  projectName: `Somnath's Project`,
35  /**
36   * Build Number to be displayed on the report.
37   * 
38   * @param buildNumber {string}
39   */
40  buildNumber: '1',
41  /**
42   * These options are use to undertsand how the reports will be presented. Below are the options
43   * 
44   * RUN_WITH_JENKINS: Defines additional menu buttons that enables integration with Jenkins.
45   * EXPAND_ALL_STEPS: Expands all scenarios by default.
46   * PARALLEL_TESTING: Add "target" column to the report, when running the same tests many times.
47   * Value of this column is same as JSON report file name.
48   * 
49   * @param presentationModes {string}
50   */
51  presentationModes: [
52    'PARALLEL_TESTING',
53  ],
54  /**
55   * Not failing statusses
56   * Any valid Status from: SKIPPED | AMBIGUOUS | FAILED | PASSED | PENDING | UNDEFINED
57   * 
58   * @param notFailingStatuses {Status[]}
59   */
60  notFailingStatuses: [
61    'SKIPPED',
62  ],
63  /**
64   * Metadata that will be displayed at the main page of the report. It is useful when there is a few reports are
65   * generated at the same time but with different parameters/configurations.
66   * 
67   * @param classifications {Record<string, string>} Key-Value-Pair of string.
68   */
69  classifications: {
70    'Platform': 'Windows',
71    'Browser': 'Firefox',
72    'Branch': 'release/1.0',
73  },
74  /**
75   * JSON files which house classifications in key value pairings. When these JSON files get
76   * processed these classifications get displayed on the main page of the report as metadata in the order in which
77   * they appear within the file.
78   * It can be absolute or relative (from Project Root) path to the file.
79   * Supports Glob Expression (https://mincong.io/2019/04/16/glob-expression-understanding/)
80   * 
81   * @param classificationFiles {string[]}
82   */
83  classificationFiles: [
84    './resources/classifications/sample_one.json',
85    './resources/classifications/sample_two.json'
86  ],
87  /**
88   * Explicit qualifier to use for the given json file.
89   * Key: JSON file name - without the extension
90   * Value: Qualifier to use
91   * 
92   * @param qualifiers {Record<string, string>}
93   */
94  qualifiers: {
95    'cucumber-report-1': 'First report',
96    'cucumber-report-2': 'Second report'
97  },
98  /**
99   * This list contains supported methods that allow to modify the way how reports are displayed.
100   * 
101   * MERGE_FEATURES_BY_ID: Merge features with different JSON files that have same ID so scenarios are be stored in single feature.
102   * MERGE_FEATURES_WITH_RETEST: Merge features and scenarios from different JSON files of different runs
103   * into a single report by features' and scenarios' ids.
104   *
105   * Merging rules:
106   * - Every new feature which is not in the result list is appended to the end.
107   *
108   * - When the results list already has a feature with such Id then we go down and apply the rules below to the scenarios:
109   *
110   *      1. if there is no scenario with a given Id in the feature's elements list
111   *         then add the scenario to the end of the list.
112   *
113   *      2. if there are no scenario with a background (which is a previous element in the elements list)
114   *        then both elements are added to the end of the current feature's elements list.
115   *        As the feature file has a structure like:
116   *        {
117   *            elements: [
118   *              {
119   *                  name: ...
120   *                  type: "background";
121   *              },
122   *              {
123   *                  name: ...
124   *                  type: "scenario";
125   *              },
126   *              {
127   *                  name: ...
128   *                  type: "background";
129   *              },
130   *              {
131   *                  name: ...
132   *                  type: "scenario";
133   *              }
134   *              ....
135   *            ]
136   *        }
137   *
138   *      3. if there is a scenario with a given Id then:
139   *          scenario + background case: replace both elements (existing element with Id and its background with new ones)
140   *          scenario only: replace only given scenario by index in the array.
141   *
142   * Example:
143   * Original cucumber report is "cucumber.json". Let's look a situation when couple of tests failed there.
144   * Cucumber runner generates a new report, for example, cucumber-rerun.json as a result of rerun the failed tests.
145   *
146   * In that case you will have a merged report where all failed tests from the original cucumber.json file
147   * are overridden with the results from the cucumber-rerun.json.
148   * SKIP_EMPTY_JSON_FILES: Skip empty JSON reports. If this flag is not selected then report generation fails on empty file.
149   * HIDE_EMPTY_HOOKS: Does not display hooks (@Before and @After) which do not have attachment or error message.
150   *
151   * @param reducingMethods {ReducingMethod[]}
152   */
153  reducingMethods: [
154    'HIDE_EMPTY_HOOKS',
155    'MERGE_FEATURES_BY_ID',
156    'MERGE_FEATURES_WITH_RETEST',
157    'SKIP_EMPTY_JSON_FILES'
158  ],
159  /**
160   * Configure how items will be sorted in the report by default.
161   *
162   * NATURAL: Same order as in JSON file.
163   * ALPHABETICAL: Order by name of the element.
164   *
165   * @param sortingMethod {SortingMethod}
166   */
167  sortingMethod: 'ALPHABETICAL',
168  resources: {
169    /**
170     * Custom static css files to each html page.
171     * It can be absolute or relative (from Project Root) path to the file.
172     *
173     * @param customCssFiles {string[]}
174     */
175    customCssFiles: [],
176    /**
177     * Custom static js files to each html page.
178     * It can be absolute or relative (from Project Root) path to the file.
179     *
180     * @param customJsFiles {string[]}
181     */
182    customJsFiles: []
183  },
184  /**
185   * File with trends
186   *
187   * @param trendsFile {string}
188   */
189  trendsFile: './trends/trends.json',
190});
191
192/**
193 * result will be undefined if Report generation failed. Else will have Reportable instance.
194 */
195const result = reportBuilder.generateReports();
196

There is a feature overview page:

And there are also feature specific results pages:

And useful information for failures:

If you have tags in your cucumber features you can see a tag overview:

And you can drill down into tag specific reports:

Live demo

You can play with the live demo report before you decide if this is worth to use.

License

Licensed under MIT

Copyright (c) 2023 Somnath Sinha