⚠️ This is an internal package; you don't need to install it in order to use the JUnit XML Formatter.

JUnit XML Formatter

Writes Cucumber message into a JUnit XML report.

The JUnit XML report is a de facto standard without an official specification. But we validate it against the Jenkins JUnit XML XSD so there should be a good chance your CI will understand it.

If not, please let us know in the issues!

Features and Limitations

Test outcome mapping

Cucumber and the JUnit XML Report support a different set of test outcomes. These are mapped according to the table below.

Additionally, it is advisable to run Cucumber in strict mode. When used in non-strict mode scenarios with a pending or undefined outcome will not fail the test run (#714). This can lead to a xml report that contains failure outcomes while the build passes.

Cucumber Outcome	XML Outcome	Passes in strict mode	Passes in non-strict mode
UNKNOWN	n/a	n/a	n/a
PASSED	passed	yes	yes
SKIPPED	skipped	yes	yes
PENDING	failure	no	yes
UNDEFINED	failure	no	yes
AMBIGUOUS	failure	no	no
FAILED	failure	no	no

Step reporting

The JUnit XML report assumes that a test is a method on a class. Yet a scenario consists of multiple steps. To provide info about which step failed, the system-out element will contain a rendition of steps and their result.

1<system-out><![CDATA[
2Given there are 12 cucumbers................................................passed
3When I eat 5 cucumbers......................................................passed
4Then I should have 7 cucumbers..............................................passed
5]]></system-out>

Naming Rules and Examples

Cucumber does not require that scenario names are unique. To disambiguate between similarly named scenarios and examples the report prefixes the rule to the scenario or example name.

1Feature: Rules
2
3  Rule: a sale cannot happen if change cannot be returned
4    Example: no change
5      ...
6    Example: exact change
7      ...
8
9  Rule: a sale cannot happen if we're out of stock
10    Example: no chocolates left
11      ...

1<testcase classname="Rules" name="a sale cannot happen if change cannot be returned - no change" time="0.007" />
2<testcase classname="Rules" name="a sale cannot happen if change cannot be returned - exact change" time="0.009" />
3<testcase classname="Rules" name="a sale cannot happen if we're out of stock - no chocolates left" time="0.009" />

Likewise for example tables, the rule (if any), scenario outline name, example name, and number are included. Additionally, if the scenario outline name is parameterized, the pickle name is included too.

1Feature: Examples Tables
2
3  Scenario Outline: Eating cucumbers
4    Given there are <start> cucumbers
5    When I eat <eat> cucumbers
6    Then I should have <left> cucumbers
7
8    Examples: These are passing
9      | start | eat | left |
10      |    12 |   5 |    7 |
11      |    20 |   5 |   15 |
12
13    Examples: These are failing
14      | start | eat | left |
15      |    12 |  20 |    0 |
16      |     0 |   1 |    0 |
17
18  Scenario Outline: Eating <color> cucumbers
19    Given I am transparent
20    When I eat <color> cucumbers
21    Then I become <color>
22
23    Examples:
24      | color | 
25      |   red | 
26      | green | 
27      |  blue | 

1<testcase classname="Examples Tables" name="Eating cucumbers - These are passing - #1.1" />
2<testcase classname="Examples Tables" name="Eating cucumbers - These are passing - #1.2" />
3<testcase classname="Examples Tables" name="Eating cucumbers - These are failing - #2.1" />
4<testcase classname="Examples Tables" name="Eating cucumbers - These are failing - #2.2" />
5<testcase classname="Examples Tables" name="Eating &lt;color&gt; cucumbers - #1.1: Eating red cucumbers" />
6<testcase classname="Examples Tables" name="Eating &lt;color&gt; cucumbers - #1.2: Eating green cucumbers" />
7<testcase classname="Examples Tables" name="Eating &lt;color&gt; cucumbers - #1.3: Eating blue cucumbers" />

Android

The Android SDK does not include javax.xml.stream.XMLOutputFactory. This and other missing java.xml.stream.* dependencies can be provided by adding com.fasterxml:aalto-xml to your dependencies.

Contributing

Each language implementation validates itself against the examples in the testdata folder. See the testdata/README.md for more information.