find-cypress-specs

Find Cypress spec files using the config settings

1$ npx find-cypress-specs
2# prints all spec files separated by a comma
3cypress/e2e/spec.js,cypress/e2e/featureA/user.js

Supports JS and TS specs

Count

You can count the number of specs found

1$ npx find-cypress-specs --count
22

Component specs

By default, it finds the E2E specs and tests. You can find component specs using --component CLI option

$ npx find-cypress-specs --component

against branch

By default, this module simply prints all spec filenames. You can add --branch parameter to only print the specs changed against that origin/branch.

1$ npx find-cypress-specs --branch main
2# prints only some specs, the ones that have changed against the "origin/main"

find specs by part of the test title

You can grep all test titles (only the test titles, not the suite names) and find the list of specs

1$ npx find-cypress-specs --grep "works, needs"

This will find all specs with tests that have "works" in the title or "needs" in the title and output the list of specs. If you use --set-gha-outputs and GitHub Actions, it sets the outputs grepSpecs and grepSpecsN.

find number of machines

If we find all the changed specs to run, we might need to decide how many machines we need. We can do a rough job by specifying the number of specs per machine plus the max number.

$ npx find-cypress-specs --branch main --specs-per-machine 4 --max-machines 5

For now, it is only useful when setting the GHA outputs.

set GitHub Actions outputs

If you add --set-gha-outputs command line switch, then the number of changed specs and the comma-separated file list will be set as GH Actions outputs changedSpecsN and changedSpecs. See pr.yml for example

1- name: Print specs changed against the parent of this branch 🌳
2  # and set GitHub Actions output
3  id: step1
4  run: node ./bin/find --branch main --parent --set-gha-outputs
5
6- name: Print set outputs
7  run: echo ${{ steps.step1.outputs.changedSpecsN }} ${{ steps.step1.outputs.changedSpecs }}

If you set the number of machines, it will set the output machinesNeeded

Write GitHub Actions job summary

You can output changes specs by using the parameter --gha-summary

against the parent commit

When dealing with a long-term branch, you do not want to see the changed files in the main branch. Instead, you want to only consider the specs changed in the current branch all the way to its parent commit. You can pass the flag --parent to only pick the modified and added specs.

1$ npx find-cypress-specs --branch main --parent
2# same as
3# git diff --name-only --diff-filter=AMR $(git merge-base origin/main HEAD)..

Note: to get the changed files, we need to fetch the repo, see pr.yml

$ checkout
$ git fetch
$ npx find-cypress-specs --branch main --parent

with traced import and require statements

Imagine you open a pull request and only change something in an utils.js file used by other specs. Which specs should you run? By default --branch main --parent would not find any changed specs, so no specs would execute, and you will have to run all specs just to be safe. This program has a mode --trace-imports <subfolder> which uses spec-change to inspect JS/TS files and find dependencies between them. Thus it can discover that when utils.js changes, the specs that import './utils' or require('./utils') should also be considered modified.

$ npx find-cypress-specs --branch main --parent --trace-imports cypress

Note: the argument is the subfolder name to limit the number of files to inspect when tracing the imports.

You can time how long tracing takes by adding option --time-trace to the command line arguments. You can also saved traced dependencies using --cache-trace argument. Next time the dependencies will be loaded from the file without recomputing. This is convenient on CI to avoid recomputing them. For example, if you need the number of affected files and their filenames

# get the number of affected specs
$ npx find-cypress-specs --branch main --parent --trace-imports cypress --cache-trace --count
# quickly get the affected specs without recomputing the dependencies
$ npx find-cypress-specs --branch main --parent --trace-imports cypress --cache-trace

The cached trace will be saved in file deps.json, you probably want to Git ignore it.

You can limit the number of added traced files using the --max-added-traced-specs <N> parameter. This avoids ALL specs added when you change some common utility that many specs import.

number of changed files

You can print just the number of changed specs

1$ npx find-cypress-specs --branch main --count
2# prints the number of spec files changed against the branch "origin/main"
35

filter by a tag

You can filter all changed specs and only report (and count) the specs that have changed AND include the given tag(s)

1$ npx find-cypress-specs --branch main --tagged @user,@preview
2# prints only some specs, the ones that have changed against the "origin/main"
3# and that have any tests or suites inside tagged "@user" or "@preview"

You can set the list and number of specs with tags into GHA outputs

1$ npx find-cypress-specs --branch main --tagged @user,@preview --set-gha-outputs

The number of found specs will be set as taggedSpecsN and the list will be set as taggedSpecs.

You can find tests / suites tagged with several tags together using AND syntax tag1+tag2. For example, to find all @user tests also tagged @smoke

1$ npx find-cypress-specs --names --tagged @user+@smoke

count tagged changed specs

Let's say we changed 2 specs that have tests tagged @user. We can output the count by adding --count option

1$ npx find-cypress-specs --branch main --tagged @user --count
22

Specs sorted by the Git modified timestamp

You can find all specs and output their list sorted by the last modified Git timestamp. The latest modified specs will be listed first. This feature can help you run the latest specs first.

1$ npx find-cypress-specs --sort-by-modified

You can set the spec filenames as GHA outputs

1# sets GitHub Actions outputs "changedSpecsN" and "changedSpecs"
2$ npx find-cypress-specs --sort-by-modified --set-gha-outputs

Important: to get the right commit dates, Git needs the full repository history. For example, when using the GHA to check out the repo, fetch the entire history:

1- name: Checkout 🛎
2  uses: actions/checkout@v4
3  with:
4    # fetch all history to get the full commit history
5    # this is needed to get the correct order of specs
6    # by modified Git timestamp
7    fetch-depth: 0

📝 Read the blog post Run Changed Cypress Specs On CI First.

Test names

You can print each spec file with the suite and test names inside of it (found using find-test-names)

1$ npx find-cypress-specs --names
2# prints something like
3
4cypress/e2e/spec.js (2 tests)
5└─ parent suite [@main]
6  ├─ works well enough
7  └─ inner suite
8    └─ shows something [@user]
9
10cypress/e2e/featureA/user.js (2 tests, 1 pending)
11├─ works
12└⊙ needs to be written
13
14found 2 specs (4 tests, 1 pending)

Where the tags are listed inside [ ... ] (see @bahmutov/cy-grep) and the pending tests are marked with ⊙ character.

Required tags are marked with [[ ... ]].

You can print the results in JSON format using --json or -j option.

You can print the tests in Markdown format using --markdown or --md option

Spec
foo/a/spec.cy.js (3 tests)
suite / test 1
suite / test 2
suite / test 3
foo/b/spec.cy.js (1 test)
another / test / test 1

Test tags

You can count tags attached to the individual tests using --tags arguments

$ npx find-cypress-specs --tags
# prints the tags table sorted by tag

Tag    Tests
-----  ----------
@sign  1
@user  2

Each tag count includes the tests that use the tag directly, and the effective tags applied from the parent suites, both tags and requiredTags.

You can print the results in JSON format using --json or -j option.

Test names filtered by a tag

1$ npx find-cypress-specs --names --tagged <single tag>
2# finds all specs and tests, then filters the output by a single tag

Test names filtered by multiple tags

1$ npx find-cypress-specs --names --tagged <tag1>,<tag2>,<tag3>,...
2# finds all specs and tests, then filters the output showing all tests
3# tagged with tag1 or tag2 or tag3 or ...

File names filtered by a tag

$ npx find-cypress-specs --tagged <single tag>
# finds all specs and tests, then filters the output showing only the file names associated with the tests


# cypress/e2e/spec.cy.js,cypress/e2e/featureA/user.cy.ts

If you pass an empty string argument like --tagged '', an empty list is returned.

You can print the number of found tagged specs by adding --count argument

$ npx find-cypress-specs --tagged <single tag> --count
3

File names filtered by multiple tags

$ npx find-cypress-specs --tagged <tag1>,<tag2>,<tag3>,...
# finds all specs and tests, then filters the output showing only the file names associated with the tests
# tagged with tag1 or tag2 or tag3 or ...

Show only the pending tests

You can show only the tests marked with "it.skip" which are called "pending" according to Mocha / Cypress terminology.

1$ npx find-cypress-specs --names --pending
2# --skipped is an alias to --pending
3$ npx find-cypress-specs --names --skipped
4# prints and counts only the pending tests
5cypress/e2e/featureA/user.cy.ts (1 test, 1 pending)
6└⊙ needs to be written [@alpha]
7
8found 1 spec (1 test, 1 pending)

Print skipped tests

$ npx find-cypress-specs --names --skipped

Prints each spec that has skipped tests.

Count tests

You can see the total number of E2E and component tests

$ npx find-cypress-specs --test-counts
4 e2e tests, 2 component tests

Update README badge

You can set or update a badge in README with test counts by adding the --update-badge argument

$ npx find-cypress-specs --test-counts --update-badge
4 e2e tests, 2 component tests
⚠️ Could not find test count badge
Insert new badge on the first line
saving updated readme with new test counts

See the badges.yml workflow

Count skipped tests

Prints the single number with the count of skipped tests

$ npx find-cypress-specs --names --skipped --count

5

cypress.config.ts

If the project uses TypeScript and cypress.config.ts then this module uses tsx to load the config and fetch the spec pattern.

If you are using import keyword in your cypress.config.ts you might get an error like this:

import { defineConfig } from 'cypress';
^^^^^^

SyntaxError: Cannot use import statement outside a module

In that case, add to your tsconfig.json file the ts-node block:

1{
2  "ts-node": {
3    "compilerOptions": {
4      "module": "commonjs"
5    }
6  }
7}

See example in bahmutov/test-todomvc-using-app-actions.

Tip: read my blog post Convert Cypress Specs from JavaScript to TypeScript.

Custom config filename

If you want to use a custom Cypress config, pass it via the environment variable CYPRESS_CONFIG_FILE

$ CYPRESS_CONFIG_FILE=path/to/cypress.config.js npx find-cypres-specs ...

Absolute spec filenames

You can return absolute filenames to the found specs

1getSpecs(config, 'e2e|component', true)

HTML Test Report

You can generate a static self-contained HTML report with all specs, tests, and test tags

1$ npx find-cypress-specs --write-html-filename report/index.html

Open the HTML report in your browser. The report should show the tests organized by spec / suite plus all test tags and allow filtering tests by the checked tags.

See an example report at https://glebbahmutov.com/find-cypress-specs/

Details

Cypress uses the resolved configuration values to find the spec files to run. It searches the integrationFolder for all patterns listed in testFiles and removes any files matching the ignoreTestFiles patterns.

You can see how Cypress finds the specs using DEBUG=cypress:cli,cypress:server:specs environment variable to see verbose logs. The logic should be in the file packages/server/lib/util/specs.ts in the repo cypress-io/cypress

Debugging

Run the utility with environment variable DEBUG=find-cypress-specs to see the verbose logs

Finding tests in the individual specs uses find-test-names so you might want to enable debugging both modules at once:

$ DEBUG=find-cypress-specs,find-test-names npx find-cypress-specs --names

To debug finding changed specs against a branch, use find-cypress-specs:git

$ DEBUG=find-cypress-specs:git npx find-cypress-specs --branch main

Videos

• Use Ava Snapshots And Execa-wrap To Write End-to-End Tests For CLI Utilities

Examples

• 📝 blog post Run Changed Traced Specs On GitHub Actions
• 📝 blog post Quickly Run The Changed Specs on GitHub Actions
• chat.io as described in the blog post Get Faster Feedback From Your Cypress Tests Running On CircleCI

NPM module

You can use this module via its NPM module API.

getSpecs

1const { getSpecs } = require('find-cypress-specs')
2// somewhere in the cypress.config.js
3setupNodeEvents(on, config) {
4  const specs = getSpecs(config)
5  // specs is a list of filenames
6}

You can pass the config object to the getSpecs method. If there is no config parameter, it will read the config file automatically.

1const specs = getSpecs({
2  e2e: {
3    specPattern: '*/e2e/featureA/*.cy.ts',
4  },
5})
6// ['cypress/e2e/featureA/spec.cy.ts']

getTests

Returns an object with individual test information

1const { getTests } = require('find-cypress-specs')
2const { jsonResults, tagTestCounts } = getTests()
3// jsonResults is an object
4// with an entry per spec file

See get-tests.js for details and examples.

Small print

Author: Gleb Bahmutov <gleb.bahmutov@gmail.com> © 2022

• @bahmutov
• glebbahmutov.com
• blog
• videos
• presentations
• cypress.tips
• Cypress Tips&Tricks newsletter
• my Cypress courses

License: MIT - do anything with the code, but don't blame me if it does not work.

Support: if you find any problems with this module, email / tweet / open issue on Github

MIT License

Copyright (c) 2022 Gleb Bahmutov <gleb.bahmutov@gmail.com>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.