gulp-dependents

Gulp plugin that tracks dependencies between files and adds any files that depend on the files currently in the stream, thus enabling incremental build of pcss, less, scss and sass files, with extensibility points to support other file types.

Problem

Gulp makes it easy to build all your files, but as the code base grows, so does the build time, significantly slowing down your workflow. The solution is incremental building - i.e. to rebuild only the files that have actually changed. Unfortunately Gulp is agnostic about the depenencies between your files, making it hard to incrementally build files that depend on other files - it doesn't know, that when a dependency changes, so does the files that depend on it.

Solution

This plugin tracks the dependencies of all the files that pass trough it, building and maintaining an in-memory dependency tree describing the dependencies between the files. For each file that passes through, it will add any files that directly or indirectly depend on that file to the stream, thus ensuring that they will also be rebuild. Combined with e.g. the gulp-cached plugin, or the "since last run" option in Gulp 4, this enables fast and reliable incremental builds.

Usage

This example shows how the plugin may be used to watch and incrementally build less files. The gulp-cached plugin will pass all files through on the first run, thus allowing gulp-dependents to set up the initial dependency graph. On subsequent runs, only changed files will be passed through, and gulp-dependents will then ensure that any dependent files are also pulled into the stream.

1
2var gulp = require('gulp'),
3    less = require('gulp-less'),
4    cached = require('gulp-cached'),
5    dependents = require('gulp-dependents');
6
7gulp.task('watch', function() {
8    gulp.watch('src/**/*.less', ['build']);
9});
10
11gulp.task('build', function() {
12    return gulp
13        .src('src/**/*.less')
14        .pipe(cached('less'))
15        .pipe(dependents())
16        .pipe(less())
17        .pipe(gulp.dest('dist'))
18});
19

Please note that although they serve a similar pourpose, gulp-cached and gulp-changed have different behavior - gulp-changed will not nessesarily pass all files through on first run. Instead, it compares the timestamps of the source and destination files, and only pass through those that appear to be different. This means, that if you want to use gulp-changed instead of gulp-cached, you must clean your output folder every time your watch task starts, as this plugin needs to process all files at least once, in order to determine the initial dependency tree - it won't know a file depends on another, until it has parsed its dependency statements at least once.

Support and limitations

Out of the box, this plugin supports pcss, less, scss and sass files, including things like comma-separated path lists, import statements spanning multiple lines and url(...) paths. For sass, which is the indent-based variant of the scss syntax, support is limited to single-line import statements. Also note, that due to the way tracking is implemented, it is currently not possible to support dependency statements with glob patterns, referencing e.g. all files in a folder.

Configuration

For the file types supported out of the box, there's generally no need to configure anything, but should the need arise, a parser configuration may be passed to the plugin function. Note that the options are merged into the default configuration, so if you only wish to override e.g. the basePaths option for scss files, then simply specify only that property.

The parser will apply each RegExp or function in the parserSteps array in sequence, such that the first receives all the file content and may e.g. extract whole dependency statements, and the second one may then extract the paths from those statements. This design enables parsing of complex statements that e.g. list multiple, comma-separated file paths. It also enables the use of external parsers, by specifying a function, which simply invokes the external parser to get the dependency paths.

1
2// The parser configuration, in which keys represents file name
3// extensions, including the dot, and values represent the config
4// to use when parsing the file type.
5var config = {
6
7    ".scss": {
8
9        // The sequence of RegExps and/or functions to use when parsing
10        // dependency paths from a source file. Each RegExp must have the
11        // 'gm' modifier and at least one capture group. Each function must
12        // accept a string and return an array of captured strings. The
13        // strings captured by each RegExp or function will be passed
14        // to the next, thus iteratively reducing the file content to an
15        // array of dependency file paths.
16        parserSteps: [
17
18            // PLEASE NOTE:
19            // The parser steps shown here are only meant as an example to
20			// illustrate the concept of the matching pipeline.
21            // The default config used for scss files is pure RegExp and
22            // reliably supports the full syntax of scss import statements.
23
24            // Match the import statements and capture the text
25            // between '@import' and ';'.
26            /^\s*@import\s+(.+?);/gm,
27
28            // Split the captured text on ',' to get each path.
29            function (text) { return text.split(","); },
30
31            // Match the balanced quotes and capture only the file path.
32            /"([^"]+)"|'([^']+)'/m
33        ],
34
35        // The file name prefixes to try when looking for dependency
36        // files, if the syntax does not require them to be specified in
37        // dependency statements. This could be e.g. '_', which is often
38        // used as a naming convention for mixin files.
39        prefixes: ['_'],
40
41        // The file name postfixes to try when looking for dependency
42        // files, if the syntax does not require them to be specified in
43        // dependency statements. This could be e.g. file name extensions.
44        postfixes: ['.scss', '.sass'],
45
46        // The additional base paths to try when looking for dependency
47        // files referenced using relative paths.
48        basePaths: [],
49    }
50};
51
52// Pass the config object to the plugin function.
53.pipe(dependents(config))
54
55// You can also pass a second config argument to enable logging.
56.pipe(dependents(config, { logDependents: true }))
57

Enjoy, and please report any issues in the issue tracker :-)