@applaud/thread-loader

Runs the following loaders in a worker pool.

Getting Started

1npm install --save-dev @applaud/thread-loader

Put this loader in front of other loaders. The following loaders run in a worker pool.

Loaders running in a worker pool are limited. Examples:

• Loaders cannot emit files.
• Loaders cannot use custom loader API (i. e. by plugins).
• Loaders cannot access the webpack options.

Each worker is a separate node.js process, which has an overhead of ~600ms. There is also an overhead of inter-process communication.

Use this loader only for expensive operations!

Examples

webpack.config.js

1module.exports = {
2  module: {
3    rules: [
4      {
5        test: /\.js$/,
6        include: path.resolve('src'),
7        use: [
8          '@applaud/thread-loader',
9          // your expensive loader (e.g babel-loader)
10        ],
11      },
12    ],
13  },
14};

with options

1use: [
2  {
3    loader: '@applaud/thread-loader',
4    // loaders with equal options will share worker pools
5    options: {
6      // the number of spawned workers, defaults to (number of cpus - 1) or
7      // fallback to 1 when require('os').cpus() is undefined
8      workers: 2,
9
10      // number of jobs a worker processes in parallel
11      // defaults to 20
12      workerParallelJobs: 50,
13
14      // additional node.js arguments
15      workerNodeArgs: ['--max-old-space-size=1024'],
16
17      // Allow to respawn a dead worker pool
18      // respawning slows down the entire compilation
19      // and should be set to false for development
20      poolRespawn: false,
21
22      // timeout for killing the worker processes when idle
23      // defaults to 500 (ms)
24      // can be set to Infinity for watching builds to keep workers alive
25      poolTimeout: 2000,
26
27      // number of jobs the poll distributes to the workers
28      // defaults to 200
29      // decrease of less efficient but more fair distribution
30      poolParallelJobs: 50,
31
32      // name of the pool
33      // can be used to create different pools with elsewise identical options
34      name: 'my-pool',
35    },
36  },
37  // your expensive loader (e.g babel-loader)
38];

prewarming

To prevent the high delay when booting workers it possible to warmup the worker pool.

This boots the max number of workers in the pool and loads specified modules into the node.js module cache.

1const threadLoader = require('@applaud/thread-loader');
2
3threadLoader.warmup(
4  {
5    // pool options, like passed to loader options
6    // must match loader options to boot the correct pool
7  },
8  [
9    // modules to load
10    // can be any module, i. e.
11    'babel-loader',
12    'babel-preset-es2015',
13    'sass-loader',
14  ]
15);

Contributing

Please take a moment to read our contributing guidelines if you haven't yet done so.

CONTRIBUTING

License

MIT