parallel-transform

Transform stream for Node.js that allows you to run your transforms in parallel without changing the order of the output.

npm install parallel-transform

It is easy to use

1var transform = require('parallel-transform');
2
3var stream = transform(10, function(data, callback) { // 10 is the parallism level
4	setTimeout(function() {
5		callback(null, data);
6	}, Math.random() * 1000);
7});
8
9for (var i = 0; i < 10; i++) {
10	stream.write(''+i);
11}
12stream.end();
13
14stream.on('data', function(data) {
15	console.log(data); // prints 0,1,2,...
16});
17stream.on('end', function() {
18	console.log('stream has ended');
19});

If you run the above example you'll notice that it runs in parallel (does not take ~1 second between each print) and that the order is preserved

Stream options

All transforms are Node 0.10 streams. Per default they are created with the options {objectMode:true}. If you want to use your own stream options pass them as the second parameter

1var stream = transform(10, {objectMode:false}, function(data, callback) {
2	// data is now a buffer
3	callback(null, data);
4});
5
6fs.createReadStream('filename').pipe(stream).pipe(process.stdout);

Unordered

Passing the option {ordered:false} will output the data as soon as it's processed by a transform, without waiting to respect the order.

License

MIT