node-goldwasher-schedule

Plugin for goldwasher, using node-schedule to schedule goldwasher-needle requests at time intervals. Requires goldwasher to work.

Installation

npm install goldwasher-schedule

Usage

1var goldwasher = require('goldwasher-schedule');
2var gs = goldwasher(targets, options);
3gs.on('result', function(results) {
4  console.log(results);
5});
6gs.start();
7gs.stop();

Parameters

Targets

The first parameter required by the setup function is an array of targets. An example:

1[
2  {
3    url: 'https://github.com',
4    rule: { second: [15, 35, 55] },
5    goldwasher: {
6      selector: 'h1'
7    }
8  }
9]

url is the only required parameter. rule is the schedule rule for node-schedule. In this case, 3 times a minute when second equals any of the three values (defaults to second: 1, e.g. once a minute). goldwasher is an object of custom goldwasher options for this target.

Additionally, all other options used by goldwasher-needle can be passed along, such as needle, goldwasherNeedle and retry. Have a look at their respective doc pages for goldwasher, needle and retry for options available.

If no other options than url are set in the target, the defaults provided by the options parameter, explained below, will be used.

Options

Options can be optionally passed in as the second parameter. It can contain the default values for targets. For instance:

1var options = {
2    rule: { second: 10 },
3    goldwasher: {
4        selector: 'h1'
5    },
6    needle: {
7        follow_max: 20
8    },
9    retry: {
10        retries: 3
11    }
12}

These options will be applied to all targets that do not explicitly define them themselves. Note that if no rule is provided, it defaults to second: 1, e.g. once a minute.

events

This module is an event emitter, that will emit events on start, stop, run, end and results. The results event will emit:

1. results - the results from goldwasher.
2. options - the options (and target) the results have been collected with.
3. response - the response from goldwasher-needle.
4. body - the raw body from goldwasher-needle.

Example

1var gs = require('goldwasher-schedule');
2
3// declare google.com a target
4var targets = [
5  {
6    url: 'https://google.com'
7  }
8];
9
10// set up the schedule
11gs(targets).start();
12
13// receive the results
14gs.on('result', function(results) {
15  console.log(results);
16});

Advanced example

1var goldwasher = require('goldwasher-schedule');
2
3// first will use default options below, second has custom options
4var targets = [
5  {
6    url: 'https://google.com'
7  },
8  {
9    url: 'https://github.com',
10    rule: { second: [15, 35, 55] },
11    goldwasher: {
12      selector: 'h1'
13    }
14  }
15];
16
17// default options
18var options = {
19  rule: { second: [1, 10, 20, 30, 40, 50] }
20};
21
22// set up the schedule
23var gs = goldwasher(targets, options);
24
25// receive the results
26gs.on('result', function(results) {
27  console.log(results);
28});
29
30// start the schedule
31gs.start();
32
33// stop the schedule after 60 seconds
34setTimeout(function() {
35  gs.stop();
36}, 60000);