request-plus

If you like request and/or request-promise-*, then you are good to go with this add-on!

It is a set of wrappers around request-promise-any module, adding the following features (all are optional and mostly independent):

• EventEmitter integration - you can set a generic listener for any fired/successful/failed request
• automatic retry for particular failures
• caching results using cache-manager
• exporting Prometheus monitoring metrics using prom-client
• simple built-in logging based on events

... and you can add your own wrappers too!

The only depency is request-promise-any, which has request as a peer dependecy, so you need to install separately the request version you like.

Basic Example

npm install request request-plus

1const request = require('request-promise')({
2  event: true,
3  retry: true,
4  log: true,
5});
6
7request('http://example.com/some/api')
8  .then(body => { console.log('response was: ' + body)})
9  .catch(error => { /*...*/ });

Advanced Example

Let's say we want to get JSON data from some resource, which fails sometimes and is in general quite slow. So we want to cache its results and do retries if it fails with a timeout or typical server errors. To implement caching we need to install additionally cache-manager

npm install request request-plus cache-manager --save

1// setup a cache object
2const cacheManager = require('cache-manager');
3const cache = cacheManager.caching({
4  store: 'memory',
5  max: 500 // keep maximum 500 different URL responses
6});
7
8const rp = require('request-plus');
9
10// create a concrete wrapper
11// you have can multiple in one project with different settings
12const request = rp({
13  // use retry wrapper
14  retry: {
15    attempts: 3
16  },
17
18  // use cache wrapper
19  cache: {
20    cache: cache,
21    cacheOptions: {
22      ttl: 3600 * 4 // 4 hours
23    }
24  }
25});
26
27// you can use all the options available in basic request module
28request({
29  uri: "http://some.service/providing/data",
30  json: true
31})
32  .then(data => {
33    // we get here if we got a 200 response within 3 retries
34  })
35  .catch(error => {
36    // well get here if the URL failed with 4xx errors,
37    // or 3 retry attempts failed
38  });

Wrappers

The wrappers can be specified in options when creating a new requestPlus wrapper (simple way), or you can add them one by one (advanced)

When specified in options, the wrappers will be added in a particular (common sense) order, namely: event, retry, cache, prom, log. Another limitation here: you can have only one wrapper of each type.

Sample:

1const rp = require('request-plus');
2const request = rp({
3  event: true,
4  retry: true,
5  prom: {
6    metric: myMetric
7  }
8});

When adding one by one you have full control of the order, and you may add wrappers of the same type.

1const rp = require('request-plus');
2const request = rp()
3  .plus.wrap('prom', {
4    metric: rawRequestHistogram
5  })
6  .plus.wrap('retry')
7  .plus.wrap('prom', {
8    metric: retriedRequestHistogram
9  });

Defaults Wrapper

Sets default options for requests. You can use it for headers or if you know all your requets expect json.

1const request = require('request-plus')({
2  defaults: {
3    headers: {
4      'User-Agent': 'My request-plus client'
5    },
6    json: true
7  }
8});
9
10// this will send a request with json:true preset
11// and with the custom User-Agent header
12request('http://some.service.com/products')
13  .then(data => {
14    if (data.product.length) {
15      /* ... */
16    }
17  });

Event Wrapper

This wrapper adds emitter to the .plus container and fires basic events for each request going through:

• request - on start of the request
• error - on error
• response - on successful response

1const request = require('request-plus')({event: true});
2
3// always output failed http requests to std error
4// together with used request param
5// independent of promise chains/catch clauses
6request.plus.emitter.on('error', (uri, error) => {
7  console.error('http request failed %j', uri);
8})
9request('http://..soooo...bad...')
10.catch(() => {
11  console.log("something happen, i don't know what");
12})

All events have uri (which can be a string or options object) as the first parameter. Other parameters depend on the event - see source code to see additional params provided for each event

Retry Wrapper

Params (all optional):

• attempts = 3 - number of attempt before giving up
• delay = 500(ms) - delay between retries. You can provide a closure and calculate it to make a progressive delay, e.g. attempt => 500 * attempt * attempt
• errorFilter - closure defining whether a retry should be done. By default it returns true for either a timeout or the statusCode is in [500, 502, 503, 504]

1const rp = require('request-plus');
2const request = rp({
3  retry: {
4    attempts: 5,
5    delay: 1500,
6
7    // retry all errors (timeout returns no response object)
8    errorFilter: error =>
9      error.response === undefined
10      || error.statusCode >= 400
11  }
12});

If there is an event wrapper initialised, then it will additionally fire events: retryRequest, retryError, retrySuccess providing the current attempt counter.

Cache Wrapper

Should be used together with a third-party module: cache-manager

Params:

• cache (required) - an instance of cache for cache-manager module
• cacheOptions - options used for set()
• getKey - closure generating string cache key for given request options. By default for string param - the full URI is used as key, for an object a hash is additionally generated and added to the URI (see below)
• hash - hash function for default getKey behavior. By default it generates a key using a very cheap algorithm, but with a significant collision probability

1const rp = require('request-plus');
2const cacheManager = require('cache-manager');
3const memeoryCache = cacheManager.caching({store: 'memory'});
4const crypto = require('crypto');
5const request = rp({
6  cache: {
7    cache: memeoryCache,
8    hash: str => crypto.createHash('md5').update(str).digest("hex")
9  }
10});

If there is an event wrapper initialised, then it will additionally fire events: cacheRequest and cacheMiss. You can use those to gather stats and calculate cache hits as count(hits) = count(cacheRequests) - count(cacheMisses).

Prometheus Wrapper

Should be used together with a third-party module: prom-client

The wrapper takes a prometheus metric and uses it to monitor both successful and error responses. It supports all basic metric types assuming that Counter just counts responses and Gauge, Histogram and Summary measure latency.

If the metric has status_code label, then it will be automatically set for each request.

If this wrapper doesn't meet your needs, you can add your own measurements using event wrapper (see above).

Params:

• metric - and instance of prom-client metric
• labels - an object with labels or a closure generating labels on the fly

1const promClient = require('prom-client');
2const testHistogram = new promClient.Histogram(
3  'test_histogram',
4  'help of test_histogram',
5  ['status_code', 'nature']
6  {buckets: [0.1, 1]}
7);
8const request = require('request-plus')({
9  prom: {
10    metric: testHistogram,
11    labels: error => {
12      if (!error) {
13        return {nature: 'success'};
14      }
15      return error.respose.statusCode === 418
16        ? {nature: 'teapot'}
17        : {}
18    }
19  }
20});

Log Wrapper

Just outputs some some of the events to stdout/stderr. Thus it requires event wrapper.

The main intention of this plugin is just to give a simple way to switch on logging when debugging. Though with some effort you can use it also in production for logging particular events

Params:

• prefix - overrides the function used to generate the prefix preceding the log information. By default it's: eventName => '[' + eventName + ']'
• loggers - overrides the default console log/error/warn. See source / unit tests for more details
• events - overrides behaviour for provided events. For each event you can provide either logger name ('info', 'warn', 'error') or a function. Additionally to event parameters this function gets eventName as the first parameter.

1const rp = require('request-plus');
2const request = rp({
3  event: true,
4  log: {
5    events: {
6      fail: 'error',
7      retryFail: (eventName, uri, error, attempt) => {
8        console.error('failed despite retries: %j, on %d attempt', uri, attempt);
9      }
10    }
11  }
12});

Adding custom wrapper

1function myWrapper(requester) {
2  return function(uri, requestOptions, callback) {
3    if (requester.plus.emitter) {
4      requester.plus.emitter.emit('myEvent', 'hello from me');
5    }
6    console.log('the uri is %j', uri);
7    return requester(uri, requestOptions, callback);
8  };
9}
10
11const request = require('request-plus')()
12  .plus.wrap('event')
13  .plus.wrap(myWrapper);