simple-get

Simplest way to make http get requests

features

This module is the lightest possible wrapper on top of node.js http, but supporting these essential features:

• follows redirects
• automatically handles gzip/deflate responses
• supports HTTPS
• supports specifying a timeout
• supports convenience url key so there's no need to use url.parse on the url when specifying options
• composes well with npm packages for features like cookies, proxies, form data, & OAuth

All this in < 100 lines of code.

install

npm install simple-get

usage

Note, all these examples also work in the browser with browserify.

simple GET request

Doesn't get easier than this:

1const get = require('simple-get')
2
3get('http://example.com', function (err, res) {
4  if (err) throw err
5  console.log(res.statusCode) // 200
6  res.pipe(process.stdout) // `res` is a stream
7})

even simpler GET request

If you just want the data, and don't want to deal with streams:

1const get = require('simple-get')
2
3get.concat('http://example.com', function (err, res, data) {
4  if (err) throw err
5  console.log(res.statusCode) // 200
6  console.log(data) // Buffer('this is the server response')
7})

POST, PUT, PATCH, HEAD, DELETE support

For POST, call get.post or use option { method: 'POST' }.

1const get = require('simple-get')
2
3const opts = {
4  url: 'http://example.com',
5  body: 'this is the POST body'
6}
7get.post(opts, function (err, res) {
8  if (err) throw err
9  res.pipe(process.stdout) // `res` is a stream
10})

A more complex example:

1const get = require('simple-get')
2
3get({
4  url: 'http://example.com',
5  method: 'POST',
6  body: 'this is the POST body',
7
8  // simple-get accepts all options that node.js `http` accepts
9  // See: http://nodejs.org/api/http.html#http_http_request_options_callback
10  headers: {
11    'user-agent': 'my cool app'
12  }
13}, function (err, res) {
14  if (err) throw err
15
16  // All properties/methods from http.IncomingResponse are available,
17  // even if a gunzip/inflate transform stream was returned.
18  // See: http://nodejs.org/api/http.html#http_http_incomingmessage
19  res.setTimeout(10000)
20  console.log(res.headers)
21
22  res.on('data', function (chunk) {
23    // `chunk` is the decoded response, after it's been gunzipped or inflated
24    // (if applicable)
25    console.log('got a chunk of the response: ' + chunk)
26  }))
27
28})

JSON

You can serialize/deserialize request and response with JSON:

1const get = require('simple-get')
2
3const opts = {
4  method: 'POST',
5  url: 'http://example.com',
6  body: {
7    key: 'value'
8  },
9  json: true
10}
11get.concat(opts, function (err, res, data) {
12  if (err) throw err
13  console.log(data.key) // `data` is an object
14})

Timeout

You can set a timeout (in milliseconds) on the request with the timeout option. If the request takes longer than timeout to complete, then the entire request will fail with an Error.

1const get = require('simple-get')
2
3const opts = {
4  url: 'http://example.com',
5  timeout: 2000 // 2 second timeout
6}
7
8get(opts, function (err, res) {})

One Quick Tip

It's a good idea to set the 'user-agent' header so the provider can more easily see how their resource is used.

1const get = require('simple-get')
2const pkg = require('./package.json')
3
4get('http://example.com', {
5  headers: {
6    'user-agent': `my-module/${pkg.version} (https://github.com/username/my-module)`
7  }
8})

Proxies

You can use the tunnel module with the agent option to work with proxies:

1const get = require('simple-get')
2const tunnel = require('tunnel')
3
4const opts = {
5  url: 'http://example.com',
6  agent: tunnel.httpOverHttp({
7    proxy: {
8      host: 'localhost'
9    }
10  })
11}
12
13get(opts, function (err, res) {})

Cookies

You can use the cookie module to include cookies in a request:

1const get = require('simple-get')
2const cookie = require('cookie')
3
4const opts = {
5  url: 'http://example.com',
6  headers: {
7    cookie: cookie.serialize('foo', 'bar')
8  }
9}
10
11get(opts, function (err, res) {})

Form data

You can use the form-data module to create POST request with form data:

1const fs = require('fs')
2const get = require('simple-get')
3const FormData = require('form-data')
4const form = new FormData()
5
6form.append('my_file', fs.createReadStream('/foo/bar.jpg'))
7
8const opts = {
9  url: 'http://example.com',
10  body: form
11}
12
13get.post(opts, function (err, res) {})

Or, include application/x-www-form-urlencoded form data manually:

1const get = require('simple-get')
2
3const opts = {
4  url: 'http://example.com',
5  form: {
6    key: 'value'
7  }
8}
9get.post(opts, function (err, res) {})

Specifically disallowing redirects

1const get = require('simple-get')
2
3const opts = {
4  url: 'http://example.com/will-redirect-elsewhere',
5  followRedirects: false
6}
7// res.statusCode will be 301, no error thrown
8get(opts, function (err, res) {})

Basic Auth

1const user = 'someuser'
2const pass = 'pa$$word'
3const encodedAuth = Buffer.from(`${user}:${pass}`).toString('base64')
4
5get('http://example.com', {
6  headers: {
7    authorization: `Basic ${encodedAuth}`
8  }
9})

OAuth

You can use the oauth-1.0a module to create a signed OAuth request:

1const get = require('simple-get')
2const crypto  = require('crypto')
3const OAuth = require('oauth-1.0a')
4
5const oauth = OAuth({
6  consumer: {
7    key: process.env.CONSUMER_KEY,
8    secret: process.env.CONSUMER_SECRET
9  },
10  signature_method: 'HMAC-SHA1',
11  hash_function: (baseString, key) => crypto.createHmac('sha1', key).update(baseString).digest('base64')
12})
13
14const token = {
15  key: process.env.ACCESS_TOKEN,
16  secret: process.env.ACCESS_TOKEN_SECRET
17}
18
19const url = 'https://api.twitter.com/1.1/statuses/home_timeline.json'
20
21const opts = {
22  url: url,
23  headers: oauth.toHeader(oauth.authorize({url, method: 'GET'}, token)),
24  json: true
25}
26
27get(opts, function (err, res) {})

Throttle requests

You can use limiter to throttle requests. This is useful when calling an API that is rate limited.

1const simpleGet = require('simple-get')
2const RateLimiter = require('limiter').RateLimiter
3const limiter = new RateLimiter(1, 'second')
4
5const get = (opts, cb) => limiter.removeTokens(1, () => simpleGet(opts, cb))
6get.concat = (opts, cb) => limiter.removeTokens(1, () => simpleGet.concat(opts, cb))
7
8var opts = {
9  url: 'http://example.com'
10}
11
12get.concat(opts, processResult)
13get.concat(opts, processResult)
14
15function processResult (err, res, data) {
16  if (err) throw err
17  console.log(data.toString())
18}

license

MIT. Copyright (c) Feross Aboukhadijeh.