Gathering detailed insights and metrics for node-fetch
Gathering detailed insights and metrics for node-fetch
Gathering detailed insights and metrics for node-fetch
Gathering detailed insights and metrics for node-fetch
A light-weight module that brings the Fetch API to Node.js
npm install node-fetch
Module System
Min. Node Version
Typescript Support
Node Version
NPM Version
8,793 Stars
640 Commits
1,027 Forks
89 Watching
10 Branches
123 Contributors
Updated on 27 Nov 2024
JavaScript (98.36%)
TypeScript (1.64%)
Cumulative downloads
Total Downloads
Last day
-0.9%
11,549,481
Compared to previous day
Last week
3.9%
62,302,928
Compared to previous week
Last month
12.4%
254,656,052
Compared to previous month
Last year
16.1%
2,705,226,621
Compared to previous year
A light-weight module that brings Fetch API to Node.js.
You might be looking for the v2 docs
Instead of implementing XMLHttpRequest
in Node.js to run browser-specific Fetch polyfill, why not go from native http
to fetch
API directly? Hence, node-fetch
, minimal code for a window.fetch
compatible API on Node.js runtime.
See Jason Miller's isomorphic-unfetch or Leonardo Quixada's cross-fetch for isomorphic usage (exports node-fetch
for server-side, whatwg-fetch
for client-side).
window.fetch
API.res.text()
and res.json()
) to UTF-8 automatically.window.fetch
offers, feel free to open an issue.Current stable release (3.x
) requires at least Node.js 12.20.0.
1npm install node-fetch
1import fetch from 'node-fetch';
node-fetch
from v3 is an ESM-only module - you are not able to import it with require()
.
If you cannot switch to ESM, please use v2 which remains compatible with CommonJS. Critical bug fixes will continue to be published for v2.
1npm install node-fetch@2
Alternatively, you can use the async import()
function from CommonJS to load node-fetch
asynchronously:
1// mod.cjs 2const fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));
To use fetch()
without importing it, you can patch the global
object in node:
1// fetch-polyfill.js 2import fetch, { 3 Blob, 4 blobFrom, 5 blobFromSync, 6 File, 7 fileFrom, 8 fileFromSync, 9 FormData, 10 Headers, 11 Request, 12 Response, 13} from 'node-fetch' 14 15if (!globalThis.fetch) { 16 globalThis.fetch = fetch 17 globalThis.Headers = Headers 18 globalThis.Request = Request 19 globalThis.Response = Response 20} 21 22// index.js 23import './fetch-polyfill' 24 25// ...
Using an old version of node-fetch? Check out the following files:
NOTE: The documentation below is up-to-date with 3.x
releases, if you are using an older version, please check how to upgrade.
1import fetch from 'node-fetch'; 2 3const response = await fetch('https://github.com/'); 4const body = await response.text(); 5 6console.log(body);
1import fetch from 'node-fetch'; 2 3const response = await fetch('https://api.github.com/users/github'); 4const data = await response.json(); 5 6console.log(data);
1import fetch from 'node-fetch'; 2 3const response = await fetch('https://httpbin.org/post', {method: 'POST', body: 'a=1'}); 4const data = await response.json(); 5 6console.log(data);
1import fetch from 'node-fetch'; 2 3const body = {a: 1}; 4 5const response = await fetch('https://httpbin.org/post', { 6 method: 'post', 7 body: JSON.stringify(body), 8 headers: {'Content-Type': 'application/json'} 9}); 10const data = await response.json(); 11 12console.log(data);
URLSearchParams
is available on the global object in Node.js as of v10.0.0. See official documentation for more usage methods.
NOTE: The Content-Type
header is only set automatically to x-www-form-urlencoded
when an instance of URLSearchParams
is given as such:
1import fetch from 'node-fetch'; 2 3const params = new URLSearchParams(); 4params.append('a', 1); 5 6const response = await fetch('https://httpbin.org/post', {method: 'POST', body: params}); 7const data = await response.json(); 8 9console.log(data);
NOTE: 3xx-5xx responses are NOT exceptions, and should be handled in then()
, see the next section.
Wrapping the fetch function into a try/catch
block will catch all exceptions, such as errors originating from node core libraries, like network errors, and operational errors which are instances of FetchError. See the error handling document for more details.
1import fetch from 'node-fetch'; 2 3try { 4 await fetch('https://domain.invalid/'); 5} catch (error) { 6 console.log(error); 7}
It is common to create a helper function to check that the response contains no client (4xx) or server (5xx) error responses:
1import fetch from 'node-fetch'; 2 3class HTTPResponseError extends Error { 4 constructor(response) { 5 super(`HTTP Error Response: ${response.status} ${response.statusText}`); 6 this.response = response; 7 } 8} 9 10const checkStatus = response => { 11 if (response.ok) { 12 // response.status >= 200 && response.status < 300 13 return response; 14 } else { 15 throw new HTTPResponseError(response); 16 } 17} 18 19const response = await fetch('https://httpbin.org/status/400'); 20 21try { 22 checkStatus(response); 23} catch (error) { 24 console.error(error); 25 26 const errorBody = await error.response.text(); 27 console.error(`Error body: ${errorBody}`); 28}
Cookies are not stored by default. However, cookies can be extracted and passed by manipulating request and response headers. See Extract Set-Cookie Header for details.
The "Node.js way" is to use streams when possible. You can pipe res.body
to another stream. This example uses stream.pipeline to attach stream error handlers and wait for the download to complete.
1import {createWriteStream} from 'node:fs'; 2import {pipeline} from 'node:stream'; 3import {promisify} from 'node:util' 4import fetch from 'node-fetch'; 5 6const streamPipeline = promisify(pipeline); 7 8const response = await fetch('https://github.githubassets.com/images/modules/logos_page/Octocat.png'); 9 10if (!response.ok) throw new Error(`unexpected response ${response.statusText}`); 11 12await streamPipeline(response.body, createWriteStream('./octocat.png'));
In Node.js 14 you can also use async iterators to read body
; however, be careful to catch
errors -- the longer a response runs, the more likely it is to encounter an error.
1import fetch from 'node-fetch'; 2 3const response = await fetch('https://httpbin.org/stream/3'); 4 5try { 6 for await (const chunk of response.body) { 7 console.dir(JSON.parse(chunk.toString())); 8 } 9} catch (err) { 10 console.error(err.stack); 11}
In Node.js 12 you can also use async iterators to read body
; however, async iterators with streams
did not mature until Node.js 14, so you need to do some extra work to ensure you handle errors
directly from the stream and wait on it response to fully close.
1import fetch from 'node-fetch'; 2 3const read = async body => { 4 let error; 5 body.on('error', err => { 6 error = err; 7 }); 8 9 for await (const chunk of body) { 10 console.dir(JSON.parse(chunk.toString())); 11 } 12 13 return new Promise((resolve, reject) => { 14 body.on('close', () => { 15 error ? reject(error) : resolve(); 16 }); 17 }); 18}; 19 20try { 21 const response = await fetch('https://httpbin.org/stream/3'); 22 await read(response.body); 23} catch (err) { 24 console.error(err.stack); 25}
1import fetch from 'node-fetch';
2
3const response = await fetch('https://github.com/');
4
5console.log(response.ok);
6console.log(response.status);
7console.log(response.statusText);
8console.log(response.headers.raw());
9console.log(response.headers.get('content-type'));
Unlike browsers, you can access raw Set-Cookie
headers manually using Headers.raw()
. This is a node-fetch
only API.
1import fetch from 'node-fetch'; 2 3const response = await fetch('https://example.com'); 4 5// Returns an array of values, instead of a string of comma-separated values 6console.log(response.headers.raw()['set-cookie']);
1import fetch, { 2 Blob, 3 blobFrom, 4 blobFromSync, 5 File, 6 fileFrom, 7 fileFromSync, 8} from 'node-fetch' 9 10const mimetype = 'text/plain' 11const blob = fileFromSync('./input.txt', mimetype) 12const url = 'https://httpbin.org/post' 13 14const response = await fetch(url, { method: 'POST', body: blob }) 15const data = await response.json() 16 17console.log(data)
node-fetch comes with a spec-compliant FormData implementations for posting multipart/form-data payloads
1import fetch, { FormData, File, fileFrom } from 'node-fetch' 2 3const httpbin = 'https://httpbin.org/post' 4const formData = new FormData() 5const binary = new Uint8Array([ 97, 98, 99 ]) 6const abc = new File([binary], 'abc.txt', { type: 'text/plain' }) 7 8formData.set('greeting', 'Hello, world!') 9formData.set('file-upload', abc, 'new name.txt') 10 11const response = await fetch(httpbin, { method: 'POST', body: formData }) 12const data = await response.json() 13 14console.log(data)
If you for some reason need to post a stream coming from any arbitrary place, then you can append a Blob or a File look-a-like item.
The minimum requirement is that it has:
Symbol.toStringTag
getter or property that is either Blob
or File
stream()
method or a arrayBuffer()
method that returns a ArrayBuffer.The stream()
must return any async iterable object as long as it yields Uint8Array (or Buffer)
so Node.Readable streams and whatwg streams works just fine.
1formData.append('upload', { 2 [Symbol.toStringTag]: 'Blob', 3 size: 3, 4 *stream() { 5 yield new Uint8Array([97, 98, 99]) 6 }, 7 arrayBuffer() { 8 return new Uint8Array([97, 98, 99]).buffer 9 } 10}, 'abc.txt')
You may cancel requests with AbortController
. A suggested implementation is abort-controller
.
An example of timing out a request after 150ms could be achieved as the following:
1import fetch, { AbortError } from 'node-fetch'; 2 3// AbortController was added in node v14.17.0 globally 4const AbortController = globalThis.AbortController || await import('abort-controller') 5 6const controller = new AbortController(); 7const timeout = setTimeout(() => { 8 controller.abort(); 9}, 150); 10 11try { 12 const response = await fetch('https://example.com', {signal: controller.signal}); 13 const data = await response.json(); 14} catch (error) { 15 if (error instanceof AbortError) { 16 console.log('request was aborted'); 17 } 18} finally { 19 clearTimeout(timeout); 20}
See test cases for more examples.
url
A string representing the URL for fetchingoptions
Options for the HTTP(S) requestPromise<Response>
Perform an HTTP(S) fetch.
url
should be an absolute URL, such as https://example.com/
. A path-relative URL (/file/under/root
) or protocol-relative URL (//can-be-http-or-https.com/
) will result in a rejected Promise
.
The default values are shown after each option key.
1{ 2 // These properties are part of the Fetch Standard 3 method: 'GET', 4 headers: {}, // Request headers. format is the identical to that accepted by the Headers constructor (see below) 5 body: null, // Request body. can be null, or a Node.js Readable stream 6 redirect: 'follow', // Set to `manual` to extract redirect headers, `error` to reject redirect 7 signal: null, // Pass an instance of AbortSignal to optionally abort requests 8 9 // The following properties are node-fetch extensions 10 follow: 20, // maximum redirect count. 0 to not follow redirect 11 compress: true, // support gzip/deflate content encoding. false to disable 12 size: 0, // maximum response body size in bytes. 0 to disable 13 agent: null, // http(s).Agent instance or function that returns an instance (see below) 14 highWaterMark: 16384, // the maximum number of bytes to store in the internal buffer before ceasing to read from the underlying resource. 15 insecureHTTPParser: false // Use an insecure HTTP parser that accepts invalid HTTP headers when `true`. 16}
If no values are set, the following request headers will be sent automatically:
Header | Value |
---|---|
Accept-Encoding | gzip, deflate, br (when options.compress === true ) |
Accept | */* |
Content-Length | (automatically calculated, if possible) |
Host | (host and port information from the target URI) |
Transfer-Encoding | chunked (when req.body is a stream) |
User-Agent | node-fetch |
Note: when body
is a Stream
, Content-Length
is not set automatically.
The agent
option allows you to specify networking related options which are out of the scope of Fetch, including and not limited to the following:
See http.Agent
for more information.
If no agent is specified, the default agent provided by Node.js is used. Note that this changed in Node.js 19 to have keepalive
true by default. If you wish to enable keepalive
in an earlier version of Node.js, you can override the agent as per the following code sample.
In addition, the agent
option accepts a function that returns http
(s).Agent
instance given current URL, this is useful during a redirection chain across HTTP and HTTPS protocol.
1import http from 'node:http'; 2import https from 'node:https'; 3 4const httpAgent = new http.Agent({ 5 keepAlive: true 6}); 7const httpsAgent = new https.Agent({ 8 keepAlive: true 9}); 10 11const options = { 12 agent: function(_parsedURL) { 13 if (_parsedURL.protocol == 'http:') { 14 return httpAgent; 15 } else { 16 return httpsAgent; 17 } 18 } 19};
Stream on Node.js have a smaller internal buffer size (16kB, aka highWaterMark
) from client-side browsers (>1MB, not consistent across browsers). Because of that, when you are writing an isomorphic app and using res.clone()
, it will hang with large response in Node.
The recommended way to fix this problem is to resolve cloned response in parallel:
1import fetch from 'node-fetch'; 2 3const response = await fetch('https://example.com'); 4const r1 = response.clone(); 5 6const results = await Promise.all([response.json(), r1.text()]); 7 8console.log(results[0]); 9console.log(results[1]);
If for some reason you don't like the solution above, since 3.x
you are able to modify the highWaterMark
option:
1import fetch from 'node-fetch'; 2 3const response = await fetch('https://example.com', { 4 // About 1MB 5 highWaterMark: 1024 * 1024 6}); 7 8const result = await res.clone().arrayBuffer(); 9console.dir(result);
Passed through to the insecureHTTPParser
option on http(s).request. See http.request
for more information.
The redirect: 'manual'
option for node-fetch is different from the browser & specification, which
results in an opaque-redirect filtered response.
node-fetch gives you the typical basic filtered response instead.
1import fetch from 'node-fetch'; 2 3const response = await fetch('https://httpbin.org/status/301', { redirect: 'manual' }); 4 5if (response.status === 301 || response.status === 302) { 6 const locationURL = new URL(response.headers.get('location'), response.url); 7 const response2 = await fetch(locationURL, { redirect: 'manual' }); 8 console.dir(response2); 9}
An HTTP(S) request containing information about URL, method, headers, and the body. This class implements the Body interface.
Due to the nature of Node.js, the following properties are not implemented at this moment:
type
destination
mode
credentials
cache
integrity
keepalive
The following node-fetch extension properties are provided:
follow
compress
counter
agent
highWaterMark
See options for exact meaning of these extensions.
(spec-compliant)
input
A string representing a URL, or another Request
(which will be cloned)options
Options for the HTTP(S) requestConstructs a new Request
object. The constructor is identical to that in the browser.
In most cases, directly fetch(url, options)
is simpler than creating a Request
object.
An HTTP(S) response. This class implements the Body interface.
The following properties are not implemented in node-fetch at this moment:
trailer
(spec-compliant)
body
A String
or Readable
streamoptions
A ResponseInit
options dictionaryConstructs a new Response
object. The constructor is identical to that in the browser.
Because Node.js does not implement service workers (for which this class was designed), one rarely has to construct a Response
directly.
(spec-compliant)
Convenience property representing if the request ended normally. Will evaluate to true if the response status was greater than or equal to 200 but smaller than 300.
(spec-compliant)
Convenience property representing if the request has been redirected at least once. Will evaluate to true if the internal redirect counter is greater than 0.
(deviation from spec)
Convenience property representing the response's type. node-fetch only supports 'default'
and 'error'
and does not make use of filtered responses.
This class allows manipulating and iterating over a set of HTTP headers. All methods specified in the Fetch Standard are implemented.
(spec-compliant)
init
Optional argument to pre-fill the Headers
objectConstruct a new Headers
object. init
can be either null
, a Headers
object, an key-value map object or any iterable object.
1// Example adapted from https://fetch.spec.whatwg.org/#example-headers-class 2import {Headers} from 'node-fetch'; 3 4const meta = { 5 'Content-Type': 'text/xml' 6}; 7const headers = new Headers(meta); 8 9// The above is equivalent to 10const meta = [['Content-Type', 'text/xml']]; 11const headers = new Headers(meta); 12 13// You can in fact use any iterable objects, like a Map or even another Headers 14const meta = new Map(); 15meta.set('Content-Type', 'text/xml'); 16const headers = new Headers(meta); 17const copyOfHeaders = new Headers(headers);
Body
is an abstract interface with methods that are applicable to both Request
and Response
classes.
(deviation from spec)
Readable
streamData are encapsulated in the Body
object. Note that while the Fetch Standard requires the property to always be a WHATWG ReadableStream
, in node-fetch it is a Node.js Readable
stream.
(spec-compliant)
Boolean
A boolean property for if this body has been consumed. Per the specs, a consumed body cannot be used again.
fetch
comes with methods to parse multipart/form-data
payloads as well as
x-www-form-urlencoded
bodies using .formData()
this comes from the idea that
Service Worker can intercept such messages before it's sent to the server to
alter them. This is useful for anybody building a server so you can use it to
parse & consume payloads.
1import http from 'node:http'
2import { Response } from 'node-fetch'
3
4http.createServer(async function (req, res) {
5 const formData = await new Response(req, {
6 headers: req.headers // Pass along the boundary value
7 }).formData()
8 const allFields = [...formData]
9
10 const file = formData.get('uploaded-files')
11 const arrayBuffer = await file.arrayBuffer()
12 const text = await file.text()
13 const whatwgReadableStream = file.stream()
14
15 // other was to consume the request could be to do:
16 const json = await new Response(req).json()
17 const text = await new Response(req).text()
18 const arrayBuffer = await new Response(req).arrayBuffer()
19 const blob = await new Response(req, {
20 headers: req.headers // So that `type` inherits `Content-Type`
21 }.blob()
22})
(node-fetch extension)
An operational error in the fetching process. See ERROR-HANDLING.md for more info.
(node-fetch extension)
An Error thrown when the request is aborted in response to an AbortSignal
's abort
event. It has a name
property of AbortError
. See ERROR-HANDLING.MD for more info.
Since 3.x
types are bundled with node-fetch
, so you don't need to install any additional packages.
For older versions please use the type definitions from DefinitelyTyped:
1npm install --save-dev @types/node-fetch@2.x
Thanks to github/fetch for providing a solid implementation reference.
David Frank | Jimmy Wärting | Antoni Kepinski | Richie Bendall | Gregor Martynus |
The latest stable version of the package.
Stable Version
2
8.8/10
Summary
node-fetch forwards secure headers to untrusted sites
Affected Versions
< 2.6.7
Patched Versions
2.6.7
8.8/10
Summary
node-fetch forwards secure headers to untrusted sites
Affected Versions
>= 3.0.0, < 3.1.1
Patched Versions
3.1.1
1
5.9/10
Summary
node-fetch Inefficient Regular Expression Complexity
Affected Versions
>= 3.0.0, < 3.2.10
Patched Versions
3.2.10
2
2.6/10
Summary
The `size` option isn't honored after following a redirect in node-fetch
Affected Versions
>= 2.0.0, < 2.6.1
Patched Versions
2.6.1
2.6/10
Summary
The `size` option isn't honored after following a redirect in node-fetch
Affected Versions
>= 3.0.0-beta.1, <= 3.0.0-beta.8
Patched Versions
3.0.0-beta.9
Reason
all changesets reviewed
Reason
no dangerous workflow patterns detected
Reason
no binaries found in the repo
Reason
0 existing vulnerabilities detected
Reason
license file detected
Details
Reason
packaging workflow detected
Details
Reason
security policy file detected
Details
Reason
0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Reason
detected GitHub workflow tokens with excessive permissions
Details
Reason
dependency not pinned by hash detected -- score normalized to 0
Details
Reason
no effort to earn an OpenSSF best practices badge detected
Reason
project is not fuzzed
Details
Reason
SAST tool is not run on all commits -- score normalized to 0
Details
Score
Last Scanned on 2024-11-18
The Open Source Security Foundation is a cross-industry collaboration to improve the security of open source software (OSS). The Scorecard provides security health metrics for open source projects.
Learn More