Gathering detailed insights and metrics for tall-plugin-meta-refresh
Gathering detailed insights and metrics for tall-plugin-meta-refresh
Installations
npm install tall-plugin-meta-refreshReleases
9
Contributors
8
Developer
Developer Guide
BETA
Module System
ESM
Min. Node Version
>=16.0.0
Typescript Support
Yes
Node Version
16.17.1
NPM Version
8.15.0
Statistics
72 Stars
48 Commits
7 Forks
4 Watching
2 Branches
8 Contributors
Updated on 04 Nov 2024
Languages
TypeScript (58.08%)
JavaScript (41.92%)
Total Downloads
Cumulative downloads
Total Downloads
25,167
Last day
-18.2%
9
Compared to previous day
Last week
5.8%
257
Compared to previous week
Last month
11.6%
834
Compared to previous month
Last year
-23%
10,138
Compared to previous year
Daily Downloads
Weekly Downloads
Monthly Downloads
Yearly Downloads
Versions
tall
Promise-based, No-dependency URL unshortner (expander) module for Node.js 16+.
Note: This library is written in TypeScript and type definitions are provided.
Install
Using npm
1npm install --save tall
or with yarn
1yarn add tall
Usage
ES6+ usage:
1import { tall } from 'tall' 2 3tall('http://www.loige.link/codemotion-rome-2017') 4 .then((unshortenedUrl) => console.log('Tall url', unshortenedUrl)) 5 .catch((err) => console.error('AAAW 👻', err))
With Async await:
1import { tall } from 'tall' 2 3async function someFunction() { 4 try { 5 const unshortenedUrl = await tall( 6 'http://www.loige.link/codemotion-rome-2017' 7 ) 8 console.log('Tall url', unshortenedUrl) 9 } catch (err) { 10 console.error('AAAW 👻', err) 11 } 12} 13 14someFunction()
ES5:
1var { tall } = require('tall') 2tall('http://www.loige.link/codemotion-rome-2017') 3 .then(function (unshortenedUrl) { 4 console.log('Tall url', unshortenedUrl) 5 }) 6 .catch(function (err) { 7 console.error('AAAW 👻', err) 8 })
Options
It is possible to specify some options as second parameter to the tall function.
Available options are the following:
method(default"GET"): any available HTTP methodmaxRedirects(default3): the number of maximum redirects that will be followed in case of multiple redirects.headers(default{}): change request headers - e.g.{'User-Agent': 'your-custom-user-agent'}timeout: (default:120000): timeout in milliseconds after which the request will be cancelledplugins: (default:[locationHeaderPlugin]): a list of plugins for adding advanced behaviours
In addition, any other options available on http.request() or https.request() are accepted. This for example includes rejectUnauthorized to disable certificate checks.
Example:
1import { tall } from 'tall' 2 3tall('http://www.loige.link/codemotion-rome-2017', { 4 method: 'HEAD', 5 maxRedirect: 10 6}) 7 .then((unshortenedUrl) => console.log('Tall url', unshortenedUrl)) 8 .catch((err) => console.error('AAAW 👻', err))
Plugins
Since tall v5+, a plugin system for extending the default behaviour of tall is available.
By default tall comes with 1 single plugin, the locationHeaderPlugin which is enabled by default. This plugin follows redirects by looking at the location header in the HTTP response received from the source URL.
You might want to write your own plugins to have more sophisticated behaviours.
Some example?
- Normalise the final URL if the final page has a
<link rel="canonical" href="http://example.com/page/" />tag in the<head>of the document - Follow HTML meta refresh redirects (
<meta http-equiv="refresh" content="0;URL='http://example.com/'" />)
Known plugins
tall-plugin-meta-refresh(official): follows redirects in<meta http-equiv="refresh">tags
Did you create a plugin for tall? Send us a PR to have it listed here!
How to write a plugin
A plugin is simply a function with a specific signature:
1export interface TallPlugin { 2 (url: URL, response: IncomingMessage, previous: Follow | Stop): Promise< 3 Follow | Stop 4 > 5}
So the only thing you need to do is to write your custom behaviour following this interface. But let's discuss briefly what the different elements mean here:
url: Is the current URL being crawledresponse: is the actual HTTP response object representing the currentprevious: the decision from the previous plugin execution (continue following a given URL or stop at a given URL)
Every plugin is executed asynchronously, so a plugin returns a Promise that needs to resolve to a Follow or a Stop decision.
Let's deep dive into these two concepts. Follow and Stop are defined as follows (touché):
1export class Follow { 2 follow: URL 3 constructor(follow: URL) { 4 this.follow = follow 5 } 6} 7 8export class Stop { 9 stop: URL 10 constructor(stop: URL) { 11 this.stop = stop 12 } 13}
Follow and Stop are effectively simple classes to express an intent: should we follow the follow URL or should we stop at the stop URL?
Plugins are executed following the middleware pattern (or chain of responsibility): they are executed in order and the information is propagated from one to the other.
For example, if we initialise tall with { plugins: [plugin1, plugin2] }, for every URL, plugin1 will be executed before plugin2 and the decision of plugin1 will be passed over onto plugin2 using the previous) parameter.
How to write and enable a plugin
Let's say we want to add a plugin that allows us to follow HTML meta refresh redirects, the code could look like this:
1// metarefresh-plugin.ts 2import { IncomingMessage } from 'http' 3import { Follow, Stop } from 'tall' 4 5export async function metaRefreshPlugin( 6 url: URL, 7 response: IncomingMessage, 8 previous: Follow | Stop 9): Promise<Follow | Stop> { 10 let html = '' 11 for await (const chunk of response) { 12 html += chunk.toString() 13 } 14 15 // note: This is just a dummy example to illustrate how to use the plugin API. 16 // It's not a great idea to parse HTML using regexes. 17 // If you are looking for a plugin that does this in a better way check out 18 // https://npm.im/tall-plugin-meta-refresh 19 const metaHttpEquivUrl = html.match( 20 /meta +http-equiv="refresh" +content="\d;url=(http[^"]+)"/ 21 )?.[1] 22 23 if (metaHttpEquivUrl) { 24 return new Follow(new URL(metaHttpEquivUrl)) 25 } 26 27 return previous 28}
Then, this is how you would use your shiny new plugin:
1import { tall, locationHeaderPlugin } from 'tall' 2import { metaRefreshPlugin } from './metarefresh-plugin' 3 4const finalUrl = await tall('https://loige.link/senior', { 5 plugins: [locationHeaderPlugin, metaRefreshPlugin] 6}) 7 8console.log(finalUrl)
Note that we have to explicitly pass the locationHeaderPlugin if we want to retain tall original behaviour.
Contributing
Everyone is very welcome to contribute to this project. You can contribute just by submitting bugs or suggesting improvements by opening an issue on GitHub.
Note: Since Tall v6, the project structure is a monorepo, so you'll need to use a recent version of npm that supports workspaces (e.g. npm 8.5+)
License
Licensed under MIT License. © Luciano Mammino.
No vulnerabilities found.
Reason
no binaries found in the repo
Reason
no dangerous workflow patterns detected
Reason
license file detected
Details
- Info: project has a license file: LICENSE:0
- Info: FSF or OSI recognized license: MIT License: LICENSE:0
Reason
Found 11/28 approved changesets -- score normalized to 3
Reason
7 existing vulnerabilities detected
Details
- Warn: Project is vulnerable to: GHSA-67hx-6x53-jw92
- Warn: Project is vulnerable to: GHSA-grv7-fg5c-xmjg
- Warn: Project is vulnerable to: GHSA-3xgq-45jj-v275
- Warn: Project is vulnerable to: GHSA-9c47-m6qq-7p4h
- Warn: Project is vulnerable to: GHSA-952p-6rrq-rcjv
- Warn: Project is vulnerable to: GHSA-c2qf-rxjj-qqgw
- Warn: Project is vulnerable to: GHSA-j8xg-fqg3-53r7
Reason
0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Reason
detected GitHub workflow tokens with excessive permissions
Details
- Warn: no topLevel permission defined: .github/workflows/main.yaml:1
- Warn: no topLevel permission defined: .github/workflows/release.yaml:1
- Info: no jobLevel write permissions found
Reason
dependency not pinned by hash detected -- score normalized to 0
Details
- Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yaml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/lmammino/tall/main.yaml/main?enable=pin
- Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yaml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/lmammino/tall/main.yaml/main?enable=pin
- Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yaml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/lmammino/tall/main.yaml/main?enable=pin
- Warn: third-party GitHubAction not pinned by hash: .github/workflows/main.yaml:42: update your workflow using https://app.stepsecurity.io/secureworkflow/lmammino/tall/main.yaml/main?enable=pin
- Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/lmammino/tall/release.yaml/main?enable=pin
- Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/lmammino/tall/release.yaml/main?enable=pin
- Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/lmammino/tall/release.yaml/main?enable=pin
- Warn: npmCommand not pinned by hash: .github/workflows/main.yaml:28
- Warn: npmCommand not pinned by hash: .github/workflows/main.yaml:33
- Warn: npmCommand not pinned by hash: .github/workflows/release.yaml:28
- Info: 0 out of 6 GitHub-owned GitHubAction dependencies pinned
- Info: 0 out of 1 third-party GitHubAction dependencies pinned
- Info: 0 out of 3 npmCommand dependencies pinned
Reason
no effort to earn an OpenSSF best practices badge detected
Reason
security policy file not detected
Details
- Warn: no security policy file detected
- Warn: no security file to analyze
- Warn: no security file to analyze
- Warn: no security file to analyze
Reason
project is not fuzzed
Details
- Warn: no fuzzer integrations found
Reason
branch protection not enabled on development/release branches
Details
- Warn: branch protection not enabled for branch 'main'
Reason
SAST tool is not run on all commits -- score normalized to 0
Details
- Warn: 0 commits out of 17 are checked with a SAST tool
Score
3.1
/10
Last Scanned on 2024-11-18
The Open Source Security Foundation is a cross-industry collaboration to improve the security of open source software (OSS). The Scorecard provides security health metrics for open source projects.
Learn More