@dimfu/recipe-scraper

Scrap food recipe data from various websites that support JSON+LD and Microdata schema, no guessing involved.

Install

1npm install --save @dimfu/recipe-scraper

Usage

Basic

Pass a url as first parameter will attempt to extract the recipe content from that website.

1import getRecipeData from '@dimfu/recipe-scraper'
2
3const url = 'https://example/recipes/creamy-courgette-potato-bake'
4
5getRecipeData(url).then((recipe) => {
6  console.log(recipe)
7})

or with async/await

1import getRecipeData from '@dimfu/recipe-scraper'
2
3async function run() {
4  const url = 'https://example/recipes/creamy-courgette-potato-bake'
5  const recipe = await getRecipeData(url)
6  console.log(recipe)
7}

HTML String

If you already have HTML string and dont want to make an http request, you can specify it in option object:

Note This can be work only if there is JSON-LD or Microdata schema tags were present.

1import getRecipeData from '@dimfu/recipe-scraper'
2
3const html = `
4    <script type="application/ld+json">
5        {
6            "@context": "http://schema.org",
7            "@type": "Recipe",
8            "name": "Classic Marinara Sauce",
9            "recipeIngredient": [
10                "1 28-ounce can whole tomatoes",
11                "1/4 cup olive oil",
12                "7 garlic peeled and slivered",
13                "Small dried whole chile",
14                "1 teaspoon kosher salt",
15                "1 large fresh basil sprig"
16            ]
17        }
18     </script>
19`
20
21const recipe = await getRecipeData({ html })

---

1import getRecipeData from '@dimfu/recipe-scraper'
2
3const html = `
4    <div itemscope itemtype="https://schema.org/Recipe">
5        <h1 itemprop="name">Simple Marinara Sauce</h1>
6        <div>
7            <span itemprop="recipeIngredient">2 cans stewed tomatoes</span>
8            <span itemprop="recipeIngredient">1 teaspoon dried oregano</span>
9            <span itemprop="recipeIngredient">1 teaspoon salt</span>
10        </div>
11    </div>
12`
13
14const recipe = await getRecipeData({ html })

It returns back the information related to the given URL

1{
2  "url": "https://example/recipes/creamy-courgette-potato-bake",
3  "name": "Creamy courgette & potato bake",
4  "image": "https://example/stryve/9ae78bc2-ad5e-449c-8626-8c9faa37054c_creamy-courgette-potato-bake.png?auto=compress,format",
5  "cookTime": "45 minutes",
6  "prepTime": "25 minutes",
7  "totalTime": "70 minutes",
8  "recipeYield": 4,
9  "recipeIngredients": [
10    "1000g Potato",
11    "2 Courgette",
12    "2 Brown onion",
13    "3tsp Olive oil",
14    "120g Cashew nuts",
15    "200ml Vegetable stock",
16    "200ml Almond milk",
17    "6 Garlic cloves",
18    "18tsp Nutritional yeast",
19    "2tsp Sea salt",
20    "2tsp Smoked paprika"
21  ],
22  "recipeInstructions": [
23    "Add cashew nuts to a bowl with enough hot water to cover",
24    "Peel and thinly slice the potatoes and courgettes",
25    "Thinly slice the onion and add to a pan with olive oil – fry for ~5 mins mixing often until lightly brown",
26    "Pre-heat the oven on 180°C (355°F)",
27    "Drain the water from cashew nuts and place in blender with vegetable stock, almond milk, garlic, nutritional yeast and salt – blend until smooth",
28    "To your oven dish add a layer potato, followed by a layer of courgette, followed by the onion",
29    "Next sprinkle half of the smoked paprika on top",
30    "Continue adding another layer of potato, followed by another layer of courgette and pour ⅔ of the creamy sauce on top",
31    "Finish off with one more layer of potatoes, the remaining sauce and the other half of the smoked paprika – place in the oven for 45 mins"
32  ]
33}

When a field can't be retrieved, the value will be undefined.

There are no guarantees about the shape of this data, because it varies with different media and scraping methods. Handle it carefully.

Configuration

You can change the behaviour by passing an options object:

1import getRecipeData from '@dimfu/recipe-scraper'
2
3const options = {
4  url: 'https://example/recipes/creamy-courgette-potato-bake', // URL of web page
5  maxRedirects: 0, // Maximum number of redirects to follow (default: 5)
6  timeout: 1000, // Request timeout in milliseconds (default: 10000ms)
7}
8
9getRecipeData(options).then((recipe) => {
10  console.log(recipe)
11})

You can specify the URL by either passing it as the first parameter, or by setting it in the options object.

Command line usage

Using without installation

1npx @dimfu/recipe-scraper https://example/recipes/creamy-courgette-potato-bake 

Note: Use this method only if you plan to use for one time, installing globally (see-below) is recommended for multiple time usages.

Installation

1npm install @dimfu/recipe-scraper -g

Note for Linux & MacOS users: DO NOT use sudo to install global packages! The correct way is to tell npm where to install its global packages: npm config set prefix ~/.local. Make sure ~/.local/bin is added to PATH.

Usage after installation

@dimfu/recipe-scraper https://example/recipes/creamy-courgette-potato-bake

License

Distributed under the MIT License. See LICENSE for more information.