node-html-markdown

NHM is a fast HTML to markdown converter, compatible with both node and the browser.

It was built with the following two goals in mind:

1. Speed

We had a need to convert gigabytes of HTML daily very quickly. All libraries we found were too slow with node. We considered using a low-level language but decided to attempt to write something that would squeeze every bit of performance out of the JIT that we could. The end result was fast enough to make the cut!

2. Human Readability

The other libraries we tested produced output that would break in numerous conditions and produced output with many repeating linefeeds, etc. Generally speaking, outside of a markdown viewer, the result was not easy to read.

We took the approach of producing a clean, concise result with consistent spacing rules.

Install

1<yarn|npm|pnpm> add node-html-markdown

Benchmarks

-----------------------------------------------------------------------------

Estimated processing times (fastest to slowest):

  [node-html-markdown (reused instance)]
    100 kB:  17ms
    1 MB:    176ms
    50 MB:   8.80sec
    1 GB:    3min, 0sec
    50 GB:   2hr, 30min, 14sec

  [turndown (reused instance)]
    100 kB:  27ms
    1 MB:    280ms
    50 MB:   13.98sec
    1 GB:    4min, 46sec
    50 GB:   3hr, 58min, 35sec

-----------------------------------------------------------------------------

Speed comparison - node-html-markdown (reused instance) is:

  1.02 times as fast as node-html-markdown
  1.57 times as fast as turndown
  1.59 times as fast as turndown (reused instance)

-----------------------------------------------------------------------------

Usage

1import { NodeHtmlMarkdown, NodeHtmlMarkdownOptions } from 'node-html-markdown'
2
3
4/* ********************************************************* *
5 * Single use
6 * If using it once, you can use the static method
7 * ********************************************************* */
8
9// Single file
10NodeHtmlMarkdown.translate(
11  /* html */ `<b>hello</b>`, 
12  /* options (optional) */ {}, 
13  /* customTranslators (optional) */ undefined,
14  /* customCodeBlockTranslators (optional) */ undefined
15);
16
17// Multiple files
18NodeHtmlMarkdown.translate(
19  /* FileCollection */ { 
20    'file1.html': `<b>hello</b>`, 
21    'file2.html': `<b>goodbye</b>` 
22  }, 
23  /* options (optional) */ {}, 
24  /* customTranslators (optional) */ undefined,
25  /* customCodeBlockTranslators (optional) */ undefined
26);
27
28
29/* ********************************************************* *
30 * Re-use
31 * If using it several times, creating an instance saves time
32 * ********************************************************* */
33
34const nhm = new NodeHtmlMarkdown(
35  /* options (optional) */ {}, 
36  /* customTransformers (optional) */ undefined,
37  /* customCodeBlockTranslators (optional) */ undefined
38);
39
40// Single file
41nhm.translate(/* html */ `<b>hello</b>`);
42
43// Multiple Files
44nhm.translate(
45  /* FileCollection */ { 
46    'file1.html': `<b>hello</b>`, 
47    'file2.html': `<b>goodbye</b>` 
48  }, 
49);

Options

1
2export interface NodeHtmlMarkdownOptions {
3  /**
4   * Use native window DOMParser when available
5   * @default false
6   */
7  preferNativeParser: boolean,
8
9  /**
10   * Code block fence
11   * @default ```
12   */
13  codeFence: string,
14
15  /**
16   * Bullet marker
17   * @default *
18   */
19  bulletMarker: string,
20
21  /**
22   * Style for code block
23   * @default fence
24   */
25  codeBlockStyle: 'indented' | 'fenced',
26
27  /**
28   * Emphasis delimiter
29   * @default _
30   */
31  emDelimiter: string,
32
33  /**
34   * Strong delimiter
35   * @default **
36   */
37  strongDelimiter: string,
38
39  /**
40   * Strong delimiter
41   * @default ~~
42   */
43  strikeDelimiter: string,
44
45  /**
46   * Supplied elements will be ignored (ignores inner text does not parse children)
47   */
48  ignore?: string[],
49
50  /**
51   * Supplied elements will be treated as blocks (surrounded with blank lines)
52   */
53  blockElements?: string[],
54
55  /**
56   * Max consecutive new lines allowed
57   * @default 3
58   */
59  maxConsecutiveNewlines: number,
60
61  /**
62   * Line Start Escape pattern
63   * (Note: Setting this will override the default escape settings, you might want to use textReplace option instead)
64   */
65  lineStartEscape: [ pattern: RegExp, replacement: string ]
66
67  /**
68   * Global escape pattern
69   * (Note: Setting this will override the default escape settings, you might want to use textReplace option instead)
70   */
71  globalEscape: [ pattern: RegExp, replacement: string ]
72
73  /**
74   * User-defined text replacement pattern (Replaces matching text retrieved from nodes)
75   */
76  textReplace?: [ pattern: RegExp, replacement: string ][]
77
78  /**
79   * Keep images with data: URI (Note: These can be up to 1MB each)
80   * @example
81   * <img src="data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSK......0o/">
82   * @default false
83   */
84  keepDataImages?: boolean
85
86  /**
87   * Place URLS at the bottom and format links using link reference definitions
88   *
89   * @example
90   * Click <a href="/url1">here</a>. Or <a href="/url2">here</a>. Or <a href="/url1">this link</a>.
91   *
92   * Becomes:
93   * Click [here][1]. Or [here][2]. Or [this link][1].
94   *
95   * [1]: /url
96   * [2]: /url2
97   */
98  useLinkReferenceDefinitions?: boolean
99
100  /**
101   * Wrap URL text in < > instead of []() syntax.
102   * 
103   * @example
104   * The input <a href="https://google.com">https://google.com</a>
105   * becomes <https://google.com>
106   * instead of [https://google.com](https://google.com)
107   * 
108   * @default true
109   */
110  useInlineLinks?: boolean
111}

Custom Translators

Custom translators are an advanced option to allow handling certain elements a specific way.

These can be modified via the NodeHtmlMarkdown#translators property, or added during creation.

For detail on how to use them see:

• translator.ts - Documentation for TranslatorConfig
• config.ts - Translators in defaultTranslators

The NodeHtmlMarkdown#codeBlockTranslators property is a collection of translators which handles elements within a <pre><code> block.

Further improvements

Being a performance-centric library, we're always interested in further improvements. There are several probable routes by which we could gain substantial performance increases over the current model.

Such methods include:

• Writing a custom parser
• Integrating an async worker-thread based model for multi-threading
• Fully replacing any remaining regex

These would be fun to implement; however, for the time being, the present library is fast enough for my purposes. That said, I welcome discussion and any PR toward the effort of further improving performance, and I may ultimately do more work in that capacity in the future!

Help Wanted!

Looking to contribute? Check out our help wanted list for a good place to start!