Gathering detailed insights and metrics for lzutf8
Gathering detailed insights and metrics for lzutf8
Gathering detailed insights and metrics for lzutf8
Gathering detailed insights and metrics for lzutf8
A high-performance Javascript string compression library
npm install lzutf8
Typescript
Module System
Node Version
NPM Version
98.7
Supply Chain
99.1
Quality
75.7
Maintenance
100
Vulnerability
100
License
TypeScript (96.69%)
CSS (1.57%)
JavaScript (1.35%)
HTML (0.39%)
Batchfile (0.01%)
Total Downloads
12,704,272
Last Day
9,541
Last Week
58,766
Last Month
259,914
Last Year
3,035,020
323 Stars
133 Commits
26 Forks
6 Watching
4 Branches
3 Contributors
Minified
Minified + Gzipped
Latest Version
0.6.3
Package Id
lzutf8@0.6.3
Unpacked Size
145.76 kB
Size
30.70 kB
File Count
6
NPM Version
8.13.2
Node Version
18.4.0
Cumulative downloads
Total Downloads
Last day
2.1%
9,541
Compared to previous day
Last week
5.4%
58,766
Compared to previous week
Last month
-1%
259,914
Compared to previous month
Last year
-14.7%
3,035,020
Compared to previous year
Note: this library is significantly out-of-date and will require a full rewrite to update with recent technologies like JS modules and WebAssembly, and for better compatibility with modern frameworks like Angular and Rect.js. The design and documentation was mostly written in 2014, before any of these were relevant. Unfortunately, it is not maintained anymore, and a rewrite is unlikely to take place in the foreseeable future.
LZ-UTF8 is a string compression library and format. Is an extension to the UTF-8 character encoding, augmenting the UTF-8 bytestream with optional compression based the LZ77 algorithm. Some of its properties:
Javascript implementation:
"BinaryString"
encoding described later in this document) when binary storage is not available or desired (e.g. when using LocalStorage or older IndexedDB).Node.js:
npm install lzutf8
1var LZUTF8 = require('lzutf8');
Browser:
1<script id="lzutf8" src="https://cdn.jsdelivr.net/npm/lzutf8/build/production/lzutf8.js"></script>
or the minified version:
1<script id="lzutf8" src="https://cdn.jsdelivr.net/npm/lzutf8/build/production/lzutf8.min.js"></script>
to reference a particular version use the pattern, where x.x.x
should be replaced with the exact version number (e.g. 0.4.6
):
1<script id="lzutf8" src="https://unpkg.com/lzutf8@x.x.x/build/production/lzutf8.min.js"></script>
note: the id
attribute and its exact value are necessary for the library to make use of web workers.
"ByteArray"
- An array of bytes. As of 0.3.2
, always a Uint8Array
. In versions up to 0.2.3
the type was determined by the platform (Array
for browsers that don't support typed arrays, Uint8Array
for supporting browsers and Buffer
for Node.js).
IE8/9 and support was dropped at 0.3.0
though these browsers can still be used with a typed array polyfill.
"Buffer"
- A Node.js Buffer
object.
"StorageBinaryString"
- A string
containing compacted binary data encoded to fit in valid UTF-16 strings. Please note the older, deprecated, "BinaryString"
encoding, is still internally supported in the library but has been removed from this document. More details are included further in this document.
"Base64"
- A base 64 string.
1var output = LZUTF8.compress(input, [options]);
Compresses the given input data.
input
can be either a String
or UTF-8 bytes stored in a Uint8Array
or Buffer
options
(optional): an object that may have any of the properties:
outputEncoding
: "ByteArray"
(default), "Buffer"
, "StorageBinaryString"
or "Base64"
returns: compressed data encoded by encoding
, or ByteArray
if not specified.
1var output = LZUTF8.decompress(input, [options]);
Decompresses the given compressed data.
input
: can be either a Uint8Array
, Buffer
or String
(where encoding scheme is then specified in inputEncoding
)
options
(optional): an object that may have the properties:
inputEncoding
: "ByteArray"
(default), "StorageBinaryString"
or "Base64"
outputEncoding
: "String"
(default), "ByteArray"
or "Buffer"
to return UTF-8 bytesreturns: decompressed bytes encoded as encoding
, or as String
if not specified.
1LZUTF8.compressAsync(input, [options], callback);
Asynchronously compresses the given input data.
input
can be either a String
, or UTF-8 bytes stored in an Uint8Array
or Buffer
.
options
(optional): an object that may have any of the properties:
outputEncoding
: "ByteArray"
(default), "Buffer"
, "StorageBinaryString"
or "Base64"
useWebWorker
: true
(default) would use a web worker if available. false
would use iterated yielding instead.callback
: a user-defined callback function accepting a first argument containing the resulting compressed data as specified by outputEncoding
(or ByteArray
if not specified) and a possible second parameter containing an Error
object.
On error: invokes the callback with a first argument of undefined
and a second one containing the Error
object.
Example:
1LZUTF8.compressAsync(input, {outputEncoding: "StorageBinaryString"}, function (result, error) { 2 if (error === undefined) 3 console.log("Data successfully compressed and encoded to " + result.length + " characters"); 4 else 5 console.log("Compression error: " + error.message); 6});
1LZUTF8.decompressAsync(input, [options], callback);
Asynchronously decompresses the given compressed input.
input
: can be either a Uint8Array
, Buffer
or String
(where encoding is set with inputEncoding
).
options
(optional): an object that may have the properties:
inputEncoding
: "ByteArray"
(default), "StorageBinaryString"
or "Base64"
outputEncoding
: "String"
(default), "ByteArray"
or "Buffer"
to return UTF-8 bytes.useWebWorker
: true
(default) would use a web worker if available. false
would use incremental yielding instead.callback
: a user-defined callback function accepting a first argument containing the resulting decompressed data as specified by outputEncoding
and a possible second parameter containing an Error
object.
On error: invokes the callback with a first argument of undefined
and a second one containing the Error
object.
Example:
1LZUTF8.decompressAsync(input, {inputEncoding: "StorageBinaryString", outputEncoding: "ByteArray"}, function (result, error) {
2 if (error === undefined)
3 console.log("Data successfully decompressed to " + result.length + " UTF-8 bytes");
4 else
5 console.log("Decompression error: " + error.message);
6});
Web workers are available if supported by the browser and the library's script source is referenced in the document with a script
tag having id
of "lzutf8"
(its src
attribute is then used as the source URI for the web worker). In cases where a script tag is not available (such as when the script is dynamically loaded or bundled with other scripts) the value of LZUTF8.WebWorker.scriptURI
may alternatively be set before the first async method call.
Workers are optimized for various input and output encoding schemes, so only the minimal amount of work is done in the main Javascript thread. Internally, conversion to or from various encodings is performed within the worker itself, reducing delays and allowing greater parallelization. Additionally, if transferable objects are supported by the browser, binary arrays will be transferred virtually instantly to and from the worker.
Only one worker instance is spawned per page - multiple operations are processed sequentially.
In case a worker is not available (such as in Node.js, IE8, IE9, Android browser < 4.4) or desired, it will iteratively process 64KB blocks while yielding to the event loop whenever a 20ms interval has elapsed. Note: In this execution method, parallel operations are not guaranteed to complete by their initiation order.
1var compressor = new LZUTF8.Compressor();
Creates a compressor object. Can be used to incrementally compress a multi-part stream of data.
returns: a new LZUTF8.Compressor
object
1var compressor = new LZUTF8.Compressor(); 2var compressedBlock = compressor.compressBlock(input);
Compresses the given input UTF-8 block.
input
can be either a String
, or UTF-8 bytes stored in a Uint8Array
or Buffer
returns: compressed bytes as ByteArray
This can be used to incrementally create a single compressed stream. For example:
1var compressor = new LZUTF8.Compressor(); 2var compressedBlock1 = compressor.compressBlock(block1); 3var compressedBlock2 = compressor.compressBlock(block2); 4var compressedBlock3 = compressor.compressBlock(block3); 5..
1var decompressor = new LZUTF8.Decompressor();
Creates a decompressor object. Can be used to incrementally decompress a multi-part stream of data.
returns: a new LZUTF8.Decompressor
object
1var decompressor = new LZUTF8.Decompressor(); 2var decompressedBlock = decompressor.decompressBlock(input);
Decompresses the given block of compressed bytes.
input
can be either a Uint8Array
or Buffer
returns: decompressed UTF-8 bytes as ByteArray
Remarks: will always return the longest valid UTF-8 stream of bytes possible from the given input block. Incomplete input or output byte sequences will be prepended to the next block.
Note: This can be used to incrementally decompress a single compressed stream. For example:
1var decompressor = new LZUTF8.Decompressor(); 2var decompressedBlock1 = decompressor.decompressBlock(block1); 3var decompressedBlock2 = decompressor.decompressBlock(block2); 4var decompressedBlock3 = decompressor.decompressBlock(block3); 5..
1var decompressor = new LZUTF8.Decompressor(); 2var decompressedBlockAsString = decompressor.decompressBlockToString(input);
Decompresses the given block of compressed bytes and converts the result to a String
.
input
can be either a Uint8Array
or Buffer
returns: decompressed String
Remarks: will always return the longest valid string possible from the given input block. Incomplete input or output byte sequences will be prepended to the next block.
1var compressionStream = LZUTF8.createCompressionStream();
Creates a compression stream. The stream will accept both Buffers and Strings in any encoding supported by Node.js (e.g. utf8
, utf16
, ucs2
, base64
, hex
, binary
etc.) and return Buffers.
example:
1var sourceReadStream = fs.createReadStream(“content.txt”); 2var destWriteStream = fs.createWriteStream(“content.txt.lzutf8”); 3var compressionStream = LZUTF8.createCompressionStream(); 4 5sourceReadStrem.pipe(compressionStream).pipe(destWriteStream);
On error: emits an error
event with the Error
object as parameter.
1var decompressionStream = LZUTF8.createDecompressionStream();
Creates a decompression stream. The stream will accept and return Buffers.
On error: emits an error
event with the Error
object as parameter.
1var output = LZUTF8.encodeUTF8(input);
Encodes a string to UTF-8.
input
as String
returns: encoded bytes as ByteArray
1var outputString = LZUTF8.decodeUTF8(input);
Decodes UTF-8 bytes to a String.
input
as either a Uint8Array
or Buffer
returns: decoded bytes as String
1var outputString = LZUTF8.encodeBase64(bytes);
Encodes bytes to a Base64 string.
input
as either a Uint8Array
or Buffer
returns: resulting Base64 string.
remarks: Maps every 3 consecutive input bytes to 4 output characters of the set A-Z
,a-z
,0-9
,+
,/
(a total of 64 characters). Increases stored byte size to 133.33% of original (when stored as ASCII or UTF-8) or 266% (stored as UTF-16).
1var output = LZUTF8.decodeBase64(input);
Decodes UTF-8 bytes to a String.
input
as String
returns: decoded bytes as ByteArray
remarks: the decoder cannot decode concatenated base64 strings. Although it is possible to add this capability to the JS version, compatibility with other decoders (such as the Node.js decoder) prevents this feature to be added.
Note: the older BinaryString
encoding has been deprecated due to a compatibility issue with the IE browser's LocalStorage/SessionStorage implementation. This newer version works around that issue by avoiding the 0
codepoint.
1var outputString = LZUTF8.encodeStorageBinaryString(input);
Encodes binary bytes to a valid UTF-16 string.
input
as either a Uint8Array
or Buffer
returns: String
remarks: To comply with the UTF-16 standard, it only uses the bottom 15 bits of each character, effectively mapping every 15 input bits to a single 16 bit output character. This Increases the stored byte size to 106.66% of original.
Note: the older BinaryString
encoding has been deprecated due to a compatibility issue with the IE browser's LocalStorage/SessionStorage implementation. This newer version works around that issue by avoiding the 0
codepoint.
1var output = LZUTF8.decodeStorageBinaryString(input);
Decodes a binary string.
input
as String
returns: decoded bytes as ByteArray
remarks: Multiple binary strings may be freely concatenated and decoded as a single string. This is made possible by ending every sequence with special marker (char code 32768 for an even-length sequence and 32769 for a an odd-length sequence).
0.1.x
: Initial release.0.2.x
: Added async error handling. Added support for TextEncoder
and TextDecoder
when available.0.3.x
: Removed support to IE8/9. Removed support for plain Array
inputs. All "ByteArray"
outputs are now Uint8Array
objects. A separate "Buffer"
encoding setting can be used to return Buffer
objects.0.4.x
: Major code restructuring. Removed support for versions of Node.js prior to 4.0
.0.5.x
: Added the "StorageBinaryString"
encoding.Copyright (c) 2014-2018, Rotem Dan <rotemdan@gmail.com>.
Source code and documentation are available under the MIT license.
No vulnerabilities found.
Reason
no binaries found in the repo
Reason
license file detected
Details
Reason
Found 3/26 approved changesets -- score normalized to 1
Reason
0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Reason
no effort to earn an OpenSSF best practices badge detected
Reason
security policy file not detected
Details
Reason
project is not fuzzed
Details
Reason
branch protection not enabled on development/release branches
Details
Reason
SAST tool is not run on all commits -- score normalized to 0
Details
Reason
10 existing vulnerabilities detected
Details
Score
Last Scanned on 2024-11-25
The Open Source Security Foundation is a cross-industry collaboration to improve the security of open source software (OSS). The Scorecard provides security health metrics for open source projects.
Learn More