Buff Utils

The swiss-army-knife of byte manipulation.

Features:

• Move between all data formats with ease!
• Encode / decode between Base58, Base64, Bech32 and more.
• Buffer object recognized as native Uint8Array.
• Prepend, append, sort, split and join arrays of multiple formats.
• Read and prefix varints.
• Convert byte blobs into consumable streams.
• Uses DataView.setUint8 for ultra-fast performace.
• Supports endianess for all the things!

Import

Example import into a browser-based project:

1<script src="https://unpkg.com/@cmdcode/buff-utils/dist/browser.js"></script>
2<script> const { Buff, Bytes } = window.buff_utils </script>

Example import into a commonjs project:

1const { Buff, Bytes } = require('@cmdcode/buff-utils')

Example import into an ES module project:

1import { Buff, Bytes } from '@cmdcode/buff-utils'

How to Use

The Buff class is an extention of the base Uint8Array class. It provides the same default functionality of a Uint8Array, and can be used as a drop-in replacement for Uint8Array. Typescript will treat Buff as a Uint8Array object.

1import { Buff, Bytes } from '@cmdcode/buff-utils'
2
3// Bytes covers value types that are convertable to Uint8Array.
4type Bytes  = string | number | bigint | Uint8Array | Buff
5// You can optionally specify the endianess of data.
6type Endian = 'le' | 'be'
7
8const bytes = new Buff (
9  data    : Bytes | Bytes[] | ArrayBuffer,  
10  size   ?: number, // Specify the size of the array (for padding)
11  endian ?: Endian  // Specify the endianess of the array.
12)
13

You can convert from many different types and formats into a Buff object.

1Buff
2  .any     = (data : any, size ?: number)        => Buff,
3  .raw     = (data : Uint8Array, size ?: number) => Buff,
4  .str     = (data : string, size ?: number)     => Buff,
5  .hex     = (data : string, size ?: number)     => Buff,
6  .bin     = (data : string, size ?: number)     => Buff,
7  .num     = (data : number, size ?: number)     => Buff,
8  .big     = (data : bigint, size ?: number)     => Buff,
9  .bytes   = (data : Bytes,  size ?: number)     => Buff,
10  .json    = (data : T,  replacer ?: Replacer)   => Buff,
11  .b58chk  = (data : string)                     => Buff,
12  .base64  = (data : string)                     => Buff,
13  .b64url  = (data : string)                     => Buff,
14
15  .bech32  = (
16    data        : string,  // Data to be encoded
17    limit      ?: number,  // Enforce a character limit.
18    chk_prefix ?: string   // Enforce a certain prefix.
19  ) => Buff,
20
21  .bech32m = (
22    data        : string,  // Data to be encoded
23    limit      ?: number,  // Enforce a character limit.
24    chk_prefix ?: string   // Enforce a certain prefix.
25  ) => Buff
26}

With Buff, you have access to an extensive API for converting between formats.

1const bytes = new Buff(data)
2
3/* Quicky convert into many formats using getters. */
4
5bytes
6  .arr     => number[]    // Convert to a number array.
7  .num     => number      // Convert to a Number.
8  .big     => bigint      // Convert to a BigInt.
9  .str     => string      // Convert to a UTF8 string.
10  .hex     => string      // Convert to a hex string.
11  .raw     => Uint8Array  // Convert to a pure Uint8Array.
12  .bin     => string      // Convert to a binary string.
13  .b58chk  => string      // Convert to base58 with checksum.
14  .base64  => string      // Convert to base64 string.
15  .b64url  => string      // Convert to base64url string.
16  .digest  => Buff        // Convert to a sha256 digest.
17  .id      => string      // Convert to a digest (hex string).
18  .stream  => Stream      // Convert to a Stream object.
19
20/* There are a few export methods that support extra params. */
21
22bytes
23  .toNum     : (endian ?: Endian)                  => number
24  .toBig     : (endian ?: Endian)                  => bigint
25  .toBin     : ()                                  => string
26  .toHash    : ()                                  => Buff
27  .toJson    : (reviver ?: Reviver)                => T
28  .toBech32  : (prefix : string, limit ?: number)  => string
29  .toBech32m : (prefix : string, limit ?: number)  => string

In addition to format conversion, you can perform many other convenient tasks.

1Buff = {
2  // Standard UTF-8 string-to-bytes encoding.
3  encode : (str : string) => Uint8Array,
4  // Standard UTF-8 bytes-to-string decoding.
5  decode : (bytes : Uint8Array) => string,
6  // Same as Uint8Array.from(), but returns a Buff object.
7  from (data : Uint8Array | number[]) => Buff
8  // Same as Uint8Array.of(), but returns a Buff object.
9  of (...data : number[]) => Buff,
10  // Join together multiple arrays of bytes.
11  join : (array : Bytes[]) => Buff,
12  // Sort multiple arrays of bytes in lexicographic order.
13  sort (arr : Bytes[], size ?: number) => Buff[],
14  // Return a buffer object with random data (uses webcrypto).
15  random (size : number) => Buff,
16  // Converts a number into a 'varint' for byte streams.
17  varInt : (num : number, endian ?: Endian) => Buff
18}
19
20const bytes = new Buff(data)
21
22bytes
23  // Append data to your ubber object
24  .append (data : Bytes) => Buff
25  // Prepend data to your buffer object.
26  .prepend (data : Bytes) => Buff
27  // Encode the size of your buffer as a varint and prepend it.
28  .prefixSize (endian ?: Endian) => Buff
29  // Same as Uint8Array.reverse(), but returns a Buff object.
30  .reverse () => Buff
31  // Identical to Uint8Array.set() method.
32  .set (array : ArrayLike<number>, offset ?: number) => void
33  // Same as Uint8Array.slice(), but returns a Buff object.
34  .slice (start ?: number, end ?: number) => Buff
35  // Same as Uint8Array.subarray(), but returns a Buff object.
36  .subarray (begin ?: number, end ?: number) => Buff
37  // Same as Uint8Array.set(), but allows Bytes as input.
38  .write (data : Bytes, offset ?: number) => void

The Stream tool will take a blob of data and allow you to consume it byte-per-byte.

1import { Stream } from '@cmdcode/buff-utils'
2
3// Convert data into a stream object.
4const stream = new Stream(data)
5
6// You can convert a buff object into a stream.
7const stream = new Buff(data).stream
8
9stream
10  // Reads x number of bytes, does not consume the stream.
11  .peek(size: number) => Buff
12  // Reads x number of bytes, consumes the stream.
13  .read(size: number) => bytes
14  // Reads the next bytes(s) as a varint, returns the number value.
15  .readSize (endian ?: Endian) => number

A number of utilities are available as stand-alone packages for import.

1import {
2  Encoder, Hash, Hex, Txt, assert, buffer, util
3} from '@cmdcode/buff-utils'

Dependencies

This library uses minimal dependences.

@scure/base
For performing encoding and decoding of many formats.
https://github.com/paulmillr/scure-base

@noble/hashes For creating hashes using sha256.
https://github.com/paulmillr/noble-hashes

Special thanks to Paul Miller for his wonderful work.

Bugs / Issues

Please feel free to post any questions or bug reports on the issues page!

Testing

This project uses tape for unit tests, and nyc for formatting.

1yarn install && yarn test
2npm  install && npm run test

Development

This project uses eslint and typescript for development, plus rollup for bundling releases.

1yarn install && yarn release
2npm  install && npm run release

Contributions

All contributions are welcome!

License

Use this code however you like! No warranty!