bitdataview (beta version)

ES6 bit addressing DataView+Stack+Queue+Buffer with any types: Uint13, Int53, BigInt61, Float17, LSB/MSB, LE/BE, .set(), .get(), .push(), .pop(), .shift(), .unshift()

Features

• Like DataView, but with bit addressing.
• .set(), .get(), .push(), .unshift(), .pop(), .shift() methods for each data type.
• Data types: boolean and custom bit size integers and floats. For example: Uint17, Int39 or BigUint59.
• Like a BitBuffer, BitArray, BitStack, BitQueue.
• Small memory using: 1 bit in memory for every 1 bit data. 23 bits data => 3 bytes in RAM.
• Full assert arguments of functions.
• Endianness: Little Endian and Big Endian byte order supported.
• Bit numbering: LSB (lest significant bit) by default, MSB (most SB) is supported.
• Can export and import to C/C++ BitDataView library (only if LSB + Little Endian).
• 100 asserted main methods, 100 unasserted fast methods, > 20 advanced methods
• Binary parsing and decrypting RS-232, HDLC, Ethernet, USB, Can-Bus, TCP/IP RAW packets.
• NodeJs and browser Javascript support.
• Can be used in external schemas. Used in Telemok schemas.
• Old browser support.
• IOT pack data for exchange between C/C++ and JavaScript ready.

Examples:

Available in folder /examples/

1. Pack Uint7, Boolean, Int43 variables to hex. And unpack.

1import { BitDataView } from '@telemok/bitdataview';
2
3let sourceData = {
4	percents: 99, // maximal 7 bits
5	isOn: true, // 1 bit
6	moneyBalance: -1234567890123, // maximal 43 bits
7};
8let source = new BitDataView();
9source.push_Byte(7, sourceData.percents);// pack 7 bits
10source.push_Bits(sourceData.isOn);// pack 1 bit
11source.push_Int(43, sourceData.moneyBalance);// pack 43 bits
12console.log("storedBits", source.getStoredBits());//storedBits 51
13let hex = source.exportHex();//export 7 + 1 + 43 = 51 bits or 7 bytes
14console.log("hex", hex);//hex e335fb048ee006
15//send hex to another device, or store to localstorage
16// [another computer another script.js]
17let dest = new BitDataView();
18dest.importHex(hex);
19let result = {
20	percents: dest.shift_Byte(7),
21	isOn: dest.shift_Bit(),
22	moneyBalance: dest.shift_Int(43),
23};
24console.log("result", result);//result { percents: 99, isOn: 1, moneyBalance: -1234567890123 }

2. Unpack Uint7, Boolean, Int43 variables in C/C++.

1/*C code*/
2#pragma pack(1)
3struct Example{
4	unsigned char percents: 7;
5	bool isOn;
6	long moneyBalance: 43;
7}__attribute__((__packed__ ));
8Example example;
9memcpy((void*)(&example), (void*)source, sizeof(Example));

3. New instance with parameters

1import { BitDataView } from '@telemok/bitdataview';
2
3let bitdataview = new BitDataView({
4    automaticMemoryExpansion: true,//Allow auto expand, when push or unshift
5    bufferBaseSizeBits: 0, //Base size, default is 256 * 8 bits
6    significantBit: "LSB", //Bits order
7    endianness: "BIG_ENDIAN", //Bytes order
8	startOffsetBits: 0, //Move start offset to begin of buffer
9});

4. Make binary packed packet

1import { BitDataView } from '@telemok/bitdataview';
2
3function userPutDataToPacket(bitdataview) {//put some user different data
4	bitdataview.push_Uint(12, driver.PACKET_TYPE_TELEMETRY_1);
5	bitdataview.push_Float32(driver.getTemperature());
6	bitdataview.push_Int(19, driver.getBalance());
7	bitdataview.push_Byte(7, driver.getPercents());
8	bitdataview.push_Uint(12, driver.getThermocouple());
9	bitdataview.push_BigUint(59, driver.getUptimeTicks());
10}
11
12let bitdataview = new BitDataView({
13    automaticMemoryExpansion: false, //Fixed buffer size, deny auto expand
14    bufferBaseSizeBits: 400, //Buffer size bits
15    significantBit: "LSB", //Bits order
16    endianness: "LITTLE_ENDIAN", //Bytes order
17	startOffsetBits: 9 + 12, //Move start offset to free space for prepend length of packet
18});
19userPutDataToPacket(bitdataview);//Get any user data
20if(bitdataview.getStoredBits() > 375)//Check data size overflow
21	throw new Error(`Overflow`);
22bitdataview.unshift_Uint(9, bitdataview.getStoredBits());//Put 9-bit length of packet BEFORE packet.
23let uint8ArrayPacket = bitdataview.exportUnit8Array();//Zeroes will be added to end of byte
24bitdataview.unshift_Uint(16, crc16(uint8ArrayPacket));//Add crc16 of length+packet to begin of packet
25bitdataview.push_Uint(8, "\n".charCodeAt(0));//Add "\n" as end of packet
26
27protocol.sendHex(bitdataview.exportHex());

5. Parse binary unpack packet

1import { BitDataView } from '@telemok/bitdataview';
2
3let bitdataview = new BitDataView({
4	automaticMemoryExpansion: false, //Fixed buffer size, deny auto expand
5	bufferBaseSizeBits: 400, //Buffer size bits
6	significantBit: "LSB", //Bits order
7	endianness: "LITTLE_ENDIAN", //Bytes order
8	startOffsetBits: 0,
9});
10
11function parsePacket(bitdataview)
12{
13	valda.instance.assert(bitdataview, BitDataView);//Check bitdataview is instanceof BitDataView
14	let length = bitdataview.shift(9);//Read 9-bit length from begin of packet
15	if(length > bitdataview.getStoredBits())
16		throw new Error(`Wrong length`);
17    let packetType = bitdataview.shift_Uint(12);
18	if(packetType === driver.PACKET_TYPE_TELEMETRY_1)
19    {
20      //Check packet size if need. But if no data will be auto throw. Length don't used, it example.
21      let result = {
22        temperature: bitdataview.shift_Float32(),
23        balance: bitdataview.shift_Int(19),
24        percents: bitdataview.shift_Byte(7),
25        thermocouple: bitdataview.shift_Uint(12),
26        uptimeTicks: bitdataview.shift_BigUint(59),
27      };
28      console.log("result","PACKET_TYPE_TELEMETRY_1", result);
29    }
30	else
31		console.error(`Parsed wrong packet type`, packetType);
32}
33
34protocol.addEventListener('readByte', event =>{
35	let detail = valda.object.parse(event, 'detail');
36	let byte = valda.integer.parse(detail, 'byte', 0, 255);
37	
38	if(byte === "\n".charCodeAt(0))//end of packet finded
39    {
40        if(bitdataview.getStoredBits() < 9 + 16 + 12)
41          throw new Error(`Low size packet`);
42        let crc16True = bitdataview.shift_Uint(16);//Read crc16 from packet
43        let uint8ArrayPacket = bitdataview.exportUnit8Array();//Zeroes will be added to end of byte
44        let crc16Calc = crc16(uint8ArrayPacket);//Calc crc16 of length+packet to end of packet
45        if(crc16True !== crc16Calc)
46          throw new Error(`Wrong crc16`);
47	    parsePacket(bitdataview);
48    }
49	if(bitdataview.getAvailableBitsToPush() < 8)
50    {
51        bitdataview.clear();
52        console.error("Wrong packet stream");
53        return;
54    }
55	bitdataview.push_Byte(8, byte);
56});

Source code:

https://github.com/Telemok/bitdataview
https://npmjs.com/@telemok/bitdataview

Installation:

1. Create your NodeJs, Browser or Webview app.
2. Run: npm import @telemok/bitdataview
3. Code: import { BitDataView } from '@telemok/bitdataview';
4. Code: var bitDataView = new BitDataView();

Main functions:

1/* setAt - set value in address at begin of bitDataView (dont't change size) */
2setAt_Bit(bitIndexAt, value)
3setAt_Byte(bitIndexAt, countBitsToSet/*0-8*/, value)
4setAt_Uint(bitIndexAt, countBitsToSet/*0-53*/, value)//use endianness
5setAt_Int(bitIndexAt, countBitsToSet/*1-53*/, value)//use endianness
6setAt_BigUint(bitIndexAt, countBitsToSet/*0-64*/, value)//use endianness
7setAt_BigInt(bitIndexAt, countBitsToSet/*1-64*/, value)//use endianness
8setAt_Float32(bitIndexAt, value)//use endianness
9setAt_Float64(bitIndexAt, value)//use endianness
10
11/* setUntil - set value in address until end of bitDataView (dont't change size) */
12setUntil_Bit(bitIndexUntil, value)
13//setUntil_Byte(bitIndexUntil, countBitsToSet/*0-8*/, value)
14//setUntil_Uint(bitIndexUntil, countBitsToSet/*0-53*/, value)//use endianness
15//setUntil_Int(bitIndexUntil, countBitsToSet/*1-53*/, value)//use endianness
16//setUntil_BigUint(bitIndexUntil, countBitsToSet/*0-64*/, value)//use endianness
17//setUntil_BigInt(bitIndexUntil, countBitsToSet/*1-64*/, value)//use endianness
18setUntil_Float32(bitIndexUntil, value)//use endianness
19setUntil_Float64(bitIndexUntil, value)//use endianness
20
21/* getAt - get value in address at begin of bitDataView (dont't change size) */
22getAt_Bit(bitIndexAt)
23getAt_Byte(bitIndexAt, countBitsToSet/*0-8*/)
24getAt_Uint(bitIndexAt, countBitsToSet/*0-53*/)//use endianness
25getAt_Int(bitIndexAt, countBitsToSet/*1-53*/)//use endianness
26getAt_BigUint(bitIndexAt, countBitsToSet/*0-64*/)//use endianness
27getAt_BigInt(bitIndexAt, countBitsToSet/*1-64*/)//use endianness
28getAt_Float32(bitIndexAt)//use endianness
29getAt_Float64(bitIndexAt)//use endianness
30
31/* getUntil - get value in address until end of bitDataView (dont't change size) */
32getUntil_Bit(bitIndexUntil)
33//getUntil_Byte(bitIndexUntil, countBitsToSet/*0-8*/)
34//getUntil_Uint(bitIndexUntil, countBitsToSet/*0-53*/)//use endianness
35//getUntil_Int(bitIndexUntil, countBitsToSet/*1-53*/)//use endianness
36//getUntil_BigUint(bitIndexUntil, countBitsToSet/*0-64*/)//use endianness
37//getUntil_BigInt(bitIndexUntil, countBitsToSet/*1-64*/)//use endianness
38//getUntil_Float32(bitIndexUntil)//use endianness
39//getUntil_Float64(bitIndexUntil)//use endianness
40
41/* push - add value before end of bitDataView (increase size) */
42push_Nothing(countBits)
43push_Bits(value, count = 1)
44push_Byte(countBitsToSet/*0-8*/, value)
45push_Uint(countBitsToSet/*0-53*/, value)//use endianness
46push_Int(countBitsToSet/*1-53*/, value)//use endianness
47push_BigUint(countBitsToSet/*0-64*/, value)//use endianness
48//push_BigInt(countBitsToSet/*1-64*/, value)//use endianness
49push_Float32(value)//use endianness
50push_Float64(value)//use endianness
51//push_Float(sign = true, exponent = 11, mantissa = 52, value)//use little endian
52push_Uint8Array(uint8Array, littleEndian = false)
53push_DataView(dataView, littleEndian = false)
54push_Hex(hexString, littleEndian = false)
55
56/* unshift - add value before begin of bitDataView (increase size) */
57unshift_Bits(value, count = 1)
58unshift_Byte(countBitsToSet/*0-8*/, value)
59unshift_Uint(countBitsToSet/*0-53*/, value)//use endianness
60unshift_Int(countBitsToSet/*1-53*/, value)//use endianness
61unshift_BigUint(countBitsToSet/*0-64*/, value)//use endianness
62//unshift_BigInt(countBitsToSet/*1-64*/, value)//use endianness
63unshift_Float32(value)//use endianness
64unshift_Float64(value)//use endianness
65//unshift_Float(sign = true, exponent = 11, mantissa = 52, value)//use little endian
66unshift_Uint8Array(uint8Array, littleEndian = false)
67unshift_DataView(dataView, littleEndian = false)
68unshift_Hex(hexString, littleEndian = false)
69
70/* pop - take value from end of bitDataView (reduce size) */
71pop_Bit(count = 1)
72pop_Byte(countBitsToSet/*0-8*/)
73pop_Uint(countBitsToSet/*0-53*/)//use endianness
74pop_Int(countBitsToSet/*1-53*/)//use endianness
75//pop_BigUint(countBitsToSet/*0-64*/)//use endianness
76//pop_BigInt(countBitsToSet/*1-64*/)//use endianness
77pop_Float32()//use endianness
78pop_Float64()//use endianness
79//pop_Float(sign = true, exponent = 11, mantissa = 52)//use little endian
80pop_Uint8Array(countBytes = this.getStoredBits() , littleEndian = false)
81pop_DataView(countBytes = undefined , littleEndian = false)
82pop_Hex(countBytes = undefined , littleEndian = false)
83
84/* shift - take value from begin of bitDataView (reduce size) */
85shift_Bit(count = 1)
86shift_Byte(countBitsToSet/*0-8*/)
87shift_Uint(countBitsToSet/*0-53*/)//use endianness
88shift_Int(countBitsToSet/*1-53*/)//use endianness
89//shift_BigUint(countBitsToSet/*0-64*/)//use endianness
90//shift_BigInt(countBitsToSet/*1-64*/)//use endianness
91shift_Float32()//use endianness
92shift_Float64()//use endianness
93//shift_Float(sign = true, exponent = 11, mantissa = 52)//use little endian
94shift_Uint8Array(countBytes = this.getStoredBits() , littleEndian = false)
95shift_DataView(countBytes = undefined , littleEndian = false)
96shift_Hex(countBytes = undefined , littleEndian = false)
97
98importUint8Array(uint8Array)
99exportUnit8Array(littleEndian = false)
100importHex(hexString)
101exportHex(/*includeLastBits = true*/)

Advanced functions:

1clear(fullClear = false, sizeBits = 256 * 8)
2clone(copyStrictPrivateStructure = false)
3
4significantBit_setLsb()
5significantBit_setMsb()
6significantBit_set(significantBitType)//"LSB" or "MSB"
7significantBit_isLsb()
8significantBit_isMsb()
9significantBit_get()//"LSB" or "MSB"
10
11endianness_setLittleEndian()
12endianness_setBigEndian()
13endianness_set(endianness)//"LITTLE_ENDIAN" or "BIG_ENDIAN"
14endianness_isLittleEndian()
15endianness_isBigEndian()
16endianness_get()//"LITTLE_ENDIAN" or "BIG_ENDIAN"
17
18getStoredBits()
19getAvailableBitsToExpandRight()
20getAvailableBitsToPush()
21getAvailableBitsToUnshift()
22expandRight(expandBits = 256 * 8)
23expandLeft(expandBits = 256 * 8)
24expandRightIfNeed(checkPushBits, bitCountIfExpandRequired = 256 * 8)
25expandLeftIfNeed(checkUnshiftBits, bitCountIfExpandRequired = 256 * 8)	
26
27toString()
28

Tests:

Test1. Compare with DataView component (70 msec)

Only LSB.

Little endian and big endian.

1000 random offsets and values

1dataView.getUint32() vs .setAt_Uint(32) and .getAt_Uint(32)
2dataView.getInt32() vs .setAt_Int(32) and .getAt_Int(32)
3dataView.getUint16() vs .setAt_Uint(16) and .getAt_Uint(16)
4dataView.getUint8() vs .setAt_Uint(8) and .getAt_Uint(8)
5dataView.getUint8() vs .setAt_Byte(8) and .getAt_Byte(8)
6dataView.getBigUint64() vs .setAt_BigUint(64) and .getAt_BigUint(64)
7dataView.getFloat64() vs .setAt_Float64() and .getAt_Float64()
8dataView.getFloat32() vs .setAt_Float32() and .getAt_Float32()

Another libs another authors:

Warning, table has many mistakes!

Url	Lng	Addr bits	Var bits	LE BE	MSB LSB	types	Buffer Stack Queue Schema	Comment
https://www.npmjs.com/package/@telemok/bitdataview https://github.com/Telemok/bitdataview	JS	x1	x1	LE BE	MSB LSB	Bool (u)int float big(U)Int	Buffer Stack Queue	This library
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView	JS	x8	x8	LE BE		(u)int float big(U)Int	Buffer	Internal
https://nodejs.org/api/buffer.html	JS	x8	x8				Buffer	Internal
https://github.com/feross/buffer	JS	x8	x8				Buffer
https://github.com/JoshGlazebrook/smart-buffer	JS	x8	x8				Buffer
https://github.com/FlorianWendelborn/bitwise	JS	x1				Bool	Array	?Many memory
https://github.com/inolen/bit-buffer	JS	x8	x8				Buffer
https://github.com/rochars/byte-data	JS	x8	x8				Buffer
https://github.com/uupaa/Bit.js/	JS	x1	x1			Bool Uint	Buffer
https://github.com/fredricrylander/bits	JS	x1	x1			Bool Uint	Buffer
https://github.com/steinwurf/bitter	C++	x1	x1	LE	MSB LSB	Bool Uint	Buffer
https://github.com/conekt/bitsandbytes	JS	x8	x8	LE BE			Queue?
https://github.com/thi-ng/umbrella/tree/develop/packages/bitstream	JS	x1	x1	BE	LSB	Bool Uint	Queue
https://www.npmjs.com/package/node-bit-stream	JS	x8	x8
https://www.npmjs.com/package/bits-bytes	JS	x1	x1			Bool Uint	???
https://www.npmjs.com/package/@binary-format/binary-format	TS	x8	x8			Uint	Queue
https://www.npmjs.com/package/hipparchos	JS	x1	x8			Uint	Buffer
https://www.npmjs.com/package/@thi.ng/rle-pack	TS	x1	x1			Uint		no direct access
https://www.npmjs.com/package/binopsy	JS
https://www.npmjs.com/package/uint-buffer	JS	x8	x8	LE BE		Uint	Buffer	like DataView
https://github.com/keichi/binary-parser	JS	x8	x8	LE BE
https://github.com/anfema/bin-grammar	JS	x8	x8	LE BE		Uint	Schema
https://www.npmjs.com/package/@astronautlabs/bitstream	ES6	--	x1	BE	MSB LSB	Uint	Queue	Many interesting code.
https://github.com/RobertBorg/node-BinaryFormat	JS	x8	x8	???	???	???	Schema	C/compatible