WSV

About WSV

The textual data format WSV - the Whitespace Separated Values format - is a modern and robust alternative to CSV. It has only a minimal set of rules and solves the main problems of CSV (watch this video for more details on that). It's human friendly and can produce documents that are beautifully formatted and readable, even without specific tools, just opened in a text editor. Here is an example WSV document:

FirstName LastName Age PlaceOfBirth
Lucas     Brown
William   Smith    30  Boston
Lucy      Reynolds 27
Olivia    Jones    -   "San Francisco"

In terms of data structure, it represents an array of arrays of string or null values. In the textual representation, values are separated by a single or consecutive whitespace. Values can contain whitespace as well and then must be escaped. Learn more about the escaping rules on the official website www.whitespacesv.com where you can find the complete specification and can try out WSV in an online editor.

WSV builds the foundation for text file formats like SML and TBL (see also the Stenway Text File Format Stack). All of these formats don't need to bother about encoding and decoding anymore, because they rely on ReliableTXT, which takes care of that aspect (see also the NPM packages reliabletxt, reliabletxt-io, and reliabletxt-browser).

Find out what can be done with WSV on the official YouTube channel from Stenway.

About this package

This package provides functionality to handle the parsing and serialization of WSV documents. It also provides functionality to encode and decode the binary version of WSV, which is called BinaryWSV. The package works both in the browser and Node.js, because it does not require environment specific functionality. If you want to read and write WSV files using Node.js's file system module, you can use the wsv-io package. The wsv-browser package on the other hand offers functionality to easily provide WSV documents as downloadable files.

If you want to get a first impression on how to use this package, you can watch this video. But always check the changelog of the presented packages for possible changes, that are not reflected in the video.

Getting started

We first have to install the Stenway WSV package with the npm install command.

npm install @stenway/wsv

We begin with creating a new WsvLine object, by passing an array of string values to the constructor. We call the toString method to serialize this WsvLine as a string.

1import { WsvLine } from "@stenway/wsv"
2const line = new WsvLine(["Value1", "Value2"])
3const lineStr = line.toString()

Both values are simply separated with a single space character:

Value1 Value2

We can change the formatting of the line, by providing an array of whitespace strings as argument to the WsvLine class constructor. In this example we indent the line with a tab character, expressed by the backslash t escape sequence and put two spaces between the first and the second value. We also add a comment by providing a string as a third argument.

1const lineWithComment = new WsvLine(["Value1", "Value2"], ["\t", "  ", " "], "My comment")
2const lineWithCommentStr = lineWithComment.toString()

These arguments are kept in the WsvLine object and the serialized line string looks like this:

	Value1  Value2 #My comment

The tab character is the first character and the comment string starts with a hash character.

We can access the values, whitespaces and the comment with their respective properties. And can of course change them, like here, where we change the first value and the comment.

1const values = lineWithComment.values
2const whitespaces = lineWithComment.whitespaces
3const comment = lineWithComment.comment
4
5lineWithComment.values[0] = "ChangedValue1"
6lineWithComment.comment = "Changed comment"
7const changedLineStr = lineWithComment.toString()

Which gives us:

	ChangedValue1  Value2 #Changed comment

Let's have a look at some of the special cases of values. Here we have a value with a space character, a value with a line feed character, a null value and a string containing doublequotes. There is also a single minus character.

1const normalValue = "Value"
2const valueWithSpace = "With space"
3const valueWithLineBreak = "With\nline break"
4const nullValue = null
5const valueWithDoublequotes = `With "Doublequotes"`
6const minusCharValue = "-"
7
8const specialValuesLine = new WsvLine([normalValue, valueWithSpace, valueWithLineBreak, nullValue, valueWithDoublequotes, minusCharValue])
9const specialValuesLineStr = specialValuesLine.toString()

Let's serialize these values and see how the resulting string will look like:

Value "With space" "With"/"line break" - "With ""Doublequotes""" "-"

The value containing the space character is put inside of doublequotes. The line feed character is replaced with this special escape sequence "/" consisting of a doublequote followed by a slash and another doublequote. The null value will result in a simple minus character. The doublequote character will be represented with two doublequotes. And the simple minus character will be enclosed by doublequotes.

Parsing

Of course the most import part is the parsing. With the static method parse of the WsvLine class, we can construct a WsvLine object from a serialized line string.

1const parsedLine = WsvLine.parse(lineStr)
2const parsedLineStr = parsedLine.toString()

The values will be correctly parsed and the serialized line string will look identical to the input of the parse method.

Now we try the same with the serialized line string, that contained whitespace formatting and a comment.

1const parsedLineWithComment = WsvLine.parse(lineWithCommentStr)
2const parsedLineWithCommentStr = parsedLineWithComment.toString()

The comment will be correctly parsed, as well as the whitespace strings. And again we will get an identical serialized string.

If you don't need to retain the formatting and comments, you can specifiy to not preserve them as a second argument to the parse method.

1const parsedLineWithoutWsAndComment = WsvLine.parse(lineWithCommentStr, false)
2const parsedLineWithoutWsAndCommentStr = parsedLineWithoutWsAndComment.toString()

And we can see, that the formatting and the comment will be stripped away.

There are case where we don't need a WsvLine class object and instead only need an array. That's where we can use the static method parseAsArray of the WsvLine class, which will return an array of string or null values.

1const emptyArray = WsvLine.parseAsArray("")
2const alsoEmpty = WsvLine.parseAsArray("     ")
3const valueArray = WsvLine.parseAsArray("Value1 Value2 - \"Value 4\"")

The first two serialized strings will lead to empty arrays and the third example will lead to an array of four values.

We can also use the static method serialize of the WsvLine class to get a serialized string from an array of values.

1const emptyArrayStr = WsvLine.serialize(emptyArray)
2const valueArrayStr = WsvLine.serialize(valueArray)

If the string we want to parse has an invalid WSV syntax, the parse method will throw a WsvParserError. In this example we will catch it and will have a look at it:

1try {
2	WsvLine.parse(`Value1 "Val`)
3} catch(error) {
4	const parserError = error as WsvParserError
5	console.log(`Parser error: ${parserError.message}`)
6}

The provided line string has a value that starts with a doublequote, but there is no closing doublequote. The parser error will therefor tell us, that the string is not closed and will give us information about the position where the error occured.

The WsvDocument class

So far we have only worked with single lines. Now we want to create a whole WSV document. For that we first create a new WsvDocument object and add two lines with the addLine method. We can either only provide values, or additionally provide formatting and comment strings. The toString method serializes the complete document as a single string.

1const document = new WsvDocument()
2document.addLine(["Value1_1", "Value1_2"])
3document.addLine(["Value2_1", null], ["\t", "  ", " "], "My comment")
4const documentStr = document.toString()

Because a WSV document is a ReliableTXT document, two lines are joined and separated by a line feed character as you can see here:

Value1_1 Value1_2
	Value2_1  - #My comment

We can also directly provide an array of lines to the WsvDocument constructor and additionally define the used ReliableTxtEncoding type. For that we also need to install and import the ReliableTXT package.

1const line1 = WsvLine.parse("a b")
2const line2 = WsvLine.parse("c d")
3const anotherDocument = new WsvDocument([line1, line2], ReliableTxtEncoding.Utf16)
4const anotherDocumentStr = anotherDocument.toString()

We can access the lines and encoding with their respective properties. With the toBytes method, we can convert the document into it's binary representation using the specified ReliableTXT encoding.

1const lines = anotherDocument.lines
2const encoding = anotherDocument.encoding
3const bytes = anotherDocument.toBytes()

Here we have specified, that the document will use the UTF-16 encoding and therefor we will see that the document is encoded as several 16-bit values with the first two bytes being the preamble bytes also called byte order mark.

To parse a complete document, we can use the static method parse of the WsvDocument class. And here we will see that the serialized string is identical to the input string of the parse method.

1const content = `a b # My first line
2                 c d`
3
4const parsedDocument = WsvDocument.parse(content)
5const parsedDocumentStr = parsedDocument.toString()

If we don't need whitespace and comments preserved, we can specify that with the second argument.

1const parsedDocumentWithoutWsAndComment = WsvDocument.parse(content, false)
2const parsedDocumentWithoutWsAndCommentStr = parsedDocumentWithoutWsAndComment.toString()

And finally if we only want an array of value arrays, we can simply use the static method parseAsJaggedArray of the WsvDocument class.

1const jaggedArray = WsvDocument.parseAsJaggedArray(`a b c d\ne\nf g - i j k`)

BinaryWSV

BinaryWSV is the binary representation of WSV documents. It starts with the magic code 'BW1'. BinaryWSV is made for scenarios, where parsing speed of the textual representation might be a limitation. It doesn't need to escape special string values, because it uses invalid UTF-8 bytes to signal line breaks, value separators, empty strings or null values, and thus non-empty string values can simply be encoded using UTF-8. The following bytes are used:

11111111  FF  255  =  Line Break Byte
11111110  FE  254  =  Value Separator Byte
11111101  FD  253  =  Null Value Byte
11111100  FC  252  =  Empty String Byte

Learn more about the BinaryWSV format in this video.

The package contains functionality to encode and decode BinaryWSV documents. The WsvDocument class offers the method toBinaryWsv which returns the WSV document encoded as BinaryWSV bytes.

1const document = WsvDocument.parse(`a b\nc - ""`)
2const bytes = document.toBinaryWsv()
3const decodedDocument = WsvDocument.fromBinaryWsv(bytes)

The static method fromBinaryWsv on the other hand returns a WsvDocument object which was decoded from the provided BinaryWSV bytes.

If you don't need a WsvDocument and instead want to encode a jagged array, you can use the the static BinaryWsvEncoder class and its static method encodeJaggedArray.

1const bytes = BinaryWsvEncoder.encodeJaggedArray([["a", "b"], ["c", null, ""]])
2const decodedJaggedArray = BinaryWsvDecoder.decodeAsJaggedArray(bytes)

The static method decodeAsJaggedArray of the static BinaryWsvDecoder class will then turn the BinaryWSV bytes into a jagged array.