Alinex Format

This package will give you an easy way to serialize and deserialize different data objects into multiple formats.

The major features are:

• easy to use
• load format libraries on demand
• auto detection

It is one of the modules of the Alinex Universe following the code standards defined in the General Docs.

Install

The easiest way is to let npm add the module directly to your modules (from within you node modules directory):

1npm install alinex-format --save

And update it to the latest version later:

1npm update alinex-format --save

Always have a look at the latest changes.

Usage

To use the format you have to load the module first:

1format = require 'alinex-format'

This gives you back the main format instance which holds two methods to use:

stringify

Serialize data object into string.

Arguments:

• obj object to be formatted
• format format to use or filename to read format from
• options (optional) specific for the format
• cb callback will be called with (err, text)

parse

Try to parse object from string. Auto detect if no format is given.

Arguments:

• string text to be parsed
• format (optional) format to use or filename to read format from
• cb callback will be called with (err, object)

Formats

This configuration class allows multiple formats to be used.

The following table will give a short comparison.

Format	Readable	Comments	Arrays	Deep	Calc	Ref.
JSON	++	no	yes	yes	no	no
JS	++	allow	yes	yes	read	no
CSON	+++	allow	yes	yes	no	no
Coffee	+++	allow	yes	yes	read	no
YAML	+++	allow	yes	yes	no	read
INI	++	allow	yes	yes	no	no
Properties	++	allow	yes	yes	no	read
XML	+	allow	yes	yes	no	no
BSON	--	no	yes	yes	no	no
CSV	+	(no)	yes	(yes)	no	no

Legend: +++ to --- = good to bad; no = not possible; allow = allowed but unused; read = only red but not written; write = only written but not red; yes = fully supported; ? = unknown

See details for each format below.

Some of the formats support comments but they won't read or write them, they only will allow them to be there in the file.

JSON

This format uses the javascript object notation a human readable structure. It is widely used in different languages not only JavaScript. See description at Wikipedia.

Common file extension json.

1{
2  "null": null,
3  "boolean": true,
4  "string": "test",
5  "number": 5.6,
6  "date": "2016-05-10T19:06:36.909Z",
7  "list": [1, 2, 3],
8  "person": {
9    "name": "Alexander Schilling",
10    "job": "Developer"
11  },
12  "complex": [
13    {"name": "Egon"},
14    {"name": "Janina"}
15  ]
16}

JSON's basic data types are:

• Number: a signed decimal number that may contain a fractional part and may use exponential E notation, but cannot include non-numbers like NaN.
• String: a sequence of zero or more Unicode characters. Strings are delimited with double-quotation marks and support a backslash escaping syntax.
• Boolean: either of the values true or false
• Array: an ordered list of zero or more values, each of which may be of any type. Arrays use square bracket notation with elements being comma-separated.
• Object: an unordered collection of name/value pairs where the keys are strings. Objects are delimited with curly brackets and use commas to separate each pair, while within each pair the colon ':' character separates the key or name from its value.
• Date: will be formatted as ISO string and stay as string after parsing
• null: An empty value, using the word null

Whitespace (space, horizontal tab, line feed, and carriage return) is allowed and ignored around or between syntactic elements.

JSON won't allow comments but you may use JavaScript like comments using // and /*...*/ like known in javascript. Therefore use the javascript parsing described below.

Format Options:

• indent - number of spaces or text to indent each level (defaults to 2 spaces)

With indent: 0 the above example would look like:

1{"null":null,"boolean":true,"string":"test","number":5.6,"date":"2016-05-10T19:06:36.909Z","list":[1,2,3],"person":{"name":"Alexander Schilling","job":"Developer"}}

JavaScript

Also allowed are normal JavaScript files. In comparison to the JSON format it is more loosely so you may use single quotes, keys don't need quotes at all and at last you may use calculations. But you may only access elements in the same file accessing data from outside is prevented by security.

Common file extension js.

1// use an object
2{
3  // null value
4  null: null,
5  // boolean setting
6  boolean: true,
7  // include a string
8  string: 'test',
9  // any integer or float number
10  number: 5.6,
11  // a date as string
12  date: "2016-05-10T19:06:36.909Z",
13  // and a list of numbers
14  list: [1, 2, 3],
15  // add a sub object
16  person: {
17    name: "Alexander Schilling",
18    job: "Developer"
19  },
20  // complex list with object
21  complex: [
22    {name: 'Egon'},
23    {name: 'Janina'}
24  ],
25  // calculate session timeout in milliseconds
26  calc: 15*60*1000,
27  math: Math.sqrt(16)
28}

Format Options:

• indent - number of spaces or text to indent each level (defaults to 2 spaces)

CSON

Like JSON but here the object is defined using CoffeeScript instead of javascript.

Common file extension cson.

1# null value
2null: null
3# boolean values
4boolean: true
5# include a string
6string: 'test'
7date: '2016-05-10T19:06:36.909Z'
8# numbers
9numberInt: -8
10numberFloat: 5.6
11# and a list of numbers
12list: [1, 2, 3]
13list2: [
14  1
15  2
16  3
17]
18# add a sub object
19person:
20  name: 'Alexander Schilling'
21  job: 'Developer'
22# complex list with object
23complex: [
24  name: 'Egon'
25,
26  name: 'Janina'
27]
28# Multi-Line Strings! Without Quote Escaping!
29emissions: '''
30  Livestock and their byproducts account for at least 32,000 million tons of carbon dioxide (CO2) per year, or 51% of all worldwide greenhouse gas emissions.
31  Goodland, R Anhang, J. “Livestock and Climate Change: What if the key actors in climate change were pigs, chickens and cows?”
32  WorldWatch, November/December 2009. Worldwatch Institute, Washington, DC, USA. Pp. 10–19.
33  http://www.worldwatch.org/node/6294
34  '''

CSON solves several major problems with hand-writing JSON by providing:

• the ability to use both single-quoted and double-quoted strings
• the ability to write multi-line strings in multiple lines
• the ability to write a redundant comma
• comments start with # and are allowed

Besides this facts it's the same as JSON and have the same types.

Format Options:

• indent - number of spaces or text to indent each level (defaults to 2 spaces)

Coffee

The Coffee Script format is nearly the same as CSON but caused by the changed parser it may contain calculations, too.

Common file extension coffee.

1null: null
2boolean: true
3# include a string
4string: 'test'
5number: 5.6
6date: '2016-05-10T19:06:36.909Z'
7# and a list of numbers
8list: [
9  1
10  2
11  3
12]
13# add a sub object
14person:
15  name: 'Alexander Schilling'
16  job: 'Developer'
17# complex structure
18complex: [
19  {name: 'Egon'}
20  {name: 'Janina'}
21]
22# Multi-Line Strings! Without Quote Escaping!
23emissions: '''
24  Livestock and their byproducts account for at least 32,000 million tons of carbon dioxide (CO2) per year, or 51% of all worldwide greenhouse gas emissions.
25  Goodland, R Anhang, J. “Livestock and Climate Change: What if the key actors in climate change were pigs, chickens and cows?”
26  WorldWatch, November/December 2009. Worldwatch Institute, Washington, DC, USA. Pp. 10–19.
27  http://www.worldwatch.org/node/6294
28  '''
29# calculate session timeout in milliseconds
30calc: 15*60*1000
31math: Math.sqrt 16

Format Options:

• indent - number of spaces or text to indent each level (defaults to 2 spaces)

YAML

This is a simplified and best human readable language to write structured information. See some examples at Wikipedia.

Common file extensions yml or yaml.

Example

1# null value
2null: null
3# boolean values
4boolean: true
5# include a string
6string: test
7unicode: "Sosa did fine.\u263A"
8control: "\b1998\t1999\t2000\n"
9hex esc: "\x0d\x0a is \r\n"
10single: '"Howdy!" he cried.'
11quoted: ' # Not a ''comment''.'
12# date support
13date: 2016-05-10T19:06:36.909Z
14# numbers
15numberInt: -8
16numberFloat: 5.6
17octal: 0o14
18hexadecimal: 0xC
19exponential: 12.3015e+02
20fixed: 1230.15
21negative infinity: -.inf
22not a number: .NaN
23# and a list of numbers
24list: [one, two, three]
25list2:
26  - one
27  - two
28  - three
29# add a sub object
30person:
31  name: Alexander Schilling
32  job: Developer
33# complex list with object
34complex:
35  - name: Egon
36  - {name: Janina}
37# multiline support
38multiline:
39  This text will be read
40  as one line without
41  linebreaks.
42multilineQuoted: "This text will be read
43  as one line without
44  linebreaks."
45lineBreaks: |
46  This text will keep
47  as it is and all line
48  breaks will be kept.
49lineSingle: >
50  This text will be read
51  as one line without
52  linebreaks.
53lineBreak: >
54  The empty line
55
56  will be a line break.
57# use references
58address1: &adr001
59  city: Stuttgart
60address2: *adr001
61# specific type casts
62numberString: "123"
63numberString2: !!str 123
64#numberFloat: !!float 123
65# binary type
66picture: !!binary |
67  R0lGODdhDQAIAIAAAAAAANn
68  Z2SwAAAAADQAIAAACF4SDGQ
69  ar3xxbJ9p0qa7R0YxwzaFME
70  1IAADs=
71# complex mapping key
72? - Detroit Tigers
73  - Chicago cubs
74: 2001-07-23

The YAML syntax is very powerful but also easy to write in it's basics:

• comments allowed starting with #
• dates are allowed as ISO string, too
• different multiline text entries
• special number values
• referenceswith *xxx to the defined '&xxx' anchor

See the example above.

INI

This is one of the oldest formats used for configurations. It is very simple but allows also complex objects through extended groups.

Common file extension ini.

1; simple text
2string = test
3
4; add a simple list
5list[] = 1
6list[] = 2
7list[] = 3
8
9; add a group
10[person]
11name = Alexander Schilling
12job = Developer
13
14; add a subgroup
15[city.address]
16name = Stuttgart

Comments start with semicolon and grous/sections are marked by square brackets. The group name defines the object to add the properties to.

Format Options:

• whitespace - (boolean) should spaces be put arround = (defaults to true)

PROPERTIES

Mainly in the Java world properties are used to setup configuration values. But it won't have support for arrays, you only may use objects with numbered keys.

Common file extension properties.

1# strings
2string = test
3other text
4multiline This text \
5  goes over multiple lines.
6
7# numbers
8integer = 15
9float: -4.6
10
11! add a simple list
12list.1 = one
13list.2 = two
14list.3 = three
15
16! add a sub object
17person.name: Alexander Schilling
18person.job: Developer
19
20#references
21ref = ${string}
22
23# add a section
24[section]
25name = Alex
26same = ${section|name}

This format supports:

• key and value may be divided by =, : with spaces or only a space
• comments start with ! or #
• structure data using sections with square brackets like in ini files
• structure data using namespaces in the key using dot as seperator
• references to other values with ${key}
• references work also as section names or reference name

XML

The XML format should only use Tags and values, but no arguments.

Common file extension xml.

1<?xml version="1.0" encoding="UTF-8" ?>
2<!-- use an object -->
3<xml>
4
5  <!-- include a string -->
6  <name>test</name>
7
8  <!-- and a list of numbers -->
9  <list>1</list>
10  <list>2</list>
11  <list>3</list>
12
13  <!-- sub object -->
14  <person>
15    <name>Alexander Schilling</name><job>Developer</job>
16  </person>
17
18  <!-- cdata section -->
19  <cdata><![CDATA[i'm not escaped: <xml>!]]></cdata>
20
21  <!-- using attributes -->
22  <attributes type="detail">
23    Hello all together
24    <sub>And specially you!</sub>
25  </attributes>
26
27</xml>

Format Options:

• root - (string) name of the root element (default: 'xml')

Parse Options:

• explicitRoot - (boolean) keep the root element (default: false)
• ignoreAttrs - (boolean) ignore attribute settings (default: false)

BSON

Binary JSON is a more compressed version of JSON but not human readable because it's a binary format. It is mainly used in the MongoDB database.

Common file extension bson.

CSV

The CSV format should only be used with table like data which is in the form of a list of lists. See the table package to transform and work with such data.

Autodetection is not possible here.

Common file extension csv.

1num;type;object
21;null;
32;undefined;
43;boolean;1
54;number;5.6
65;text;Hello
76;quotes;"Give me a ""hand up"""
87;date;128182440000
98;list;[1,2,3]
109;object;"{""name"":""Egon""}"
1110;complex;"[{""name"":""Valentina""},{""name"":""Nadine""},{""name"":""Sven""}]"

While some types are fully supported: string, number

Others are partly supported and won't be automatically detectable:

• boolean as integer
• date as unix time integer
• null, undefined and empty strings are stored the same way and wil be red as null

And lastly complex sub objects will be stored as JSON text and be automatically parsed on read again.

Format Options:

• columns List of fields, applied when transform returns an object, order matters, read the transformer documentation for additionnal information, columns are auto discovered in the first record when the user write objects, can refer to nested properties of the input JSON, see the "header" option on how to print columns names on the first line.
• delimiter Set the field delimiter (default: ';')
• escape - (char) Set the escape character (Default: '"')
• quote - (char) Optionnal character surrounding a field, one character only (Default: '"')
• quoted - (boolean) quote all the non-empty fields even if not required (default: false)
• quotedEmpty - (boolean) quote empty fields? (default: false)
• quotedString - (boolean) quote all fields of type string even if not required (default: false)

Parse Options:

• delimiter - (char) Set the field delimiter (default: ';')
• quote - (char) Optionnal character surrounding a field, one character only (Default: '"')
• escape - (char) Set the escape character (Default: '"')
• comment - (char) Treat all the characters after this one as a comment, default to '' (disabled).

License

Copyright 2016 Alexander Schilling

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.