obj-walker

Walk objects like this guy.

Map over an object in a preorder or postoder depth-first manner. Also, provides functions for serializing and deserializng self-referential objects using JSON pointer.

This library is designed to work well with functions that traverse an object in the same way JSON.stringify and JSON.parse do. Namely, preorder and postorder. To mimic that behavior entirely set the jsonCompat option to true.

Custom traversal functions are supported for some functions. This allows you to walk tree-like structures, such as a JSON schema, in a more efficient and logical way. Prefer walkEach in these scenarios.

map, walkEach, walkEachAsync, mapLeaves, compact, and truncate support the option modifyInPlace for in-place modification. Otherwise, the object is deep cloned.

1export interface MutationOption {
2    /** Set to true to modify the object instead of returning a new object. */
3    modifyInPlace?: boolean
4}

walker

1walker(obj: object, walkFn: WalkFn, options: Options = {}) => void

Generic walking fn that traverses an object in preorder (default) or postorder, calling walkFn for each node. Can be used directly, but probably shouldn't.

1export interface Node {
2    key: string | undefined
3    val: any
4    parents: any[]
5    path: string[]
6    isLeaf: boolean
7    isRoot: boolean
8}
9
10export interface Options {
11    postOrder?: boolean
12    jsonCompat?: boolean
13    traverse?(x: any): any
14}

1import { walker } from 'obj-walker'
2
3const obj = {
4    a: {
5        b: 23,
6        c: 24,
7    },
8    d: {
9        e: 'Bob',
10        f: [10, 20, 30],
11    },
12}
13
14const nodes: Node[] = []
15const walkFn = (node: Node) => {
16    nodes.push(node)
17}
18walker(obj, walkFn, options)
19nodes

Returns an array of nodes. Note this is how walk works, so prefer that fn.

1[
2  {
3    key: undefined,
4    parents: [],
5    val: { a: { b: 23, c: 24 }, d: { e: 'Bob', f: [10, 20, 30] } },
6    path: [],
7    isRoot: true,
8    isLeaf: false,
9  },
10  {
11    key: 'a',
12    val: { b: 23, c: 24 },
13    parents: [{ a: { b: 23, c: 24 }, d: { e: 'Bob', f: [10, 20, 30] } }],
14    path: ['a'],
15    isLeaf: false,
16    isRoot: false,
17  },
18  {
19    key: 'b',
20    val: 23,
21    parents: [
22      { b: 23, c: 24 },
23      { a: { b: 23, c: 24 }, d: { e: 'Bob', f: [10, 20, 30] } },
24    ],
25    path: ['a', 'b'],
26    isLeaf: true,
27    isRoot: false,
28  },
29  ...
30]

walk

1walk(obj: object, options: WalkOptions = {}) => Node[]

Walk an object. Returns an array of all nodes in the object in either preorder or postorder.

1export interface WalkOptions extends Options {
2    leavesOnly?: boolean
3}

1import { walk } from 'obj-walker'
2
3const obj = {
4    a: {
5        b: 23,
6        c: 24,
7    },
8    d: {
9        e: 'Bob',
10        f: [10, 20, 30],
11    },
12}
13walk(obj).map((x) => x.path)

Produces:

1[
2    [],
3    ["a"],
4    ["a", "b"],
5    ["a", "c"],
6    ["d"],
7    ["d", "e"],
8    ["d", "f"],
9    ["d", "f", "0"],
10    ["d", "f", "1"],
11    ["d", "f", "2"]
12]

walkEach

1walkEach(obj: object, walkFn: WalkFn, options: options?: WalkOptions & MutationOption) => object

1export type WalkFn = (node: Node) => void

Walk over an object calling walkFn for each node. The original object is deep-cloned by default making it possible to simply mutate each node as needed in order to transform the object. The cloned object is returned if options.modifyInPlace is not set to true.

Below I want to walk a MongoDB JSON schema and set additionalProperties to true wherever it exists. I traverse this tree using a custom traverse fn. The original object is not modified.

1import { walkEach } from 'obj-walker'
2
3const obj = {
4    bsonType: 'object',
5    additionalProperties: false,
6    required: ['name'],
7    properties: {
8        _id: {
9            bsonType: 'objectId',
10        },
11        name: { bsonType: 'string' },
12        addresses: {
13            bsonType: 'array',
14            items: {
15                bsonType: 'object',
16                additionalProperties: false,
17                properties: {
18                    address: {
19                        bsonType: 'object',
20                        additionalProperties: false,
21                        properties: {
22                            zip: { bsonType: 'string' },
23                            country: { bsonType: 'string' },
24                        },
25                    },
26                },
27            },
28        },
29    },
30}
31
32const traverse = (x: any) => x.properties || (x.items && { items: x.items })
33const walkFn = ({ val }: Node) => {
34    if ('additionalProperties' in val) {
35        val.additionalProperties = true
36    }
37}
38walkEach(obj, walkFn, { traverse })

Produces:

1{
2  bsonType: 'object',
3  additionalProperties: true,
4  required: ['name'],
5  properties: {
6    _id: { bsonType: 'objectId' },
7    name: { bsonType: 'string' },
8    addresses: {
9      bsonType: 'array',
10      items: {
11        bsonType: 'object',
12        additionalProperties: true,
13        properties: {
14          address: {
15            bsonType: 'object',
16            additionalProperties: true,
17            properties: {
18              zip: { bsonType: 'string' },
19              country: { bsonType: 'string' },
20            },
21          },
22        },
23      },
24    },
25  },
26}

walkEachAsync

Like walkEach but awaits the promise returned by walkFn before proceeding to the next node.

map

Map over an object modifying values with a fn depth-first in a preorder or postorder manner. The output of the mapper fn will be traversed if possible when traversing preorder.

By default, nodes will be excluded by returning undefined. Undefined array values will not be excluded. To customize pass a fn for options.shouldSkip.

1map(obj: object, mapper: Mapper, options?: MapOptions & MutationOption) => object

1export type Mapper = (node: Node) => any
2
3export type MapOptions = Omit<Options, 'traverse'> & {
4    shouldSkip?(val: any, node: Node): boolean
5}

1import { map } from 'obj-walker'
2
3const obj = {
4    a: {
5        b: 23,
6        c: 24,
7    },
8    d: {
9        e: 'Bob',
10        f: [10, null, 30, [31, undefined, 32], 40],
11    },
12    g: [25, '', { h: [null, 26, 27] }],
13    i: 'Frank',
14}
15map(obj, ({ val }) => (Array.isArray(val) ? _.compact(val) : val))

Produces:

1{
2  a: { b: 23, c: 24 },
3  d: { e: 'Bob', f: [10, 30, [31, 32], 40] },
4  g: [25, { h: [26, 27] }],
5  i: 'Frank',
6}

Postorder

1const obj = {
2    bob: {
3        scores: ['87', 'x97', 95, false],
4    },
5    joe: {
6        scores: [92, 92.5, '73.2', ''],
7    },
8    frank: {
9        scores: ['abc', ''],
10    },
11}
12const result = map(
13    obj,
14    ({ val, isLeaf }) => {
15        if (isLeaf) {
16            return parseFloat(val)
17        }
18        return Array.isArray(val) ? _.compact(val) : val
19    },
20    { postOrder: true }
21)

Produces:

1{
2  bob: { scores: [87, 95] },
3  joe: { scores: [92, 92.5, 73.2] },
4  frank: { scores: [] },
5}

Custom shouldSkip fn

1const obj = {
2    bob: {
3        scores: ['87', 'x97', 95, false],
4    },
5    joe: {
6        scores: [92, 92.5, '73.2', ''],
7    },
8    frank: {
9        scores: ['abc', ''],
10    },
11}
12
13const shouldSkip = (val: any, node: Node) =>
14    _.isEmpty(val) && !parentIsArray(node)
15const result = map(
16    obj,
17    ({ val, isLeaf }) => {
18        if (isLeaf) {
19            return parseFloat(val)
20        }
21        return Array.isArray(val) ? _.compact(val) : val
22    },
23    { postOrder: true, shouldSkip }
24)

Produces:

1{
2  bob: { scores: [87, 95] },
3  joe: { scores: [92, 92.5, 73.2] },
4}

mapLeaves

1mapLeaves(obj: object, mapper: Mapper, options?: MapOptions) => object

Map over the leaves of an object with a fn. By default, nodes will be excluded by returning undefined. Undefined array values will not be excluded. To customize pass a fn for options.shouldSkip.

1import { mapLeaves } from 'obj-walker'
2
3const obj = {
4    a: {
5        b: 23,
6        c: 24,
7    },
8    d: {
9        e: 100,
10        f: [10, 20, 30],
11    },
12}
13mapLeaves(obj, ({ val }) => val + 1)

Produces:

1{
2  a: { b: 24, c: 25 },
3  d: { e: 101, f: [11, 21, 31] },
4}

findNode

1findNode(obj: object, findFn: FindFn, options?: Options) => Node | undefined

Search for a node and short-circuit the tree traversal if it's found.

1import { findNode } from 'obj-walker'
2
3const obj = {
4    name: 'Joe',
5    address: {
6        city: 'New York',
7        state: 'NY',
8        zipCode: '10001',
9    },
10    likes: ['Stock Market', 'Running'],
11}
12
13findNode(obj, (node) => {
14    return _.isEqual(node.path, ['address', 'zipCode'])
15})

Produces:

1{
2  key: 'zipCode',
3  val: '10001',
4  parents: [
5    { city: 'New York', state: 'NY', zipCode: '10001' },
6    {
7      name: 'Joe',
8      address: { city: 'New York', state: 'NY', zipCode: '10001' },
9      likes: ['Stock Market', 'Running'],
10    },
11  ],
12  path: ['address', 'zipCode'],
13  isLeaf: true,
14  isRoot: false,
15}

flatten

1flatten(obj: object, options?: WalkOptions & FlattenOptions) => object

1interface FlattenOptions {
2    /** Defaults to '.' */
3    separator?: string
4    /** Flatten objects and not arrays */
5    objectsOnly?: boolean
6}

Flatten an object's keys. Optionally pass separator to determine what character to join keys with. Defaults to '.'. If an array is passed, an object of path to values is returned unless the objectsOnly option is set.

1import { flatten } from 'obj-walker'
2
3const obj = {
4    a: {
5        b: 23,
6        c: 24,
7    },
8    d: {
9        e: 100,
10        f: [10, 20, 30],
11    },
12}
13flatten(obj)

Produces:

1{
2  'a.b': 23,
3  'a.c': 24,
4  'd.e': 100,
5  'd.f.0': 10,
6  'd.f.1': 20,
7  'd.f.2': 30,
8}

unflatten

1unflatten(obj: object, options?: UnflattenOptions) => object

1interface UnflattenOptions {
2    /** Defaults to '.' */
3    separator?: string | RegExp
4}

Unflatten an object previously flattened. Optionally pass separator to determine what character or RegExp to split keys with. Defaults to '.'.

1import { unflatten } from 'obj-walker'
2
3const obj = {
4    'a.b': 23,
5    'a.c': 24,
6    'd.e': 100,
7    'd.f.0': 10,
8    'd.f.1': 20,
9    'd.f.2.g': 30,
10    'd.f.2.h.i': 40,
11}
12unflatten(obj)

Produces:

1{
2  a: {
3    b: 23,
4    c: 24,
5  },
6  d: {
7    e: 100,
8    f: [10, 20, { g: 30, h: { i: 40 } }],
9  },
10}

compact

1compact(obj: object, options: CompactOptions & MutationOption) => object

1interface CompactOptions {
2    removeUndefined?: boolean
3    removeNull?: boolean
4    removeEmptyString?: boolean
5    removeFalse?: boolean
6    removeNaN?: boolean
7    removeEmptyObject?: boolean
8    removeEmptyArray?: boolean
9    compactArrays?: boolean
10    removeFn?: (val: any, node: Node) => boolean
11}

Compact an object, removing fields recursively according to the supplied options. All option flags are false by default. If compactArrays is set to true, arrays will be compacted based on the enabled remove option flags.

1const obj = {
2    a: {
3        b: [null, null, 21, '', { b1: null }, { b2: 26 }],
4    },
5    c: [],
6    d: [42, null],
7    e: {
8        f: {
9            g: '',
10            h: undefined,
11            i: 'null',
12        },
13    },
14}
15const result = compact(obj, {
16    removeUndefined: true,
17    removeEmptyString: true,
18    removeNull: true,
19    compactArrays: true,
20    removeEmptyArray: true,
21    removeEmptyObject: true,
22    removeFn: (val: any) => val === 'null',
23})

Produces:

1{
2  a: { b: [21, { b2: 26 }] },
3  d: [42],
4}

truncate

1truncate(obj: object, options: TruncateOptions & MutationOption) => object

1export interface TruncateOptions {
2    /** Max allowed depth of objects/arrays. Default to Infinity */
3    maxDepth?: number
4    /** What to replace an object/array at the maximum depth with. Defaults to '[Truncated]' */
5    replacementAtMaxDepth?: any
6    /** Max allowed length of a string. Defaults to Infinity */
7    maxStringLength?: number
8    /** What to replace the last characters of the truncated string with. Defaults to '...' */
9    replacementAtMaxStringLength?: string
10    /** Max allowed length of an array. Defaults to Infinity */
11    maxArrayLength?: number
12    /** Transform instances of Error into plain objects so that truncation can be performed. Defautls to false */
13    transformErrors?: boolean
14}

Truncate allows you to limit the depth of nested objects/arrays, the length of strings, and the length of arrays. Instances of Error can be converted to plain objects so that the enabled truncation options also apply to the error fields. All truncation methods are opt-in.

Note: For the best performance you should consider setting modifyInPlace to true.

1const obj = {
2    a: {
3        b: 'Frank',
4        c: {
5            d: 'Joe',
6        },
7        e: null,
8    },
9    f: 42,
10}
11truncate(obj, { depth: 2 })

Produces:

1{
2  a: {
3    b: 'Frank',
4    c: '[Truncated]',
5    e: null,
6  },
7  f: 42,
8}

size

1size(val: any) => number

Estimate the size in bytes.

1const obj = {
2    a: {
3        b: 'hello',
4    },
5    c: Symbol('hello'),
6    d: {
7        e: [true, false],
8    },
9    f: [42, 10n],
10}
11size(obj)
12// 44

Helper fns

These helper fns are exported for your convenience.

1export const isObjectOrArray = _.overSome([_.isPlainObject, _.isArray])
2
3export const defShouldSkip = (val: any, node: Node) =>
4    val === undefined && !parentIsArray(node)
5
6export const parentIsArray = (node: Node) => {
7    const parent = node.parents[0]
8    return Array.isArray(parent)
9}
10
11export const defTraverse = (x: any) => isObjectOrArray(x) && !_.isEmpty(x) && x

addRefs

1addRefs(obj: object, options?: RefOptions): object

Replace duplicate objects refs with pointers to the first object seen.

1import { addRefs } from 'obj-walker'
2
3const apiOutput = {
4    1: 'foo',
5    2: 'bar',
6    3: 'baz',
7}
8
9const detailsOutput = {
10    1: 'bla',
11    2: 'bla',
12    3: 'bla',
13}
14
15const obj = {
16    api: {
17        input: [1, 2, 3],
18        output: apiOutput,
19    },
20    details: {
21        input: apiOutput,
22        output: detailsOutput,
23    },
24    writeToDB: {
25        input: detailsOutput,
26    },
27}
28addRefs(obj)

Produces:

1{
2  api: {
3    input: [1, 2, 3],
4    output: { '1': 'foo', '2': 'bar', '3': 'baz' },
5  },
6  details: {
7    input: { $ref: '#/api/output' },
8    output: { '1': 'bla', '2': 'bla', '3': 'bla' },
9  },
10  writeToDB: { input: { $ref: '#/details/output' } },
11}

deref

1deref(obj: object, options?: RefOptions): object

Rehydrate objects by replacing refs with actual objects.

1import { deref } from 'obj-walker'
2
3const obj = {
4    api: {
5        input: [1, 2, 3],
6        output: { '1': 'foo', '2': 'bar', '3': 'baz' },
7    },
8    details: {
9        input: { $ref: '#/api/output' },
10        output: { '1': 'bla', '2': 'bla', '3': 'bla' },
11    },
12    writeToDB: { input: { $ref: '#/details/output' } },
13}
14deref(obj)

Produces:

1{
2  api: {
3    input: [1, 2, 3],
4    output: { '1': 'foo', '2': 'bar', '3': 'baz' },
5  },
6  details: {
7    input: { '1': 'foo', '2': 'bar', '3': 'baz' },
8    output: { '1': 'bla', '2': 'bla', '3': 'bla' },
9  },
10  writeToDB: { input: { '1': 'bla', '2': 'bla', '3': 'bla' } },
11}