AST Types

This module provides an efficient, modular, Esprima-compatible implementation of the abstract syntax tree type hierarchy pioneered by the Mozilla Parser API.

Installation

From NPM:

npm install ast-types

From GitHub:

cd path/to/node_modules
git clone git://github.com/benjamn/ast-types.git
cd ast-types
npm install .

Basic Usage

1import assert from "assert";
2import {
3  namedTypes as n,
4  builders as b,
5} from "ast-types";
6
7var fooId = b.identifier("foo");
8var ifFoo = b.ifStatement(fooId, b.blockStatement([
9  b.expressionStatement(b.callExpression(fooId, []))
10]));
11
12assert.ok(n.IfStatement.check(ifFoo));
13assert.ok(n.Statement.check(ifFoo));
14assert.ok(n.Node.check(ifFoo));
15
16assert.ok(n.BlockStatement.check(ifFoo.consequent));
17assert.strictEqual(
18  ifFoo.consequent.body[0].expression.arguments.length,
19  0,
20);
21
22assert.strictEqual(ifFoo.test, fooId);
23assert.ok(n.Expression.check(ifFoo.test));
24assert.ok(n.Identifier.check(ifFoo.test));
25assert.ok(!n.Statement.check(ifFoo.test));

AST Traversal

Because it understands the AST type system so thoroughly, this library is able to provide excellent node iteration and traversal mechanisms.

If you want complete control over the traversal, and all you need is a way of enumerating the known fields of your AST nodes and getting their values, you may be interested in the primitives getFieldNames and getFieldValue:

1import {
2  getFieldNames,
3  getFieldValue,
4} from "ast-types";
5
6const partialFunExpr = { type: "FunctionExpression" };
7
8// Even though partialFunExpr doesn't actually contain all the fields that
9// are expected for a FunctionExpression, types.getFieldNames knows:
10console.log(getFieldNames(partialFunExpr));
11// [ 'type', 'id', 'params', 'body', 'generator', 'expression',
12//   'defaults', 'rest', 'async' ]
13
14// For fields that have default values, types.getFieldValue will return
15// the default if the field is not actually defined.
16console.log(getFieldValue(partialFunExpr, "generator"));
17// false

Two more low-level helper functions, eachField and someField, are defined in terms of getFieldNames and getFieldValue:

1// Iterate over all defined fields of an object, including those missing
2// or undefined, passing each field name and effective value (as returned
3// by getFieldValue) to the callback. If the object has no corresponding
4// Def, the callback will never be called.
5export function eachField(object, callback, context) {
6  getFieldNames(object).forEach(function(name) {
7    callback.call(this, name, getFieldValue(object, name));
8  }, context);
9}
10
11// Similar to eachField, except that iteration stops as soon as the
12// callback returns a truthy value. Like Array.prototype.some, the final
13// result is either true or false to indicates whether the callback
14// returned true for any element or not.
15export function someField(object, callback, context) {
16  return getFieldNames(object).some(function(name) {
17    return callback.call(this, name, getFieldValue(object, name));
18  }, context);
19}

So here's how you might make a copy of an AST node:

1import { eachField } from "ast-types";
2const copy = {};
3eachField(node, function(name, value) {
4  // Note that undefined fields will be visited too, according to
5  // the rules associated with node.type, and default field values
6  // will be substituted if appropriate.
7  copy[name] = value;
8})

But that's not all! You can also easily visit entire syntax trees using the powerful types.visit abstraction.

Here's a trivial example of how you might assert that arguments.callee is never used in ast:

1import assert from "assert";
2import {
3  visit,
4  namedTypes as n,
5} from "ast-types";
6
7visit(ast, {
8  // This method will be called for any node with .type "MemberExpression":
9  visitMemberExpression(path) {
10    // Visitor methods receive a single argument, a NodePath object
11    // wrapping the node of interest.
12    var node = path.node;
13
14    if (
15      n.Identifier.check(node.object) &&
16      node.object.name === "arguments" &&
17      n.Identifier.check(node.property)
18    ) {
19      assert.notStrictEqual(node.property.name, "callee");
20    }
21
22    // It's your responsibility to call this.traverse with some
23    // NodePath object (usually the one passed into the visitor
24    // method) before the visitor method returns, or return false to
25    // indicate that the traversal need not continue any further down
26    // this subtree.
27    this.traverse(path);
28  }
29});

Here's a slightly more involved example of transforming ...rest parameters into browser-runnable ES5 JavaScript:

1import { builders as b, visit } from "ast-types";
2
3// Reuse the same AST structure for Array.prototype.slice.call.
4var sliceExpr = b.memberExpression(
5  b.memberExpression(
6    b.memberExpression(
7      b.identifier("Array"),
8      b.identifier("prototype"),
9      false
10    ),
11    b.identifier("slice"),
12    false
13  ),
14  b.identifier("call"),
15  false
16);
17
18visit(ast, {
19  // This method will be called for any node whose type is a subtype of
20  // Function (e.g., FunctionDeclaration, FunctionExpression, and
21  // ArrowFunctionExpression). Note that types.visit precomputes a
22  // lookup table from every known type to the appropriate visitor
23  // method to call for nodes of that type, so the dispatch takes
24  // constant time.
25  visitFunction(path) {
26    // Visitor methods receive a single argument, a NodePath object
27    // wrapping the node of interest.
28    const node = path.node;
29
30    // It's your responsibility to call this.traverse with some
31    // NodePath object (usually the one passed into the visitor
32    // method) before the visitor method returns, or return false to
33    // indicate that the traversal need not continue any further down
34    // this subtree. An assertion will fail if you forget, which is
35    // awesome, because it means you will never again make the
36    // disastrous mistake of forgetting to traverse a subtree. Also
37    // cool: because you can call this method at any point in the
38    // visitor method, it's up to you whether your traversal is
39    // pre-order, post-order, or both!
40    this.traverse(path);
41
42    // This traversal is only concerned with Function nodes that have
43    // rest parameters.
44    if (!node.rest) {
45      return;
46    }
47
48    // For the purposes of this example, we won't worry about functions
49    // with Expression bodies.
50    n.BlockStatement.assert(node.body);
51
52    // Use types.builders to build a variable declaration of the form
53    //
54    //   var rest = Array.prototype.slice.call(arguments, n);
55    //
56    // where `rest` is the name of the rest parameter, and `n` is a
57    // numeric literal specifying the number of named parameters the
58    // function takes.
59    const restVarDecl = b.variableDeclaration("var", [
60      b.variableDeclarator(
61        node.rest,
62        b.callExpression(sliceExpr, [
63          b.identifier("arguments"),
64          b.literal(node.params.length)
65        ])
66      )
67    ]);
68
69    // Similar to doing node.body.body.unshift(restVarDecl), except
70    // that the other NodePath objects wrapping body statements will
71    // have their indexes updated to accommodate the new statement.
72    path.get("body", "body").unshift(restVarDecl);
73
74    // Nullify node.rest now that we have simulated the behavior of
75    // the rest parameter using ordinary JavaScript.
76    path.get("rest").replace(null);
77
78    // There's nothing wrong with doing node.rest = null, but I wanted
79    // to point out that the above statement has the same effect.
80    assert.strictEqual(node.rest, null);
81  }
82});

Here's how you might use types.visit to implement a function that determines if a given function node refers to this:

1function usesThis(funcNode) {
2  n.Function.assert(funcNode);
3  var result = false;
4
5  visit(funcNode, {
6    visitThisExpression(path) {
7      result = true;
8
9      // The quickest way to terminate the traversal is to call
10      // this.abort(), which throws a special exception (instanceof
11      // this.AbortRequest) that will be caught in the top-level
12      // types.visit method, so you don't have to worry about
13      // catching the exception yourself.
14      this.abort();
15    },
16
17    visitFunction(path) {
18      // ThisExpression nodes in nested scopes don't count as `this`
19      // references for the original function node, so we can safely
20      // avoid traversing this subtree.
21      return false;
22    },
23
24    visitCallExpression(path) {
25      const node = path.node;
26
27      // If the function contains CallExpression nodes involving
28      // super, those expressions will implicitly depend on the
29      // value of `this`, even though they do not explicitly contain
30      // any ThisExpression nodes.
31      if (this.isSuperCallExpression(node)) {
32        result = true;
33        this.abort(); // Throws AbortRequest exception.
34      }
35
36      this.traverse(path);
37    },
38
39    // Yes, you can define arbitrary helper methods.
40    isSuperCallExpression(callExpr) {
41      n.CallExpression.assert(callExpr);
42      return this.isSuperIdentifier(callExpr.callee)
43          || this.isSuperMemberExpression(callExpr.callee);
44    },
45
46    // And even helper helper methods!
47    isSuperIdentifier(node) {
48      return n.Identifier.check(node.callee)
49          && node.callee.name === "super";
50    },
51
52    isSuperMemberExpression(node) {
53      return n.MemberExpression.check(node.callee)
54          && n.Identifier.check(node.callee.object)
55          && node.callee.object.name === "super";
56    }
57  });
58
59  return result;
60}

As you might guess, when an AbortRequest is thrown from a subtree, the exception will propagate from the corresponding calls to this.traverse in the ancestor visitor methods. If you decide you want to cancel the request, simply catch the exception and call its .cancel() method. The rest of the subtree beneath the try-catch block will be abandoned, but the remaining siblings of the ancestor node will still be visited.

NodePath

The NodePath object passed to visitor methods is a wrapper around an AST node, and it serves to provide access to the chain of ancestor objects (all the way back to the root of the AST) and scope information.

In general, path.node refers to the wrapped node, path.parent.node refers to the nearest Node ancestor, path.parent.parent.node to the grandparent, and so on.

Note that path.node may not be a direct property value of path.parent.node; for instance, it might be the case that path.node is an element of an array that is a direct child of the parent node:

1path.node === path.parent.node.elements[3]

in which case you should know that path.parentPath provides finer-grained access to the complete path of objects (not just the Node ones) from the root of the AST:

1// In reality, path.parent is the grandparent of path:
2path.parentPath.parentPath === path.parent
3
4// The path.parentPath object wraps the elements array (note that we use
5// .value because the elements array is not a Node):
6path.parentPath.value === path.parent.node.elements
7
8// The path.node object is the fourth element in that array:
9path.parentPath.value[3] === path.node
10
11// Unlike path.node and path.value, which are synonyms because path.node
12// is a Node object, path.parentPath.node is distinct from
13// path.parentPath.value, because the elements array is not a
14// Node. Instead, path.parentPath.node refers to the closest ancestor
15// Node, which happens to be the same as path.parent.node:
16path.parentPath.node === path.parent.node
17
18// The path is named for its index in the elements array:
19path.name === 3
20
21// Likewise, path.parentPath is named for the property by which
22// path.parent.node refers to it:
23path.parentPath.name === "elements"
24
25// Putting it all together, we can follow the chain of object references
26// from path.parent.node all the way to path.node by accessing each
27// property by name:
28path.parent.node[path.parentPath.name][path.name] === path.node

These NodePath objects are created during the traversal without modifying the AST nodes themselves, so it's not a problem if the same node appears more than once in the AST (like Array.prototype.slice.call in the example above), because it will be visited with a distict NodePath each time it appears.

Child NodePath objects are created lazily, by calling the .get method of a parent NodePath object:

1// If a NodePath object for the elements array has never been created
2// before, it will be created here and cached in the future:
3path.get("elements").get(3).value === path.value.elements[3]
4
5// Alternatively, you can pass multiple property names to .get instead of
6// chaining multiple .get calls:
7path.get("elements", 0).value === path.value.elements[0]

NodePath objects support a number of useful methods:

1// Replace one node with another node:
2var fifth = path.get("elements", 4);
3fifth.replace(newNode);
4
5// Now do some stuff that might rearrange the list, and this replacement
6// remains safe:
7fifth.replace(newerNode);
8
9// Replace the third element in an array with two new nodes:
10path.get("elements", 2).replace(
11  b.identifier("foo"),
12  b.thisExpression()
13);
14
15// Remove a node and its parent if it would leave a redundant AST node:
16//e.g. var t = 1, y =2; removing the `t` and `y` declarators results in `var undefined`.
17path.prune(); //returns the closest parent `NodePath`.
18
19// Remove a node from a list of nodes:
20path.get("elements", 3).replace();
21
22// Add three new nodes to the beginning of a list of nodes:
23path.get("elements").unshift(a, b, c);
24
25// Remove and return the first node in a list of nodes:
26path.get("elements").shift();
27
28// Push two new nodes onto the end of a list of nodes:
29path.get("elements").push(d, e);
30
31// Remove and return the last node in a list of nodes:
32path.get("elements").pop();
33
34// Insert a new node before/after the seventh node in a list of nodes:
35var seventh = path.get("elements", 6);
36seventh.insertBefore(newNode);
37seventh.insertAfter(newNode);
38
39// Insert a new element at index 5 in a list of nodes:
40path.get("elements").insertAt(5, newNode);

Scope

The object exposed as path.scope during AST traversals provides information about variable and function declarations in the scope that contains path.node. See scope.ts for its public interface, which currently includes .isGlobal, .getGlobalScope(), .depth, .declares(name), .lookup(name), and .getBindings().

Custom AST Node Types

The ast-types module was designed to be extended. To that end, it provides a readable, declarative syntax for specifying new AST node types, based primarily upon the require("ast-types").Type.def function:

1import {
2  Type,
3  builtInTypes,
4  builders as b,
5  finalize,
6} from "ast-types";
7
8const { def } = Type;
9const { string } = builtInTypes;
10
11// Suppose you need a named File type to wrap your Programs.
12def("File")
13  .bases("Node")
14  .build("name", "program")
15  .field("name", string)
16  .field("program", def("Program"));
17
18// Prevent further modifications to the File type (and any other
19// types newly introduced by def(...)).
20finalize();
21
22// The b.file builder function is now available. It expects two
23// arguments, as named by .build("name", "program") above.
24const main = b.file("main.js", b.program([  // Pointless program contents included for extra color.  b.functionDeclaration(b.identifier("succ"), [    b.identifier("x")  ], b.blockStatement([    b.returnStatement(      b.binaryExpression(        "+", b.identifier("x"), b.literal(1)      )    )  ]))
25]));
26
27assert.strictEqual(main.name, "main.js");
28assert.strictEqual(main.program.body[0].params[0].name, "x");
29// etc.
30
31// If you pass the wrong type of arguments, or fail to pass enough
32// arguments, an AssertionError will be thrown.
33
34b.file(b.blockStatement([]));
35// ==> AssertionError: {"body":[],"type":"BlockStatement","loc":null} does not match type string
36
37b.file("lib/types.js", b.thisExpression());
38// ==> AssertionError: {"type":"ThisExpression","loc":null} does not match type Program

The def syntax is used to define all the default AST node types found in babel-core.ts, babel.ts, core.ts, es-proposals.ts, es6.ts, es7.ts, es2020.ts, esprima.ts, flow.ts, jsx.ts, type-annotations.ts, and typescript.ts, so you have no shortage of examples to learn from.