Yuku

Semantic Analysis

Semantic analysis turns a tree into a model of the program: every scope, symbol, and reference, with each reference resolved to the binding it names. The same pass reports the early errors parsing alone cannot catch, such as redeclarations and private fields used outside their class, so together with the parser it covers every early error the specification requires.

It is a separate pass, so the parse stays fast and only the tools that need semantics pay for them.

JavaScript

npm install yuku-analyzer

import { Analyzer } from "yuku-analyzer";

const project = new Analyzer();

project.setFile("a.ts", `export const value = 1;`);
project.setFile("b.ts", `export { value as renamed } from "./a.ts";`);
project.setFile("c.ts", `import { renamed } from "./b.ts"; renamed;`);

project.module("c.ts").rootScope.find("renamed").definition().binding.name;
// "value"

The same model as plain objects, computed natively, with every import linked across a project. See yuku-analyzer on npm for the full API.

Zig

var tree = try parser.parse(allocator, source, .{});
defer tree.deinit();

const semantic = try parser.semantic.analyze(&tree);

analyze returns the Semantic model and appends its errors to tree.diagnostics. The model lives in the tree’s arena and describes the tree as parsed, so analyze again after a transform. To run your own hooks during the analysis walk, use the semantic traverser.

The model

  • A scope is a lexical environment, with its kind, its parent, its strictness, and the scope its vars hoist to.
  • A symbol is a declared binding, with its flags and every declaration.
  • A reference is a use of a name, resolved to its symbol, or to .none for a global or undeclared name. It records whether it writes, as assignments, ++, and for-of targets do. JSX component tags and type-predicate parameters are references too, so a rename sees every site.

Every node leads into the model, and every symbol leads back to its nodes.

semantic.symbolOf(node)             // the symbol a node declares or refers to, or null
semantic.referenceOf(node)          // the reference at an identifier, or null
semantic.scopeOf(node)              // the innermost scope around a node
semantic.parentOf(node)             // its parent, or null at the root
semantic.ancestors(node)            // the node, then each parent up to the root

semantic.symbol(id)                 // and scope(id), reference(id)
semantic.decls(symbol)              // each declaration's name node
semantic.uses(symbol)               // each reference to it

semantic.bindings(scope)            // the symbols a scope declares
semantic.binding(scope, "x")        // one of them by name
semantic.lookup(scope, "x", .value) // name resolution from any scope

semantic.iterSymbols()              // and iterScopes, iterReferences

Symbol, Scope, and Reference are in parser.traverser.semantic. A declaration’s name node is an identifier, or a string literal for a quoted enum member. Ids are dense and ordered, symbols by first declaration and references by source position, so “first declaration” and “earliest use” are index comparisons. Renaming a binding everywhere is two loops.

const name = try tree.addString("renamed");
for (semantic.decls(symbol)) |node| tree.setIdentifierName(node, name);
for (semantic.uses(symbol)) |ref| tree.setIdentifierName(semantic.reference(ref).node, name);

Unused bindings and undeclared names are one loop each.

var symbols = semantic.iterSymbols();
while (symbols.next()) |entry| {
    if (semantic.uses(entry.id).len == 0) std.debug.print("unused {s}\n", .{tree.string(entry.symbol.name)});
}

var references = semantic.iterReferences();
while (references.next()) |entry| {
    if (entry.reference.symbol == .none) std.debug.print("undeclared {s}\n", .{tree.string(entry.reference.name)});
}

Scopes

The scope tree follows the specification exactly. A catch clause shares its scope with its body unless its parameter has a default value or a computed key, a named function or class expression gets a scope of its own for the name, every function body scopes its let and const apart from the parameters as TypeScript does, a class decorator sees the scope around its class, and a member decorator the class scope. Strict mode flows from modules, classes, and "use strict" directives.

Symbols

A symbol’s flags record its declaration kinds, such as function_scoped_var, block_scoped_var, function, class, interface, regular_enum, and import, and its modifiers, such as const_var, exported, and ambient. Masks answer a category in one call, as flags.intersects(Symbol.value_space) does, with variable, any_import, value_space, type_space, and namespace_space. flags.visibleIn(space) tells whether a reference in a space can bind to it.

One rule decides both merging and redeclaration. Each declaration kind carries an exclusion mask in Symbol.Excludes, and a new declaration conflicts with an existing symbol whose flags intersect it, and merges into that symbol otherwise. Function overloads, a class and an interface, and a namespace and an enum all merge this way. A namespace or enum declared in several blocks merges too, and each block sees the exports or members of the others.

Declaration spaces

Resolution follows TypeScript. A name resolves in one space, and a binding outside that space does not shadow it.

type T = string;
function f() {
  const T = 1;
  let x: T; // the outer `type T`, as in tsc
  T;        // the inner `const T`
}

A reference resolves in .value for runtime uses, .type for annotations, .namespace for the head of a dotted name such as ns.T or import x = ns.T, .typeof for value uses inside a type, or .any for alias positions such as export { x }. A class and an enum live in both the value and type spaces, and every reference carries its space, so a rename can change a value without touching a same-named type.

Module records

module_record.collect lists a module’s imports and exports, modeled on the specification’s ImportEntry and ExportEntry records, each with the local symbol behind it.

const records = try parser.semantic.module_record.collect(&tree, &semantic);
// records.imports, records.exports, records.flags

An import has a kind, name, symbol, specifier, type_only, phase, node, and scope. Its kind is named, namespace, side_effect, import_equals, dynamic, require, or augmentation, and dynamic import() and require() calls count when their specifier is a string literal. An augmentation is a declare module "m" block in a module, which merges the declarations of its body scope into the exports of m. An export has a kind, name, symbol, from_name, specifier, type_only, and node, and its kind is named, re_export, namespace, star, equals, or global. As in TypeScript, a declaration file that is a module but has no export statement exports every declaration but its imports. CommonJS exports have no static shape, so records.flags records whether the file uses require, module, exports, or import.meta.