# Yuku — A JavaScript/TypeScript Compiler Toolchain in Zig > Yuku is a high-performance JavaScript and TypeScript compiler toolchain written in Zig. Spec-compliant, zero dependencies, fast by design. > Full documentation, every page concatenated for LLMs. Index: https://yuku.fyi/llms.txt # Introduction Source: https://yuku.fyi/ A high-performance JavaScript/TypeScript compiler toolchain written in Zig, bringing modern JavaScript tooling infrastructure to the Zig ecosystem. [Try it in the playground →](https://playground.yuku.fyi) ## Why Yuku Yuku is a JavaScript/TypeScript toolchain built from the ground up in Zig. It is designed for correctness, performance, and clarity. **Correctness first.** Yuku is 100% ECMAScript spec compliant. It passes all 55,000+ tests from [Test262](https://github.com/tc39/test262), Babel, TypeScript, etc., with full AST matching, covering every edge case in the specification. Zero failures, zero AST mismatches. See the [test results](https://github.com/yuku-toolchain/yuku/tree/main/test/parser/results). **Fast by design.** The parser is built using data-oriented design principles and generous performance engineering. **Pure Zig, zero dependencies.** The entire toolchain is written in Zig with no external C libraries or runtime dependencies. This makes it easy to build, embed, and cross-compile. **Modern JavaScript.** Full support for modern and experimental features including decorators, source phase imports, deferred imports, `using`/`await using` declarations, and more. ## Current Status The `.js(x)`/`.ts(x)` parser is stable and ready for production use in both Node.js and Zig. The API is stable, the implementation is 100% spec-compliant, and the AST output is fully validated against Test262, Babel, and TypeScript fixtures. The [Traverser](https://yuku.fyi/parser/traverse/) is usable today and powers Yuku's semantic checker, but its API surface may still evolve as we build more tools on top of it. ## Performance On native (Zig/Rust) parsing, Yuku is faster than Oxc and roughly 2× faster than SWC: ![Parsing `typescript.js` (7.8 MB) · macOS (ARM), Apple M3, 16 GB · [benchmark source](https://github.com/yuku-toolchain/ecmascript-parser-benchmark-native)](https://raw.githubusercontent.com/yuku-toolchain/ecmascript-parser-benchmark-native/refs/heads/main/charts/typescript.png "Native benchmark comparing Yuku, Oxc, and other Zig/Rust parsers parsing typescript.js") On the JavaScript side (npm), Yuku is **3-10x faster** than alternatives: ![Parsing `react.js` (72 KB) via npm · macOS (ARM), Apple M3, 16 GB · [benchmark source](https://github.com/yuku-toolchain/ecmascript-parser-benchmark-js)](https://raw.githubusercontent.com/yuku-toolchain/ecmascript-parser-benchmark-js/refs/heads/main/charts/react.png "npm benchmark comparing Yuku, Oxc, Acorn, and other parsers parsing react.js") But does it scale? Yes: ![Parsing `typescript.js` (7.8 MB) via npm · macOS (ARM), Apple M3, 16 GB · [benchmark source](https://github.com/yuku-toolchain/ecmascript-parser-benchmark-js)](https://raw.githubusercontent.com/yuku-toolchain/ecmascript-parser-benchmark-js/refs/heads/main/charts/typescript.png "npm benchmark comparing Yuku, Oxc, Acorn, and other parsers parsing typescript.js") > **A note on Oxc's npm performance** >You may wonder why Oxc is slower in this npm benchmark, even slower than Babel. This is because Oxc's npm package has the overhead of passing the AST from Rust to JavaScript. What it does is serialize the AST to a JSON string on the Rust side and then call `JSON.parse` on the JavaScript side. This overhead makes it slower, even though Oxc is very fast at raw parsing speed. > >If you benchmark Babel and Oxc by just calling their `parse` functions, Oxc will appear faster than Babel. This is because the `program` field returned by Oxc's parse call is a getter that only runs `JSON.parse` when you actually access it. That deserialization is the main bottleneck, and it grows as the AST gets larger (i.e., as the source file gets longer). The npm benchmarks above measure the time to actually obtain the full AST for all parsers. > >Yuku takes a different approach. Its AST is designed from the ground up to be transfer-friendly: flat, compact, and near-binary. This makes passing the AST from Zig to JavaScript fast, lightweight, and simple, without modifying the core parser. Zig's comptime makes this safe by design. There are no multi-gigabyte allocations, only the memory the source being parsed actually needs. It works reliably on any platform and is already fully validated, passing all 55,000+ Test262 cases with exact AST matching between Zig and JavaScript. This is still an early version, with further AST transfer performance improvements already planned. # Parser Source: https://yuku.fyi/parser/ Yuku's parser turns JavaScript and TypeScript source code into an [Abstract Syntax Tree](https://yuku.fyi/parser/ast/). ## Node.js ```bash npm install yuku-parser ``` ```js import { parse } from "yuku-parser"; const { program, comments, diagnostics } = parse("const x = 1 + 2;"); ``` Outputs an [ESTree](https://github.com/estree/estree) / [TS-ESTree](https://www.npmjs.com/package/@typescript-eslint/typescript-estree)-compatible AST matching [Oxc](https://oxc.rs). Runs 3-10x faster than alternatives on npm. See [yuku-parser on npm](https://www.npmjs.com/package/yuku-parser) for the full API. ## Zig ```bash zig fetch --save git+https://github.com/yuku-toolchain/yuku.git ``` In your `build.zig`: ```zig const yuku_dep = b.dependency("yuku", .{ .target = target, .optimize = optimize, }); my_module.addImport("parser", yuku_dep.module("parser")); ``` > **Note** >Yuku requires Zig 0.16.0 or later. ## Quick Start ```zig const std = @import("std"); const parser = @import("parser"); pub fn main() !void { // the smp allocator is used as the backing allocator for the tree's internal arena var tree = try parser.parse(std.heap.smp_allocator, "const x = 5;", .{}); defer tree.deinit(); for (tree.diagnostics.items) |d| { std.debug.print("{s}\n", .{d.message}); } } ``` ## Options `parse` takes an `Options` struct to configure the parsing mode: ```zig const tree = try parser.parse(allocator, source, .{ .source_type = .module, .lang = .jsx, }); ``` | Field | Values | Default | Description | | ------------------------------- | ------------------------------------ | --------- | --------------------------------------------------------------------------------- | | `source_type` | `.script`, `.module` | `.module` | Script mode or ES module mode (strict mode) | | `lang` | `.js`, `.ts`, `.jsx`, `.tsx`, `.dts` | `.js` | Language variant and syntax features to enable | | `preserve_parens` | `true`, `false` | `true` | Keep `ParenthesizedExpression` nodes in the AST | | `allow_return_outside_function` | `true`, `false` | `false` | Allow `return` statements at the top level | | `comments` | `.none`, `.flat`, `.attached`, `.both` | `.flat` | Collect comments: as a flat list (`tree.comments`), attached to host nodes (`tree.commentsOf`), or both. See [Comments](https://yuku.fyi/parser/ast/#comments) | Both fields can be inferred from a file path: ```zig const tree = try parser.parse(allocator, source, .{ .source_type = .fromPath("app.cjs"), // .script .lang = .fromPath("app.tsx"), // .tsx }); ``` ## The Tree `parse` returns a `Tree` containing the full AST, diagnostics, and source metadata. The allocator passed to `parse` is used as the backing allocator for the tree's internal arena, so `tree.deinit()` frees everything at once. The AST is a flat array of nodes referenced by integer index. `tree.root` is always a `program` node, so unpack it directly. Everything below it is a tagged union you `switch` on: ```zig var tree = try parser.parse(allocator, source, .{}); defer tree.deinit(); const program = tree.data(tree.root).program; for (tree.extra(program.body)) |child_idx| { switch (tree.data(child_idx)) { .variable_declaration => |decl| { for (tree.extra(decl.declarators)) |d| { _ = d; } }, .function => |func| { _ = func; }, else => {}, } } ``` The four read primitives are `tree.data(idx)` for a node's typed payload, `tree.span(idx)` for its source range, `tree.extra(range)` for a variadic child list, and `tree.string(handle)` for string content. See the [AST reference](https://yuku.fyi/parser/ast/) for the full node catalog, the field conventions, and the eight categorical predicates on `NodeData`. ## Diagnostics The parser recovers from errors and continues, so a single parse produces the full AST alongside all diagnostics: ```zig for (tree.diagnostics.items) |d| { std.debug.print("[{s}] {s} at {d}..{d}\n", .{ d.severity.toString(), d.message, d.span.start, d.span.end, }); for (d.labels) |label| { std.debug.print(" {d}..{d}: {s}\n", .{ label.span.start, label.span.end, label.message }); } if (d.help) |help| { std.debug.print(" help: {s}\n", .{help}); } } ``` Each diagnostic has a `severity` (`.error`, `.warning`, `.hint`, `.info`), a `message`, a source `span`, optional `labels` pointing to related code regions, and optional `help` text. ## Going Further - [Traverse](https://yuku.fyi/parser/traverse/) — walk the AST with typed visitor hooks, in four modes: basic, scoped, semantic, and transform. - [Semantic Analysis](https://yuku.fyi/parser/semantic/) — scope tree, symbol table, scope-dependent early errors, and module import/export records. - [Codegen](https://yuku.fyi/parser/codegen/) — print a tree back to JavaScript/TypeScript source. # AST Source: https://yuku.fyi/parser/ast/ The AST that comes out of `parser.parse()` is a flat array of nodes that reference each other by integer index. Reading and walking it is fast, predictable, and explicit. There are no boxed structs, no virtual dispatch, no surprise allocations. Every operation is a tagged-union switch and a slice index away. The same tree, when exposed through the [`yuku-parser`](https://www.npmjs.com/package/yuku-parser) npm package, becomes [ESTree](https://github.com/estree/estree)-compatible output matching [Oxc](https://oxc.rs): - **JavaScript / JSX**: fully conformant with [ESTree](https://github.com/estree/estree), identical to [Acorn](https://www.npmjs.com/package/acorn). - **TypeScript**: conforms to [TS-ESTree](https://www.npmjs.com/package/@typescript-eslint/typescript-estree) used by `@typescript-eslint`. Comments can be attached to the AST nodes they belong to, exposed as a flat offset-indexed array, or both. See [Comments](#comments). On top of the base specs, the AST also carries Stage 3 [decorators](https://github.com/tc39/proposal-decorators), [import defer](https://github.com/tc39/proposal-defer-import-eval), [import source](https://github.com/tc39/proposal-source-phase-imports), and a `hashbang` field on `program`. These extensions are present in Oxc as well. > **Building tools on the AST?** >The [traverser](https://yuku.fyi/parser/traverse/) is the recommended way to work with the AST. It gives you ergonomic visitor hooks, scopes, symbols, and transforms, everything you need to walk, analyze, and rewrite the tree without managing indices yourself. Reach for it first when building lints, codemods, or any pass over the tree. > >This page covers the AST itself, the node types, their fields, and how to read them directly when you need to. ## Memory model The AST is not a graph of heap-allocated structs. Every node lives in a single flat array (`Tree.nodes`), and child references are indices into that array. Variable-length child lists live in a second flat array (`Tree.extras`), and string content lives in a string pool. Three arrays, one arena. ``` Tree nodes NodeList flat array of all nodes (data + span, struct-of-arrays) extras []NodeIndex variable-length child lists (IndexRange points here) strings StringPool all string content (source refs + interned extras) ``` `NodeList` is a `MultiArrayList(Node)`, so `data` and `span` are stored in two separate parallel arrays. Code that only reads spans, or only reads data, touches one array. All memory is owned by a single `ArenaAllocator`. `tree.deinit()` frees the entire tree at once. ## The Tree `Tree` is the root container returned by `parser.parse()`. The fields you read directly: | Field | Type | Description | | ------------- | ----------------------- | ---------------------------------------------------------- | | `root` | `NodeIndex` | Index of the root node (a `program`) | | `diagnostics` | `ArrayList(Diagnostic)` | Parse errors, warnings, hints | | `source` | `[]const u8` | Original source text | | `source_type` | `SourceType` | `.script` or `.module` | | `lang` | `Lang` | `.js`, `.ts`, `.jsx`, `.tsx`, or `.dts` | Everything else is reached through methods: ```zig tree.data(idx) // NodeData for the node at idx tree.span(idx) // Span (source byte range) for the node at idx tree.extra(range) // []const NodeIndex for an IndexRange tree.string(handle) // []const u8 for a String handle tree.commentsOf(idx) // []const AttachedComment attached to the node at idx tree.isTs() // language is .ts, .tsx, or .dts tree.isJsx() // language is .jsx or .tsx tree.isModule() // source_type is .module tree.hasErrors() // any diagnostic with severity .error ``` ## Core types Four small types carry every reference inside the AST. ### NodeIndex ```zig pub const NodeIndex = enum(u32) { null = std.math.maxInt(u32), _ }; ``` Every node is identified by its position in `Tree.nodes`. Optional child slots use `.null` to signal absence. ```zig // if_statement.alternate is .null when there is no else branch if (node.alternate != .null) { const else_data = tree.data(node.alternate); } ``` ### IndexRange ```zig pub const IndexRange = struct { start: u32, len: u32 }; ``` Variable-length children are stored as a contiguous slice in `Tree.extras`. An `IndexRange` is a `(start, len)` window into that array. Resolve it with `tree.extra(range)`: ```zig const children = tree.extra(node.body); // []const NodeIndex for (children) |child| { const child_data = tree.data(child); } ``` `IndexRange.empty` is the zero-length range. ### String ```zig pub const String = struct { start: u32, end: u32 }; ``` `String` is a lightweight handle to text. It points into one of two backing stores: - **Source slice (zero-copy)**: most identifiers and string literals parsed from input. The bytes live inside `tree.source` directly. - **Pool entry**: interned strings such as escaped identifiers and names produced by transforms. These live in the string pool's extra buffer. `tree.string(handle)` resolves both transparently and always returns `[]const u8`: ```zig const name = tree.string(node.name); ``` ### Span ```zig pub const Span = struct { start: u32, end: u32 }; ``` Byte offsets into the source text. `start` is inclusive, `end` is exclusive. ```zig const span = tree.span(idx); const text = tree.source[span.start..span.end]; ``` ## Reading a node `NodeData` is a tagged union with one variant per node type. The variant tags are snake_case (`binary_expression`, `if_statement`, `ts_type_alias_declaration`, ...). `tree.data(idx)` returns one. `switch` on the tag and unpack: ```zig switch (tree.data(idx)) { .binary_expression => |expr| { // expr.left and expr.right are NodeIndex (recurse with data) // expr.operator is a BinaryOperator enum const left = tree.data(expr.left); }, .variable_declaration => |decl| { // decl.kind is VariableKind (.var, .let, .const, .using, .await_using) // decl.declarators is IndexRange (read with extra) for (tree.extra(decl.declarators)) |d| { /* ... */ } }, .identifier_reference => |id| { // id.name is a String (resolve with string) const text = tree.string(id.name); }, else => {}, } ``` The same snake_case tag names are used as visitor hook names: a method called `enter_binary_expression` on your visitor struct fires when the [traverser](https://yuku.fyi/parser/traverse/) enters that node kind. ### Reading children A node's children sit in two kinds of fields: - **Single child**: `NodeIndex`. Either a real index, or `.null` for an absent optional slot. Read with `tree.data(field)`. - **Variadic children**: `IndexRange`. Resolve with `tree.extra(range)` to get `[]const NodeIndex`, then iterate. ```zig switch (tree.data(idx)) { .function => |func| { // single child const body = tree.data(func.body); // variadic children: function params -> formal_parameters -> items const params = tree.data(func.params).formal_parameters; for (tree.extra(params.items)) |param| { /* ... */ } }, else => {}, } ``` For tree-wide walks, use the [traverser](https://yuku.fyi/parser/traverse/) instead of writing recursion by hand. It handles every node kind correctly without per-tag bookkeeping. ## Predicates Eight methods on `NodeData` answer the categorical questions linters and analyzers ask most often. They collapse a family of tags into a single boolean. | Method | True for | | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `isExpression()` | Any node that produces a value at runtime: literals, identifier references, operator expressions, member access, calls, function and class expression forms, JSX elements, and the TypeScript value-position wrappers. | | `isStatement()` | Any node valid at statement position: control flow, structural statements, declarations, imports, exports, and TypeScript top-level declarations. Function and class declaration forms are included, expression forms are not. | | `isLiteral()` | `string_literal`, `numeric_literal`, `bigint_literal`, `boolean_literal`, `null_literal`, `regexp_literal`, `template_literal`. | | `isCallable()` | `function` (any form) and `arrow_function_expression`. Excludes `method_definition`, which wraps a `function` in its `value` field. | | `isPattern()` | `binding_identifier`, `array_pattern`, `object_pattern`, `assignment_pattern`. | | `isDeclaration()` | `variable_declaration`, function and class declaration forms, `import_declaration`, `export_named_declaration`, `export_default_declaration`, `export_all_declaration`, `ts_type_alias_declaration`, `ts_interface_declaration`, `ts_enum_declaration`, `ts_module_declaration`, `ts_global_declaration`, `ts_import_equals_declaration`. | | `isIteration()` | `for_statement`, `for_in_statement`, `for_of_statement`, `while_statement`, `do_while_statement`. Useful for `break` and `continue` scope checks. | | `isTypeContext()` | Any node that roots a TypeScript type-only subtree: type annotations, type references, function and constructor types, conditional and mapped types, type literals, interface heritage, and the various TS signatures. The semantic traverser uses this to drive its `inTypePosition()` context flag. | For dual-purpose nodes (`function` and `class`) the predicates consult the `type` field internally, so `isExpression()` returns true only for the expression forms and `isStatement()` / `isDeclaration()` only for the declaration forms. ```zig const data = tree.data(idx); if (data.isExpression()) { // any value-producing node } if (data.isCallable()) { // function or arrow_function_expression // The body, params, etc. are still type-specific, // so switch on the tag to access them. } ``` For anything narrower than these eight, `switch` directly: ```zig switch (data) { .arrow_function_expression => |arrow| { /* ... */ }, else => {}, } ``` ## Node reference Every entry in `NodeData` is a distinct node tag. The tag name is the exact name used in `tree.data()` switches and visitor hooks (`enter_`). Optional child fields are noted with `.null`. Optional child lists are noted with `.empty`. ### Program The root of every tree. There is always exactly one `program` node at `tree.root`. ```zig pub const Program = struct { source_type: SourceType, // .script or .module body: IndexRange, // (any statement | directive)[] hashbang: ?Hashbang, // non-null for #!/usr/bin/env node lines }; ``` `directive` nodes (such as `"use strict";`) appear at the start of the body. Imports and exports appear in source order alongside other statements. ### Statements | Tag | Syntax | Description | | ---------------------- | ------------------------------- | -------------------------------------------------------------------------- | | `expression_statement` | `expr;` | An expression used as a statement. | | `block_statement` | `{ ... }` | A braced block. | | `empty_statement` | `;` | A standalone semicolon. | | `debugger_statement` | `debugger;` | A debugger breakpoint. | | `if_statement` | `if (test) cons else alt` | An `if` / `else` branch. | | `switch_statement` | `switch (d) { cases }` | A `switch` with one or more `case` and `default` clauses. | | `switch_case` | `case x: ...` / `default: ...` | A single clause inside a `switch`. | | `for_statement` | `for (init; test; update) body` | A C-style `for` loop. | | `for_in_statement` | `for (x in y) body` | A `for ... in` loop iterating over enumerable property keys. | | `for_of_statement` | `for (x of y) body` | A `for ... of` or `for await ... of` loop iterating over an iterable. | | `while_statement` | `while (test) body` | A `while` loop. | | `do_while_statement` | `do body while (test)` | A `do ... while` loop. | | `break_statement` | `break;` / `break label;` | A `break` exiting the nearest loop, switch, or labeled statement. | | `continue_statement` | `continue;` / `continue label;` | A `continue` jumping to the next iteration of the nearest or labeled loop. | | `labeled_statement` | `label: stmt` | A statement prefixed with a label that `break` and `continue` can target. | | `return_statement` | `return;` / `return expr;` | A `return` from the enclosing function. | | `throw_statement` | `throw expr;` | A `throw` raising an exception. | | `try_statement` | `try {} catch {} finally {}` | A `try` with optional `catch` and `finally` clauses. | | `catch_clause` | `catch (e) { body }` | The `catch` clause of a `try`, with an optional binding. | | `with_statement` | `with (obj) body` | A `with` block. Forbidden in strict mode. | ### Declarations | Tag | Syntax | Description | | ---------------------- | ----------------------------- | ---------------------------------------------------------------------------- | | `variable_declaration` | `var/let/const/using x = ...` | A `var`, `let`, `const`, `using`, or `await using` declaration. | | `variable_declarator` | `x = init` | A single binding inside a variable declaration. | | `directive` | `"use strict";` | A directive prologue, only valid at the top of a function or module body. | | `function` | `function foo() {}` | Every function form (declaration, expression, ambient, body-less signature). | | `class` | `class Foo {}` | Both class declarations and class expressions. | `function` and `class` are dual-purpose nodes. The `type` field distinguishes the form: ```zig // FunctionType function_declaration // function foo() {} function_expression // const x = function () {} ts_declare_function // declare function foo(): void // also: plain overload signatures ts_empty_body_function_expression // body-less class methods (overloads, // abstract, ambient) // ClassType class_declaration // class Foo {} class_expression // const x = class {} ``` ### Expressions | Tag | Syntax | Description | | ---------------------------- | --------------------------------------- | -------------------------------------------------------------------------------- | | `binary_expression` | `a + b`, `a === b`, `a instanceof b` | A non-logical, non-assignment binary operation. | | `logical_expression` | `a && b`, `a \|\| b`, `a ?? b` | A short-circuiting logical operation. | | `unary_expression` | `!x`, `typeof x`, `void x`, `delete x` | A unary prefix operation. | | `update_expression` | `x++`, `++x`, `x--` | A prefix or postfix increment or decrement. | | `assignment_expression` | `x = y`, `x += y`, `x ??= y` | An assignment or compound assignment. | | `conditional_expression` | `test ? a : b` | A ternary expression. | | `sequence_expression` | `a, b, c` | A comma-separated sequence of expressions. | | `parenthesized_expression` | `(expr)` | An expression wrapped in parentheses, preserved in the tree. | | `member_expression` | `obj.prop`, `obj[x]`, `obj.#priv` | Property access, in static, computed, or optional form. | | `call_expression` | `fn(args)`, `fn?.()` | A function call, optionally with type arguments or optional invocation. | | `new_expression` | `new Foo(args)` | A `new` constructor invocation. | | `chain_expression` | `a?.b`, `a?.()` | A wrapper that scopes optional-chain short-circuiting to a member or call chain. | | `tagged_template_expression` | `` tag`hello` `` | A template literal preceded by a tag function. | | `await_expression` | `await expr` | An `await` of a promise inside an async context. | | `yield_expression` | `yield expr`, `yield* expr` | A `yield` or delegating `yield*` inside a generator. | | `meta_property` | `import.meta`, `new.target` | A meta property reference such as `import.meta` or `new.target`. | | `array_expression` | `[a, , b, ...c]` | An array literal, including holes and spread elements. | | `object_expression` | `{a: 1, b, ...c}` | An object literal, including spread elements. | | `object_property` | `key: value`, getters, setters, methods | A property entry inside an object literal. | | `spread_element` | `...expr` | A spread element used in arrays, calls, and object literals. | | `import_expression` | `import(src)`, `import.source(src)` | A dynamic `import()` call or phased import. | | `this_expression` | `this` | The `this` keyword used as an expression. | | `super` | `super` | The `super` keyword used as an expression head. | ### Literals | Tag | Syntax | Description | | ------------------ | ----------------------------- | ---------------------------------------------------- | | `string_literal` | `"hello"`, `'world'` | A string literal with escape sequences resolved. | | `numeric_literal` | `42`, `0xFF`, `0o7`, `0b1010` | A numeric literal in decimal, hex, octal, or binary. | | `bigint_literal` | `42n` | A BigInt literal. | | `boolean_literal` | `true`, `false` | The `true` or `false` keyword as a value. | | `null_literal` | `null` | The `null` literal. | | `regexp_literal` | `/pattern/flags` | A regular expression literal. | | `template_literal` | `` `hello ${name}` `` | A template literal with zero or more interpolations. | | `template_element` | text part between `${...}` | A static text span inside a template literal. | ### Identifiers Five tags, all carrying a single `name: String` field. They are structurally identical but appear in different syntactic positions and resolve differently. | Tag | Used for | | ---------------------- | -------------------------------------------------------------------------------------------------------------------------- | | `identifier_reference` | A name used as a value: `x`, `console`, `Math` | | `binding_identifier` | A name being declared: `const x`, `function foo`, `import { x }` | | `identifier_name` | A bare name in non-expression position: object keys (`{foo: 1}`), member access right-hand side (`obj.foo`), `import.meta` | | `label_identifier` | A label name in `break label`, `continue label`, or `label: stmt` | | `private_identifier` | A private class member: `#field` (the `#` is not part of `name`) | ```js const foo = bar.baz; // ^^^ ^^^ ^^^ // | | identifier_name (property, never resolved) // | identifier_reference (variable use, resolved by scope chain) // binding_identifier (declaration, recorded as a symbol) ``` `binding_identifier` additionally carries decorators, an optional type annotation, and an optional `?` flag when it appears in a parameter position. ### Patterns (destructuring) | Tag | Syntax | Description | | ---------------------- | ------------------------------- | ---------------------------------------------------------------------------------- | | `array_pattern` | `[a, , b, ...rest]` | An array destructuring pattern. | | `object_pattern` | `{a, b: c, ...rest}` | An object destructuring pattern. | | `binding_property` | `key: value` or shorthand `key` | A single property inside an object pattern. | | `assignment_pattern` | `x = default` | A binding pattern with a default value, used in destructuring and parameter lists. | | `binding_rest_element` | `...rest` | A `...rest` element inside a binding pattern or parameter list. | | `formal_parameters` | `(a, b = 1, ...rest)` | The parameter list of a function. | | `formal_parameter` | a single parameter slot | A single parameter slot wrapping a binding pattern. | ### Functions ```zig pub const Function = struct { type: FunctionType, // declaration, expression, or TS forms id: NodeIndex, // binding_identifier (.null for anonymous) generator: bool, // true for function* async: bool, // true for async function declare: bool, // true for declare function params: NodeIndex, // formal_parameters body: NodeIndex, // function_body (.null for TS overloads / abstract) type_parameters: NodeIndex, // ts_type_parameter_declaration or .null return_type: NodeIndex, // ts_type_annotation or .null }; ``` | Tag | Description | | --------------------------- | ------------------------------------------------------------------------------ | | `function` | Every named and anonymous function form, including ambient and body-less ones. | | `function_body` | The braced body of a function. | | `arrow_function_expression` | An arrow function, with either an expression or a block body. | ### Classes ```zig pub const Class = struct { type: ClassType, // class_declaration or class_expression decorators: IndexRange, // decorator[] (empty if none) id: NodeIndex, // binding_identifier (.null for anonymous expressions) super_class: NodeIndex, // any expression (.null if no extends clause) body: NodeIndex, // class_body type_parameters: NodeIndex, // ts_type_parameter_declaration or .null super_type_arguments: NodeIndex, // ts_type_parameter_instantiation or .null implements: IndexRange, // ts_class_implements[] (empty if none) abstract: bool, // true for abstract class declare: bool, // true for declare class }; ``` | Tag | Description | | --------------------- | --------------------------------------------------------- | | `class` | Both class declarations and class expressions. | | `class_body` | The braced body of a class, holding its members. | | `method_definition` | A method, getter, setter, or constructor inside a class. | | `property_definition` | A class field or auto-accessor declaration. | | `static_block` | A `static { ... }` initialization block inside a class. | | `decorator` | A decorator (`@expr`) applied to a class or class member. | | `super` | The `super` keyword used as an expression head. | ### Modules | Tag | Syntax | Description | | ---------------------------- | ------------------------------ | ---------------------------------------------------------------------------- | | `import_declaration` | `import x from 'y'` | A static `import` declaration, including side-effect and phased forms. | | `import_specifier` | `{ imported as local }` | A named binding specifier in an import declaration. | | `import_default_specifier` | `import x from ...` | The default-binding specifier in an import declaration. | | `import_namespace_specifier` | `import * as x from ...` | A `* as local` namespace import specifier. | | `import_attribute` | `{ type: "json" }` | A single attribute in a `with { ... }` clause on an import or export. | | `export_named_declaration` | `export { x }`, `export var x` | An `export { ... }` or `export ` declaration, with optional re-export. | | `export_default_declaration` | `export default expr` | An `export default` declaration. | | `export_all_declaration` | `export * from 'y'` | An `export * from "m"` or `export * as ns from "m"` declaration. | | `export_specifier` | `{ local as exported }` | A named binding specifier in an export declaration. | ### JSX JSX nodes are only present in `.jsx` and `.tsx` trees. | Tag | Syntax | Description | | -------------------------- | --------------------------- | -------------------------------------------------------- | | `jsx_element` | `...` | A JSX element, possibly self-closing. | | `jsx_opening_element` | `` | The opening tag of a JSX element. | | `jsx_closing_element` | `` | The closing tag of a JSX element. | | `jsx_fragment` | `<>...` | A JSX fragment. | | `jsx_opening_fragment` | `<>` | The opening `<>` of a JSX fragment. | | `jsx_closing_fragment` | `` | The closing `` of a JSX fragment. | | `jsx_identifier` | `Foo` in JSX position | An identifier used as a JSX tag or attribute name. | | `jsx_namespaced_name` | `namespace:name` | A namespaced JSX name. | | `jsx_member_expression` | `Foo.Bar.Baz` | A dotted JSX tag name. | | `jsx_attribute` | `foo="bar"` or `foo={expr}` | A single JSX attribute, including boolean-only forms. | | `jsx_spread_attribute` | `{...props}` | A spread attribute on a JSX element. | | `jsx_expression_container` | `{expression}` | An `{ expression }` slot inside JSX. | | `jsx_empty_expression` | `{}` | The empty `{}` placeholder inside a JSX expression slot. | | `jsx_text` | text content between tags | A span of raw text inside a JSX element or fragment. | | `jsx_spread_child` | `{...children}` | A spread child inside a JSX element. | A JSX tag name (the `name` field on `jsx_opening_element`, `jsx_closing_element`, and one form of `jsx_attribute`) is one of `jsx_identifier`, `jsx_namespaced_name`, or `jsx_member_expression`. A JSX child (entries in the `children` list on `jsx_element` and `jsx_fragment`) is one of `jsx_text`, `jsx_expression_container`, `jsx_spread_child`, `jsx_element`, or `jsx_fragment`. ## TypeScript TypeScript nodes are present in `.ts`, `.tsx`, and `.dts` trees. ### Type wrapper | Tag | Syntax | Description | | -------------------- | ------ | ---------------------------------------------------------------------------- | | `ts_type_annotation` | `: T` | A `: T` annotation wrapping an inner type. The span starts at the `:` token. | ### Keyword types Each keyword is its own zero-field node. | Tag | Syntax | | ---------------------- | ----------- | | `ts_any_keyword` | `any` | | `ts_unknown_keyword` | `unknown` | | `ts_never_keyword` | `never` | | `ts_void_keyword` | `void` | | `ts_null_keyword` | `null` | | `ts_undefined_keyword` | `undefined` | | `ts_string_keyword` | `string` | | `ts_number_keyword` | `number` | | `ts_bigint_keyword` | `bigint` | | `ts_boolean_keyword` | `boolean` | | `ts_symbol_keyword` | `symbol` | | `ts_object_keyword` | `object` | | `ts_intrinsic_keyword` | `intrinsic` | | `ts_this_type` | `this` | ### Type references | Tag | Syntax | Description | | ------------------- | -------------------- | ---------------------------------------------------------------------------- | | `ts_type_reference` | `Foo`, `Promise` | A reference to a named type, optionally with type arguments. | | `ts_qualified_name` | `A.B.C` | A left-associative dotted type name. | | `ts_type_query` | `typeof console.log` | The `typeof` type operator applied to a value reference. | | `ts_import_type` | `import("m").Foo` | A reference to a type imported from a module path, written in type position. | ### Type parameters and arguments | Tag | Syntax | Description | | --------------------------------- | ---------------------- | ------------------------------------------------------------------------------ | | `ts_type_parameter` | `T`, `T extends U = V` | A single type parameter introduced by a generic declaration. | | `ts_type_parameter_declaration` | `` | The `<...>` parameter list introduced by a generic declaration. | | `ts_type_parameter_instantiation` | `` | The `<...>` argument list applied at a call site, reference, or instantiation. | ### Literal and template types | Tag | Syntax | Description | | -------------------------- | ----------------------------- | -------------------------------------------------------------------- | | `ts_literal_type` | `"hello"`, `42`, `true`, `-1` | A literal value used in type position. | | `ts_template_literal_type` | `` `Hello, ${N}!` `` | A template literal in type position with one or more interpolations. | ### Composite types | Tag | Syntax | Description | | ------------------------ | ----------------------- | ---------------------------------------------------------------------------- | | `ts_array_type` | `T[]` | A postfix array type. | | `ts_indexed_access_type` | `T[K]` | An indexed access type that looks up a property type. | | `ts_tuple_type` | `[T, U?, ...V[]]` | A fixed-length tuple type with positional, optional, rest, or named entries. | | `ts_named_tuple_member` | `label: T`, `label?: T` | A labeled element inside a tuple type. | | `ts_optional_type` | `T?` (in tuple slot) | An optional element inside a tuple type. | | `ts_rest_type` | `...T` (in tuple slot) | A rest element inside a tuple type. | ### Set-operation types | Tag | Syntax | Description | | ---------------------- | ----------------------------------- | ------------------------------------------------------------------ | | `ts_union_type` | `A \| B \| C` | A union of two or more types. | | `ts_intersection_type` | `A & B & C` | An intersection of two or more types. | | `ts_conditional_type` | `T extends U ? X : Y` | A conditional type selecting between two branches. | | `ts_infer_type` | `infer R`, `infer R extends string` | An `infer` placeholder inside a conditional type's extends branch. | ### Type operators | Tag | Syntax | Description | | ----------------------- | ------------------------------------------ | ----------------------------------------------------------- | | `ts_type_operator` | `keyof T`, `unique symbol`, `readonly T[]` | A `keyof`, `unique`, or `readonly` prefix on an inner type. | | `ts_parenthesized_type` | `(T)` | A parenthesized type used for grouping or precedence. | ### Callable types | Tag | Syntax | Description | | --------------------- | --------------------------------------- | ----------------------------------------------------------------------------- | | `ts_function_type` | `(x: T) => U` | A callable signature in type position. | | `ts_constructor_type` | `new (x: T) => U`, `abstract new ...` | A constructor signature in type position, optionally `abstract`. | | `ts_type_predicate` | `x is T`, `asserts x is T`, `asserts x` | A type predicate that narrows a parameter or `this` in control-flow analysis. | ### Object-shape types | Tag | Syntax | Description | | ----------------- | ----------------- | ------------------------------------------------------------------------ | | `ts_type_literal` | `{ x: T; y: U }` | An anonymous object type holding a list of signatures. | | `ts_mapped_type` | `{ [K in T]: V }` | A mapped type that projects every key in a union to a new property type. | ### JSDoc types | Tag | Syntax | Description | | ---------------------------- | ----------------- | --------------------------------------------------------------- | | `ts_jsdoc_nullable_type` | `?T` or `T?` | A JSDoc-style nullable type marker. | | `ts_jsdoc_non_nullable_type` | `!T` or `T!` | A JSDoc-style non-nullable type marker. | | `ts_jsdoc_unknown_type` | `?` (in `Foo`) | A JSDoc-style unknown type, valid only in a type argument slot. | ### Signature members These appear inside `ts_type_literal.members` and `ts_interface_body.body`. | Tag | Syntax | Description | | ------------------------------------ | ----------------------------------------- | -------------------------------------------------------------------------------- | | `ts_property_signature` | `key: T`, `readonly key?: T` | A property declaration inside a type literal or interface body. | | `ts_method_signature` | `m(x: T): U`, `get x(): T`, `set x(v: T)` | A method, getter, or setter declaration inside a type literal or interface body. | | `ts_call_signature_declaration` | `(x: T): U` | A bare call signature inside a type literal or interface body. | | `ts_construct_signature_declaration` | `new (x: T): U` | A bare construct signature inside a type literal or interface body. | | `ts_index_signature` | `[k: K]: V`, `readonly [...]: V` | An index signature inside a type literal, interface body, or class body. | ### Type, interface, and enum declarations | Tag | Syntax | Description | | --------------------------- | -------------------------------------- | ---------------------------------------------------------------------- | | `ts_type_alias_declaration` | `type Maybe = T \| null` | A `type` alias declaration, optionally generic and optionally ambient. | | `ts_interface_declaration` | `interface Foo extends Bar { ... }` | An `interface` declaration, optionally generic and optionally ambient. | | `ts_interface_body` | `{ ... }` of an interface | The body of an interface, holding its signature members. | | `ts_interface_heritage` | one entry of an `extends` clause | A single parent listed in an interface's `extends` clause. | | `ts_class_implements` | one entry of an `implements` clause | A single interface listed in a class's `implements` clause. | | `ts_enum_declaration` | `enum Color { ... }` | An `enum` declaration, optionally `const` and optionally ambient. | | `ts_enum_body` | `{ ... }` of an enum | The body of an enum, holding its members in source order. | | `ts_enum_member` | `A = 1` inside an enum body | A single member of an enum body, with an optional initializer. | ### Module and namespace declarations | Tag | Syntax | Description | | ----------------------- | ---------------------------------------- | --------------------------------------------------------------------- | | `ts_module_declaration` | `namespace Foo { ... }`, `module "x" {}` | A `namespace` or `module` declaration, optionally ambient. | | `ts_module_block` | the `{ ... }` of a module | The body of a `namespace`, `module`, or `declare global` declaration. | | `ts_global_declaration` | `declare global { ... }` | A `declare global` augmentation block. | ### Parameters and `this` | Tag | Syntax | Description | | ----------------------- | ----------------------------- | ------------------------------------------------------------------------------- | | `ts_parameter_property` | `constructor(public x: T) {}` | A constructor parameter that implicitly declares a class field. | | `ts_this_parameter` | `function f(this: T)` | An explicit `this` parameter declaring the type of `this` in the function body. | ### TypeScript expressions | Tag | Syntax | Description | | ----------------------------- | ------------------ | --------------------------------------------------------- | | `ts_as_expression` | `expr as T` | A postfix `as` type assertion. | | `ts_satisfies_expression` | `expr satisfies T` | A postfix `satisfies` constraint check. | | `ts_type_assertion` | `expr` | A prefix `` type assertion. Forbidden in `.tsx`. | | `ts_non_null_expression` | `expr!` | A postfix non-null assertion. | | `ts_instantiation_expression` | `expr` | A type instantiation expression without call parentheses. | ### TypeScript module forms | Tag | Syntax | Description | | --------------------------------- | ----------------------------------------------- | --------------------------------------------------------------------------------- | | `ts_export_assignment` | `export = expr` | A CommonJS-style ambient export. | | `ts_namespace_export_declaration` | `export as namespace Name` | A UMD ambient namespace export. | | `ts_import_equals_declaration` | `import x = require("m")`, `import x = Foo.Bar` | An `import =` declaration binding to a require or entity name. | | `ts_external_module_reference` | `require("m")` | The `require("module")` form on the right-hand side of `import x = require(...)`. | ## Comments The `comments` option controls collection. It has four modes: - `.flat` (default): every comment lands in a flat, source-ordered list, `tree.comments`, each carrying its source span. No per-node attachment. - `.attached`: each comment is attached to the AST node it sits next to, read with `tree.commentsOf(idx)`. - `.both`: the flat list and per-node attachment. - `.none`: comments are skipped like whitespace. ```zig var tree = try parser.parse(allocator, source, .{ .comments = .flat }); defer tree.deinit(); for (tree.comments) |c| { // c.span is the whole comment, delimiters included std.debug.print("{s} @ {d}..{d}\n", .{ tree.string(c.value), c.span.start, c.span.end }); } ``` Each entry is a `Comment`: ```zig pub const Comment = struct { type: Type, // .line or .block value: String, // body without `//` or `/* */` delimiters span: Span, // full comment span, delimiters included }; ``` ### Attached comments `.attached` (and `.both`) bind each comment to its host node, read with `tree.commentsOf(idx)`: ```zig var tree = try parser.parse(allocator, source, .{ .comments = .attached }); defer tree.deinit(); for (tree.commentsOf(some_node_idx)) |c| { std.debug.print("{s} {s} {s}\n", .{ @tagName(c.position), @tagName(c.type), tree.string(c.value), }); } ``` Each is an `AttachedComment`: ```zig pub const AttachedComment = struct { type: Comment.Type, // .line or .block position: Position, // .before, .after, or .inside (relative to host) same_line: bool, // shares a source line with the host's adjacent edge value: String, // body without `//` or `/* */` delimiters }; ``` `position` tells you where the comment sits relative to its host: - `.before`: leading the host node. - `.after`: trailing the host node. - `.inside`: interior to an otherwise empty host, like `function f() { /* hi */ }`. `same_line` is true when the comment shares a source line with the host's adjacent edge (host start for `.before`, host end for `.after`). For `.inside` it is always false. # Traverse Source: https://yuku.fyi/parser/traverse/ The traverser is how you do real work on a Yuku AST. It walks the tree for you, calls your visitor hooks at every node, and (depending on the mode you pick) hands you the surrounding lexical scopes, the symbol table, or a mutable handle on the tree itself. | Mode | What you get | Returns | | ------------- | ------------------------------------------------- | --------------------------- | | **Basic** | Path from root, plus the full immutable tree | nothing | | **Scoped** | Path + lexical scopes (with strict-mode tracking) | `ScopeTree` | | **Semantic** | Path + scopes + symbols and references | `ScopeTree` + `SymbolTable` | | **Transform** | Path + a mutable `*Tree` for in-place rewrites | nothing | The three read-only modes (basic, scoped, semantic) hand your visitor a `*const Tree`, so the type system guarantees you cannot accidentally mutate the AST while analysing it. Transform is the only mode that gives you `*Tree`. This is intentional: tracked state (scopes, symbols) and tree mutation cannot safely coexist in a single pass, so the API splits them. > **Note** >The traverser is stable and powers Yuku's semantic checker in production. The surface may still grow as we build more tools on top of it (minifier, bundler, formatter). Breaking changes will be called out in release notes. ## Your First Visitor A visitor is just a struct with `enter_*` and `exit_*` methods. Pick a node type, write a method named after it, and the walker will call you when it gets there. ```zig const std = @import("std"); const parser = @import("parser"); const ast = parser.ast; const traverser = parser.traverser; const basic = traverser.basic; const Counter = struct { functions: u32 = 0, pub fn enter_function( self: *Counter, _: ast.Function, _: ast.NodeIndex, _: *basic.Ctx, ) traverser.Action { self.functions += 1; return .proceed; } }; var tree = try parser.parse(allocator, source, .{}); defer tree.deinit(); var counter = Counter{}; try basic.traverse(Counter, &tree, &counter); std.debug.print("found {d} functions\n", .{counter.functions}); ``` That is the entire shape of every traverser-based tool you will write. The rest of this page is "what else you can do inside a hook". ## Hooks Every hook follows the same four-argument shape: ```zig pub fn enter_( self: *V, // your visitor payload: ast., // the node's unpacked payload index: ast.NodeIndex, // the node's index in the tree ctx: *.Ctx, // mode-specific context ) traverser.Action { ... } pub fn exit_( self: *V, payload: ast., index: ast.NodeIndex, ctx: *.Ctx, ) void { ... } ``` The hook name must match a field in `ast.NodeData`. Misspell it (`enter_funciton`) or use the wrong payload type and Zig produces a clear compile error at the `traverse` call site. There are no silent mismatches, and no runtime "method not found" surprises. Enter hooks may return either `traverser.Action` or `Allocator.Error!traverser.Action`. Both are accepted, so a hook that never allocates can write `return .proceed;` directly, while one that calls `addNode` writes `return .proceed;` from inside a `try` block. Exit hooks always return `void`. ### Receiving the Payload Already Unpacked Each typed hook receives its node's payload pre-extracted from the tagged union, so you never write a `switch` to "unwrap" the node you just matched on: ```zig pub fn enter_binary_expression( _: *V, expr: ast.BinaryExpression, // already unpacked, no switch needed _: ast.NodeIndex, _: *basic.Ctx, ) traverser.Action { if (expr.operator == .add) { ... } return .proceed; } ``` When you do need to peek at a child node before the walker reaches it, switch on `ctx.tree.data(child_index)`: ```zig pub fn enter_call_expression( _: *V, call: ast.CallExpression, _: ast.NodeIndex, ctx: *basic.Ctx, ) traverser.Action { switch (ctx.tree.data(call.callee)) { .member_expression => |mem| { // callee looks like obj.method(...) _ = mem; }, .identifier_reference => |id| { const name = ctx.tree.string(id.name); // callee is a bare identifier _ = name; }, else => {}, } return .proceed; } ``` See the [AST reference](https://yuku.fyi/parser/ast/) for every node type and its fields. ### Catch-All Hooks If you want one hook that fires for every node regardless of type, define `enter_node` and/or `exit_node`. The payload is `ast.NodeData` (the full union): ```zig pub fn enter_node( self: *V, data: ast.NodeData, index: ast.NodeIndex, ctx: *basic.Ctx, ) traverser.Action { return .proceed; } pub fn exit_node( self: *V, data: ast.NodeData, index: ast.NodeIndex, ctx: *basic.Ctx, ) void {} ``` When both a typed hook and a catch-all are defined, the order is: - **Enter**: `enter_node` first, then `enter_` - **Exit**: `exit_` first, then `exit_node` That ordering lets a catch-all enter "gate" a subtree (return `.skip` to opt out before the typed hooks run) and a catch-all exit "summarise" what just happened. ### Actions Every enter hook returns one of three actions: | Action | Effect | | ---------- | ---------------------------------------- | | `.proceed` | Walk into this node's children | | `.skip` | Do not descend, move to the next sibling | | `.stop` | End the entire traversal immediately | `.skip` is what you return after hand-walking a subtree yourself, or after a transform that should not be re-entered. `.stop` is how you implement "find the first X and exit". ### Running Without Hooks Sometimes the only thing you want from the traverser is its output: a `ScopeTree` from the scoped mode, or a `ScopeTree` + `SymbolTable` from the semantic mode. In that case you do not need to define any hooks. Pass an empty struct as the visitor and the walker still runs end-to-end, the trackers still produce their result, and nothing fires in between. ```zig const NoopVisitor = struct {}; var noop = NoopVisitor{}; // Just the scope tree: const scope_tree = try scoped.traverse(NoopVisitor, &tree, &noop); // Scope tree + symbol table: var result = try sem.traverse(NoopVisitor, &tree, &noop); try result.symbol_table.resolveAll(result.scope_tree); ``` The empty struct makes the absence of hooks visible in the code, which is why there is no hidden `analyze`-style helper inside the traverser. If a tool wants the result of a walk without any per-node logic, the caller spells that out exactly. ## The Path Every mode tracks the path from the root down to the current node. The path is a small fixed-capacity stack of `NodeIndex` values you can read at any time through `ctx.path`. ```zig ctx.path.parent() // immediate parent NodeIndex, or null at root ctx.path.ancestor(0) // current node ctx.path.ancestor(1) // parent (same as parent()) ctx.path.ancestor(2) // grandparent ctx.path.depth() // 0 at root, grows as you descend var it = ctx.path.ancestors(); while (it.next()) |idx| { // walks from current node up to root } ``` Combined with the full tree (always available as `ctx.tree`), the path lets you navigate freely: ```zig pub fn enter_identifier_reference( _: *V, id: ast.IdentifierReference, _: ast.NodeIndex, ctx: *basic.Ctx, ) traverser.Action { // is this identifier the callee of a call expression? if (ctx.path.parent()) |parent_idx| { if (ctx.tree.data(parent_idx) == .call_expression) { const name = ctx.tree.string(id.name); _ = name; } } return .proceed; } ``` The path has a hard cap of 256 entries. In practice this is far beyond any realistic ECMAScript nesting depth, so you can treat it as effectively unbounded. ## Basic Traverser The minimum: path tracking plus the full immutable tree. No allocator needed, nothing returned. Use it for tools that only need structural pattern matching: - `eslint`-style rules that look at "this node and its parent" - counters and statistics - AST-shape assertions in tests - pretty-printers that read but never write the tree ```zig const basic = traverser.basic; var visitor = MyVisitor{}; try basic.traverse(MyVisitor, &tree, &visitor); ``` `basic.Ctx` carries: ```zig ctx.tree // *const ast.Tree, full read access ctx.path // NodePath, the current path stack ``` That is it. Step up to scoped or semantic when you need to know what bindings are in scope. ## Scoped Traverser Scoped mode adds automatic lexical scope tracking. Whenever the walker enters a scope-creating node, the tracker pushes a new scope. On exit, it pops. Your hooks see `ctx.scope.currentScope()` already pointing at the right place. ```zig const scoped = traverser.scoped; var visitor = MyVisitor{}; const scope_tree = try scoped.traverse(MyVisitor, &tree, &visitor); // scope_tree contains every scope the walk produced ``` ### What Creates a Scope The tracker recognises every construct in the spec that introduces a new lexical environment, plus the TypeScript-specific ones that scope type parameters and infer bindings. | Node | Scope kind | | ------------------------------------------------------------ | ------------------------------------------------------------- | | Program | `global` (plus a child `module` if `source_type` is `module`) | | Function declaration / expression | `function` | | Arrow function | `function` | | Block statement | `block` | | `for` / `for...in` / `for...of` | `block` | | `catch` clause | `block` (the body block reuses this scope) | | `switch` statement | `block` | | Class declaration / expression | `class` (always strict per spec) | | Class static block | `static_block` | | Named function or class expression | `expression_name` wrapping a `function` / `class` scope | | TS interface / type alias | `block` | | TS function/constructor type, call/construct/index signature | `block` | | TS method signature | `block` | | TS mapped / conditional type | `block` (conditional isolates `infer T` per branch) | | TS namespace body | `ts_module` (its own kind, see below) | A few details worth pinning down because they trip up first-time readers: **Catch clauses share their body block.** Per spec section 14.15.2 the catch parameter and the block body live in the same `catchEnv`. The tracker pushes one `block` scope on `catch_clause`, and the body's `block_statement` reuses it instead of pushing a second one. This is what lets `findInScopeOrHoisted` naturally detect the early-error case where a `var` inside the body collides with the parameter. **Named function and class expressions create two scopes.** For `const x = function foo() { ... }`, ECMAScript section 15.2.5 wraps the body in an extra environment that holds an immutable binding for `foo`: ``` outer scope (x lives here) expression_name (foo lives here, immutable) function scope (body bindings live here) ``` Without this, `const foo = 1` inside the body would conflict with the expression name. Same pattern for `const C = class D { ... }` per section 15.7.14. **`ts_module` is its own scope kind, not a block.** TypeScript namespace bodies act as a `var`-hoist target, so a `var` inside a namespace stays inside the namespace instead of escaping to the surrounding scope. That difference is encoded as a separate `Scope.Kind` so `Kind.isHoistTarget()` returns true for it. ### Strict Mode Strict mode propagates automatically: - Module scopes are always strict. - Class scopes are always strict. - A `"use strict"` directive at the top of any scope sets `flags.strict` on that scope. - Child scopes inherit strict mode from their parent. - Functions whose body opens with `"use strict"` get `strict = true` set **at function-scope creation time**, before any parameter hooks fire. This is needed because the directive applies retroactively to the parameter list, where rules like "no duplicate parameters" only kick in under strict mode. ### Querying the Tracker Inside a hook, `ctx.scope` is a live `ScopeTracker` you can interrogate: ```zig pub fn enter_node( _: *V, _: ast.NodeData, _: ast.NodeIndex, ctx: *scoped.Ctx, ) traverser.Action { const id = ctx.scope.currentScopeId(); // ScopeId of current scope const cur = ctx.scope.currentScope(); // Scope value (kind, flags, parent, ...) const hoist = ctx.scope.currentHoistScopeId(); // where `var` declarations would land const strict = ctx.scope.isStrict(); if (cur.kind == .function and !strict) { ... } var it = ctx.scope.ancestors(id); while (it.next()) |ancestor_id| { const ancestor = ctx.scope.getScope(ancestor_id); _ = ancestor; } return .proceed; } ``` `getScope(id)` and `getScopeMut(id)` look up any scope by id. `currentScopeMut()` is there if you need to flip a flag on the active scope yourself (rarely needed, the tracker handles `"use strict"` automatically). ### Using the ScopeTree After Traversal `scoped.traverse` returns an immutable `ScopeTree` containing every scope that was created. Each scope is: ```zig pub const Scope = struct { node: ast.NodeIndex, // the AST node that created this scope parent: ScopeId, // parent scope, .none at the root hoist_target: ScopeId, // nearest ancestor (or self) where `var` hoists to kind: Kind, // .global, .module, .function, .block, .class, ... flags: Flags, // flags.strict }; ``` `ScopeId.root` (the global scope) is always `0`, and when `source_type` is `.module`, `ScopeId.module` is always `1`. ```zig const scope_tree = try scoped.traverse(MyVisitor, &tree, &visitor); const root = scope_tree.getScope(.root); var it = scope_tree.ancestors(some_scope_id); while (it.next()) |id| { const scope = scope_tree.getScope(id); _ = scope; } ``` `scope_tree.scopes` is the raw slice, indexed by `ScopeId`. `getScope(id)` and `ancestors(start)` are the only two methods; everything else you do with scopes goes through the [SymbolTable](#symboltable-reference) lookups that take a `ScopeId`. The tree is backed by the parser's arena, so it lives as long as the source `Tree` does. Calling `tree.deinit()` invalidates it. ## Semantic Traverser Semantic mode is the full-power one: path, scopes, symbols, references, redeclaration handling, TypeScript context tracking. ```zig const sem = traverser.semantic; var visitor = MyVisitor{}; const result = try sem.traverse(MyVisitor, &tree, &visitor); // result.scope_tree - every scope // result.symbol_table - every symbol and reference ``` `sem.Ctx` exposes: ```zig ctx.tree // *const ast.Tree ctx.path // NodePath ctx.scope // ScopeTracker (same API as scoped mode) ctx.symbols // SymbolTracker ctx.inTypePosition() // true inside a TS type-only subtree ctx.inTsNamespace() // true inside a TS `namespace` body ``` ### Two-Phase Binding Symbol declaration is split across two phases per node: 1. **Phase 1, on enter**: when entering a parent declaration node (`variable_declaration`, `function`, `class`, `import_declaration`, `formal_parameters`, `catch_clause`, `ts_interface_declaration`, etc.), the tracker records _what kind_ of binding the next `binding_identifier` should produce: its flags, its redeclaration excludes, and its target scope. This happens **before** your enter hook runs. 2. **Phase 2, on `post_enter`**: after your enter hook returns, but before the walker descends into children, the tracker materialises the actual symbol or reference. `binding_identifier` becomes a `Symbol`, `identifier_reference` becomes a `Reference`. Why the split? It guarantees a useful invariant for your visitor: > **Note** >Your enter hook on a `binding_identifier` sees the scope state **before** that binding has been declared. You can inspect what is _about to_ be declared, look up whether something with the same name already exists, and decide what to do, before the tracker commits the new symbol. ```zig pub fn enter_binding_identifier( _: *V, id: ast.BindingIdentifier, _: ast.NodeIndex, ctx: *sem.Ctx, ) !traverser.Action { const flags = ctx.symbols.currentBindingFlags(); // what the new symbol will be const excludes = ctx.symbols.currentBindingExcludes(); // what it conflicts with const target = ctx.symbols.currentTarget(); // which scope it lands in const name = ctx.tree.string(id.name); if (ctx.symbols.findInScopeOrHoisted(target, name)) |existing_id| { const existing = ctx.symbols.getSymbol(existing_id); if (existing.flags.intersects(excludes)) { // genuine conflict: emit a redeclaration diagnostic } else { // compatible merge (e.g. function overload, class + interface, // namespace + value). The tracker will merge them automatically // in post_enter. } } return .proceed; } ``` `currentBindingFlags`, `currentBindingExcludes`, and `currentTarget` are the three readers for the pending state. Reading them inside an enter hook on a `binding_identifier` is always safe. Reading them at any other node is undefined. ### The Symbol Once a binding is materialised, you get a `Symbol`: ```zig pub const Symbol = struct { name: String, // index into the tree's string pool flags: Flags, // declaration kind + modifiers (see below) scope: ScopeId, // the scope this symbol was declared in decls: Range, // every declarator of this symbol, as a slice // into SymbolTable.decl_nodes. Use // table.symbolDecls(id) to read it. }; ``` A single symbol can collect multiple declarators when the language allows merging: TS function overloads, `class` + `interface` merging, `namespace` + `enum` merging, ambient module patterns. The tracker records every declarator node, so a renamer or a "go to definition" feature can show all of them. ### Symbol Flags `Symbol.Flags` describes everything the tracker knows about a binding. A single symbol can carry several flags at once: a `class` lives in both value and type space, an exported `var` is both `function_scoped_var` and `exported`, and an interface and a class of the same name merge into a single symbol that satisfies both kinds. The flags group into three categories. **Declaration kind** (what created the binding): ```zig symbol.flags.function_scoped_var // var, parameter, catch_var symbol.flags.block_scoped_var // let, const, using, await_using symbol.flags.function // function declaration / expression symbol.flags.class // class declaration / expression symbol.flags.interface // TS interface symbol.flags.type_alias // TS type alias symbol.flags.type_parameter // TS , infer T, mapped key symbol.flags.regular_enum // TS enum symbol.flags.const_enum // TS const enum symbol.flags.value_module // TS namespace whose body has runtime content symbol.flags.namespace_module // TS namespace (any kind) symbol.flags.import // value or unspecified-kind import symbol.flags.type_import // `import type ...` or `import { type x }` ``` **Modifiers** (qualifiers on the binding): ```zig symbol.flags.const_var // const or using binding symbol.flags.parameter // function/method parameter symbol.flags.catch_var // catch (e) binding symbol.flags.ambient // TS `declare` symbol.flags.exported // exported from a module symbol.flags.is_default // default export ``` **Helpers** on the flag struct itself: ```zig flags.intersects(other) // true if `flags` and `other` share at least one flag flags.merge(other) // union of two flag sets (used when merging compatible declarations) flags.isHoistingVar() // true for a real `var` (not a parameter, not a catch_var) flags.toString() // human-readable category for diagnostics ``` **Space predicates.** A symbol can occupy JS value space (visible at runtime), TS type space (referenced from annotations), or both (a `class` straddles them by design). These are common enough questions that they have direct predicates, no manual flag combinations needed: ```zig flags.inValueSpace() // var, let, const, function, class, enum, value namespace flags.inTypeSpace() // class, enum, interface, type alias, type parameter flags.isBlockScopedLike() // names a hoisting `var` cannot pass through // (block_scoped_var, class, function) ``` `class` and `regular_enum` deliberately satisfy both `inValueSpace` and `inTypeSpace`. That is what makes "use a class as a type" work without special-casing. ### Per-Kind Redeclaration Excludes For each declaration kind, the tracker has a precomputed `Symbol.Excludes.X` flag set. The rule is uniform: > A new declaration with `Excludes.X` conflicts with any existing symbol whose flags **intersect** `Excludes.X`. Otherwise the two declarations merge into a single symbol with the union of their flags. ```zig Symbol.Excludes.block_var // let / const / using Symbol.Excludes.function_var // var Symbol.Excludes.function // function (allows overloads in TS, var-merge in sloppy) Symbol.Excludes.class Symbol.Excludes.interface Symbol.Excludes.type_alias Symbol.Excludes.regular_enum Symbol.Excludes.const_enum Symbol.Excludes.value_module Symbol.Excludes.namespace_module Symbol.Excludes.import_binding Symbol.Excludes.parameter Symbol.Excludes.catch_param Symbol.Excludes.type_parameter ``` This single mechanism handles function overloads, `class` + `interface` declaration merging, `namespace` + `enum` merging, and ambient module patterns without any per-construct branching. If you ever need to teach the tracker a new merging rule, you change one flag set, not a tangle of `if` statements. ### TypeScript Context Flags Two booleans on `sem.Ctx` track whether the walker is currently inside TS-only territory: ```zig pub fn enter_identifier_reference( _: *V, id: ast.IdentifierReference, _: ast.NodeIndex, ctx: *sem.Ctx, ) traverser.Action { if (ctx.inTypePosition()) { // Inside a type annotation, type reference, type parameter, // type literal, mapped/conditional type, etc. // References here are tagged as `.type` automatically. } if (ctx.inTsNamespace()) { // Inside a TS `namespace` body. } _ = id; return .proceed; } ``` `inTypePosition()` is also what the tracker uses internally to decide that a `binding_identifier` inside a function-type or index signature is a parameter _label_ (not a real declaration). Only `type_parameter` bindings are real in type position. ### References and Their Kind Every `identifier_reference` node produces one `Reference`. Each reference carries a `kind`: ```zig pub const Reference = struct { name: String, scope: ScopeId, node: ast.NodeIndex, kind: Kind = .value, // .value or .type }; ``` `.value` means the reference is a runtime use (the receiver of a property access, an argument, the LHS of an assignment, etc.). `.type` means it appears inside a type annotation, an `extends` / `implements` clause, a type argument, or any other TS type-position context. Rename-aware tooling distinguishes the two so it can change a value without touching a same-named type, and vice versa. ### Resolving References During traversal, references are recorded but not yet linked to their declarations. After traversal, call `resolveAll` with the scope tree to walk every reference up its chain and build the cross-index: ```zig var result = try sem.traverse(MyVisitor, &tree, &visitor); try result.symbol_table.resolveAll(result.scope_tree); ``` Once resolved you have the full bidirectional map. Crucially, the reverse iterators yield `NodeIndex` values directly, so you can rewrite or annotate the source without an extra lookup: ```zig const table = result.symbol_table; // Forward: what does this reference point to? const sym_id = table.referenceSymbol(some_ref_id); if (sym_id != .none) { const sym = table.getSymbol(sym_id); _ = sym; } // Every declaration site of a symbol. Returns a slice you can index. for (table.symbolDecls(my_sym_id)) |decl_node| { _ = decl_node; } // Every use site of a symbol. var uses = table.symbolUses(my_sym_id); while (uses.next()) |use_node| { _ = use_node; } // Every site (declarations first in source order, then uses). // This is the iterator a renamer wants. var sites = table.symbolSites(my_sym_id); while (sites.next()) |node| { _ = node; } // Quick check: is this binding used at all? if (!table.isReferenced(my_sym_id)) { // candidate for an "unused variable" diagnostic } // References that did not resolve to any local binding. // These are globals, undeclared names, or free variables. var it = table.iterUnresolved(); while (it.next()) |entry| { _ = entry; // entry.id, entry.reference } ``` `iterUnresolved` is exactly what a "no-undef" linter wants: every name the parser saw that is not bound anywhere in the tree. You can also resolve a name manually from any starting scope: ```zig if (table.resolve(result.scope_tree, scope_id, "myVar")) |found| { const sym = table.getSymbol(found); _ = sym; } ``` `resolve` walks up the scope chain just like JavaScript does at runtime, also matching hoisted `var`s passing through intermediate block scopes. ### Single-Scope Lookups For tools that do _not_ want a full chain walk (a minifier checking "is this name shadowed in this exact scope"), both the tracker and the table expose tight single-scope lookups: ```zig findInScope(scope, name) // bindings declared directly in `scope` findInScopeOrHoisted(scope, name) // also matches a `var` passing through `scope` ``` `findInScopeOrHoisted` is the same lookup the tracker uses internally to detect block-scoped redeclarations of names that a hoisting `var` is travelling through. ### Iterating the Table Three iterators walk the whole table. Each yields an entry so you never reach back through the table for a lookup you were just handed: ```zig var syms = table.iterSymbols(); while (syms.next()) |entry| { _ = entry; // entry.id, entry.symbol } var refs = table.iterReferences(); while (refs.next()) |entry| { _ = entry; // entry.id, entry.reference } var unresolved = table.iterUnresolved(); while (unresolved.next()) |entry| { _ = entry; // unresolved refs only, requires resolveAll } ``` For tight per-scope loops (a minifier checking shadowing in one scope, a renamer enumerating bindings in a function body), `scopeSymbols(scope_id)` yields raw `*SymbolId`s straight out of the per-scope hash map, so you avoid copying full `Symbol` structs: ```zig // During traversal (live tracker): var it = ctx.symbols.scopeSymbols(scope_id); while (it.next()) |sym_id_ptr| { const sym = ctx.symbols.getSymbol(sym_id_ptr.*); const name = ctx.tree.string(sym.name); _ = name; } // After traversal (immutable table): var it2 = result.symbol_table.scopeSymbols(scope_id); while (it2.next()) |sym_id_ptr| { const sym = result.symbol_table.getSymbol(sym_id_ptr.*); _ = sym; } ``` ### SymbolTable Reference The table's public surface, in one place: ```zig table.symbols // []const Symbol, in declaration order table.references // []const Reference, in source order table.getSymbol(sym_id) // Symbol by id table.getReference(ref_id) // Reference by id table.iterSymbols() // (id, symbol) entries table.iterReferences() // (id, reference) entries table.iterUnresolved() // unresolved (id, reference), requires resolveAll table.scopeSymbols(scope_id) // *SymbolId per binding in the scope table.findInScope(scope, name) // single-scope lookup table.findInScopeOrHoisted(scope, name)// + hoisting var passing through table.resolve(scope_tree, scope, name) // scope-chain lookup try table.resolveAll(scope_tree) // build the cross-index table.referenceSymbol(ref_id) // forward (ref -> sym), after resolveAll table.symbolDecls(sym_id) // []const NodeIndex of declarators table.symbolUses(sym_id) // iterator over use sites, after resolveAll table.symbolSites(sym_id) // iterator over decls + uses, after resolveAll table.isReferenced(sym_id) // shorthand for "any uses?" ``` The table aliases the tree's arena, so it lives as long as the tree. Calling `tree.deinit()` invalidates everything inside. ## Transform Traverser Transform mode is for rewrites: codemods, desugaring passes, AST-level optimisations. Your visitor receives `*Tree` (mutable) and can call `setData`, `setSpan`, `setIdentifierName`, `addNode`, and `addExtra` from inside any hook. ```zig const transform = traverser.transform; var visitor = MyTransform{}; try transform.traverse(MyTransform, &tree, &visitor); ``` `transform.Ctx` is intentionally minimal: ```zig ctx.tree // *ast.Tree, full read AND write access ctx.path // NodePath ``` There is no scope or symbol tracking in this mode. Mutating the tree would invalidate any tracked state mid-walk, so the design splits "analyse" from "rewrite" at the type level. If you need both, run two passes (see [Combining Modes](#combining-modes) below). ### Replacing a Node In Place The simplest transform replaces a node's data inside its enter hook. The walker re-reads the node after every enter, so the replacement's children are walked automatically: ```zig pub fn enter_binary_expression( _: *MyTransform, expr: ast.BinaryExpression, index: ast.NodeIndex, ctx: *transform.Ctx, ) traverser.Action { if (expr.operator == .add) { ctx.tree.setData(index, .{ .binary_expression = .{ .left = expr.left, .right = expr.right, .operator = .multiply, }}); } return .proceed; } ``` ### Renaming an Identifier `setIdentifierName` rewrites the name field of any identifier-shaped node (`binding_identifier`, `identifier_reference`, `identifier_name`, `label_identifier`, `private_identifier`, `jsx_identifier`) without changing its variant or span: ```zig const new_name = try ctx.tree.addString("a"); ctx.tree.setIdentifierName(node_index, new_name); ``` This is the primitive Yuku's minifier uses to rename every site of a symbol in lock-step. Combined with `symbolSites`, you can rewrite a whole binding in a few lines. ### Creating New Nodes Use `addNode` to append a brand-new node and get its index. Use `addExtra` to allocate variable-length child lists for fields typed as `IndexRange`: ```zig const lit = try ctx.tree.addNode( .{ .numeric_literal = .{ .raw = "42" } }, .none, // span: .none if it has no source location ); const args = try ctx.tree.addExtra(&.{ child1, child2, child3 }); ``` Both are safe to call during traversal and use the tree's arena, so there is nothing to free. ### Wrapping a Node A common pattern: "take this node, move it to a fresh node, replace this one with a wrapper pointing at the moved copy". Useful for parenthesising expressions, wrapping in `await`, etc. ```zig pub fn enter_binary_expression( _: *MyTransform, expr: ast.BinaryExpression, index: ast.NodeIndex, ctx: *transform.Ctx, ) !traverser.Action { const span = ctx.tree.span(index); // 1. Move the original data into a new node, keeping its span. const inner = try ctx.tree.addNode( .{ .binary_expression = expr }, span, ); // 2. Replace the current node with a wrapper that points at the moved copy. ctx.tree.setData(index, .{ .parenthesized_expression = .{ .expression = inner } }); // 3. Skip so the walker does not re-enter and re-wrap the moved node. return .skip; } ``` Returning `.skip` here is essential. If you let the walker descend, it will re-read the new wrapper, find its child (the moved copy), and call `enter_binary_expression` again on the same data, infinitely. ### Self-Reference Safety > **Caution** >Never set a node's child to its own index. The walker re-reads node data after every enter hook, so a self-referential node causes infinite recursion. ```zig // WRONG: cycle. The wrapper points to its own index. const wrapper = try ctx.tree.addNode( .{ .parenthesized_expression = .{ .expression = index } }, span, ); ctx.tree.setData(index, ctx.tree.data(wrapper)); // RIGHT: move original data to a fresh node, wrap that. const inner = try ctx.tree.addNode(original_data, span); ctx.tree.setData(index, .{ .parenthesized_expression = .{ .expression = inner } }); ``` ## Building ASTs From Scratch `Tree.initEmpty(allocator)` creates a tree with no source text, intended for programmatic AST construction. Because there is no source backing it, every string must be created with `tree.addString(...)`. A valid tree starts from a `program` root node: ```zig var out = ast.Tree.initEmpty(allocator); defer out.deinit(); const hello = try out.addString("hello"); const lit = try out.addNode( .{ .string_literal = .{ .value = hello } }, .none, ); const stmt = try out.addNode( .{ .expression_statement = .{ .expression = lit } }, .none, ); const body = try out.addExtra(&.{stmt}); out.root = try out.addNode( .{ .program = .{ .source_type = .module, .body = body } }, .none, ); ``` That is enough for a tree that any read-only consumer (a printer, an emitter, another traverser pass) will accept. ### Building One Tree While Walking Another A particularly powerful pattern, central to transpilers and source-to-source compilers, is walking an _input_ tree with full semantic context (scopes, symbols, path) while building a completely separate _output_ tree: ```zig const sem = traverser.semantic; const Transpiler = struct { out: *ast.Tree, pub fn enter_function( self: *Transpiler, func: ast.Function, index: ast.NodeIndex, ctx: *sem.Ctx, ) !traverser.Action { // Read context from the *source* tree: const is_strict = ctx.scope.isStrict(); const span = ctx.tree.span(index); _ = is_strict; _ = func; // Build into the *output* tree: const name = try self.out.addString("transpiledFn"); const id = try self.out.addNode( .{ .binding_identifier = .{ .name = name } }, span, ); _ = id; return .proceed; } }; var source_tree = try parser.parse(allocator, source, .{}); defer source_tree.deinit(); var out = ast.Tree.initEmpty(allocator); defer out.deinit(); var transpiler = Transpiler{ .out = &out }; _ = try sem.traverse(Transpiler, &source_tree, &transpiler); // `out` is a fresh AST you built using full knowledge of the source's // scopes, symbols, and structure. The two trees have independent arenas. ``` The two trees never touch each other's storage, and either can be freed independently. This is the recommended structure for any tool that produces a transformed AST from an input AST without mutating the input. ## Combining Modes The four modes compose via multiple passes. A typical pipeline looks like: ```zig var tree = try parser.parse(allocator, source, .{}); defer tree.deinit(); // Pass 1: rewrite syntax (sugar lowering, JSX transform, etc.) var rewriter = MyTransform{}; try transform.traverse(MyTransform, &tree, &rewriter); // Pass 2: semantic analysis on the rewritten tree var analyser = MyAnalyser{}; var result = try sem.traverse(MyAnalyser, &tree, &analyser); try result.symbol_table.resolveAll(result.scope_tree); // Pass 3: emit, lint, minify, etc. ``` Because read-only modes hand visitors a `*const Tree` and transform hands a `*Tree`, the type system enforces that you cannot accidentally smuggle a tracking pass into a mutation pass. If a function compiles with `*const Tree` you have a mathematical guarantee it will not change the AST. # Semantic Analysis Source: https://yuku.fyi/parser/semantic/ The ECMAScript specification defines a set of [early errors](https://tc39.es/ecma262/#early-error) that conformant implementations must report before execution. Some are detectable during parsing from local context alone: `return` outside a function, `yield` outside a generator, invalid destructuring. Others require knowledge of the program's scope structure and bindings: redeclarations, unresolved exports, private fields used outside their class, and more. Yuku defers these scope-dependent checks to a separate semantic analysis pass. This keeps parsing fast and lets each consumer opt in only to the work it actually needs. A formatter, for example, only needs the AST and should not pay the cost of scope resolution. ## Analyzing a Tree `semantic.analyze` builds a scope tree and symbol table, resolves identifier references to their declarations, and reports the remaining early errors. Together, parsing and semantic analysis cover the full set of early errors required by the specification. ```zig var tree = try parser.parse(allocator, source, .{}); defer tree.deinit(); // run semantic analysis const result = try parser.semantic.analyze(&tree); // result.scope_tree - all lexical scopes // result.symbol_table - all symbols and references ``` Semantic diagnostics are appended directly to `tree.diagnostics` alongside parse errors. After analysis, `tree.hasErrors()` reflects both. All allocations (scope tree, symbol table) use the tree's arena, so they are valid for the lifetime of the tree and freed by `tree.deinit()`. See [Traverse](https://yuku.fyi/parser/traverse/#semantic-traverser) for everything the returned `ScopeTree` and `SymbolTable` can do. > **Note** >If you need the scope tree and symbol table without running the early-error checks, call the [semantic traverser](https://yuku.fyi/parser/traverse/#semantic-traverser) directly with your own visitor (or an empty `struct {}` if you only want the tables): > >```zig >const sem = parser.traverser.semantic; > >var noop = struct {}{}; >const result = try sem.traverse(@TypeOf(noop), &tree, &noop); > >// result.scope_tree - all lexical scopes >// result.symbol_table - all symbols and references (no diagnostics emitted) >``` > >`semantic.analyze` is built [exactly that way](https://github.com/yuku-toolchain/yuku/blob/main/src/parser/semantic/root.zig), with a real visitor that emits the early-error diagnostics on top. ## Module Records `semantic.module_record` produces a module's flat import/export records, mirroring the specification's [ImportEntry](https://tc39.es/ecma262/#table-importentry-record-fields) / [ExportEntry](https://tc39.es/ecma262/#table-exportentry-records) model. The [Analyzer](https://yuku.fyi/analyzer/)'s cross-file linking is built on it. Records carry the `SymbolId` of each local binding, so a linker joins an importing module's record to the defining module's symbol with no name re-resolution. That is why `collect` takes the symbol table: ```zig var tree = try parser.parse(allocator, source, .{}); defer tree.deinit(); const result = try parser.semantic.analyze(&tree); const records = try parser.semantic.module_record.collect(&tree, &result.symbol_table); for (records.imports) |imp| { std.debug.print("imports '{s}' from \"{s}\"\n", .{ tree.string(imp.name), tree.string(imp.specifier), }); } for (records.exports) |exp| { std.debug.print("exports '{s}'\n", .{tree.string(exp.name)}); } ``` Both slices are in source order, allocated in the tree's arena, and freed by `tree.deinit()`. ### NameKind Both record types describe what is taken from (or exposed by) a module with a `NameKind`: | Kind | Meaning | | --------- | --------------------------------------------------------------------------------------------------- | | `.named` | A specific export name. Default imports/exports use the name `"default"`, exactly as the spec models them. | | `.star` | The module namespace object (`* as ns`) or all names (`export *`) | | `.none` | No binding at all: a side-effect import (`import "m"`) | | `.equals` | The module's entire export value: TS `export = expr`. Exports only. | | `.global` | A UMD global alias: TS `export as namespace N`. Exports only, not reachable from the import graph. | ### ImportRecord One imported binding (or side-effect import) of the module: | Field | Type | Meaning | | ----------- | ----------------------- | ---------------------------------------------------------------------------- | | `symbol` | `SymbolId` | The local binding symbol. `.none` for side-effect imports. | | `name_kind` | `NameKind` | What is taken from the source module | | `name` | `ast.String` | The imported name when `name_kind` is `.named` (`"default"` for default imports). `.empty` otherwise. | | `specifier` | `ast.String` | Decoded module specifier text | | `kind` | `ast.ImportOrExportKind`| `.type` for `import type` and `import { type x }` bindings | | `phase` | `?ast.ImportPhase` | Stage 3 phase modifier (`.source`, `.defer`), or `null` for a regular import | | `node` | `ast.NodeIndex` | The specifier node, or the whole declaration for side-effect imports | How each import form maps: | Source | `name_kind` | `name` | `symbol` | | ---------------------------- | ----------- | ----------- | ---------------- | | `import x from "m"` | `.named` | `"default"` | binding of `x` | | `import { a } from "m"` | `.named` | `"a"` | binding of `a` | | `import { a as b } from "m"` | `.named` | `"a"` | binding of `b` | | `import * as ns from "m"` | `.star` | `.empty` | binding of `ns` | | `import "m"` | `.none` | `.empty` | `.none` | | `import x = require("m")` | `.star` | `.empty` | binding of `x` | TS `import a = B.C` aliases a namespace, not a module, so it produces no record. ### ExportRecord One exported name of the module: | Field | Type | Meaning | | ----------- | ----------------------- | --------------------------------------------------------------------------- | | `name_kind` | `NameKind` | `.named` for every named export including `"default"`. `.star` only for `export * from "m"` without an alias. `.equals` for `export =`, `.global` for `export as namespace`. | | `name` | `ast.String` | The exported name when `.named`, or the global name when `.global`. `.empty` for `export *` and `export =`. | | `symbol` | `SymbolId` | The local symbol backing the export. `.none` for re-exports, anonymous default exports, and expression exports that do not resolve to a module-scope binding. | | `from_kind` | `NameKind` | What a re-export takes from its source module. `.none` when the export is local (no `from` clause). | | `from_name` | `ast.String` | The source-module name when `from_kind` is `.named` | | `specifier` | `ast.String` | Decoded specifier of the `from` clause. `.empty` when local. | | `kind` | `ast.ImportOrExportKind`| `.type` for type-only exports | | `node` | `ast.NodeIndex` | The specifier or declaration node, for spans and diagnostics | How each export form maps: | Source | `name_kind` | `name` | `symbol` | `from_kind` | | -------------------------------- | ----------- | ----------- | -------------- | ------------------------- | | `export const a = 1` | `.named` | `"a"` | binding of `a` | `.none` | | `export default function f() {}` | `.named` | `"default"` | binding of `f` | `.none` | | `export default expr` | `.named` | `"default"` | `.none` | `.none` | | `export { a as b }` | `.named` | `"b"` | binding of `a` | `.none` | | `export { a as b } from "m"` | `.named` | `"b"` | `.none` | `.named` (`from_name = "a"`) | | `export * from "m"` | `.star` | `.empty` | `.none` | `.star` | | `export * as ns from "m"` | `.named` | `"ns"` | `.none` | `.star` | | `export = expr` | `.equals` | `.empty` | see below | `.none` | | `export as namespace N` | `.global` | `"N"` | `.none` | `.none` | `export ` produces one record per bound name: `export const { a, b: [c] } = x` records `a` and `c`. `export default expr` and `export = expr` resolve `symbol` only when the expression is an identifier naming a module-scope binding. > **Note** >CommonJS (`require`, `module.exports`) is ordinary code, not module syntax, and produces no records. # Codegen Source: https://yuku.fyi/parser/codegen/ The codegen takes a `Tree` and writes it back as source code. It walks the AST directly and emits each node from its tag, so the output is always syntactically valid and faithful to the tree's structure. ## Node.js ```bash npm install yuku-codegen ``` ```js import { parse } from "yuku-parser"; import { print } from "yuku-codegen"; const { program } = parse("const x = 1 + 2;"); const { code } = print(program); ``` `print`, `strip`, and `minify` take the `Program` node off the `ParseResult` returned by [yuku-parser](https://www.npmjs.com/package/yuku-parser). For source maps, pass the original source text via `sourceMaps: { source }`. See [yuku-codegen on npm](https://www.npmjs.com/package/yuku-codegen) for the full API. ## Zig ```bash zig fetch --save git+https://github.com/yuku-toolchain/yuku.git ``` ```zig const std = @import("std"); const parser = @import("parser"); pub fn main() !void { const allocator = std.heap.smp_allocator; var tree = try parser.parse(allocator, "const x = 1 + 2;", .{}); defer tree.deinit(); const result = try parser.codegen.print(allocator, &tree, .{}); defer result.deinit(allocator); std.debug.print("{s}", .{result.code}); } ``` `print` reads from the tree, writes a fresh buffer, and never mutates anything. The `Tree` remains valid after the call. The returned `Result` owns its buffers, freed via `result.deinit(allocator)`. ## Result ```zig pub const Result = struct { code: []const u8, errors: []const Diagnostic, map: ?SourceMap = null, }; ``` | Field | Type | Description | | -------- | -------------- | --------------------------------------------------- | | `code` | `[]const u8` | Generated source | | `errors` | `[]Diagnostic` | Codegen-detected problems, empty for a clean print | | `map` | `?SourceMap` | Source Map V3 when `source_maps` was set, else null | The only return-value error from `print` is allocation failure. Codegen-detected problems are reported in `errors` and do not abort the run. For a plain `print`, `errors` is always empty. They appear only when stripping (see [Type stripping](#type-stripping)). A `Diagnostic` carries: ```zig pub const Diagnostic = struct { message: []const u8, start: u32, end: u32, }; ``` ## Options ```zig const result = try parser.codegen.print(allocator, &tree, .{ .format = .pretty, .indent = 2, .quotes = .preserve, .comments = .some, .source_maps = null, }); ``` | Field | Type | Default | Description | | ------------- | ------------------- | --------- | ---------------------------------------------------------- | | `format` | `Format` | `.pretty` | `.pretty` (indented) or `.compact` (no extra whitespace) | | `indent` | `u8` | `2` | Spaces per level when `format == .pretty` | | `quotes` | `Quotes` | `.preserve` | `.preserve` keeps each string's source quote style; `.double` / `.single` force one (content always re-escaped) | | `comments` | `Comments` | `.some` | Comment passthrough filter. See [Comments](#comments). | | `source_maps` | `?SourceMapOptions` | `null` | Set to emit a Source Map V3 alongside the code | `Format` controls only discretionary whitespace. Grammar-required separators (semicolons, commas, parentheses) are always emitted regardless of mode. ## Source maps Pass the original `source` to `source_maps` to emit a Source Map V3 alongside the generated code. Without a `source` there is nothing to map back to, so no map is produced. ```zig var tree = try parser.parse(allocator, source, .{}); defer tree.deinit(); const result = try parser.codegen.print(allocator, &tree, .{ .source_maps = .{ .source = source, .file = "out.js", .source_file_name = "in.js", .sources_content = source, }, }); defer result.deinit(allocator); // result.map is non-null when a source was provided. const map = result.map.?; ``` ### `SourceMapOptions` ```zig pub const SourceMapOptions = struct { source: ?[]const u8 = null, file: ?[]const u8 = null, source_file_name: ?[]const u8 = null, source_root: ?[]const u8 = null, sources_content: ?[]const u8 = null, }; ``` | Field | Type | Description | | ------------------ | ------------- | -------------------------------------------------------------------- | | `source` | `?[]const u8` | The original source text. Required to emit a map | | `file` | `?[]const u8` | Output filename, embedded as the map's `file` | | `source_file_name` | `?[]const u8` | Source filename, embedded as the single entry of `sources` | | `source_root` | `?[]const u8` | Prefix embedded as `sourceRoot` | | `sources_content` | `?[]const u8` | When set, embedded as the single entry of the map's `sourcesContent` | ### Result shape `SourceMap` is the Source Map V3 wire format, ready to serialize: ```zig pub const SourceMap = struct { version: u8 = 3, file: ?[]const u8, source_root: ?[]const u8, sources: []const []const u8, sources_content: ?[]const ?[]const u8, names: []const []const u8, mappings: []const u8, }; ``` Columns are 0-indexed UTF-16 code units, matching the convention used by Chrome DevTools and consumer-side libraries (`@jridgewell/trace-mapping`, `source-map`, etc.). ## Comments Comments live on the AST nodes they were attached to during parsing (see [Comments](https://yuku.fyi/parser/ast/#comments) in the AST reference). For codegen to print them, the tree must have been parsed with `comments = .attached` (or `.both`). The `comments` option selects which attached comments are emitted. The default is `.some`, matching the bundler convention of preserving legal banners, JSDoc, and tree-shaking annotations while dropping plain noise. ```zig pub const Comments = enum { none, // drop every comment all, // emit every comment some, // legal banners, jsdoc, and tree-shaking annotations line, // emit `// ...` only block, // emit `/* ... */` only }; ``` Because comments are attached to nodes, they survive AST transforms: move or replace a node and its comments come with it. ## Type stripping Strip TypeScript syntax from a `Tree`, leaving JavaScript. Same codegen, same options, with TypeScript-only nodes and fields removed from the output. ```zig var tree = try parser.parse(allocator, source, .{ .lang = .ts }); defer tree.deinit(); const result = try parser.codegen.strip(allocator, &tree, .{}); defer result.deinit(allocator); std.debug.print("{s}", .{result.code}); ``` ### How it works Stripping is not regex, not a separate transform pass, not a whitespace overlay on top of the original source. It is the codegen with one extra rule per node visit: skip nodes that are TypeScript-only, and skip TypeScript-only fields on shared nodes. That makes it always accurate and reliable. Nothing is parsed by hand a second time. The parser already classified every byte, and the codegen reads that classification directly. Comments, whitespace, nested template literal types, generic call expressions, conditional types, and every other awkward boundary case are tree nodes like any other, not regex edge cases. And it is extremely fast, one traversal of the parsed tree, writing directly into an output buffer. ### What stripping does not do A few TypeScript features (`enum`, `namespace`, `module`, `export =`, `import = require()`, parameter properties) emit JavaScript runtime values. Converting them to JavaScript equivalents is a transpilation step, not a syntax-stripping step. The stripper does exactly what its name says, it strips TypeScript syntax. When a runtime-emitting construct is encountered, it is reported as a `Diagnostic` and skipped, and the rest of the file is still emitted. The ambient forms of these constructs (`declare enum`, `declare namespace`, `declare module`, `import type X = require(...)`) carry no runtime, and are stripped silently along with the rest of the type system. All other TypeScript syntax (types, interfaces, type aliases, generics, type assertions, `satisfies`, non-null `!`, `declare`, `abstract`) strips cleanly today. ## Minification `minify` applies size-reducing rewrites at print time: ```zig const result = try parser.codegen.minify(allocator, &tree, .{ .format = .compact }); defer result.deinit(allocator); ``` The substitutions: - `true` / `false` → `!0` / `!1` - `undefined` → `void 0` (in expression position) - `Infinity` → `1/0` - numeric literals shortened to their shortest form (`1000000` → `1e6`, `0.5` → `.5`, etc.) - `obj["foo"]` → `obj.foo` when the key is a valid identifier - `{ "foo": x }` → `{ foo: x }` when safe Combine with `format = .compact` for full minification. # Analyzer Source: https://yuku.fyi/analyzer/ `yuku-analyzer` is full semantic analysis for JavaScript and TypeScript: scopes, symbols, resolved references, closures, and cross-file module linking, computed natively in Zig and queried as plain JavaScript objects. **No single library gives you all of this.** Scopes and resolved references mean `eslint-scope` or `@typescript-eslint/scope-manager`. Cross-file go-to-definition means the TypeScript compiler or `ts-morph`. A parser sits underneath both. `yuku-analyzer` is all of them in one native pass behind one API. **At native speed.** Up to ~15× faster per file than `eslint-scope`, `@typescript-eslint/scope-manager`, and `@babel/traverse`, with zero per-query cost after the single native call. Stitch those separate tools together yourself and the gap only widens: each re-walks the AST, you re-parse to resolve across files, and you keep the indexes between them in sync by hand. `yuku-analyzer` pays all of that once, in Zig. ```bash npm install yuku-analyzer ``` ```js import { Analyzer } from "yuku-analyzer"; const analyzer = new Analyzer(); analyzer.addFile("config.ts", `export const flags = { debug: true };`); const app = analyzer.addFile("app.ts", `import { flags } from "./config.ts";`); const def = app.rootScope.find("flags").definition(); def.module.path; // "config.ts" def.symbol.name; // "flags" ``` That is real cross-file resolution, not a string search: it follows import, re-export, and `export *` chains to the binding that actually defines the name, the same as an editor's go-to-definition, in plain JavaScript. ## The problem it solves Assembling this yourself is not only slower, it is harder to get right. The lightweight tools give you a scope stack but leave the binding rules to you: hoisting, catch clauses, named function expressions, TypeScript declaration merging, value space versus type space. Each tool implements a subset, and each subset has its own bugs. `yuku-analyzer` computes none of that in JavaScript. The binder, scope tree, reference resolution, and module records all come from the same well-tested native analyzer that powers the rest of Yuku, so there is one implementation to keep correct, not a JavaScript copy that drifts from it. JavaScript receives a finished model to query, not events to track. ## Architecture The design rests on one observation: a semantic model is mostly integers. Scopes point to parents, symbols point to scopes, references point to symbols, and everything points to AST nodes. Integers serialize for free. **One native call.** `addFile` parses the source, runs scope construction, binding, and reference resolution in Zig, and serializes the result into a single binary buffer: the AST in Yuku's flat transfer format, followed by the semantic tables as fixed-stride sections. One FFI crossing per file, total. **Zero-copy decode.** On the JavaScript side, the semantic sections are read through typed-array views directly over the transferred buffer. Nothing is parsed, nothing is copied. A symbol's name, flags, scope, and declaration list are reads at computed offsets. **Lazy objects, eager answers.** `Scope`, `Symbol`, `Reference`, `Import`, and `Export` are flyweight objects over the tables: tiny, allocated once per row on first access, with getters that read the buffer. Cross-indexes (which references belong to which symbol, which symbols belong to which scope) build lazily on first use and amortize across every later query. **Node identity.** AST nodes decode lazily and are memoized by node index. The node you reach by walking `module.ast` and the node a semantic query hands back are the same JavaScript object. `symbol.declarations[0] === someNodeYouWalkedTo` is a meaningful comparison, and a `WeakMap` resolves any node back to its index, which is what makes `symbolOf(node)` a lookup instead of a search. The result: native-code analysis speed, JavaScript-object ergonomics, and a wire format that is provably synchronized with the code that reads it. ## The Analyzer The `Analyzer` is the project: a set of modules plus the links between them. ```js import { Analyzer } from "yuku-analyzer"; const analyzer = new Analyzer(); const module = analyzer.addFile("src/app.tsx", source); analyzer.removeFile("src/app.tsx"); // true if it existed analyzer.module("src/app.tsx"); // Module | undefined analyzer.modules; // ReadonlyMap ``` `addFile` accepts the same options as `yuku-parser`'s `parse`, with `lang` and `sourceType` defaulting from the file extension: ```js analyzer.addFile("legacy.cjs", source, { // lang: "js" inferred from the extension // sourceType: "script" inferred from the extension preserveParens: true, allowReturnOutsideFunction: false, attachComments: false, }); ``` Adding a path that already exists replaces the module and marks the graph for relinking. The call returns a new `Module`, and any scopes, symbols, or nodes you held from the previous version belong to that earlier parse. A change in `analyzer.module(path)` identity is the signal to drop a cache keyed on the old one. ### Module resolution Cross-file linking needs to map import specifiers to added files. The default resolver handles relative specifiers with standard extension probing (`./util` matches `util.ts`, `util/index.ts`, and so on). For anything else, supply your own: ```js const analyzer = new Analyzer({ resolve(specifier, importerPath) { // return the path of an added file, or null for external modules return myAliasMap.get(specifier) ?? null; }, }); ``` Returning `null` marks the import as external: `import.resolvedModule` stays `null` and definition chains stop there, without diagnostics. ## The Module `addFile` returns a `Module`, the per-file unit of the analysis. Everything on it is local JavaScript: no native calls happen after `addFile` returns. ```js module.path; // the path it was added under module.source; // the original source text module.ast; // ESTree / TS-ESTree Program, lazily decoded module.diagnostics; // syntax and semantic errors for this file module.comments; // every comment in source order ``` The AST is the same ESTree / TypeScript-ESTree output as `yuku-parser`, and nodes are plain mutable objects. Edit them, run them through any ESTree tool, print them with `yuku-codegen`. The semantic surface: ```js module.scopes; // Scope[], index is the scope id module.rootScope; // the scope top-level code runs in module.symbols; // Symbol[], index is the symbol id module.references; // Reference[], in source order module.unresolvedReferences; // references that resolve to no binding module.imports; // Import[], in source order module.exports; // Export[], in source order ``` Ids are stable within a parse: `(module.path, symbol.id)` is a persistable key for caches and incremental tooling. Re-adding a path reparses it into a new `Module` and can renumber, so pair the key with module identity and invalidate when `analyzer.module(path)` changes. ## Scopes Every lexical environment in the file, as a tree: ```js const scope = module.scopes[3]; scope.kind; // "global" | "module" | "function" | "block" | "class" // | "staticBlock" | "expressionName" | "tsModule" scope.node; // the AST node that created the scope scope.parent; // parent Scope, or null at the global scope scope.strict; // strict mode, propagated per spec scope.hoistTarget; // the scope where a `var` declared here actually lands scope.bindings; // symbols declared directly in this scope scope.find("x"); // direct binding lookup, no chain walk scope.contains(other); // is `other` this scope or a descendant? for (const s of scope.ancestors()) { /* this scope up to global */ } ``` The scope tree is the native binder's exact output, so the spec subtleties are already right. ## Symbols A `Symbol` is one declared binding: ```js const sym = module.rootScope.find("render"); sym.name; // "render" sym.scope; // the Scope it is declared in sym.declarations; // every declarator node, in source order sym.references; // every resolved use site in this module sym.id; // stable index into module.symbols ``` One symbol can have several declarations when the language merges them: TypeScript function overloads, `class` + `interface` merging, `namespace` + `enum` merging. The analyzer records every declarator, which is exactly what go-to-definition and rename need. ### Flags What a symbol is lives in a bitset. There is exactly one way to query it: `has` (any of the given flags) and `hasAll` (all of them), against the exported `SymbolFlags` constants. No parallel boolean getters, so the API stays small and predictable. ```js import { SymbolFlags } from "yuku-analyzer"; sym.has(SymbolFlags.Function); // is it a function? sym.has(SymbolFlags.TypeAlias | SymbolFlags.Interface); // either kind? sym.hasAll(SymbolFlags.Function | SymbolFlags.Exported); // an exported function? ``` Alongside the single-bit flags, four **composites** answer the common categorical questions directly: ```js sym.has(SymbolFlags.Variable); // var / let / const, parameters and catch bindings included sym.has(SymbolFlags.Import); // any import binding, value or `import type` sym.has(SymbolFlags.ValueSpace); // visible at runtime sym.has(SymbolFlags.TypeSpace); // referencable from a TS type position ``` A `class` satisfies both `ValueSpace` and `TypeSpace`, which is what makes "use a class as a type" work without special cases. The flag values, composites included, are generated from the native binder's bit layout at build time, so they can never disagree with what the binder wrote. ## References A `Reference` is one identifier in use position, already resolved: ```js const ref = module.references[0]; ref.name; // the identifier text ref.node; // the Identifier node, identity-shared with the AST ref.scope; // the scope the use occurs in ref.symbol; // the resolved Symbol, or null for free names ref.kind; // "value" for runtime uses, "type" for TS type positions ref.isWrite; // true when this use (re)assigns the binding ``` `kind` lets rename and dead-code tools treat a value and a same-named type independently. `isWrite` is computed structurally in the native pass. `module.unresolvedReferences` is the complement: every name that resolves to no local binding. That list is precisely what a no-undef lint rule or a globals collector wants. ## Node queries These methods connect AST nodes to the semantic model. All of them work on node object identity, not positions or names: ```js module.symbolOf(node); // the symbol a node declares or references, or null module.referenceOf(node); // the Reference for an identifier node, or null module.scopeOf(node); // the innermost scope whose extent contains the node module.parentOf(node); // the node that structurally contains it, or null module.resolve("fetch"); // scope-chain lookup from the root scope module.resolve("x", someScope); // or from any scope, like the engine would ``` `symbolOf` is the workhorse: hand it a declaration identifier and you get the symbol it declares, hand it a reference identifier and you get the symbol it resolves to. `parentOf` walks upward from a node you already hold, with no ancestor stack and no full walk. Because nodes are memoized by index, it is the same constant-time lookup as the others. It returns `null` at the program root and for any node that is not part of this module's AST. ## Editing the AST Node identity is what makes the analyzer a refactoring engine, not just a query layer. The `node` on every symbol and reference is the same object you reach by walking `module.ast` (`===` holds), so a transform is a plain assignment, and [`yuku-codegen`](https://yuku.fyi/parser/codegen/) prints the mutated tree back to source: ```js import { print } from "yuku-codegen"; const m = analyzer.addFile("util.ts", `const tmp = load();\nexport const data = tmp.value + tmp.size;`); const tmp = m.rootScope.find("tmp"); tmp.declarations[0].name = "raw"; // rename the binding for (const ref of tmp.references) ref.node.name = "raw"; // and every resolved use // across files, analyzer.referencesOf(symbol) returns these same live nodes for every use print(m.ast).code; // const raw = load(); // export const data = raw.value + raw.size; ``` The uses come from resolved references, not a name search, so a shadowing inner `tmp` is left untouched. The walk in the next section is the same edit applied with full context at every node. ## Walking `module.walk` is a typed visitor walk with the semantic model in context. Handlers are keyed by node type and receive the exact node type, not a generic node: ```js module.walk({ // bare function = enter handler CallExpression(node, ctx) { if (node.callee.type === "Identifier") { const target = ctx.module.symbolOf(node.callee); if (target?.has(SymbolFlags.Import)) { console.log(`calls imported ${node.callee.name}`); } } }, // or an enter/leave pair FunctionDeclaration: { enter(node, ctx) { console.log("entering", node.id.name); }, leave(node, ctx) { console.log("leaving", node.id.name); }, }, // universal catch-alls enter(node, ctx) {}, leave(node, ctx) {}, }); ``` Per node, the order is: catch-all `enter`, typed enter, children, typed leave, catch-all `leave`. Pass a node as the second argument to walk only a subtree: `module.walk(visitors, someFunction)`. ### The context One context object is reused across the whole walk (do not store it). It carries the position and the semantics: ```js ctx.node; // the current node ctx.parent; // its parent, or null at the walk root ctx.key; // the field on the parent holding this node ctx.index; // position in an array field, or null ctx.ancestors(); // a copy of the ancestor chain, root first ctx.scope; // the innermost Scope at this node ctx.symbol; // shorthand for module.symbolOf(node) ctx.reference; // shorthand for module.referenceOf(node) ctx.module; // the module being walked ``` `ctx.scope` is not tracked during the walk. The binder records the scope at every node and ships it as a per-node table, so `ctx.scope` (like `module.scopeOf`) is a single read off that table. No scoping rule is evaluated in JavaScript, and the answer is exact even where scopes do not nest with spans, such as decorators. ### Mutation The walk mutates the AST in place, with precise semantics: | Operation | Effect | | ----------------------- | ------------------------------------------------------------------------------------------------------------- | | `ctx.skip()` | Do not descend into this node's children. `leave` still fires. | | `ctx.stop()` | End the walk immediately. | | `ctx.replace(node)` | Swap the current node. The walk continues into the replacement's children and `leave` fires for its new type. | | `ctx.remove()` | Splice the node out of an array field, or null a plain field. Children are not walked, `leave` does not fire. | | `ctx.insertBefore(node)`| Insert a sibling before the current node. The inserted node is not visited. | | `ctx.insertAfter(node)` | Insert a sibling after the current node. The walk visits it. | A replacement node created with `start: 0, end: 0` inherits the original node's span, which keeps source maps meaningful through `yuku-codegen`. ```js module.walk({ DebuggerStatement(node, ctx) { ctx.remove(); }, Identifier(node, ctx) { if (ctx.symbol === legacyName) node.name = "modernName"; }, }); ``` One rule to remember: the semantic tables are a snapshot of the parsed source. Nodes you create have no symbols or references of their own. Analyze, transform, print, and re-analyze the output if you need fresh semantics for the transformed code. ## findAll For the simplest queries there is a one-liner: ```js module.findAll("FunctionDeclaration"); // FunctionDeclaration[] module.findAll(["ClassDeclaration", "TSInterfaceDeclaration"]); ``` ## Closure analysis `capturesOf` computes the free variables of a function: every binding referenced inside it (nested closures included) that is declared outside it. ```js const source = ` let count = 0; const step = 2; export function tick() { count += step; return () => count; } `; const module = analyzer.addFile("counter.ts", source); const [tick] = module.findAll("FunctionDeclaration"); for (const capture of module.capturesOf(tick)) { console.log(capture.symbol.name, capture.isWritten); } // count true (tick writes to it) // step false (read only) ``` Each `Capture` carries the outer `symbol`, the capturing `references` inside the function, and `isWritten`. Type-only references are excluded, since they do not exist at runtime. Only bindings appear: `this`, `arguments`, and unresolved globals carry no symbol and are never reported, while module-scope and imported bindings count like any other outer binding. Because the computation rides the resolved reference table, it is shadowing-correct and alias-correct by construction. A local `count` declared inside the function does not produce a false capture, and a reference is attributed to the binding it actually resolves to, not to the nearest matching name. ## Cross-file analysis ### Import and export records Each module carries spec-true records of its module surface, computed natively: ```js for (const imp of module.imports) { imp.specifier; // "./lib.ts" imp.name; // imported export name, "default" for default imports, // null for namespace and side-effect imports imp.local; // the local binding Symbol, or null for side effects imp.isNamespace; // import * as ns imp.isSideEffect; // import "m" imp.typeOnly; // import type / import { type x } imp.phase; // "source" | "defer" | null (stage 3 phase imports) imp.resolvedModule; // the defining Module, or null when external } for (const exp of module.exports) { exp.name; // exported name, "default" included, null for export * exp.local; // backing local Symbol, when there is one exp.isStar; // export * from "m" exp.specifier; // re-export source, or null for local exports exp.fromName; // the name taken from the source module exp.isNamespaceReexport; // export * as ns from "m" exp.isExportEquals; // TS export = expr (the module's entire value) exp.globalName; // TS export as namespace N, else null exp.typeOnly; // export type exp.resolvedModule; // the source Module for re-exports } ``` Following the specification, `default` is modeled as an export *name*, not a separate kind, and `export *` never forwards `default`. TypeScript's legacy module forms (`export =`, `export as namespace`) are recorded with their own kinds, so ESM tooling never mistakes them for named exports. Tools built on these records inherit the spec behavior instead of approximating it. These records cover ECMAScript module syntax and TypeScript's module forms (`import` / `export`, `import type`, `export =`, `export as namespace`). CommonJS is ordinary code rather than module syntax, so `require`, `module.exports`, and `exports.x` produce no import or export records and take no part in linking. Everything per file (scopes, symbols, references, captures) is computed for CommonJS sources the same way. ### Linking `analyzer.link()` joins the graph: resolves every specifier through the resolver, populates `resolvedModule` on imports and re-exports, builds `dependencies` / `dependents`, and validates every imported name and named re-export. Name resolution implements the spec's `ResolveExport`: renaming re-export chains are followed per name, `default` is never satisfied by `export *`, and a name supplied by multiple `export *` declarations through different bindings is reported as ambiguous, the same conditions an engine raises at link time. Calling it is optional. Every cross-file surface links on demand after files change, so reading `import.resolvedModule` or `module.dependencies` is always correct. Call `link()` explicitly when you want to control when the work happens and collect the diagnostics at a known point: ```js analyzer.link(); for (const d of analyzer.diagnostics) { console.log(`${d.module}: ${d.message}`); // "main.ts: Module './lib.ts' has no export 'helpr'" } ``` ### Definitions across modules `definitionOf` follows import, re-export, and `export *` chains to the place a binding is actually defined, however many files away: ```js // a.ts: export const value = 1; // b.ts: export { value as renamed } from "./a.ts"; // c.ts: import { renamed } from "./b.ts"; const c = analyzer.module("c.ts"); const sym = c.rootScope.find("renamed"); const def = analyzer.definitionOf(sym); def.module.path; // "a.ts" def.symbol.name; // "value" ``` `symbol.definition()` is the instance-method shorthand. A result with `symbol: null` means the definition is a whole module namespace (`import * as ns`). A `null` result means the chain leaves the added file set (an external package, by design not an error), cannot be resolved, or is ambiguous. Chains with cycles terminate safely: a circular request is detected per (module, name) pair, so a chain may legitimately pass through the same module twice under different names. ### References across modules The inverse direction: every use of a symbol anywhere in the graph, with imports followed back to the definition: ```js const uses = analyzer.referencesOf(def.symbol); for (const { module, reference } of uses) { console.log(module.path, reference.name, reference.isWrite); } ``` This is find-all-references as a compiler primitive: rename across files, unused-export detection, impact analysis. ```js // unused exports, whole project for (const module of analyzer.modules.values()) { for (const exp of module.exports) { if (exp.local && analyzer.referencesOf(exp.local).length === 0) { console.log(`${module.path}: '${exp.name}' is exported but never used`); } } } ``` ### Exported names `module.exportedNames()` lists everything a module exports with `export *` chains followed, the spec's `GetExportedNames`. Per the spec, ambiguous star names are included (ambiguity is a resolution error, not an enumeration one) and `default` never arrives through a star: ```js // a.ts: export const one = 1; // lib.ts: export const two = 2; export default x; export * from "./a.ts"; analyzer.module("lib.ts").exportedNames(); // ["two", "default", "one"] analyzer.module("a.ts").exportedNames(); // ["one"] ``` This is what namespace-member completion and re-export expansion build on. ## SymbolFlags reference The full bitset, generated from the native binder's layout: | Flag | Meaning | | ----------------------- | ------------------------------------------------ | | `FunctionScopedVariable`| `var`, parameter, or catch variable | | `BlockScopedVariable` | `let`, `const`, `using`, `await using` | | `Function` | function declaration or expression | | `Class` | class declaration or expression | | `RegularEnum` | TS `enum` | | `ConstEnum` | TS `const enum` | | `ValueModule` | TS namespace with runtime content | | `Interface` | TS `interface` | | `TypeAlias` | TS `type` alias | | `TypeParameter` | TS ``, `infer T`, mapped-type key | | `NamespaceModule` | TS namespace of any kind | | `ValueImport` | a value import binding (`import x` / `import { x }`) | | `TypeImport` | `import type` / `import { type x }` binding | | `Const` | `const` or `using` binding | | `Ambient` | TS `declare` | | `Parameter` | function or method parameter | | `CatchVariable` | `catch (e)` binding | | `Exported` | exported from its module | | `Default` | the default export | Plus four composites (unions of the above), for the common categorical questions: | Composite | Matches | | ------------- | ----------------------------------------------------------------- | | `Variable` | `var` / `let` / `const`, parameters and catch bindings included | | `Import` | any import binding, value or `import type` | | `ValueSpace` | visible at runtime (var, function, class, enum, value namespace) | | `TypeSpace` | referencable from a type position (class, enum, interface, alias, type param) | ## Performance Analysis runs in the native parser pass, so full semantics cost roughly half of parsing time on top of the parse itself. Validated against 55,000+ real-world files. Concretely, on an Apple M-series machine: parsing plus complete semantic analysis of a typical source file lands well under a millisecond, walking sustains tens of millions of nodes per second, and linking a 2,000-module graph takes about a millisecond. ## TypeScript Everything is fully typed. Visitor handlers receive exact node types, and the semantic surface (`Module`, `Scope`, `Symbol`, `Reference`, `Import`, `Export`, `Capture`) is exported: ```ts import type { Module, Symbol, Capture } from "yuku-analyzer"; ```