dtolnay/syn
Syn
syn is a Rust syntax tree parser and token stream converter. It serves as a toolkit for procedural macro development, providing a framework to parse Rust source code into structured syntax trees for analysis and transformation.
The project enables the manipulation of Rust abstract syntax trees through specialized visitor and folder patterns for traversing and mutating nodes. It provides a bidirectional mapping that allows developers to convert token streams into structured trees and print those trees back into tokens for code generation.
The library covers a broad range of syntax analysis capabilities, including the parsing of expressions, type definitions, and top-level items. It also includes mechanisms for tracking source code spans to support precise error reporting and the ability to define custom syntax for specific identifiers or symbols.
Features
- Toolkits - Provides a complete toolkit for parsing Rust source code into structured trees for procedural macro development.
- Abstract Syntax Tree Parsing - Converts token streams into structured syntax trees for analysis and transformation.
- Token Stream Serialization - Transforms structured syntax tree nodes back into token streams for use in code generation.
- Recursive Descent Parsers - Uses a recursive descent parsing strategy to convert Rust token streams into structured syntax trees.
- Visitor Pattern Traversers - Implements the visitor pattern to separate syntax tree traversal logic from node-specific operations.
- Source Span Tracking - Provides a tree structure that attaches original source code offsets to every node for precise error reporting.
- Source Code Error Location Reporters - Identifies the precise location and range of a syntax tree node within the original source code.
- Syntax-to-Token Conversion - Converts syntax tree nodes back into token streams to regenerate the original source code.
- Procedural Macros - Offers a comprehensive set of tools for manipulating Rust tokens and syntax trees within the procedural macro system.
- Token Stream Converters - Provides a bidirectional mapping to convert between Rust token streams and structured syntax tree representations.
- Source Code Abstract Syntax Trees - Provides a tree-based representation of source code that identifies the location and range of nodes.
- Rust Code Generators - Turns structured data or modified syntax trees back into valid Rust tokens for automated source file creation.
- Source File Parsing - Converts complete source files into structured syntax trees for analysis and transformation.
- Source Span Trackers - Retrieves the precise source code location and range for syntax nodes to enable accurate error reporting.
- Source Token Generation - Prints syntax tree nodes back into tokens of source code for use in Rust macro expansion.
- Symmetric Token Printing - Provides a bidirectional mapping to convert structured syntax nodes back into compatible token streams.
- Syntax-to-Token Printers - Transforms represented syntax tree nodes back into token streams for use in code generation.
- Token Stream Intermediation - Implements a translation layer between raw source text and structured syntax trees using token sequences.
- Rust Parsers - Parses Rust source code into a structured syntax tree for analysis and transformation.
- Type Definition Parsing - Converts token streams into syntax trees representing paths, references, pointers, and trait objects.
- AST Visitor Patterns - Provides a framework for traversing and mutating Rust abstract syntax trees using visitor and folder patterns.
- Syntax Parsing Engines - Parses token streams into structured syntax trees representing top-level Rust declarations.
- Syntax Traversal - Provides visitor patterns to traverse syntax tree nodes for read-only analysis.
- Syntax Tree Mutation - Traverses syntax trees to modify nodes in place or transform them into new owned trees.
- Tree Mutations - Implements specialized visitor and folder patterns for traversing and mutating Rust abstract syntax trees.
- Tree Mutators - Provides specialized folder patterns to analyze and mutate syntax tree nodes during traversal.
- Syntax Tree Printing - Converts structured syntax tree nodes back into token streams for use in code generation.
- Syntax Tree Transformers - Implements folder patterns to transform an owned syntax tree into a modified version for code generation.
- Tree Mutations - Implements a folder-based transformation pattern to mutate Rust abstract syntax trees for code generation.
- Custom Syntax Handlers - Allows the definition of custom types to parse specific identifiers or symbols as keywords.
- Grammatical Ambiguity Resolution - Parses tokens using specific boundary rules to resolve grammatical ambiguities between structural elements.
- Lookahead Parsing - Includes a mechanism to inspect future tokens to resolve grammatical ambiguities during tree construction.
- Parsing Lookahead Analysis - Determines if the next token in a stream can validly start an expression to guide parsing decisions.
- Source Code Analysis Tools - Converts Rust tokens into searchable tree representations to analyze code patterns and enforce architectural constraints.
- Speculative Parsers - Provides a lookahead mechanism to decide the parsing path without consuming tokens from the stream.
- Compiler Diagnostic Toolkits - Tracks source code spans throughout the parsing and transformation process to deliver precise error messages.
- Parse Error Reporters - Generates detailed error messages identifying the specific tokens causing syntax failures.
- Error Reporting - Reports precise errors by tracking span information across the macro expansion process.
- Compiler Error Emissions - Tracks span information during expansion to trigger precise compiler error messages.
- Developer Utilities - Parser for Rust source code into an AST.
