awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
sebastianbergmann avatar

sebastianbergmann/php-token-stream

0
View on GitHub↗
6,386 stars·58 forks·PHP·8 views

Php Token Stream

php-token-stream is a lexical analysis tool and tokenizer wrapper for PHP. It functions as a source code streamer that reads tokens one by one, preventing the need to load entire source files into memory.

The project provides memory-efficient parsing by wrapping the native PHP tokenizer extension. This allows for the sequential processing of source code tokens to analyze structural components and syntax.

The tool is designed for static code analysis and the development of compiler tooling. It supports linear token processing and sequential traversal to examine language constructs and project structures.

Features

  • Lexical Analyzers - Functions as a lexical analyzer that breaks PHP source code into a stream of tokens for analysis.
  • Tokenizer Wrappers - Wraps the native PHP tokenizer extension to provide a memory-efficient streaming interface.
  • Memory-Efficient Parsing - Minimizes memory overhead when analyzing large PHP files by streaming tokens instead of loading the full source.
  • Lexical Token Streams - Represents PHP source code as a sequence of discrete tokens for syntactic and structural analysis.
  • PHP Code Analysis - Analyzes the structure and syntax of PHP source code token by token without loading entire files.
  • Source Code Streamers - Reads PHP tokens one by one as a stream to prevent loading entire source files into memory.
  • Source Code Token Streaming - Implements an iterator-based approach to stream PHP tokens without loading entire source files into memory.
  • Compiler Tooling - Provides the sequential token parsing necessary for building PHP compiler and transformation tools.
  • Lazy Sequences - Provides a lazy-loading sequence that retrieves tokens only when requested to reduce memory overhead.
  • Token Stream Navigators - Maintains a stateful pointer within the token sequence to support sequential traversal and look-ahead operations.
  • Linear Tokenizers - Processes source code as a one-dimensional stream of tokens to simplify structural analysis.
  • Static Code Analysis Tools - Enables the examination of PHP source code for bugs and structural patterns without executing it.

Star history

Star history chart for sebastianbergmann/php-token-streamStar history chart for sebastianbergmann/php-token-stream

How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Open-source alternatives to Php Token Stream

Similar open-source projects, ranked by how many features they share with Php Token Stream.
  • phpdocumentor/reflectioncommonphpDocumentor avatar

    phpDocumentor/ReflectionCommon

    9,058View on GitHub↗

    ReflectionCommon is a PHP reflection interface library and code analysis abstraction. It serves as a foundation for static analysis by providing a shared specification for representing classes, methods, and properties during programmatic code inspection. The project standardizes the reflection API to decouple analysis tools from specific PHP reflection implementations. This ensures that different analysis implementations can work interchangeably through a consistent layer of interfaces. The library covers the domain of PHP code analysis and static analysis tooling, establishing a common way

    PHPhacktoberfest
    View on GitHub↗9,058
  • nunomaduro/phpinsightsnunomaduro avatar

    nunomaduro/phpinsights

    5,607View on GitHub↗

    PhpInsights is a static analysis tool and code quality analyzer for PHP. It evaluates source code to identify bugs, style violations, and technical debt without executing the application. The tool functions as a complexity metric utility, calculating architectural and cyclomatic complexity to locate overly complicated logic. It measures overall software health and maintainability by comparing code against industry standards. The system manages technical debt through rule-based validation and metric-driven scoring. It uses a static analysis engine to parse source code, delivering the results

    PHP
    View on GitHub↗5,607
  • doctrine/lexerdoctrine avatar

    doctrine/lexer

    11,156View on GitHub↗

    This project is a regular expression lexer library and lexical analysis engine used to break input strings into typed token streams. It serves as a foundational component for constructing compilers or interpreters by identifying and categorizing substrings into discrete tokens. The library provides a token stream navigator featuring a cursor-based interface. This allows for sequential traversal of tokenized input and non-destructive lookahead, enabling the inspection of future tokens without advancing the internal position pointer. It includes specific support for recursive descent parsing t

    PHP
    View on GitHub↗11,156
  • antlr/antlr4antlr avatar

    antlr/antlr4

    18,928View on GitHub↗

    ANTLR is a grammar-based code generator and multi-language parser generator used to design and implement custom languages. It functions as a toolkit for transforming formal language definitions into executable source code for processing structured text or binary files, while providing a framework for automatically constructing and traversing hierarchical parse trees. The project is distinguished by its ability to generate lexers and parsers in various target programming languages from a single shared grammar definition. It supports grammars containing direct left recursion and utilizes adapti

    Java
    View on GitHub↗18,928
See all 30 alternatives to Php Token Stream→

Frequently asked questions

What does sebastianbergmann/php-token-stream do?

php-token-stream is a lexical analysis tool and tokenizer wrapper for PHP. It functions as a source code streamer that reads tokens one by one, preventing the need to load entire source files into memory.

What are the main features of sebastianbergmann/php-token-stream?

The main features of sebastianbergmann/php-token-stream are: Lexical Analyzers, Tokenizer Wrappers, Memory-Efficient Parsing, Lexical Token Streams, PHP Code Analysis, Source Code Streamers, Source Code Token Streaming, Compiler Tooling.

What are some open-source alternatives to sebastianbergmann/php-token-stream?

Open-source alternatives to sebastianbergmann/php-token-stream include: phpdocumentor/reflectioncommon — ReflectionCommon is a PHP reflection interface library and code analysis abstraction. It serves as a foundation for… nunomaduro/phpinsights — PhpInsights is a static analysis tool and code quality analyzer for PHP. It evaluates source code to identify bugs,… doctrine/lexer — This project is a regular expression lexer library and lexical analysis engine used to break input strings into typed… antlr/antlr4 — ANTLR is a grammar-based code generator and multi-language parser generator used to design and implement custom… felixfbecker/php-language-server — This project is a backend service that implements the Language Server Protocol to provide IDE-integrated intelligence… alecthomas/participle — Participle is a Go parser generator and toolkit for building language frontends. It provides a declarative grammar…