Awesome GitHub RepositoriesToken-Level Incremental Processing

Harmony 是一个 AI SDK，旨在对对话进行 Token 化、格式化推理布局、解析原始输出并定义工具调用模式。它提供了一个系统，用于将结构化对话和工具调用转换为大型语言模型推理和训练所需的 Token 序列。 该项目包括一个输出格式化程序，将推理链和多通道输出结构化为一致的布局，以防止 Token 丢失。它还具有一个响应解析器，将原始完成 Token 和实时流转换回结构化的消息对象和角色。 该 SDK 通过用于定义可调用函数和命名空间的框架来管理工具集成。它还提供了实时 Token 解析、模型行为配置和有状态对话序列化的功能。

Nearley is a JavaScript parser toolkit used to define context-free grammars and generate corresponding parsers. It features an EBNF grammar compiler that transforms language definitions written in extended Backus-Naur Form into executable JavaScript code, utilizing an Earley parser implementation to process any context-free grammar. The toolkit distinguishes itself by its ability to handle left-recursion and ambiguity without failing, allowing it to identify and return multiple valid derivations for a single input string. It also includes a grammar fuzzing generator to produce random strings

Nearley is a JavaScript parser toolkit and context-free grammar compiler. It functions as an Earley parser generator that transforms BNF-like grammar definitions into executable code capable of analyzing text and generating abstract syntax trees. The project is distinguished by its ability to handle any context-free grammar, including those with left recursion and ambiguity, by maintaining multiple valid derivations for a single input. It further supports incremental parsing, allowing input strings to be processed in chunks to provide partial results and real-time feedback. Beyond core parsi

The YAML Language Server is a backend service that provides intelligent editing support for YAML files by implementing the Language Server Protocol. It functions as a validation and analysis engine, enabling code editors to offer real-time diagnostics, structural feedback, and automated assistance for configuration-heavy development workflows. The project distinguishes itself through its schema-driven approach to data integrity, utilizing a JSON Schema validation engine to verify document structure against formal definitions. It supports complex configuration management by allowing developers