Awesome GitHub RepositoriesToken-Level Incremental Processing

Harmony ist ein KI-SDK, das für das Tokenisieren von Konversationen, das Formatieren von Reasoning-Layouts, das Parsen von Rohausgaben und das Definieren von Tool-Call-Schemata entwickelt wurde. Es bietet ein System zur Konvertierung strukturierter Dialoge und Tool-Calls in die Token-Sequenzen, die für die Inferenz und das Training von Large Language Models erforderlich sind. Das Projekt enthält einen Output-Formatter, der Reasoning-Ketten und Multi-Channel-Ausgaben in konsistente Layouts strukturiert, um Token-Verlust zu vermeiden. Es bietet zudem einen Response-Parser, der rohe Completion-Token und Live-Streams zurück in strukturierte Nachrichtenobjekte und Rollen transformiert. Das SDK verwaltet die Tool-Integration durch ein Framework zur Definition aufrufbarer Funktionen und Namespaces. Es bietet zudem Funktionen für Echtzeit-Token-Parsing, Konfiguration des Modellverhaltens und zustandsbehaftete Konversations-Serialisierung.

Nearley is a JavaScript parser toolkit used to define context-free grammars and generate corresponding parsers. It features an EBNF grammar compiler that transforms language definitions written in extended Backus-Naur Form into executable JavaScript code, utilizing an Earley parser implementation to process any context-free grammar. The toolkit distinguishes itself by its ability to handle left-recursion and ambiguity without failing, allowing it to identify and return multiple valid derivations for a single input string. It also includes a grammar fuzzing generator to produce random strings

Nearley is a JavaScript parser toolkit and context-free grammar compiler. It functions as an Earley parser generator that transforms BNF-like grammar definitions into executable code capable of analyzing text and generating abstract syntax trees. The project is distinguished by its ability to handle any context-free grammar, including those with left recursion and ambiguity, by maintaining multiple valid derivations for a single input. It further supports incremental parsing, allowing input strings to be processed in chunks to provide partial results and real-time feedback. Beyond core parsi

The YAML Language Server is a backend service that provides intelligent editing support for YAML files by implementing the Language Server Protocol. It functions as a validation and analysis engine, enabling code editors to offer real-time diagnostics, structural feedback, and automated assistance for configuration-heavy development workflows. The project distinguishes itself through its schema-driven approach to data integrity, utilizing a JSON Schema validation engine to verify document structure against formal definitions. It supports complex configuration management by allowing developers