This project is a framework for building voice and text agents using the OpenAI Realtime API. It implements architectural patterns for multi-agent orchestration, hybrid model distribution, state-managed prompting, and real-time response validation.
The framework utilizes a hybrid task distributor to split workloads between fast conversational models and high-intelligence models for complex reasoning. It employs an orchestration system that routes user requests between specialized agents using a graph to manage complex task requirements.
Additional capabilities include a state machine prompt manager to enforce strict data collection sequences and a real-time output filter to scan model responses against safety and compliance rules. The system also features a tool-call execution pipeline and supports full-duplex communication via WebSockets.