# openai/openai-realtime-agents

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/openai-openai-realtime-agents).**

6,911 stars · 1,094 forks · TypeScript · MIT

## Links

- GitHub: https://github.com/openai/openai-realtime-agents
- awesome-repositories: https://awesome-repositories.com/repository/openai-openai-realtime-agents.md

## Description

This project is a framework for building voice and text agents using the OpenAI Realtime API. It implements architectural patterns for multi-agent orchestration, hybrid model distribution, state-managed prompting, and real-time response validation.

The framework utilizes a hybrid task distributor to split workloads between fast conversational models and high-intelligence models for complex reasoning. It employs an orchestration system that routes user requests between specialized agents using a graph to manage complex task requirements.

Additional capabilities include a state machine prompt manager to enforce strict data collection sequences and a real-time output filter to scan model responses against safety and compliance rules. The system also features a tool-call execution pipeline and supports full-duplex communication via WebSockets.

## Tags

### Artificial Intelligence & ML

- [Realtime AI Session Managers](https://awesome-repositories.com/f/artificial-intelligence-ml/realtime-ai-session-managers.md) — Builds conversational agents using the OpenAI Realtime API with support for switching between models and roles.
- [Multi-Agent Routing Systems](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/agent-orchestration-multi-agent/autonomous-agents/agent-orchestration-frameworks/multi-agent-routing-systems.md) — Directs users between different specialized AI agents based on their intent to handle complex task requirements.
- [Agent Routing Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/agent-orchestration-multi-agent/coordination-and-routing/agent-routing-frameworks.md) — Provides a framework for directing user requests to specialized agents using a defined intent graph. ([source](https://cdn.jsdelivr.net/gh/openai/openai-realtime-agents@main/README.md))
- [Tool Call Execution Loops](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/conversational-voice-interaction/human-in-the-loop/tool-call-execution-loops.md) — Processes model-generated function requests by executing local code and feeding results back into the context.
- [Hybrid LLM Task Distributors](https://awesome-repositories.com/f/artificial-intelligence-ml/hybrid-llm-task-distributors.md) — Implements a pattern for splitting workloads between fast conversational models and high-intelligence models.
- [Hybrid Model Task Distribution](https://awesome-repositories.com/f/artificial-intelligence-ml/hybrid-model-task-distribution.md) — Implements a hybrid task distributor to split workloads between fast conversational models and high-intelligence models. ([source](https://cdn.jsdelivr.net/gh/openai/openai-realtime-agents@main/README.md))
- [Multi-Agent Orchestration Systems](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-agent-orchestration-systems.md) — Routes user requests between specialized agents using a graph to manage complex task requirements.
- [Full-Duplex Multimodal Interaction](https://awesome-repositories.com/f/artificial-intelligence-ml/multimodal-processing/full-duplex-multimodal-interaction.md) — Maintains a persistent open connection for simultaneous audio and text streaming between client and server.
- [Realtime API Agent Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/realtime-api-agent-frameworks.md) — Provides a framework for building voice and text agents using the OpenAI Realtime API with agentic patterns.
- [Tiered Model Workload Splitting](https://awesome-repositories.com/f/artificial-intelligence-ml/tiered-model-workload-splitting.md) — Distributes workload between a low-latency model for interaction and a high-reasoning model for complex tool execution.
- [Conversational State Managers](https://awesome-repositories.com/f/artificial-intelligence-ml/conversational-state-managers.md) — Guides a voice AI through a strict sequence of steps to collect and verify user information.
- [Finite State Machine Managers](https://awesome-repositories.com/f/artificial-intelligence-ml/conversational-state-managers/finite-state-machine-managers.md) — Enforces strict sequences of steps to collect and verify data points during model conversations using a finite state machine.
- [Sequential State Enforcement](https://awesome-repositories.com/f/artificial-intelligence-ml/structured-prompting-tools/prompt-based-schema-enforcement/sequential-state-enforcement.md) — Guides models through a strict sequence of steps to collect and verify specific data points sequentially. ([source](https://cdn.jsdelivr.net/gh/openai/openai-realtime-agents@main/README.md))

### Web Development

- [Intent-Based Routing](https://awesome-repositories.com/f/web-development/request-routing/intent-based-routing.md) — Directs user requests between specialized model instances based on a predefined map of intents.

### Development Tools & Productivity

- [Streaming Output Modifiers](https://awesome-repositories.com/f/development-tools-productivity/real-time-output-streaming/streaming-output-modifiers.md) — Intercepts model responses in the stream to validate content against safety and compliance rules.

### Security & Cryptography

- [AI Output Safety Filters](https://awesome-repositories.com/f/security-cryptography/ai-output-safety-filters.md) — Scans generated voice and text responses against safety and compliance rules before they reach the end user.
- [Model Safety Filters](https://awesome-repositories.com/f/security-cryptography/model-safety-filters.md) — Scans generated model responses against safety and compliance rules in real-time to validate content. ([source](https://cdn.jsdelivr.net/gh/openai/openai-realtime-agents@main/README.md))

### Software Engineering & Architecture

- [Conversational State Machines](https://awesome-repositories.com/f/software-engineering-architecture/state-machine-logic/lightweight-state-machines/automation-state-machines/entity-state-machines/architectural-state-machines/conversational-state-machines.md) — Guides model behavior through a defined sequence of states to ensure structured data collection.