vocodedev/vocode-core

Features

• Voice Agents - Provides a framework for building agents that use speech-to-text and text-to-speech for real-time voice interactions.
• Conversational Voice AI - Serves as a conversational orchestrator that coordinates real-time audio streaming, LLM reasoning, and tool execution.
• Real-Time Conversational AI Frameworks - Provides a complete framework for building low-latency voice agents by integrating STT, LLM, and TTS.
• Conversational Audio Streams - Coordinates real-time audio data flow between speech-to-text, reasoning engines, and text-to-speech for low-latency voice interactions.
• API-Driven Call Automation - Handles inbound and outbound telephony conversations using configurable agents initiated and managed via API.
• Agent Tool Definitions - Implements mechanisms for defining custom agent tools, including names, parameters, and validation logic for live conversations.
• Conversational Interaction Settings - Enables configuration of prompts, languages, and voice providers to define how agents interact with users.
• Agent Persona Configurations - Defines personas and behavioral traits to govern the identity and conversational style of voice agents.
• Tool Assigners - Provides interfaces for linking specific functional tools, such as call termination or API triggers, to agents.
• Agent Configurations - Provides structured settings and configuration for defining AI agent behavior, providers, and operational parameters.
• Conversational Turn Detection - Implements logic to analyze speech context and determine when a user has finished speaking or interrupted the agent.
• Conversational Behavior Policies - Provides structured flows and policies to govern the behavior, greetings, and response parameters of conversational LLMs.
• Multi-turn Interaction Managers - Manages stateful conversations and turn-based interaction logic.
• Phone Call Handlers - Provides agents capable of conducting natural language interactions and executing tasks over traditional phone lines.
• Realtime Voice Conversation Facilitators - Facilitates low-latency, two-way spoken interactions with integrated emotion tracking.
• Voice Activity Detection - Implements automated identification of speech segments to manage conversational turn-taking.
• Agent Prompt Templates - Implements reusable system prompt structures to govern the personality, goals, and reasoning of voice agents.
• Agent Tool Execution - Provides mechanisms for agents to invoke external functions and tools, such as email or scheduling, during live conversations.
• Model Provider Integrations - Provides unified interfaces for specifying and configuring different large language model providers to power agent responses.
• AI Telephony Systems - Manages inbound and outbound phone calls and provisions numbers for AI voice agents.
• Language Model Integrations - Provides adapters and streaming interfaces to connect voice agents to various hosted or local language model providers.
• Multilingual Transcription - Supports multi-language audio transcription using configurable models to enable global voice interactions.
• Real-Time Transcription - Provides instantaneous conversion of live microphone audio streams into text transcripts for real-time processing.
• Conversational Voice Pipelines - Coordinates a real-time pipeline of transcription, reasoning, and synthesis for continuous voice interaction.
• Function Calling Interfaces - Implements systems that enable language models to execute external tools and API functions based on model output.
• Model-Driven Action Triggering - The project runs agent actions based on the model's determination using prompts and tool descriptions.
• Real-Time Speech Transcription - Coordinates a pipeline of speech-to-text, LLMs, and text-to-speech for live audio interactions.
• Speech Service Integrations - Connects to both speech-to-text and text-to-speech providers to enable full audio-text-audio conversion.
• Speech-to-Text Conversions - Transforms real-time audio streams into machine-processable text using both cloud providers and local models.
• Speech Transcription - Converts spoken audio to text using local model files and shared libraries for offline processing.
• Text-to-Speech - Implements a system for synthesizing natural human speech from text input using various provider models.
• Asynchronous Pipeline Coordination - Coordinates speech recognition, response generation, and synthesis to manage asynchronous audio streams.
• Tool Calling - Provides mechanisms for language models to request and execute external functions or scripts to perform real-time actions.
• Voice Assistants - Allows the definition of agent personas, system prompts, and synthetic voice characteristics.
• Cross-Channel Audio Routing - Streams voice data across telephony, WebRTC, and meeting platforms using a unified backend routing mechanism.
• Call Lifecycle Management - Provides comprehensive management of inbound and outbound call lifecycles, including start, termination, and metadata retrieval.
• AI-Powered Inbound Call Answerers - Provides an AI-powered system to answer incoming phone calls and interact with users using conversational agents.
• Webhook-Based Call Acceptors - Implements a webhook-based system to accept incoming phone calls and establish bidirectional media streams for AI processing.
• Outbound Call Initiators - Enables the programmatic initiation of outbound phone calls to specific numbers using defined prompts.
• Real-Time Voice and Video Communication - Provides comprehensive control over the voice call lifecycle, including starting, ending, and monitoring real-time status.
• Real-Time Voice Backend Hosting - Exposes websocket routes to handle real-time streaming voice interactions between clients and the server.
• Call Lifecycle Management - Manages the full lifecycle of voice conversations, including starting, terminating, and tracking active session status.
• Telephony Session Managers - Provides a specialized integration layer to attach conversational agents to phone lines, WebRTC, and virtual meetings.
• Call Traffic Routing - Maps phone numbers to specific AI agents to route incoming and outgoing telephony traffic.
• Speech Synthesis Providers - Integrates modular plugins for connecting external text-to-speech services to customize agent voice characteristics.
• Local Agent Deployments - Executes real-time voice conversations on local hardware to ensure privacy and low latency.
• Agent Instantiation - Implements mechanisms to create active agent instances based on specific configuration for custom behavior integration.
• Multi-Provider Swapping - Provides a swappable interface to switch between different speech-to-text, text-to-speech, and language model services.
• Phrase-Based Action Execution - Executes specific behaviors and controls conversation flow after the agent speaks a predefined phrase.
• Custom Agent Flow Definitions - Allows developers to define custom response logic by subclassing a base agent class for real-time conversations.
• Reasoning Pipelines - Connects agents to multi-stage prompting chains to enable retrieval augmented generation and complex decision making.
• Hybrid Local-Remote AI Routing - Routes audio and text processing tasks between local binaries and cloud APIs to balance latency and connectivity.
• Local Model Generation - Enables the generation of conversational responses using local large language model binaries to avoid external network calls.
• Synthetic Speech Generation - Utilizes speech synthesis providers to generate natural-sounding synthetic voices from text input.
• Voice Discovery Interfaces - Provides APIs and utilities to query and retrieve available voice identifiers and metadata from synthesis providers.
• Latency Adjustments - Scales pauses before AI responses based on speed coefficients or user speaking rates.
• User Interruption Detection - Detects and manages user interruptions to determine when the AI should stop speaking.
• Provider-Agnostic Model Interfaces - Provides abstraction layers that standardize inputs and outputs across multiple transcription, synthesis, and LLM providers.
• Responsiveness Tuning - Adjusts speech endpoint detection and response speed to refine the natural flow of dialogue.
• Speech Synthesis - Converts text to spoken audio using configurable voices across a wide range of supported locales.
• Incremental Synthesis - Generates speech incrementally via WebSockets or gRPC to minimize latency in real-time conversations.
• Unified Model Interfaces - Provides a unified interface for connecting voice agents to diverse speech-to-text, language, and text-to-speech models.
• Telephony-Integrated Autonomous Agents - Provides telephony capabilities as a tool for autonomous agents to make calls and act on outcomes.
• Local Speech Synthesis - Provides the ability to run speech synthesis models locally for improved privacy and lower latency.
• Tool Integrations - Equips voice assistants with the ability to trigger external APIs, webhooks, and software actions.
• Voice Cloning - Supports the creation of synthetic replicas of human voices for personalized AI interactions.
• Telephony Routing - Routes voice calls through centralized telephony infrastructure to enable phone-based conversations.
• Multi-Platform Meeting Services - Integrates AI agents with multiple virtual meeting platforms to interact with live participants.
• AI Meeting Participants - Integrates AI agents into virtual meeting platforms via dial-in to facilitate real-time conversations.
• Agent Tooling and Event Configurations - Equips agents with specific actions and configures webhooks to notify external systems of conversation events.
• Conversation Event Webhooks - Triggers external HTTP webhooks based on conversation lifecycle events such as call endings or machine detection.
• Cross-Platform Audio Routing - Streams voice data across telephony, web interfaces, and meeting platforms for cross-platform communication.
• Speech Synthesis & TTS - Converts text into spoken audio and directs the output to available system speakers.
• Answering Machine Detection - Determines if a call is answered by a voicemail system to decide whether the agent should continue or hang up.
• Call Transferring - Redirects current voice conversations to different phone numbers to perform hand-offs to other users.
• Multi-Channel Voice Deployment - Enables voice interactions over telephony, web, and meeting software across different communication channels.
• Automated Outbound Dialers - Provides automated outbound dialing capabilities to start voice conversations triggered by scripts.
• WebRTC Audio Streaming - Connects agents to web applications using the WebRTC protocol for real-time audio interaction.
• Programmatic Termination Logic - Implements programmatic logic to terminate active conversations based on user phrases or internal agent state.
• Voice-Enabled Web Interfaces - Creates web-based applications that allow users to interact with AI via spoken conversation.

Star history