Open-source alternatives to Vocode Python

LiveKit is a comprehensive framework for building and orchestrating real-time, multimodal AI agents that interact with users through voice, video, and text. It provides a centralized, event-driven architecture to manage the entire lifecycle of automated participants, from initialization and session state management to graceful shutdown. By utilizing a selective forwarding unit, the platform efficiently routes media streams between participants and agents, ensuring low-latency communication and secure, token-based authentication for all connections. The platform distinguishes itself through it

This project is a Python framework for building autonomous, event-driven agent systems. It provides a unified runtime for orchestrating multi-agent workflows, managing persistent conversation state, and executing code within secure, isolated sandbox environments. The framework is designed to handle complex task delegation, allowing agents to invoke other agents as tools while maintaining context across multi-turn interactions. The framework distinguishes itself through its deep integration with the Model Context Protocol, enabling agents to connect to external data sources and remote services

PraisonAI is an autonomous AI agent platform that coordinates multiple LLM-powered agents for research, planning, and execution of complex workflows. It functions as a multi-agent orchestration framework, a workflow builder, and a Model Context Protocol server, while also providing retrieval-augmented generation through vector knowledge bases. Agents can interact via CLI, web, or standardized protocols with sandboxed code execution. The platform distinguishes itself with a rich set of agent communication protocols, including A2A, REST, WebSocket, voice and telephony integration, and MCP, allo

This project is a framework for building local voice assistants and a real-time audio streaming server. It functions as a containerized inference engine and a multilingual speech pipeline that orchestrates speech-to-text, language models, and text-to-speech components to convert spoken input into spoken output. The system is distinguished by its use of WebSocket-based bidirectional streaming for low-latency interactions. It features a voice activity detection system that manages speech boundaries and handles user barge-in interruptions during assistant playback. It also supports custom voice

This project is a comprehensive toolkit for on-device speech recognition, synthesis, and audio processing, specifically engineered for Apple Silicon. It provides a framework for building real-time, full-duplex voice agents that operate entirely offline, leveraging native hardware acceleration to maintain performance and privacy. By utilizing optimized machine learning models, the library enables local execution of complex audio tasks without reliance on external cloud services. The library distinguishes itself through its specialized focus on local, high-performance voice interaction. It incl

Personaplex is an LLM speech-to-speech framework and conversational AI persona engine designed for real-time voice interfaces. It provides a system for defining AI identities and vocal characteristics through a combination of text-based role prompts and audio reference files. The project features a real-time AI voice interface that supports full-duplex human-AI dialogue, enabling multiple parties to speak and listen simultaneously via bidirectional audio streaming. It includes a GPU-accelerated audio processor and a speech-to-speech pipeline to facilitate low-latency conversations. The frame

NeMo is a multimodal AI framework and toolkit designed for the development, training, and scaling of large language models, generative AI systems, and speech-based models. It functions as an automatic speech recognition toolkit, a text-to-speech engine, and a framework for building models that process and generate combinations of text, image, and audio data. The project serves as a conversational AI orchestrator capable of managing real-time, interruptible voice interactions. It provides specialized workflows for speech translation, converting spoken audio from one language into text or speec

LiveTalking is an interactive talking head engine and AI avatar management platform designed to synchronize synthetic speech with facial movements. It functions as a real-time orchestrator that connects large language models and text-to-speech services to neural-rendered digital humans. The project distinguishes itself through low-latency streaming capabilities and the ability to handle real-time conversational interruptions. It supports advanced audio-visual customization, including human voice cloning and the ability to drive avatar expressions using real-time webcam data. The platform cov

Fonoster is a conversational AI framework and multi-tenant communications platform as a service. It serves as a programmable voice gateway and SIP telephony platform, enabling the creation of voice-based assistants and automated communication workflows using large language models. The project distinguishes itself through a vendor-agnostic speech integration engine that abstracts speech-to-text and text-to-speech providers. It features a multi-tenant architecture that isolates telephony resources and user identities into distinct organizational workspaces. The system covers a broad range of t

This project is an Android SDK designed to integrate conversational voice interfaces into mobile applications. It utilizes a Model-View-ViewModel architecture to separate business logic from user interface components, providing a structured framework for managing complex voice interactions. The library features a modular pipeline that handles the entire flow of a conversation, including microphone input capture, speech transcription, intelligent response generation, and audio synthesis. A key differentiator is its ability to analyze acoustic features from audio input to detect user emotional

RealtimeVoiceChat is a conversational AI gateway and voice chat interface that enables real-time spoken interactions with large language models. It functions as a streaming audio pipeline that integrates speech-to-text transcription and text-to-speech synthesis to facilitate natural voice conversations. The system utilizes WebSocket-based audio streaming and an asynchronous text-to-speech pipeline to maintain low latency. It includes specialized conversational controls such as silence-based turn detection and client-side interrupt handling, which stops active audio playback when new user spee

MiniCPM-V is a multimodal large language model and vision-language system designed for complex visual and linguistic understanding. It functions as an on-device AI model, providing the capacity to process text, images, and video as a compact neural network. The project is specifically developed as an edge AI framework, utilizing quantization and weight sharding to run on memory-constrained mobile chipsets. This allows for the deployment of multimodal intelligence directly on mobile operating systems for local inference. Its capabilities cover multimodal content analysis of high-resolution im

This project is a comprehensive framework for building AI-powered applications, providing a unified toolkit for orchestrating language models, autonomous agents, and interactive user interfaces. It serves as a central library for managing the entire lifecycle of AI interactions, from initial prompt generation and model provider abstraction to complex, multi-step reasoning and tool execution. The framework distinguishes itself through its deep integration with frontend development, specifically by enabling generative user interfaces that render dynamic components directly from model outputs. I

This project is a framework for building custom AI chatbots capable of PDF document analysis. It implements Retrieval Augmented Generation to connect a large language model to private document data. The system utilizes graph-based agent orchestration to control conversation flow and decision logic. It maintains context across interactions through thread-based state management and delivers AI responses to the user interface via real-time streaming. The project covers PDF document ingestion through chunk-based processing and vector-store retrieval. It includes mechanisms for query-based data r

This project is a Java-based framework integration that provides an AI agent runtime, a graph-based AI workflow engine, and an LLM orchestration framework for Spring applications. It enables the development of stateful autonomous agents and the implementation of retrieval-augmented generation systems using document processing and vector databases. The framework distinguishes itself through a graph-based workflow runtime for designing complex AI pipelines with conditional routing and persistent state. It supports multi-agent orchestration via service-discovery coordination and provides human-i

Mastra is an orchestration framework designed for building, deploying, and managing autonomous AI agents and multi-agent systems. It provides a comprehensive suite of primitives for creating resilient AI applications, including durable workflow orchestration, event-driven agent loops, and semantic memory management. By integrating these core components, the platform enables developers to build complex, multi-step processes that can reason about goals and execute tasks without manual intervention. The framework distinguishes itself through its focus on observability and secure, isolated execut

Casibase is an open-source platform that orchestrates multi-turn conversations with large language models and manages retrieval-augmented knowledge bases from a single interface. It provides a unified system for connecting to over 30 AI model providers, ingesting documents into vector embeddings for semantic search, and running autonomous agent loops that can drive a browser, search the web, execute commands, and integrate with external tools. The platform distinguishes itself by combining AI conversation management with infrastructure and application orchestration capabilities. It includes a

Ten Framework is a multimodal large language model agent framework designed for building low-latency conversational agents. It integrates voice, text, and visual inputs in real time to facilitate human interaction. The project includes a real-time speech processing pipeline for streaming transcription, voice activity detection, and speaker diarization. It also features an avatar synchronization engine that coordinates character lip animations and visual outputs with synthesized speech. The framework covers edge AI deployment through containerized packaging and direct integration with embedde

This project is a Python speech recognition library that serves as a unified interface for converting spoken audio into text. It functions as a bridge between Python applications and a variety of speech-to-text engines, providing a consistent way to interact with both local and cloud-based recognition services. The library distinguishes itself as a multi-engine transcription tool, wrapping diverse online APIs and offline recognition backends into a standardized format. This allows for interchangeable recognition engines and supports multilingual audio transcription through various language pa

Qwen2.5-Omni is an omnichannel multimodal large language model designed to process and generate content across text, audio, vision, and video. It functions as a real-time speech AI, utilizing an end-to-end architecture to maintain synchronous voice conversations with low-latency responses. The project emphasizes efficiency through quantized edge models, allowing for local inference on mobile hardware and resource-constrained devices. It employs 4-bit weight quantization, CPU-based process offloading, and on-demand weight loading to reduce GPU memory requirements. The system integrates specia

This is an open-source Python SDK for building and orchestrating production-grade AI agents. It provides a unified framework for creating conversational agents that can use tools, maintain state, and coordinate across multiple language model providers including OpenAI, Anthropic, Google, Amazon Bedrock, and locally-hosted models. The SDK supports multi-agent orchestration through graphs, teams, and swarms, allowing several specialized agents to collaborate on complex tasks. Agents can be composed as callable tools that other agents invoke, and the framework includes policy handlers that inspe

ollama-js is a JavaScript client library and API wrapper that provides a programmatic interface for interacting with and managing large language models. It enables the execution of models in both local and cloud-based environments, facilitating the generation of conversational text and the management of model lifecycles. The project distinguishes itself by offering specialized tools for model administration, including the ability to download, create, and remove models, as well as the capacity to define custom model blueprints and prompt templates. It also provides a vector embedding client fo

Context-Engineering is a prompt engineering framework and cognitive architecture for large language models. It provides a set of patterns and methodologies for designing structured prompts and modular reasoning flows that decompose complex tasks into specialized, step-by-step problem solving templates. The project distinguishes itself through stateful prompt management and context window optimization. It maintains persistent memory across multiple interaction turns by compressing conversation history into compact internal state cells and employs techniques to maximize information density per

Chatterbox is a comprehensive machine learning platform designed for multilingual speech synthesis and real-time audio generation. It functions as an engine that converts text into natural-sounding speech, capable of replicating specific human vocal characteristics and emotional expressions from short audio samples. The platform distinguishes itself through advanced control over the synthesis process, allowing for the manipulation of emotional intensity and the injection of non-verbal vocalizations such as laughter or coughing. It is engineered for low-latency performance, utilizing an optimi

OpenHands is an autonomous agent framework designed for software engineering workflows. It provides a modular platform for orchestrating AI agents that reason, plan, and execute tasks within isolated, containerized development environments. By integrating with standard version control and development tools, the system enables agents to autonomously navigate codebases, implement features, and resolve issues through iterative reasoning and tool execution. The platform distinguishes itself through a model-agnostic orchestrator that connects diverse language models to a unified tool registry. It