Projects sharing features with Open LLM VTuber

Vocode-core is a framework for building real-time conversational AI voice agents. It serves as a conversational orchestrator and pipeline that integrates speech-to-text, large language models, and text-to-speech services to enable low-latency voice interactions. The project features a provider-agnostic interface that allows for swappable speech and language model providers, including support for both cloud APIs and local binaries. It distinguishes itself through a specialized telephony integration layer that enables agents to be deployed across phone lines, WebRTC, and virtual meeting platfor

Personaplex is an LLM speech-to-speech framework and conversational AI persona engine designed for real-time voice interfaces. It provides a system for defining AI identities and vocal characteristics through a combination of text-based role prompts and audio reference files. The project features a real-time AI voice interface that supports full-duplex human-AI dialogue, enabling multiple parties to speak and listen simultaneously via bidirectional audio streaming. It includes a GPU-accelerated audio processor and a speech-to-speech pipeline to facilitate low-latency conversations. The frame

hexo-helper-live2d is a plugin for adding interactive 2D animated characters to static websites generated with the Hexo framework. It functions as a configuration-driven system that embeds and manages character assets within static pages via build-time script automation. The project provides a manager for loading and positioning character assets from local directories or remote URLs. It uses an HTML canvas to render animated models that respond to mouse and touch movements. The system covers character visual configuration, including controls for scale, screen position, opacity, and mobile vi

gptel is an LLM Emacs client and multi-backend AI integration system that allows users to interact with large language models directly inside the Emacs text editor. It serves as an AI-powered text refactoring tool and a context-aware prompt manager, providing a unified interface to connect with various AI providers, including local Ollama instances, AWS Bedrock, and Gemini. The project distinguishes itself as a Model Context Protocol client, connecting to MCP servers to provide language models with external tools and data sources. It enables context-augmented prompting by aggregating text fro

Duix-Mobile is a software development kit for deploying real-time conversational AI characters on mobile devices. It enables the creation of interactive digital humans capable of fluid voice-to-voice interactions, featuring low-latency speech recognition and synchronized lip movements. The project distinguishes itself through the ability to integrate custom external language models and speech providers to define an avatar's intelligence and voice. It supports the generation of real-time multilingual subtitles and provides mechanisms to track the training status of newly created digital charac

Vocode-python is an LLM voice AI framework and orchestration library used to build real-time conversational agents. It functions as a speech-to-speech pipeline that integrates large language models with speech-to-text and text-to-speech services to facilitate continuous spoken dialogue. The framework acts as a telephony integration gateway, connecting voice agents to phone lines and video conferencing platforms through automated calling infrastructure. It provides the means to deploy voice channels to external communication platforms and host telephony servers. The library manages the end-to

lollms-webui is a web-based user interface and local AI model orchestrator designed for interacting with and managing large language models and multimodal AI on local hardware. It functions as a generative AI multimedia suite that enables the creation of text, images, video, and music through integrated diffusion and language models. The project features a dedicated persona manager to configure behavioral profiles and distinct personalities, controlling the style and tone of model responses. It includes a local memory system for maintaining long-term conversation context and chat history via

Pipecat is a framework and software development kit for building real-time multimodal AI agents and speech-to-speech systems. It utilizes a frame-based data pipeline to route audio, video, and text through a modular sequence of processors, enabling the orchestration of low-latency conversational AI. The project is distinguished by its ability to coordinate complex multimodal services, including speech-to-text, language models, and text-to-speech, within a single pipeline. It features semantic voice activity detection for natural turn-taking, state-machine conversation flows for dialogue manag

NeMo is a multimodal AI framework and toolkit designed for the development, training, and scaling of large language models, generative AI systems, and speech-based models. It functions as an automatic speech recognition toolkit, a text-to-speech engine, and a framework for building models that process and generate combinations of text, image, and audio data. The project serves as a conversational AI orchestrator capable of managing real-time, interruptible voice interactions. It provides specialized workflows for speech translation, converting spoken audio from one language into text or speec

Fonoster is a conversational AI framework and multi-tenant communications platform as a service. It serves as a programmable voice gateway and SIP telephony platform, enabling the creation of voice-based assistants and automated communication workflows using large language models. The project distinguishes itself through a vendor-agnostic speech integration engine that abstracts speech-to-text and text-to-speech providers. It features a multi-tenant architecture that isolates telephony resources and user identities into distinct organizational workspaces. The system covers a broad range of t

This project is a framework for developing multimodal AI agents that function as programmable participants in real-time communication rooms. It enables the construction of agents that can see, hear, and speak by integrating speech-to-text, large language models, and text-to-speech pipelines to facilitate low-latency, natural conversations. The system is distinguished by its advanced orchestration of real-time media and conversational flow, including support for full-duplex speech, preemptive response generation, and sophisticated interruption management. It further differentiates itself throu

Airi is an interactive digital companion engine designed to bridge large language models with local animation rendering. It functions as a middleware platform that synchronizes conversational text streams with skeletal and facial movements to drive virtual avatars in real time. The framework distinguishes itself by integrating desktop context awareness, allowing characters to maintain situational awareness of a user's screen activity across both desktop and web environments. It utilizes a hybrid execution model that splits computational workloads between cloud-based language processing and lo

This is an open-source Python SDK for building and orchestrating production-grade AI agents. It provides a unified framework for creating conversational agents that can use tools, maintain state, and coordinate across multiple language model providers including OpenAI, Anthropic, Google, Amazon Bedrock, and locally-hosted models. The SDK supports multi-agent orchestration through graphs, teams, and swarms, allowing several specialized agents to collaborate on complex tasks. Agents can be composed as callable tools that other agents invoke, and the framework includes policy handlers that inspe

🎙️🤖Create, Customize and Talk to your AI Character/Companion in Realtime (All in One Codebase!). Have a natural seamless conversation with AI everywhere (mobile, web and terminal) using LLM OpenAI GPT3.5/4, Anthropic Claude2, Chroma Vector DB, Whisper Speech2Text, ElevenLabs Text2Speech🎙️🤖

This project is a ChatGPT-powered bot integrated into the Feishu chat platform, enabling conversational AI, image generation, and voice interaction within both private and group chats. It connects large language models to the team messaging environment, allowing users to engage in multi-topic conversations, generate images from text prompts, and analyze uploaded images using vision models. The bot supports persona preset switching, allowing predefined role templates to modify the AI's behavior and tone with a single command. It maintains separate conversation histories per topic using session

WebGAL is a web-native development environment designed for the creation and publication of interactive narrative games. It provides a browser-based platform that enables users to build visual novels featuring branching dialogue and multimedia assets without requiring manual coding. The platform distinguishes itself through a visual editor that utilizes node-based logic definition to manage narrative flow and scene transitions. It includes specialized tooling for the configuration and integration of animated character models, allowing for the direct implementation of interactive character seq

This project is a self-hosted web interface and desktop application designed for interacting with language models. It provides a private platform for managing conversational sessions, allowing users to connect to external AI services while maintaining control over their interaction history and configuration settings. The application distinguishes itself by offering a unified interface that supports multimodal inputs and outputs, including voice interaction processing and generative image creation. It secures sensitive credentials by routing requests through a backend proxy and ensures data pr

PilotDeck is an LLM agent orchestration platform and agentic workflow manager designed to build and manage task-oriented AI agents. It serves as a multi-model router and an MCP server integration hub, connecting language models to external tools and data sources via the Model Context Protocol. The platform distinguishes itself through a transparent AI memory management system that allows users to view, edit, and delete memory entries to correct hallucinations. It implements a routing layer that directs tasks to the most suitable model based on complexity and performance requirements, supporti

Short video factory is a local AI content generator and automated video editing tool. It provides a production pipeline that uses large language models to transform text prompts into marketing scripts and rendered short-form videos. The system is designed for local-first execution, running all processing and asset management on the host machine to maintain data privacy. It distinguishes itself through a batch-processing workflow that can sequentially execute copywriting and rendering for multiple items using predefined presets. The software covers a broad range of media capabilities, includi

MiniCPM is a collection of small language models designed for local, on-device deployment in resource-constrained environments. The project focuses on running dense Transformer models on consumer hardware, including GPUs, CPUs, and Apple Silicon, without requiring custom code forks. The project distinguishes itself through heavy optimization for edge hardware, utilizing quantized weight compression in GGUF and MLX formats to reduce memory overhead. It implements advanced inference techniques such as speculative sampling and radix-tree prefix caching to accelerate generation speed and throughp

BiliNote is a tool that converts video URLs into structured, organized notes. It works by extracting video content and metadata from major platforms, transcribing audio to text entirely on-device using a local speech recognition model, and then summarizing the transcript with a language model to produce clean notes that can include screenshots and timestamp links. What sets BiliNote apart is its configurable AI backend, which lets you choose and switch between different language model providers for generating summaries. All transcription happens offline and locally, preserving privacy and ena

Ten Framework is a multimodal large language model agent framework designed for building low-latency conversational agents. It integrates voice, text, and visual inputs in real time to facilitate human interaction. The project includes a real-time speech processing pipeline for streaming transcription, voice activity detection, and speaker diarization. It also features an avatar synchronization engine that coordinates character lip animations and visual outputs with synthesized speech. The framework covers edge AI deployment through containerized packaging and direct integration with embedde

RealtimeVoiceChat is a conversational AI gateway and voice chat interface that enables real-time spoken interactions with large language models. It functions as a streaming audio pipeline that integrates speech-to-text transcription and text-to-speech synthesis to facilitate natural voice conversations. The system utilizes WebSocket-based audio streaming and an asynchronous text-to-speech pipeline to maintain low latency. It includes specialized conversational controls such as silence-based turn detection and client-side interrupt handling, which stops active audio playback when new user spee

Mate-Engine is a 3D desktop avatar engine designed to render interactive characters that float on the desktop and react to system events. It functions as a virtual assistant platform, combining an interactive character renderer with an interface that connects local language models to 3D avatars for desktop conversations. The engine features a custom 3D model loader that imports third-party humanoid models using standard rigging and bone naming conventions. It includes an audio-reactive animation system that monitors live audio output from applications to automatically trigger dance sequences

WebContainer is a browser-based runtime environment designed to execute server-side code, operating system commands, and full-stack development toolchains directly within a web tab. It provides the infrastructure for cloud IDEs and zero-install development workflows by simulating a runtime that eliminates the need for local installations or remote virtual machines. The system leverages WebAssembly to map system calls and implements a virtual POSIX-compliant filesystem and network interception layer. This allows the runtime to spawn command-line processes, execute shell commands, and route int

This project is an AI voice assistant backend and gateway server designed to connect ESP32 hardware to large language models. It enables real-time conversational AI by processing streaming speech-to-text and text-to-speech interactions, allowing hardware devices to engage in natural language dialogue. The system is distinguished by a modular plugin framework that loads custom feature extensions at runtime and a retrieval-augmented generation engine that queries external knowledge bases for factual accuracy. It further personalizes interactions by using voiceprint mapping to identify individua