Open-LLM-VTuber/Open-LLM-VTuber

Features

• AI Desktop Companions - Ships a fully local AI companion with a Live2D avatar, voice interaction, and offline processing.
• Realtime Voice Conversation Facilitators - Enables spoken dialogue with an AI companion that listens, responds, and can be interrupted mid-speech.
• AI Backend Integrations - Connects to multiple LLM, speech, and TTS backends through a modular configuration interface.
• Conversational Voice AI - Engages in real-time spoken conversations with an AI character using speech recognition and synthesis.
• AI Companion Customizers - Enables importing custom Live2D models, editing prompts, and cloning voices to define a unique companion.
• Environmental Perception Captures - Captures camera feed, screen recordings, or screenshots so the AI can perceive the user's environment and display content.
• Companion Appearance and Persona Customizers - Provides tools to import Live2D models, modify prompts, and clone voices for companion personalization.
• Companion Customizers - Allows defining AI character through prompts, Live2D models, and voice cloning for unique companions.
• OpenAI-Compatible Backend Configurators - Connects to any OpenAI-compatible LLM backend, including local runners and cloud APIs.
• Local LLM Configurations - Connects to locally hosted language models via Ollama or LM Studio for offline inference.
• Multi-Modal AI Toolkits - Connects diverse AI backends for language, vision, and speech through a unified interface with browser control.
• Companion Persona Customizers - Provides tools to define character appearance, voice, and personality through models, prompts, and cloning.
• Real-Time Conversational AI Frameworks - Enables real-time spoken dialogue with a Live2D avatar using low-latency voice interaction.
• Visual Input Captures - Captures camera feed or screen content so the AI can see the user and their environment.
• Character Appearance and Voice Customizers - Provides tools to import Live2D models, modify prompts, and clone voices for companion personalization.
• Voice-Enabled Agents - Provides a fully local AI companion with speech input, on-device LLM responses, and a Live2D avatar.
• Interactive Desktop Companions - Launches a transparent, always-on-top window that stays on the desktop as a virtual pet-like companion.
• Local AI Operations - Operates all core AI functions—voice, vision, and language—entirely offline using local models.
• Live2D Avatar Renderers - Renders and animates Live2D avatars in real-time using WebGL with expressions and click interactions.
• Real-Time Interaction Engines - Enables low-latency bidirectional data streaming between the AI backend and the avatar interface.
• Browser-Based Runtimes - Operates the entire application within a web browser for cross-platform access and remote connectivity.
• Multi-Character Conversations - Enable multiple AI characters to participate in the same conversation simultaneously.
• LLM API Configurations - Connects to language models through OpenAI-compatible APIs or Groq for cloud-based inference.
• AI Backend Abstractions - Provides a unified configuration interface for swapping between LLM, speech, and TTS backends.
• Local-First Tool Processing - Runs all core AI functions on-device without internet, using local models for privacy and independence.
• Live2D Avatar Configurators - Provides a setup interface for Live2D avatars that visually represent AI companions.
• Live2D Model Configurators - Provides a dedicated interface for registering and configuring Live2D models with emotion and interaction mappings.
• Realtime Avatar Renderers - Renders customizable Live2D characters that react with expressions and emotions as AI companions.
• Live2D Avatar Renderers - Renders customizable Live2D models that display expressions and react to clicks based on AI state.
• Live2D Animation Configurators - Ships a configuration system for Live2D model expressions, gestures, and click-triggered animations.
• Character Appearance and Persona Customizers - Enables importing Live2D models, modifying prompts, and cloning voices for companion personalization.
• Long-term Memory Stores - Stores conversation history and uses a memory system for recalling past interactions across sessions.
• External Memory Integrations - Integrates with Letta (MemGPT) to store and recall conversation history across sessions.
• Group Chat Routing - Coordinates multiple AI characters to participate simultaneously in a single conversation.
• Multi-Agent Session Facilitators - Orchestrates multiple AI characters to participate in the same conversation simultaneously.
• Speech Interruption Handlers - Allows the user to speak over the AI without the AI hearing its own output, enabling natural turn-taking.
• Live Camera and Screen Perception - Captures and interprets live camera feed or screen content so the AI can see the user and environment.
• External Memory Integrations - Retains conversation history and long-term context by integrating with external memory systems like Letta.
• Avatar Tap Interaction Configurators - Allows configuration of click-triggered motion sequences on different avatar body parts.
• Persistent Chat Histories - Saves conversation logs so users can resume previous interactions without losing context.
• Tool Invocation Handlers - Provides MCP-based tool invocation for web searches and time queries with status feedback.
• Conversation Memory Retrieval - Stores chat history and long-term memories so the AI recalls previous interactions across sessions.
• Multi-Agent Orchestrators - Manages multiple AI characters in a group chat by coordinating their responses through a plugin system.
• Live2D Model Registrations - Implements a registration system for Live2D models via file placement and JSON configuration.
• Facial Expression Modulators - Links the AI's emotional state to specific Live2D facial expressions for appropriate visual reactions.
• Web-Based Interfaces - Provides a web-based interface accessible from any device on the same network.
• Desktop Overlays - Displays the avatar as a click-through, always-on-top window that integrates with the desktop environment.
• Character Profile Switchers - Loads different Live2D models and settings by selecting named character configurations.
• Live2D Character Importers - Provides a defined setup process for importing new Live2D characters into the system.
• Model-to-Character Assignments - Provides a mechanism to assign Live2D models to character profiles for avatar selection.
• Desktop Overlay Toggles - Toggles the client between a standard window and a transparent, always-on-top overlay that follows the cursor.
• AI-Driven Browser Controllers - Commands an AI to operate a web browser and execute tasks while showing a live view of the browser.
• Remote Browser Controllers - Commands an AI agent to operate a web browser remotely and perform tasks with a live view.
• Browser Interaction Controls - Commands an AI to navigate and interact with a web browser, showing a live view to the user.