Mate Engine

Mate-Engine is a 3D desktop avatar engine designed to render interactive characters that float on the desktop and react to system events. It functions as a virtual assistant platform, combining an interactive character renderer with an interface that connects local language models to 3D avatars for desktop conversations.

The engine features a custom 3D model loader that imports third-party humanoid models using standard rigging and bone naming conventions. It includes an audio-reactive animation system that monitors live audio output from applications to automatically trigger dance sequences and movements.

The system manages character behavior through a looping state machine and interactive controls, such as gaze tracking that follows the mouse cursor and gestures triggered by defined touch regions. It renders content on a transparent window overlay that can be anchored to the system taskbar or active windows.

Users can adjust visual performance settings, including lighting, bloom, and animation speed, to optimize the engine for different hardware.

Features

Interactive Desktop Companions - Renders an interactive 3D character as a persistent on-screen companion that reacts to user activity.

AI Desktop Assistants - Functions as a desktop-resident AI assistant that connects local language models to an interactive 3D avatar.

AI Chat Interfaces - Provides a conversational interface connecting users to local language models via 3D avatars.

Local AI Integrations - Connects 3D avatars to self-hosted local language models for interactive conversations.

API-Free Local Inference - Integrates local language models to generate character dialogue without requiring external API calls.

3D Model Importers - Provides a system for loading user-provided 3D models that adhere to standard rigging conventions.

Interactive Gaze and Gesture Systems - Implements a visual engine that animates character gaze and gestures by tracking mouse position and touch regions.

3D Rendering Engines - Provides a rendering engine for interactive 3D characters that float on the desktop.

Model Loaders - Provides utilities for importing external 3D humanoid models into the rendering pipeline using standard rigging conventions.

Skeletal Rig Mapping - Enables the import of third-party humanoid models by matching their bone naming to a standard skeletal rig.

Desktop Overlay Renderers - Renders 3D models as idle desktop companions with looping animations and floating movement on a transparent overlay.

Transparent Overlays - Renders the 3D character on a transparent system window overlay positioned above other applications.

Audio Trigger Detection - Monitors audio output from applications to automatically trigger visual movements and dance sequences.

Avatar Tap Interaction Configurators - Uses mouse tracking and designated touch regions to control character gaze and trigger animation sequences.

Audio-Reactive Avatars - Triggers character dance sequences and movements automatically by monitoring real-time audio output.

Lightweight State Machines - Uses a state machine to manage transitions between idle, dancing, and reacting animations.

Audio-Driven Animation Engines - Implements a system that triggers dance and movement sequences based on real-time system audio output.

Bone Rotation Tracking - Updates 3D model joint rotations so the character's gaze and posture follow the mouse cursor.

Gaze Tracking Animations - Moves the character's eyes, head, and spine to follow the mouse cursor for a natural appearance.

System Window Anchoring - Provides a positioning system to anchor the character overlay on top of active windows or the system taskbar.

shinyflvreMate-Engine

Features

Star history