Awesome GitHub RepositoriesSpeaker Embeddings

This project is a neural text-to-speech engine and voice cloning toolkit designed to generate synthetic speech that mimics the vocal characteristics of a target speaker. It functions as a real-time audio synthesizer, utilizing a deep learning pipeline to convert written text into high-fidelity speech output with minimal latency. The system employs a transfer learning framework that leverages pre-trained speaker verification models to adapt synthesis to new, unseen vocal identities. By using an encoder-based speaker embedding process, the toolkit maps variable-length audio samples into a laten

LocalAI is a local generative AI platform and inference engine designed to host large language, vision, and audio models on private hardware. It functions as an API compatible gateway that mimics proprietary service endpoints, allowing existing third-party software to integrate with a self-hosted backend. The platform distinguishes itself as a distributed AI model orchestrator, capable of scaling inference across machine clusters using VRAM-aware routing and hardware coordination. It provides a unified interface for diverse open-source backends and supports self-hosted RAG infrastructure thro

هذا المشروع عبارة عن مجموعة أدوات لتحويل النص إلى كلام تعتمد على التعلم العميق وتستخدم لتدريب ونشر نماذج تركيب الكلام العصبية. يوفر إطار عمل شاملاً لتحويل النص المكتوب إلى صوت منطوق، باستخدام مشفرات صوتية عصبية لتحويل المخططات الطيفية المركبة إلى أشكال موجية صوتية عالية الدقة. تتضمن مجموعة الأدوات نظام استنساخ صوتي يكرر أصواتاً بشرية محددة عن طريق استخراج تضمينات المتحدث من عينات صوتية قصيرة. كما يدعم تركيب الصوت متعدد المتحدثين، مما يسمح بتوليد الكلام عبر هويات صوتية مختلفة باستخدام بنيات نماذج متخصصة. يغطي النظام خط أنابيب تركيب الكلام بالكامل، بما في ذلك أدوات لتنظيم مجموعة بيانات الكلام، وتدريب النماذج المخصصة مع تتبع الأداء، وواجهة سطر أوامر لتوليد الصوت. للوصول إلى الشبكة، يوفر خادم HTTP مستضاف ذاتياً لنشر نماذج تركيب الكلام كواجهة برمجة تطبيقات (API).

ChatTTS is a conversational text-to-speech generative model designed to convert written dialogue into natural sounding audio. It functions as a multilingual speech synthesis framework capable of producing human-like audio across different languages and speaker profiles. The system is distinguished by its ability to generate interactive dialogue with realistic vocal nuances. It utilizes a speech nuance controller to insert specific tokens that trigger non-verbal elements, such as laughter, pauses, and interjections, during the synthesis process. The project includes a streaming audio generato

This project is a generative speech synthesis engine that converts text into high-fidelity human speech. It utilizes a two-stage autoregressive transformer architecture that separates semantic token prediction from acoustic detail reconstruction to balance linguistic accuracy with audio quality. The system is designed to support multilingual output and conversational AI development, enabling the generation of context-aware speech that maintains flow across multiple dialogue turns. The platform distinguishes itself through a production-ready inference server that employs continuous batching to

TensorFlow.js is a JavaScript machine learning library used for training and deploying models in web browsers and server-side environments. It functions as a browser-based model trainer, a WebAssembly inference engine, and a WebGPU accelerated tensor library for low-level linear algebra. The project also includes a model converter to transform Python-based models into optimized formats for JavaScript execution. The library distinguishes itself through a pluggable backend architecture that allows mathematical operations to be executed via CPU, WebGL, or WebGPU. It supports the conversion of Py

Index-tts is a neural audio generation engine designed to convert written text into high-fidelity human speech. By utilizing deep learning models and phoneme-based sequence modeling, the system transforms text into natural-sounding audio waveforms suitable for a variety of accessibility and media applications. The platform functions as a server-side inference pipeline that provides a programmatic interface for integrating voice generation into external applications. It distinguishes itself through asynchronous audio streaming, which buffers and delivers generated speech chunks in real time to

Sherpa-ONNX is an ONNX-based speech processing toolkit that provides a local speech recognition engine, an on-device voice synthesis tool, and a speaker identification framework. It is designed as a cross-platform speech API that enables speech-to-text, text-to-speech, and speaker verification tasks to be executed locally on a device without requiring network access. The project is distinguished by its ability to perform zero-shot voice cloning and speaker diarization on-device. It supports a wide range of hardware accelerations, including GPU and various NPU architectures, and provides a Web

هذا المشروع عبارة عن مكتبة وإطار عمل للذكاء الاصطناعي القابل للتفسير لرؤية الحاسوب لـ PyTorch، يوفر مجموعة من الأدوات لتصور وتدقيق عمليات صنع القرار الداخلية للشبكات العصبية العميقة. يعمل كأداة إسناد للشبكة العصبية وأداة تصحيح أخطاء لتحديد مناطق الصورة التي تدفع تنبؤات النموذج. تتميز المكتبة بدعمها لطرق الإسناد القائمة على التدرج وغير القائمة على التدرج، مما يسمح بتوليد خرائط حرارية مرئية وخرائط إسناد دون الحاجة إلى تعديلات على كود مصدر النموذج الأصلي. وتتميز أيضاً باكتشاف المفهوم المرئي، باستخدام تحليل المصفوفة لتفكيك التنشيطات الداخلية إلى أنماط قابلة للتفسير ورسم خرائط التضمينات الكامنة لأهمية البكسل. يغطي إطار العمل مجموعة واسعة من القدرات بما في ذلك توليد الخرائط الحرارية وتحسينها، والتحويل المكاني لبنيات مثل محولات الرؤية، والتكيفات لأهداف الرؤية متعددة المهام مثل اكتشاف الكائنات والتجزئة الدلالية. يتضمن أيضاً مجموعة تقييم دقة النموذج التي تستخدم تحليل الاضطراب، ودراسات الاستئصال، وقياسات التوطين لتحديد مدى إخلاص التفسيرات المولدة. يوفر المشروع آليات لربط التنشيط الديناميكي، وتكيف البنية المخصص، وتكوين الهدف الموجه لربط أدوات القابلية للتفسير بمخرجات النماذج المختلفة.

Omi is an open-source wearable AI platform that captures audio and screen data to provide real-time conversational assistance and memory. It integrates a wearable hardware development kit with a vector memory database and large language model capabilities to create a persistent digital record of user interactions. The platform is distinguished by its BLE audio streaming pipeline, which transmits raw audio from wearable hardware for real-time transcription and speaker identification. It utilizes a plugin-based agent tool framework that allows AI assistants to autonomously invoke custom functio

PaddleSpeech is a comprehensive toolkit of neural models for speech recognition, synthesis, and translation built on the PaddlePaddle deep learning framework. It provides a collection of frameworks and tools for converting spoken audio into written text, synthesizing natural audio from text, and performing direct speech translation. The toolkit includes specialized capabilities for keyword spotting to detect trigger words and speaker verification systems that extract unique voiceprints to identify and distinguish between individuals. It also features end-to-end translation tools that map audi

Spark-TTS is a deep learning text-to-speech synthesis engine designed to convert written text into high-fidelity audio. It utilizes a transformer-based architecture and autoregressive sequence modeling to generate coherent speech, transforming linguistic input into natural-sounding waveforms through neural speech codec synthesis. The platform distinguishes itself through zero-shot voice cloning, which allows users to mimic a target speaker’s unique vocal identity using only a short reference audio sample without requiring additional model training. It also features cross-lingual phonetic mapp

Piper is a local neural text-to-speech engine designed to convert written text into natural human speech entirely on your own hardware. By utilizing a neural synthesis framework, it operates without the need for internet connectivity, ensuring that all audio generation remains private and secure. The system distinguishes itself through a modular architecture that allows for the dynamic loading of speaker embeddings and voice configurations. This enables users to switch between various vocal personas and styles without requiring a full reload of the core synthesis model. By processing input th

This project is a comprehensive suite for neural speech synthesis, featuring a deep learning text-to-speech engine, a neural speech synthesis trainer, and a voice cloning toolkit. It provides a system for synthesizing human-like speech from text using neural network models and high-fidelity vocoders. The suite includes a speech model conversion utility to transform deep learning models between different formats for deployment across various hardware runtimes. It also provides a self-contained HTTP server to expose pre-trained text-to-speech models as a remote audio API. Capabilities include

Monolith is a distributed recommendation model framework and asynchronous training engine designed to build and train large-scale deep learning architectures. It functions as a distributed model trainer that processes massive datasets across multiple compute nodes using asynchronous update mechanisms. The system features a dedicated embedding table manager that creates unique, feature-isolated tables to prevent representation collisions. It also includes a real-time weight updater to capture immediate changes in user interest and data hotspots through continuous parameter synchronization. Th

This project is an AI voice assistant backend and gateway server designed to connect ESP32 hardware to large language models. It enables real-time conversational AI by processing streaming speech-to-text and text-to-speech interactions, allowing hardware devices to engage in natural language dialogue. The system is distinguished by a modular plugin framework that loads custom feature extensions at runtime and a retrieval-augmented generation engine that queries external knowledge bases for factual accuracy. It further personalizes interactions by using voiceprint mapping to identify individua

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

BERTopic is a topic modeling library used to extract interpretable themes from collections of text documents and images. It functions as a document clustering framework that transforms unstructured data into numerical vectors to group semantically similar content. The project distinguishes itself through a multimodal embedding tool that allows for joint clustering of text and images in a shared vector space. It also features a class-based TF-IDF representation engine to identify representative words for clusters and an integrated system for using large language models to generate natural lang

PyTorch Metric Learning is an open-source library for training neural networks to produce similarity-preserving embedding spaces. It provides a modular framework where interchangeable loss functions, mining strategies, and evaluation tools can be composed to learn representations that map similar items to nearby points and dissimilar items to distant points in the embedding space. The library distinguishes itself through a highly configurable architecture that separates concerns across several interchangeable components. Users can assemble custom loss functions from pluggable distance metrics

StyleTTS2 is an adversarial text-to-speech model that uses style diffusion and large speech language models to generate natural-sounding speech from text input. It combines adversarial training with large pre-trained speech models to improve speech quality and reduce artifacts, while employing a style diffusion process that extracts prosodic and timbral features from reference audio to guide speech generation. The model supports multi-speaker voice synthesis by conditioning the diffusion process on speaker-specific embeddings derived from reference utterances, enabling voice cloning and adapt