Open-source alternatives to Lyra

Mattermost Server is an open source collaboration suite and self-hosted team communication platform. It functions as an enterprise chat server and extensible messaging backend that provides a secure environment for professional organizations to exchange messages and coordinate software development. The platform is designed as an AI-integrated communication hub, embedding artificial intelligence into messaging flows to assist users with information retrieval and task completion. It supports a high degree of customization through a plugin-based extension model, allowing for the creation of cust

Heartlib is an audio processing library for large language models that provides tools for audio tokenization, compression, and cross-modal alignment. It implements core models for audio-text embedding, automatic speech recognition, neural codecs, and text-driven audio synthesis. The project features a text-to-audio synthesis engine capable of generating high-fidelity music and speech from text descriptions or reference files. It also includes a neural audio codec designed for low-bitrate compression that preserves acoustic structure and sound quality. Additional capabilities cover audio-text

Moshi is a real-time voice foundation model and speech-to-speech framework designed for bidirectional, low-latency conversations. It functions as a full-duplex voice interface that processes audio and text concurrently in a single stream, enabling natural human-machine dialogue without sequential processing delays. The system utilizes a neural audio codec to compress high-fidelity audio into low-bitrate tokens for efficient transmission. To manage complex responses and reasoning, it employs internal monologue modeling, which generates a hidden stream of thought tokens alongside audible speech

This project is a PyTorch implementation of a discrete variational autoencoder designed to compress high-resolution imagery into discrete latent representations. It functions as an image autoencoder that encodes visual data into discrete codes and decodes those codes back into reconstructed images. The system utilizes a latent space image compressor to convert images into a compressed, discrete format. This allows for generative image synthesis and the analysis of image compression by transitioning between raw pixels and discrete code sequences. The implementation covers latent space visuali

This project is an educational course and collection of training materials focused on generative diffusion models. It provides a curriculum and practical guides for training, fine-tuning, and deploying models capable of synthesizing images, audio, and video. The material covers specific implementation strategies including noise-based synthesis, iterative refinement, and latent space compression. It provides instruction on guiding generative outputs through conditional synthesis and prompt adherence optimization, as well as techniques for image inpainting and text-based editing. The project i

Bert-VITS2 is a neural speech synthesis system and AI voice generator designed to convert written text into natural sounding audio. It utilizes a VITS2 engine and a neural speech synthesis model to produce high-fidelity human voices. The system incorporates a multilingual BERT language processor to improve the prosody and emotional accuracy of the generated speech. It supports multilingual voice generation and custom voice cloning to replicate specific human speech patterns and tones. The architecture covers text-to-speech synthesis through a multi-stage pipeline involving phoneme alignment,

DiT is a latent diffusion model and transformer-based generative AI framework implemented in PyTorch. It functions as a class-conditional image generator that replaces traditional convolutional backbones with a transformer architecture to synthesize high-fidelity images. The project utilizes patch-based latent processing and latent space compression to operate on low-dimensional image representations. It incorporates class-conditional guidance and adjustable guidance scales to control the visual content of generated images during the sampling process. The framework covers distributed model t

Amphion is an audio generation toolkit designed for the research and development of models that synthesize speech, music, and environmental sound effects. It provides a standardized framework for reproducible audio synthesis, incorporating a text-to-speech engine and a voice conversion framework. The project specializes in transforming audio identities, allowing for the modification of speaker accents and voice identities while preserving original rhythm and style. It also includes capabilities for singing voice synthesis and the generation of environmental soundscapes from text descriptions

Stable-audio-tools is a toolkit for training and deploying latent diffusion models for high-fidelity audio synthesis. It provides a framework for generating audio by iteratively refining noise within a compressed latent space, using specialized encoders to preserve temporal and spectral features of the audio signal. The project features a system for adapting pre-trained audio checkpoints to new datasets through modular initialization and configuration files. It includes utilities for weight extraction and inference model export, which remove training metadata and optimizer states to create li

DiffSinger is an AI vocal synthesizer and neural audio generator designed to produce high-fidelity singing and speech. It functions as a text-to-speech system and a diffusion-based singing voice synthesis tool that transforms text and pitch into audible audio. The system utilizes a shallow diffusion mechanism and iterative noise refinement to generate realistic vocal performances. It incorporates specialized sampling plugins and numerical solvers to accelerate inference and reduce the time required to generate synthetic voices. The project covers acoustic modeling, mel-spectrogram synthesis,

This project is a comprehensive suite for neural speech synthesis, featuring a deep learning text-to-speech engine, a neural speech synthesis trainer, and a voice cloning toolkit. It provides a system for synthesizing human-like speech from text using neural network models and high-fidelity vocoders. The suite includes a speech model conversion utility to transform deep learning models between different formats for deployment across various hardware runtimes. It also provides a self-contained HTTP server to expose pre-trained text-to-speech models as a remote audio API. Capabilities include

Jukebox is a generative audio model and AI music synthesis tool designed to create high-fidelity music samples and singing voices. It functions as a deep learning system that synthesizes raw audio conditioned on genre and artist metadata, utilizing a neural audio codec to convert raw audio into discrete codes for generative modeling and reconstruction. The system enables musical style steering and AI music composition by conditioning generation on specific artists, genres, and lyrics. It supports audio priming, allowing existing wave files to guide the creation of new musical sequences, and p

This project is a deep learning research toolkit and generative model library providing implementations of Variational Autoencoders using the PyTorch framework. It serves as a framework for training and evaluating autoencoder architectures to learn latent representations for data reconstruction and the generation of synthetic data samples. The toolkit focuses on unsupervised feature learning and generative model training, featuring a system for mapping external configuration files to model hyperparameters to ensure reproducible experimental runs. It includes mechanisms for tracking training p

TensorFlowTTS is a neural speech synthesis framework used to convert text into high-fidelity audio waveforms. It provides a toolkit for training and fine-tuning sequence-to-sequence or generative adversarial network architectures to produce natural sounding speech. The system includes neural vocoder implementations that transform intermediate acoustic representations into final audio waveforms. It also features playback speed control to adjust the rate of synthesized speech output. The framework covers the end-to-end pipeline for speech synthesis, including audio data preprocessing to create

ToonCrafter is a model that combines latent diffusion, reference-based colorization, and sketch-guided control for cartoon animation and interpolation. It functions as a cartoon video interpolation model, a reference-based colorization model, and a sketch-guided animation tool, all built on a latent diffusion animation framework. The project distinguishes itself by integrating three core capabilities into a single pipeline: generating smooth intermediate frames between two cartoon images using diffusion-based priors, transferring color and style from a reference image onto black-and-white ske

apprtc is a WebRTC video chat application and signaling server designed to establish peer-to-peer audio and video communication between browsers. It provides a coordination layer using a websocket-based signaling server to exchange session descriptions and network candidates. The project is delivered as a dockerized communication app, allowing for a containerized deployment of the calling service for local development or cloud hosting. It includes a network gateway that integrates STUN and TURN servers to facilitate media flow through firewalls and NATs. The implementation covers peer discov

Jitsi is an open-source communication suite providing a self-hosted platform for real-time video conferencing, audio calls, and instant messaging. It functions as a comprehensive toolkit for streaming voice and video across different devices and platforms, allowing for the deployment of private team collaboration environments. The project is distinguished by its multi-protocol communication bridging, which translates and exchanges data between diverse network standards including SIP, XMPP, and IRC. This allows users on different protocols to interact within a unified communication interface.

This project is a WebRTC plugin for Flutter that provides a real-time communication framework for implementing peer-to-peer voice, video, and data streaming. It functions as a cross-platform media streamer and hardware media controller, allowing applications to manage device cameras and microphones across mobile, desktop, and web platforms. The framework includes a dedicated peer-to-peer data channel interface for exchanging arbitrary data packets with low latency. It ensures privacy and data integrity through end-to-end encryption for all audio, video, and data transmissions. Broad capabili

SimpleWebRTC is a communication framework and real-time media streaming library designed to establish peer-to-peer audio, video, and data streams between web clients. It provides a conference room manager to organize multiple participants into virtual rooms for group interaction. The framework includes a dedicated system for peer-to-peer file transfers and low-latency data messaging. It also features a network traversal configuration tool for managing the servers required to maintain connectivity across firewalls and restrictive network environments. The project covers broad capability areas

This project is an RTMP media streaming SDK and a real-time communication framework designed for pushing and playing audio and video streams. It provides tools for interactive broadcasting, low-latency voice and video calls, and a cross-platform media player compatible with Windows, iOS, and Android. The toolkit enables interactive live broadcasting with support for multi-host interactions and the ability to push streams to distribution servers via CDN. It includes a cloud recording manager for capturing live sessions and saving them as files to cloud storage, along with a system for composit

This project is a music information retrieval library and research dataset designed for audio feature extraction and music genre classification. It provides a framework for training and evaluating machine learning models that categorize audio tracks into hierarchical genre structures, supported by a collection of open-licensed MP3 tracks and pre-computed features. The project includes a music metadata API client to fetch structured track, album, and artist information from external data sources. It utilizes these external integrations to map parent-child relationships between genres and organ

flutter-webrtc is a real-time communication SDK and plugin for the Flutter framework. It provides a set of tools for establishing peer-to-peer media connections and low-latency data exchange across mobile, desktop, and web environments. The project enables the creation of applications with live audio and video calling, real-time media streaming, and peer-to-peer data channels for sending encrypted arbitrary data packets without a central server. It supports secure media communication through end-to-end encryption for audio, video, and data streams. The SDK covers broad capabilities including

Bark Voice Cloning is a text-to-speech synthesis engine designed to generate natural-sounding audio and replicate specific vocal characteristics. The system utilizes a transformer-based autoregressive model to convert written text into high-fidelity speech, supporting multilingual output and expressive delivery. The project distinguishes itself through zero-shot voice cloning, which extracts speaker identity embeddings from short audio samples to condition the generative model without requiring extensive fine-tuning. It also provides specialized workflows for voice identity conversion, allowi

This project is an AI voice training framework and singing voice conversion tool. It uses VITS and SoftVC technologies to transform the timbre of singing and spoken audio recordings, allowing a user to change the vocal characteristics of a recording to match a specific target speaker. The system provides a web-based voice converter interface for managing model checkpoints and performing timbre transformation and pitch shifting. It supports exporting trained models to the ONNX format for use in external interfaces and lightweight runtimes. The framework covers the full production pipeline, in

This project is a browser-based video conferencing platform that facilitates direct, peer-to-peer communication between participants. By utilizing browser-native media capture and direct network connections, it enables real-time audio, video, and screen sharing without relying on central media relay servers. The platform distinguishes itself through its focus on ephemeral, private communication. It generates unique, short-lived session identifiers for virtual rooms that exist only in memory, ensuring that no persistent user accounts or long-term storage are required. To maintain connection st

AudioKit is an audio framework for iOS, macOS, and tvOS that provides tools for digital audio synthesis, signal processing, and audio analysis. It functions as a synthesis engine for generating audio waveforms and textures, a processing library for modifying tonal characteristics, and a toolkit for extracting frequency and amplitude data from sonic signals. The framework utilizes a modular node architecture and graph-based signal routing to connect audio generators, processors, and outputs. It wraps low-level audio primitives in high-level classes to facilitate sound generation and modificati

HuLa is an open-source communication suite and cross-platform chat client designed for real-time instant messaging, voice, and video calls. It functions as an LLM-integrated messaging platform and an AI-powered social manager that synchronizes user data and settings across different operating systems. The platform integrates large language models to provide automated AI assistants, real-time message translation, and chat log summarization. It further utilizes artificial intelligence for user sentiment analysis to assist with social matching and risk management. The system includes a high-per