Projects sharing features with Stable Audio Tools

MedSAM is a deep learning framework designed for automating the segmentation of anatomical structures in 2D and 3D medical imagery. It provides specialized tools for fine-tuning pretrained segmentation weights on custom medical datasets and evaluating the accuracy of those predictions against ground truth labels. The project focuses on adapting the Segment Anything Model architecture for medical use, enabling the isolation of specific anatomical structures through prompt-guided methods such as bounding boxes and point prompts. The system covers a full medical AI workflow, including data engi

This project is a Dreambooth implementation designed to personalize Stable Diffusion models. It serves as an AI image personalization tool and model tuner that enables the creation of unique subject identifiers to generate consistent, personalized images. The system focuses on subject-driven image synthesis by fine-tuning pre-trained diffusion models on small, custom datasets. This allows the model to recognize specific people, objects, or artistic styles and place those learned subjects into diverse contexts via text-to-image conditioning. The implementation includes a diffusion model optim

DiT is a latent diffusion model and transformer-based generative AI framework implemented in PyTorch. It functions as a class-conditional image generator that replaces traditional convolutional backbones with a transformer architecture to synthesize high-fidelity images. The project utilizes patch-based latent processing and latent space compression to operate on low-dimensional image representations. It incorporates class-conditional guidance and adjustable guidance scales to control the visual content of generated images during the sampling process. The framework covers distributed model t

AudioLDM is a latent diffusion framework for generating high-fidelity audio, music, and sound effects. It functions as a text-to-audio generator that converts natural language descriptions into synthetic audio signals with control over pitch and environment. The system provides specialized tools for audio-to-audio synthesis and generative repair. This includes the ability to perform audio style transfer and replicate specific acoustic events based on existing files. The project covers a broad range of audio transformation tasks, including audio super-resolution for increasing signal fidelity

This is a framework for training and sampling diffusion models to generate high-fidelity images, video, and 4D assets. It provides a modular environment for managing generative AI training pipelines, including the handling of datasets, noise sampling, and loss weighting to stabilize the creation of synthetic content. The project features a modular model configuration system that uses YAML-based assembly to define network submodules and conditioners. It also includes a dedicated toolset for AI image watermarking, allowing for the embedding and detection of invisible markers to verify the origi

HunyuanDiT is a bilingual text-to-image generative model and diffusion transformer image generator. It uses a latent diffusion system to synthesize high-resolution images from text prompts, with a specific focus on understanding and generating content from both Chinese and English language descriptions. The project features a multi-resolution transformer architecture and a bilingual embedding space to map different scripts into a shared semantic area. It supports iterative multi-turn image refinement, which translates conversational dialogue into updated prompts to progressively modify visual

This project is an educational course and collection of training materials focused on generative diffusion models. It provides a curriculum and practical guides for training, fine-tuning, and deploying models capable of synthesizing images, audio, and video. The material covers specific implementation strategies including noise-based synthesis, iterative refinement, and latent space compression. It provides instruction on guiding generative outputs through conditional synthesis and prompt adherence optimization, as well as techniques for image inpainting and text-based editing. The project i

This is a PyTorch implementation of a text-to-image model designed for synthesizing high-fidelity images from natural language descriptions. It utilizes a diffusion image generator to transform latent embeddings into visual data through an iterative denoising process. The system employs a two-stage latent mapping process, using a CLIP-based latent prior to map text embeddings to image embeddings before decoding them into pixels. It features a cascading diffusion decoder that produces high-resolution imagery by passing low-resolution outputs through a sequence of models at increasing scales.

This project is a TensorFlow implementation of a neural network for raw audio waveform generation. It functions as a conditioned speech synthesis model that produces synthetic audio samples using a dilated convolutional neural network architecture. The system supports custom voice modeling by incorporating global conditioning and categorical identifiers during training and generation. This allows the model to mimic specific speakers or distinct audio characteristics for neural text-to-speech applications. The framework covers deep learning audio synthesis, including audio dataset processing,

ComfyUI-LTXVideo is a generative framework and ComfyUI custom node extension for synthesizing high-fidelity video. It utilizes a latent diffusion and transformer-based system to create cinematic clips from text, image, and audio inputs, providing a modular interface for precise control over subject behavior and temporal consistency. The tool distinguishes itself with production-grade capabilities, including the generation of High Dynamic Range video in linear formats such as ARRI LogC3. It supports multimodal synchronization for audio-driven animation and lip-syncing, and allows for the creat

Tensorpack is a high-level TensorFlow neural network framework and research library designed for building and training deep learning models. It provides a collection of reproducible neural network architectures for computer vision, generative tasks, reinforcement learning, and natural language processing. The project distinguishes itself through a specialized deep learning data pipeline that uses pure Python for parallel data loading and streaming. It includes a multi-GPU training orchestrator for distributing workloads via data-parallel strategies and a dedicated interpretability toolkit for

DeepKE is a knowledge extraction toolkit and framework designed to transform unstructured text into structured knowledge graphs. It provides a pipeline for identifying and classifying named entities, semantic relations, and events, converting raw datasets into structured triples. The project utilizes large language models as tool callers through a standardized context protocol to drive automated data extraction processes. It supports schema-driven extraction across multiple domains and bilingual text, employing joint entity and relation extraction to identify components in a single structured

This application is a platform for AI voice synthesis and neural voice cloning. It provides a comprehensive toolkit for converting text into natural-sounding human speech by applying custom-trained neural network models to specific audio samples. The system facilitates the entire lifecycle of voice model development, including the preparation of raw audiobooks and video transcriptions into structured training datasets. It supports the training of these models on local or remote hardware, utilizing multi-GPU distributed processing to handle large-scale data and accelerate model convergence. B

mmpretrain is a modular PyTorch computer vision framework designed for developing, training, and benchmarking deep learning architectures. It serves as a comprehensive toolkit for vision tasks, providing a specialized platform for multimodal machine learning and self-supervised learning. The project features a computer vision model zoo containing architectural definitions and pre-trained weights for backbones such as ViT, ConvNeXt, and Swin Transformer. It distinguishes itself through a dedicated self-supervised learning toolkit that implements algorithms like MAE and DINO to train models wit

Caffe is a high-performance deep learning framework and convolutional neural network library designed for training and deploying neural networks. It functions as a GPU-accelerated machine learning engine with a core implemented in C++ to enable high-throughput tensor operations. The project utilizes a declarative configuration system where model architectures and hyperparameters are defined in external text files, separating the network design from the execution code. It includes a model serialization system to export trained weights and topologies into binary files for efficient deployment a

Neuraltalk2 is a deep learning vision system designed for automatic image captioning. Built with PyTorch, it utilizes a hybrid architecture that combines a convolutional neural network encoder with a recurrent neural network decoder to generate textual descriptions from visual input. The project features a GPU-accelerated training pipeline capable of distributing workloads across multiple graphics processing units through multi-process distribution. It supports the generation of descriptions for both static image files and real-time video streams. The framework includes capabilities for enco

This project is a deep learning library built for single-image super-resolution and visual enhancement. It provides a framework for training and deploying neural network architectures designed to reconstruct high-resolution images from low-resolution sources, effectively recovering fine details and removing artifacts caused by downscaling or compression. The library distinguishes itself through the implementation of generative adversarial networks and residual block architectures, which work together to improve the realism and clarity of upscaled outputs. It supports training through both pix

This project is a PyTorch object detection framework that implements the Faster R-CNN architecture. It serves as a vision model for predicting precise bounding boxes around multiple objects within images and live video feeds. The system is optimized for multi-GPU training to reduce the time required for model convergence. It utilizes a GPU-accelerated design to handle the training and inference of complex detection networks. The framework covers the full object detection lifecycle, including custom network training and inference for static images and real-time video streams. It includes capa

Muzic is a deep learning platform and framework for AI-driven music analysis, composition, and synthesis. It functions as a music generation framework and analysis tool, utilizing large language models and autonomous agents to orchestrate the creation and interpretation of symbolic and audio music. The project is distinguished by its cross-modal capabilities, mapping natural language and symbolic music into a shared joint embedding space for zero-shot classification and information retrieval. It employs a variety of specialized architectures, including diffusion frameworks for audio synthesis

Jukebox is a generative audio model and AI music synthesis tool designed to create high-fidelity music samples and singing voices. It functions as a deep learning system that synthesizes raw audio conditioned on genre and artist metadata, utilizing a neural audio codec to convert raw audio into discrete codes for generative modeling and reconstruction. The system enables musical style steering and AI music composition by conditioning generation on specific artists, genres, and lyrics. It supports audio priming, allowing existing wave files to guide the creation of new musical sequences, and p

Video-Pre-Training is a machine learning framework designed for training autonomous agents to perform complex tasks by observing and mimicking human behavior from video recordings. It provides a comprehensive toolkit for imitation learning and reinforcement learning research, enabling the development of agents that can replicate human actions within simulated digital environments. The framework distinguishes itself through its ability to process large-scale, unlabeled video datasets to bootstrap agent capabilities. It utilizes inverse dynamics modeling to infer control inputs from frame trans

Amphion is an audio generation toolkit designed for the research and development of models that synthesize speech, music, and environmental sound effects. It provides a standardized framework for reproducible audio synthesis, incorporating a text-to-speech engine and a voice conversion framework. The project specializes in transforming audio identities, allowing for the modification of speaker accents and voice identities while preserving original rhythm and style. It also includes capabilities for singing voice synthesis and the generation of environmental soundscapes from text descriptions

This project is an AI singing voice conversion system and vocal processor used for training generative voice models and converting vocal recordings or live input into a target voice. It functions as a VITS model trainer and a real-time voice changer that transforms vocal timbre and pitch to change the identity of a singer. The system provides a graphical management dashboard for controlling training hyperparameters and voice conversion presets. It supports low-latency audio streaming for live microphone input and employs pitch estimation to ensure precise matching between source and target vo

This project is a deep learning framework designed for end-to-end speech-to-text transcription. It utilizes the WaveNet neural network architecture to process spoken audio input and generate written text transcripts, leveraging connectionist temporal classification to map variable-length audio sequences to character-level outputs. The system distinguishes itself through a comprehensive training pipeline that supports distributed execution across multiple graphics processing units. It includes specialized utilities for audio data augmentation and the transformation of raw audio files into opti

xtuner is a comprehensive training engine for large language models, offering a toolkit for pre-training, supervised fine-tuning, and the optimization of vision-language multimodal models. It serves as a distributed training accelerator and a specialized framework for scaling Mixture-of-Experts models and aligning model behavior through reinforcement learning from human feedback. The project distinguishes itself through advanced memory and compute optimizations, such as sequence parallelism for ultra-long context windows and interleaved pipeline parallelism to reduce GPU idle time. It provide

This repository serves as the documentation source for the Hugging Face Hub, a collaborative platform designed for hosting, versioning, and discovering machine learning models, datasets, and interactive applications. It provides the foundational infrastructure for managing machine learning assets through Git-based repositories, which support large file storage, branching, and comprehensive commit history. The platform distinguishes itself by integrating metadata-driven discovery and structured management systems that allow users to attach licensing, task categories, and performance metrics to

StableCascade is a generative AI system and latent diffusion framework designed for text-to-image synthesis and image-to-image transformations. It utilizes a multi-stage cascade architecture that encodes and decodes images via a latent space to produce high-fidelity visual imagery. The system includes a cascade diffusion pipeline for controlling image structure through inpainting, outpainting, and super-resolution. It also provides a toolkit for image-to-image generation and the creation of image variations using embeddings. The framework supports model optimization through low-rank adaptati