Awesome GitHub RepositoriesAutoregressive Models

Bark is a generative audio engine and machine learning inference library designed to convert written text into high-fidelity speech and sound effects. It functions as a text-to-audio transformer, utilizing multi-stage neural network architectures to map semantic input tokens into detailed audio codebooks for synthesis. The system distinguishes itself through a hierarchical transformer stacking approach that separates semantic understanding from acoustic realization. By employing autoregressive token prediction and vector quantized codebook mapping, the engine bridges linguistic and sonic doma

This project is a comprehensive Chinese translation of a technical deep learning textbook, providing an educational resource on the theory and implementation of neural networks. It functions as a collaborative technical translation project designed to make complex academic AI literature accessible to non-English speakers. The project utilizes a community-driven translation model that integrates external suggestions and pull requests to refine linguistic accuracy and reduce bias. It employs standardized terminology mapping to ensure a uniform vocabulary throughout the translated content. To i

minGPT is a minimal implementation of the Transformer architecture designed for training and experimenting with language models. It functions as a neural network training framework and a text generation engine, providing the necessary tools to manage data loading, backpropagation, and parameter updates for custom deep learning models. The project is structured as an educational resource for understanding how transformer architectures function by building and training models from scratch. It utilizes a modular block architecture and transformer-based self-attention to process sequences, allowi

CosyVoice is a speech synthesis framework that utilizes large language models to generate expressive, multilingual audio. The system functions as an audio generation engine capable of producing natural-sounding speech across multiple languages while preserving regional dialects and specific emotional tones. The platform distinguishes itself through its zero-shot voice cloning capabilities, which allow for the creation of synthetic voice profiles from short audio samples without requiring additional model training. It provides fine-grained control over vocal attributes, enabling users to adjus

Qwen3-VL is a multimodal vision-language model designed to process and reason across images, videos, and text. It functions as a computer vision framework capable of identifying objects, extracting structured data from documents, and interpreting spatial elements within visual media. The system operates as an automated user interface interaction agent, interpreting screen data to navigate software and mobile applications. By utilizing a unified transformer architecture, it performs complex visual reasoning to execute user-defined tasks without manual input. Beyond interface navigation, the m

Janus is a multimodal large language model and unified framework that integrates visual understanding and image generation within a single neural network. It functions as both a visual understanding model for analyzing images and a text-to-image generator. The system uses a unified transformer backbone and a multimodal latent space to bridge the gap between text and visual data. This architecture employs decoupled visual encoding and cross-modal tokenization to separate the paths for discriminative understanding and generative tasks, representing images as grids of discrete codes. The projec

FramePack is a neural video synthesis engine and generation framework designed to produce long, temporally consistent video sequences. It functions as a diffusion model optimizer, providing a suite of techniques to manage the computational demands of high-parameter video models while maintaining visual stability during extended generation tasks. The system distinguishes itself through a hierarchical approach to frame prediction, which plans distant anchor frames before filling in intermediate content to prevent cumulative temporal drift. By utilizing constant-length context compression and to

ChatGLM2-6B is a bilingual chat large language model designed for natural conversation and text generation in both English and Chinese. It functions as a fine-tunable language model that supports updating weights via specialized scripts to adapt to specific datasets and tasks. The project serves as a quantized inference engine and multi-GPU model orchestrator, enabling the execution of large models on consumer-grade hardware. It is capable of processing long context sequences up to 32K tokens to maintain understanding across extended documents. The system covers capabilities for multilingual

Spark-TTS is a deep learning text-to-speech synthesis engine designed to convert written text into high-fidelity audio. It utilizes a transformer-based architecture and autoregressive sequence modeling to generate coherent speech, transforming linguistic input into natural-sounding waveforms through neural speech codec synthesis. The platform distinguishes itself through zero-shot voice cloning, which allows users to mimic a target speaker’s unique vocal identity using only a short reference audio sample without requiring additional model training. It also features cross-lingual phonetic mapp

CodeGeeX is an open-source code model and multilingual large language model designed to generate, translate, and complete source code across multiple programming languages. It functions as an AI coding assistant and a cross-lingual code translator that produces executable code and technical documentation. The project enables natural language programming by turning plain English descriptions into functional programs. It also provides the ability to convert source code from one programming language to another while preserving the original logic and functionality. The system covers a range of c

VAR is a visual autoregressive model and image generation framework that applies large language model scaling laws to visual data. It functions as an image generator that uses a coarse-to-fine next-scale prediction approach rather than traditional raster-scan tokenization. The system utilizes scale-based tokenization to represent images as a hierarchy of discrete tokens. It generates high-resolution content by iteratively predicting the next resolution level, refining coarse predictions into fine-grained details. The project covers a broad range of capabilities including autoregressive image

Jukebox is a generative audio model and AI music synthesis tool designed to create high-fidelity music samples and singing voices. It functions as a deep learning system that synthesizes raw audio conditioned on genre and artist metadata, utilizing a neural audio codec to convert raw audio into discrete codes for generative modeling and reconstruction. The system enables musical style steering and AI music composition by conditioning generation on specific artists, genres, and lyrics. It supports audio priming, allowing existing wave files to guide the creation of new musical sequences, and p

GraphCast is a machine learning model that uses graph neural networks to produce global weather forecasts up to ten days ahead at high spatial resolution. The system represents the Earth's surface as an icosahedral mesh, enabling message passing between mesh nodes to capture atmospheric dynamics, and combines this with a learned multiscale processor that operates across coarse-to-fine mesh resolutions. The model is trained on historical ERA5 reanalysis data through a supervised learning objective, and its autoregressive rollout loop feeds predictions back as input to generate multi-step forec

Taming Transformers is a generative system for high-resolution image synthesis that combines a vector-quantized GAN image encoder with an autoregressive transformer. It utilizes a discrete latent space to represent images as codebook tokens, enabling the production of high-fidelity visuals through a hybrid architecture. The project provides specialized capabilities for layout-based scene synthesis, allowing for the creation of complex images by placing objects according to defined bounding box coordinates. It also includes tools for image inpainting to fill missing sections of an image by ana

Ce projet est un framework de traitement du langage naturel axé sur un pré-entraîneur autorégressif généralisé conçu pour la représentation de langage non supervisée. Il implémente un modèle de langage qui combine un entraînement basé sur la permutation avec une architecture Transformer-XL pour fonctionner comme un processeur de texte à long contexte. Le système se distingue par sa capacité à gérer des séquences de texte dépassant les limites de longueur standard grâce à l'utilisation de la récurrence au niveau du segment et du codage positionnel relatif. Il met à l'échelle le pré-entraînement haute performance sur plusieurs clusters de GPU et TPU en utilisant des implémentations d'entraînement distribué. La base de code couvre l'ensemble du workflow de machine learning, incluant le nettoyage de texte et la tokenisation par sous-mots pour le prétraitement des données, ainsi que le fine-tuning spécifique à une tâche pour la réponse aux questions, la compréhension de lecture et la classification de texte. Il inclut des utilitaires pour l'optimisation des paramètres, la planification du taux d'apprentissage et l'évaluation des probabilités de réponse via des métriques de précision-rappel. Le projet fournit des configurations pour gérer les hyperparamètres du modèle et l'entraînement accéléré par le matériel sur plusieurs hôtes.

x-transformers est une bibliothèque PyTorch et un toolkit de recherche pour construire des architectures transformer. Il fournit un framework modulaire pour implémenter la recherche transformer expérimentale, incluant une suite de mécanismes d'attention avancés, des outils de modélisation de séquences longues et un framework pour les vision transformers. Le projet se distingue par son accent sur les composants haute performance et économes en mémoire, tels que Flash Attention avec des noyaux tuilés et l'attention multi-requêtes. Il implémente également des méthodes spécialisées pour étendre les fenêtres de contexte, incluant la récurrence de séquence et les plongements positionnels rotatifs. La bibliothèque couvre un large éventail de capacités architecturales, incluant divers schémas de normalisation pour stabiliser l'entraînement, des réseaux feedforward à portes et des topologies de couches personnalisées comme les réseaux Macaron. Elle prend en charge les constructions d'encodeur et de décodeur, fournissant des outils pour la génération de séquence autorégressive et les tâches vision-langage comme la légende d'image.

PyTorch Forecasting is a deep learning framework designed for building and training neural network architectures specifically for time series forecasting. It serves as a comprehensive toolkit for implementing autoregressive models, multi-horizon forecasting, and probabilistic prediction intervals using PyTorch tensors. The library distinguishes itself through a probabilistic forecasting toolkit that generates prediction intervals and quantile forecasts using both parametric and non-parametric distributions. It further provides a neural network model optimizer for automated hyperparameter tuni

OmniGen2 est un modèle de génération d'images unifié et un grand modèle de langage multimodal conçu pour gérer la génération texte-vers-image, les tâches image-vers-image et l'édition d'images au sein d'un seul framework. Il fonctionne comme un moteur visuel de modèle de langage causal capable de générer et d'éditer des images basées sur des entrées combinées de texte et d'images. Le système propose une composition visuelle en contexte et une génération pilotée par le sujet, lui permettant d'extraire des sujets d'images de référence et de les placer dans de nouvelles scènes. Il prend également en charge l'édition d'images basée sur des instructions, où des objets ou styles spécifiques sont modifiés via des commandes en langage naturel tout en préservant le reste de l'image. Les capacités du modèle s'étendent à l'analyse et au raisonnement de contenu visuel, permettant la reconnaissance d'objets à travers des entrées combinées de texte et de vision. Pour améliorer la qualité de sortie, il emploie un processus de raffinement visuel itératif avec un mécanisme d'auto-correction. Les performances sont gérées via l'optimisation de l'utilisation de la VRAM par le déchargement dynamique des poids et l'accélération de la vitesse d'inférence utilisant des techniques de mise en cache.

FastVideo is a comprehensive system for accelerated video generation, serving as a video generation inference engine, a video diffusion training framework, and a modular pipeline orchestrator. It provides a distributed transformer optimizer and a distillation toolkit designed to reduce denoising steps and model complexity to increase frame rates. The project distinguishes itself through specialized acceleration techniques, including joint distillation and sparse attention training. It implements low-step video generation and weight quantization to FP8 or FP4 precision to increase throughput a