Projects sharing features with Slow Fast Sampling

DMax is a new dLLM paradigm achieving aggressive parallel decoding while preserving generation quality.

Efficiency: DPad-enhanced dLLMs achieve up to a 61.39× speedup over vanilla dLLM baselines. Accuracy: DPad-enhanced dLLMs achieve up to a +26.46% improvement over vanilla dLLM baselines. (Evaluation conducted on NVIDIA A100-PCIe-80GB GPUs).

Official repository for the paper: Accelerating Diffusion LLMs via Adaptive Parallel Decoding

Dream-Coder 7B is a diffusion LLM for code trained exclusively on open-source data across its development stages—adaptation, supervised fine-tuning, and reinforcement learning. It achieves an impressive 21.4% pass@1 on LiveCodeBench (2410-2505), outperforming other open-source diffusion LLMs by…

](https://huggingface.co/Dream-org/DreamOn-v0-7B)

By Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, Thomas Hofmann

Constrained Decoding of Diffusion LLMs with Context-Free Grammars

dVoting: Fast Voting for dLLMs

Current Diffusion Language Models (DLMs) have been studied at a smaller scale compared to their autoregressive (AR) counterparts and lack fair comparison on language modeling benchmarks. Additionally, training diffusion models from scratch at scale remains challenging. We propose adapting…

please use requirementsgpu.txt if your accelerator is GPUs, use requirementstpu.txt when using Google Cloud TPUs.

This repository contains code for training and evaluating the models in the paper Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning.

](https://huggingface.co/Dream-org/Dream-v0-Base-7B)

This repository contains the official implementation of paper A Reparameterized Discrete Diffusion Model for Text Generation.

The pipeline of dKV-Cache

Official implementation of DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models.

This repository contains code for training and evaluating the models in the paper Likelihood-Based Diffusion Language Models.

https://github.com/user-attachments/assets/09c67a58-b425-463d-a998-c1a6049bc171

dInfer is an efficient and extensible inference framework for dLLMs. As illustrated in the following architecture, it modularizes inference into four components: model, diffusion iteration manager, decoder and KV-cache manager. It provides well-designed APIs for flexible algorithms combinations…

SparseD is a novel sparse attention method for diffusion language models (DLMs), delivering near lossless acceleration in performance. It uses full attention and computes sparse patterns during early denoising steps, then reuses these patterns in later steps to restrict computation and improve…

By Justin Deschenaux and Caglar Gulcehre.

We introduce SDAR (Synergy of Diffusion and AutoRegression), a large-scale diffusion language model that unites the complementary strengths of autoregressive and discrete diffusion modeling. By merging the training efficiency of autoregressive methods with the highly parallel decoding ability of…

MaskKV: Fine-Grained Cache Eviction for Efficient dLLM Inference

🚀 dParallel: Learnable Parallel Decoding for dLLMs