Open-source alternatives to Diffullama

Are Autoregressive Large Language Models Implicit Teachers for Diffusion Large Language Models? A comprehensive framework for transferring alignment knowledge from AR-LLMs to Diffusion Models

The repository contains the code for the DRAKES method presented in the paper: Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein design (ICLR 2025). DRAKES is a fine-tuning method for reward optimization or alignment in discrete diffusion models,…

This thesis aimstoinvestigate thepotential of discrete diffusion models in the context ofnaturallanguagegeneration.

🚀 dParallel: Learnable Parallel Decoding for dLLMs

d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning A two-stage approach combining masked SFT with diffu -GRPO—a novel policy gradient method based on GRPO that features efficient log probability estimation designed for masked dLLMs—to scale reasoning capabilities…

Dream-Coder 7B is a diffusion LLM for code trained exclusively on open-source data across its development stages—adaptation, supervised fine-tuning, and reinforcement learning. It achieves an impressive 21.4% pass@1 on LiveCodeBench (2410-2505), outperforming other open-source diffusion LLMs by…

](https://huggingface.co/Dream-org/DreamOn-v0-7B)

By Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, Thomas Hofmann

please use requirementsgpu.txt if your accelerator is GPUs, use requirementstpu.txt when using Google Cloud TPUs.

Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model

Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models Most comprehensive framework for dLLM's and multimodal dLLM's post-training

This is the official implementation of the paper d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation (ICML'26), where we introduce a novel recipe for building an ultra-fast diffusion language model named d3LLM (pseuDo-Distilled Diffusion LLM) 🚀.

NeurIPS 2024 Code for the paper "Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models"

This repository contains code for training and evaluating the models in the paper Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning.

SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models

This repository contains the official implementation of paper A Reparameterized Discrete Diffusion Model for Text Generation.

Official implementation of DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models.

This repository contains code for training and evaluating the models in the paper Likelihood-Based Diffusion Language Models.

SparseD is a novel sparse attention method for diffusion language models (DLMs), delivering near lossless acceleration in performance. It uses full attention and computes sparse patterns during early denoising steps, then reuses these patterns in later steps to restrict computation and improve…

By Justin Deschenaux and Caglar Gulcehre.

We introduce SDAR (Synergy of Diffusion and AutoRegression), a large-scale diffusion language model that unites the complementary strengths of autoregressive and discrete diffusion modeling. By merging the training efficiency of autoregressive methods with the highly parallel decoding ability of…

Diffusion Language Models are Super Data Learners

Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding