Open-source alternatives to Dream Coder

](https://huggingface.co/Dream-org/DreamOn-v0-7B)

By Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, Thomas Hofmann

This repository includes implementations to reproduce the R1 pipeline for code generation:

please use requirementsgpu.txt if your accelerator is GPUs, use requirementstpu.txt when using Google Cloud TPUs.

Current Diffusion Language Models (DLMs) have been studied at a smaller scale compared to their autoregressive (AR) counterparts and lack fair comparison on language modeling benchmarks. Additionally, training diffusion models from scratch at scale remains challenging. We propose adapting…

This repository contains code for training and evaluating the models in the paper Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning.

](https://huggingface.co/Dream-org/Dream-v0-Base-7B)

This repository contains the official implementation of paper A Reparameterized Discrete Diffusion Model for Text Generation.

🚀 dParallel: Learnable Parallel Decoding for dLLMs

Official implementation of DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models.

This repository contains code for training and evaluating the models in the paper Likelihood-Based Diffusion Language Models.

SparseD is a novel sparse attention method for diffusion language models (DLMs), delivering near lossless acceleration in performance. It uses full attention and computes sparse patterns during early denoising steps, then reuses these patterns in later steps to restrict computation and improve…

CodeV-R1 is an innovative open-source Large Language Model (LLM) specifically designed for the generation of high-quality Verilog code, addressing the challenges faced by existing models in this domain.

By Justin Deschenaux and Caglar Gulcehre.

We introduce SDAR (Synergy of Diffusion and AutoRegression), a large-scale diffusion language model that unites the complementary strengths of autoregressive and discrete diffusion modeling. By merging the training efficiency of autoregressive methods with the highly parallel decoding ability of…

Diffusion Language Models are Super Data Learners

MegaDLMs

OpenMoE 2: Sparse Diffusion Language Models

Training Optimal Large Diffusion Language Models Jinjie Ni†, Qian Liu, Chao Du, Longxu Dou, Hang Yan, Zili Wang, Tianyu Pang, Michael Qizhe Shieh

This repository contains the official implementation for the paper "C²DLM: Causal Concept-Guided Diffusion Large Language Models".

By Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, Volodymyr Kuleshov

By Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov

This thesis aimstoinvestigate thepotential of discrete diffusion models in the context ofnaturallanguagegeneration.