Projects sharing features with Score Entropy Discrete Diffusion

](https://huggingface.co/Dream-org/DreamOn-v0-7B)

This thesis aimstoinvestigate thepotential of discrete diffusion models in the context ofnaturallanguagegeneration.

This repository serves as a comprehensive research platform and toolkit for advancing machine learning, quantum computing, and large-scale scientific data analysis. It provides foundational frameworks for developing complex algorithmic systems, offering the necessary infrastructure for distributed training, computational graph execution, and high-performance model development. The project distinguishes itself by integrating specialized research domains with robust, privacy-preserving methodologies. It supports diverse scientific discovery through tools for quantum simulation, physics-informed

🚀 dParallel: Learnable Parallel Decoding for dLLMs

This is the official implementation of Pix2Seq in Tensorflow 2 with efficient TPUs/GPUs support. The original Pix2Seq code aims to be a general framework that turns RGB pixels into semantically meaningful sequences. We now extend it to be a generic codebase, with task-centric organization that…

Current Diffusion Language Models (DLMs) have been studied at a smaller scale compared to their autoregressive (AR) counterparts and lack fair comparison on language modeling benchmarks. Additionally, training diffusion models from scratch at scale remains challenging. We propose adapting…

This repository contains code for training and evaluating the models in the paper Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning.

](https://huggingface.co/Dream-org/Dream-v0-Base-7B)

This repository contains the official implementation of paper A Reparameterized Discrete Diffusion Model for Text Generation.

please use requirementsgpu.txt if your accelerator is GPUs, use requirementstpu.txt when using Google Cloud TPUs.

This repository contains code for training and evaluating the models in the paper Likelihood-Based Diffusion Language Models.

SparseD is a novel sparse attention method for diffusion language models (DLMs), delivering near lossless acceleration in performance. It uses full attention and computes sparse patterns during early denoising steps, then reuses these patterns in later steps to restrict computation and improve…

By Justin Deschenaux and Caglar Gulcehre.

We introduce SDAR (Synergy of Diffusion and AutoRegression), a large-scale diffusion language model that unites the complementary strengths of autoregressive and discrete diffusion modeling. By merging the training efficiency of autoregressive methods with the highly parallel decoding ability of…

Diffusion Language Models are Super Data Learners

MegaDLMs

OpenMoE 2: Sparse Diffusion Language Models

Training Optimal Large Diffusion Language Models Jinjie Ni†, Qian Liu, Chao Du, Longxu Dou, Hang Yan, Zili Wang, Tianyu Pang, Michael Qizhe Shieh

This repository contains the official implementation for the paper "C²DLM: Causal Concept-Guided Diffusion Large Language Models".

By Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, Volodymyr Kuleshov

Official implementation of DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models.

1.we propose a new solution for integrating Pre-trained LM into Diffusion Model, to perform discrete diffusion for text-to-text generation. It requires only low-cost fine-tuning and can performs better than vanilla fine-tuning;

Official PyTorch implementation of the paper "Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles" (Slow Fast Sampling).

By Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, Thomas Hofmann