What are the best Awesome Language Diffusion Models GitHub Repositories?

Research and implementations of diffusion models applied to text generation. Explore 65 awesome GitHub repositories matching part of an awesome list · Language Diffusion Models. Refine with filters or upvote what's useful. Top picks: ml-gsai/llada, zhziszz/dllm, hkunlp/dream, kuleshov-group/bd3lms, louaaron/score-entropy-discrete-diffusion, kuleshov-group/mdlm, hkunlp/diffullama, ml-gsai/smdm, jetastra/sdar, hzfinfdu/diffusion-bert.

Why is ml-gsai/llada a recommended Language Diffusion Models GitHub Repositories repository?

Large language diffusion models for text generation.

Why is zhziszz/dllm a recommended Language Diffusion Models GitHub Repositories repository?

Simple implementation of diffusion language modeling.

Why is hkunlp/dream a recommended Language Diffusion Models GitHub Repositories repository?

Diffusion-based language model implementation.

Why is kuleshov-group/bd3lms a recommended Language Diffusion Models GitHub Repositories repository?

Interpolating between autoregressive and diffusion language models.

Why is louaaron/score-entropy-discrete-diffusion a recommended Language Diffusion Models GitHub Repositories repository?

Discrete diffusion modeling via score-entropy estimation.

Why is kuleshov-group/mdlm a recommended Language Diffusion Models GitHub Repositories repository?

Simple and effective masked diffusion language models.

Why is hkunlp/diffullama a recommended Language Diffusion Models GitHub Repositories repository?

Adapting autoregressive models to diffusion language models.

Why is ml-gsai/smdm a recommended Language Diffusion Models GitHub Repositories repository?

Scaling masked diffusion models on text.

Why is jetastra/sdar a recommended Language Diffusion Models GitHub Repositories repository?

Synergistic diffusion-autoregression paradigm for sequence generation.

Why is hzfinfdu/diffusion-bert a recommended Language Diffusion Models GitHub Repositories repository?

Improving generative masked language models with diffusion.

65 个仓库

Awesome GitHub RepositoriesLanguage Diffusion Models

Research and implementations of diffusion models applied to text generation.

Explore 65 awesome GitHub repositories matching part of an awesome list · Language Diffusion Models. Refine with filters or upvote what's useful.

用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。

ml-gsai/llada
ML-GSAI/LLaDA
3,580在 GitHub 上查看
LLaDA is a masked diffusion language model and conditional text generator. It generates text by iteratively refining masked tokens through a diffusion process rather than predicting the next token in a sequence. The project functions as a vision-language diffusion model, converting visual inputs into text responses. It also serves as a preference optimization framework that uses log-likelihood estimation and evidence lower bounds to tune model responses. The system supports multi-round conversational AI and text sequence evaluation. It integrates vision-language embedding for cross-modal con
Large language diffusion models for text generation.
Python
在 GitHub 上查看3,580
zhziszz/dllm
ZHZisZZ/dllm
2,581在 GitHub 上查看
dLLM: Simple Diffusion Language Modeling
Simple implementation of diffusion language modeling.
Pythondiscrete-diffusion-modelsllmnlp
在 GitHub 上查看2,581
hkunlp/dream
HKUNLP/Dream
1,249在 GitHub 上查看
](https://huggingface.co/Dream-org/Dream-v0-Base-7B)
Diffusion-based language model implementation.
Python
在 GitHub 上查看1,249
kuleshov-group/bd3lms
kuleshov-group/bd3lms
1,013在 GitHub 上查看
By Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, Volodymyr Kuleshov
Interpolating between autoregressive and diffusion language models.
Python
在 GitHub 上查看1,013
louaaron/score-entropy-discrete-diffusion
louaaron/Score-Entropy-Discrete-Diffusion
732在 GitHub 上查看
This repo contains a PyTorch implementation for the paper Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution by Aaron Lou, Chenlin Meng and Stefano Ermon.
Discrete diffusion modeling via score-entropy estimation.
Python
在 GitHub 上查看732
kuleshov-group/mdlm
kuleshov-group/mdlm
698在 GitHub 上查看
By Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov
Simple and effective masked diffusion language models.
Python
在 GitHub 上查看698
hkunlp/diffullama
hkunlp/diffullama
395在 GitHub 上查看
Current Diffusion Language Models (DLMs) have been studied at a smaller scale compared to their autoregressive (AR) counterparts and lack fair comparison on language modeling benchmarks. Additionally, training diffusion models from scratch at scale remains challenging. We propose adapting…
Adapting autoregressive models to diffusion language models.
Python
在 GitHub 上查看395
ml-gsai/smdm
ml-gsai/smdm
383在 GitHub 上查看
Masked diffusion models (MDMs) have shown promise in language modeling, yet their scalability and effectiveness in core language tasks, such as text generation and language understanding, remain underexplored. This paper establishes the first scaling law for MDMs, demonstrating a scaling rate…
Scaling masked diffusion models on text.
Python
在 GitHub 上查看383
jetastra/sdar
JetAstra/SDAR
359在 GitHub 上查看
We introduce SDAR (Synergy of Diffusion and AutoRegression), a large-scale diffusion language model that unites the complementary strengths of autoregressive and discrete diffusion modeling. By merging the training efficiency of autoregressive methods with the highly parallel decoding ability of…
Synergistic diffusion-autoregression paradigm for sequence generation.
Python
在 GitHub 上查看359
hzfinfdu/diffusion-bert
Hzfinfdu/Diffusion-BERT
343在 GitHub 上查看
Official implementation of DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models.
Improving generative masked language models with diffusion.
Python
在 GitHub 上查看343
jinjieni/megadlms
JinjieNi/MegaDLMs
339在 GitHub 上查看
MegaDLMs
Large-scale diffusion language model training.
Python
在 GitHub 上查看339
s-sahoo/duo
s-sahoo/duo
234在 GitHub 上查看
By Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, Volodymyr Kuleshov
Exploring the diffusion duality in language modeling.
Python
在 GitHub 上查看234
jinjieni/dlms-are-super-data-learners
JinjieNi/dlms-are-super-data-learners
228在 GitHub 上查看
Diffusion Language Models are Super Data Learners
Exploring diffusion language models as efficient data learners.
Python
在 GitHub 上查看228
google-deepmind/md4
google-deepmind/md4
161在 GitHub 上查看
please use requirementsgpu.txt if your accelerator is GPUs, use requirementstpu.txt when using Google Cloud TPUs.
Simplified and generalized masked diffusion for discrete data.
Python
在 GitHub 上查看161
introspective-diffusion/i-dlm
Introspective-Diffusion/I-DLM
148在 GitHub 上查看
Introspective diffusion language models for improved generation.
Python
在 GitHub 上查看148
dvruette/gidd
dvruette/gidd
119在 GitHub 上查看
By Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, Thomas Hofmann
Generalized interpolating discrete diffusion.
Python
在 GitHub 上查看119
dreamlm/dreamon
DreamLM/DreamOn
117在 GitHub 上查看
](https://huggingface.co/Dream-org/DreamOn-v0-7B)
Diffusion language models for code infilling.
Python
在 GitHub 上查看117
s-sahoo/eso-lms
s-sahoo/Eso-LMs
119在 GitHub 上查看
Subham Sekhar Sahoo \ 1 , Zhihan Yang \ 2 , Yash Akhauri †1 , Johnna Liu †1 , Deepansha Singh †1 , Zhoujun Cheng †3 , Zhengzhong Liu 3 , Eric Xing 3 , John Thickstun 2 , Arash Vahdat 4
Esoteric language models based on diffusion.
Python
在 GitHub 上查看119
igul222/plaid
igul222/plaid
115在 GitHub 上查看
This repository contains code for training and evaluating the models in the paper Likelihood-Based Diffusion Language Models.
Likelihood-based diffusion language models.
Python
在 GitHub 上查看115
dreamlm/dream-coder
DreamLM/Dream-Coder
104在 GitHub 上查看
Dream-Coder 7B is a diffusion LLM for code trained exclusively on open-source data across its development stages—adaptation, supervised fine-tuning, and reinforcement learning. It achieves an impressive 21.4% pass@1 on LiveCodeBench (2410-2505), outperforming other open-source diffusion LLMs by…
Open diffusion language model for code generation.
Python
在 GitHub 上查看104

65 个仓库

Awesome GitHub RepositoriesLanguage Diffusion Models

Research and implementations of diffusion models applied to text generation.

Explore 65 awesome GitHub repositories matching part of an awesome list · Language Diffusion Models. Refine with filters or upvote what's useful.

用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。

ml-gsai/llada
ML-GSAI/LLaDA
3,580在 GitHub 上查看
LLaDA is a masked diffusion language model and conditional text generator. It generates text by iteratively refining masked tokens through a diffusion process rather than predicting the next token in a sequence. The project functions as a vision-language diffusion model, converting visual inputs into text responses. It also serves as a preference optimization framework that uses log-likelihood estimation and evidence lower bounds to tune model responses. The system supports multi-round conversational AI and text sequence evaluation. It integrates vision-language embedding for cross-modal con
Large language diffusion models for text generation.
Python
在 GitHub 上查看3,580
zhziszz/dllm
ZHZisZZ/dllm
2,581在 GitHub 上查看
dLLM: Simple Diffusion Language Modeling
Simple implementation of diffusion language modeling.
Pythondiscrete-diffusion-modelsllmnlp
在 GitHub 上查看2,581
hkunlp/dream
HKUNLP/Dream
1,249在 GitHub 上查看
](https://huggingface.co/Dream-org/Dream-v0-Base-7B)
Diffusion-based language model implementation.
Python
在 GitHub 上查看1,249
kuleshov-group/bd3lms
kuleshov-group/bd3lms
1,013在 GitHub 上查看
By Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, Volodymyr Kuleshov
Interpolating between autoregressive and diffusion language models.
Python
在 GitHub 上查看1,013
louaaron/score-entropy-discrete-diffusion
louaaron/Score-Entropy-Discrete-Diffusion
732在 GitHub 上查看
This repo contains a PyTorch implementation for the paper Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution by Aaron Lou, Chenlin Meng and Stefano Ermon.
Discrete diffusion modeling via score-entropy estimation.
Python
在 GitHub 上查看732
kuleshov-group/mdlm
kuleshov-group/mdlm
698在 GitHub 上查看
By Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov
Simple and effective masked diffusion language models.
Python
在 GitHub 上查看698
hkunlp/diffullama
hkunlp/diffullama
395在 GitHub 上查看
Current Diffusion Language Models (DLMs) have been studied at a smaller scale compared to their autoregressive (AR) counterparts and lack fair comparison on language modeling benchmarks. Additionally, training diffusion models from scratch at scale remains challenging. We propose adapting…
Adapting autoregressive models to diffusion language models.
Python
在 GitHub 上查看395
ml-gsai/smdm
ml-gsai/smdm
383在 GitHub 上查看
Masked diffusion models (MDMs) have shown promise in language modeling, yet their scalability and effectiveness in core language tasks, such as text generation and language understanding, remain underexplored. This paper establishes the first scaling law for MDMs, demonstrating a scaling rate…
Scaling masked diffusion models on text.
Python
在 GitHub 上查看383
jetastra/sdar
JetAstra/SDAR
359在 GitHub 上查看
We introduce SDAR (Synergy of Diffusion and AutoRegression), a large-scale diffusion language model that unites the complementary strengths of autoregressive and discrete diffusion modeling. By merging the training efficiency of autoregressive methods with the highly parallel decoding ability of…
Synergistic diffusion-autoregression paradigm for sequence generation.
Python
在 GitHub 上查看359
hzfinfdu/diffusion-bert
Hzfinfdu/Diffusion-BERT
343在 GitHub 上查看
Official implementation of DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models.
Improving generative masked language models with diffusion.
Python
在 GitHub 上查看343
jinjieni/megadlms
JinjieNi/MegaDLMs
339在 GitHub 上查看
MegaDLMs
Large-scale diffusion language model training.
Python
在 GitHub 上查看339
s-sahoo/duo
s-sahoo/duo
234在 GitHub 上查看
By Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, Volodymyr Kuleshov
Exploring the diffusion duality in language modeling.
Python
在 GitHub 上查看234
jinjieni/dlms-are-super-data-learners
JinjieNi/dlms-are-super-data-learners
228在 GitHub 上查看
Diffusion Language Models are Super Data Learners
Exploring diffusion language models as efficient data learners.
Python
在 GitHub 上查看228
google-deepmind/md4
google-deepmind/md4
161在 GitHub 上查看
please use requirementsgpu.txt if your accelerator is GPUs, use requirementstpu.txt when using Google Cloud TPUs.
Simplified and generalized masked diffusion for discrete data.
Python
在 GitHub 上查看161
introspective-diffusion/i-dlm
Introspective-Diffusion/I-DLM
148在 GitHub 上查看
Introspective diffusion language models for improved generation.
Python
在 GitHub 上查看148
dvruette/gidd
dvruette/gidd
119在 GitHub 上查看
By Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, Thomas Hofmann
Generalized interpolating discrete diffusion.
Python
在 GitHub 上查看119
dreamlm/dreamon
DreamLM/DreamOn
117在 GitHub 上查看
](https://huggingface.co/Dream-org/DreamOn-v0-7B)
Diffusion language models for code infilling.
Python
在 GitHub 上查看117
s-sahoo/eso-lms
s-sahoo/Eso-LMs
119在 GitHub 上查看
Subham Sekhar Sahoo \ 1 , Zhihan Yang \ 2 , Yash Akhauri †1 , Johnna Liu †1 , Deepansha Singh †1 , Zhoujun Cheng †3 , Zhengzhong Liu 3 , Eric Xing 3 , John Thickstun 2 , Arash Vahdat 4
Esoteric language models based on diffusion.
Python
在 GitHub 上查看119
igul222/plaid
igul222/plaid
115在 GitHub 上查看
This repository contains code for training and evaluating the models in the paper Likelihood-Based Diffusion Language Models.
Likelihood-based diffusion language models.
Python
在 GitHub 上查看115
dreamlm/dream-coder
DreamLM/Dream-Coder
104在 GitHub 上查看
Dream-Coder 7B is a diffusion LLM for code trained exclusively on open-source data across its development stages—adaptation, supervised fine-tuning, and reinforcement learning. It achieves an impressive 21.4% pass@1 on LiveCodeBench (2410-2505), outperforming other open-source diffusion LLMs by…
Open diffusion language model for code generation.
Python
在 GitHub 上查看104

Awesome Language Diffusion Models GitHub Repositories

ML-GSAI/LLaDA

ZHZisZZ/dllm

HKUNLP/Dream

kuleshov-group/bd3lms

louaaron/Score-Entropy-Discrete-Diffusion

kuleshov-group/mdlm

hkunlp/diffullama

ml-gsai/smdm

JetAstra/SDAR

Hzfinfdu/Diffusion-BERT

JinjieNi/MegaDLMs

s-sahoo/duo

JinjieNi/dlms-are-super-data-learners

google-deepmind/md4

Introspective-Diffusion/I-DLM

dvruette/gidd

DreamLM/DreamOn

s-sahoo/Eso-LMs

igul222/plaid

DreamLM/Dream-Coder