52 Repos
Awesome GitHub RepositoriesInference Optimization
Methods and frameworks to accelerate decoding and improve inference efficiency.
Explore 52 awesome GitHub repositories matching part of an awesome list · Inference Optimization. Refine with filters or upvote what's useful.
Awesome Inference Optimization GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- inclusionai/dinfer
inclusionAI/dInfer
dInfer is an efficient and extensible inference framework for dLLMs. As illustrated in the following architecture, it modularizes inference into four components: model, diffusion iteration manager, decoder and KV-cache manager. It provides well-designed APIs for flexible algorithms combinations…
Efficient inference framework for diffusion language models.
Python - zhijie-group/discrete-diffusion-forcing
zhijie-group/Discrete-Diffusion-Forcing
📝 Blog Post • 🚀 Online Demo • 🤗 D2F-Dream LoRA • 🤗 D2F-LLaDA LoRA
Faster-than-AR inference via discrete diffusion forcing.
Python - s-sahoo/duo
s-sahoo/duo
By Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, Volodymyr Kuleshov
Diffusion duality framework for inference.
Python - maomaocun/dllm-cache
maomaocun/dLLM-cache
Official PyTorch implementation of the paper "dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching" (dLLM-Cache) in ICML 2026.
Adaptive caching for diffusion large language models.
Python - li-jinsong/daedal
Li-Jinsong/DAEDAL
Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models
Variable-length denoising for diffusion LLMs.
Python - kamichanw/d2cache
Kamichanw/d2Cache
Research dLLM Codebase for Diffusion Language Models Research
Dual adaptive caching for diffusion-based LLMs.
Python - horseee/dkv-cache
horseee/dkv-cache
The pipeline of dKV-Cache
Dedicated cache for diffusion language models.
Python - czg1225/dmax
czg1225/DMax
DMax is a new dLLM paradigm achieving aggressive parallel decoding while preserving generation quality.
Aggressive parallel decoding for diffusion LLMs.
Python - s-sahoo/eso-lms
s-sahoo/Eso-LMs
Subham Sekhar Sahoo \ 1 , Zhihan Yang \ 2 , Yash Akhauri †1 , Johnna Liu †1 , Deepansha Singh †1 , Zhoujun Cheng †3 , Zhengzhong Liu 3 , Eric Xing 3 , John Thickstun 2 , Arash Vahdat 4
Esoteric language model inference techniques.
Python - liusulin/ddpd
liusulin/DDPD
Code repository for the paper Think While You Generate: Discrete Diffusion with Planned Denoising, by Sulin Liu, Juno Nam, Andrew Campbell, Hannes Stärk, Yilun Xu, Tommi Jaakkola, Rafael Gómez-Bombarelli. Tweet and video for the main idea.
Discrete diffusion with planned denoising.
Python - kuleshov-group/remdm
kuleshov-group/remdm
We introduce ReMDM, a simple and general framework to design remasking samplers for masked discrete diffusion models. In this repo, we provide our implementation of different ReMDM strategies for unconditional text generation on OpenWebText. We also provide a demo in this notebook showing how to…
Remasking discrete diffusion with inference-time scaling.
Python - crys-chen/dpad
Crys-Chen/DPad
Efficiency: DPad-enhanced dLLMs achieve up to a 61.39× speedup over vanilla dLLM baselines. Accuracy: DPad-enhanced dLLMs achieve up to a +26.46% improvement over vanilla dLLM baselines. (Evaluation conducted on NVIDIA A100-PCIe-80GB GPUs).
Efficient diffusion models with suffix dropout.
Python - pixeli99/prophet
pixeli99/Prophet
Official implementation of "Diffusion Language Models Know the Answer Before Decoding"
Predictive sampling for diffusion language models.
Jupyter Notebook - eth-sri/constrained-diffusion
eth-sri/constrained-diffusion
Constrained Decoding of Diffusion LLMs with Context-Free Grammars
Constrained decoding with context-free grammars.
Rust - liangrunflora/slow-fast-sampling
LiangrunFlora/Slow-Fast-Sampling
Official PyTorch implementation of the paper "Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles" (Slow Fast Sampling).
Three-principle acceleration for diffusion LLMs.
Python - neuir/pc-sampler
NEUIR/PC-Sampler
ACL '26 Source code for paper "Empirical Analysis of Decoding Biases in Masked Diffusion Models"
Position-aware calibration of decoding bias.
Python - maomaocun/dllm-var
maomaocun/dLLM-Var
Yicun Yang 1 , Cong Wang 1 , Shaobo Wang 1 , Zichen Wen 1 , Biqing Qi 2 , Hanlin Xu 3 , Linfeng Zhang 1 †
Native variable-length generation for diffusion LLMs.
Python
