Awesome GitHub RepositoriesModel Merging

MergeKit is a toolkit for combining multiple pre-trained large language models into a single entity using algorithmic blending. It provides a specialized system for parameter interpolation and weight extraction to unify model capabilities. The project distinguishes itself through an evolutionary merge optimizer that tunes parameters based on quantitative evaluation metrics. It also features a mixture of experts orchestrator capable of converting dense models into sparse architectures and a tokenizer alignment tool for transplanting embeddings between different models. The toolkit covers a br

Mergekit is a toolkit for combining multiple pretrained large language models into a single model. It functions as an architecture assembler and merging system that transfers capabilities between models using weighted algorithms and layer-wise assembly without requiring additional training. The project provides specialized utilities for extracting low-rank approximations from fine-tuned models to create portable parameter updates. It also includes a framework for converting dense language models into a mixture of experts architecture by constructing gating mechanisms to route inputs to specia

sd-scripts is a suite of utilities designed for fine-tuning generative models, preprocessing datasets, and converting model weights. It provides a collection of scripts for executing Stable Diffusion training through methods such as DreamBooth, textual inversion, and full fine-tuning, alongside a framework for creating and managing Low-Rank Adaptation weights. The project features specialized capabilities for model weight conversion between different architectures and precision formats. It includes tools for merging adaptation weights into base models, extracting weights from trained models,

ComfyUI-nunchaku is a 4-bit diffusion inference engine and a set of nodes for running low-precision quantized diffusion models within ComfyUI visual workflows. It provides a backend that reduces memory overhead and increases generation speed for transformer models. The project includes specialized tools for identity-preserving generation and an image-to-image guidance toolkit that uses depth maps and reference images. It also features a multimodal visual question answering implementation and a utility for merging multiple quantized model files into single unified files. The engine covers a b