Awesome GitHub RepositoriesMemory-Constrained Inference

DiffSynth-Studio is a comprehensive platform for the lifecycle management of generative diffusion models, providing a unified environment for inference, fine-tuning, and training. It utilizes a modular pipeline architecture and a standardized abstraction layer to support consistent workflows across diverse model configurations for image and video generation. The platform distinguishes itself through a memory-optimized inference engine that dynamically manages resources to facilitate high-resolution generation on constrained hardware. It also integrates specialized training capabilities, inclu

Airllm is a framework designed to execute and fine-tune large language models on consumer-grade hardware. By employing layer-wise model decomposition and memory-efficient loading techniques, the engine enables the operation of massive models that would otherwise exceed available system or video memory. The project distinguishes itself through a suite of optimization strategies that balance memory footprint with performance. It utilizes block-wise weight quantization and asynchronous layer prefetching to reduce resource consumption and hide data transfer latency. Additionally, the framework su

PowerInfer is a high-performance local large language model inference engine and sparse inference framework. It provides a runtime for executing models on consumer-grade hardware, utilizing a GPU acceleration backend to optimize tensor operations for graphics processors. The system distinguishes itself through a sparse inference framework that increases generation speed by skipping computations based on activation sparsity in model weights. It includes a GGUF model converter for transforming weights and metadata into a unified binary format, as well as an OpenAI API compatible server for inte

StoryDiffusion is a generative AI system designed for consistent character image and video generation. It utilizes a pluggable cross-attention module to inject shared character representations into pretrained diffusion models, allowing for visual identity stability across multiple images and scenes without retraining the base model. The project features a video generation pipeline that produces temporally coherent sequences from text prompts or condition images. It employs a latent space motion interpolator to predict intermediate frames and semantic motion, enabling long-range video generati

FedML es una biblioteca de entrenamiento de aprendizaje automático distribuido, framework de aprendizaje federado y orquestador de cargas de trabajo de GPU. Proporciona los componentes centrales del sistema necesarios para ejecutar el entrenamiento y ajuste fino de modelos a gran escala en clústeres de GPU multi-nube, on-premise y descentralizados, ofreciendo además un motor dedicado para el despliegue escalable de modelos y un gestor de pipelines MLOps para la gestión del ciclo de vida de extremo a extremo. La plataforma se distingue por permitir el aprendizaje federado que preserva la privacidad en dispositivos edge descentralizados y silos organizacionales, manteniendo los datos sin procesar en el hardware local. También cuenta con un mercado de cómputo de agrupación de recursos que permite a los usuarios contribuir con capacidad de GPU no utilizada a un pool compartido para la ejecución de tareas distribuidas. El sistema cubre una amplia gama de capacidades, incluyendo orquestación de GPU multi-nube, gestión automatizada de pipelines de aprendizaje automático y despliegue de IA en el edge para dispositivos IoT y smartphones. Además, integra herramientas para el ajuste fino de modelos fundamentales, despliegue de inferencia de baja latencia y seguimiento de experimentos de entrenamiento con perfilado de rendimiento de hardware. Los usuarios pueden lanzar y programar cargas de trabajo utilizando una interfaz de línea de comandos y archivos de configuración declarativos.