Awesome GitHub RepositoriesInference Optimization Utilities

Unsloth is a high-performance training and inference platform designed to optimize the lifecycle of large language and multimodal models. It provides a comprehensive engine for fine-tuning, executing, and managing models locally, with a focus on reducing memory consumption and increasing compute speed on consumer-grade hardware. The platform distinguishes itself through hand-optimized kernels and automated computational graph techniques that maximize hardware throughput. It supports advanced training methodologies, including reinforcement learning for reasoning and efficient adapter-based fin

Keras is a high-level deep learning framework designed for constructing and training neural networks through the composition of modular, functional layers. It serves as a comprehensive modeling toolkit that provides standardized procedures for defining, evaluating, and deploying complex architectures. By utilizing a directed acyclic graph approach, the framework allows users to build intricate models with multiple inputs, outputs, and shared layers, ensuring consistent numerical execution through functional state management. The project distinguishes itself as a multi-backend machine learning

YOLOv5 is a comprehensive computer vision framework designed for end-to-end deep learning, specializing in real-time object detection, image classification, and instance segmentation. It provides a unified toolkit that manages the entire lifecycle of a model, from initial dataset configuration and hyperparameter tuning to high-speed inference and deployment. The framework utilizes a modular neural architecture, allowing users to swap backbone and head components to tailor models for specific visual tasks. What distinguishes this project is its focus on production-ready deployment and model ef

Faceswap is a comprehensive framework for automated media manipulation and neural face synthesis. It provides a modular pipeline that manages the entire lifecycle of facial feature extraction, deep learning model training, and image conversion. By coordinating complex computer vision workflows, the system enables users to map facial identities between source and destination datasets while maintaining structural alignment and lighting consistency across video frames. The project distinguishes itself through a highly extensible plugin-based architecture that handles hardware-accelerated process

PyTorch Lightning is a high-level deep learning framework for PyTorch that automates training loops and removes repetitive engineering boilerplate. It functions as a structured pipeline for managing machine learning experiments, providing a distributed training orchestrator and tools for mixed-precision training. The framework decouples scientific model architecture from the engineering required for infrastructure and scaling. This separation allows the same model code to execute across CPUs, GPUs, or TPUs through a hardware-agnostic execution engine and a centralized trainer that manages the

Facefusion is a modular framework designed for automated image and video manipulation, specializing in tasks such as face swapping, enhancement, and restoration. It functions as a computer vision processing pipeline that chains independent machine learning modules to perform complex transformations, including facial animation, age modification, and lip synchronization. The system is built to handle both real-time interactive feeds and large-scale batch processing tasks. The platform distinguishes itself through a highly extensible architecture that supports custom processing modules and inter

This project is a singing voice conversion tool based on VITS generative modeling. It transforms the identity of a singing voice to a target speaker while preserving the original melody, lyrics, and intonation. The system distinguishes itself through hybrid voice synthesis, allowing for the blending of multiple speaker identities via linear model interpolation. It utilizes cluster-based feature retrieval to increase target voice similarity and employs a diffusion probabilistic model as a post-processor to remove electronic artifacts and improve vocal clarity. The software covers a broad rang

Label Studio este un instrument de etichetare a datelor de mai multe tipuri și un spațiu de lucru pentru adnotarea datelor, conceput pentru a pregăti seturi de date pentru antrenarea învățării automate. Acesta funcționează ca un pipeline de date integrat în cloud care importă date brute din stocare, gestionează procesul de adnotare și exportă etichete în formate standardizate. Platforma dispune de un cadru de integrare a modelelor de învățare automată care se conectează la servere de modele externe. Acest lucru permite adnotarea asistată de model și învățarea activă, permițând sistemului să efectueze pre-etichetarea și să rafineze predicțiile pe baza feedback-ului uman. Software-ul oferă instrumente de gestionare a proiectelor pentru organizarea seturilor de date și atribuirea sarcinilor utilizatorilor prin acces bazat pe roluri. Suportă diverse tipuri de date și utilizează adaptoare de stocare agnostice față de backend pentru a se conecta cu sisteme de fișiere locale sau furnizori de stocare în cloud. Aplicația poate fi instalată prin configurare manuală sau implementări cu un singur clic pe infrastructura cloud.

labelImg este un instrument desktop de adnotare a imaginilor și un utilitar de pregătire a seturilor de date utilizat pentru a crea seturi de date etichetate pentru antrenarea viziunii computerizate. Acesta oferă o interfață grafică pentru desenarea casetelor de delimitare în jurul obiectelor din imagini și atribuirea lor de etichete de clasă pentru a construi date de referință (ground truth) pentru modelele de învățare automată. Software-ul suportă în mod specific formatul de adnotare Pascal VOC XML, exportând coordonatele imaginilor și numele claselor în structuri XML sau text standard. Acesta permite utilizatorilor să încarce liste de clase predefinite din fișiere text pentru a standardiza denumirea în cadrul întregului proiect. Dincolo de etichetarea inițială, instrumentul acoperă fluxuri de lucru de adnotare a imaginilor, inclusiv vizualizarea adnotărilor salvate și verificarea manuală a setului de date. Aceasta include capacitatea de a marca imaginile ca fiind verificate sau dificile pentru a menține calitatea setului de date.

Paddle is a deep learning framework designed for building, training, and deploying large-scale machine learning models. It incorporates a distributed training engine for optimizing performance across multiple chips and a model inference engine for transforming trained models into production-ready formats for cross-platform execution. The platform features a heterogeneous hardware abstraction and a standardized software stack that allows models to run across diverse hardware architectures through a common interface. It also includes a scientific computing library capable of solving complex dif

This repository serves as a comprehensive collection of reference implementations for the PyTorch machine learning library. It provides practical examples for building, training, and deploying deep learning models, functioning as a toolkit for developers to explore neural network architectures and training workflows. The project distinguishes itself by offering concrete demonstrations of complex machine learning operations, ranging from computer vision tasks like object detection and depth estimation to the training of large-scale transformer models. These examples illustrate how to implement

MLC LLM is a machine learning compiler and inference engine designed to execute large language models locally across diverse hardware platforms, including desktop, mobile, and web environments. By utilizing machine learning compilation, the project transforms high-level model definitions into specialized, hardware-specific binary libraries. This process optimizes model weights and generates compute kernels tailored to the unique memory and processing characteristics of target graphics and mobile hardware. The engine distinguishes itself by providing a unified runtime abstraction that enables

ONNX is an open-source standard for machine learning interoperability that provides a unified format for representing neural network models. By defining a common set of operators and a standardized file structure, it enables models to be shared, exported, and executed consistently across different training frameworks and software ecosystems. The project functions as an intermediate representation layer that decouples model development from deployment. It utilizes a language-neutral binary serialization format to store model structures and weights, ensuring that computational graphs remain por

Candle is a minimalist machine learning framework and deep learning inference engine designed for the Rust programming language. It functions as a low-level tensor computation library, providing the necessary primitives for multi-dimensional array operations and mathematical transformations required to execute pre-trained neural network models. The framework distinguishes itself through a focus on memory efficiency and hardware utilization. It employs static-typed tensor operations to enforce shape validation and memory safety at compile time, while utilizing a lazy-loaded computational graph

FunASR is an automatic speech recognition toolkit and multilingual speech-to-text engine designed to convert spoken audio into written text across more than fifty languages. It provides a framework for speaker diarization, an OpenAI-compatible transcription API for local server hosting, and speech models compatible with the ONNX format. The project distinguishes itself by supporting high-performance inference on edge hardware via self-contained binaries and portable model exports. It incorporates specialized capabilities for natural speech generation with adjustable timbre and emotional expre

This project is a comprehensive computer vision library for the PyTorch ecosystem, providing a standardized collection of neural network architectures, datasets, and high-performance transformation utilities. It serves as a foundational framework for building, training, and deploying deep learning models, offering a centralized model registry that allows developers to instantiate architectures with pre-trained weights for tasks such as image classification, object detection, and semantic segmentation. The library distinguishes itself through its modular approach to data and compute management

LitGPT is a training and deployment framework for large language models, providing a suite of tools for pretraining, finetuning, quantizing, evaluating, and serving models within a production environment. It includes a dedicated training pipeline for adapting pretrained models to specific tasks, a quantization tool for reducing weight precision, and an inference server for hosting models via web interfaces. The framework supports high-performance model development through custom architecture implementation and the use of predefined recipes to standardize pretraining and finetuning. It enables

LoRA is a framework for parameter-efficient fine-tuning of large-scale neural networks. It functions by injecting trainable low-rank decomposition matrices into frozen model layers, allowing for task-specific adaptation while preserving the integrity of the original base model weights. The project distinguishes itself by enabling the direct merging of these trained low-rank matrices into primary model weights. This process eliminates additional computational overhead during inference, ensuring that adapted models maintain the same performance characteristics as the original architecture. Furt

PaddleFormers is a framework for the training, fine-tuning, and deployment of large language models. It provides a full lifecycle pipeline for executing large-scale model training and applying adaptation methods to align models with specialized tasks. The project focuses on scaling model operations through distributed training and hardware accelerator integration. It employs pipeline parallelism and mixed-precision training to manage memory and increase throughput across multiple hardware devices. The library includes a curated model zoo for serving pre-trained architectures and tools for pr

TensorRT-LLM is a platform and toolkit designed for compiling, optimizing, and serving transformer-based models on accelerated hardware. It functions as a framework that transforms machine learning models into efficient execution graphs, providing an engine to refine these models for specific hardware to maximize throughput and minimize latency during text generation. The project distinguishes itself through advanced execution strategies that manage the entire inference pipeline. It utilizes kernel-level fusion and static graph execution to optimize mathematical operations and computational f