Awesome GitHub RepositoriesInference Optimization Tools

Keras is a high-level deep learning framework designed for constructing and training neural networks through the composition of modular, functional layers. It serves as a comprehensive modeling toolkit that provides standardized procedures for defining, evaluating, and deploying complex architectures. By utilizing a directed acyclic graph approach, the framework allows users to build intricate models with multiple inputs, outputs, and shared layers, ensuring consistent numerical execution through functional state management. The project distinguishes itself as a multi-backend machine learning

Facefusion is a modular framework designed for automated image and video manipulation, specializing in tasks such as face swapping, enhancement, and restoration. It functions as a computer vision processing pipeline that chains independent machine learning modules to perform complex transformations, including facial animation, age modification, and lip synchronization. The system is built to handle both real-time interactive feeds and large-scale batch processing tasks. The platform distinguishes itself through a highly extensible architecture that supports custom processing modules and inter

ONNX is an open-source standard for machine learning interoperability that provides a unified format for representing neural network models. By defining a common set of operators and a standardized file structure, it enables models to be shared, exported, and executed consistently across different training frameworks and software ecosystems. The project functions as an intermediate representation layer that decouples model development from deployment. It utilizes a language-neutral binary serialization format to store model structures and weights, ensuring that computational graphs remain por

This project is a comprehensive computer vision library for the PyTorch ecosystem, providing a standardized collection of neural network architectures, datasets, and high-performance transformation utilities. It serves as a foundational framework for building, training, and deploying deep learning models, offering a centralized model registry that allows developers to instantiate architectures with pre-trained weights for tasks such as image classification, object detection, and semantic segmentation. The library distinguishes itself through its modular approach to data and compute management

LitGPT is a training and deployment framework for large language models, providing a suite of tools for pretraining, finetuning, quantizing, evaluating, and serving models within a production environment. It includes a dedicated training pipeline for adapting pretrained models to specific tasks, a quantization tool for reducing weight precision, and an inference server for hosting models via web interfaces. The framework supports high-performance model development through custom architecture implementation and the use of predefined recipes to standardize pretraining and finetuning. It enables

LoRA is a framework for parameter-efficient fine-tuning of large-scale neural networks. It functions by injecting trainable low-rank decomposition matrices into frozen model layers, allowing for task-specific adaptation while preserving the integrity of the original base model weights. The project distinguishes itself by enabling the direct merging of these trained low-rank matrices into primary model weights. This process eliminates additional computational overhead during inference, ensuring that adapted models maintain the same performance characteristics as the original architecture. Furt

This project is a deep learning model compiler and parser that converts ONNX models into optimized TensorRT engines. It functions as a bridge that maps standardized ONNX operators to vendor-specific kernels to enable high-performance inference on NVIDIA GPUs. The system operates as a GPU inference optimizer, selecting hardware-specific kernels and tuning memory allocation to maximize throughput. It transforms neural network graphs into serialized binary execution plans to reduce runtime overhead. The toolset covers deep learning model deployment and edge AI performance tuning. It includes ca

This application is a real-time computer vision system designed to identify and label objects within live video feeds, recorded files, and static images. It functions as a comprehensive framework that integrates pre-trained machine learning models with video processing pipelines to perform multi-object localization and visual data tracking. The system distinguishes itself through a multithreaded architecture that decouples frame acquisition from detection logic, ensuring the interface remains responsive during continuous analysis. It provides specialized scripts for training and optimizing cu