Awesome GitHub RepositoriesOn-Load Quantizers

Acest proiect oferă o suită de modele ușoare de detectare a feței concepute pentru inferență de mare viteză pe dispozitive de edge computing. Se concentrează pe o arhitectură de rețea neuronală compactă care permite detectarea feței umane în medii caracterizate prin resurse de calcul limitate și constrângeri de putere. Sistemul dispune de detectoare de fețe cuantizate disponibile în mai multe formate pentru a asigura compatibilitatea între diverse arhitecturi hardware. Include utilitare pentru exportul și cuantizarea modelelor, permițând convertirea ponderilor antrenate în formate standardizate pentru implementare hardware-agnostică. Proiectul acoperă fluxuri de lucru pentru antrenarea modelelor personalizate și procesarea seturilor de date de imagini pentru a ajusta profilurile de precizie și viteză. Suportă sarcini de computer vision în timp real prin utilizarea aritmeticii doar cu numere întregi și a unui runtime de inferență bazat pe C pentru a minimiza overhead-ul pe hardware-ul embedded.

mistral.rs is an inference engine for large language models that runs locally and exposes models behind OpenAI and Anthropic-compatible APIs. It serves as a multi-model serving platform, capable of loading several models in a single server process with per-request routing and on-demand loading and unloading. The engine supports multimodal inference, processing text alongside images, video, audio, and speech inputs, and includes a quantized model deployment runtime that reduces memory use and speeds up inference on consumer hardware. The project distinguishes itself through an agentic tool exe

YOLO-World is a vision-language framework and open-vocabulary object detection model. It identifies objects in images and video based on free-form text prompts without requiring predefined category labels. The system enables the identification of arbitrary objects by fusing image features with text embeddings. It includes a specialized tool for automated image labeling, which generates bounding box annotations for custom datasets using text-based prompts. The project provides a deployment pipeline for converting models into quantized ONNX and TFLite formats, supporting real-time inference on

Gluon-CV este o bibliotecă de computer vision pentru MXNet care oferă o colecție cuprinzătoare de arhitecturi de viziune pre-implementate și pipeline-uri de antrenament. Servește drept toolkit de cercetare în deep learning și o grădină zoologică de modele (model zoo) care conține ponderi pre-antrenate de ultimă generație pentru analiza imaginilor și a videoclipurilor. Proiectul include o bibliotecă specializată de estimare a posturii umane și un toolkit de compresie a modelelor. Aceste instrumente permit tăierea (pruning) și cuantizarea modelelor de deep learning pentru a crește viteza de inferență și a facilita implementarea pe hardware edge cu resurse limitate. Biblioteca acoperă o gamă largă de capabilități de viziune, inclusiv clasificarea imaginilor, detectarea obiectelor și segmentarea semantică și de instanță. De asemenea, oferă instrumente pentru analiza video, cum ar fi recunoașterea acțiunilor, urmărirea obiectelor și estimarea adâncimii monoculare. Antrenamentul este susținut prin pipeline-uri automatizate și sarcini de lucru distribuite multi-GPU pentru a accelera convergența modelului.

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo

CTranslate2 is a C++ inference engine and runtime for Transformer models, designed to execute models on both CPU and GPU with optimizations for speed and memory efficiency. It functions as a model format converter, quantization tool, and REST API server, enabling deployment of neural machine translation, automatic speech recognition, and text generation models. The engine distinguishes itself through a suite of runtime optimizations including layer fusion, weight-matrix quantization, batch-by-length grouping, and a caching allocator that reuses GPU memory. It supports tensor-parallel model di

This project is an automated audio production system that converts document content, such as PDFs, into spoken dialogue and audio files. It functions as a pipeline that transforms static text into natural two-person scripts for podcast generation. The system synthesizes realistic multilingual speech that includes regional accents and nonverbal cues like laughing or sighing. These voice tracks are combined with generated ambient background music and atmospheric noise to create layered audio compositions. The project also includes capabilities for conversational AI agents, utilizing generation

Sherpa-ncnn is an edge-based speech recognition and synthesis engine designed to run neural network models locally on mobile, embedded, and desktop hardware. It provides a cross-platform framework for offline speech-to-text transcription and text-to-speech synthesis, ensuring that all audio processing occurs on-device without requiring an internet connection or external cloud services. The project distinguishes itself through its use of the ncnn inference engine, which is optimized for low-latency execution on resource-constrained devices. It incorporates on-device model quantization to reduc