Awesome GitHub RepositoriesOn-Load Quantizers

Dieses Projekt bietet eine Suite leichtgewichtiger Gesichtserkennungsmodelle, die für die Inferenz mit hoher Geschwindigkeit auf Edge-Computing-Geräten entwickelt wurden. Es basiert auf einer kompakten neuronalen Netzwerkarchitektur, die die Erkennung menschlicher Gesichter in Umgebungen mit begrenzten Rechenressourcen und Leistungsbeschränkungen ermöglicht. Das System bietet quantisierte Gesichtserkenner in mehreren Formaten, um die Kompatibilität über verschiedene Hardwarearchitekturen hinweg sicherzustellen. Es enthält Dienstprogramme für den Modellexport und die Quantisierung, wodurch trainierte Gewichte in standardisierte Formate für hardwareunabhängige Bereitstellungen konvertiert werden können. Das Projekt deckt Workflows für das Training benutzerdefinierter Modelle und die Verarbeitung von Bilddatensätzen ab, um Präzisions- und Geschwindigkeitsprofile anzupassen. Es unterstützt Echtzeit-Computer-Vision-Aufgaben durch die Verwendung von Integer-only-Arithmetik und einer C-basierten Inferenz-Runtime, um den Overhead auf eingebetteter Hardware zu minimieren.

mistral.rs is an inference engine for large language models that runs locally and exposes models behind OpenAI and Anthropic-compatible APIs. It serves as a multi-model serving platform, capable of loading several models in a single server process with per-request routing and on-demand loading and unloading. The engine supports multimodal inference, processing text alongside images, video, audio, and speech inputs, and includes a quantized model deployment runtime that reduces memory use and speeds up inference on consumer hardware. The project distinguishes itself through an agentic tool exe

YOLO-World is a vision-language framework and open-vocabulary object detection model. It identifies objects in images and video based on free-form text prompts without requiring predefined category labels. The system enables the identification of arbitrary objects by fusing image features with text embeddings. It includes a specialized tool for automated image labeling, which generates bounding box annotations for custom datasets using text-based prompts. The project provides a deployment pipeline for converting models into quantized ONNX and TFLite formats, supporting real-time inference on

Gluon-CV ist eine MXNet-Computer-Vision-Bibliothek, die eine umfassende Sammlung vortrainierter Vision-Architekturen und Trainings-Pipelines bereitstellt. Sie dient als Deep-Learning-Research-Toolkit und Model-Zoo mit State-of-the-Art-Gewichten für die Bild- und Videoanalyse. Das Projekt enthält eine spezialisierte Bibliothek für Human-Pose-Estimation sowie ein Toolkit zur Modellkompression. Diese Tools ermöglichen das Pruning und die Quantisierung von Deep-Learning-Modellen, um die Inferenzgeschwindigkeit zu erhöhen und die Bereitstellung auf ressourcenbeschränkter Edge-Hardware zu erleichtern. Die Bibliothek deckt ein breites Spektrum an Vision-Funktionen ab, darunter Bildklassifizierung, Objekterkennung sowie semantische und Instanz-Segmentierung. Sie bietet zudem Tools für die Videoanalyse, wie Action-Recognition, Objekt-Tracking und monokulare Tiefenschätzung. Das Training wird durch automatisierte Pipelines und verteilte Multi-GPU-Workloads unterstützt, um die Modellkonvergenz zu beschleunigen.

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo

CTranslate2 is a C++ inference engine and runtime for Transformer models, designed to execute models on both CPU and GPU with optimizations for speed and memory efficiency. It functions as a model format converter, quantization tool, and REST API server, enabling deployment of neural machine translation, automatic speech recognition, and text generation models. The engine distinguishes itself through a suite of runtime optimizations including layer fusion, weight-matrix quantization, batch-by-length grouping, and a caching allocator that reuses GPU memory. It supports tensor-parallel model di

This project is an automated audio production system that converts document content, such as PDFs, into spoken dialogue and audio files. It functions as a pipeline that transforms static text into natural two-person scripts for podcast generation. The system synthesizes realistic multilingual speech that includes regional accents and nonverbal cues like laughing or sighing. These voice tracks are combined with generated ambient background music and atmospheric noise to create layered audio compositions. The project also includes capabilities for conversational AI agents, utilizing generation

Sherpa-ncnn is an edge-based speech recognition and synthesis engine designed to run neural network models locally on mobile, embedded, and desktop hardware. It provides a cross-platform framework for offline speech-to-text transcription and text-to-speech synthesis, ensuring that all audio processing occurs on-device without requiring an internet connection or external cloud services. The project distinguishes itself through its use of the ncnn inference engine, which is optimized for low-latency execution on resource-constrained devices. It incorporates on-device model quantization to reduc