Hardware acceleration libraries

ncnn is a high-performance neural network inference framework designed for executing deep learning models locally on mobile and desktop hardware. It functions as a specialized engine that enables the deployment of artificial intelligence tasks directly on resource-constrained devices, eliminating the need for external network connectivity or cloud-based processing services. The framework provides a comprehensive toolset for model optimization, allowing users to convert and quantize machine learning models into specialized binary structures. By utilizing static model graph compilation and zero

torch2trt is a tool for transforming PyTorch model modules into optimized TensorRT engines to improve inference performance on NVIDIA GPUs. It functions as a deep learning model optimizer and engine generator that converts neural network layers into high-performance runtime formats for hardware-accelerated graphics processors. The project features a custom layer conversion tool that allows users to define and register Python-based conversion logic to handle specialized operations not supported by default. This extensibility is paired with a registry-based system for mapping specific layer typ

Burn is a deep learning framework designed for building, training, and deploying neural networks using a modular architecture. As a machine learning library built in Rust, it provides a backend-agnostic computational engine that enables the execution of models across diverse hardware, including central processors, graphics processors, and web runtimes. The framework distinguishes itself through a highly portable design that allows developers to maintain a single workflow for both training and inference across heterogeneous environments. It incorporates advanced optimization techniques such as

Deep-Live-Cam is a generative video transformation tool designed for real-time facial manipulation and cinematic enhancement. It functions as a local-first AI runtime, performing all media processing directly on the user's hardware to ensure complete data privacy without external network dependencies. By utilizing a high-performance processing pipeline, the application enables live face swapping and interactive video modifications during active streaming sessions or on pre-recorded media. The system distinguishes itself through a hardware-abstraction execution layer that dynamically routes co

This project is a cross-platform machine learning inference engine designed to execute pre-trained models across diverse operating systems and hardware environments. It functions as a standardized execution framework that manages the entire lifecycle of model inference, from loading and graph optimization to hardware-accelerated execution and generative sequence management. The runtime distinguishes itself through a highly modular architecture that decouples model logic from hardware-specific kernels. By utilizing an execution provider abstraction, it enables developers to offload computation

Anime4K is a collection of graphics shaders and image processing algorithms designed to enhance the visual quality of animated media. It functions as a real-time upscaling engine that increases the resolution of video content during playback, allowing for higher fidelity viewing without the need to permanently re-encode source files. The project distinguishes itself by utilizing hardware-accelerated rendering to perform complex image reconstruction directly on the graphics card. By employing a pass-based pipeline, it chains multiple processing stages to refine frames iteratively, ensuring tha

BitNet is a quantized inference engine designed to execute highly compressed language models by performing arithmetic on low-precision, bit-level weight data. It functions as a model optimization toolkit and a high-performance kernel library, enabling the execution of large language models on consumer hardware by reducing memory footprints and increasing processing speeds. The project distinguishes itself through hardware-specific kernel optimizations that leverage native processor instructions to accelerate matrix multiplication. By utilizing packed integer arithmetic and memory-aligned weig

FFmpeg is a cross-platform multimedia framework designed for the recording, conversion, and streaming of audio and video content. It functions as a comprehensive toolkit that provides both a command-line utility for direct media manipulation and a collection of low-level libraries for integration into custom applications. At its core, the project utilizes a packet-based stream engine and a format-agnostic abstraction layer to handle diverse media standards, containers, and network protocols. The framework distinguishes itself through a modular, graph-based filter execution model that allows f

Video2x is a modular processing framework designed for AI-enhanced video upscaling and frame rate conversion. It functions as a comprehensive toolset for increasing the resolution and visual clarity of media files while generating intermediate frames to improve motion smoothness. The system is built to handle intensive media transformation tasks by leveraging hardware acceleration and custom encoding pipelines. The project distinguishes itself through a plugin-based architecture that allows for the integration of custom machine learning models and specialized algorithms. It utilizes a modular

This project is a PyTorch implementation of the YOLOv4 object detection framework. It provides a system for training and deploying neural networks that identify and locate multiple objects within images and video streams. The framework includes tools for converting trained weights into universal formats and hardware-specific optimized engines, specifically supporting ONNX and TensorRT. It features a TensorRT inference optimizer to reduce latency and increase throughput, as well as a model architecture compatible with NVIDIA DeepStream streaming analytics pipelines. The system covers model tr

DeepSpeed is a high-performance library designed to scale deep learning model training and inference across massive clusters of GPUs and compute nodes. It provides a comprehensive suite of tools for distributed training, enabling the execution of models that exceed the memory capacity of single devices through advanced parameter partitioning, pipeline-based model parallelism, and memory-efficient state offloading. The framework distinguishes itself through specialized communication-efficient optimizers and hardware-aware acceleration techniques. By utilizing gradient compression, quantization

whisper.cpp is a C++ implementation of the Whisper speech-to-text model, serving as a lightweight machine learning inference engine and quantized runtime. It provides high-performance automatic speech recognition and real-time audio transcription without requiring a Python environment. The project utilizes model quantization to reduce memory usage and increase inference speed on local hardware. It incorporates hardware acceleration to optimize processing speed across different processors. The system covers audio processing capabilities including voice activity detection, speaker diarization,

Neural Compressor is a deep learning model compression toolkit and AI inference acceleration engine. It functions as an automated model quantization tool and hardware-aware model compiler designed to reduce the memory footprint of neural networks and decrease execution latency. The project provides specialized frameworks for optimizing large language models, utilizing weight-only quantization and hardware-specific kernels to improve the operational efficiency of generative AI workloads. It maps neural network operators to specialized CPU and GPU vector instructions to accelerate model executi

Transformers.js is a JavaScript library and web machine learning framework designed to run pretrained transformer models directly in the browser. It serves as a client-side inference engine and a wrapper for the ONNX Runtime, enabling the execution of multimodal AI tasks on user devices without the need for a backend server. The library distinguishes itself by providing a unified toolkit for processing text, image, and audio data locally. This architecture supports privacy-preserving model inference and reduces latency by performing all computations on the client's hardware. Its capabilities

OpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models. The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and

Axolotl is a configuration-driven framework designed for the fine-tuning, evaluation, and quantization of large language models. It functions as a comprehensive orchestrator for distributed training, enabling users to manage complex workflows across multi-node and multi-GPU environments. By utilizing structured configuration files, the platform streamlines the setup of training parameters, dataset paths, and hardware distribution strategies. The project distinguishes itself through its support for diverse training methodologies, including full-parameter tuning, parameter-efficient adaptation,

llama-cpp-python provides a Python interface for the llama.cpp library, enabling the execution of large language models with hardware acceleration. It functions as a GGUF model loader and a structured text generator capable of running inference servers and multimodal runtimes for processing both text and image inputs. The project distinguishes itself through a local inference server that exposes model capabilities via an OpenAI-compatible web API. It supports advanced execution techniques including speculative decoding, weight quantization, and layer-based GPU offloading to manage memory acro

Sherpa-ONNX is an ONNX-based speech processing toolkit that provides a local speech recognition engine, an on-device voice synthesis tool, and a speaker identification framework. It is designed as a cross-platform speech API that enables speech-to-text, text-to-speech, and speaker verification tasks to be executed locally on a device without requiring network access. The project is distinguished by its ability to perform zero-shot voice cloning and speaker diarization on-device. It supports a wide range of hardware accelerations, including GPU and various NPU architectures, and provides a Web

Brain.js is a JavaScript neural network library for building, training, and running machine learning models in the browser or Node.js. It provides implementations for several network types, including feedforward networks, recurrent neural networks for time series forecasting, and autoencoders for data compression and denoising. The library features WebGL-based GPU acceleration to increase the speed of neural network computations on the graphics processor. It also includes a visualization tool that generates SVG images to represent the topology and layers of a feedforward network. The framewo

Frigate is a self-hosted network video recorder that functions as a private, local AI-powered vision engine. It manages video streams by performing real-time object detection, tracking, and classification directly on local hardware, ensuring that security monitoring and activity recording remain independent of cloud services. The system distinguishes itself through a modular, hardware-accelerated video pipeline that offloads intensive decoding and machine learning inference to dedicated GPUs, NPUs, or specialized accelerators like Coral TPUs and Hailo modules. It utilizes state-based object t

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

LiteRT is a runtime and API for executing machine learning and generative AI models on mobile, desktop, and IoT hardware. It consists of an inference engine and a specialized environment for running quantized large language and diffusion models locally on edge hardware. The system includes an ahead-of-time model compiler that translates models into hardware-specific bytecode to reduce startup latency and memory overhead. It provides a unified interface for Neural Processing Units with automatic fallback routing to CPUs or GPUs when specific subgraph support is unavailable. An edge model conve

PaddleDetection is an object detection framework designed for the end-to-end development, training, and deployment of computer vision models. It provides a comprehensive library of modular neural network architectures and pipelines that support object detection, instance segmentation, and multi-object tracking tasks. The project distinguishes itself through a configuration-driven approach that decouples model components like backbones and heads, allowing for the flexible assembly of custom vision workflows. It incorporates advanced techniques such as anchor-free detection logic, joint detecti

Facefusion is a modular framework designed for automated image and video manipulation, specializing in tasks such as face swapping, enhancement, and restoration. It functions as a computer vision processing pipeline that chains independent machine learning modules to perform complex transformations, including facial animation, age modification, and lip synchronization. The system is built to handle both real-time interactive feeds and large-scale batch processing tasks. The platform distinguishes itself through a highly extensible architecture that supports custom processing modules and inter

IREE is an MLIR-based compiler toolchain and runtime designed to translate machine learning models from various frameworks into optimized binaries for execution across diverse hardware targets. It provides a unified pipeline to ingest models from PyTorch, TensorFlow, JAX, and ONNX, lowering them into a common intermediate representation for deployment on CPUs, GPUs, and bare-metal embedded systems. The project distinguishes itself through a bytecode virtual machine and a hardware abstraction layer that decouple high-level model logic from specific hardware instruction sets. It supports sophis

MXNet is a deep learning framework and distributed machine learning engine designed for training and deploying neural networks. It functions as a hardware-agnostic backend that allows for the development of deep learning models through a hybrid of symbolic and imperative programming. The system distinguishes itself through automatic distributed parallelism, which scales training workloads across multiple GPUs and machines. It features an extensible hardware backend interface that enables the integration of custom accelerators and proprietary libraries without modifying the core source code.

Accelerate is a framework for high-performance array computing that provides a domain-specific language for expressing complex mathematical and parallel computations. By utilizing a declarative programming interface, it allows users to define high-level array transformations that are automatically translated into optimized machine code for diverse hardware architectures. The system distinguishes itself through a modular architecture that decouples high-level array operations from hardware-specific instructions. It employs just-in-time compilation and kernel fusion to transform programs into e

Buzz is a desktop application that provides a local speech-to-text engine for transcribing and translating audio and video files. By leveraging local machine inference, the software ensures data privacy and offline performance, removing the need for cloud connectivity during media processing. The application distinguishes itself through a modular plugin architecture that allows for the integration of custom functionality, such as content summarization and automated text formatting, without modifying the core codebase. It also features a speaker diarization pipeline that identifies and labels

Ijkplayer is a cross-platform media playback engine designed to provide consistent audio and video rendering across mobile devices. Built upon established open-source multimedia frameworks, it functions as a unified engine that leverages hardware-accelerated decoding to process diverse media formats. The project distinguishes itself by providing a comprehensive toolchain for compiling and configuring low-level media source code into native binary libraries. This allows developers to integrate high-performance playback directly into mobile applications, utilizing a pluggable output architectur

DeepFaceLive is a desktop application designed for real-time facial replacement and animation within live video streams. By utilizing deep learning models, the software performs high-speed identity mapping and facial feature analysis to transform video content as it is captured. The engine relies on GPU-accelerated inference to execute these complex image manipulation tasks at interactive frame rates. The application distinguishes itself through a modular video processing pipeline that chains specialized tasks to maintain high throughput and low latency. It features a virtual camera streaming

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

Upscayl is a cross-platform desktop application designed to increase the resolution and visual quality of digital images using artificial intelligence. By executing all processing tasks locally on the user's machine, the software ensures that sensitive media files remain private and never leave the host system for cloud-based services. The application distinguishes itself through a hardware-agnostic architecture that offloads intensive rendering workloads directly to the local graphics unit. It utilizes a hardware abstraction layer to translate enhancement commands into instructions compatibl

This library is a web-based framework designed for general-purpose parallel processing and high-performance computation on graphics hardware. It provides a toolkit for executing complex mathematical operations, physics simulations, and procedural visualizations directly within the browser by leveraging the graphics processor for data-intensive tasks. The framework distinguishes itself through automated shader management, which ensures consistent execution across diverse hardware and browser environments. It achieves this by transpiling modern shader syntax into legacy versions and automatical

The Intel GPU Media Driver is a hardware-accelerated driver designed to facilitate video decoding, encoding, and transcoding operations on Intel graphics processing units. It functions as a low-level abstraction layer that enables media applications to offload compute-intensive processing tasks to dedicated graphics hardware engines through a standardized software interface. The driver distinguishes itself by providing a unified hardware abstraction layer that translates high-level media requests into platform-specific instructions across diverse graphics hardware generations. It manages comp

SmarterStreaming is a cross-platform SDK for hardware-accelerated audio and video capture, encoding, and streaming. It provides a complete media pipeline for low-latency RTMP and RTSP streaming, and includes an embedded lightweight RTSP server that can serve live feeds directly from the source device to local network clients without requiring a separate server. The SDK also integrates with GB28181 surveillance platforms, enabling compliant device registration and streaming for standardized video monitoring systems. The project distinguishes itself through a set of integrated capabilities for

PaddleFormers is a framework for the training, fine-tuning, and deployment of large language models. It provides a full lifecycle pipeline for executing large-scale model training and applying adaptation methods to align models with specialized tasks. The project focuses on scaling model operations through distributed training and hardware accelerator integration. It employs pipeline parallelism and mixed-precision training to manage memory and increase throughput across multiple hardware devices. The library includes a curated model zoo for serving pre-trained architectures and tools for pr

PlaidML is a deep learning compiler framework and cross-platform runtime designed to execute machine learning models on a wide variety of hardware targets. It functions as a hardware agnostic tensor engine that translates tensor models into executable code, allowing deep learning networks to run across different compute devices without requiring specific driver dependencies. The system enables the execution of models on custom or limited hardware by using JSON specifications to define device hardware. It employs a domain-specific language to describe tensor computations and provides a middle

Accelerate is a PyTorch distributed training library that abstracts the boilerplate required to run models across multiple GPUs, TPUs, and CPUs. It functions as a deep learning model scaler and distributed hardware orchestrator, allowing the same training script to run on different hardware backends without modifying the core logic. The project provides a distributed training command line interface for configuring compute environments and launching jobs across single or multi-node clusters. It includes a mixed precision training framework to implement FP16 and BF16 precision, reducing memory

Deeplearnjs is a JavaScript deep learning framework and automatic differentiation engine designed for building and training artificial intelligence models within a web browser environment. It functions as a machine learning library that leverages WebGL to provide hardware acceleration for neural networks. The project serves as a high-performance linear algebra library, using the GPU to execute operations on multi-dimensional arrays. This enables the implementation of deep learning models and the execution of client-side machine learning inference. The framework covers the complete automatic

TensorFlow.js is a JavaScript machine learning library and browser-based runtime used to build, train, and execute models. It functions as a WebGL accelerated tensor engine, providing a foundation for high-performance linear algebra operations and an automatic differentiation framework for computing gradients. The project distinguishes itself through its ability to run machine learning directly in web environments, supporting both client-side inference and browser-based training. It enables the deployment of Python-based models by converting Keras or TensorFlow models into compatible formats

face-api.js is a TensorFlow.js face recognition library and browser-based computer vision API. It provides tools for performing face detection, recognition, and landmark prediction within browsers and Node.js. The library includes a biometric identity descriptor generator that creates numerical vectors to compare identity and similarity between images. It features a facial landmark detection tool for mapping sixty-eight specific coordinate points on a face, as well as an age and gender estimation model. Its capabilities cover real-time facial analysis, including the recognition of facial exp

JavaCV provides a Java-based interface for native computer vision and video processing libraries. It functions as a wrapper for native vision libraries, allowing Java applications to perform image analysis, object detection, and video stream processing. The project integrates comprehensive computer vision capabilities, including facial recognition, image segmentation, and optical flow analysis for motion tracking. It also provides tools for hardware geometry calibration and projector-camera alignment to ensure accurate spatial representation. The system covers high-performance media renderin