Open-source alternatives to Mace

oneDNN is a cross-architecture compute library and hardware acceleration framework designed as a oneAPI deep learning library. It functions as a neural network inference engine that provides optimized primitives to accelerate deep learning operations across diverse CPU and GPU architectures. The project distinguishes itself through a combination of just-in-time instruction generation based on detected processor features and microarchitecture-specific tuning. It utilizes graph-based operation compilation to minimize overhead and manages layout-aware tensors to optimize data access patterns acr

TensorRT is a deep learning inference engine and software development kit designed to optimize and deploy neural networks for high-performance execution on NVIDIA GPUs. It functions as a GPU acceleration framework that reduces latency and increases throughput for trained models during production deployment. The toolkit imports models from the Open Neural Network Exchange format and transforms them into optimized engines. It utilizes graph-based model optimization, layer-fusion kernel generation, and precision-based quantization to convert floating point weights into lower precision formats.

This repository is a collection of sample applications and reference projects for integrating on-device machine learning APIs into mobile applications. It provides implementation examples for both Android and iOS, demonstrating how to embed computer vision and natural language processing models. The project covers the implementation of mobile computer vision, including text recognition, face detection, and barcode scanning. It includes workflows for real-time image processing and the integration of pre-trained models that process data locally on the device.

AI-on-the-edge-device is an edge AI meter digitizer and computer vision image processor designed to convert images of analog and digital utility meters into numeric values. It functions as an IoT gateway that runs neural network inference locally on hardware to monitor water, power, and gas readings. The system is distinguished by its ability to handle both analog pointers and digital digits through custom-trained neural networks. It includes specialized tools for image alignment, region-of-interest extraction, and hardware-level lighting control to minimize glare on glass surfaces. To mainta

Auto-subs is an AI transcription and automatic captioning tool that converts spoken audio from video files into synchronized subtitles. It functions as a subtitle generator and a transcription bridge, enabling the conversion of speech to text with automatic speaker identification and multi-language translation support. The software prioritizes data privacy by utilizing on-device AI inference to process audio and video files locally on the user's hardware. It distinguishes itself by offering deep integration with professional video editing workflows, allowing users to export timing and transcr

IREE is an MLIR-based compiler toolchain and runtime designed to translate machine learning models from various frameworks into optimized binaries for execution across diverse hardware targets. It provides a unified pipeline to ingest models from PyTorch, TensorFlow, JAX, and ONNX, lowering them into a common intermediate representation for deployment on CPUs, GPUs, and bare-metal embedded systems. The project distinguishes itself through a bytecode virtual machine and a hardware abstraction layer that decouple high-level model logic from specific hardware instruction sets. It supports sophis

The nexa-sdk is an on-device AI SDK and multimodal inference engine designed to run large language, vision, and audio models locally on mobile and desktop hardware. It functions as a local LLM runtime and NPU acceleration framework, enabling the execution of generative and discriminative models without reliance on cloud services. The project distinguishes itself through a dedicated NPU acceleration framework that optimizes model execution on Neural Processing Units to reduce latency and power consumption. It employs hardware-agnostic backend routing to dynamically distribute computations acro

gemma.cpp is a C++ inference engine for Gemma, PaliGemma, and Griffin language models, designed to run directly on-device without Python dependencies. It provides a self-contained runtime that loads quantized model weights and performs text generation on CPU or GPU, along with a model checkpoint converter that transforms PyTorch or Keras checkpoints into a compact binary format for fast loading. The engine supports multiple model architectures, including the Griffin recurrent architecture with gated linear recurrent layers and sliding-window attention for efficient long-sequence handling, as

Local-File-Organizer is a local-first file classification system that uses on-device machine learning models to categorize documents and media into structured directories. It functions as an automated file classifier and asset manager that leverages local inference to sort files based on content, meaning, and metadata. The project emphasizes privacy by performing all data processing and analysis on the local device, eliminating the need to send sensitive files to external cloud services. It utilizes local models to analyze text and image content to generate descriptive filenames and thematic

oneDNN is a library for deep learning acceleration that provides optimized building blocks for neural network training and inference. It manages tensor computation across CPU and GPU hardware, enabling the execution of high-performance primitives for model training and neural network inference optimization. The project distinguishes itself through hardware-specific kernel optimization and the use of just-in-time compilation to target specific processor instruction sets. It supports quantized neural network execution using both static and dynamic quantization to reduce memory usage and increas

AidLearning-Framework is an integrated development platform for building and deploying AI applications on ARM-based devices. It combines Android and Linux operating systems running simultaneously on a single device, providing a unified runtime environment for cross-system AI development. The platform includes hardware acceleration across CPU, GPU, and NPU, with a unified API that automatically selects the optimal compute backend for inference. The framework distinguishes itself by enabling Python-based AI projects to be packaged directly into Android APK files for installation on mobile devic

Moonshine is a complete on-device voice interface toolkit that provides speech recognition, text-to-speech synthesis, phonetic processing, speaker diarization, and intent recognition, all running locally on edge hardware without any cloud dependency. It executes quantized neural networks for speech and language tasks directly on the device, enabling fully offline conversational AI capabilities. The toolkit distinguishes itself by orchestrating multi-turn spoken exchanges through a conversational flow manager that maintains context across interactions and manages branching dialog flows. It inc

Porcupine is an on-device wake word detection engine that listens for a specific spoken phrase in real-time audio and triggers actions, all processed locally without any cloud connectivity. It includes a custom wake word model creator that generates production-ready models from just a few spoken examples in seconds, requiring no training data. Beyond wake word detection, Porcupine also provides on-device speech recognition for real-time transcription with custom vocabulary, an on-device audio content searcher that indexes and finds spoken phrases in audio files or streams, and a lightweight vo

ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,

ubuntu-rockchip is a Rockchip Ubuntu distribution and GPU-accelerated Linux image designed for RK35XX series single-board computers. It provides a 64-bit ARM64 Linux environment that deploys a standard Ubuntu desktop on top of specialized board-support packages. The system features hardware drivers for 3D graphics acceleration and 4K video rendering and streaming. These optimizations allow for high-resolution media playback and increased rendering speeds for 3D applications on Rockchip hardware. The distribution includes tools for single-board computer administration, such as a guided first-

This project is a PyTorch object detection framework that implements the Faster R-CNN architecture. It serves as a vision model for predicting precise bounding boxes around multiple objects within images and live video feeds. The system is optimized for multi-GPU training to reduce the time required for model convergence. It utilizes a GPU-accelerated design to handle the training and inference of complex detection networks. The framework covers the full object detection lifecycle, including custom network training and inference for static images and real-time video streams. It includes capa

DeepDanbooru is a deep learning tool for tagging anime-style images with Danbooru-style tags. It uses a pre-trained convolutional neural network to analyze images and predict tags identifying characters, attributes, and artwork details. The project provides a complete pipeline for training custom tag recognition models. Users can prepare datasets by downloading tag definitions from a remote Danbooru server using authenticated API requests, then store image-tag pairs in a structured SQLite database. The training workflow supports filtering datasets by rating or score criteria, configuring hype

Neural Compressor is a deep learning model compression toolkit and AI inference acceleration engine. It functions as an automated model quantization tool and hardware-aware model compiler designed to reduce the memory footprint of neural networks and decrease execution latency. The project provides specialized frameworks for optimizing large language models, utilizing weight-only quantization and hardware-specific kernels to improve the operational efficiency of generative AI workloads. It maps neural network operators to specialized CPU and GPU vector instructions to accelerate model executi

Brain.js is a JavaScript neural network library for building, training, and running machine learning models in the browser or Node.js. It provides implementations for several network types, including feedforward networks, recurrent neural networks for time series forecasting, and autoencoders for data compression and denoising. The library features WebGL-based GPU acceleration to increase the speed of neural network computations on the graphics processor. It also includes a visualization tool that generates SVG images to represent the topology and layers of a feedforward network. The framewo

LightGlue is a deep learning framework designed for local feature matching and high-speed correspondence estimation between pairs of images. It functions as a computer vision matching model that identifies corresponding keypoints across different viewpoints. The system utilizes an adaptive neural network architecture that dynamically optimizes inference speed by pruning its own depth and width based on the input image pairs. This approach employs a transformer-style attention mechanism and cross-image attention to compute correlations between feature descriptors. The matching process include

GhostNet provides a set of efficient AI model architectures and neural network design patterns designed to reduce computation and memory overhead. It serves as a computer vision backbone and a lightweight vision transformer, optimizing the balance between predictive accuracy and inference speed. The project focuses on reducing resource consumption for deployment on mobile devices and edge hardware. It achieves this through the use of lightweight vision transformer implementations and architectures that minimize the total number of parameters. The codebase covers a range of capabilities for i

Jellyfin-desktop is a cross-platform media client for Windows, macOS, and Linux. It functions as a native wrapper that provides a standalone desktop shell for browsing and playing movies, music, and shows from a remote home media server. The application integrates Chromium with native operating system GPU drivers to provide hardware-accelerated video playback for high-resolution media streams. It utilizes an Electron-based runtime to render a web-based user interface within a native desktop environment. The client manages local file-system caching for session data and application assets to r

GoCV is a computer vision library and Go language binding for OpenCV. It serves as an image processing toolkit and deep learning inference engine, providing programmatic access to a wide range of algorithms for image manipulation, object detection, and video analysis. The project differentiates itself through high-performance native bindings and hardware acceleration. It utilizes a foreign function interface to map Go calls to C++ functions and includes a hardware-agnostic backend dispatch to route neural network tasks to computation engines such as CUDA and OpenVINO. The library covers a br

warp-ctc is a high-performance library for calculating connectionist temporal classification loss to train sequence-to-sequence deep learning models. It provides a numerical stability layer using log-space computation to prevent underflow and precision errors during probability calculations for long sequences. The library utilizes hardware-accelerated kernels to compute loss in parallel across CPU and GPU architectures. It focuses on increasing training throughput by optimizing the dynamic programming steps of the CTC algorithm. These capabilities support the training of models for speech re

waifu2x-caffe is a deep learning image upscaler and denoiser that uses the Caffe framework to increase image resolution and remove noise from illustrations and photographs. It functions as a neural network image processor that reduces compression artifacts and pixelation while maintaining visual clarity. The project provides specialized neural network weights optimized separately for 2D illustrations and real-world photographs. It includes distinct processing for alpha channels to preserve transparency and employs test-time augmentation to improve output precision. The tool supports both a c