AITemplate is an ahead-of-time deep learning compiler that translates PyTorch neural networks into standalone C++ source code. It functions as a PyTorch to C++ compiler and a GPU kernel fusion engine, producing self-contained executable binaries that run inference without requiring a Python interpreter or deep learning framework runtime. The project generates optimized CUDA and HIP C++ code specifically for NVIDIA TensorCores and AMD MatrixCores. It focuses on maximizing throughput for half-precision floating-point operations through a system that combines multiple neural network operators in
Local-File-Organizer is a local-first file classification system that uses on-device machine learning models to categorize documents and media into structured directories. It functions as an automated file classifier and asset manager that leverages local inference to sort files based on content, meaning, and metadata. The project emphasizes privacy by performing all data processing and analysis on the local device, eliminating the need to send sensitive files to external cloud services. It utilizes local models to analyze text and image content to generate descriptive filenames and thematic
CTranslate2 is a C++ inference engine and runtime for Transformer models, designed to execute models on both CPU and GPU with optimizations for speed and memory efficiency. It functions as a model format converter, quantization tool, and REST API server, enabling deployment of neural machine translation, automatic speech recognition, and text generation models. The engine distinguishes itself through a suite of runtime optimizations including layer fusion, weight-matrix quantization, batch-by-length grouping, and a caching allocator that reuses GPU memory. It supports tensor-parallel model di
AidLearning-Framework is an integrated development platform for building and deploying AI applications on ARM-based devices. It combines Android and Linux operating systems running simultaneously on a single device, providing a unified runtime environment for cross-system AI development. The platform includes hardware acceleration across CPU, GPU, and NPU, with a unified API that automatically selects the optimal compute backend for inference.
The main features of aidlearning/aidlearning-framework are: ARM Hardware Accelerators, Python-to-APK Bundlers, Local On-Device AI, GPU-Accelerated Inference, ARM Multi-Hardware Accelerators, On-Device Inference, Model Inference Accelerators, ARM AIOT Platforms.
Open-source alternatives to aidlearning/aidlearning-framework include: nvidia/isaac-gr00t. facebookincubator/aitemplate — AITemplate is an ahead-of-time deep learning compiler that translates PyTorch neural networks into standalone C++… qiuyannnn/local-file-organizer — Local-File-Organizer is a local-first file classification system that uses on-device machine learning models to… pytorch/executorch — ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It… opennmt/ctranslate2 — CTranslate2 is a C++ inference engine and runtime for Transformer models, designed to execute models on both CPU and… chainner-org/chainner — chaiNNer is a GPU-accelerated AI image upscaling application that uses a visual node-based interface for constructing…