The nexa-sdk is an on-device AI SDK and multimodal inference engine designed to run large language, vision, and audio models locally on mobile and desktop hardware. It functions as a local LLM runtime and NPU acceleration framework, enabling the execution of generative and discriminative models without reliance on cloud services.
The project distinguishes itself through a dedicated NPU acceleration framework that optimizes model execution on Neural Processing Units to reduce latency and power consumption. It employs hardware-agnostic backend routing to dynamically distribute computations across CPUs, GPUs, and NPUs, and supports GGUF-based model loading for efficient memory mapping and layer offloading.
Its capabilities cover a broad spectrum of AI tasks, including conversational text generation, text-to-image synthesis, and automatic speech recognition. It also provides tools for vector embedding generation and document reranking for local semantic search, as well as a REST-based inference server with an OpenAI-compatible interface for external integration.
The SDK supports cross-platform deployment across Android and Linux environments and includes a Python library for developer integration.
