a-ghorbani/pocketpal-ai
Pocketpal Ai
PocketPal AI is an on-device LLM chat application for Android that runs small language models locally, enabling private AI conversations without requiring an internet connection. It functions as an offline inference engine that downloads and executes quantized language models directly on the device, with adjustable parameters like temperature and chat templates to control how the AI behaves.
The application lets users create custom AI personalities by configuring unique system prompts and contextual settings for different conversational roles. It integrates with the Hugging Face Hub to download and load both public and gated models, supporting authentication tokens for models that require special permissions. Users can download, load, and switch between multiple small language models from a built-in list or external hub, and benchmark model performance by measuring tokens per second and memory usage on the device.
Features
- Offline Chat Clients - Runs small language models directly on the device for private conversations without needing an internet connection.
- Custom AI Assistant Development - Building personalized AI personalities with custom system prompts and settings for different conversational roles.
- Inference Parameters - Adjusts model parameters like system prompt, temperature, and chat templates to control how the AI behaves.
- Local Chat Applications - An Android application that runs small language models locally for private AI conversations without internet connectivity.
- Inference Configuration Parameters - Adjusts model behavior through configurable parameters like system prompt, temperature, and chat templates.
- On-Device Inference Engines - Runs quantized language models on-device with adjustable temperature and chat template parameters.
- On-Device Models - Runs small language models directly on the device using local inference without requiring an internet connection.
- Personal AI Assistants - Builds custom AI personalities by setting unique system prompts and contextual settings for different conversation roles.
- Model Downloaders - Downloads and loads models from the Hugging Face Hub, including gated models requiring authentication tokens.
- Model Performance Benchmarking - Measures tokens-per-second and memory usage of loaded models to compare local AI performance.
- Runtime Model Swapping - Supports downloading, loading, and switching between multiple small language models from a built-in list or external hub.
- Model Downloaders - Downloads, loads, and switches between multiple small language models from a built-in list or the Hugging Face Hub.
- Hugging Face Authenticators - Authenticates with a Hugging Face token to download and run models that require special permissions.
