Local LLM Chat Clients

GPT4All is a cross-platform runtime environment designed to execute large language models directly on local consumer hardware. By leveraging an optimized C++ inference backend, it enables private, offline AI interactions without requiring an internet connection or external cloud services. The project provides a comprehensive ecosystem for managing the entire model lifecycle, including discovery, downloading, and configuration of local weights. What distinguishes the platform is its integrated retrieval-augmented generation engine, which allows users to index local documents into semantic vect

Chatbox is a cross-platform desktop application that provides a unified interface for interacting with a wide range of artificial intelligence models. It functions as a model-agnostic client, allowing users to connect to various third-party AI providers or execute open-source models directly on their own hardware. By centralizing these diverse services into a single workspace, the application enables users to manage multiple chat sessions, adjust model parameters, and switch between different AI backends with ease. The project distinguishes itself through a local-first architecture that prior

Jan is a desktop application that functions as a local artificial intelligence model runtime and an open-standard API server. It enables the execution of large language models directly on local hardware, ensuring that data remains private and accessible offline while providing a unified interface for managing model weights and inference runtimes. The platform distinguishes itself by offering a modular inference backend that allows users to swap execution engines based on hardware compatibility and performance needs. It acts as a cross-platform orchestrator, providing the ability to switch bet

This project is a comprehensive platform for hosting and interacting with large language models directly on local hardware. It provides a web-based graphical interface that allows users to manage model loading, configure generation parameters, and execute text or chat interactions entirely offline. By running models locally, the software ensures complete data privacy and eliminates reliance on external cloud services for generative tasks. Beyond basic inference, the platform functions as a versatile workbench for generative AI development. It includes an integrated pipeline for fine-tuning mo

SillyTavern is a comprehensive interface and orchestration platform designed for immersive AI roleplay and interactive chat experiences. It functions as a unified gateway that connects users to a wide array of local and cloud-based large language models, providing a centralized environment to manage complex character personas, narrative context, and model-driven interactions. The platform distinguishes itself through its advanced prompt engineering and automation capabilities. It utilizes a sophisticated macro-based templating engine and vector-database retrieval to dynamically inject lore, c

Chatbox is a desktop client and multi-provider chat interface for interacting with large language model APIs across various service providers and local installations. It functions as a local-first AI conversation manager that stores chat history and user settings directly on the device. The application provides a unified interface to connect multiple AI backends for text generation and image creation. It includes a specialized rendering system for AI responses that supports technical documentation through syntax highlighting, Markdown, and Latex mathematical notation. The platform manages pr

Cherry Studio is a cross-platform desktop application that serves as a centralized workspace for managing and interacting with multiple artificial intelligence models. It functions as a local-first orchestrator, prioritizing user privacy by storing all conversation history and knowledge bases directly on your device. By providing a unified interface for both cloud-based and local AI services, the platform simplifies API key management and allows for consistent model interaction across different operating systems. The application distinguishes itself through a robust retrieval-augmented genera

Serge is a self-hosted web chat interface for running large language models locally using the llama.cpp inference engine. It loads GGUF-format model files directly on your own machine, removing the need for internet connectivity or external API keys, and streams responses to the browser in real time via WebSocket connections. The project is packaged for containerized deployment using Docker and Docker Compose, with a Traefik reverse proxy that handles HTTP and WebSocket routing along with automatic TLS certificate management. Ready-made Kubernetes manifests are also provided, enabling deploym

picoGPT is a lightweight, low-level runtime environment and inference engine designed to load pre-trained checkpoints and execute generative transformer model inference. It provides a minimal implementation of the generative pre-trained transformer architecture to facilitate local language model execution. The project includes a C++ machine learning library for converting model parameters and executing greedy token generation without heavy external dependencies. It handles remote asset synchronization by downloading pre-trained weights, hyperparameters, and vocabulary files from remote server

Maid is a mobile large language model chat client and local runner. It provides a unified interface for interacting with AI models via cloud APIs or by executing model files directly on mobile hardware for offline generation. The project functions as a multi-provider manager that handles API keys and a system for downloading and loading curated model files to the device. It utilizes a provider-agnostic interface to allow switching between local and remote backends. The application includes tools for organizing and exporting chat histories, synchronizing user data across multiple devices, and

PocketPal AI is an on-device LLM chat application for Android that runs small language models locally, enabling private AI conversations without requiring an internet connection. It functions as an offline inference engine that downloads and executes quantized language models directly on the device, with adjustable parameters like temperature and chat templates to control how the AI behaves. The application lets users create custom AI personalities by configuring unique system prompts and contextual settings for different conversational roles. It integrates with the Hugging Face Hub to downlo

assistant-ui is a React interface framework and component kit designed for building conversational user interfaces for artificial intelligence services. It provides a collection of reusable components to render chat windows and interaction histories that connect to large language model backends. The framework uses a composable architecture and a provider pattern to decouple UI components from specific backend implementations. Visual customization is handled through a CSS-variable theming system, while adapter-based integration standardizes different API responses into a consistent format for