LocalGPT

localGPT is a private AI knowledge base and retrieval-augmented generation application. It provides a local document indexer, a hybrid search engine, and an inference interface to enable chatting with private documents and managing a self-hosted information repository without sending data to external servers.

The system distinguishes itself through a dual-pass verification pipeline that ensures generated answers are grounded in retrieved sources, accompanied by explicit source attribution. It employs a hybrid retrieval approach combining semantic vector search with keyword matching and reranking, and utilizes recursive query decomposition to break complex requests into smaller parallel sub-queries.

The platform covers broad capability areas including multi-format document processing, dynamic query routing, and semantic query caching. It also manages conversation history tracking and provides a RESTful API for integrating document retrieval and language model functionality into external applications.

The project integrates with open-source models across different hardware accelerators and includes system health monitoring via structured logs and health endpoints.

Features

Local RAG Implementations - Provides a complete local system for chatting with private documents using retrieval-augmented generation and local language models.

Retrieval-Augmented Generation - Combines local document retrieval with a large language model to generate grounded responses using private context.

Retrieval-Augmented Generation - Implements a local retrieval-augmented generation pipeline to improve AI response accuracy using private documents.

Documentation Query Interfaces - Enables answering natural language questions using indexed document collections with integrated source citations.

Hybrid Search Systems - Combines semantic vector search with keyword-based search and reranking to improve retrieval accuracy.

Local Model Execution - Executes embedding and text generation models directly on local hardware to ensure data privacy.

Private Document Retrieval - Enables querying private files and documents using local language models without external data exposure.

Local AI Search - Uses a hybrid search engine and smart router to summarize information found within local files.

Local Document Indexing - Processes uploaded files into a searchable local collection for retrieval-augmented generation.

Hybrid Search Engines - Integrates vector-based semantic retrieval with traditional keyword-based indexing to improve document discovery precision.

Local Knowledge Bases - Parses, indexes, and manages local document collections for private information retrieval and answer extraction.

Response Grounding Verifications - Employs a dual-pass verification pipeline to ensure generated answers are strictly grounded in retrieved sources.

Indexing Context Enrichment - Adds supplemental information to documents during indexing to enhance the model's understanding of the content.

Grounded Answer Generation - Ensures generated responses are supported by traceable citations and source verification from retrieved documents.

Groundedness Verifications - Performs an independent verification pass to ensure generated responses match the retrieved document context.

AI Model Integrations - Includes adapters and interfaces to connect various open-source models for inference and embeddings across different hardware.

Query Routers - Directs user inputs to either a direct model response or a retrieval pipeline based on request complexity.

Semantic Chunking - Implements semantic chunking to split documents into meaningful segments that fit within model context windows.

Late Chunking - Implements late chunking to maintain the semantic meaning of data by splitting text after embedding.

Recursive Decomposers - Breaks complex natural language requests into smaller sub-queries for parallel execution and final synthesis.

Multi-Format Document Ingestion - Ingests various file types and applies generated context to improve the retrieval of relevant information.

Recursive Query Decomposition - Breaks complex user requests into multiple sub-queries executed in parallel to synthesize a final comprehensive answer.

Semantic Search Engines - Utilizes vector embeddings and keyword search to retrieve information from large sets of files based on conceptual meaning.

Source Attribution Interfaces - Provides UI components that list document references for every response to verify the origin of information.

Model Deployment and Platforms - Fully private, on-premise document intelligence platform for chatting with your documents using local LLMs.

Data Interaction & Memory - Private, local document interaction using vector databases and LLMs.

PromtEngineerlocalGPT

Features

Star history