PromtEngineer/localGPT
LocalGPT
localGPT is a private AI knowledge base and retrieval-augmented generation application. It provides a local document indexer, a hybrid search engine, and an inference interface to enable chatting with private documents and managing a self-hosted information repository without sending data to external servers.
The system distinguishes itself through a dual-pass verification pipeline that ensures generated answers are grounded in retrieved sources, accompanied by explicit source attribution. It employs a hybrid retrieval approach combining semantic vector search with keyword matching and reranking, and utilizes recursive query decomposition to break complex requests into smaller parallel sub-queries.
The platform covers broad capability areas including multi-format document processing, dynamic query routing, and semantic query caching. It also manages conversation history tracking and provides a RESTful API for integrating document retrieval and language model functionality into external applications.
The project integrates with open-source models across different hardware accelerators and includes system health monitoring via structured logs and health endpoints.
Features
- Local RAG Implementations - Provides a complete local system for chatting with private documents using retrieval-augmented generation and local language models.
- Retrieval-Augmented Generation - Combines local document retrieval with a large language model to generate grounded responses using private context.
- Retrieval-Augmented Generation - Implements a local retrieval-augmented generation pipeline to improve AI response accuracy using private documents.
- Documentation Query Interfaces - Enables answering natural language questions using indexed document collections with integrated source citations.
- Hybrid Search Systems - Combines semantic vector search with keyword-based search and reranking to improve retrieval accuracy.
- Local Model Execution - Executes embedding and text generation models directly on local hardware to ensure data privacy.
- Private Document Retrieval - Enables querying private files and documents using local language models without external data exposure.
- Local AI Search - Uses a hybrid search engine and smart router to summarize information found within local files.
- Local Document Indexing - Processes uploaded files into a searchable local collection for retrieval-augmented generation.
- Hybrid Search Engines - Integrates vector-based semantic retrieval with traditional keyword-based indexing to improve document discovery precision.
- Local Knowledge Bases - Parses, indexes, and manages local document collections for private information retrieval and answer extraction.
- Response Grounding Verifications - Employs a dual-pass verification pipeline to ensure generated answers are strictly grounded in retrieved sources.
- Indexing Context Enrichment - Adds supplemental information to documents during indexing to enhance the model's understanding of the content.
- Grounded Answer Generation - Ensures generated responses are supported by traceable citations and source verification from retrieved documents.
- Groundedness Verifications - Performs an independent verification pass to ensure generated responses match the retrieved document context.
- AI Model Integrations - Includes adapters and interfaces to connect various open-source models for inference and embeddings across different hardware.
- Query Routers - Directs user inputs to either a direct model response or a retrieval pipeline based on request complexity.
- Semantic Chunking - Implements semantic chunking to split documents into meaningful segments that fit within model context windows.
- Late Chunking - Implements late chunking to maintain the semantic meaning of data by splitting text after embedding.
- Recursive Decomposers - Breaks complex natural language requests into smaller sub-queries for parallel execution and final synthesis.
- Multi-Format Document Ingestion - Ingests various file types and applies generated context to improve the retrieval of relevant information.
- Recursive Query Decomposition - Breaks complex user requests into multiple sub-queries executed in parallel to synthesize a final comprehensive answer.
- Semantic Search Engines - Utilizes vector embeddings and keyword search to retrieve information from large sets of files based on conceptual meaning.
- Source Attribution Interfaces - Provides UI components that list document references for every response to verify the origin of information.
- Model Deployment and Platforms - Fully private, on-premise document intelligence platform for chatting with your documents using local LLMs.
- Data Interaction & Memory - Private, local document interaction using vector databases and LLMs.
