Awesome GitHub RepositoriesMulti-Format Document Ingestion

localGPT is a private AI knowledge base and retrieval-augmented generation application. It provides a local document indexer, a hybrid search engine, and an inference interface to enable chatting with private documents and managing a self-hosted information repository without sending data to external servers. The system distinguishes itself through a dual-pass verification pipeline that ensures generated answers are grounded in retrieved sources, accompanied by explicit source attribution. It employs a hybrid retrieval approach combining semantic vector search with keyword matching and rerank

This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable i

WeKnora is a multi-tenant retrieval-augmented generation (RAG) knowledge platform and autonomous AI agent framework. It transforms raw documents into queryable knowledge bases and integrates large language models with vector databases to provide grounded AI responses. The system also functions as a Model Context Protocol (MCP) tool server, exposing knowledge search and agentic capabilities to external AI clients. The platform distinguishes itself through an autonomous agent framework that utilizes iterative reasoning, tool calling, and web search to solve multi-step tasks. It implements a sta

This project is an LLM knowledge base builder and personal knowledge management tool. It is a desktop application designed to transform diverse documents into a persistent, interlinked wiki through LLM analysis and incremental ingestion. The system distinguishes itself with a knowledge graph visualizer that uses community detection algorithms to map relationships between concepts and identify topical clusters. It features a hybrid retrieval system that combines keyword matching, vector embeddings, and graph relevance to locate information. The platform covers a wide range of capabilities inc

A fast, helpful, and open-source document parser

Eino is an AI agent development kit and LLM application framework designed for building autonomous agents and orchestrating complex language model workflows. It serves as a multi-agent orchestration engine and workflow orchestrator, providing a graph-based execution model to route data between models, tools, and retrievers. The framework distinguishes itself through a robust set of multi-agent coordination patterns, including supervisor-led management, sequential flows, and autonomous reasoning loops like ReAct. It features advanced agent execution controls such as active turn preemption, che

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

Paper-qa is a retrieval augmented generation system designed for question answering and analysis of scientific literature and technical documents. It functions as an LLM-powered research assistant that extracts grounded answers and summaries with citations from a document library. The system utilizes an agentic RAG orchestrator to iteratively refine search queries and gather evidence through multi-step tool calling. It features a multimodal document parser that extracts text, tables, and images from PDFs, alongside a vector-based indexer that embeds and caches document libraries for efficient

Verba is a retrieval-augmented generation interface and chatbot that uses Weaviate to provide factual answers based on private datasets. It functions as a vector database knowledge base, combining a hybrid search engine with an orchestration interface to connect various large language model providers and embedding services. The system differentiates itself through a RAG pipeline manager for adjusting text chunking rules and retrieval settings, alongside a 3D vector space visualization tool for analyzing the spatial organization and clustering of high-dimensional embeddings. It employs a modul

This project is a reference implementation and application template for Retrieval-Augmented Generation (RAG). It integrates Azure OpenAI with Azure AI Search to enable conversational chat interfaces that provide grounded responses based on private enterprise data. The system is distinguished by its multimodal AI interface, allowing it to process and reason over combined text, image, and PDF content. It employs a hybrid search architecture that combines vector and keyword retrieval with semantic reranking to prioritize the most relevant documents for prompt augmentation. The project covers a

Megaparse is a document parsing tool and RAG data preprocessor designed to convert PDFs, Word documents, and presentations into clean text formats. It functions as a vision-based document extractor that recovers high-fidelity information from images and complex layouts to optimize data for large language model ingestion. The system employs multimodal AI and vision models to perform schema-preserving parsing, which maintains structural hierarchies such as tables and headers. It utilizes lossless structural transformation to turn layout-heavy binary files into text sequences while preserving th

Cocoindex is an incremental data processing engine that builds and maintains live indexes for AI agents, with a core focus on codebase indexing and knowledge graph extraction. The engine uses a function-graph execution model where user-defined Python functions are composed into a directed acyclic graph, and it processes data incrementally so only changed source records or code paths are re-computed, avoiding full recomputation at any scale. It supports automatic schema inference from transformation pipeline type annotations and provides full data lineage tracing, tagging every output record wi

Ce projet est un système de gestion de contenu .NET Core et une plateforme de gestion multi-sites conçue pour organiser et publier du contenu numérique structuré à travers des sites web indépendants depuis une interface centralisée. Il fonctionne à la fois comme un CMS headless et un générateur de site statique, rendant des templates dynamiques en fichiers HTML pour augmenter la vitesse de chargement et l'évolutivité. Le système intègre la génération augmentée par récupération (RAG) pour transformer les documents et le contenu du site web en bases de connaissances IA interrogeables. Il inclut un orchestrateur de flux de travail IA visuel pour définir la logique entre les requêtes des utilisateurs et les sorties des modèles de langage de grande taille. La plateforme couvre de larges domaines de capacités incluant la modélisation de données à schéma flexible, l'organisation des actifs numériques et le contrôle d'accès basé sur les rôles. Elle prend en charge la publication multi-terminaux pour divers appareils, une architecture d'extensibilité basée sur des plugins pour des modules personnalisés, et une suite de sécurité complète incluant le chiffrement de données SM4 et la défense contre les vulnérabilités web. Le logiciel est conçu pour une infrastructure web auto-hébergée et peut être déployé via des conteneurs Docker sur Windows, Linux et macOS.

OpenViking is a multi-tenant context server and knowledge base administration system designed to provide AI agents with persistent long-term memory. It enables the indexing of diverse documents and codebases to support retrieval-augmented generation, allowing agents to recall past interactions, user preferences, and learned experiences across sessions. The project is distinguished by its use of a URI-based virtual filesystem to organize memories, resources, and skills. It implements a tiered context loading system that balances retrieval precision with token budgets by structuring data into a

This project is a comprehensive framework for constructing, managing, and evaluating knowledge graphs through multi-agent reasoning and deep search capabilities. It provides an end-to-end pipeline that ingests multi-format documents, extracts entities and relationships based on configurable schemas, and maintains structured knowledge bases to support evidence-based retrieval. The system distinguishes itself through its multi-agent orchestration, which decomposes complex queries into parallel research steps and synthesizes long-form reports. It leverages advanced graph-based techniques, includ