# Hybrid Search Engines for RAG > Search results for `hybrid search combining keyword and vector retrieval for RAG` on awesome-repositories.com. 112 total matches; showing the first 50. Explore on the web: https://awesome-repositories.com/q/hybrid-search-combining-keyword-and-vector-retrieval-for-rag **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [this search on awesome-repositories.com](https://awesome-repositories.com/q/hybrid-search-combining-keyword-and-vector-retrieval-for-rag).** ## Results - [openai/chatgpt-retrieval-plugin](https://awesome-repositories.com/repository/openai-chatgpt-retrieval-plugin.md) (21,192 ⭐) — This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable information and secures private data stores using OAuth-based user authentication. The capability surface covers multi-format file indexing for PDF, DOCX, and PPTX files, alongside document ingestion from JSON and ZIP archives. It supports multiple vector storage backends, including PostgreSQL with pgvector, Redis, and cloud-native services. The architecture is designed for containerized deployment via Docker and includes tools for metadata extraction and real-time data synchronization through webhooks. The project provides a local development server with pre-configured routing and security to verify plugin functionality before deployment. - [datawhalechina/all-in-rag](https://awesome-repositories.com/repository/datawhalechina-all-in-rag.md) (3,989 ⭐) — This project is a retrieval augmented generation framework designed to build pipelines that connect unstructured data and knowledge graphs with large language models. It functions as a vector database orchestrator for indexing text and multimodal content, as well as a system for translating natural language queries into structured database commands. The framework integrates a hybrid retrieval engine that combines dense vector search with sparse keyword matching to increase the precision of retrieved contexts. It further enhances reasoning and relationship mapping through a graph-augmented retrieval system. The system includes a toolkit for measuring the quality of retrieval and generation processes using standardized metrics. It also provides mechanisms to enforce predefined schemas and patterns on model responses to ensure consistent output for downstream applications. The project is implemented in Python. - [mastra-ai/mastra](https://awesome-repositories.com/repository/mastra-ai-mastra.md) (21,221 ⭐) — Mastra is an orchestration framework designed for building, deploying, and managing autonomous AI agents and multi-agent systems. It provides a comprehensive suite of primitives for creating resilient AI applications, including durable workflow orchestration, event-driven agent loops, and semantic memory management. By integrating these core components, the platform enables developers to build complex, multi-step processes that can reason about goals and execute tasks without manual intervention. The framework distinguishes itself through its focus on observability and secure, isolated execution. It features a built-in telemetry pipeline that captures structured execution traces, logs, and performance metrics, allowing for real-time debugging and evaluation of agent behavior. Furthermore, it utilizes sandboxed environments to isolate code execution and filesystem operations, ensuring that agent interactions remain secure and reproducible. Mastra covers a broad capability surface, including multi-agent delegation hierarchies, schema-validated tool execution, and real-time voice interaction. It supports advanced orchestration patterns such as human-in-the-loop approvals, persistent state management for long-running workflows, and retrieval-augmented generation using vector-based semantic memory. These features are designed to work together to support the entire lifecycle of AI-powered applications, from initial development and testing to production deployment. The project is built for TypeScript environments and provides a modular architecture that integrates with existing web stacks and infrastructure. It includes a client SDK for interacting with remote agents and supports various authentication providers to secure API endpoints and agent resources. - [milvus-io/milvus](https://awesome-repositories.com/repository/milvus-io-milvus.md) (44,804 ⭐) — Milvus is a specialized vector database engine designed for the indexing, management, and high-speed similarity retrieval of high-dimensional vector embeddings. It functions as a similarity search engine capable of identifying nearest neighbors within large-scale vector spaces, supporting the storage and retrieval of billions of data points while maintaining consistent performance. The system utilizes a distributed architecture that decouples storage, query, and coordination into independent services, allowing for horizontal scaling across clusters. It employs a global indexing mechanism that builds specialized data structures across immutable, independently indexed segments. This design, combined with a shared-storage decoupled model, enables compute and storage resources to scale independently in cloud environments, while a log-based persistence layer ensures data durability and state recovery. The platform supports a wide range of data retrieval patterns, including retrieval-augmented generation, hybrid search, and multimodal data retrieval for text, images, and graphs. Deployment options range from lightweight local instances for rapid prototyping to robust standalone setups and fully managed distributed clusters. Documentation includes sizing tools to assist in estimating hardware requirements based on specific data volumes and operational patterns. - [camel-ai/camel](https://awesome-repositories.com/repository/camel-ai-camel.md) (17,253 ⭐) — This project is a comprehensive framework for building and managing autonomous agent systems. It provides a unified architecture for orchestrating multi-agent societies, where specialized agents collaborate through roleplay to decompose and solve complex tasks. The system integrates language models with external environments, enabling agents to perform real-world actions through a standardized tool-calling abstraction layer. The framework distinguishes itself through its focus on iterative reasoning and data reliability. It employs automated feedback loops to refine agent outputs and self-evaluate reasoning traces, ensuring high-quality results. To maintain operational integrity, the system enforces schema-based output parsing for reliable workflow integration and utilizes sandboxed environments for secure, isolated code execution. Beyond its core orchestration capabilities, the project includes a suite of utilities for retrieval-augmented generation and synthetic data production. It supports persistent memory management via vector-based context retrieval and provides extensive tooling for web automation, API integration, and human-in-the-loop oversight. The platform is designed to be model-agnostic, offering a consistent interface for interacting with a wide range of proprietary and open-source language models. - [oramasearch/orama](https://awesome-repositories.com/repository/oramasearch-orama.md) (10,436 ⭐) — Orama is a search engine and vector database that provides full-text indexing, geospatial calculations, and semantic vector storage. It functions as an LLM retrieval engine designed to provide grounded context to language models for conversational interfaces. The project implements hybrid search by combining dense vector embeddings with inverted keyword indices to retrieve documents based on both semantic meaning and exact text matches. It utilizes a WebAssembly module to execute search logic across different JavaScript environments and platforms. The system covers a broad range of retrieval capabilities, including faceted search with category counts, geographical distance filtering, and typo tolerance. It also includes a middleware pipeline for integrating external plugins and tools for search result merchandising to influence document ranking via custom rules. - [labring/fastgpt](https://awesome-repositories.com/repository/labring-fastgpt.md) (27,132 ⭐) — FastGPT is a comprehensive platform for building, deploying, and managing context-aware artificial intelligence applications. It provides a unified environment that integrates custom data sources with language models, utilizing a retrieval-augmented generation engine to ground responses in accurate, domain-specific information. The system is designed for enterprise-scale use, featuring multi-tenant architecture, administrative controls, and secure authentication protocols including OAuth 2.0 and custom single sign-on integration. The platform distinguishes itself through a visual, node-based workflow orchestrator that allows users to design complex business logic and automated task sequences without manual coding. It offers sophisticated knowledge base management, supporting multi-vector data mapping, hybrid search fusion, and automated website content synchronization. To ensure high-quality outputs, the system includes tools for search query optimization, result reranking, and automated performance evaluation, allowing developers to score and analyze the accuracy of their applications across multiple iterations. Beyond its core generation and retrieval capabilities, the platform provides extensive utilities for data handling and organizational management. This includes intelligent parsing of complex document formats, flexible search modes, and granular access controls for team management. Users can also leverage secure, sandboxed rendering for rich content and export cited documents for offline review, ensuring a complete lifecycle for production-ready AI services. - [falkordb/falkordb](https://awesome-repositories.com/repository/falkordb-falkordb.md) (3,437 ⭐) — FalkorDB is a high-performance graph database management system and vector graph database. It serves as a knowledge graph construction tool and a GraphRAG knowledge store, integrating structured property graphs with vector search to provide grounded context for large language models. The engine is designed as a multi-tenant graph engine, capable of hosting thousands of isolated datasets within a single instance. The system distinguishes itself by using linear algebra for query execution, treating relationship tensors as matrix multiplications to achieve low-latency multi-hop traversals. It utilizes sparse-matrix graph storage and vectorized traversals to process thousands of relationships simultaneously. These capabilities are combined with hybrid vector-graph indexing to unify semantic similarity search with structural graph exploration. The platform covers a broad range of capabilities, including GraphRAG orchestration, AI agent memory implementation, and advanced graph analytics such as community detection and centrality ranking. It supports OpenCypher query execution and provides connectivity via the Bolt and RESP protocols. Additional functionality includes automated ontology loading, temporal data tracking, and real-time binary replication for high availability. The database supports migration from Neo4j and can be deployed as a distributed cluster or as an embedded graph engine. - [lorenzoromani1983/wayback-keyword-search](https://awesome-repositories.com/repository/lorenzoromani1983-wayback-keyword-search.md) (180 ⭐) — This tool downloads each page from the Wayback Machine for a specific domain and enables further keyword search on each saved page. - [microsoft/generative-ai-for-beginners](https://awesome-repositories.com/repository/microsoft-generative-ai-for-beginners.md) (112,045 ⭐) — This project is a comprehensive, open-source educational curriculum designed to guide developers through the mastery of generative artificial intelligence. It provides a structured learning path that covers foundational concepts, prompt engineering, and the practical application of large language models. The repository serves as a central hub for skill acquisition, offering sequential modules that progress from basic model mechanics to advanced architectural patterns. The curriculum distinguishes itself by focusing on the end-to-end lifecycle of intelligent software, including the implementation of retrieval-augmented generation and agentic workflow orchestration. It provides technical guidance on integrating diverse models—ranging from open-source options to cloud-based services—while emphasizing responsible development through systematic safety guardrails and ethical design practices. Learners are equipped to build functional applications, such as conversational interfaces, semantic search tools, and automated content generators, using standardized interfaces and modern development techniques. Beyond core model implementation, the resource covers operational practices for monitoring and maintaining AI systems in production. It includes practical modules on fine-tuning, vector-based indexing, and designing intuitive user experiences for intelligent systems. The repository is structured to support developers through every stage of the process, from initial environment configuration and dependency management to deployment readiness and troubleshooting. - [willwulfken/midjourney-styles-and-keywords-reference](https://awesome-repositories.com/repository/willwulfken-midjourney-styles-and-keywords-reference.md) (12,285 ⭐) — This project serves as a comprehensive reference tool for prompt engineering within generative image models. It provides a structured guide for exploring artistic styles, technical parameters, and keyword combinations to assist in achieving specific aesthetic outcomes and consistent visual themes. The resource distinguishes itself by enabling direct comparisons between different model versions, allowing users to observe how specific keywords and settings influence output quality over time. By organizing visual examples and technical data into a hierarchical taxonomy, it facilitates the iterative testing and refinement of prompts to improve the predictability of generated imagery. The documentation is maintained as a version-controlled repository and rendered as a static site, featuring a responsive grid layout for browsing collections. It includes a client-side search index that allows for immediate filtering of keywords and parameters without requiring server-side requests. - [infiniflow/ragflow](https://awesome-repositories.com/repository/infiniflow-ragflow.md) (82,922 ⭐) — This project is a comprehensive retrieval-augmented generation platform designed for building, managing, and deploying knowledge-based AI applications. It provides a unified environment for organizing datasets, configuring conversational chat assistants, and developing autonomous agents that execute multi-step reasoning workflows. By integrating document intelligence with advanced retrieval pipelines, the platform enables the creation of grounded, verifiable responses supported by traceable citations. The platform distinguishes itself through deep document understanding and sophisticated knowledge orchestration. It supports complex document parsing, including the extraction of tables and images, and utilizes graph-based indexing to enhance reasoning over large document collections. Users can configure multiple recall strategies and fused re-ranking to optimize retrieval accuracy, while the system maintains context through multi-turn dialogue management and flexible tool-use frameworks. The architecture is built on a modular, containerized microservice foundation that supports both local inference engines and external language model APIs. It includes asynchronous task processing for document ingestion and indexing, ensuring system responsiveness during heavy workloads. The platform also provides a standardized interface for model abstraction, allowing for seamless integration with existing language model ecosystems. Developers can interact with the platform through a comprehensive suite of RESTful endpoints and Python client libraries, which cover the full lifecycle of agents, datasets, and knowledge graphs. The system is designed for flexible deployment, offering configurable environment settings and support for custom containerized environments to facilitate local development and infrastructure portability. - [toneli/rt-retrieving-and-thinking](https://awesome-repositories.com/repository/toneli-rt-retrieving-and-thinking.md) (0 ⭐) — This is the source code of the model RT (Retrieving and Thinking). For the full project, please check the file RTBC5CDR/3RT and RTNCBI/3RT, the implementation of GPT-NER and PromptNER is in the BC5CDR.zip and NCBI.zip. we refer to the source of code of GPT-NER and paper of GPT-NER in our project… - [cinnamon/kotaemon](https://awesome-repositories.com/repository/cinnamon-kotaemon.md) (25,139 ⭐) — Kotaemon is an orchestration framework designed for building modular, agentic workflows that integrate document processing, retrieval-augmented generation, and multi-step reasoning. It provides a comprehensive platform for developing document-based question answering systems, allowing users to chain language models, prompt templates, and external tools into complex, automated pipelines. The system distinguishes itself through a highly modular architecture that emphasizes component-based composition and schema-driven data exchange. It supports autonomous agents capable of decomposing complex queries through iterative processing and tool-calling, while its hybrid retrieval orchestration combines vector similarity and full-text search with re-ranking to improve the accuracy of retrieved context. The framework also features event-driven streaming, which delivers incremental results from long-running pipelines to the user interface in real-time. Beyond its core reasoning capabilities, the platform includes a suite of functional modules for the entire lifecycle of document-based applications. This includes multi-modal parsing for extracting text, tables, and visual elements from diverse file formats, as well as administrative tools for managing document collections, vector stores, and multi-user access. The system is designed to be interface-agnostic, allowing developers to wrap third-party libraries and external services into standardized, reusable processing units. The project provides a web-based user interface for interactive querying and configuration, and it supports deployment of private, isolated instances through predefined templates. - [bragai/brag-langchain](https://awesome-repositories.com/repository/bragai-brag-langchain.md) (4,028 ⭐) — bRAG-langchain is a framework for building retrieval augmented generation pipelines using LangChain to connect documents with language models. It functions as a vector store orchestrator that manages document indexing and retrieval strategies to improve context accuracy. The system implements an advanced retrieval pipeline featuring a semantic query router that directs natural language inputs to specific data sources or prompts. It includes a metadata filtering engine that translates natural language queries into structured schemas to narrow search results. The project covers hybrid search optimization through query expansion and reciprocal rank fusion. It supports multi-vector indexing and the storage of multiple document representations to increase retrieval precision. - [marwes/combine](https://awesome-repositories.com/repository/marwes-combine.md) (1,358 ⭐) — A parser combinator library for Rust - [promtengineer/localgpt](https://awesome-repositories.com/repository/promtengineer-localgpt.md) (22,215 ⭐) — localGPT is a private AI knowledge base and retrieval-augmented generation application. It provides a local document indexer, a hybrid search engine, and an inference interface to enable chatting with private documents and managing a self-hosted information repository without sending data to external servers. The system distinguishes itself through a dual-pass verification pipeline that ensures generated answers are grounded in retrieved sources, accompanied by explicit source attribution. It employs a hybrid retrieval approach combining semantic vector search with keyword matching and reranking, and utilizes recursive query decomposition to break complex requests into smaller parallel sub-queries. The platform covers broad capability areas including multi-format document processing, dynamic query routing, and semantic query caching. It also manages conversation history tracking and provides a RESTful API for integrating document retrieval and language model functionality into external applications. The project integrates with open-source models across different hardware accelerators and includes system health monitoring via structured logs and health endpoints. - [meilisearch/meilisearch](https://awesome-repositories.com/repository/meilisearch-meilisearch.md) (58,118 ⭐) — Meilisearch is a Rust-based search engine providing typo-tolerant full-text and vector-based semantic search with real-time conversational capabilities. - [cmavro/gnn-rag](https://awesome-repositories.com/repository/cmavro-gnn-rag.md) (0 ⭐) — This is the code for GNN-RAG: Graph Neural Retrieval for Large Language Modeling Reasoning. - [mongodb/mongo](https://awesome-repositories.com/repository/mongodb-mongo.md) (28,158 ⭐) — This project is a distributed, document-oriented database system designed to store information in flexible, hierarchical structures. It supports horizontal scaling through automated sharding and maintains high availability across global clusters using a multi-node replication protocol. By executing multi-document operations as atomic units, the system ensures data integrity and consistency across distributed environments. The platform distinguishes itself by integrating advanced vector-based indexing, which enables semantic similarity searches alongside traditional geospatial and lexical queries. It functions as an enterprise-grade data platform, incorporating granular access controls, encryption, and auditing mechanisms to meet the requirements of regulated production environments. These capabilities allow for the management of large-scale datasets while maintaining the flexibility of a schema-less storage model. The system provides a comprehensive suite of tools for database administration, including command-line utilities for infrastructure management, data migration, and performance monitoring. It supports integration with container orchestration platforms and offers standardized client libraries to facilitate connectivity across various programming languages and business intelligence tools. - [ra1028/swiftui-combine](https://awesome-repositories.com/repository/ra1028-swiftui-combine.md) (0 ⭐) — This is an example project of SwiftUI and Combine using GitHub GET /search/users API. - [dair-ai/prompt-engineering-guide](https://awesome-repositories.com/repository/dair-ai-prompt-engineering-guide.md) (75,678 ⭐) — This project is a comprehensive educational resource and technical guide focused on the development, optimization, and application of large language models. It provides a structured curriculum for mastering prompt engineering, ranging from foundational principles of instruction design to advanced techniques for improving model reasoning, accuracy, and reliability. The guide distinguishes itself by offering deep technical insights into agentic workflows and autonomous system design. It covers the implementation of multi-step reasoning chains, tool integration through function calling, and stateful memory management. Beyond basic prompting, it explores sophisticated frameworks that combine reasoning and acting, as well as methodologies for retrieval-augmented generation and the creation of synthetic datasets to address data scarcity in specialized domains. The documentation also addresses the broader engineering surface of AI development, including defensive strategies for application security and automated evaluation loops for model verification. These resources are designed to support developers in building complex, task-oriented AI systems that can interact with external APIs and maintain continuity across long-running processes. - [mahyarmirrashed/search-and-replace.nvim](https://awesome-repositories.com/repository/mahyarmirrashed-search-and-replace-nvim.md) (7 ⭐) — Search and replace functionality in Neovim. - [deepset-ai/haystack](https://awesome-repositories.com/repository/deepset-ai-haystack.md) (24,253 ⭐) — Haystack is an orchestration framework designed for building complex search and generative AI pipelines. It functions as an agentic workflow engine, enabling the construction of automated sequences that allow AI agents to perform multi-step reasoning and data analysis. The framework utilizes a modular, component-based architecture that connects processing steps into directed acyclic graphs. By employing a provider-agnostic integration layer, it decouples core logic from specific external AI services and vector databases, allowing for the flexible exchange of underlying technologies. This design supports the development of custom retrieval systems that provide context-aware answers from large datasets. Beyond text-based retrieval, the platform includes tools for multimodal data processing and indexing. It normalizes diverse media formats, including images and audio, into a unified representation to ensure consistent analysis across different types of content. The system also incorporates observability hooks to monitor state changes during the execution of complex workflows. - [activeloopai/deeplake](https://awesome-repositories.com/repository/activeloopai-deeplake.md) (9,175 ⭐) — DeepLake is AI data infrastructure consisting of a multimodal data lake, a hybrid search engine, and a serverless vector database. It provides a PostgreSQL-based AI data runtime that combines multimodal storage with streaming pipelines to load and shuffle datasets from cloud storage directly into deep learning training pipelines. The system utilizes lazy indexing to store and slice images, audio, and video without loading entire files into memory. It enables retrieval-augmented generation by persisting high-dimensional embeddings in a serverless vector store and implementing hybrid search that combines vector similarity with full-text keyword matching. The project covers a broad capability surface including structured metadata indexing for numeric and JSON fields, cloud-local data synchronization, and visualization tools for inspecting dataset annotations such as bounding boxes and masks. - [zilliztech/claude-context](https://awesome-repositories.com/repository/zilliztech-claude-context.md) (5,373 ⭐) — Claude-context is a retrieval-augmented generation pipeline and semantic code search tool. It functions as an LLM codebase indexer and RAG context provider, designed to index local directories and retrieve relevant code files to provide context for large language models. The system operates as a hybrid search engine that combines keyword matching with dense vector search. This allows for the retrieval of code snippets and logic using natural language queries based on meaning rather than exact text matches. The project covers codebase indexing and search index management, utilizing asynchronous processing and recursive directory traversal. It incorporates index filtering rules to manage which files are included and employs a combination of semantic encoding and local vector storage to maintain a searchable representation of the source code. - [mthcht/threathunting-keywords](https://awesome-repositories.com/repository/mthcht-threathunting-keywords.md) (0 ⭐) — 🎯 List of keywords for ThreatHunting sessions - [docling-project/docling](https://awesome-repositories.com/repository/docling-project-docling.md) (61,674 ⭐) — Docling is a modular framework designed for document parsing, layout analysis, and structured data extraction. It transforms unstructured files and web content into a unified, hierarchical data model that preserves the spatial and semantic relationships between text, tables, images, and layout elements. By normalizing diverse input formats into a consistent internal representation, the library enables uniform processing across various document types. The project distinguishes itself through a schema-driven approach that maps document regions to strongly-typed objects, ensuring data accuracy through validation against predefined templates. Its pipeline-based architecture supports pluggable processing backends, allowing for the dynamic integration of specialized engines for optical character recognition and complex visual layout analysis. Users can control parsing behavior and extraction parameters through declarative configuration files, facilitating integration into automated workflows and server-based architectures. The library provides both a programmatic interface and a command-line toolkit to support automated document processing and format conversion. It utilizes optional dependency management to allow for modular installation of specific features, such as media rendering or advanced processing capabilities, depending on the requirements of the application. - [s1n7ax/nvim-search-and-replace](https://awesome-repositories.com/repository/s1n7ax-nvim-search-and-replace.md) (70 ⭐) — Really simple plugin to search and replace multiple files - [alibaba/zvec](https://awesome-repositories.com/repository/alibaba-zvec.md) (5,198 ⭐) — zvec is an embedded vector database engine and indexing library designed for high-dimensional similarity search. It functions as a hybrid search engine and a retrieval-augmented generation knowledge base, allowing for the storage and retrieval of dense and sparse vectors. The system is distinguished by its hybrid retrieval pipeline, which fuses vector similarity, full-text keyword matching, and scalar metadata filtering into single query operations. It supports a plugin-based model integration system for registering custom embedding models and rerankers, as well as language bindings for native application integration. The project provides comprehensive data management through isolated local collection persistence, write-ahead logging, and dynamic schema mapping. Its search capabilities cover approximate nearest neighbor search at billion-scale, multimodal semantic search, and result reranking, while optimizing performance via memory-mapped I/O and vector index compression. The engine facilitates AI agent integration by exposing database interfaces and reusable operation skill sets to connect agents to structured data stores. - [yichuan-w/leann](https://awesome-repositories.com/repository/yichuan-w-leann.md) (11,985 ⭐) — LEANN is a framework for local retrieval augmented generation and vector indexing. It functions as a system for building local knowledge bases and source code search engines that combine large language models with retrieved private data to generate context-aware responses. The project distinguishes itself through a vision-model based document layout extractor for parsing complex PDF figures and diagrams, and a source code search engine that employs structure-aware chunking to preserve function and class boundaries. It also implements the Model Context Protocol to integrate real-time data sources into the retrieval pipeline. The system provides hybrid information retrieval combining semantic search, exact keyword matching, and boolean metadata filtering. It supports the indexing of diverse data sources, including web browsing history, communication logs, and technical documentation. - [jamwithai/production-agentic-rag-course](https://awesome-repositories.com/repository/jamwithai-production-agentic-rag-course.md) (6,972 ⭐) — This project is an educational course and technical blueprint for building production-ready retrieval-augmented generation systems. It provides a curriculum and implementation strategies for designing agentic workflows, containerized AI infrastructure, and retrieval pipelines using large language models. The materials focus on agentic design patterns, utilizing state-based decision nodes to rewrite queries and grade retrieved documents. It differentiates its approach by providing a deployment framework for managing databases, search engines, and API services through container orchestration. The project covers a broad range of architectural capabilities, including hybrid search with reciprocal rank fusion, OCR-based document parsing for PDF ingestion, and input-validation guardrails to prevent hallucinations. It also addresses operational requirements such as distributed request tracing, automatic query caching, and server-sent event streaming for real-time responses. - [noppefoxwolf/combinative](https://awesome-repositories.com/repository/noppefoxwolf-combinative.md) (106 ⭐) — UI event handling using Apple's combine framework. - [chroma-core/chroma](https://awesome-repositories.com/repository/chroma-core-chroma.md) (26,198 ⭐) — Chroma is a specialized vector database designed to index and retrieve high-dimensional data representations for semantic similarity search. It functions as a comprehensive platform for information retrieval, enabling the storage and management of unstructured documents alongside structured metadata. By mapping data into numerical representations, the system facilitates rapid similarity lookups across large datasets. The platform distinguishes itself through a hybrid search infrastructure that combines dense vector embeddings with sparse keyword and regular expression matching to balance semantic relevance with exact term precision. It supports multi-modal data, allowing for the indexing and querying of text, images, and audio within a unified interface. Furthermore, the system provides an agentic retrieval framework that enables autonomous agents to perform iterative search cycles and refine results for complex, multi-step queries. Beyond its core search capabilities, the platform includes specialized tools for codebase analysis, utilizing syntax-aware chunking to preserve logical structure for development tasks. It features a pluggable embedding pipeline that decouples vector generation from storage, allowing integration with diverse third-party machine learning models. The system also supports metadata-filtered query execution, ensuring precise retrieval by applying boolean constraints to document attributes. Operational support is provided through a programmatic interface for managing database instances in both self-hosted and cloud-based environments, including automated provisioning for scalable deployments. - [openai/openai-cookbook](https://awesome-repositories.com/repository/openai-openai-cookbook.md) (74,196 ⭐) — This project is a technical learning resource and developer knowledge base focused on the integration of large language models into software applications. It provides a structured collection of guides and code examples designed to teach developers how to implement intelligent features using proven patterns and best practices. The repository distinguishes itself through a library of functional demonstrations that cover complex topics such as retrieval-augmented generation, function calling, and prompt engineering workflows. These materials are organized into a modular structure, allowing for the rapid development and testing of prototypes and proof-of-concept applications before moving toward production-ready software. The content is delivered as a version-controlled knowledge base, utilizing markdown-based documentation and executable code blocks. These resources are designed to be copied directly into external development environments or cloud-based notebooks for hands-on experimentation. The entire collection is compiled into a static site to ensure consistent accessibility and navigation. - [run-llama/rags](https://awesome-repositories.com/repository/run-llama-rags.md) (6,540 ⭐) — Rags is an orchestration tool for building retrieval-augmented generation pipelines and managing conversational data interfaces. It serves as a system for creating these pipelines from local files and web pages using natural language instructions to query, retrieve, and summarize information from connected datasets. The project features a multimodal retrieval system that identifies and extracts information across different data types and modalities. It includes a vector search orchestrator to manage chunking strategies and search parameters, alongside a pipeline builder that translates conversational instructions into structured retrieval workflows. The platform provides capabilities for agent management, including session tracking to isolate conversation states and caches. System configuration is handled through a visual interface and natural language tuning of prompts and model parameters. - [weaviate/weaviate](https://awesome-repositories.com/repository/weaviate-weaviate.md) (15,620 ⭐) — Weaviate is an AI-native vector database designed to store and index high-dimensional vector embeddings alongside traditional data objects. It serves as a backend infrastructure for retrieval-augmented generation, enabling applications to ground language model responses in private, context-aware data. The platform distinguishes itself by combining vector similarity search with traditional keyword filtering through a hybrid storage architecture. It integrates directly with external machine learning models to automate the generation of embeddings and perform complex inference tasks during ingestion and query time. Beyond standard search, the database provides persistent state and memory for autonomous agents, allowing them to recall past interactions and maintain context across sessions. The system supports a range of operational requirements, from local development instances to distributed, sharded clusters capable of horizontal scaling. It utilizes a graph-oriented query language to traverse data relationships and execute multi-modal search operations, while background processing ensures consistent performance during index updates. - [opendataloader-project/opendataloader-pdf](https://awesome-repositories.com/repository/opendataloader-project-opendataloader-pdf.md) (25,769 ⭐) — This project is a PDF data extraction tool and document preprocessor designed to convert PDF files into structured formats such as Markdown, JSON, and HTML. It functions as an OCR document parser for scanned files, an accessibility automator for generating PDF/UA compliant metadata, and a loader for AI orchestration frameworks like LangChain. The software distinguishes itself through specialized handling of complex document elements, including the conversion of mathematical formulas into LaTeX and the generation of natural-language descriptions for charts and images. It utilizes recursive segmentation to determine correct reading orders in multi-column layouts and employs border-cluster detection to preserve the integrity of merged-cell tables. Broad capabilities include optical character recognition, semantic document chunking for retrieval optimization, and noise reduction to strip headers and footers. It also features security utilities for decrypting password-protected files, sanitizing sensitive private data, and filtering invisible content to prevent prompt injection. The project supports high-throughput batch processing and provides structure visualization tools to overlay detected semantic elements onto original documents for verification. - [mrrezaeiuoft/amg-rag](https://awesome-repositories.com/repository/mrrezaeiuoft-amg-rag.md) (0 ⭐) — AMG-RAG (Agentic Medical Graph-RAG) is a comprehensive framework that automates the construction and continuous updating of Medical Knowledge Graphs (MKGs), integrates reasoning, and retrieves current external evidence for medical Question Answering (QA). Our approach addresses the challenge of… - [raudaschl/rag-fusion](https://awesome-repositories.com/repository/raudaschl-rag-fusion.md) (940 ⭐) — RAG-Fusion: multi-query generation + Reciprocal Rank Fusion for better retrieval-augmented generation. Includes evaluation harness with NFCorpus/BEIR. - [aishwaryanr/awesome-generative-ai-guide](https://awesome-repositories.com/repository/aishwaryanr-awesome-generative-ai-guide.md) (24,755 ⭐) — This project is a community-driven knowledge repository and technical learning resource focused on the field of generative artificial intelligence. It serves as a centralized hub for developers and practitioners to access curated research, tutorials, and foundational concepts necessary for building and deploying modern artificial intelligence applications. The platform distinguishes itself through a collaborative, distributed contribution model that aggregates diverse learning materials into a structured, searchable knowledge base. It covers a wide range of specialized topics, including retrieval-augmented generation, large language model training, fine-tuning techniques, and agentic workflows. Beyond technical skill development, the repository functions as a professional development hub, offering interview preparation resources and guidance for those pursuing careers in the artificial intelligence industry. The content is organized through a hierarchical taxonomy, allowing users to navigate complex subjects such as system evaluation, multimodal models, and security tools. The repository provides access to comprehensive code notebooks and structured tutorials, all maintained as static documentation within a version control system to ensure accessibility and ease of discovery. - [weaviate/verba](https://awesome-repositories.com/repository/weaviate-verba.md) (7,715 ⭐) — Verba is a retrieval-augmented generation interface and chatbot that uses Weaviate to provide factual answers based on private datasets. It functions as a vector database knowledge base, combining a hybrid search engine with an orchestration interface to connect various large language model providers and embedding services. The system differentiates itself through a RAG pipeline manager for adjusting text chunking rules and retrieval settings, alongside a 3D vector space visualization tool for analyzing the spatial organization and clustering of high-dimensional embeddings. It employs a modular provider system that allows for swapping between different local and cloud text generation and embedding services. The platform covers multi-modal data ingestion, processing unstructured documents, audio transcriptions, web crawls, and version control repositories into a searchable knowledge base. Its retrieval capabilities combine semantic and keyword search to extract relevant context from vector stores, utilizing configurable text chunking to optimize retrieval precision. - [vectorize-io/vectorize-mcp-server](https://awesome-repositories.com/repository/vectorize-io-vectorize-mcp-server.md) (108 ⭐) — Official Vectorize MCP Server - [pathwaycom/pathway](https://awesome-repositories.com/repository/pathwaycom-pathway.md) (62,959 ⭐) — Pathway is a high-performance data processing framework designed for building unified batch and streaming pipelines. It functions as an orchestrator for complex data transformations, utilizing a differential dataflow engine to process updates incrementally. By treating static datasets and continuous event streams with identical logic, the platform ensures exactly-once processing semantics and consistent results across diverse data sources. The framework distinguishes itself through its specialized support for real-time artificial intelligence and retrieval-augmented generation. It features integrated vector-aware data ingestion, which automates the creation and maintenance of searchable document indexes that update instantly as new data arrives. Developers can connect language models directly into their pipelines, utilizing built-in capabilities for document chunking, embedding generation, and result reranking to maintain synchronized, context-aware information retrieval. Beyond its core processing capabilities, the platform provides a robust infrastructure for deploying data applications. It supports the transition from batch to streaming workflows by simply updating input connectors, while its containerized deployment model allows for scaling services across local and cloud environments. The system is designed to handle large-scale event-driven tasks, providing a consistent programming model for both analytics and automated content generation workflows. - [vectifyai/pageindex](https://awesome-repositories.com/repository/vectifyai-pageindex.md) (33,103 ⭐) — PageIndex is an agent-ready knowledge engine that processes documents into hierarchical tree structures to enable reasoning-based information retrieval. By organizing content into logical trees rather than relying on traditional vector database chunking, the platform preserves the original structure and flow of complex documents. It functions as a Model Context Protocol server, allowing external AI agents to connect to and query indexed knowledge bases through standardized communication protocols. The platform distinguishes itself by using vision-language models to process raw document images directly, capturing tables, lists, and layout information without requiring optical character recognition. This visual processing is paired with agentic reasoning, which allows the system to navigate document hierarchies based on semantic intent. To ensure transparency, the engine provides retrieval traceability, offering inline citations and step-by-step reasoning paths for every generated response. The system supports a comprehensive document lifecycle, including management of storage, conversational memory, and indexing status. Its retrieval capabilities combine logical tree navigation with hybrid search techniques and metadata filtering to identify precise information. The platform is secured through credential-based authentication for all protocol-based API interactions. - [facebookresearch/parlai](https://awesome-repositories.com/repository/facebookresearch-parlai.md) (10,625 ⭐) — ParlAI is a conversational AI research framework designed for training, evaluating, and sharing dialogue models using a unified interface for datasets and agents. It functions as a PyTorch-based training platform and a dialogue data collection system, providing a centralized model zoo for the distribution of versioned pretrained agents. The project distinguishes itself through a knowledge-grounded retrieval system that combines dense and sparse indexing to ground responses in external information. It also provides a comprehensive infrastructure for gathering human-AI interaction data via integrated crowdsourcing workflows, comparative evaluations, and human-model chat facilitation. The framework covers a broad range of capabilities, including multimodal dialogue development for visual content, safety classification for toxicity detection, and complex model evaluation through self-chat simulations. It supports diverse data management tasks such as disk-based dataset streaming, multi-task weighted sampling, and the implementation of custom teacher agents. The system is implemented in Python and utilizes a centralized registry to manage pretrained model checkpoints and metadata. - [embedchain/embedchain](https://awesome-repositories.com/repository/embedchain-embedchain.md) (58,769 ⭐) — Embedchain is an LLM memory management framework and RAG orchestration engine designed to provide AI agents with a persistent storage layer. It functions as a long-term memory pipeline that extracts facts from unstructured interactions and stores them as permanent knowledge base entries to retain user preferences and interaction history across sessions. The system employs a hybrid vector database interface that combines semantic embeddings with traditional keyword search. It utilizes an entity-linking knowledge graph to connect related information points and applies temporal ranking to distinguish current states from historical data. The framework covers multi-level state management across user, session, and agent tiers and implements multi-signal retrieval to surface relevant context. It includes a command line interface for administering stored data and interaction history. - [whyhow-ai/rule-based-retrieval](https://awesome-repositories.com/repository/whyhow-ai-rule-based-retrieval.md) (0 ⭐) — The Rule-based Retrieval package is a Python package that enables you to create and manage Retrieval Augmented Generation (RAG) applications with advanced filtering capabilities. It seamlessly integrates with OpenAI for text generation and Pinecone for efficient vector database management. - [eto-ai/lance](https://awesome-repositories.com/repository/eto-ai-lance.md) (6,671 ⭐) — Lance is a versioned columnar data format and storage engine designed as a multimodal AI lakehouse. It serves as a vector database storage engine and a cloud object store dataset manager, organizing images, video, audio, and embeddings into a unified format optimized for machine learning workflows. The project distinguishes itself by combining a columnar layout for structured data with a specialized blob store for large multimodal tensors. It implements a hybrid search engine that integrates vector similarity search, full-text search, and SQL analytics on a single dataset, supported by a storage model that allows high-performance random access to specific records without scanning entire files. The system covers broad capability areas including ACID data versioning with support for time travel and branching, metadata-driven schema evolution, and distributed data writing. It provides diverse indexing options such as inverted file indexes for vectors, BTree range indexing, and roaring-bitmap scalar indexing to accelerate data retrieval. The project persists datasets across S3-compatible storage and distributed filesystems using URI schemes. - [microsoft/ai-agents-for-beginners](https://awesome-repositories.com/repository/microsoft-ai-agents-for-beginners.md) (67,369 ⭐) — This project is a structured educational resource and technical guide for designing and implementing autonomous systems using large language models. It provides a comprehensive curriculum and code samples focused on agentic design patterns, autonomous development, and the creation of systems capable of planning and executing multi-step tasks. The resource details the implementation of agentic retrieval-augmented generation, where models autonomously plan and refine data searches. It covers a wide array of orchestrators and design patterns, including metacognitive reflection for self-correcting reasoning and human-in-the-loop oversight for critical action approval. The materials extend to the coordination of multi-agent systems through task decomposition and communication protocols, as well as the management of short-term session context and long-term persistent memory. Further technical coverage includes agent observability, secure deployment practices, and the integration of external tools and data sources. The project is delivered primarily as a collection of Jupyter Notebooks.