Awesome GitHub RepositoriesMulti-Model Vector Storage

SurrealDB is a multi-model database engine designed to store and query document, graph, relational, and vector data within a single ACID-compliant platform. It functions as an AI-native data store, integrating vector search, graph traversal, and machine learning model execution directly into its query layer. By providing a unified declarative query language, the platform eliminates the need for external middleware to synchronize data across different storage models. The platform distinguishes itself through its ability to manage agent memory and complex workflows natively. It allows developer

PostgresML is a machine learning database extension for PostgreSQL that integrates model training and inference directly into the database. It functions as an in-database AI platform and vector database, enabling the execution of large language models and natural language processing tasks on stored records without exporting data to external services. The system distinguishes itself by utilizing GPU acceleration to minimize latency during model predictions and employing a hybrid storage engine that maintains relational data alongside high-dimensional vectors. It allows for the building and fin

Cocoindex is an incremental data processing engine that builds and maintains live indexes for AI agents, with a core focus on codebase indexing and knowledge graph extraction. The engine uses a function-graph execution model where user-defined Python functions are composed into a directed acyclic graph, and it processes data incrementally so only changed source records or code paths are re-computed, avoiding full recomputation at any scale. It supports automatic schema inference from transformation pipeline type annotations and provides full data lineage tracing, tagging every output record wi

Chonkie 是一个专为检索增强生成 (RAG) 流水线设计的文本分块库。它充当语义文本分割器和 RAG 数据摄取流水线，将原始文本转换为嵌入片段，以便存储在向量数据库中。 该项目通过专门的分割策略脱颖而出，包括用于保留源代码逻辑边界的基于 AST 的代码分割器，以及使用嵌入模型根据语义确定边界的语义文本分割器。它还提供了一个向量数据库摄取器，用于自动化生成嵌入并将其导出到各种存储中。 该库涵盖了广泛的功能，包括通过 OCR 和 Markdown 提取进行文档解析，多种分割方法（如基于 Token 计数和分层分割），以及通过可重用流水线进行工作流编排。它支持多种向量存储集成，包括 Qdrant、Milvus、Weaviate 和 Elasticsearch，以及将数据导出为 JSON 和 Hugging Face 数据集。 用户可以通过命令行界面执行这些操作，或将系统部署为容器化的 API 服务。