Awesome GitHub RepositoriesVector Index Compression

ParlAI is a conversational AI research framework designed for training, evaluating, and sharing dialogue models using a unified interface for datasets and agents. It functions as a PyTorch-based training platform and a dialogue data collection system, providing a centralized model zoo for the distribution of versioned pretrained agents. The project distinguishes itself through a knowledge-grounded retrieval system that combines dense and sparse indexing to ground responses in external information. It also provides a comprehensive infrastructure for gathering human-AI interaction data via inte

zvec is an embedded vector database engine and indexing library designed for high-dimensional similarity search. It functions as a hybrid search engine and a retrieval-augmented generation knowledge base, allowing for the storage and retrieval of dense and sparse vectors. The system is distinguished by its hybrid retrieval pipeline, which fuses vector similarity, full-text keyword matching, and scalar metadata filtering into single query operations. It supports a plugin-based model integration system for registering custom embedding models and rerankers, as well as language bindings for nativ

RAGatouille 是一个检索框架和搜索引擎，旨在实现和训练后期交互 (late-interaction) 检索模型。它作为生成式 AI 流水线的模块化检索组件，专注于高性能文档排序以提高搜索准确性。 该项目提供了一个使用对和三元组训练及微调检索模型的工具包，具有用于领域适应的自动硬负样本挖掘功能。它实现了一种后期交互机制，通过利用压缩嵌入在检索速度和精度之间取得平衡。 该系统涵盖了文档索引和检索操作，利用基于磁盘的向量存储来处理超出可用系统内存的数据集。它进一步支持通过映射标记级嵌入来创建检索增强生成工作流，以保留细粒度的语义信息。

USearch is a high-performance vector similarity search engine and approximate nearest neighbor index designed for dense embeddings. It functions as a low-level vector database core and high-dimensional vector indexer, providing the primitives necessary to store and retrieve vectors across massive datasets. The engine distinguishes itself through hardware-level SIMD acceleration for distance kernels and a proximity-graph indexing system that enables fast retrieval across billions of vectors. It supports multi-precision vector quantization to balance memory usage and accuracy, and utilizes memo