nmslib/hnswlib
Hnswlib
hnswlib 是一个仅包含头文件的 C++ 库和向量索引引擎，专为高维近似最近邻搜索而设计。它将大型嵌入集合组织成可搜索的图结构，以实现快速的邻近查询和距离计算。
该系统利用分层可导航小世界（HNSW）图来实现快速向量相似度搜索。其独特之处在于允许定义自定义距离度量和相似度函数，以适应特定数据需求。
该引擎涵盖了完整的索引生命周期，包括增量索引构建以及通过添加和元素移除来管理数据点。查询功能包括近似和精确最近邻搜索，并辅以布尔搜索过滤，以根据元素标签排除候选对象。
该库支持通过二进制文件序列化进行索引持久化，并提供并行执行配置，以将查询和索引任务分配到多个 CPU 核心上。
Features
- Hierarchical Proximity Graphs - Utilizes Hierarchical Navigable Small World graphs to achieve logarithmic search complexity through coarse-to-fine navigation.
- Vector Indexing - Implements a high-performance indexing engine specifically designed for high-dimensional vector data and similarity search.
- Vector Similarity Search - Implements algorithms and data structures for performing similarity search on high-dimensional vector embeddings.
- Approximate Nearest Neighbor Search - Utilizes Hierarchical Navigable Small World graphs to trade absolute precision for significantly faster proximity lookups.
- Graph - Constructs hierarchical proximity graphs by configuring link density to balance memory usage and retrieval recall.
- Vector Indexing Engines - Acts as a high-performance engine for organizing vectors to enable rapid proximity queries and distance calculations.
- HNSW Indexes - Provides a high-performance implementation of Hierarchical Navigable Small World (HNSW) indexes for approximate nearest neighbor search.
- Incremental Vector Sync - Supports incremental index management by adding or removing elements without requiring a full rebuild.
- Nearest Neighbor Searches - Implements algorithms for finding the closest vectors in high-dimensional space based on distance calculations.
- Vector Distance Metrics - Provides the mathematical framework for implementing specialized similarity functions between high-dimensional vectors.
- Customizable Distance Metrics - Allows the definition of custom similarity functions to adapt distance calculations to specific data requirements.
- Greedy Graph Search - Implements greedy search traversal to efficiently navigate the graph layers toward the closest vector.
- Header-only Libraries - Distributed as a header-only C++ library to enable compiler inlining and simplify integration.
- Search Index Deletions - Allows for the removal of specific data points from the search index via unique identifiers.
- Incremental Indexing Engines - Supports adding and updating vectors within the index without necessitating a full rebuild of the graph structure.
- Index Persistence Utilities - Enables saving and loading the current index state to binary files for persistence across sessions.
- Search Index Management - Includes tools for managing the index lifecycle, including marking elements as deleted to reclaim memory.
- Dynamic Index Updating - Provides mechanisms to update index content and replace elements to ensure the dataset remains current.
- Vector Databases - Provides the core indexing and retrieval layer necessary for building high-performance vector databases.
- Multi-threaded Execution - Distributes independent search and indexing requests across multiple CPU cores to increase throughput.
- Template-Based Metrics - Uses C++ template polymorphism to inject custom similarity functions without the overhead of virtual function calls.
- AI & Machine Learning - Fast approximate nearest neighbors library
- Machine Learning - Fast approximate nearest neighbor search library.
