nmslib/hnswlib
Hnswlib
hnswlib is a header-only C++ library and vector indexing engine designed for high-dimensional approximate nearest neighbor search. It organizes large collections of embeddings into a searchable graph structure to enable rapid proximity queries and distance calculations.
The system utilizes Hierarchical Navigable Small World graphs to achieve fast vector similarity search. It distinguishes itself by allowing the definition of custom distance metrics and similarity functions to adapt calculations to specific data requirements.
The engine covers the full indexing lifecycle, including incremental index construction and the management of data points through additions and element removal. Query capabilities include both approximate and exact nearest neighbor search, supplemented by boolean search filtering to exclude candidates based on element labels.
The library supports index persistence via binary file serialization and provides configurations for parallel execution to distribute query and indexing tasks across multiple CPU cores.
Features
- Hierarchical Proximity Graphs - Utilizes Hierarchical Navigable Small World graphs to achieve logarithmic search complexity through coarse-to-fine navigation.
- Vector Indexing - Implements a high-performance indexing engine specifically designed for high-dimensional vector data and similarity search.
- Vector Similarity Search - Implements algorithms and data structures for performing similarity search on high-dimensional vector embeddings.
- Approximate Nearest Neighbor Search - Utilizes Hierarchical Navigable Small World graphs to trade absolute precision for significantly faster proximity lookups.
- Graph - Constructs hierarchical proximity graphs by configuring link density to balance memory usage and retrieval recall.
- Vector Indexing Engines - Acts as a high-performance engine for organizing vectors to enable rapid proximity queries and distance calculations.
- HNSW Indexes - Provides a high-performance implementation of Hierarchical Navigable Small World (HNSW) indexes for approximate nearest neighbor search.
- Incremental Vector Sync - Supports incremental index management by adding or removing elements without requiring a full rebuild.
- Nearest Neighbor Searches - Implements algorithms for finding the closest vectors in high-dimensional space based on distance calculations.
- Vector Distance Metrics - Provides the mathematical framework for implementing specialized similarity functions between high-dimensional vectors.
- Customizable Distance Metrics - Allows the definition of custom similarity functions to adapt distance calculations to specific data requirements.
- Greedy Graph Search - Implements greedy search traversal to efficiently navigate the graph layers toward the closest vector.
- Header-only Libraries - Distributed as a header-only C++ library to enable compiler inlining and simplify integration.
- Search Index Deletions - Allows for the removal of specific data points from the search index via unique identifiers.
- Incremental Indexing Engines - Supports adding and updating vectors within the index without necessitating a full rebuild of the graph structure.
- Index Persistence Utilities - Enables saving and loading the current index state to binary files for persistence across sessions.
- Search Index Management - Includes tools for managing the index lifecycle, including marking elements as deleted to reclaim memory.
- Dynamic Index Updating - Provides mechanisms to update index content and replace elements to ensure the dataset remains current.
- Vector Databases - Provides the core indexing and retrieval layer necessary for building high-performance vector databases.
- Multi-threaded Execution - Distributes independent search and indexing requests across multiple CPU cores to increase throughput.
- Template-Based Metrics - Uses C++ template polymorphism to inject custom similarity functions without the overhead of virtual function calls.
- AI & Machine Learning - Fast approximate nearest neighbors library
- Machine Learning - Fast approximate nearest neighbor search library.
Star history
How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
