spotify/annoy
Annoy
Annoy is a C++ library designed for approximate nearest neighbor search in high-dimensional vector spaces. It functions as a vector similarity search engine that constructs static, disk-based data structures to facilitate fast lookups. By mapping identifiers to vector data and persisting these structures to disk, the library enables efficient, memory-mapped access to large datasets.
The project distinguishes itself through the use of random projection trees and distance-metric-based partitioning, which organize data into hierarchical binary trees to balance search precision against computational overhead. Because the resulting indices are immutable and memory-mapped, they can be shared across multiple independent system processes without requiring the entire dataset to reside in active memory.
The library supports a broad range of indexing and retrieval capabilities, including the ability to scale to datasets that exceed available system memory. It provides cross-language integration through generated bindings and standard build system support, allowing the core search engine to be utilized across various programming environments.
Features
- Approximate Nearest Neighbor Search - Provides a high-performance library for approximate nearest neighbor search using random projection trees and memory-mapped indices.
- Memory-Mapped Indexing - Enables efficient access to large datasets by persisting search structures to disk and mapping them directly into process memory.
- Vector Search Indexes - Constructs memory-mapped vector search indexes that enable fast, memory-efficient similarity queries.
- Similarity Search Engines - Constructs static, disk-based data structures to perform high-dimensional nearest neighbor lookups with configurable precision.
- Vector Similarity Search - Performs similarity queries by applying distance metrics to find nearest neighbors in high-dimensional space.
- K-Nearest Neighbor Retrieval - Retrieves the closest items to a query vector using pre-built, memory-mapped data structures.
- Memory-Disk Layering - Persists search structures to disk, allowing large datasets to be shared across processes without requiring full memory residency.
- Trees - Organizes high-dimensional data into hierarchical random projection trees to enable rapid approximate nearest neighbor lookups.
- AI & Machine Learning - Approximate nearest neighbors library optimized for memory
- General Machine Learning - Library for approximate nearest neighbor search.
- Nearest Neighbor Search - Approximate nearest neighbor search with memory-mapped files.
- NLP - Approximate nearest neighbor search library.
- Recommender Systems - Approximate Nearest Neighbors in C++/Python optimized for memory usage.
- Vector Databases - Library for efficient approximate nearest neighbor search in vector spaces.
- Search and Indexing - C++ library for fast approximate nearest neighbor search.
- Search Engines - C++ library for searching points in space.
- Similarity Search - Memory-optimized approximate nearest neighbor search library.
- Vector Databases - Listed in the “Vector Databases” section of the Llm Course awesome list.
- Search Index Management - Enables saving and loading of static search indices to disk for distribution and reuse across system environments.
- Search and Indexing - Supports scaling indexing to disk to accommodate datasets that exceed available system memory.
- Static Content Indexing - Constructs immutable, pre-computed binary search structures that remain fixed after creation for fast, shared lookups.
- Native Library Integrations - Exposes high-performance search capabilities to multiple programming environments through native bindings and build system support.
- Language Bindings - Generates interface wrappers to allow core search functionality to be utilized from various programming languages.
- Distance Metrics - Provides configurable distance metrics to partition vector spaces for efficient similarity search.
