Projects sharing features with Annoy

SPTAG is a vector approximate nearest neighbor search library and distributed vector search engine. It provides a large-scale vector index designed to organize and retrieve similar vectors from massive datasets using high-performance similarity search and proximity queries. The system functions as a dynamic vector index manager, supporting incremental updates, insertions, and deletions of vectors without requiring a full index rebuild. It scales search operations across multiple machines to handle large-scale datasets and high volumes of online requests through distributed search request hand

Vector similarity search extension for PostgreSQL.

Hora is a vector similarity search library written in Rust designed for efficient approximate nearest neighbor retrieval in high-dimensional data spaces. It functions as a compiled search engine that organizes vector spaces using hierarchical graphs, trees, and product quantization, while computing proximity through various mathematical distance formulas including Euclidean, Manhattan, cosine, and dot product calculations. The library supports high-performance execution through hardware-accelerated instruction sets and concurrent thread dispatch across multiple hardware processors. To accommo

LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters

TensorFlow Similarity is a Python framework designed for training neural networks to learn high-dimensional vector representations and perform similarity-based retrieval. It provides a comprehensive toolkit for metric learning, enabling the development of systems that group similar items together in vector space and identify them through distance-based comparisons. The library distinguishes itself by integrating specialized training techniques, such as contrastive and triplet-based learning, with robust data management tools that ensure stable model convergence. It supports self-supervised re

zvec is an embedded vector database engine and indexing library designed for high-dimensional similarity search. It functions as a hybrid search engine and a retrieval-augmented generation knowledge base, allowing for the storage and retrieval of dense and sparse vectors. The system is distinguished by its hybrid retrieval pipeline, which fuses vector similarity, full-text keyword matching, and scalar metadata filtering into single query operations. It supports a plugin-based model integration system for registering custom embedding models and rerankers, as well as language bindings for nativ

hnswlib is a header-only C++ library and vector indexing engine designed for high-dimensional approximate nearest neighbor search. It organizes large collections of embeddings into a searchable graph structure to enable rapid proximity queries and distance calculations. The system utilizes Hierarchical Navigable Small World graphs to achieve fast vector similarity search. It distinguishes itself by allowing the definition of custom distance metrics and similarity functions to adapt calculations to specific data requirements. The engine covers the full indexing lifecycle, including incrementa

This project is a full text search engine and enterprise search infrastructure designed for indexing and retrieving large sets of documents. It provides a comprehensive framework for information discovery using ranked results and linguistic analysis. The system integrates high-dimensional vector similarity search for semantic retrieval alongside traditional full-text capabilities. It distinguishes itself through support for geospatial data retrieval, multilingual text processing, and a search suggestion workflow that includes typo-tolerant query completion and spellchecking. The platform cov

Milvus is a specialized vector database engine designed for the indexing, management, and high-speed similarity retrieval of high-dimensional vector embeddings. It functions as a similarity search engine capable of identifying nearest neighbors within large-scale vector spaces, supporting the storage and retrieval of billions of data points while maintaining consistent performance. The system utilizes a distributed architecture that decouples storage, query, and coordination into independent services, allowing for horizontal scaling across clusters. It employs a global indexing mechanism that

Gensim is a natural language processing toolkit designed for large-scale text analysis and the training of semantic vector embeddings. It provides a framework for identifying latent thematic structures within document collections and calculating semantic similarity between text segments using unsupervised statistical algorithms. The project is distinguished by its ability to handle datasets that exceed available system memory through incremental corpus streaming, which processes documents one at a time from disk. It utilizes sparse vector representations and dictionary-based token mapping to

Mastra is an orchestration framework designed for building, deploying, and managing autonomous AI agents and multi-agent systems. It provides a comprehensive suite of primitives for creating resilient AI applications, including durable workflow orchestration, event-driven agent loops, and semantic memory management. By integrating these core components, the platform enables developers to build complex, multi-step processes that can reason about goals and execute tasks without manual intervention. The framework distinguishes itself through its focus on observability and secure, isolated execut

Bleve is a search indexing engine library written in Go, designed to provide full-text search and document retrieval capabilities for embedded application data. It functions as a framework for indexing structured or unstructured information, allowing developers to build searchable collections that support complex query logic and data analysis. The engine distinguishes itself through a pluggable analysis pipeline that normalizes text before indexing, alongside support for vector similarity search to identify semantically related content. It utilizes finite-state transducer automata for efficie

Deepface is a comprehensive deep learning library for facial recognition and demographic analysis. It provides a modular pipeline that handles the entire lifecycle of facial processing, including detection, geometric alignment, and the transformation of facial images into high-dimensional numerical vector embeddings for identity verification and similarity comparison. The library distinguishes itself through a model ensemble approach, which combines predictions from multiple pre-trained neural networks to improve classification accuracy and reduce bias. It also integrates advanced security fe

This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d

XGBoost is a distributed machine learning library for implementing scalable gradient boosting decision trees used for regression, classification, and ranking. It functions as a predictive model framework and a cross-language toolkit, providing a core implementation with native bindings for Python, R, Java, Scala, and C++. The system is designed as a GPU-accelerated library that utilizes CUDA and NCCL to speed up the training of decision tree ensembles. It operates as a distributed framework capable of scaling training and prediction across multi-node clusters and GPU environments to process m

Vald is a distributed, cloud-native search engine designed for high-dimensional vector data. It functions as an approximate nearest neighbor search platform, enabling the identification of similar data points across massive datasets through horizontal scaling and distributed indexing. The system is built for container orchestration environments, utilizing custom resource controllers to automate cluster lifecycle management and infrastructure state. It employs graph-based indexing to perform rapid similarity lookups and supports zero-downtime operations by decoupling index construction from qu

Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

Cozo is a logic-based database engine that functions as a relational data store, an embedded graph database, and a temporal vector database. It utilizes a Datalog-inspired query language to execute relational, recursive, and graph queries. The system distinguishes itself through specialized indexing for high-dimensional vector similarity searches and near-duplicate detection using locality sensitive hashing. It also provides built-in temporal versioning, allowing for historical state retrieval and time-travel queries to access data as it existed at specific points in time. Its broader capabi

X-algorithm is a modular recommendation engine framework designed to orchestrate personalized content feeds. It functions as a machine learning ranking system that manages the end-to-end lifecycle of content delivery, from initial candidate retrieval to final display ordering. The system distinguishes itself through a multi-stage pipeline that integrates vector-based similarity search with transformer-based engagement prediction. By mapping user history and content features into high-dimensional embeddings, it performs rapid approximate nearest neighbor searches to identify relevant items. Th

Vespa is a distributed search engine, vector database, and machine learning ranking engine. It serves as an AI search platform designed to handle large-scale document indexing and complex query processing across a cluster of nodes, combining keyword retrieval with high-dimensional embedding storage for semantic similarity search. The platform distinguishes itself by integrating machine learning models directly into the search pipeline to perform real-time inference and ranking. It converts these models into ranking expressions to score and order results based on relevance, while providing a s

This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable i

RedisInsight is a graphical user interface and management tool for browsing, analyzing, and administering Redis databases. It provides a visual environment for exploring key-value data structures, managing database instances, and performing data analysis across different operating systems and deployments. The tool distinguishes itself by providing dedicated visual managers for complex operations, including a vector database manager for configuring embeddings and similarity searches, a query workbench for executing raw commands and Lua scripts, and a performance monitoring dashboard for tracki

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

This project is a C++ vector similarity engine and implementation of the Hierarchical Navigable Small World algorithm. It provides a header-only library for performing approximate nearest neighbor searches in high-dimensional spaces, alongside Python bindings that expose these indexing and search capabilities to data science environments. The engine enables real-time embedding retrieval and high-dimensional similarity search using a multi-layered graph structure to balance search speed and accuracy. It supports custom distance metrics to calculate similarity between vectors in various mathema

sqlite-vec is a C-based vector library and SQLite extension that adds virtual tables for storing and querying high-dimensional embeddings. It functions as a database plugin for performing nearest neighbor searches using distance metrics such as L2, cosine, and Hamming distance. The project provides a portable embedding store that supports deployment across Android, iOS, desktop environments, and web browsers via WebAssembly. It distinguishes itself by converting numerical arrays into compact binary formats and utilizing quantization to reduce the memory footprint and storage size of vector in

Lance is a versioned columnar data format and storage engine designed as a multimodal AI lakehouse. It serves as a vector database storage engine and a cloud object store dataset manager, organizing images, video, audio, and embeddings into a unified format optimized for machine learning workflows. The project distinguishes itself by combining a columnar layout for structured data with a specialized blob store for large multimodal tensors. It implements a hybrid search engine that integrates vector similarity search, full-text search, and SQL analytics on a single dataset, supported by a stor

RavenDB is a multi-model NoSQL document database designed for high-performance, ACID-compliant data storage. It persists structured information as schema-flexible JSON documents and utilizes a unit-of-work session pattern to track entity changes and batch modifications into atomic transactions. The platform is built on a distributed architecture that supports horizontal scaling through sharding and ensures high availability via multi-node, master-to-master cluster replication. The database distinguishes itself through a self-optimizing query engine that automatically creates and maintains ind