3 repository-uri
Automatic triggering of index updates upon data ingestion to minimize latency.
Distinct from Index Versioning: Focuses on the automatic trigger mechanism upon ingestion, whereas versioning tracks revisions.
Explore 3 awesome GitHub repositories matching data & databases · Automatic Background Indexing. Refine with filters or upvote what's useful.
LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters
Triggers index updates automatically as new data is added to minimize latency impact.
TagStudio is a database-driven digital asset manager designed to organize and categorize files without altering their physical location on disk. It utilizes a non-destructive metadata store to attach tags and custom fields to files via a separate database rather than modifying original file headers. The project is distinguished by a hierarchical tagging system that supports parent-child relationships and custom color palettes for visual organization. It includes a multi-format media previewer that generates thumbnails and playback for images, videos, audio, documents, and archives, as well as
Automatically triggers index updates upon directory scanning to keep the local library synchronized with the filesystem.
zvec is an embedded vector database engine and indexing library designed for high-dimensional similarity search. It functions as a hybrid search engine and a retrieval-augmented generation knowledge base, allowing for the storage and retrieval of dense and sparse vectors. The system is distinguished by its hybrid retrieval pipeline, which fuses vector similarity, full-text keyword matching, and scalar metadata filtering into single query operations. It supports a plugin-based model integration system for registering custom embedding models and rerankers, as well as language bindings for nativ
Implements background buffering for vector index construction to maintain retrieval speed during updates.