Awesome GitHub RepositoriesData File Compaction

Lance is a versioned columnar data format and storage engine designed as a multimodal AI lakehouse. It serves as a vector database storage engine and a cloud object store dataset manager, organizing images, video, audio, and embeddings into a unified format optimized for machine learning workflows. The project distinguishes itself by combining a columnar layout for structured data with a specialized blob store for large multimodal tensors. It implements a hybrid search engine that integrates vector similarity search, full-text search, and SQL analytics on a single dataset, supported by a stor

RoseDB 是一个嵌入式键值数据库和日志结构化存储引擎。它作为一个基于库的数据库，在应用进程内运行，提供本地数据持久化，而无需单独的数据库服务器。 该项目实现了 Bitcask 风格的存储，利用仅追加 (Append-only) 日志格式来最大化写入吞吐量并最小化寻道时间。它通过将多个读写操作分组为单个原子单元来确保数据一致性。 存储架构依赖于内存键索引进行常量时间查找，并使用基于段的文件管理来处理数据压缩和磁盘空间回收。

Indigo is a Go-based framework and toolkit designed for building, hosting, and scaling services within the AT Protocol ecosystem. It provides the foundational infrastructure for decentralized social networking, enabling developers to implement relay services, manage cryptographically signed user repositories, and handle identity resolution across federated environments. The project distinguishes itself through a robust architecture that decouples content hosting from discovery, allowing for independent moderation and algorithmic feed generation. It utilizes content-addressed storage and Merkl