5 个仓库
Automatic compression and decompression of specific database columns to reduce disk footprint.
Distinct from JSON Field Compression: The candidates focus on JSON fields or general archiving, not transparent database column-level compression.
Explore 5 awesome GitHub repositories matching data & databases · Transparent Column Compression. Refine with filters or upvote what's useful.
WCDB is a cross-platform storage layer and embedded database engine that serves as a framework for SQLite. It functions as an object relational mapper, linking application classes to database tables to enable data operations via objects rather than raw queries. The project is distinguished by an integrated encryption layer for securing data at rest and a full-text search engine that uses language-specific tokenizers for text lookups. It also features transparent field compression to reduce storage footprints and a connection-pooling model to coordinate simultaneous read and write operations a
Reduces storage footprints by automatically compressing and decompressing specific table fields.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Applies dictionary encoding or bit compression to reduce the storage footprint of column data.
CppGuide is a curated collection of educational resources and practical guides focused on C++ server development, Linux kernel internals, concurrent programming, network protocols, and security exploitation. It provides structured learning paths for backend developers, covering everything from interview preparation to building high-performance network servers and understanding operating system fundamentals. The guide distinguishes itself by offering in-depth, hands-on tutorials that walk through real-world implementations, including building a Redis-like server from scratch, designing custom
Encodes numeric fields using fewer bytes than native representation to reduce wire size.
YDB 是一个分布式 SQL 数据库和分析引擎,专为水平扩展和强一致性而设计。它作为一个多模型系统,通过提供可序列化 ACID 事务的分布式架构支持事务和分析工作负载。 该系统以其广泛的协议兼容性而著称,实现了用于标准 SQL 驱动程序的 PostgreSQL 有线协议和用于消息传递与流处理的 Kafka 协议。它进一步作为向量数据库,支持用于语义搜索和嵌入的向量索引以及近似最近邻搜索。 该平台使用具有行式和列式格式的混合存储模型管理数据,利用向量化查询执行进行 PB 级分析。其操作范围包括变更数据捕获流、精确一次(exactly-once)持久队列和多区域高可用性。 部署和生命周期管理通过 Kubernetes Operator 和基础设施即代码(IaC)配置提供支持。
Organizes columns into groups to apply specialized compression modes and optimize physical hardware allocation.
RoaringBitmap is a compressed integer set library designed for memory-efficient storage and high-speed in-memory analytics. It provides a professional implementation of the Roaring Bitmap format to maintain compact representations of integer collections. The project features a standardized binary serialization format that enables the exchange of compressed bitmap data across different programming languages and hardware architectures. This allows for consistent data portability and cross-language data exchange. The library performs rapid logical set operations, including intersections, unions
Dynamically switches between bitsets, sorted arrays, and run-length encoding based on data density.