5 Repos
Automatic compression and decompression of specific database columns to reduce disk footprint.
Distinct from JSON Field Compression: The candidates focus on JSON fields or general archiving, not transparent database column-level compression.
Explore 5 awesome GitHub repositories matching data & databases · Transparent Column Compression. Refine with filters or upvote what's useful.
WCDB is a cross-platform storage layer and embedded database engine that serves as a framework for SQLite. It functions as an object relational mapper, linking application classes to database tables to enable data operations via objects rather than raw queries. The project is distinguished by an integrated encryption layer for securing data at rest and a full-text search engine that uses language-specific tokenizers for text lookups. It also features transparent field compression to reduce storage footprints and a connection-pooling model to coordinate simultaneous read and write operations a
Reduces storage footprints by automatically compressing and decompressing specific table fields.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Applies dictionary encoding or bit compression to reduce the storage footprint of column data.
CppGuide is a curated collection of educational resources and practical guides focused on C++ server development, Linux kernel internals, concurrent programming, network protocols, and security exploitation. It provides structured learning paths for backend developers, covering everything from interview preparation to building high-performance network servers and understanding operating system fundamentals. The guide distinguishes itself by offering in-depth, hands-on tutorials that walk through real-world implementations, including building a Redis-like server from scratch, designing custom
Encodes numeric fields using fewer bytes than native representation to reduce wire size.
YDB ist eine verteilte SQL-Datenbank und analytische Engine, die auf horizontale Skalierbarkeit und starke Konsistenz ausgelegt ist. Sie fungiert als Multi-Model-System, das transaktionale und analytische Workloads durch eine verteilte Architektur unterstützt, die serialisierbare ACID-Transaktionen bietet. Das System zeichnet sich durch seine breite Protokollkompatibilität aus und implementiert das PostgreSQL-Wire-Protokoll für Standard-SQL-Treiber sowie das Kafka-Protokoll für Messaging und Streaming. Es dient zudem als Vektordatenbank und unterstützt Vektorindizes sowie die Suche nach ungefähren nächsten Nachbarn (Approximate Nearest Neighbor) für semantische Suche und Embeddings. Die Plattform verwaltet Daten unter Verwendung eines hybriden Speichermodells mit zeilen- und spaltenorientierten Formaten und nutzt vektorisierte Abfrageausführung für Analysen im Petabyte-Maßstab. Ihr operativer Bereich umfasst Change-Data-Capture-Streaming, persistente Queues mit Exactly-Once-Garantie und Multi-Zone-Hochverfügbarkeit. Deployment und Lebenszyklusmanagement werden durch einen Kubernetes-Operator und Infrastructure-as-Code-Provisioning unterstützt.
Organizes columns into groups to apply specialized compression modes and optimize physical hardware allocation.
RoaringBitmap is a compressed integer set library designed for memory-efficient storage and high-speed in-memory analytics. It provides a professional implementation of the Roaring Bitmap format to maintain compact representations of integer collections. The project features a standardized binary serialization format that enables the exchange of compressed bitmap data across different programming languages and hardware architectures. This allows for consistent data portability and cross-language data exchange. The library performs rapid logical set operations, including intersections, unions
Dynamically switches between bitsets, sorted arrays, and run-length encoding based on data density.