jankotek/MapDB
MapDB
MapDB es un motor de base de datos embebido y una biblioteca de colecciones respaldada por disco que almacena colecciones de Java en disco o fuera del heap (off-heap). Funciona como un motor de procesamiento de datos local diseñado para manejar conjuntos de datos que exceden la RAM física disponible.
El proyecto utiliza almacenamiento de datos off-heap para eliminar la sobrecarga de la recolección de basura y emplea caché de desbordamiento en disco para equilibrar el uso de memoria y disco. Proporciona utilidades especializadas para filtrar y analizar grandes volúmenes de datos locales en una sola máquina.
El sistema garantiza la integridad de los datos mediante transacciones compatibles con ACID y control de concurrencia multiversión. Soporta mapas, conjuntos y colas persistentes utilizando indexación B-tree y un registro de transacciones secuencial para la durabilidad.
El comportamiento del sistema y las operaciones internas del motor pueden ser monitorizados a través de niveles de registro y formatos de salida configurables.
Features
- Disk-Backed Data Structures - Provides implementations of standard Java data structures that persist their state directly to the local filesystem.
- ACID Transactional Cores - Implements a core transactional engine that ensures atomic and durable data modifications via a sequential log.
- Persistent Mappings - Provides disk-backed key-value mappings and other collections that persist across application restarts.
- Disk-Persistent Data Structures - Maps collections like maps, sets, and queues directly to persistent storage to survive process restarts.
- Embedded Database Engines - Functions as a database engine integrated directly into application processes without requiring external server infrastructure.
- Multi-Version Concurrency Control - Utilizes multi-version concurrency control to allow simultaneous non-blocking read and write operations.
- ACID-Compliant - Ensures atomic, consistent, isolated, and durable operations with crash safety and rollback support.
- B-Tree - Uses B-Tree structures to organize stored data for efficient disk-based range queries and lookups.
- Off-Heap Memory Managers - Manages memory outside the JVM heap to eliminate garbage collection overhead for massive datasets.
- Hybrid Memory-Disk Caches - Employs a hybrid memory-disk cache that spills less active data to disk to optimize throughput.
- Disk Caching Systems - Offloads less frequently accessed data to persistent storage to manage memory constraints and maintain performance.
- Local Data Engines - Offers an engine designed for local data storage, indexing, and high-performance processing.
- Large Dataset Optimizations - Provides specialized utilities for high-performance filtering and analysis of massive local datasets.
- Databases and Storage - Embedded database engine backed by disk or memory.
- Embedded Databases - Concurrent collections backed by disk or off-heap memory.
