OpenHFT/Chronicle-Queue
Chronicle Queue
Chronicle Queue is a high-performance data handling system featuring off-heap message queues, memory-mapped file stores, and replicated message stores. It provides a binary compatible memory layout that enables different programming languages to share data without serialization overhead.
The system utilizes a replicated message store to synchronize data across multiple nodes, ensuring high availability and instant failover. Its memory-mapped architecture supports deterministic replay from disk and low-latency data recording.
The project implements off-heap memory management and zero-allocation processing to eliminate garbage collection pauses and system jitter. It covers capability areas including inter-process communication, append-only sequential logging, and deterministic event sourcing.
Features
- Memory-Mapped File Access - Provides a persistence engine that maps disk files directly into process memory for low-latency recording and replay.
- Memory Layouts - Defines a strict binary memory layout that allows multiple programming languages to read shared data without translation.
- Memory-Mapped Storage - Uses memory-mapped files to map disk storage directly into the process address space for low-latency persistence.
- Shared Memory Data Exchange - Provides high-performance data transfer between processes using binary compatible shared memory buffers to eliminate serialization overhead.
- Off-Heap Queues - Ships a high-performance binary queue that stores data off-heap to eliminate Java garbage collection pauses.
- Append-Only Log Storage - Implements an immutable, chronological log structure to maximize disk throughput and ensure deterministic data replay.
- Low Latency Messaging - Implements low-latency messaging by utilizing off-heap memory to bypass garbage collection delays.
- Inter-Process Communication - Facilitates high-performance inter-process communication using shared memory primitives and binary compatible layouts.
- Off-Heap Memory Managers - Manages data outside the managed heap to eliminate garbage collection pauses and reduce system jitter.
- Zero-Allocation Architectures - Employs a zero-allocation architecture and off-heap memory to completely eliminate garbage collection pauses.
- Local Message Loggers - Maintains a local binary log of messages on disk to ensure recovery and deterministic replay after crashes.
- Event Sourcing Engines - Implements deterministic event sourcing by recording message sequences to disk for exact system state replay.
- Queue High Availability - Ensures high availability and failover by replicating queue data across multiple network nodes.
- High-Frequency Trading Infrastructure - Provides the high-performance, nanosecond-precision infrastructure required for reliable financial trading and tick data persistence.
- Shared Memory Management - Employs shared memory management with atomic operations and memory barriers for lock-free concurrency across processes.
- Zero-Allocation Buffer Encoders - Provides serialization mechanisms that write directly to buffers to avoid creating temporary objects during processing.
- Data Structures and Memory - Micro second messaging that stores everything to disk.
- Collection Libraries - Low-latency messaging library with disk-based persistence.
