OpenHFT/Chronicle-Queue
Chronicle Queue
Chronicle Queue is a high-performance data handling system featuring off-heap message queues, memory-mapped file stores, and replicated message stores. It provides a binary compatible memory layout that enables different programming languages to share data without serialization overhead.
The system utilizes a replicated message store to synchronize data across multiple nodes, ensuring high availability and instant failover. Its memory-mapped architecture supports deterministic replay from disk and low-latency data recording.
The project implements off-heap memory management and zero-allocation processing to eliminate garbage collection pauses and system jitter. It covers capability areas including inter-process communication, append-only sequential logging, and deterministic event sourcing.
Features
- Memory-Mapped File Access - Provides a persistence engine that maps disk files directly into process memory for low-latency recording and replay.
- Memory Layouts - Defines a strict binary memory layout that allows multiple programming languages to read shared data without translation.
- Memory-Mapped Storage - Uses memory-mapped files to map disk storage directly into the process address space for low-latency persistence.
- Shared Memory Data Exchange - Provides high-performance data transfer between processes using binary compatible shared memory buffers to eliminate serialization overhead.
- Off-Heap Queues - Ships a high-performance binary queue that stores data off-heap to eliminate Java garbage collection pauses.
- Append-Only Log Storage - Implements an immutable, chronological log structure to maximize disk throughput and ensure deterministic data replay.
- Low Latency Messaging - Implements low-latency messaging by utilizing off-heap memory to bypass garbage collection delays.
- Inter-Process Communication - Facilitates high-performance inter-process communication using shared memory primitives and binary compatible layouts.
- Off-Heap Memory Managers - Manages data outside the managed heap to eliminate garbage collection pauses and reduce system jitter.
- Zero-Allocation Architectures - Employs a zero-allocation architecture and off-heap memory to completely eliminate garbage collection pauses.
- Local Message Loggers - Maintains a local binary log of messages on disk to ensure recovery and deterministic replay after crashes.
- Event Sourcing Engines - Implements deterministic event sourcing by recording message sequences to disk for exact system state replay.
- Queue High Availability - Ensures high availability and failover by replicating queue data across multiple network nodes.
- High-Frequency Trading Infrastructure - Provides the high-performance, nanosecond-precision infrastructure required for reliable financial trading and tick data persistence.
- Shared Memory Management - Employs shared memory management with atomic operations and memory barriers for lock-free concurrency across processes.
- Zero-Allocation Buffer Encoders - Provides serialization mechanisms that write directly to buffers to avoid creating temporary objects during processing.
- Data Structures and Memory - Micro second messaging that stores everything to disk.
- Collection Libraries - Low-latency messaging library with disk-based persistence.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
