9 repositorios
Assesses storage and retrieval speed for key-value operations.
Distinct from Key-Value Stores: Distinct from general key-value stores: focuses on performance testing and benchmarking.
Explore 9 awesome GitHub repositories matching data & databases · Performance Benchmarking. Refine with filters or upvote what's useful.
RocksDB is a high-performance, embeddable persistent key-value library and storage engine based on Log-Structured Merge-trees. It is designed to provide durable storage for large-scale datasets, integrating directly into applications to manage data on flash and RAM-based hardware. The engine is distinguished by its focus on minimizing read and write amplification through multi-threaded compaction and custom memory allocators. It features specialized optimizations for flash storage, including support for zoned block devices, and provides the ability to extend store behavior via external plugin
Executes isolated operations with synthetic data to benchmark internal storage component performance.
NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t
Assesses the speed of data storage and retrieval operations within key-value buckets by simulating concurrent workloads.
FoundationDB is an ACID-compliant distributed transactional key-value store. It functions as a scalable database engine that ensures strict serializability and data consistency across a cluster of servers using a shared-nothing architecture. The system is distinguished by its multi-region replication capabilities, allowing data to be synchronized across different datacenters for high availability and disaster recovery. It utilizes optimistic concurrency control to manage distributed transactions and employs a majority-based coordination system to maintain cluster state. The platform provides
Executes automated workloads with specific data distributions to track performance regressions.
bbolt is an ACID-compliant embedded key-value store for Go applications. It persists all data in a single memory-mapped file on disk, organizing information using B+ trees to facilitate sorted key iteration and efficient range queries. The project distinguishes itself through a hierarchical data organization model, allowing buckets to be nested within other buckets to create a tree-like structure. It employs a single-writer, multi-reader locking mechanism and copy-on-write transactions to ensure serializable isolation and data integrity. The system includes comprehensive data management capa
Includes a utility to execute synthetic read and write tests to determine processing throughput.
LMCache is a distributed key-value cache manager and tiering system designed to accelerate large language model inference. It functions as a tiered storage layer that offloads tensors from GPU memory to CPU RAM, local disks, or remote object stores, enabling the reuse of cached prefixes across different inference sessions and serving engines. The system differentiates itself through a disaggregated prefill-decode model, which separates prompt processing from token generation by transferring caches between distributed compute nodes. It utilizes peer-to-peer orchestration to share and retrieve
Generates test key sets with configurable concurrency and offsets to measure cache performance and throughput.
Pebble is an embedded key-value storage engine written in Go, designed as a library that provides durable, write-optimized data persistence directly within applications. It organizes data using a log-structured merge-tree (LSM-tree) structure, where writes are first buffered in an in-memory skiplist memtable and persisted to a write-ahead log before being flushed to block-based SSTable files on disk. The engine supports atomic batch commits, configurable write synchronization, and automatic background compaction that merges and rewrites sorted runs to reclaim space and maintain read performanc
Includes benchmarks for mixed read-update-scan workloads using configurable key distributions and value sizes.
mergerfs is a FUSE-based union filesystem that pools multiple independent filesystems or directories into a single unified mount point. It acts as a proxy to underlying storage, forwarding file operations directly to the filesystem for near-native performance while merging directory listings and attribute changes. The project provides a live, read-write pooled view of storage that aggregates drives of any size without requiring reformatting or data redistribution, and it isolates individual drive failures so that the pool continues serving data from remaining filesystems. The filesystem offer
Provides a null read/write benchmarking mode that prevents data modification during performance testing.
seL4 is a formally verified microkernel whose C implementation is backed by machine-checked mathematical proofs of correctness, confidentiality, integrity, and availability. It enforces strict isolation between processes through hardware-enforced address space separation and a capability-based access control system, where each process holds explicit rights only to the resources it has been granted. The kernel exposes hardware resources through a minimal API of system calls that manage threads, address spaces, and inter-process communication, with synchronous IPC supporting sender-identifying b
Stops benchmark logging and returns the index of the last log entry.
libjxl is an open-source library for encoding and decoding images in the JPEG XL format. It provides the core codec functionality needed to compress source images into JPEG XL and decompress JPEG XL files back into common raster image representations. The library supports adjustable quality, distance, and effort settings during encoding, and handles conversion between different color spaces during encode or decode operations. It also enables reading and writing of embedded metadata such as EXIF, XMP, and color profiles. For performance, libjxl includes benchmarking tools for decode speed and
Includes benchmarking tools for measuring JPEG XL decode throughput across different threading configurations.