7 个仓库
Decompression processes that operate within a fixed memory budget to prevent exhaustion during stream processing.
Distinct from High-Throughput Decompression: Focuses on strict memory bounds during decompression, whereas High-Throughput Decompression focuses on speed.
Explore 7 awesome GitHub repositories matching data & databases · Bounded-Memory Decompression. Refine with filters or upvote what's useful.
This project provides a lossless compression algorithm and a byte-oriented compression library designed for high-speed data reduction and maximum decompression speed. It functions as a stream-oriented compression engine, a software library for encoding and decoding data blocks, and a command-line tool for managing interoperable compressed frames. The system distinguishes itself through the use of predefined pattern dictionaries to improve compression ratios for small data sets and small packets. It supports multiple processing modes, including high-speed block compression for minimal latency
Implements decompression that uses a fixed amount of intermediate storage to prevent memory exhaustion during stream processing.
Thumbor is a dynamic image processing service and proxy server that resizes, crops, and filters images on demand via URL parameters. It functions as a URL-based image manipulator that generates specific image versions from source assets to optimize web delivery. The system includes a smart cropping engine that uses facial feature detection and computer vision to automatically center thumbnails on the most relevant visual content. To prevent unauthorized parameter tampering and malicious requests, it employs signature-based request validation and domain-based source whitelisting. Broad capabi
Prevents memory exhaustion attacks by enforcing pixel limits during the image decompression phase.
Ristretto is a high-performance in-memory cache and concurrent key-value store for Go applications. It provides a thread-safe memory store that manages strict memory bounds and employs probabilistic set filters to reduce lookup overhead. The system is distinguished by an admission-policy cache that utilizes frequency sketches and cost-based eviction to maximize hit ratios. It minimizes contention and improves throughput through the use of striped ring buffers and concurrent map sharding. The project covers a broad range of data management capabilities, including time-based expiration, item f
Manages strict memory bounds by automatically evicting low-value data to prevent out-of-memory errors.
DiskLruCache is a disk-backed key-value cache library for Android and Java environments that stores data on the filesystem within a configurable byte limit. It uses a least-recently-used (LRU) eviction policy to automatically remove old entries when the cache exceeds its allocated size, and maintains a journal file to reconstruct its in-memory state after a process restart. The cache provides atomic write operations that first write new data to a temporary file before atomically renaming it, preventing partial or corrupted reads during concurrent access. It returns frozen snapshots of cache e
Enforces a configurable disk byte limit with per-entry size tracking and automatic eviction.
Mozjpeg 是一个高性能 C 库,用于编码、解码和转码 JPEG 图像。它作为标准 JPEG 库的二进制兼容、直接替换方案,保持现有的函数签名,从而在无需更改应用逻辑的情况下提高压缩效率。 该库作为一个图像优化器,通过无损渐进式编码和系数优化来减小文件大小。它利用基于网格的量化和 SIMD 加速处理来优化视觉质量与文件大小之间的权衡。 其更广泛的功能包括全面的图像编码与解码、色彩空间转换和分量下采样。该架构设计为可重入且线程安全,允许并发处理多个图像。它还具有专门的元数据管理功能,用于通过标记拆分处理大型 ICC 颜色配置文件。
Optimizes memory usage by controlling the flow of downsampled data between the decompressor and post-processor.
webfunny_monitor is an open-source frontend monitoring and analytics platform that tracks page load performance, JavaScript errors, and user behavior. It serves as a comprehensive tool for measuring page load speed, API response times, client-side errors, and custom user events, with real-time dashboards that stream aggregated metrics via WebSocket for sub-second updates. The project distinguishes itself through its architectural approach: events are collected asynchronously via lightweight HTTP beacons and batched before transmission, while a memory-bounded local storage buffer ensures relia
Caches event data in a memory-bounded local storage buffer to ensure reliable delivery under intermittent network conditions.
这是一个用于 Go 的线程安全内存缓存库,当达到容量时会自动驱逐最近最少使用的项目。它提供了一个固定大小的内存缓存,旨在通过在有新数据进入时移除最旧的条目来保持恒定的内存占用。 该系统包括一个驱逐管理器,在清除条目时执行自定义回调,以促进资源清理。它还支持基于时间的过期,允许在指定的生存时间 (TTL) 后自动移除项目。 该库涵盖了通用的缓存功能,包括用于检查键是否存在的缓存检查、全量缓存清除,以及通过防止新数据突发挤占频繁访问的项目来优化命中率的策略。所有读写操作均已同步,以允许跨多个协程的并发访问。
Enforces a strict capacity limit on the number of stored elements to maintain a constant memory footprint.