7 repositorios
Decompression processes that operate within a fixed memory budget to prevent exhaustion during stream processing.
Distinct from High-Throughput Decompression: Focuses on strict memory bounds during decompression, whereas High-Throughput Decompression focuses on speed.
Explore 7 awesome GitHub repositories matching data & databases · Bounded-Memory Decompression. Refine with filters or upvote what's useful.
This project provides a lossless compression algorithm and a byte-oriented compression library designed for high-speed data reduction and maximum decompression speed. It functions as a stream-oriented compression engine, a software library for encoding and decoding data blocks, and a command-line tool for managing interoperable compressed frames. The system distinguishes itself through the use of predefined pattern dictionaries to improve compression ratios for small data sets and small packets. It supports multiple processing modes, including high-speed block compression for minimal latency
Implements decompression that uses a fixed amount of intermediate storage to prevent memory exhaustion during stream processing.
Thumbor is a dynamic image processing service and proxy server that resizes, crops, and filters images on demand via URL parameters. It functions as a URL-based image manipulator that generates specific image versions from source assets to optimize web delivery. The system includes a smart cropping engine that uses facial feature detection and computer vision to automatically center thumbnails on the most relevant visual content. To prevent unauthorized parameter tampering and malicious requests, it employs signature-based request validation and domain-based source whitelisting. Broad capabi
Prevents memory exhaustion attacks by enforcing pixel limits during the image decompression phase.
Ristretto is a high-performance in-memory cache and concurrent key-value store for Go applications. It provides a thread-safe memory store that manages strict memory bounds and employs probabilistic set filters to reduce lookup overhead. The system is distinguished by an admission-policy cache that utilizes frequency sketches and cost-based eviction to maximize hit ratios. It minimizes contention and improves throughput through the use of striped ring buffers and concurrent map sharding. The project covers a broad range of data management capabilities, including time-based expiration, item f
Manages strict memory bounds by automatically evicting low-value data to prevent out-of-memory errors.
DiskLruCache is a disk-backed key-value cache library for Android and Java environments that stores data on the filesystem within a configurable byte limit. It uses a least-recently-used (LRU) eviction policy to automatically remove old entries when the cache exceeds its allocated size, and maintains a journal file to reconstruct its in-memory state after a process restart. The cache provides atomic write operations that first write new data to a temporary file before atomically renaming it, preventing partial or corrupted reads during concurrent access. It returns frozen snapshots of cache e
Enforces a configurable disk byte limit with per-entry size tracking and automatic eviction.
Mozjpeg es una librería en C de alto rendimiento para codificar, decodificar y transcodificar imágenes JPEG. Sirve como un reemplazo binario compatible y directo para las librerías JPEG estándar, manteniendo las firmas de funciones existentes para mejorar la eficiencia de la compresión sin requerir cambios en la lógica de la aplicación. La librería funciona como un optimizador de imágenes que reduce los tamaños de archivo mediante codificación progresiva sin pérdida y optimización de coeficientes. Utiliza cuantización basada en trellis y procesamiento acelerado por SIMD para optimizar el equilibrio entre la calidad visual y el tamaño del archivo. Sus capacidades más amplias incluyen codificación y decodificación de imágenes integral, conversión de espacio de color y submuestreo de componentes. La arquitectura está diseñada para ser reentrante y segura para hilos, permitiendo el procesamiento concurrente de múltiples imágenes. También cuenta con gestión de metadatos especializada para manejar perfiles de color ICC grandes mediante la división de marcadores.
Optimizes memory usage by controlling the flow of downsampled data between the decompressor and post-processor.
webfunny_monitor is an open-source frontend monitoring and analytics platform that tracks page load performance, JavaScript errors, and user behavior. It serves as a comprehensive tool for measuring page load speed, API response times, client-side errors, and custom user events, with real-time dashboards that stream aggregated metrics via WebSocket for sub-second updates. The project distinguishes itself through its architectural approach: events are collected asynchronously via lightweight HTTP beacons and batched before transmission, while a memory-bounded local storage buffer ensures relia
Caches event data in a memory-bounded local storage buffer to ensure reliable delivery under intermittent network conditions.
Esta es una biblioteca de caché en memoria segura para hilos (thread-safe) para Go que expulsa automáticamente los elementos menos utilizados recientemente (LRU) cuando se alcanza la capacidad. Proporciona una caché de memoria de tamaño fijo diseñada para mantener una huella de memoria constante eliminando las entradas más antiguas a medida que llegan nuevos datos. El sistema incluye un gestor de expulsión que ejecuta callbacks personalizados cuando se purgan las entradas para facilitar la limpieza de recursos. También admite la expiración basada en tiempo, permitiendo la eliminación automática de elementos después de una duración de tiempo de vida (TTL) especificada. La biblioteca cubre capacidades generales de caché, incluyendo inspección de caché para verificar la existencia de claves, purga completa de caché y estrategias para optimizar las tasas de acierto evitando que nuevas ráfagas de datos desplacen elementos accedidos frecuentemente. Todas las operaciones de lectura y escritura están sincronizadas para permitir el acceso concurrente a través de múltiples rutinas.
Enforces a strict capacity limit on the number of stored elements to maintain a constant memory footprint.