awesome-repositories.com
Blog
awesome-repositories.com

Descoperă cele mai bune repository-uri open source cu căutare AI.

ExploreazăCăutări recomandateAlternative open-sourceSoftware self-hostedBlogHartă site
ProiectDespreCum realizăm clasamentulPresăServer MCP
LegalConfidențialitateTermeni
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

27 repository-uri

Awesome GitHub RepositoriesDisk Caching Systems

Mechanisms for offloading data to persistent storage to manage memory constraints.

Distinguishing note: Focuses on disk-backed storage for large-scale analysis tasks.

Explore 27 awesome GitHub repositories matching data & databases · Disk Caching Systems. Refine with filters or upvote what's useful.

Awesome Disk Caching Systems GitHub Repositories

Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
  • skylot/jadxAvatar skylot

    skylot/jadx

    49,088Vezi pe GitHub↗

    Jadx is a comprehensive Java decompilation suite designed to transform compiled binary application files into readable source code. It functions as a static analysis workbench, providing a graphical interface for navigating, searching, and inspecting the internal logic of complex software packages. By utilizing a bytecode-to-Java pipeline, the project reconstructs high-level logical structures from low-level binary instructions, making it a primary tool for Android application reverse engineering. The project distinguishes itself through a sophisticated control flow reconstruction engine and

    Offloads large intermediate data structures to the file system to minimize memory pressure during analysis.

    Javaandroiddecompilerdex
    Vezi pe GitHub↗49,088
  • alibaba/easyexcelAvatar alibaba

    alibaba/easyexcel

    33,703Vezi pe GitHub↗

    EasyExcel is a Java processing library designed for reading and writing XLS, XLSX, and CSV files. It functions as a memory-efficient spreadsheet parser, an object-relational mapper that binds spreadsheet columns to Java class fields, and a stream-based exporter for handling high-volume data. The library distinguishes itself through a streaming model that processes large files row-by-row via listeners to prevent heap memory overflow. It also operates as a template engine, allowing the population of predefined spreadsheet files with dynamic data while preserving original layouts and styles. Br

    Utilizes disk-backed caching to offload cell data to temporary files, preventing memory overflow with massive datasets.

    Javaexceljavajxl
    Vezi pe GitHub↗33,703
  • blankj/androidutilcodeAvatar Blankj

    Blankj/AndroidUtilCode

    33,657Vezi pe GitHub↗

    AndroidUtilCode is an Android utility library and system API wrapper designed to reduce development boilerplate. It provides a collection of helper classes for common tasks including system settings management, file I/O, and hardware access. The project distinguishes itself through a comprehensive toolset for device management and UI assistance. It includes specialized capabilities for monitoring battery status, managing system volume and brightness, and implementing UI helpers to prevent duplicate click events. It also provides a dedicated system for coordinate conversion between different m

    Saves bitmaps and JSON data to the local filesystem to reduce network requests and computation.

    Javaandroidandroidxapp
    Vezi pe GitHub↗33,657
  • square/picassoAvatar square

    square/picasso

    18,820Vezi pe GitHub↗

    Picasso is an Android image loading library designed for downloading, caching, and displaying images in applications. It provides an automated system for memory and disk management to reduce network requests and improve load times. The library includes a bitmap transformation tool for resizing, cropping, and modifying assets before they are rendered. It also functions as a bridge for Jetpack Compose, converting image requests into painter objects for use within declarative Android user interfaces. The system manages image retrieval from multiple sources, including web URLs, local files, and

    Maintains images in tiered memory and disk caches for fast retrieval and reduced network usage.

    Kotlin
    Vezi pe GitHub↗18,820
  • prestodb/prestoAvatar prestodb

    prestodb/presto

    16,711Vezi pe GitHub↗

    Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

    Moves intermediate results to local storage when memory usage exceeds defined limits to allow completion of large-scale operations.

    Javabig-datadatahadoop
    Vezi pe GitHub↗16,711
  • imagemagick/imagemagickAvatar ImageMagick

    ImageMagick/ImageMagick

    15,742Vezi pe GitHub↗

    ImageMagick is a comprehensive software suite for the creation, editing, composition, and conversion of digital images. It functions as both a command-line utility for batch processing and automation, and as a programming library that allows developers to integrate advanced image manipulation capabilities into external applications. The project is distinguished by its modular architecture, which supports hundreds of image formats through a pluggable coder system and external delegate libraries. It is designed for high-performance environments, utilizing memory-mapped pixel caching, stream-ori

    Forces image data to flush to disk during processing to ensure file integrity and detect storage issues.

    Ccommand-line-image-tooldigital-image-editingimage-conversion
    Vezi pe GitHub↗15,742
  • dgraph-io/badgerAvatar dgraph-io

    dgraph-io/badger

    15,666Vezi pe GitHub↗

    Badger is an embeddable key-value store written in Go that provides persistent data storage for byte keys and values. It is a persistent database that utilizes a tiered LSM tree storage model to optimize disk storage and retrieval efficiency. The system features an ACID transaction engine that ensures data integrity through serializable snapshot isolation and multi-version concurrency control. It also provides an encrypted key-value store with data-at-rest encryption and a managed encrypted key registry to secure stored information. The engine covers a broad set of capabilities including hig

    Synchronizes directory entries to disk to ensure consistency of file operations after a crash.

    Godatabasedocument-databasego
    Vezi pe GitHub↗15,666
  • antirez/ds4Avatar antirez

    antirez/ds4

    15,143Vezi pe GitHub↗

    ds4 is a local inference engine for DeepSeek models that includes a distributed runtime for splitting transformer layers across networked computers. It functions as a reasoning controller with a local weight streamer and an API server that streams chat completions via industry standard endpoints. The system employs a memory management model that loads model experts from disk on demand to execute models that exceed available system RAM. It provides controls for reasoning effort and model behavior steering, allowing the modification of response characteristics through activation directions. Th

    Stores processed prompt prefixes on disk to allow session resumption without recalculating input tokens.

    C
    Vezi pe GitHub↗15,143
  • boltdb/boltAvatar boltdb

    boltdb/bolt

    14,642Vezi pe GitHub↗

    Bolt is a single-file embedded key-value store for Go applications. It is an ACID transactional database that organizes data in B+trees on disk to provide efficient sorted key retrieval and range scans. The system uses a memory-mapped model to map the database file directly into the process address space for fast random-access reads. The project distinguishes itself through a multi-version concurrency control architecture that allows multiple simultaneous readers to access a consistent snapshot of data without blocking a writer. It employs a single-writer multi-reader locking model and uses a

    Writes modified memory data to physical storage to ensure durability and prevent data loss during crashes.

    Go
    Vezi pe GitHub↗14,642
  • apple/turicreateAvatar apple

    apple/turicreate

    11,171Vezi pe GitHub↗

    This project is an automated machine learning framework and toolkit designed for training and tuning custom models for classification, regression, and recommendations. It functions as a multimodal machine learning toolkit capable of processing and training models using a combination of text, image, audio, and sensor data. The framework distinguishes itself as a multimodal data processor that can handle and visualize large datasets on a single machine using column-oriented disk storage. It includes a core machine learning model generator that converts trained models into formats compatible wit

    Manages large-scale data by swapping chunks between disk and memory to prevent system crashes.

    C++
    Vezi pe GitHub↗11,171
  • dicedb/diceAvatar DiceDB

    DiceDB/dice

    10,760Vezi pe GitHub↗

    Dice is a low-latency key-value store based on Valkey that functions as a hierarchical storage engine. It combines a high-speed memory store with a tiered cache system to manage data across multiple storage levels. The system extends memory capacity by implementing a disk-backed cache that automatically persists evicted items to disk and restores them during cache misses. This storage tiering balances retrieval speed and operational cost by distributing data between fast memory and slower disk storage. The engine includes a query subscription service that monitors specific data patterns and

    Implements a system to restore evicted data from disk into memory during cache misses.

    C
    Vezi pe GitHub↗10,760
  • etcd-io/bboltAvatar etcd-io

    etcd-io/bbolt

    9,573Vezi pe GitHub↗

    bbolt is an ACID-compliant embedded key-value store for Go applications. It persists all data in a single memory-mapped file on disk, organizing information using B+ trees to facilitate sorted key iteration and efficient range queries. The project distinguishes itself through a hierarchical data organization model, allowing buckets to be nested within other buckets to create a tree-like structure. It employs a single-writer, multi-reader locking mechanism and copy-on-write transactions to ensure serializable isolation and data integrity. The system includes comprehensive data management capa

    Implements a mechanism to force written data from memory to physical storage to ensure durability.

    Go
    Vezi pe GitHub↗9,573
  • mit-pdos/xv6-publicAvatar mit-pdos

    mit-pdos/xv6-public

    9,466Vezi pe GitHub↗

    xv6 is a Unix-like educational operating system designed for teaching kernel concepts. It is implemented as an instructional kernel for x86 and RISC-V architectures, featuring a virtual memory manager and support for symmetric multiprocessing. The system is designed to demonstrate core OS principles through a simplified re-implementation of Unix Version 6. It provides a controlled environment for studying system calls, trap handling, and process lifecycles. Its capability surface covers process management, including scheduling and context switching, and memory management via page tables for

    Implements a buffer cache to store disk blocks in memory, reducing physical hardware access and synchronizing concurrent reads.

    C
    Vezi pe GitHub↗9,466
  • risingwavelabs/risingwaveAvatar risingwavelabs

    risingwavelabs/risingwave

    9,093Vezi pe GitHub↗

    RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

    Caches frequently accessed data and state on local disks to reduce retrieval costs from cloud object storage.

    Rustapache-icebergdata-engineeringdatabase
    Vezi pe GitHub↗9,093
  • hashicorp/raftAvatar hashicorp

    hashicorp/raft

    9,037Vezi pe GitHub↗

    This is a Raft consensus library and distributed consensus engine implemented in Go. It provides the primitives necessary to build fault-tolerant distributed services by implementing a replicated state machine that ensures a group of servers agree on a shared system state through leader election and log replication. The project distinguishes itself through a pluggable architecture for storage backends and snapshot storage, decoupling the consensus logic from physical persistence. It includes specialized mechanisms for leadership transfer, protocol version management to support rolling upgrade

    Stores specific log entries in a fast-access ring buffer to reduce read latency from the primary store.

    Go
    Vezi pe GitHub↗9,037
  • atom-archive/xrayAvatar atom-archive

    atom-archive/xray

    8,420Vezi pe GitHub↗

    Xray is a collaborative text editor and distributed workspace manager that utilizes conflict-free replicated data types to synchronize real-time edits and directory structures across peers. It functions as both an Electron-based desktop application and a headless editor server that manages workspaces and file systems remotely for connected clients. The project distinguishes itself by integrating fine-grained version control tracking, recording keystroke-level changes and uncommitted edits between Git commits. It employs a decentralized synchronization model for working copies and uses operati

    Implements ordered operations when writing internal directory changes to disk to prevent the creation of circular directory loops.

    Rust
    Vezi pe GitHub↗8,420
  • path/fastimagecacheAvatar path

    path/FastImageCache

    8,068Vezi pe GitHub↗

    FastImageCache is an iOS image caching library that provides a persistent disk-based image store. It utilizes a persistent bitmap cache to store images in uncompressed formats and incorporates an image pre-processing pipeline to optimize assets before they are committed to storage. The library optimizes rendering performance by using memory-mapped image tables for constant-time retrieval and byte-aligned data layouts to prevent memory copies. It organizes images of identical dimensions into shared tables and manages disk space through a least-recently-used cache eviction system. The project

    Stores images on disk using a model-based approach to ensure fast loading in graphics-heavy applications.

    Objective-C
    Vezi pe GitHub↗8,068
  • cg123/mergekitAvatar cg123

    cg123/mergekit

    7,158Vezi pe GitHub↗

    Mergekit is a toolkit for combining multiple pretrained large language models into a single model. It functions as an architecture assembler and merging system that transfers capabilities between models using weighted algorithms and layer-wise assembly without requiring additional training. The project provides specialized utilities for extracting low-rank approximations from fine-tuned models to create portable parameter updates. It also includes a framework for converting dense language models into a mixture of experts architecture by constructing gating mechanisms to route inputs to specia

    Implements disk-based sharded loading to enable large-scale model merging on hardware with limited RAM.

    Python
    Vezi pe GitHub↗7,158
  • chyingp/nodejs-learning-guideAvatar chyingp

    chyingp/nodejs-learning-guide

    6,874Vezi pe GitHub↗

    This project is a learning guide and collection of study notes designed to teach Node.js backend development. It provides a comprehensive core API reference and practical demonstrations for implementing server-side logic, network programming, and system APIs. The guide specifically covers advanced technical domains including process management for scaling applications via clusters and child processes, as well as network programming for building TCP, UDP, and HTTP services. It also includes detailed instructional material on security implementation, focusing on cryptographic hashing and encryp

    Forces data flushes to persistent storage to ensure file integrity via disk synchronization.

    Rubycryptoexpressnodejs
    Vezi pe GitHub↗6,874
  • axboe/fioAvatar axboe

    axboe/fio

    6,268Vezi pe GitHub↗

    fio este un instrument de benchmarking pentru performanța stocării și un generator sintetic de sarcini I/O. Acesta funcționează ca un profiler de dispozitive de stocare și un motor de replay pentru urme I/O, permițând măsurarea throughput-ului și a latenței pentru dispozitivele de stocare și sistemele de fișiere. Proiectul se distinge prin capacitatea sa de a acționa ca un tester de stres pentru stocarea distribuită, gestionând mai multe backend-uri de server la distanță printr-un singur controler pentru a evalua stocarea în rețea. Include, de asemenea, capabilități specializate pentru analiza deduplicării stocării prin generarea de buffere de date redundante pentru a testa eficiența subsistemelor de deduplicare. Instrumentul acoperă o gamă largă de capabilități, inclusiv simularea sarcinilor I/O complexe cu dimensiuni de bloc și adâncime I/O controlabile, analiza inactivității CPU și colectarea statisticilor dispozitivelor. Oferă observabilitate prin vizualizarea metricilor de performanță, histograme de latență și monitorizarea stării de echilibru (steady-state). Sarcinile pot fi declanșate prin argumente de linie de comandă sau fișiere de configurare detaliate pentru a automatiza scenarii complexe de benchmarking.

    Forces the operating system to synchronize cached file data or metadata to the physical storage medium.

    C
    Vezi pe GitHub↗6,268
Înapoi12Înainte
  1. Home
  2. Data & Databases
  3. Disk Caching Systems

Explorează sub-etichetele

  • Buffer CachesIn-memory caches that store disk blocks to reduce physical I/O operations. **Distinct from Disk Caching Systems:** Distinct from Disk Caching Systems: specifically refers to the kernel-level block cache used for filesystem synchronization and performance, not large-scale analysis offloading.
  • Disk Synchronization1 sub-tagMechanisms for forcing data flushes to persistent storage to ensure file integrity. **Distinct from Disk Caching Systems:** Distinct from general disk caching: focuses on forced synchronization for integrity rather than performance caching.
  • Disk-Based Offloading Systems2 sub-tag-uriMechanisms for moving intermediate query results to local storage when memory limits are reached. **Distinct from Disk Caching Systems:** Distinct from general disk caching: focuses on offloading intermediate results during large-scale analytical operations.
  • Function Result CachingDisk-based caching systems that specifically store return values of functions based on input arguments. **Distinct from Disk Caching Systems:** Focuses on memoizing function results based on input hashes, not just offloading large data for memory management.