awesome-repositories.com
Blog
awesome-repositories.com

Entdecke die besten Open-Source-Repositories mit KI-gestützter Suche.

EntdeckenKuratierte SuchenOpen-Source-AlternativenSelf-hosted SoftwareBlogSitemap
ProjektÜber unsRanking-MethodikPresseMCP-Server
RechtlichesDatenschutzAGB
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

27 Repos

Awesome GitHub RepositoriesDisk Caching Systems

Mechanisms for offloading data to persistent storage to manage memory constraints.

Distinguishing note: Focuses on disk-backed storage for large-scale analysis tasks.

Explore 27 awesome GitHub repositories matching data & databases · Disk Caching Systems. Refine with filters or upvote what's useful.

Awesome Disk Caching Systems GitHub Repositories

Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
  • skylot/jadxAvatar von skylot

    skylot/jadx

    49,088Auf GitHub ansehen↗

    Jadx is a comprehensive Java decompilation suite designed to transform compiled binary application files into readable source code. It functions as a static analysis workbench, providing a graphical interface for navigating, searching, and inspecting the internal logic of complex software packages. By utilizing a bytecode-to-Java pipeline, the project reconstructs high-level logical structures from low-level binary instructions, making it a primary tool for Android application reverse engineering. The project distinguishes itself through a sophisticated control flow reconstruction engine and

    Offloads large intermediate data structures to the file system to minimize memory pressure during analysis.

    Javaandroiddecompilerdex
    Auf GitHub ansehen↗49,088
  • alibaba/easyexcelAvatar von alibaba

    alibaba/easyexcel

    33,703Auf GitHub ansehen↗

    EasyExcel is a Java processing library designed for reading and writing XLS, XLSX, and CSV files. It functions as a memory-efficient spreadsheet parser, an object-relational mapper that binds spreadsheet columns to Java class fields, and a stream-based exporter for handling high-volume data. The library distinguishes itself through a streaming model that processes large files row-by-row via listeners to prevent heap memory overflow. It also operates as a template engine, allowing the population of predefined spreadsheet files with dynamic data while preserving original layouts and styles. Br

    Utilizes disk-backed caching to offload cell data to temporary files, preventing memory overflow with massive datasets.

    Javaexceljavajxl
    Auf GitHub ansehen↗33,703
  • blankj/androidutilcodeAvatar von Blankj

    Blankj/AndroidUtilCode

    33,657Auf GitHub ansehen↗

    AndroidUtilCode is an Android utility library and system API wrapper designed to reduce development boilerplate. It provides a collection of helper classes for common tasks including system settings management, file I/O, and hardware access. The project distinguishes itself through a comprehensive toolset for device management and UI assistance. It includes specialized capabilities for monitoring battery status, managing system volume and brightness, and implementing UI helpers to prevent duplicate click events. It also provides a dedicated system for coordinate conversion between different m

    Saves bitmaps and JSON data to the local filesystem to reduce network requests and computation.

    Javaandroidandroidxapp
    Auf GitHub ansehen↗33,657
  • square/picassoAvatar von square

    square/picasso

    18,820Auf GitHub ansehen↗

    Picasso is an Android image loading library designed for downloading, caching, and displaying images in applications. It provides an automated system for memory and disk management to reduce network requests and improve load times. The library includes a bitmap transformation tool for resizing, cropping, and modifying assets before they are rendered. It also functions as a bridge for Jetpack Compose, converting image requests into painter objects for use within declarative Android user interfaces. The system manages image retrieval from multiple sources, including web URLs, local files, and

    Maintains images in tiered memory and disk caches for fast retrieval and reduced network usage.

    Kotlin
    Auf GitHub ansehen↗18,820
  • prestodb/prestoAvatar von prestodb

    prestodb/presto

    16,711Auf GitHub ansehen↗

    Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

    Moves intermediate results to local storage when memory usage exceeds defined limits to allow completion of large-scale operations.

    Javabig-datadatahadoop
    Auf GitHub ansehen↗16,711
  • imagemagick/imagemagickAvatar von ImageMagick

    ImageMagick/ImageMagick

    15,742Auf GitHub ansehen↗

    ImageMagick is a comprehensive software suite for the creation, editing, composition, and conversion of digital images. It functions as both a command-line utility for batch processing and automation, and as a programming library that allows developers to integrate advanced image manipulation capabilities into external applications. The project is distinguished by its modular architecture, which supports hundreds of image formats through a pluggable coder system and external delegate libraries. It is designed for high-performance environments, utilizing memory-mapped pixel caching, stream-ori

    Forces image data to flush to disk during processing to ensure file integrity and detect storage issues.

    Ccommand-line-image-tooldigital-image-editingimage-conversion
    Auf GitHub ansehen↗15,742
  • dgraph-io/badgerAvatar von dgraph-io

    dgraph-io/badger

    15,666Auf GitHub ansehen↗

    Badger is an embeddable key-value store written in Go that provides persistent data storage for byte keys and values. It is a persistent database that utilizes a tiered LSM tree storage model to optimize disk storage and retrieval efficiency. The system features an ACID transaction engine that ensures data integrity through serializable snapshot isolation and multi-version concurrency control. It also provides an encrypted key-value store with data-at-rest encryption and a managed encrypted key registry to secure stored information. The engine covers a broad set of capabilities including hig

    Synchronizes directory entries to disk to ensure consistency of file operations after a crash.

    Godatabasedocument-databasego
    Auf GitHub ansehen↗15,666
  • antirez/ds4Avatar von antirez

    antirez/ds4

    15,143Auf GitHub ansehen↗

    ds4 is a local inference engine for DeepSeek models that includes a distributed runtime for splitting transformer layers across networked computers. It functions as a reasoning controller with a local weight streamer and an API server that streams chat completions via industry standard endpoints. The system employs a memory management model that loads model experts from disk on demand to execute models that exceed available system RAM. It provides controls for reasoning effort and model behavior steering, allowing the modification of response characteristics through activation directions. Th

    Stores processed prompt prefixes on disk to allow session resumption without recalculating input tokens.

    C
    Auf GitHub ansehen↗15,143
  • boltdb/boltAvatar von boltdb

    boltdb/bolt

    14,642Auf GitHub ansehen↗

    Bolt is a single-file embedded key-value store for Go applications. It is an ACID transactional database that organizes data in B+trees on disk to provide efficient sorted key retrieval and range scans. The system uses a memory-mapped model to map the database file directly into the process address space for fast random-access reads. The project distinguishes itself through a multi-version concurrency control architecture that allows multiple simultaneous readers to access a consistent snapshot of data without blocking a writer. It employs a single-writer multi-reader locking model and uses a

    Writes modified memory data to physical storage to ensure durability and prevent data loss during crashes.

    Go
    Auf GitHub ansehen↗14,642
  • apple/turicreateAvatar von apple

    apple/turicreate

    11,171Auf GitHub ansehen↗

    This project is an automated machine learning framework and toolkit designed for training and tuning custom models for classification, regression, and recommendations. It functions as a multimodal machine learning toolkit capable of processing and training models using a combination of text, image, audio, and sensor data. The framework distinguishes itself as a multimodal data processor that can handle and visualize large datasets on a single machine using column-oriented disk storage. It includes a core machine learning model generator that converts trained models into formats compatible wit

    Manages large-scale data by swapping chunks between disk and memory to prevent system crashes.

    C++
    Auf GitHub ansehen↗11,171
  • dicedb/diceAvatar von DiceDB

    DiceDB/dice

    10,760Auf GitHub ansehen↗

    Dice is a low-latency key-value store based on Valkey that functions as a hierarchical storage engine. It combines a high-speed memory store with a tiered cache system to manage data across multiple storage levels. The system extends memory capacity by implementing a disk-backed cache that automatically persists evicted items to disk and restores them during cache misses. This storage tiering balances retrieval speed and operational cost by distributing data between fast memory and slower disk storage. The engine includes a query subscription service that monitors specific data patterns and

    Implements a system to restore evicted data from disk into memory during cache misses.

    C
    Auf GitHub ansehen↗10,760
  • etcd-io/bboltAvatar von etcd-io

    etcd-io/bbolt

    9,573Auf GitHub ansehen↗

    bbolt is an ACID-compliant embedded key-value store for Go applications. It persists all data in a single memory-mapped file on disk, organizing information using B+ trees to facilitate sorted key iteration and efficient range queries. The project distinguishes itself through a hierarchical data organization model, allowing buckets to be nested within other buckets to create a tree-like structure. It employs a single-writer, multi-reader locking mechanism and copy-on-write transactions to ensure serializable isolation and data integrity. The system includes comprehensive data management capa

    Implements a mechanism to force written data from memory to physical storage to ensure durability.

    Go
    Auf GitHub ansehen↗9,573
  • mit-pdos/xv6-publicAvatar von mit-pdos

    mit-pdos/xv6-public

    9,466Auf GitHub ansehen↗

    xv6 is a Unix-like educational operating system designed for teaching kernel concepts. It is implemented as an instructional kernel for x86 and RISC-V architectures, featuring a virtual memory manager and support for symmetric multiprocessing. The system is designed to demonstrate core OS principles through a simplified re-implementation of Unix Version 6. It provides a controlled environment for studying system calls, trap handling, and process lifecycles. Its capability surface covers process management, including scheduling and context switching, and memory management via page tables for

    Implements a buffer cache to store disk blocks in memory, reducing physical hardware access and synchronizing concurrent reads.

    C
    Auf GitHub ansehen↗9,466
  • hashicorp/raftAvatar von hashicorp

    hashicorp/raft

    9,037Auf GitHub ansehen↗

    This is a Raft consensus library and distributed consensus engine implemented in Go. It provides the primitives necessary to build fault-tolerant distributed services by implementing a replicated state machine that ensures a group of servers agree on a shared system state through leader election and log replication. The project distinguishes itself through a pluggable architecture for storage backends and snapshot storage, decoupling the consensus logic from physical persistence. It includes specialized mechanisms for leadership transfer, protocol version management to support rolling upgrade

    Stores specific log entries in a fast-access ring buffer to reduce read latency from the primary store.

    Go
    Auf GitHub ansehen↗9,037
  • risingwavelabs/risingwaveAvatar von risingwavelabs

    risingwavelabs/risingwave

    9,093Auf GitHub ansehen↗

    RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

    Caches frequently accessed data and state on local disks to reduce retrieval costs from cloud object storage.

    Rustapache-icebergdata-engineeringdatabase
    Auf GitHub ansehen↗9,093
  • atom-archive/xrayAvatar von atom-archive

    atom-archive/xray

    8,420Auf GitHub ansehen↗

    Xray is a collaborative text editor and distributed workspace manager that utilizes conflict-free replicated data types to synchronize real-time edits and directory structures across peers. It functions as both an Electron-based desktop application and a headless editor server that manages workspaces and file systems remotely for connected clients. The project distinguishes itself by integrating fine-grained version control tracking, recording keystroke-level changes and uncommitted edits between Git commits. It employs a decentralized synchronization model for working copies and uses operati

    Implements ordered operations when writing internal directory changes to disk to prevent the creation of circular directory loops.

    Rust
    Auf GitHub ansehen↗8,420
  • path/fastimagecacheAvatar von path

    path/FastImageCache

    8,068Auf GitHub ansehen↗

    FastImageCache is an iOS image caching library that provides a persistent disk-based image store. It utilizes a persistent bitmap cache to store images in uncompressed formats and incorporates an image pre-processing pipeline to optimize assets before they are committed to storage. The library optimizes rendering performance by using memory-mapped image tables for constant-time retrieval and byte-aligned data layouts to prevent memory copies. It organizes images of identical dimensions into shared tables and manages disk space through a least-recently-used cache eviction system. The project

    Stores images on disk using a model-based approach to ensure fast loading in graphics-heavy applications.

    Objective-C
    Auf GitHub ansehen↗8,068
  • cg123/mergekitAvatar von cg123

    cg123/mergekit

    7,158Auf GitHub ansehen↗

    Mergekit is a toolkit for combining multiple pretrained large language models into a single model. It functions as an architecture assembler and merging system that transfers capabilities between models using weighted algorithms and layer-wise assembly without requiring additional training. The project provides specialized utilities for extracting low-rank approximations from fine-tuned models to create portable parameter updates. It also includes a framework for converting dense language models into a mixture of experts architecture by constructing gating mechanisms to route inputs to specia

    Implements disk-based sharded loading to enable large-scale model merging on hardware with limited RAM.

    Python
    Auf GitHub ansehen↗7,158
  • chyingp/nodejs-learning-guideAvatar von chyingp

    chyingp/nodejs-learning-guide

    6,874Auf GitHub ansehen↗

    This project is a learning guide and collection of study notes designed to teach Node.js backend development. It provides a comprehensive core API reference and practical demonstrations for implementing server-side logic, network programming, and system APIs. The guide specifically covers advanced technical domains including process management for scaling applications via clusters and child processes, as well as network programming for building TCP, UDP, and HTTP services. It also includes detailed instructional material on security implementation, focusing on cryptographic hashing and encryp

    Forces data flushes to persistent storage to ensure file integrity via disk synchronization.

    Rubycryptoexpressnodejs
    Auf GitHub ansehen↗6,874
  • axboe/fioAvatar von axboe

    axboe/fio

    6,268Auf GitHub ansehen↗

    fio is a storage performance benchmarking tool and synthetic I/O workload generator. It functions as a storage device profiler and I/O trace replay engine, enabling the measurement of throughput and latency for storage devices and file systems. The project is distinguished by its ability to act as a distributed storage stress tester, managing multiple remote server backends via a single controller to evaluate network storage. It also includes specialized capabilities for storage deduplication analysis by generating redundant data buffers to test the efficiency of deduplication subsystems. Th

    Forces the operating system to synchronize cached file data or metadata to the physical storage medium.

    C
    Auf GitHub ansehen↗6,268
Vorherige12Nächste
  1. Home
  2. Data & Databases
  3. Disk Caching Systems

Unter-Tags erkunden

  • Buffer CachesIn-memory caches that store disk blocks to reduce physical I/O operations. **Distinct from Disk Caching Systems:** Distinct from Disk Caching Systems: specifically refers to the kernel-level block cache used for filesystem synchronization and performance, not large-scale analysis offloading.
  • Disk Synchronization1 Sub-TagMechanisms for forcing data flushes to persistent storage to ensure file integrity. **Distinct from Disk Caching Systems:** Distinct from general disk caching: focuses on forced synchronization for integrity rather than performance caching.
  • Disk-Based Offloading Systems2 Sub-TagsMechanisms for moving intermediate query results to local storage when memory limits are reached. **Distinct from Disk Caching Systems:** Distinct from general disk caching: focuses on offloading intermediate results during large-scale analytical operations.
  • Function Result CachingDisk-based caching systems that specifically store return values of functions based on input arguments. **Distinct from Disk Caching Systems:** Focuses on memoizing function results based on input hashes, not just offloading large data for memory management.