3 Repos
Techniques for reducing storage overhead and redundancy in resource-constrained environments.
Distinct from Data Storage Optimizers: Focuses on redundancy reduction for small clusters rather than data format compression.
Explore 3 awesome GitHub repositories matching data & databases · Resource Optimization. Refine with filters or upvote what's useful.
This project is a GitOps infrastructure framework designed for managing bare metal servers, container clusters, and networking. It serves as a declarative system for orchestrating the deployment and lifecycle of self-hosted services, using Git as the source of truth to synchronize the desired state of the environment. The framework differentiates itself through a comprehensive automation suite that covers the entire hardware-to-service pipeline. It includes a PXE-based bare metal provisioner for network booting and operating system installation, alongside a lightweight container orchestration
Conserves resources in small environments by reducing data redundancy and limiting replica sizes.
go-fastdfs is a distributed file system and object storage server designed for building private cloud storage. It provides a FastDFS compatible storage implementation that manages clusters of storage nodes to handle large-scale file uploads and downloads. The system focuses on high availability through a decentralized architecture that automatically synchronizes data and repairs failures across multiple machines without a central coordinator. It specifically supports resumable file storage via HTTP, allowing large transfers to be paused and resumed from the last successful byte to handle netw
Reduces filesystem overhead by merging small files and removing redundant data through SHA1 content hashing.
meta-rules-dat is a collection of binary-encoded network datasets used to identify and categorize traffic for routing on resource-constrained devices. It provides a structured domain categorization list and a geographic IP routing dataset to map network traffic to specific countries or service providers. The project utilizes trie-based lookup data and compact binary serialization to enable high-performance prefix matching and fast domain-to-category resolution. To minimize memory and storage overhead, it employs stripped-down GeoIP mapping that removes non-essential metadata. The datasets co
Employs compressed binary datasets to reduce memory and storage overhead on low-power hardware.