3 个仓库
Techniques for reducing storage overhead and redundancy in resource-constrained environments.
Distinct from Data Storage Optimizers: Focuses on redundancy reduction for small clusters rather than data format compression.
Explore 3 awesome GitHub repositories matching data & databases · Resource Optimization. Refine with filters or upvote what's useful.
This project is a GitOps infrastructure framework designed for managing bare metal servers, container clusters, and networking. It serves as a declarative system for orchestrating the deployment and lifecycle of self-hosted services, using Git as the source of truth to synchronize the desired state of the environment. The framework differentiates itself through a comprehensive automation suite that covers the entire hardware-to-service pipeline. It includes a PXE-based bare metal provisioner for network booting and operating system installation, alongside a lightweight container orchestration
Conserves resources in small environments by reducing data redundancy and limiting replica sizes.
go-fastdfs 是一个专为构建私有云存储而设计的分布式文件系统和对象存储服务器。它提供了一个兼容 FastDFS 的存储实现,管理存储节点集群以处理大规模文件上传和下载。 该系统通过去中心化架构实现高可用性,无需中央协调器即可自动同步数据并修复多台机器上的故障。它特别支持通过 HTTP 进行断点续传,允许大文件传输在网络不稳定时从最后一个成功字节处暂停并恢复。 核心功能包括通过基于 SHA1 的内容去重进行存储资源优化,以及合并小文件以减少文件系统 inode 消耗。该项目还集成了一个图像处理流水线,可在下载过程中执行动态缩放和调整大小,并使用基于令牌的认证来保护文件访问。 该系统可通过 Docker 容器部署。
Reduces filesystem overhead by merging small files and removing redundant data through SHA1 content hashing.
meta-rules-dat is a collection of binary-encoded network datasets used to identify and categorize traffic for routing on resource-constrained devices. It provides a structured domain categorization list and a geographic IP routing dataset to map network traffic to specific countries or service providers. The project utilizes trie-based lookup data and compact binary serialization to enable high-performance prefix matching and fast domain-to-category resolution. To minimize memory and storage overhead, it employs stripped-down GeoIP mapping that removes non-essential metadata. The datasets co
Employs compressed binary datasets to reduce memory and storage overhead on low-power hardware.