# deuxfleurs-org/garage

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/deuxfleurs-org-garage).**

2,944 stars · 110 forks · Rust · agpl-3.0

## Links

- GitHub: https://github.com/deuxfleurs-org/garage
- Homepage: https://garagehq.deuxfleurs.fr
- awesome-repositories: https://awesome-repositories.com/repository/deuxfleurs-org-garage.md

## Topics

`object-storage` `rust` `s3`

## Description

Garage is a distributed object storage system that provides an S3-compatible API gateway. It is designed to synchronize metadata across distributed nodes using conflict-free replicated data types and Merkle-tree state alignment to maintain cluster-wide consistency.

The system ensures data resilience through zone-aware replication, distributing data copies across multiple physical locations. It employs quorum-based request routing and versioned layout management to validate and commit cluster configuration changes.

The project covers a broad range of operational capabilities, including automated data rebalancing, pluggable metadata engines, and hardware failure recovery. It includes administrative tools via CLI and REST APIs for node role assignment, topology mapping, and service discovery.

Monitoring and security are handled through Prometheus and OpenTelemetry metrics, inter-node RPC encryption, and bucket-level permission controls. Deployment is supported via Kubernetes and Helm orchestration.

## Tags

### Networking & Communication

- [S3 Compatible APIs](https://awesome-repositories.com/f/networking-communication/api-integration-frameworks/api-management-integration/api-compatibility/s3-compatible-apis.md) — Provides an S3-compatible API gateway that translates standard object storage requests into internal cluster operations.
- [Gateway Routing](https://awesome-repositories.com/f/networking-communication/traffic-routing/node-aware-routing/gateway-routing.md) — Utilizes gateway nodes to route requests to the appropriate storage nodes for improved performance. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/))
- [Zone-Aware Data Replication](https://awesome-repositories.com/f/networking-communication/traffic-routing/zone-aware-routing/zone-aware-data-replication.md) — Distributes data copies across multiple physical availability zones to ensure resilience against site-wide failures.
- [Storage Block Compression](https://awesome-repositories.com/f/networking-communication/data-compression/storage-block-compression.md) — Reduces disk space and network bandwidth usage by applying compression to data blocks. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/))
- [Latency-Aware](https://awesome-repositories.com/f/networking-communication/load-balancers/latency-aware.md) — Improves data retrieval speed by prioritizing local nodes and low-latency zones. ([source](https://garagehq.deuxfleurs.fr/documentation/design/internals/))
- [Multipart Upload Utilities](https://awesome-repositories.com/f/networking-communication/multipart-upload-utilities.md) — Supports chunked file transfers to allow the interruption and completion of large object uploads. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/s3-compatibility/))
- [S3 Bucket Exposures](https://awesome-repositories.com/f/networking-communication/remote-storage-protocols/s3-protocol-clients/s3-bucket-exposures.md) — Serves S3 API and website interfaces to clients without requiring physical data storage on the local node. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/gateways/))
- [REST Administrative APIs](https://awesome-repositories.com/f/networking-communication/rest-administrative-apis.md) — Provides a dedicated REST API for programmatic management of nodes, buckets, and access credentials. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/features/))
- [Service Discovery Integrations](https://awesome-repositories.com/f/networking-communication/service-discovery-integrations.md) — Integrates with external service discovery tools to automatically locate other nodes within the cluster. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/from-source/))

### Data & Databases

- [Cross-Location Data Replication](https://awesome-repositories.com/f/data-databases/client-server-data-synchronization/core-data-sync-servers/cross-location-data-replication.md) — Distributes data copies across multiple physical locations and zones to ensure resilience against site failures. ([source](https://cdn.jsdelivr.net/gh/deuxfleurs-org/garage@main-v2/README.md))
- [Distributed Data Replication](https://awesome-repositories.com/f/data-databases/client-server-data-synchronization/core-data-sync-servers/distributed-data-replication.md) — Uses conflict-free replicated data types to synchronize state and maintain consistency across distributed nodes. ([source](https://garagehq.deuxfleurs.fr/documentation/design/))
- [Metadata Restorations](https://awesome-repositories.com/f/data-databases/data-governance-modeling/data-management-governance/backup-recovery-systems/database-backup-restoration/metadata-restorations.md) — Repairs the metadata database through synchronization from peer nodes or by restoring from snapshots. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/recovering/))
- [Data Restoration Tools](https://awesome-repositories.com/f/data-databases/data-restoration-tools.md) — Restores missing data blocks to replacement drives by synchronizing from healthy replicas. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/recovering/))
- [CRDT Databases](https://awesome-repositories.com/f/data-databases/distributed-databases/crdt-databases.md) — Implements a distributed database architecture using conflict-free replicated data types for metadata consistency.
- [Distributed Storage Recovery](https://awesome-repositories.com/f/data-databases/distributed-storage-recovery.md) — Ensures service and data availability are restored after physical hardware crashes to maintain system resilience. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/))
- [Bucket Management](https://awesome-repositories.com/f/data-databases/object-storage/bucket-management.md) — Provides administrative controls for creating isolated storage containers and managing their access permissions. ([source](https://garagehq.deuxfleurs.fr/documentation/))
- [Bucket Access Policies](https://awesome-repositories.com/f/data-databases/object-storage/bucket-management/bucket-access-policies.md) — Links API keys to specific storage buckets to restrict or grant read and write access. ([source](https://garagehq.deuxfleurs.fr/documentation/quick-start/))
- [Quorum-Based Consistency](https://awesome-repositories.com/f/data-databases/quorum-based-consistency.md) — Ensures data integrity by requiring a quorum of replica acknowledgments for routing and resolving object requests.
- [Block-based State Synchronization](https://awesome-repositories.com/f/data-databases/block-based-state-synchronization.md) — Audits disk storage against reference tables to identify and resolve missing data blocks. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/durability-repairs/))
- [Missing Block Resolution](https://awesome-repositories.com/f/data-databases/block-storage/missing-block-resolution.md) — Identifies blocks that cannot be fetched from the cluster to facilitate manual retries or purging. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/durability-repairs/))
- [Storage Deduplication](https://awesome-repositories.com/f/data-databases/block-storage/storage-deduplication.md) — Reduces storage footprints by identifying and eliminating duplicate data blocks. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/features/))
- [Zone Capacity Mapping](https://awesome-repositories.com/f/data-databases/cluster-topology-caching/static-topology-definitions/logical-topology-mapping/zone-capacity-mapping.md) — Maps nodes to logical zones and assigns storage capacities to ensure data replication across physical locations. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/real-world/))
- [Consistency Tuning](https://awesome-repositories.com/f/data-databases/consistency-tuning.md) — Allows adjusting quorum requirements for read and write operations to balance consistency against availability. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/))
- [Database Metadata Snapshots](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-extraction-ingestion/data-import-and-export/metadata-exporters/database-metadata-snapshots.md) — Creates consistent and simultaneous backups of metadata databases across all nodes for recovery from corruption. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/upgrading/))
- [Automatic Rebalancing](https://awesome-repositories.com/f/data-databases/data-partitioning-strategies/automatic-rebalancing.md) — Automatically redistributes data across the cluster when nodes are added or removed to maintain balance. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/features/))
- [Database Version Migrations](https://awesome-repositories.com/f/data-databases/database-version-migrations.md) — Runs migration procedures on metadata and cluster state to transition between software versions. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/upgrading/))
- [Key-Value Store APIs](https://awesome-repositories.com/f/data-databases/distributed-storage/key-value-store-apis.md) — Provides a specialized API for bulk storage and retrieval of small metadata pairs and hash table data. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/features/))
- [Capacity Analysis](https://awesome-repositories.com/f/data-databases/distributed-storage/storage-sharing/capacity-analysis.md) — Reports total and available disk space for data and metadata across all cluster nodes. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/monitoring/))
- [Metadata Reference Repairs](https://awesome-repositories.com/f/data-databases/metadata-reference-repairs.md) — Purges orphan object versions and block references to resolve inconsistencies caused by node unavailability. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/durability-repairs/))
- [Node Identity Replacements](https://awesome-repositories.com/f/data-databases/node-identity-replacements.md) — Provides mechanisms to swap lost node IDs with new ones to trigger automated data rebalancing. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/recovering/))
- [Object Lifecycle Operations](https://awesome-repositories.com/f/data-databases/object-storage/object-lifecycle-operations.md) — Automatically expires objects or aborts incomplete multipart uploads after a specified period. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/s3-compatibility/))
- [Pluggable Database Backends](https://awesome-repositories.com/f/data-databases/persistent-storage-backends/pluggable-database-backends.md) — Provides a pluggable architecture allowing the use of different database backends like LMDB or SQLite for metadata.
- [Pluggable Storage Engines](https://awesome-repositories.com/f/data-databases/pluggable-storage-engines.md) — Supports interchangeable database backends such as LMDB, Sqlite3, and Fjall to optimize performance. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/))
- [Data Integrity](https://awesome-repositories.com/f/data-databases/storage-engines/data-integrity.md) — Periodically scans data directories to detect corruption and rebuilds damaged files from healthy replicas. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/durability-repairs/))

### DevOps & Infrastructure

- [Cluster Node Management](https://awesome-repositories.com/f/devops-infrastructure/cluster-node-management.md) — Manages node identities and roles, distinguishing between storage providers and API gateways. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/layout/))
- [Node Retirement](https://awesome-repositories.com/f/devops-infrastructure/cluster-node-management/node-retirement.md) — Excises broken nodes from the cluster and redistributes data to maintain the required replica count. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/recovering/))
- [Gateway High Availability](https://awesome-repositories.com/f/devops-infrastructure/gateway-high-availability.md) — Employs load balancers and failover tools to ensure the S3 API gateway remains a stable entry point. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/ansible/))
- [Cluster Configuration Management](https://awesome-repositories.com/f/devops-infrastructure/cluster-configuration-management.md) — Includes a CLI tool specifically for defining and managing cluster-wide storage and node configurations. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/cli/))
- [Cluster Lifecycle Managers](https://awesome-repositories.com/f/devops-infrastructure/cluster-lifecycle-managers.md) — Provides utilities to propose, apply, or revert versioned changes to node roles and partition assignments. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/layout/))
- [Automated Node Discovery](https://awesome-repositories.com/f/devops-infrastructure/cluster-node-management/automated-node-discovery.md) — Implements transitive discovery using unique identifiers to allow distributed nodes to automatically join the cluster. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/real-world/))
- [Registrar-Based Node Discoveries](https://awesome-repositories.com/f/devops-infrastructure/cluster-node-management/automated-node-discovery/registrar-based-node-discoveries.md) — Integrates with orchestrators to automatically discover and connect cluster nodes without manual configuration. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/features/))
- [Automated Partition Balancers](https://awesome-repositories.com/f/devops-infrastructure/load-balancing/partitioning-algorithms/automated-partition-balancers.md) — Automatically redistributes data partitions across storage nodes to balance load and maintain replica counts. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/layout/))
- [Node Availability Verification](https://awesome-repositories.com/f/devops-infrastructure/node-availability-verification.md) — Uses periodic TCP pings as heartbeats to verify node availability and ensure correct request routing. ([source](https://garagehq.deuxfleurs.fr/documentation/design/internals/))

### Security & Cryptography

- [API Access Security](https://awesome-repositories.com/f/security-cryptography/api-access-security.md) — Generates unique credentials to authenticate requests and control access to specific storage buckets. ([source](https://garagehq.deuxfleurs.fr/documentation/))
- [Secure Node Networking](https://awesome-repositories.com/f/security-cryptography/secure-node-networking.md) — Secures remote procedure calls between cluster nodes using a mandatory secret handshake. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/encryption/))
- [Scoped Token Issuance](https://awesome-repositories.com/f/security-cryptography/access-token-management/offline-access-tokens/scoped-token-issuance.md) — Creates scoped, time-limited API tokens to control access to specific administrative functions. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/admin-api/))
- [Client-Side Encryption](https://awesome-repositories.com/f/security-cryptography/client-side-encryption.md) — Encrypts objects using client-provided keys before writing them to disk. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/encryption/))
- [Presigned URL Generation](https://awesome-repositories.com/f/security-cryptography/presigned-url-generation.md) — Creates temporary, cryptographically signed URLs that grant timed access to specific objects. ([source](https://garagehq.deuxfleurs.fr/documentation/connect/apps/))

### Software Engineering & Architecture

- [Merkle-Tree Storage Engines](https://awesome-repositories.com/f/software-engineering-architecture/string-processing-algorithms/tree-based-storage/merkle-tree-storage-engines.md) — Uses Merkle trees to efficiently compare and align metadata state across distributed cluster nodes.
- [Cluster Layout Versioning](https://awesome-repositories.com/f/software-engineering-architecture/configuration-versioning/version-requirement-validators/cluster-layout-versioning.md) — Stages and validates versioned cluster configuration changes before committing them to the active environment.
- [Rolling Updates](https://awesome-repositories.com/f/software-engineering-architecture/instance-management/rolling-updates.md) — Implements strategies for updating binaries and configurations across the cluster incrementally to ensure continuous availability. ([source](https://garagehq.deuxfleurs.fr/documentation/operations/upgrading/))

### Development Tools & Productivity

- [CLI Administration Tools](https://awesome-repositories.com/f/development-tools-productivity/cli-administration-tools.md) — Ships a command-line interface for performing cluster administrative tasks via an internal RPC protocol. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/admin-api/))

### System Administration & Monitoring

- [API Performance Monitoring](https://awesome-repositories.com/f/system-administration-monitoring/api-performance-monitoring.md) — Tracks latency, throughput, and error rates for S3 and administrative API endpoints. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/monitoring/))
- [Cluster Health Monitoring](https://awesome-repositories.com/f/system-administration-monitoring/cluster-health-monitoring.md) — Exposes Prometheus metrics and health check endpoints to monitor distributed cluster status and quorum availability. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/))
- [Prometheus-Based Metric Exporters](https://awesome-repositories.com/f/system-administration-monitoring/prometheus-exporters/prometheus-based-metric-exporters.md) — Gathers operational telemetry and execution traces using Prometheus and OpenTelemetry formats. ([source](https://garagehq.deuxfleurs.fr/documentation/cookbook/from-source/))

### Web Development

- [Replication Progress Monitoring](https://awesome-repositories.com/f/web-development/reactive-networking/progress-tracking/replication-progress-monitoring.md) — Tracks the progress of block resyncs and queue lengths to ensure data consistency across the cluster. ([source](https://garagehq.deuxfleurs.fr/documentation/reference-manual/monitoring/))

### Part of an Awesome List

- [Database Systems](https://awesome-repositories.com/f/awesome-lists/data/database-systems.md) — S3-compatible distributed object storage for self-hosting.