9 dépôts
Systems for managing data as objects and providing scalable storage services.
Explore 9 awesome GitHub repositories matching part of an awesome list · Object Storage. Refine with filters or upvote what's useful.
MinIO is a software-defined, cloud-native object storage server designed to manage large volumes of unstructured data. It functions as a distributed storage cluster that aggregates multiple independent nodes into a unified, scalable pool, providing a high-performance infrastructure compatible with standard cloud storage protocols and application programming interfaces. The system utilizes a shared-nothing architecture that eliminates central metadata servers, relying instead on a decentralized hash table to map objects across the cluster. Data availability and resilience are maintained throug
S3-compatible object storage server.
SeaweedFS is a distributed object store and high-performance file system designed to manage massive volumes of unstructured data. It utilizes a decoupled architecture that separates metadata management from raw data storage, allowing for independent scalability and the efficient handling of billions of files. By providing a POSIX-compliant interface, it enables applications to interact with a unified namespace while maintaining the performance characteristics of a distributed object store. The system distinguishes itself through a multi-region data fabric that supports active-active replicati
Distributed file system optimized for small files.
SFTPGo is a multi-protocol file server and secure file transfer gateway that enables the exchange and storage of files via SFTP, FTP, HTTP, and WebDAV. It provides a unified management interface and a web-based file manager for configuring server settings, user permissions, and collaborating on documents. The project functions as a cloud storage gateway, bridging legacy file transfer protocols to modern backends such as S3, Google Cloud Storage, and Azure Blob. It features an automated data workflow engine that uses dynamic logic and virtual folders to route files and manage data lifecycles a
Configurable SFTP server with WebDAV and FTP support.
Sui is a blockchain platform featuring an object-centric state model and resource-oriented smart contracts. It utilizes parallel transaction execution to increase network throughput and supports programmable transaction blocks that bundle multiple operations into single atomic units. The platform distinguishes itself with a capability-based access control system and zero-knowledge login mechanisms, enabling users to authenticate via identity providers without seed phrases. It also implements deterministic object addressing to allow predictable state lookups and supports the creation of soulbo
Represents all digital assets as first-class objects with unique IDs to enable parallel processing.
The AWS SDK for PHP is a software development kit that provides HTTP client classes for every supported Amazon Web Service, enabling PHP applications to send authenticated requests and receive structured, typed response objects. It includes a credential resolution chain that automatically locates credentials from environment variables, instance profiles, or configuration files, and supports promise-based asynchronous execution for running multiple API calls concurrently to improve throughput. The SDK distinguishes itself through a middleware pipeline architecture that allows interception and
Copies large objects between 5 GB and 5 TB within S3 using a multipart copy process.
Cocoindex is an incremental data processing engine that builds and maintains live indexes for AI agents, with a core focus on codebase indexing and knowledge graph extraction. The engine uses a function-graph execution model where user-defined Python functions are composed into a directed acyclic graph, and it processes data incrementally so only changed source records or code paths are re-computed, avoiding full recomputation at any scale. It supports automatic schema inference from transformation pipeline type annotations and provides full data lineage tracing, tagging every output record wi
Builds real-time transformation pipelines triggered by events from object storage and message queues.
Incus is a unified orchestration platform for managing system containers, OCI application containers, and virtual machines through a single control plane. It brings together cluster infrastructure management, secure multi-tenancy, software-defined networking, and pluggable storage backend orchestration into one cohesive system exposed via a full REST API and command-line interface. What distinguishes Incus is its ability to run multiple instance types side by side—full Linux system containers, OCI application containers, and QEMU virtual machines—all managed with consistent tooling. Networkin
Provides Ceph Object Gateway-backed storage buckets accessible across cluster members without synchronization.
supabase-js est une bibliothèque client complète conçue pour intégrer des applications frontend avec un backend-as-a-service hébergé. Elle fournit une interface unifiée pour interagir avec une base de données PostgreSQL, des systèmes de gestion des identités, le stockage d'objets cloud et la synchronisation de données en temps réel. La bibliothèque présente une conception client isomorphe qui fonctionne à la fois dans les environnements navigateur et serveur. Elle se distingue par une approche typée, utilisant TypeScript pour mapper les schémas de base de données directement aux définitions côté client, et emploie une API basée sur PostgREST pour traduire les appels JavaScript en requêtes RESTful. Le client couvre un large éventail de capacités, incluant l'authentification utilisateur via OAuth, OIDC et passkeys, ainsi que la gestion de session utilisant des jetons signés. Il gère des données binaires à grande échelle via une interface de stockage compatible S3 et permet des mises à jour d'application en direct via des abonnements basés sur WebSocket pour les changements de base de données et la synchronisation de présence. Une fonctionnalité supplémentaire inclut l'invocation de fonctions edge serverless et l'exécution de recherches de similarité utilisant des plongements vectoriels (vector embeddings).
Provides a scalable object storage system for uploading and managing large binary data.
Implementation of the Amazon S3 protocol.