9 Repos
Systems for managing data as objects and providing scalable storage services.
Explore 9 awesome GitHub repositories matching part of an awesome list · Object Storage. Refine with filters or upvote what's useful.
MinIO is a software-defined, cloud-native object storage server designed to manage large volumes of unstructured data. It functions as a distributed storage cluster that aggregates multiple independent nodes into a unified, scalable pool, providing a high-performance infrastructure compatible with standard cloud storage protocols and application programming interfaces. The system utilizes a shared-nothing architecture that eliminates central metadata servers, relying instead on a decentralized hash table to map objects across the cluster. Data availability and resilience are maintained throug
S3-compatible object storage server.
SeaweedFS is a distributed object store and high-performance file system designed to manage massive volumes of unstructured data. It utilizes a decoupled architecture that separates metadata management from raw data storage, allowing for independent scalability and the efficient handling of billions of files. By providing a POSIX-compliant interface, it enables applications to interact with a unified namespace while maintaining the performance characteristics of a distributed object store. The system distinguishes itself through a multi-region data fabric that supports active-active replicati
Distributed file system optimized for small files.
SFTPGo is a multi-protocol file server and secure file transfer gateway that enables the exchange and storage of files via SFTP, FTP, HTTP, and WebDAV. It provides a unified management interface and a web-based file manager for configuring server settings, user permissions, and collaborating on documents. The project functions as a cloud storage gateway, bridging legacy file transfer protocols to modern backends such as S3, Google Cloud Storage, and Azure Blob. It features an automated data workflow engine that uses dynamic logic and virtual folders to route files and manage data lifecycles a
Configurable SFTP server with WebDAV and FTP support.
Sui is a blockchain platform featuring an object-centric state model and resource-oriented smart contracts. It utilizes parallel transaction execution to increase network throughput and supports programmable transaction blocks that bundle multiple operations into single atomic units. The platform distinguishes itself with a capability-based access control system and zero-knowledge login mechanisms, enabling users to authenticate via identity providers without seed phrases. It also implements deterministic object addressing to allow predictable state lookups and supports the creation of soulbo
Represents all digital assets as first-class objects with unique IDs to enable parallel processing.
The AWS SDK for PHP is a software development kit that provides HTTP client classes for every supported Amazon Web Service, enabling PHP applications to send authenticated requests and receive structured, typed response objects. It includes a credential resolution chain that automatically locates credentials from environment variables, instance profiles, or configuration files, and supports promise-based asynchronous execution for running multiple API calls concurrently to improve throughput. The SDK distinguishes itself through a middleware pipeline architecture that allows interception and
Copies large objects between 5 GB and 5 TB within S3 using a multipart copy process.
Cocoindex is an incremental data processing engine that builds and maintains live indexes for AI agents, with a core focus on codebase indexing and knowledge graph extraction. The engine uses a function-graph execution model where user-defined Python functions are composed into a directed acyclic graph, and it processes data incrementally so only changed source records or code paths are re-computed, avoiding full recomputation at any scale. It supports automatic schema inference from transformation pipeline type annotations and provides full data lineage tracing, tagging every output record wi
Builds real-time transformation pipelines triggered by events from object storage and message queues.
Incus is a unified orchestration platform for managing system containers, OCI application containers, and virtual machines through a single control plane. It brings together cluster infrastructure management, secure multi-tenancy, software-defined networking, and pluggable storage backend orchestration into one cohesive system exposed via a full REST API and command-line interface. What distinguishes Incus is its ability to run multiple instance types side by side—full Linux system containers, OCI application containers, and QEMU virtual machines—all managed with consistent tooling. Networkin
Provides Ceph Object Gateway-backed storage buckets accessible across cluster members without synchronization.
supabase-js ist eine umfassende Client-Bibliothek, die entwickelt wurde, um Frontend-Anwendungen mit einem gehosteten Backend-as-a-Service zu integrieren. Sie bietet ein einheitliches Interface für die Interaktion mit einer PostgreSQL-Datenbank, Identitätsmanagementsystemen, Cloud-Objektspeicherung und Echtzeit-Datensynchronisierung. Die Bibliothek bietet ein isomorphes Client-Design, das sowohl in Browser- als auch in Serverumgebungen funktioniert. Sie zeichnet sich durch einen Typ-sicheren Ansatz aus, nutzt TypeScript, um Datenbankschemata direkt auf Client-seitige Definitionen zu mappen, und verwendet eine PostgREST-basierte API, um JavaScript-Aufrufe in RESTful-Requests zu übersetzen. Der Client deckt ein breites Spektrum an Funktionen ab, einschließlich Nutzerauthentifizierung via OAuth, OIDC und Passkeys sowie Sitzungsmanagement unter Verwendung signierter Token. Er verwaltet große Binärdaten über ein S3-kompatibles Speicher-Interface und ermöglicht Live-Anwendungsupdates via WebSocket-basierter Abonnements für Datenbankänderungen und Präsenzsynchronisierung. Zusätzliche Funktionalität umfasst den Aufruf von Serverless-Edge-Functions und die Durchführung von Ähnlichkeitssuchen unter Verwendung von Vektor-Embeddings.
Provides a scalable object storage system for uploading and managing large binary data.
Implementation of the Amazon S3 protocol.