9 repositorios
Systems for managing data as objects and providing scalable storage services.
Explore 9 awesome GitHub repositories matching part of an awesome list · Object Storage. Refine with filters or upvote what's useful.
MinIO is a software-defined, cloud-native object storage server designed to manage large volumes of unstructured data. It functions as a distributed storage cluster that aggregates multiple independent nodes into a unified, scalable pool, providing a high-performance infrastructure compatible with standard cloud storage protocols and application programming interfaces. The system utilizes a shared-nothing architecture that eliminates central metadata servers, relying instead on a decentralized hash table to map objects across the cluster. Data availability and resilience are maintained throug
S3-compatible object storage server.
SeaweedFS is a distributed object store and high-performance file system designed to manage massive volumes of unstructured data. It utilizes a decoupled architecture that separates metadata management from raw data storage, allowing for independent scalability and the efficient handling of billions of files. By providing a POSIX-compliant interface, it enables applications to interact with a unified namespace while maintaining the performance characteristics of a distributed object store. The system distinguishes itself through a multi-region data fabric that supports active-active replicati
Distributed file system optimized for small files.
SFTPGo is a multi-protocol file server and secure file transfer gateway that enables the exchange and storage of files via SFTP, FTP, HTTP, and WebDAV. It provides a unified management interface and a web-based file manager for configuring server settings, user permissions, and collaborating on documents. The project functions as a cloud storage gateway, bridging legacy file transfer protocols to modern backends such as S3, Google Cloud Storage, and Azure Blob. It features an automated data workflow engine that uses dynamic logic and virtual folders to route files and manage data lifecycles a
Configurable SFTP server with WebDAV and FTP support.
Sui is a blockchain platform featuring an object-centric state model and resource-oriented smart contracts. It utilizes parallel transaction execution to increase network throughput and supports programmable transaction blocks that bundle multiple operations into single atomic units. The platform distinguishes itself with a capability-based access control system and zero-knowledge login mechanisms, enabling users to authenticate via identity providers without seed phrases. It also implements deterministic object addressing to allow predictable state lookups and supports the creation of soulbo
Represents all digital assets as first-class objects with unique IDs to enable parallel processing.
The AWS SDK for PHP is a software development kit that provides HTTP client classes for every supported Amazon Web Service, enabling PHP applications to send authenticated requests and receive structured, typed response objects. It includes a credential resolution chain that automatically locates credentials from environment variables, instance profiles, or configuration files, and supports promise-based asynchronous execution for running multiple API calls concurrently to improve throughput. The SDK distinguishes itself through a middleware pipeline architecture that allows interception and
Copies large objects between 5 GB and 5 TB within S3 using a multipart copy process.
Cocoindex is an incremental data processing engine that builds and maintains live indexes for AI agents, with a core focus on codebase indexing and knowledge graph extraction. The engine uses a function-graph execution model where user-defined Python functions are composed into a directed acyclic graph, and it processes data incrementally so only changed source records or code paths are re-computed, avoiding full recomputation at any scale. It supports automatic schema inference from transformation pipeline type annotations and provides full data lineage tracing, tagging every output record wi
Builds real-time transformation pipelines triggered by events from object storage and message queues.
Incus is a unified orchestration platform for managing system containers, OCI application containers, and virtual machines through a single control plane. It brings together cluster infrastructure management, secure multi-tenancy, software-defined networking, and pluggable storage backend orchestration into one cohesive system exposed via a full REST API and command-line interface. What distinguishes Incus is its ability to run multiple instance types side by side—full Linux system containers, OCI application containers, and QEMU virtual machines—all managed with consistent tooling. Networkin
Provides Ceph Object Gateway-backed storage buckets accessible across cluster members without synchronization.
supabase-js is a comprehensive client library designed to integrate frontend applications with a hosted backend-as-a-service. It provides a unified interface for interacting with a PostgreSQL database, identity management systems, cloud object storage, and real-time data synchronization. The library features an isomorphic client design that operates across both browser and server environments. It distinguishes itself through a type-safe approach, utilizing TypeScript to map database schemas directly to client-side definitions, and employs a PostgREST-based API to translate JavaScript calls in
Provides a scalable object storage system for uploading and managing large binary data.
Implementation of the Amazon S3 protocol.