11 Repos
Architectures that leverage cloud object storage for scalable and stateless data persistence.
Distinguishing note: Focuses on the integration of object storage as a primary data layer.
Explore 11 awesome GitHub repositories matching devops & infrastructure · Object Storage Integration. Refine with filters or upvote what's useful.
SurrealDB is a multi-model database engine designed to store and query document, graph, relational, and vector data within a single ACID-compliant platform. It functions as an AI-native data store, integrating vector search, graph traversal, and machine learning model execution directly into its query layer. By providing a unified declarative query language, the platform eliminates the need for external middleware to synchronize data across different storage models. The platform distinguishes itself through its ability to manage agent memory and complex workflows natively. It allows developer
Saves transactional data directly in cloud object storage to enable stateless, elastic compute.
Apache Flink is a distributed processing engine designed for both high-throughput, low-latency data streams and finite batch workloads. It functions as a stateful stream processor and a SQL stream processing engine, providing a unified runtime to execute relational queries and event-based transformations. The system is distinguished by its ability to manage persistent operator state to ensure exactly-once processing guarantees and consistency during failures. It features specialized capabilities for complex event processing to detect temporal patterns and handles out-of-order events using eve
Integrates cloud object storage for scalable and stateless data persistence and as a sink/source.
SpringBoot-Labs is a collection of educational resources and reference implementations for Java backend architecture and distributed systems. It provides practical lab guides and code samples focused on building applications with the Spring Boot framework and designing scalable microservices architectures. The project specifically covers service governance and distributed cloud deployment patterns using Spring Cloud and Spring Cloud Alibaba. It includes a dedicated kit for microservices and a guide for executing remote procedure calls and managing service discovery via the Dubbo protocol. Th
Provides reference implementations for leveraging cloud object storage for stateless data persistence.
OpenObserve is a unified observability data platform designed to ingest, store, and analyze logs, metrics, and traces. It functions as a cloud-native monitoring tool that centralizes telemetry from diverse sources, including standard collectors and cloud service providers, into a single, scalable system. By utilizing a columnar storage engine backed by object storage, the platform enables efficient long-term data retention and high-performance analytical querying. The platform distinguishes itself through deep integration with artificial intelligence, allowing users to query data using natura
Utilizes cloud-native object storage buckets as the primary data repository to ensure durability, scalability, and low-cost retention.
QuestDB is a high-performance, distributed time-series database designed for the ingestion, storage, and analysis of massive datasets. It functions as a real-time analytics platform that utilizes a columnar storage engine to optimize disk input and output, enabling efficient analytical scans and complex windowing operations on streaming data. The platform distinguishes itself through specialized capabilities for handling asynchronous time-series streams, including advanced join algorithms that align disparate data sets based on precise timestamp lookups. It supports high-volume ingestion thro
Synchronizes state across distributed replicas by utilizing centralized cloud object storage as the primary source of truth.
immudb is a tamperproof database that maintains an immutable record of entries using cryptographic commit logging. It ensures verifiable database integrity by utilizing Merkle trees to generate membership and consistency proofs that detect unauthorized data alterations. The system employs a multi-model storage engine that unifies key-value, document, and relational data structures within a single immutable backend. It provides compatibility with the PostgreSQL wire protocol, allowing it to integrate with standard SQL clients, ORMs, and database tools. The project covers broad capabilities in
Leverages cloud object storage as a primary data layer using access keys and cloud credentials.
paopao-ce ist eine soziale Community-Plattform, die für die Veröffentlichung von Inhalten und das Management von Nutzerbeziehungen entwickelt wurde. Sie bietet eine skalierbare Networking-Umgebung, die sowohl Freundschafts- als auch Follower-basierte soziale Modelle implementiert, um Nutzerinteraktion und Content-Sharing zu regulieren. Die Plattform integriert eine digitale Wallet und eine Payment-Gateway-Anbindung, um Finanztransaktionen abzuwickeln und Nutzerguthaben zu verwalten. Sie nutzt eine Kombination aus einem Go-basierten Backend und einem JavaScript-basierten Frontend zur Bereitstellung ihrer Dienste. Das System umfasst Funktionen für Identitätsmanagement via SMS-Verifizierung und sichere Token-basierte Sitzungen. Es bietet zudem Community-Content-Suche mittels invertierter Indizierung, Medienpersistenz durch Objektspeicherung und die automatisierte Generierung maschinenlesbarer API-Spezifikationen. Die Observability wird durch System-Log-Capture und Performance-Profiling gehandhabt, um Verarbeitungsengpässe zu identifizieren.
Integrates cloud object storage to persist user media and maintain a stateless application server.
Photoshot is a commercial SaaS image platform and web application used for creating personalized AI avatars and portraits. It functions as an AI avatar creator that trains custom machine learning models on user-uploaded photos to produce consistent digital personas. The platform includes an LLM prompt generator that uses large language models to craft detailed text descriptions for image generation engines. It integrates a secure third-party payment gateway to manage user access to these creative tools and services. The system architecture handles asynchronous task queueing for machine learn
Integrates cloud object storage to handle large binary assets like generated images and trained model weights.
Tusd is a server implementation of the open protocol for resumable file uploads over HTTP. It functions as a gateway that streams incoming data directly to local disk or remote object storage backends, ensuring that large files can be processed without exhausting system memory. The server is designed as a modular package that can be embedded directly into existing applications, allowing developers to integrate file transfer capabilities without requiring external service dependencies. It maintains persistent state for all active transfers, enabling clients to pause and resume uploads from the
Streams uploaded files directly into remote object storage services for scalable data persistence.
Helix DB is a distributed graph database and knowledge graph platform that persists nodes and edges on object storage for durable and unlimited scaling. It operates as an ACID-compliant system, ensuring data consistency through serializable snapshot isolation during concurrent operations. The project distinguishes itself by combining a vector search engine and a property graph, utilizing hybrid vector and full-text search to locate entry points for graph traversals. It enables dynamic graph querying through a domain-specific language, allowing complex logic and recursive queries to be execute
Persists graph nodes and edges on cloud object storage to enable durable and unlimited scaling.
Dieses Projekt bietet einen selbst gehosteten Remote-Cache-Server, der darauf ausgelegt ist, Build-Artefakte für verteilte Entwicklungsteams und CI/CD-Pipelines zu speichern und bereitzustellen. Als zentraler Proxy für Build-Outputs ermöglicht es Teams, Arbeitsergebnisse zu synchronisieren, redundante Ausführungen zu minimieren und die Build-Zeiten für Monorepo-Projekte zu beschleunigen. Das System zeichnet sich durch eine modulare Architektur aus, die verschiedene Storage-Backends unterstützt, sodass Artefakte auf lokalen Festplatten gespeichert oder in verschiedene Cloud-basierte Objektspeicher integriert werden können. Es nutzt einen Stream-basierten Netzwerktransport, um große Build-Outputs effizient zu verarbeiten, und verwendet kryptografische Signaturen, um die Integrität und Authentizität der Artefakte während der Übertragung zu verifizieren. Die Sicherheit wird durch tokenbasierte Request-Authentifizierung und Middleware-gesteuerte Zugriffskontrolle verwaltet, die es Administratoren ermöglichen, Read-Only-Modi zu erzwingen und die Interaktion mit dem Cache einzuschränken. Die Infrastruktur ist auf Flexibilität ausgelegt und unterstützt die automatisierte Bereitstellung in containerisierten, serverlosen und Cloud-Umgebungen, während Health-Monitoring-Endpunkte den Betriebsstatus verifizieren.
Persists build outputs to local disks or cloud-based object storage providers for scalable artifact management.