Awesome GitHub RepositoriesStorage Configurations

MindsDB is an AI-native database engine that treats machine learning models and autonomous agents as virtual tables. By mapping external data sources, predictive models, and third-party services directly into the database schema, it enables users to perform inference, data retrieval, and complex orchestration using standard SQL syntax. The platform distinguishes itself through an autonomous agent orchestrator that executes iterative reasoning loops, allowing agents to plan data access and synthesize natural language responses from connected knowledge bases. It functions as a federated data ga

Karakeep is a self-hosted, open-source platform designed for personal knowledge management and web content archiving. It functions as a centralized repository where users can capture, organize, and preserve bookmarks, notes, and media files, ensuring long-term access to digital information even if original sources are removed or modified. The system distinguishes itself through its automated content processing and security-focused architecture. It utilizes headless browser crawling and optical character recognition to ingest and index web content, while a modular artificial intelligence pipel

Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep

Cube is a semantic data layer that provides a unified framework for defining business metrics, dimensions, and relationships across diverse data sources. By acting as a headless business intelligence engine, it transforms raw data into a governed model that can be queried via SQL, REST, and GraphQL interfaces. This architecture ensures consistent data definitions and logic across all downstream analytical applications and reporting tools. The platform distinguishes itself through its integrated conversational AI capabilities, which allow users to explore data using natural language. It orches

Docmost is an open-source knowledge management system designed as a collaborative documentation platform for teams. It functions as an enterprise wiki that centralizes organizational information into structured, searchable workspaces, enabling users to create, organize, and share content through a hierarchical system of spaces and pages. The platform distinguishes itself by integrating artificial intelligence directly into the documentation lifecycle. It utilizes vector-based semantic search to allow for natural language queries across stored content and provides AI-assisted tools for draftin

OpenObserve is a unified observability data platform designed to ingest, store, and analyze logs, metrics, and traces. It functions as a cloud-native monitoring tool that centralizes telemetry from diverse sources, including standard collectors and cloud service providers, into a single, scalable system. By utilizing a columnar storage engine backed by object storage, the platform enables efficient long-term data retention and high-performance analytical querying. The platform distinguishes itself through deep integration with artificial intelligence, allowing users to query data using natura

Calibre-web is a self-hosted web application that provides a browser-based interface for browsing, managing, and reading digital book collections stored in a library database. It functions as a comprehensive library management system, allowing users to organize large collections, edit metadata, and perform automated content updates through a centralized administrative dashboard. The platform distinguishes itself by integrating directly with external infrastructure to extend the capabilities of a standard digital library. It supports remote storage mapping to host files on cloud providers, uti

Kubo is a peer-to-peer implementation of the InterPlanetary File System (IPFS) designed for decentralized data storage and content delivery. It uses content-addressing, directed acyclic graphs, and distributed hash tables to identify, distribute, and retrieve data across a network without relying on central servers. The project differentiates itself by providing a virtual filesystem via FUSE, which maps decentralized network namespaces to local operating system directories for direct file access. It also includes integrated HTTP gateways that translate peer-to-peer content into standard web t

Cap is a self-hosted screen recording and video collaboration platform designed for teams to replace synchronous meetings with asynchronous video updates. It provides a comprehensive suite for capturing high-resolution desktop activity, including system audio, microphone input, and camera overlays, which are then processed through an integrated post-production workflow. The platform distinguishes itself by offering full data sovereignty through containerized deployment and object storage abstractions, allowing users to host their media assets on private infrastructure or S3-compatible buckets

LibSQL is a high-performance, distributed SQL database engine that extends SQLite to support remote network access, edge computing, and real-time synchronization. It functions as an embedded database library that integrates directly into application processes while providing the infrastructure to maintain consistency across multiple geographic regions. The platform distinguishes itself by enabling database interaction over standard HTTP protocols, allowing applications to query remote data sources in serverless and edge environments without requiring local filesystem access. It includes nativ

Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

TigerBeetle is a distributed financial accounting database designed for high-volume transaction processing. It functions as a specialized transaction engine that enforces strict double-entry bookkeeping invariants, ensuring that every debit and credit is balanced and accounted for with absolute consistency. By utilizing a consensus-based replication model, the system provides high availability and data durability across geographically distributed clusters, making it suitable for mission-critical financial infrastructure. The system distinguishes itself through a performance-oriented architect

Unstructured is an enterprise-grade data orchestration engine designed to transform raw, unstructured files into structured, machine-readable formats. It functions as a comprehensive platform for document ingestion, partitioning, and enrichment, specifically engineered to prepare complex data for retrieval-augmented generation and agentic AI workflows. The platform distinguishes itself through its sophisticated document processing strategies, which combine rule-based extraction with vision-language models to handle diverse file layouts, tables, and images. It provides a modular architecture t

PiggyMetrics is a demonstration project that illustrates a complete microservice architecture built with Spring Boot, Spring Cloud, and Docker, centered around a personal finance tracking application. It records incomes, expenses, and savings for individual accounts while computing spending statistics and time-series data to track cash flow dynamics over time. The project showcases a full Spring Cloud service mesh, including an API gateway for request routing, service discovery for dynamic instance location, and a centralized configuration server that manages application settings from a singl

Dask هو إطار عمل للحوسبة المتوازية وجدول مهام موزع مصمم لتوسيع نطاق سير عمل علوم البيانات في Python من أجهزة فردية إلى مجموعات (clusters) كبيرة. يعمل كمدير موارد للمجموعة يقوم بتنسيق المنطق الحسابي من خلال تمثيل المهام وتبعياتها كرسوم بيانية موجهة غير دورية. تسمح هذه البنية للنظام بأتمتة توزيع أعباء العمل عبر الأجهزة المتاحة مع إدارة متطلبات التنفيذ المعقدة. يتميز المشروع بمحرك تقييم كسول يؤجل عمليات البيانات حتى يتم طلبها صراحة، مما يتيح تحسين الرسم البياني العالمي وتخصيص الموارد بكفاءة. يتضمن خاصية تسريب البيانات الواعية بالذاكرة لمنع تعطل النظام عند معالجة مجموعات البيانات التي تتجاوز الذاكرة المتاحة، ويستخدم دمج الرسم البياني للمهام لدمج تسلسلات العمليات في خطوات تنفيذ واحدة، مما يقلل من عبء الجدولة والاتصال بين العقد. توفر المنصة سطح قدرات شاملاً لتحليلات البيانات واسعة النطاق، بما في ذلك دعم التعلم الآلي الموزع، وتكامل الحوسبة عالية الأداء، ومعالجة البيانات المتوازية. توفر أدوات واسعة النطاق لإدارة دورة حياة المجموعة، وتوصيف الأداء، والمراقبة في الوقت الفعلي لتنفيذ المهام. يمكن للمستخدمين نشر هذه البيئات عبر بنية تحتية متنوعة، بما في ذلك الأجهزة المحلية، ومزودي السحابة، والأنظمة الحاوية، ومجموعات الحوسبة عالية الأداء.

Boto3 is the AWS SDK for Python, providing a programmatic interface for managing and automating AWS cloud infrastructure and services. It serves as a cloud management API client and resource manager for provisioning, configuring, and scaling virtual servers, databases, and storage. The library enables the implementation of infrastructure-as-code through declarative templates and scripts, allowing for the deployment of identical resource stacks across multiple accounts and geographic regions. It also provides a framework for coordinating distributed workflows, serverless functions, and contain

Mage AI is a Python-based data pipeline orchestrator and self-hosted data integrated development environment. It is designed for building, scheduling, and monitoring data workflows using a block-based pipeline design and interactive notebook interface. The platform distinguishes itself by integrating generative AI capabilities, allowing users to connect large language model providers via API to incorporate artificial intelligence into automated data streams. It also functions as an Apache Spark data processor, managing the kernels and infrastructure required for high-volume analytics and larg

Moto is a cloud service mockery framework and API mock server that simulates AWS infrastructure locally. It allows developers to test cloud-dependent code and verify infrastructure-as-code templates without deploying real resources or incurring costs. The project functions as an SDK interceptor that can patch existing service clients to redirect requests to a local mock environment. It can also be run as a standalone HTTP server, enabling any programming language to interact with the simulated endpoints. The framework covers a vast array of simulated capabilities, including data storage, com

SpringCloud-Learning is an educational project that demonstrates how to build microservices using Spring Cloud, covering the core patterns of service discovery, centralized configuration management, and API gateway routing. The project provides hands-on examples for registering and discovering microservice instances with Nacos, Eureka, or Consul, and for routing external API requests through Spring Cloud Gateway with support for filters and load balancing. The tutorials explore service resilience through circuit breakers and rate limiting with Sentinel and Hystrix, including custom fallback l

Afero is a Go library that provides a unified filesystem abstraction, allowing applications to interact with local disk, in-memory storage, cloud services, archives, and remote systems through a single, consistent interface. At its core, it defines a standard interface that all filesystem backends implement, enabling developers to swap storage implementations without changing application code. The library distinguishes itself through its composable architecture, which includes layered filesystem composition for creating cached, sandboxed, or restricted storage views. It offers a copy-on-write