Awesome GitHub RepositoriesPersistent Storage Backends

This project is a Python framework for building autonomous, event-driven agent systems. It provides a unified runtime for orchestrating multi-agent workflows, managing persistent conversation state, and executing code within secure, isolated sandbox environments. The framework is designed to handle complex task delegation, allowing agents to invoke other agents as tools while maintaining context across multi-turn interactions. The framework distinguishes itself through its deep integration with the Model Context Protocol, enabling agents to connect to external data sources and remote services

Colly is a high-performance web scraping framework designed for the automated extraction of structured data from websites. It provides a programmable toolkit that manages the complexities of large-scale data collection, including concurrent request orchestration, automatic cookie handling, and robots.txt compliance. By utilizing an asynchronous execution model, the engine maintains high throughput while preventing resource exhaustion during recursive or distributed crawling tasks. The framework is distinguished by its modular, event-driven architecture, which allows developers to hook into sp

Crawlee is a web scraping framework designed for building scalable, reliable, and distributed data extraction pipelines. It provides a unified interface for managing headless browser automation and lightweight HTTP requests, allowing developers to handle complex web navigation, dynamic content rendering, and large-scale data collection within a single, modular architecture. The project distinguishes itself through its resource-aware concurrency controller, which dynamically scales task execution based on real-time CPU and memory usage to prevent host machine exhaustion. It also features a rob

FastMCP is a Python framework designed for building servers that expose functions, resources, and prompts to AI models using the Model Context Protocol. It simplifies the development process by automatically deriving tool metadata, input schemas, and documentation directly from Python function signatures and type hints. The framework provides a unified container for managing these components, allowing developers to build modular applications that integrate seamlessly with AI assistants. The project distinguishes itself through its support for interactive, server-defined user interface compone

Mastra is an orchestration framework designed for building, deploying, and managing autonomous AI agents and multi-agent systems. It provides a comprehensive suite of primitives for creating resilient AI applications, including durable workflow orchestration, event-driven agent loops, and semantic memory management. By integrating these core components, the platform enables developers to build complex, multi-step processes that can reason about goals and execute tasks without manual intervention. The framework distinguishes itself through its focus on observability and secure, isolated execut

Nautilus Trader is a high-performance algorithmic trading framework built in Rust, designed for the development, backtesting, and live execution of automated trading strategies. It provides a comprehensive platform for managing multi-asset portfolios and interacting with diverse financial markets through a standardized connectivity suite. The system is engineered to handle high-frequency data processing and complex order execution while maintaining precise numerical accuracy across various asset classes. The framework distinguishes itself through an architecture centered on deterministic even

This project is a cross-platform chatbot framework designed to integrate generative artificial intelligence models into messaging services. It provides a unified architecture for building and deploying automated bots that maintain consistent conversation state, user identity, and interaction logic across multiple messaging platforms from a single codebase. The framework distinguishes itself through a modular adapter system that normalizes platform-specific webhooks and events into a standardized internal schema. It includes a comprehensive toolkit for constructing rich, interactive user inter

Optuna is a Python-based hyperparameter optimization framework designed to automate the search for optimal machine learning model configurations. It functions as a Bayesian optimization library that systematically tests parameter combinations to maximize or minimize objective functions, streamlining the model development process through iterative evaluation. The project distinguishes itself through a define-by-run dynamic construction model, which allows users to build complex, conditional search spaces using standard programming logic. Its architecture is highly modular, featuring a pluggabl

Grist is a relational spreadsheet platform that combines the flexibility of a spreadsheet with the power of a relational database. At its core, it manages structured data across multiple linked tables, using a relational database engine to organize information while providing a familiar grid interface. The platform supports Python-based formulas for complex calculations and data transformations, with automatic recalculation when referenced cells change. The system is designed for self-hosted deployment, storing data in either portable SQLite files or enterprise-grade PostgreSQL databases. It

This project is a full-stack project management web application designed for organizing work and managing project workflows. It consists of a Node.js TypeScript REST API for server-side business logic and a React component UI library for the frontend interface. The application implements a board-based task visualization system with customizable status workflows. It provides mechanisms for issue workflow management, including task prioritization and team member assignment to track responsibilities and progress. The system is built with a decoupled client-server architecture using full-stack T

FerretDB is an open-source database emulator and protocol translator that mimics a MongoDB environment to support existing drivers and client tools on a relational backend. It functions as a stateless database proxy that converts binary wire protocol messages into SQL statements, allowing a relational engine to handle document-oriented requests. The project serves as a migration tool for moving applications from MongoDB to PostgreSQL without rewriting queries or changing client drivers. It achieves this by using PostgreSQL as a document store, storing and querying BSON documents through a tra

Dex is an OpenID Connect provider and identity federation proxy that translates authentication signals from various upstream sources into a unified OpenID Connect interface. It functions as a multi-protocol identity broker, enabling client applications to implement a single standard while delegating user verification to external identity providers. The project distinguishes itself through a pluggable connector architecture that bridges disparate protocols including LDAP, SAML, and OAuth2. It provides specific integrations for services such as GitHub, Google, GitLab, and Microsoft, while offer

EmDash is an open-source content management system built on Astro that combines a visual admin panel with a plugin-driven architecture and server-side rendering. It provides a complete content management system with structured content modeling, a rich text editor using Portable Text format, and a TypeScript API for type-safe content queries. The system supports authentication through passkeys, OAuth 2.1, and external providers, with role-based access control and fine-grained permission scopes. What distinguishes EmDash is its plugin development framework, which supports both native plugins ru

dtm is a distributed transaction framework and polyglot transaction coordinator designed to maintain data consistency across microservices. It functions as a Saga orchestration engine and a two-phase message coordinator, ensuring that multi-service operations either succeed completely or roll back to a consistent state. The project distinguishes itself by supporting multiple consistency patterns, including Saga, TCC, XA, and outbox patterns, allowing users to select the appropriate model for their specific application requirements. It provides a polyglot integration layer via HTTP and gRPC, e

Cadence is a distributed workflow orchestration engine designed to execute long-running, asynchronous business logic with built-in durability and resilience across distributed systems. It functions as a stateful process manager that ensures processes resume from their last known state following system crashes or network outages. The platform utilizes a distributed task queue to manage work across independent worker nodes and supports persistence via SQL or Cassandra backend storage. It includes a workflow visualization dashboard for inspecting execution histories and state traces, alongside a

Gorush is a dedicated notification dispatch server used to send real-time alerts to iOS and Android mobile devices. It functions as a centralized infrastructure for hosting a notification API and managing the delivery of push messages through high-throughput queuing. The system supports both gRPC streams and HTTP endpoints to receive notification triggers and includes a command-line interface for triggering alerts without a running server process. It manages network traffic through proxy routing via HTTP or SOCKS5 and secures server communication using automated TLS certificate management. O

TeslaMate 是一个自托管系统，用于将来自 Tesla 车辆的遥测数据记录并存档到本地数据库。它作为一个车辆数据记录器和遥测仪表板，旨在在本地维护完整的驾驶和车辆状态数据历史记录。 该系统包括一个车辆充电成本跟踪器，用于根据能源使用量和电价监控充电会话的财务成本。它还具有一个用于定义自定义地理边界的地理围栏位置管理器，以自动标记访问过的位置，以及一个家庭自动化数据代理，将实时车辆状态发布到本地代理。 该项目提供了电池健康分析、能耗监控和车辆指标可视化的功能。它进一步支持历史数据导入、自定义位置管理以及与外部智能家居工具的集成。

Floccus is a browser extension that synchronizes bookmarks across different browsers and devices. It runs directly within the browser to access the native bookmark API, and offers both one-way and two-way sync modes to control how changes flow between devices. The extension includes an optional end-to-end encryption layer that encrypts bookmark data client-side before transmission, ensuring only the user can decrypt it on other devices. The extension supports multiple storage backends through a unified sync interface, including WebDAV, Google Drive, and Nextcloud, allowing users to choose bet

PyMySQL is a MySQL database connector and SQL database driver for Python environments. It serves as a client library that enables Python applications to establish network connections and interact with MySQL database servers. The library is a pure Python implementation of the MySQL client-server wire protocol. This design allows for MySQL integration and data management without requiring native C extensions. The project provides capabilities for database connectivity, query execution, and secure access control. It follows the DB-API 2.0 specification to provide a consistent interface for mana

Pholcus 是一个用 Go 编写的分布式网络爬虫框架，专为高并发数据提取而设计。它作为一个分布式爬虫编排器和动态数据提取引擎，利用服务器-客户端架构在多个节点间协调任务。 该系统集成了无头浏览器引擎来渲染动态内容并执行 JavaScript，从而能够从单页应用 (SPA) 中提取数据。它具有用于配置爬虫参数和监控执行进度的 Web 管理界面，并支持通过热重载配置文件更新提取规则，无需重启系统。 流量管理通过代理池轮换和请求随机化来处理，以规避机器人检测并避免速率限制。该框架包括基于状态的检查点恢复功能，可在故障后恢复任务，并提供可插拔的存储适配器，将提取的数据导出到数据库、消息队列或文件中。