113 repositorios
Tools that facilitate the connection and exchange of information between disparate software systems and data sources.
Explore 113 awesome GitHub repositories matching data & databases · Data Integration. Refine with filters or upvote what's useful.
AutoGPT is an orchestration platform designed for building, managing, and deploying autonomous agents. It provides a visual canvas-based environment where users can assemble agents by connecting modular blocks that represent actions, data flows, and conditional logic. The platform supports the entire agent lifecycle, including task scheduling, execution monitoring, and configuration management, while offering a marketplace for discovering and sharing community-built workflows. The project includes a legacy framework for command-line agent execution and an extensible component system for devel
Integrates relational database storage with automated schema generation and migration capabilities.
Bun is a high-performance runtime environment designed to execute JavaScript and TypeScript applications with minimal latency and high throughput. Built on a native core implemented in Zig, it provides a unified execution engine that leverages JavaScriptCore for efficient memory management and low-latency startup. The project functions as an all-in-one toolchain, integrating a native bundler, transpiler, package manager, and test runner into a single command-line interface. What distinguishes Bun is its focus on native system integration and developer productivity. It features a high-performa
Groups multiple database operations into atomic transactions to ensure data consistency and integrity during complex updates.
Superset is a web-based business intelligence platform designed for data exploration, visualization, and interactive dashboarding. It functions as a query-driven analytics engine that connects to various SQL databases, allowing users to perform ad-hoc analysis, define virtual metrics, and build complex data visualizations through a centralized interface. The platform distinguishes itself through a robust semantic layer that transforms raw database schemas into calculated columns and virtual metrics, enabling consistent business logic across an organization. It features a plugin-based visualiz
Handles database connection credentials and drivers to link external SQL data sources to a centralized management platform.
OpenBB is a financial data platform and investment research terminal designed to aggregate, normalize, and distribute market data across analytical workflows. It functions as a comprehensive ecosystem that bridges disparate financial data providers with custom applications, spreadsheets, and internal modeling infrastructure. The platform distinguishes itself through a provider-based data abstraction layer that normalizes heterogeneous financial APIs into a consistent, schema-driven format. This architecture supports quantitative research automation and the construction of interactive, widget-
Consolidates market data from multiple external providers into a single, standardized interface for research and analysis.
Prometheus is a comprehensive monitoring and alerting platform designed to track infrastructure health and application performance. It functions as a time series database that ingests, indexes, and queries high-frequency numerical data points. By utilizing a pull-based model, the system periodically collects multi-dimensional metrics from monitored targets, storing them in an optimized block storage format that supports high-throughput ingestion and efficient historical analysis. The platform distinguishes itself through a specialized query engine that enables real-time analysis of performanc
Transmits data to external storage systems using standardized protocols to ensure reliable communication across distributed services.
NocoDB is a visual platform that transforms relational databases into collaborative, spreadsheet-style workspaces. By acting as a headless database backend, it provides a unified environment for designing database structures, managing record relationships, and interacting with data without requiring manual SQL queries. The platform normalizes interactions across various SQL and NoSQL data sources, allowing users to manage complex datasets through a centralized interface. The project distinguishes itself by automatically generating RESTful and GraphQL APIs from existing database schemas, enabl
Handles connections to external data sources, allowing for the integration of high-volume datasets into a centralized environment.
Memos is a self-hosted, container-native knowledge management platform designed for capturing and organizing personal notes. It functions as a private workspace where users can create content using markdown, tags, and media embeds to streamline daily productivity. The system is built to be deployed as a portable service, allowing individuals to maintain full control over their data and hosting environment. Beyond its core note-taking capabilities, the platform operates as a headless content service that exposes a structured RESTful API. This interface allows for programmatic interaction, enab
Link external database services by defining connection strings and drivers through environment variables.
Appwrite is a backend-as-a-service platform that provides a unified development environment for building full-stack applications. It integrates essential infrastructure components—including authentication, databases, storage, and serverless functions—into a single, centralized interface to simplify application development and resource management. The platform distinguishes itself through a container-based microservices architecture that ensures consistent execution across diverse infrastructure. It features a versatile connectivity layer that links frontend applications with third-party servi
Provides seamless connectivity to MongoDB clusters for reading and writing documents within a unified backend environment.
LevelDB is an embedded database library and persistent storage engine that provides a sorted key-value store. It uses a log-structured merge-tree architecture to map byte arrays to values, running directly within a process to provide storage without the need for a separate server process. The system is distinguished by its use of custom comparison functions to define key ordering, enabling efficient range scans and sequenced lookups. It ensures data reliability through atomic batch execution, consistent snapshot generation, and log-based recovery after failures. The engine covers broad capab
Supports grouping multiple mutations into a single atomic batch to ensure data consistency.
This project is a comprehensive reference collection of practical implementation examples and patterns for building applications with Spring Boot. It serves as a Java web application template and a showcase for developing functional web services featuring REST endpoints, template engines, and global exception handling. The repository distinguishes itself by providing detailed demonstrations of enterprise-grade features, including distributed locking, task scheduling, and asynchronous message exchange using brokers like RabbitMQ. It also includes reference implementations for automated API doc
Implements connectors for reading from and writing to MongoDB document database instances.
Redash is a self-hosted analytics platform and SQL data visualization tool. It provides a web-based SQL query editor for writing, executing, and scheduling database queries, and functions as a business intelligence dashboard for monitoring metrics via visual widgets. The platform distinguishes itself through its data source connectors, which integrate with various SQL, NoSQL, and API-based stores to retrieve information for analysis. It enables self-service analytics by allowing users to run queries with dynamic parameters and supports shared data reporting via public links or embedded dashbo
Provides interfaces for establishing connections to a wide variety of external SQL, NoSQL, and API-based data sources.
OpenZeppelin Contracts is a library of modular, secure, and reusable smart contract components designed for the development of decentralized applications. It provides a foundational framework for building standard-compliant contracts, offering battle-tested implementations for token standards, access control, and common utility patterns. The project distinguishes itself through its comprehensive support for complex architectural patterns, including proxy-based upgradeability, role-based access control, and account abstraction. It enables developers to implement modular logic injection via hoo
Allows bundling multiple function calls into a single transaction for atomic execution.
React-admin is a framework for building data-driven administrative interfaces that connect to REST or GraphQL backends. It provides a comprehensive suite of tools for managing the full lifecycle of administrative applications, including resource-oriented routing, declarative form scaffolding, and context-driven state management. By utilizing a modular adapter-based architecture, the framework abstracts backend communication, allowing developers to build consistent CRUD interfaces that handle data fetching, authentication, and synchronization automatically. The project distinguishes itself thr
Implements standardized methods to connect administrative interfaces with any REST or GraphQL backend.
DataEase is an open-source, self-hosted business intelligence platform designed for building interactive data visualizations and managing analytical reporting. It provides a centralized environment where users can construct dashboards through a drag-and-drop interface, connecting to diverse data sources including relational databases, data warehouses, and external APIs. The platform distinguishes itself through its focus on embedded analytics and enterprise-grade governance. It allows for the seamless integration of charts, dashboards, and management modules into third-party web applications
Integrates with a wide range of relational databases, data warehouses, data lakes, flat files, and API endpoints.
This project is a Python-based framework that functions as a generative AI agent for programmatic data analysis. It enables users to interact with structured data sources through natural language prompts, translating these requests into executable code to perform analysis, data cleaning, and visualization. By maintaining conversational context across multi-turn interactions, the system allows for iterative exploration and the building of complex data narratives. The framework distinguishes itself through a robust semantic layer and secure execution model. It maps raw datasets to descriptive m
Connects to local files, relational databases, and cloud platforms to unify data for analysis.
This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d
Groups multiple document updates into a single atomic write to ensure complex state changes are applied consistently.
This project provides a framework for managing multi-agent systems, designed to automate complex software development, infrastructure, and business workflows. It functions as a multi-agent workflow orchestrator that routes tasks to domain-specific workers while maintaining state persistence and infrastructure automation. By leveraging large language models, the system decomposes high-level objectives into actionable plans, ensuring that complex operations are executed with consistency and reliability. The framework distinguishes itself through its hierarchical agent registry and policy-driven
Connects clinical and administrative data sources to facilitate interoperability and reporting.
Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep
Stores and retrieves secure connection details for external database services using reusable blocks managed through a centralized interface.
Letta is a framework for building, deploying, and managing autonomous AI agents that maintain persistent state across long-term interactions. It provides a comprehensive suite of primitives for defining agents with configurable personas, modular memory blocks, and tool-use capabilities, enabling them to retain user preferences and conversation history over extended sessions. The platform distinguishes itself through its advanced memory management and orchestration capabilities. It allows agents to autonomously update their own memory, perform retrieval-augmented generation, and coordinate com
Connects external data sources and document folders to agents for information retrieval.
Claude Code Templates is a comprehensive framework for orchestrating specialized AI agents and automating development workflows within local environments. It provides a structured system for defining, configuring, and deploying AI personas that handle specific technical tasks, ranging from backend architecture and frontend implementation to security auditing and infrastructure management. The project distinguishes itself through a configuration-driven approach that allows teams to standardize development environments and share reusable agent definitions across projects. It includes a robust C
Connects development environments to cloud database services for streamlined migration and synchronization.