12 Repos
Links various storage systems to enable data exploration and analysis.
Distinguishing note: Focuses on the connectivity layer for diverse data sources, distinct from data modeling.
Explore 12 awesome GitHub repositories matching data & databases · Database Connectors. Refine with filters or upvote what's useful.
Context7 is an AI-powered documentation retrieval engine designed to provide developers and AI agents with real-time, context-aware access to technical documentation and code snippets. By integrating external library documentation as callable tools, the platform equips AI coding assistants with project-specific knowledge, helping to improve generation accuracy and reduce hallucinations during inference. The platform distinguishes itself through a robust security and governance framework that manages documentation as a centralized knowledge base. It employs a multi-source ingestion pipeline to
Links external database services to development environments to enable seamless data querying.
This project is a programmable, high-performance drawing engine designed for building collaborative whiteboards, diagramming tools, and infinite canvas applications. It provides a reactive graphics runtime that manages complex canvas interactions, viewport animations, and input handling through a unified signal-based API. The framework is built on a schema-driven data store that maintains application state in a strictly typed, centralized record system, enabling efficient UI updates and persistent data management. The engine distinguishes itself through a highly modular architecture that supp
Provides pluggable storage support including native SQLite persistence.
Metabase is a business intelligence platform designed to connect to various storage systems and relational databases for data exploration, visualization, and reporting. It provides a centralized environment where users can build queries through a graphical interface or raw code, transforming raw information into interactive dashboards and charts. The platform is built to support self-service analytics, allowing non-technical team members to extract insights without requiring deep knowledge of database syntax. The platform distinguishes itself through a metadata-driven modeling layer that abst
Link various storage systems and relational databases to enable comprehensive data exploration, analysis, and visualization.
Airflow is a platform for programmatically authoring, scheduling, and monitoring complex data pipelines. It functions as a workflow automation engine that manages the lifecycle of recurring business processes by executing code-defined task dependencies. By representing workflows as directed acyclic graphs, the system ensures that task execution order and data flow are explicitly defined and reliably maintained across distributed computing environments. The platform distinguishes itself through a highly modular, provider-based architecture that decouples core orchestration logic from external
Execute queries and perform data operations across multi-model and distributed database instances to interact with persistent storage layers during task execution.
Ray is a distributed computing framework designed to scale Python and Java applications across clusters by abstracting task scheduling and resource management. It functions as a resource-aware execution engine that manages task dependencies, placement, and fault tolerance across networked compute nodes. At its core, the system provides a stateful actor model, allowing developers to define classes that run in dedicated processes to maintain and mutate internal state across remote method calls. The framework distinguishes itself through a robust cross-language interoperability layer, enabling f
Queries SQL databases using standard connectors to ingest data directly into distributed datasets for large-scale processing.
Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e
Saves dataset contents to relational database tables using connection strings and native drivers.
This project is an automated trading and agentic workflow platform designed to orchestrate complex financial tasks through state-based graphs. It provides a comprehensive framework for building, deploying, and managing autonomous agents that execute multi-step analytical processes, monitor real-time market conditions, and perform high-speed trade execution. The platform distinguishes itself through a robust agentic plugin ecosystem that integrates directly with popular AI-powered development environments and command-line interfaces. It features a specialized financial analysis engine capable
Configures database connections using flexible drivers for local and production environments.
Druid is a database connection management and monitoring framework designed to maintain persistent, high-performance links between applications and relational databases. It functions as a resource manager that automates the lifecycle of connection pools, reducing the overhead associated with repeatedly opening and closing network connections. The project distinguishes itself through an integrated query analysis engine that decomposes database statements into structured components. This capability enables real-time security auditing, syntax validation, and metadata extraction, allowing for the
The project enables manual definition of connection parameters, including pool size, timeout limits, and validation rules, to establish stable links to storage.
Materialize is a streaming SQL database that continuously ingests live data from sources such as Kafka, Redpanda, PostgreSQL, and MySQL, and incrementally maintains materialized views. It provides a PostgreSQL-compatible query engine that accepts standard SQL over the PostgreSQL wire protocol, enabling any existing SQL client or BI tool to query real-time data. The system also includes a Model Context Protocol (MCP) server that exposes live materialized view data to AI agents, providing fresh context without polling. Materialize distinguishes itself through its ability to offer configurable c
Continuously ingests MySQL changes via GTID-based binlog replication with transactional consistency.
Arroyo is a high-performance stream processing platform built in Rust. It executes continuous SQL queries on streaming data with event-time semantics, enabling accurate windowed aggregations, joins, and stateful computations on unbounded event streams. The platform uses native Rust execution for high throughput and low latency, with periodic checkpointing for exactly-once fault tolerance and horizontal scaling across distributed workers. The system integrates deeply with Kafka for reading and writing topics with exactly-once delivery and supports change data capture (CDC) from MySQL and Postg
Ingests change data capture streams from MySQL and Postgres databases via Debezium.
Dinky is a real-time data platform for developing, deploying, and operating streaming applications based on Apache Flink. It functions as a SQL streaming IDE and a real-time data pipeline orchestrator, providing a web-based environment for writing and verifying queries with integrated logic plan visualization and lineage tracking. The platform acts as a distributed cluster manager, allowing the registration, monitoring, and administration of multiple processing clusters from a centralized interface. It also serves as a change data capture integration tool, synchronizing real-time database cha
Optimizes CDC source connections by merging multiple sources from one origin into a single node to prevent connection exhaustion.
Leafmap is a Python geospatial visualization library designed for creating interactive maps and performing geospatial analysis within Jupyter environments. It provides a comprehensive set of tools for building interactive map interfaces, browsing and visualizing SpatioTemporal Asset Catalog items, and connecting to PostGIS databases for spatial data rendering. The project distinguishes itself through a backend-agnostic rendering system that allows users to switch between different mapping engines while maintaining a consistent API. It features specialized capabilities for Cloud Optimized GeoT
Connects PostGIS databases to retrieve and render spatial data directly onto interactive maps.