Awesome GitHub RepositoriesMulti-Source Data Integration

This project is a structured educational resource and technical guide for designing and implementing autonomous systems using large language models. It provides a comprehensive curriculum and code samples focused on agentic design patterns, autonomous development, and the creation of systems capable of planning and executing multi-step tasks. The resource details the implementation of agentic retrieval-augmented generation, where models autonomously plan and refine data searches. It covers a wide array of orchestrators and design patterns, including metacognitive reflection for self-correctin

Minds Platform is an automation system and application platform designed for building and deploying custom AI tools and workflows. It functions as a machine learning integration layer and self-hosted orchestrator that connects predictive models and large language models to external data sources. The platform enables the execution of multi-step tasks that read and write data to automate reports and operational activities. It supports deployment across cloud, on-premises, and virtual private cloud environments to maintain control over models and data. Capabilities include event-driven workflow

DB-GPT is an AI-driven database management system that uses agentic reasoning to execute data tasks. It converts natural language prompts into executable database queries and combines structured database records with unstructured knowledge bases to provide grounded analysis. The system orchestrates multi-step reasoning chains that integrate database queries, custom scripts, and external tool calls. It allows for the packaging of domain knowledge into reusable analysis skills and executes generated code within sandboxed environments for system safety. The platform covers data orchestration ac

DB-GPT is an agentic data analysis platform and business intelligence AI that functions as a large language model data assistant. It provides a text-to-SQL interface and a sandboxed code execution environment to translate natural language into executable database queries and Python scripts. The platform utilizes iterative agentic reasoning to plan and execute multi-step data analysis workflows through tool calls. It features a modular skill-based extension system that allows domain knowledge and analysis workflows to be packaged into reusable functional components. The system integrates data

JimuReport is an open-source reporting and dashboard engine designed to be embedded directly into Spring Boot applications. Its core identity centers on generating data reports and full-screen dashboards from natural language descriptions, eliminating the need for manual design. The platform also provides a conversational query interface that translates plain-language questions into database queries, returning results as tables and charts without requiring SQL knowledge. What distinguishes JimuReport is its integration of AI skills that can be installed with a single command, enabling report

Feast is a machine learning feature store and MLOps data infrastructure layer. It provides a centralized system for managing and serving features across offline training and online production environments, utilizing an online feature serving layer for low-latency retrieval. The project centers on a feature registry that acts as a central catalog for defining, governing, and discovering feature services. It employs a unified data access layer to decouple feature retrieval from physical storage and includes a point-in-time data generator to create historically accurate training datasets that pr

🔥 基于大模型和 RAG 的智能问数系统，对话式数据分析神器。Text-to-SQL Generation via LLMs using RAG.

DataX Web is a web-based management platform for scheduling, building, executing, and monitoring distributed data synchronization jobs powered by DataX. It provides a visual console for creating and managing DataX tasks without manual JSON configuration, with a distributed executor cluster that auto-registers worker nodes and supports configurable routing and blocking strategies for task distribution. The platform offers cron-based task scheduling with dynamic start, stop, and immediate status changes, along with incremental sync capabilities that pass dynamic parameters to extract only new o

Data on COVID-19 (coronavirus) cases, deaths, hospitalizations, tests • All countries • Updated daily by Our World in Data

Fluvio ist eine verteilte Event-Streaming-Plattform und eine Cloud-native Streaming-Engine, die für das Sammeln, Persistieren und Replizieren von Echtzeit-Datenströmen über einen verteilten Cluster hinweg entwickelt wurde. Sie fungiert als Echtzeit-Datenpipeline für den Aufbau zustandsbehafteter Workflows, die Daten zwischen externen Quellen und Senken aufnehmen, anreichern und exportieren. Die Plattform zeichnet sich durch die Verwendung von WebAssembly zur Ausführung kompilierter Module für In-Line-Datentransformationen und -filterung aus. Dies ermöglicht die Ausführung benutzerdefinierter Geschäftslogik, um Informationen während der Übertragung umzuformen, ohne den Cluster neu starten zu müssen. Das System deckt ein breites Spektrum an Funktionen ab, einschließlich connector-basierter Datenaufnahme aus externen Protokollen, log-strukturierter unveränderlicher Speicherung mit Zero-Copy-IO und horizontaler Clusterskalierung. Es unterstützt die Erstellung komplexer ereignisgesteuerter Pipelines, die zustandsbehaftete Verarbeitung, fensterbasierte Aggregationen und partitionierte Datenverteilung nutzen. Die Engine kann als leichtgewichtiges Binärprogramm auf diversen Systemarchitekturen bereitgestellt werden, einschließlich ARM64-IoT-Geräten für die Datenverarbeitung am Edge.

Daft is a distributed dataframe library and multimodal data processor designed to handle large-scale structured and unstructured data. It functions as a vectorized execution engine that processes tables alongside images, audio, and video, utilizing a unified schema to manage diverse data types. The project distinguishes itself by combining distributed data engineering with large-scale AI inference. It provides an AI data pipeline for batch-optimizing model prompts and generating high-dimensional text embeddings, while utilizing zero-copy memory sharing to execute custom Python functions witho

MineContext is a context management system designed to collect, store, and retrieve multimodal data to build targeted context windows for large language models. It functions as an orchestration tool and retrieval augmented generation framework that utilizes a local vector data store to index documents and enable similarity searches. The system differentiates itself through a multimodal context collector that gathers information from screen captures, files, and version control systems. It provides mechanisms for proactive information retrieval, extracting summaries and activity records from ca

Amundsen is a data catalog and discovery platform that provides a centralized directory for indexing tables and dashboards. It functions as a metadata management system and search engine, allowing users to locate and understand available data assets across diverse distributed sources. The platform includes capabilities for data lineage tracking to map the origin and movement of datasets between systems. It also serves as a data profiling tool, calculating distribution and quality statistics for individual table columns to provide automated insights into the nature of the data. The system man

CSGHub ist eine Plattform für Modellmanagement und ein Datensatz-Register, das für das Speichern, Verteilen und Verwalten von Large Language Models entwickelt wurde. Es bietet einen zentralen Hub für KI-Assets, der über eine Weboberfläche, ein Software Development Kit (SDK) und die Kommandozeile zugänglich ist. Die Plattform fungiert als selbstgehostete KI-Infrastruktur und ermöglicht die On-Premise-Installation innerhalb privater Netzwerke für sichere und Offline-Operationen. Sie beinhaltet ein Modellspeichersystem, das Python-SDK-Kompatibilität mit dem Hugging-Face-Ökosystem beibehält, um die Migration bestehender Skripte zu erleichtern. Das System deckt den gesamten Modell-Lifecycle ab, einschließlich Versionierung, Verteilung und der Vorbereitung von Trainingsdaten durch intelligente Annotation und Multi-Source-Synchronisation. Die organisatorische Sicherheit wird durch Enterprise-Zugriffskontrolle und rollenbasierte Berechtigungen verwaltet, um den Zugriff auf sensible Machine-Learning-Assets einzuschränken.

Mapnik ist eine kartografische Rendering-Bibliothek und Karten-Rendering-Engine, die entwickelt wurde, um Daten aus geografischen Informationssystemen in visuelle Karten und druckbare Layouts umzuwandeln. Sie dient als Werkzeug zur Visualisierung räumlicher Daten, das Symbologie- und Styling-Regeln auf diverse geografische Datenquellen anwendet. Das Projekt nutzt ein XML-Kartenkonfigurations-Framework, um das visuelle Erscheinungsbild von Karten zu definieren, was die Trennung von Styling-Logik und Layout-Eigenschaften von der Kern-Rendering-Engine ermöglicht. Ihre Fähigkeiten decken die Visualisierung geografischer Informationen und das Management kartografischer Stile ab. Das System integriert räumliche Daten aus mehreren Formaten über eine Plugin-Architektur und verwaltet die Transformation von Koordinaten, um sicherzustellen, dass räumliche Daten auf Kartenbildern korrekt ausgerichtet sind.

Langroid is a multi-agent orchestration framework and tool integration suite designed for building complex AI applications. It serves as a multi-modal integration layer that connects diverse local and remote language models with an agentic retrieval-augmented generation system. The project distinguishes itself through a collaborative message-exchange paradigm, allowing specialized agents to delegate tasks hierarchically and coordinate via structured communication. It features an advanced state management system for conversational AI, including the ability to rewind and prune conversation hist

This project is a client-side data management library and query orchestrator designed to synchronize remote server state with local client state. It functions as a type-safe state manager and cache orchestrator that coordinates data loading across diverse backends, including REST, GraphQL, and WebSockets. The system distinguishes itself through a durable workflow engine for executing asynchronous functions with persisted state and deterministic replay. It also provides a standardized AI integration adapter to connect large language models to application data, supporting real-time response str