Awesome GitHub RepositoriesEnterprise Data Services

Refine is a React-based framework for building data-intensive internal tools, admin panels, and B2B applications. It functions as a data-driven UI library and a headless admin panel generator that connects frontends to external backend services using standardized logic for state management and network request handling. The project decouples business logic from the presentation layer, allowing any custom design system or interface library to be applied to the application. It includes a CRUD application generator that automatically creates user interfaces for managing records based on the struc

This project is an AI agent orchestration platform that provides a visual environment for building, testing, and deploying complex automation workflows. It functions as a low-code development interface where users can chain discrete functional blocks into dependency-aware pipelines to integrate artificial intelligence with external data and services. The platform supports the creation of intelligent conversational agents, automated business processes, and multi-service API orchestrations within a unified workspace. The platform distinguishes itself through its event-driven integration engine,

This project is a distributed, document-oriented database system designed to store information in flexible, hierarchical structures. It supports horizontal scaling through automated sharding and maintains high availability across global clusters using a multi-node replication protocol. By executing multi-document operations as atomic units, the system ensures data integrity and consistency across distributed environments. The platform distinguishes itself by integrating advanced vector-based indexing, which enables semantic similarity searches alongside traditional geospatial and lexical quer

This platform is a modular, metadata-driven framework designed for building custom business applications and data management systems without traditional coding. It functions as a low-code environment where data models, user interfaces, and business logic are defined through visual configurations rather than hardcoded views. The architecture supports multi-tenant isolation, allowing multiple independent applications to run within a single shared memory space while maintaining strict logical separation of data and configurations. What distinguishes this system is its deep integration of artific

Airbyte is a data integration platform designed to synchronize information between diverse applications, databases, and data warehouses. It functions as an extract, transform, and load orchestrator that manages automated data movement workflows across cloud, on-premise, and hybrid environments. The platform provides a standardized interface for connectors, enabling the movement of structured and unstructured data while maintaining stateful checkpoints for reliable incremental syncing. The platform distinguishes itself through a containerized architecture that isolates connectors to prevent de

Weaviate is a cloud-native vector database and distributed vector store designed to save high-dimensional vectors alongside structured data. It functions as a hybrid search engine that combines vector similarity, keyword matching, and structured metadata filtering within a single query. The system is optimized for retrieval-augmented generation, integrating vector search with generative AI and reranking to power question-and-answer workflows. It distinguishes itself through the ability to merge semantic search with traditional keyword queries and structured metadata filters to improve result

GenericAgent is an LLM agent framework and autonomous system controller designed to manage local systems, web browsers, and hardware interfaces through action and observation loops. It functions as a tool orchestrator that routes model calls to local executors, enabling the automation of complex tasks on a host machine. The project is distinguished by its self-evolving AI agent capabilities, which convert successful execution paths into reusable procedural scripts and skill trees to reduce future reasoning overhead. It employs a context optimization engine that utilizes layered memory hierarc

DataHub is a metadata management platform designed to unify technical, operational, and business context across diverse data ecosystems. By utilizing a graph-based metadata model and an event-driven ingestion architecture, it creates a centralized source of truth that maps complex data relationships, lineage, and ownership. This foundational framework enables organizations to maintain a synchronized view of their data landscape, supporting both human-led discovery and automated data operations. The platform distinguishes itself through its focus on grounding artificial intelligence and autono

Midway 是一套用于 Node.js 的后端框架，旨在构建企业级微服务和全栈应用。它以依赖注入架构为核心，使用控制反转（IoC）容器和装饰器来组织复杂的业务逻辑并维护可扩展的服务器端结构。 该框架提供针对 HTTP、gRPC 和 WebSocket 的多协议支持，允许单一环境处理多样化的服务间网络需求。它还包括一个多运行时适配器层，使应用能够部署在传统容器和各种无服务器云平台上。 其他功能包括用于模块化扩展的组件化插件系统、用于请求处理的中间件执行流水线，以及用于前端界面与后端逻辑之间统一 API 连接的工具。该系统还结合了请求数据验证和标准化的错误处理。

Pachyderm is a containerized, versioned, and lineage-tracked data pipeline platform that runs natively on Kubernetes. It combines a distributed file system backend with immutable data versioning, so every commit to a data repository creates an auditable snapshot, and every pipeline step executes as an isolated container. The platform is defined by a data-centric pipeline model where pipelines are specified by their input and output data repositories rather than explicit task sequences, and provenance is recorded as a directed acyclic graph of commits linking output data to its input sources an

Fleet is an open-source device management platform that provides centralized control over computing devices running macOS, Linux, Windows, Chromebooks, iOS, and Android. It enables organizations to enroll devices, collect real-time telemetry, enforce security compliance policies, and manage software remotely from a single system. The platform can be deployed as a single binary, run locally for testing, or scaled horizontally across cloud infrastructure on AWS, Kubernetes, GCP, or Render, with support for high availability through database replication and load balancing. The platform distingui

Fluvio is a distributed event streaming platform and cloud-native streaming engine designed for collecting, persisting, and replicating real-time data streams across a distributed cluster. It functions as a real-time data pipeline for building stateful workflows that ingest, enrich, and export data between external sources and sinks. The platform is distinguished by its use of WebAssembly to execute compiled modules for in-line data transformations and filtering. This allows for the execution of custom business logic to reshape information in motion without requiring a restart of the cluster.

Scraperr 是一个自托管的 Web 抓取和爬虫平台，旨在通过 XPath 选择器从网站提取结构化数据。它作为一个容器化系统，通过队列管理抓取任务，并使用人工智能分析结果内容。 该项目的特色在于其 Kubernetes 原生架构，允许通过包管理器进行可扩展的部署和管理。它包括一个能够进行域名级爬取的爬虫引擎，以及一个使用人工智能查询提取出的 Web 内容的数据分析器。 该平台涵盖了广泛的功能，包括自动化数据提取、批量 Web 爬取和媒体文件下载。它提供了将抓取数据可视化为表格、配置自定义请求头以模拟浏览器身份，以及将结果导出为 CSV 或 Markdown 格式的工具。 该应用支持通过 Kubernetes 部署配置进行自定义安装参数和版本更新。

Briefer is an interactive data notebook platform and business intelligence dashboard tool used for collaborative data analysis and reporting. It provides a containerized environment for building reports that combine SQL, Python, and Markdown with native visualizations. The platform features an integrated code assistant that uses large language models to generate SQL and Python snippets from natural language prompts. It is designed as a Kubernetes data application, deploying via Helm charts to manage isolated compute environments and ensure separate resources per page through pod-based isolati

Vald is a distributed, cloud-native search engine designed for high-dimensional vector data. It functions as an approximate nearest neighbor search platform, enabling the identification of similar data points across massive datasets through horizontal scaling and distributed indexing. The system is built for container orchestration environments, utilizing custom resource controllers to automate cluster lifecycle management and infrastructure state. It employs graph-based indexing to perform rapid similarity lookups and supports zero-downtime operations by decoupling index construction from qu

这是一个开源、自托管的平台，专为企业级数据收集、问卷构建和自动化运营工作流而设计。它为组织提供了一个全面的环境，在维护完全数据主权的同时，在其私有基础设施内管理交互式问卷、在线考试和复杂的信息收集项目。 该平台通过专注于高并发、可扩展的部署和细粒度的组织控制脱颖而出。它具有一个动态的拖放式构建器，支持基于逻辑的分支和 AI 辅助内容生成，允许创建能够适应用户输入的复杂表单。除了简单的数据录入，系统还集成了先进的业务功能，如多阶段审批工作流、安全支付处理和自动化文档生成，所有这些都通过中心化的、基于角色的访问控制系统进行管理。 该系统涵盖了广泛的能力范围，包括实时数据管理、多平台分发，以及用于人才评估和资源调度的专业工具。它支持多种存储后端，并提供强大的提交跟踪功能，确保收集的信息保持安全且易于分析。该应用专为容器化部署而设计，简化了私有云或本地环境的设置过程。

Flow-like is a workflow orchestration engine designed for building and executing strictly typed automated processes. It provides a secure, sandboxed runtime environment that supports the integration of local artificial intelligence models, allowing for the processing of data entirely on host hardware without reliance on external cloud services. The platform distinguishes itself through its event-sourced execution tracing, which records every state change and data movement to enable full auditability and the replay of past processes. It combines this with a hybrid storage system that integrate