7 个仓库
Systems that collect and consolidate information from multiple disparate sources into a unified format.
Distinct from Bug Bounty Report Mappings: Existing candidates focus on reporting templates or specific CVE mappings rather than the process of aggregating feeds into a dataset.
Explore 7 awesome GitHub repositories matching data & databases · Data Aggregators. Refine with filters or upvote what's useful.
Edict is a multi-agent orchestration system and framework designed to coordinate specialized large language model agents. It functions as a workflow designer and orchestrator that decomposes complex objectives into structured plans, using directed acyclic graphs and role-based hierarchies to execute sub-tasks. The system is distinguished by its event-driven architecture, utilizing a publish-subscribe event bus and transactional outbox to manage agent communications and task transitions. It features a dedicated skill management system that allows for the importation, updating, and sandboxed ex
Collects and summarizes information from multiple third-party platforms into consolidated datasets for agent processing.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Sources events from streaming platforms into a unified SQL-queryable view with tenant isolation for teams.
hackerone-reports is a bug bounty dataset aggregator and vulnerability intelligence tool designed to scrape and parse public security reports from the HackerOne platform. It functions as a security report parser that transforms raw bug bounty feeds into structured datasets for analysis. The system automates the collection of public vulnerability reports to identify common security patterns and facilitate security research. It specializes in organizing these disclosures by bug type, payout amount, and target program to enable vulnerability trend analysis. The tool covers capabilities for scra
Collects high-impact security reports from public feeds and organizes them by type, payout, and program.
Akka.NET is an actor model framework used for building concurrent and distributed applications. It functions as a distributed computing platform and state manager that enables isolated actors to communicate via asynchronous message passing, ensuring thread-safe state management without manual locks. The project is distinguished by its decentralized coordination capabilities, including a distributed state manager that uses sharding and dynamic rebalancing to maintain high availability. It incorporates an event sourcing engine that persists state as a sequence of events in an append-only log an
Collects and consolidates information from groups of actors to generate reports or status queries.
m3 是一个分布式时间序列数据库,专为高分辨率指标和高基数数据管理而设计。它作为一个可扩展的存储系统和多集群查询引擎,提供了一个分布式指标聚合器,能够在数据提交到存储之前进行降采样和汇总。 该项目以其使用 etcd 进行节点成员管理和分片放置的协调集群模型而脱颖而出。它支持多种摄取协议,包括 Prometheus 远程写入协议、InfluxDB 行协议和 Graphite Carbon 纯文本协议,并提供与 PromQL 和 Graphite 兼容的查询接口。 该系统涵盖了广泛的功能领域,包括列式时间序列存储、同步数据复制和分布式查询扇出。它集成了数据生命周期自动化、基于法定人数 (Quorum) 的一致性调整,以及基于标签的序列索引,以在隔离的命名空间中保持数据完整性和检索速度。 集群编排和组件放置通过自动化工具和 Operator 进行管理,以确保高可用性和均衡的数据分布。
Provides a mechanism to output aggregated metrics to long-term storage for persistence.
mmocr 是一个基于 PyTorch 的光学字符识别(OCR)框架,旨在训练和部署文本检测、识别和关键信息提取模型。它作为一个全面的场景文本检测和识别工具箱,提供用于定位文本区域并将视觉文本转换为机器编码字符串的专用库。 该项目的独特之处在于用于关键信息提取的研究框架和高级文本定位功能。这些包括使用 Transformer 的基于点的定位,以及使用参数化贝塞尔曲线来识别和转录任意形状的文本。 该框架涵盖了广泛的计算机视觉功能,包括用于增强和标准化多样化 OCR 数据集的流水线管理、具有分布式扩展的模型训练,以及使用标准 OCR 指标的性能评估。它还提供用于几何多边形操作和结果可视化的实用程序,以便根据真实标注审计预测。 该系统使用 Python 实现,并支持通过 Docker 环境打包进行安装。
Aggregates multiple distinct data sources into a single unified dataset for training or evaluation.
Open Health 是一个安全的健康数据平台和个人健康记录管理器,旨在将分散的医疗记录收集并存储在单一的中心化位置。它作为一个 AI 驱动的医疗对话工具和数据解析器,将非结构化的健康文档转换为结构化文件,以便进行分析和处理。 该平台集成了大语言模型,通过将结构化的个人医疗记录注入模型上下文来提供个性化的健康指导。这使得系统能够根据用户的特定健康数据生成量身定制的医疗建议。 该系统通过用户账户注册和身份验证来管理患者数据,以保护敏感信息。它还提供了医疗记录结构化以及从多个来源整合健康数据的功能。
Provides a system to collect and consolidate disparate medical records into a unified format for a comprehensive health overview.