7 repositorios
Systems that collect and consolidate information from multiple disparate sources into a unified format.
Distinct from Bug Bounty Report Mappings: Existing candidates focus on reporting templates or specific CVE mappings rather than the process of aggregating feeds into a dataset.
Explore 7 awesome GitHub repositories matching data & databases · Data Aggregators. Refine with filters or upvote what's useful.
Edict is a multi-agent orchestration system and framework designed to coordinate specialized large language model agents. It functions as a workflow designer and orchestrator that decomposes complex objectives into structured plans, using directed acyclic graphs and role-based hierarchies to execute sub-tasks. The system is distinguished by its event-driven architecture, utilizing a publish-subscribe event bus and transactional outbox to manage agent communications and task transitions. It features a dedicated skill management system that allows for the importation, updating, and sandboxed ex
Collects and summarizes information from multiple third-party platforms into consolidated datasets for agent processing.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Sources events from streaming platforms into a unified SQL-queryable view with tenant isolation for teams.
hackerone-reports is a bug bounty dataset aggregator and vulnerability intelligence tool designed to scrape and parse public security reports from the HackerOne platform. It functions as a security report parser that transforms raw bug bounty feeds into structured datasets for analysis. The system automates the collection of public vulnerability reports to identify common security patterns and facilitate security research. It specializes in organizing these disclosures by bug type, payout amount, and target program to enable vulnerability trend analysis. The tool covers capabilities for scra
Collects high-impact security reports from public feeds and organizes them by type, payout, and program.
Akka.NET is an actor model framework used for building concurrent and distributed applications. It functions as a distributed computing platform and state manager that enables isolated actors to communicate via asynchronous message passing, ensuring thread-safe state management without manual locks. The project is distinguished by its decentralized coordination capabilities, including a distributed state manager that uses sharding and dynamic rebalancing to maintain high availability. It incorporates an event sourcing engine that persists state as a sequence of events in an append-only log an
Collects and consolidates information from groups of actors to generate reports or status queries.
m3 es una base de datos de series temporales distribuida, diseñada para métricas de alta resolución y gestión de datos de alta cardinalidad. Funciona como un sistema de almacenamiento escalable y un motor de consultas multiclúster, proporcionando un agregador de métricas distribuido capaz de realizar downsampling y resumir datos antes de que se confirmen en el almacenamiento. El proyecto se distingue por un modelo de clúster coordinado que utiliza etcd para la pertenencia a nodos y la colocación de shards. Soporta múltiples protocolos de ingesta, incluyendo el protocolo de escritura remota de Prometheus, el protocolo de línea de InfluxDB y el protocolo de texto plano de Graphite Carbon, y proporciona interfaces de consulta compatibles para PromQL y Graphite. El sistema cubre amplias áreas de capacidad, incluyendo almacenamiento de series temporales en columnas, replicación de datos síncrona y distribución de consultas (fan-out) distribuida. Incorpora automatización del ciclo de vida de los datos, ajuste de consistencia basado en quórum e indexación de series basada en etiquetas para mantener la integridad de los datos y la velocidad de recuperación en espacios de nombres aislados. La orquestación del clúster y la colocación de componentes se gestionan mediante herramientas y operadores automatizados para garantizar la alta disponibilidad y una distribución equilibrada de los datos.
Provides a mechanism to output aggregated metrics to long-term storage for persistence.
mmocr es un framework de reconocimiento óptico de caracteres basado en PyTorch diseñado para entrenar y desplegar modelos de detección de texto, reconocimiento y extracción de información clave. Sirve como una caja de herramientas integral para la detección y reconocimiento de texto en escenas, proporcionando bibliotecas especializadas para localizar regiones de texto y convertir texto visual en cadenas codificadas por máquina. El proyecto se distingue por un framework de investigación para la extracción de información clave y capacidades avanzadas de detección de texto. Estas incluyen la detección basada en puntos utilizando transformers y el uso de curvas de Bezier parametrizadas para identificar y transcribir texto con formas arbitrarias. El framework cubre una amplia superficie de capacidades de visión artificial, incluyendo la gestión de pipelines de datos para aumentar y estandarizar diversos conjuntos de datos OCR, entrenamiento de modelos con escalado distribuido y evaluación del rendimiento utilizando métricas OCR estándar. También proporciona utilidades para la manipulación de polígonos geométricos y visualización de resultados para auditar predicciones contra anotaciones de verdad fundamental. El sistema está implementado en Python y admite la instalación mediante empaquetado de entorno Docker.
Aggregates multiple distinct data sources into a single unified dataset for training or evaluation.
Open Health is a secure health data platform and personal health record manager designed to collect and store disparate medical histories in a single centralized location. It functions as an AI-powered medical conversation tool and data parser that transforms unstructured health documents into structured files for analysis and processing. The platform integrates large language models to provide personalized health guidance by injecting structured personal medical records into the model context. This allows for the generation of tailored medical responses based on the user's specific health da
Provides a system to collect and consolidate disparate medical records into a unified format for a comprehensive health overview.