Awesome GitHub RepositoriesSchema-Driven Data Normalizers

OpenBB is a financial data platform and investment research terminal designed to aggregate, normalize, and distribute market data across analytical workflows. It functions as a comprehensive ecosystem that bridges disparate financial data providers with custom applications, spreadsheets, and internal modeling infrastructure. The platform distinguishes itself through a provider-based data abstraction layer that normalizes heterogeneous financial APIs into a consistent, schema-driven format. This architecture supports quantitative research automation and the construction of interactive, widget-

Kotaemon is an orchestration framework designed for building modular, agentic workflows that integrate document processing, retrieval-augmented generation, and multi-step reasoning. It provides a comprehensive platform for developing document-based question answering systems, allowing users to chain language models, prompt templates, and external tools into complex, automated pipelines. The system distinguishes itself through a highly modular architecture that emphasizes component-based composition and schema-driven data exchange. It supports autonomous agents capable of decomposing complex q

Figma-Context-MCP is a design-to-code automation tool that functions as a server for the Model Context Protocol. It acts as a bridge between visual design platforms and development environments, enabling large language models to access design file metadata and component properties directly. The project distinguishes itself by providing a standard-compliant interface that translates design specifications into structured data. By extracting layout and styling information, it facilitates the programmatic conversion of design tokens and component requirements into actionable code structures. Thi

This project is a command-line forensic toolkit designed for the investigation and security auditing of mobile devices. It provides a framework for collecting system logs, application data, and forensic artifacts to identify potential security breaches, unauthorized access, or evidence of malicious activity. The utility employs a modular extraction architecture that parses diverse file formats and system logs into a standardized, normalized data structure. By utilizing this unified format, the tool performs both heuristic analysis of system metadata and pattern matching against structured thr

Joyagent-jdgenie is an automated data orchestrator designed to centralize the retrieval and processing of information from disparate remote sources. It functions as a framework for building repeatable data pipelines that fetch, clean, and normalize raw input into consistent, structured formats. The system utilizes a schema-driven engine to apply validation rules and structural templates to incoming data, ensuring compatibility across enterprise systems. By employing configuration-based workflow definitions, it allows for the orchestration of modular tasks into automated execution flows, separ

Sigma is a generic SIEM signature format and log event pattern standard used to describe malicious activity. It provides a vendor-neutral system for defining security event patterns in YAML, ensuring that detection logic remains portable across different monitoring platforms. The project maintains a curated library of peer-reviewed detection rules that identify threats and compliance violations. This standardized approach allows for the exchange of threat hunting logic and the translation of generic signatures into specific queries for various security information and event management systems

OmniAuth is a rack authentication framework that allows applications to verify user identities through third-party service providers using a single standardized interface. It functions as middleware to separate identity verification from core application logic by intercepting incoming requests. The project employs a strategy-pattern provider model to encapsulate provider-specific logic into interchangeable classes. It provides a custom authentication strategy framework and base classes for building new providers based on industry standards. The framework handles the multi-step authentication

easyquotation is a Python library that provides access to Chinese stock market data, including real-time quotes, historical daily candlestick prices, exchange-traded fund details, and a stock code database sync utility. It retrieves live trading data from Chinese exchanges, A-shares, and Hong Kong listed stocks without requiring manual API key configuration, offering a unified interface to multiple public data feeds. The library combines several market data providers behind a single query interface, using asynchronous I/O to handle parallel requests and a polling engine that delivers sub-seco

This project is a manga source extension repository and content aggregator. It functions as an HTTP content scraping engine that retrieves images and metadata from external provider websites by parsing HTML and making network requests to display digital manga within a unified reader. The system utilizes a JSON extension repository to allow reader applications to discover and install third-party content providers. It employs an interface-based plugin framework that defines a common set of methods to ensure external sources remain compatible with a standardized internal format. The project cov

JobSpy is a job board scraper and listing aggregator designed to extract employment opportunities from multiple websites and compile them into a unified dataset. It functions as a job search automation tool that programmatically collects vacancies based on keywords, locations, and specific filters. The project serves as a web scraping framework that utilizes proxy routing and user-agent rotation to bypass rate limits and avoid server-side blocking during data extraction. It includes infrastructure for concurrent request aggregation and schema-based data normalization to ensure consistent form

This project is a bug bounty target dataset and security asset list. It serves as a structured repository of reachable network assets, domains, and applications eligible for security testing across multiple vulnerability disclosure programs. The dataset is designed to support bug bounty reconnaissance, attack surface mapping, and security target analysis. It provides organized scopes and target lists to help identify valid assets for security testing and vulnerability research workflows. The repository utilizes automated scraping pipelines and platform API integration to synchronize data. It

OpenAddresses is an open-source geospatial data aggregator and directory that collects public domain and open-license address, parcel, and building datasets from governments and organizations worldwide. It functions as a global index and data warehouse for locating and distributing free geospatial records. The project operates a normalization pipeline that cleans and standardizes diverse source formats into a consistent global coordinate and attribute schema. This process includes a crowdsourced curation pipeline and programmatic quality validation to verify the spatial accuracy and formattin

OSV is a distributed database and aggregator of open-source security advisories that uses a standardized vulnerability schema to track security flaws. It functions as a system for collecting and normalizing security data from diverse ecosystems into a single unified format, providing a web API for querying package vulnerabilities and submitting standardized records. The project distinguishes itself through a security advisory distribution service that supports bulk dataset exports via cloud storage buckets and incremental synchronization of security record updates. It also employs sandbox-bas

El servidor actúa como un motor de ingesta centralizado diseñado para recolectar, normalizar e indexar datos de telemetría distribuidos. Funciona como un procesador de backend que recibe métricas de rendimiento, trazas y registros de errores de agentes de aplicación, transformándolos en documentos estructurados para su almacenamiento y análisis dentro de plataformas de búsqueda y analítica. El sistema se distingue por un pipeline de ingesta de alto rendimiento que utiliza procesamiento de eventos asíncrono y control de flujo consciente de la contrapresión (backpressure) para mantener la estabilidad durante picos de tráfico. Emplea etapas de transformación modulares basadas en plugins para sanitizar y enriquecer payloads entrantes, asegurando que los datos heterogéneos sean mapeados a un esquema unificado antes de ser confirmados en el backend mediante indexación masiva optimizada. Más allá de la ingesta central, el proyecto proporciona capacidades de observabilidad completas, incluyendo trazas de peticiones distribuidas, mapeo de dependencias de servicios y rastreo de errores de aplicación. Soporta monitoreo especializado para instrumentación móvil y aplicaciones de modelos de lenguaje, mientras ofrece herramientas de diagnóstico para la correlación de anomalías de rendimiento y recolección de métricas a nivel de sistema. El repositorio incluye una suite de utilidades para la orquestación de desarrollo local, sincronización de contenedores y pruebas automatizadas de rutas de actualización de despliegue.

Esta librería es un framework de procesamiento de datos para la JVM que proporciona un entorno con seguridad de tipos para manipular datos tabulares estructurados. Funciona como un conjunto de herramientas integral para realizar transformaciones de datos complejas, agregaciones y análisis estadístico, aprovechando la validación de esquemas en tiempo de compilación para garantizar la integridad estructural en los pipelines de datos. El proyecto se distingue por su profunda integración con entornos de notebooks interactivos y su uso de generación de código en tiempo de compilación. Al derivar y aplicar esquemas automáticamente a partir de entradas sin procesar, genera descriptores de acceso con seguridad de tipos que permiten el autocompletado en el IDE y la verificación estática de nombres de columnas. Esta arquitectura permite a los desarrolladores realizar procesamiento de pipelines funcional manteniendo una estricta seguridad de tipos, evitando eficazmente errores en tiempo de ejecución durante la manipulación de datos. La librería admite una amplia gama de flujos de trabajo de datos, incluyendo la importación y mapeo de esquemas de bases de datos relacionales, la realización de análisis geoespaciales y la ejecución de pivotes de datos complejos. Incluye amplias utilidades para la construcción, filtrado, ordenamiento y cálculo de estadísticas descriptivas de datos. Además, el framework proporciona capacidades robustas de visualización y generación de informes, permitiendo a los usuarios renderizar tablas HTML interactivas, componer documentos y generar gráficos directamente a partir de conjuntos de datos estructurados. La librería está diseñada para un uso fluido dentro de entornos de desarrollo Kotlin y Java, con soporte especializado para la gestión automatizada de dependencias e integración de kernels en notebooks interactivos.

El asistente de seguridad automatizado es una utilidad de línea de comandos diseñada para orquestar múltiples herramientas de análisis de seguridad en un flujo de trabajo unificado y basado en configuración. Funciona como un motor central que ejecuta pruebas de seguridad de aplicaciones estáticas y escaneos de infraestructura, agregando diversas salidas de herramientas en un formato estandarizado legible por máquina para garantizar una detección de vulnerabilidades consistente a lo largo de los ciclos de vida de desarrollo. La herramienta se distingue por una arquitectura de plugins modular que permite la integración de escáneres personalizados o propietarios, junto con una capa de inteligencia externa que transmite los hallazgos a servicios de inteligencia artificial para el análisis de remediación automatizado. Al admitir formatos de informes estandarizados como SARIF, permite una revisión y seguimiento consistentes de los hallazgos de seguridad en varias plataformas de informes y sistemas de monitoreo. Más allá de sus capacidades de orquestación central, el framework facilita la verificación de cumplimiento automatizada y la aplicación de políticas de seguridad. Se integra directamente en entornos de desarrollo locales y tuberías de integración continua, permitiendo a los equipos definir parámetros de escaneo específicos, umbrales de severidad y exclusiones de archivos para adaptar el análisis de seguridad a los requisitos del proyecto.