Awesome GitHub RepositoriesRelational Transformations

Apache Spark is a unified distributed data processing engine designed for large-scale data analysis and computation graphs. It functions as a distributed machine learning framework, a graph processing system, a real-time stream processor, and a SQL analytics engine. The system enables the execution of distributed SQL querying, large-scale graph analysis, and real-time stream analytics across clusters of machines. It also provides a scalable environment for implementing machine learning algorithms and predictive model development on massive datasets. The engine incorporates relational query e

FerretDB is an open-source database emulator and protocol translator that mimics a MongoDB environment to support existing drivers and client tools on a relational backend. It functions as a stateless database proxy that converts binary wire protocol messages into SQL statements, allowing a relational engine to handle document-oriented requests. The project serves as a migration tool for moving applications from MongoDB to PostgreSQL without rewriting queries or changing client drivers. It achieves this by using PostgreSQL as a document store, storing and querying BSON documents through a tra

q is a command-line utility for the processing, filtering, and aggregation of tabular text and database files using standard SQL syntax. It functions as a query engine that treats CSV and TSV files, as well as standard input, as relational database tables. The tool distinguishes itself by providing a persistent cache layer that stores processed tabular data in a binary format to accelerate repeated queries on large datasets. It also maps individual filenames or stream identifiers to relational table names, enabling SQL joins across disparate text files. The project covers a broad range of da

This project is a comprehensive geographic location dataset and reference library providing standardized data for countries, states, and cities. It serves as a source of truth for regional hierarchies, ISO codes, coordinates, and timezone information, available as both a relational SQL database and a document-based JSON library. The project includes a custom dataset export tool that functions as a filtering engine. This allows for the generation of tailored geographic files in JSON, CSV, and GeoJSON formats by selecting only the specific regions or fields required. The dataset covers global

Featuretools is an automated feature engineering library and data transformation framework written in Python. It automatically generates machine learning feature vectors from multi-table datasets by applying synthesis patterns to relational and timestamped data. The system functions as a distributed feature synthesis engine, allowing the process of creating feature vectors to scale across multiple cores or clusters to handle large-scale datasets. The library supports the synthesis of multi-table datasets, time series feature generation, and the creation of custom machine learning primitives

هذا المشروع عبارة عن نظام لالتقاط بيانات التغيير (CDC) وطبقة مزامنة تنقل البيانات من قواعد بيانات MySQL إلى فهارس Elasticsearch. يعمل كأداة تعيين من علائقية إلى مستند، حيث يحول جداول قاعدة البيانات إلى مستندات قابلة للبحث لتمكين تكامل البيانات في الوقت الفعلي والبحث بالنص الكامل. يتميز المزامِن بدعم إلغاء تطبيع البيانات العلائقية، والذي يحول عمليات الربط (joins) من واحد إلى متعدد في قاعدة البيانات إلى هياكل مستندات أب-ابن. كما يسمح بتجميع الجداول المقسمة، باستخدام أنماط التعبير النمطي لتجميع جداول قاعدة بيانات متعددة في فهرس بحث واحد. يغطي النظام تعيين وتحويل البيانات بشكل شامل، بما في ذلك تحويل نوع الحقل، وتعيين المخطط، وتصفية الحقول المتزامنة. يوظف نموذج معالجة قائماً على خط الأنابيب لفك تشفير ودمج الحقول، مستخدماً كلاً من التحميل الأولي القائم على اللقطات (snapshots) للأساسيات وبث سجلات النظام الثنائية للتحديثات في الوقت الفعلي.

RavenDB is a multi-model NoSQL document database designed for high-performance, ACID-compliant data storage. It persists structured information as schema-flexible JSON documents and utilizes a unit-of-work session pattern to track entity changes and batch modifications into atomic transactions. The platform is built on a distributed architecture that supports horizontal scaling through sharding and ensures high availability via multi-node, master-to-master cluster replication. The database distinguishes itself through a self-optimizing query engine that automatically creates and maintains ind