Awesome GitHub RepositoriesRelational Transformations

Apache Spark is a unified distributed data processing engine designed for large-scale data analysis and computation graphs. It functions as a distributed machine learning framework, a graph processing system, a real-time stream processor, and a SQL analytics engine. The system enables the execution of distributed SQL querying, large-scale graph analysis, and real-time stream analytics across clusters of machines. It also provides a scalable environment for implementing machine learning algorithms and predictive model development on massive datasets. The engine incorporates relational query e

FerretDB is an open-source database emulator and protocol translator that mimics a MongoDB environment to support existing drivers and client tools on a relational backend. It functions as a stateless database proxy that converts binary wire protocol messages into SQL statements, allowing a relational engine to handle document-oriented requests. The project serves as a migration tool for moving applications from MongoDB to PostgreSQL without rewriting queries or changing client drivers. It achieves this by using PostgreSQL as a document store, storing and querying BSON documents through a tra

q is a command-line utility for the processing, filtering, and aggregation of tabular text and database files using standard SQL syntax. It functions as a query engine that treats CSV and TSV files, as well as standard input, as relational database tables. The tool distinguishes itself by providing a persistent cache layer that stores processed tabular data in a binary format to accelerate repeated queries on large datasets. It also maps individual filenames or stream identifiers to relational table names, enabling SQL joins across disparate text files. The project covers a broad range of da

This project is a comprehensive geographic location dataset and reference library providing standardized data for countries, states, and cities. It serves as a source of truth for regional hierarchies, ISO codes, coordinates, and timezone information, available as both a relational SQL database and a document-based JSON library. The project includes a custom dataset export tool that functions as a filtering engine. This allows for the generation of tailored geographic files in JSON, CSV, and GeoJSON formats by selecting only the specific regions or fields required. The dataset covers global

Featuretools is an automated feature engineering library and data transformation framework written in Python. It automatically generates machine learning feature vectors from multi-table datasets by applying synthesis patterns to relational and timestamped data. The system functions as a distributed feature synthesis engine, allowing the process of creating feature vectors to scale across multiple cores or clusters to handle large-scale datasets. The library supports the synthesis of multi-table datasets, time series feature generation, and the creation of custom machine learning primitives

Acest proiect este un sistem de captură a datelor modificate (CDC) și un strat de sincronizare care mută datele din bazele de date MySQL în indecși Elasticsearch. Funcționează ca un mapper relațional-la-document, transformând tabelele bazei de date în documente căutabile pentru a permite integrarea datelor în timp real și căutarea full-text. Sincronizatorul se diferențiază prin suportul pentru denormalizarea datelor relaționale, care transformă join-urile de tip unu-la-mai-mulți din baza de date în structuri de documente părinte-copil. De asemenea, permite agregarea tabelelor partiționate, utilizând modele de expresii regulate pentru a grupa mai multe tabele de bază de date într-un singur index de căutare. Sistemul acoperă maparea și transformarea cuprinzătoare a datelor, inclusiv conversia tipurilor de câmpuri, maparea schemei și filtrarea câmpurilor sincronizate. Utilizează un model de procesare bazat pe pipeline pentru a decoda și îmbina câmpurile, folosind atât încărcarea inițială bazată pe snapshot-uri pentru linii de bază, cât și streaming-ul jurnalului binar pentru actualizări în timp real.

RavenDB is a multi-model NoSQL document database designed for high-performance, ACID-compliant data storage. It persists structured information as schema-flexible JSON documents and utilizes a unit-of-work session pattern to track entity changes and batch modifications into atomic transactions. The platform is built on a distributed architecture that supports horizontal scaling through sharding and ensures high availability via multi-node, master-to-master cluster replication. The database distinguishes itself through a self-optimizing query engine that automatically creates and maintains ind