Awesome GitHub RepositoriesMapReduce Processing Engines

Redisson is a Java client library for Redis and Valkey that provides a distributed data structure library, a distributed lock manager, and a distributed MapReduce framework. It enables application instances in a cluster to share state through thread-safe collections and objects. The project implements a JCache compliant caching layer for standardized data storage and retrieval. It also functions as a probabilistic data store, providing memory-efficient structures such as Bloom filters and HyperLogLog for high-volume data membership testing. The library covers distributed state management usi

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

PouchDB is a JavaScript NoSQL document store designed to persist JSON data within web browsers or Node.js environments. It functions as an offline-first data store that caches information on a local device and synchronizes with a remote server when connectivity is available. The database implements the CouchDB API to ensure compatibility for bidirectional data replication. This allows for the synchronization of documents between a local client and remote CouchDB servers to maintain consistency across multiple devices. The project provides capabilities for local browser data storage and remot

Hadoop is a big data infrastructure suite and distributed data processing framework designed to store and process massive datasets across clusters of computers. It consists of a distributed storage system for managing large files across multiple nodes and a parallel computing engine for processing data across a distributed cluster. The framework implements a distributed file system to ensure fault tolerance and high throughput, paired with a programming model that processes large datasets in parallel. It manages the underlying hardware and software environment required for distributed big dat

This project is a collection of foundational machine learning algorithms and data science tools implemented in Python. It focuses on building the logic of these tools using basic programming primitives rather than relying on specialized libraries. The implementation covers several core domains, including a linear algebra library for matrix and vector operations, a statistical analysis toolkit for probability and hypothesis testing, and a framework for map-reduce distributed processing. It also includes implementations for natural language processing, graph theory for network analysis, and var

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac

HBase este un magazin NoSQL distribuit de tip wide-column și un motor de stocare big data conceput pentru seturi de date rare (sparse). Funcționează ca o bază de date coloanară scalabilă construită deasupra Hadoop Distributed File System pentru a oferi acces în timp real la citirea și scrierea unor volume masive de date structurate și nestructurate. Sistemul acționează ca un gateway de bază de date cross-language, oferind conectivitate prin apeluri de procedură remote native, REST și interfețe Thrift. Se distinge printr-un model de coordonare master-worker care permite scalarea orizontală și toleranța la erori într-un cluster. Proiectul acoperă un set larg de capabilități, inclusiv controlul accesului granular prin etichete de vizibilitate la nivel de celulă, compresia datelor pluggable și agregarea datelor pe partea de server. De asemenea, suportă fluxuri de lucru de analiză big data prin integrarea map-reduce și permite execuția logicii personalizate pe partea de server. Monitorizarea operațională este oferită prin urmărirea metricilor de sistem și exportul de metrici bazat pe plugin-uri.

This is a collection of academic programming projects that accompany an operating systems textbook, designed to teach core OS concepts through hands-on implementation. The projects span the major subsystems of an operating system, including process scheduling, memory management, file systems, and concurrency, with students building components from scratch in a simulated environment. The projects are structured to cover the full range of OS internals, from low-level kernel development to user-space system programming. Students implement lottery-based CPU schedulers, dynamic heap memory allocat

Acesta este un repository educațional care oferă implementări și tutoriale pentru deep learning, arhitecturi de rețele neuronale și fundamentele machine learning. Servește ca referință pentru construirea de perceptroni multistrat, rețele convoluționale și recurente, utilizând backpropagation și gradient descent. Proiectul include framework-uri specializate pentru modelare generativă prin autoencodere și rețele generative adversariale (GAN), precum și un toolkit pentru reinforcement learning care implementează metode bazate pe valoare, pe politică și actor-critic. De asemenea, oferă referințe practice pentru arhitecturile transformer și BERT, concentrându-se pe mecanismele de atenție pentru procesarea limbajului natural și sarcini de date vizuale. Repository-ul acoperă o gamă largă de capabilități, inclusiv procesarea viziunii computerizate, modelarea secvențială și analiza robusteții adversariale. Oferă, de asemenea, ghiduri pentru machine learning distribuit, detaliind strategii de scalare a antrenării pe mai multe noduri folosind MapReduce, servere de parametri și învățare federată. Proiectul oferă suport fundamental pentru algoritmi tradiționali de machine learning, acoperind specific regresia, clasificarea și clustering-ul.