Awesome GitHub RepositoriesCache-Aware Memory Access

simdjson is a high-performance JSON parser that utilizes SIMD instructions to process gigabytes of data per second. It functions as a SIMD JSON parser, a multithreaded NDJSON processing library, a UTF-8 validation engine, and a tool for JSON minification and string building. The project focuses on high-throughput data processing, enabling the ingestion of massive JSON volumes and the verification of UTF-8 encoding standards. It includes dedicated capabilities for constructing JSON strings with optimized memory usage and removing unnecessary whitespace from documents to reduce file size. The

The Disruptor is a lock-free inter-thread messaging library and high-performance event bus. It implements a concurrent ring buffer designed for high-concurrency and low-latency message sequencing. The project utilizes a specific messaging architecture to eliminate lock contention, enabling high-throughput event routing and the exchange of continuous event streams between threads. It ensures strict first-in-first-out ordering and immediate data visibility across processing threads. The library provides capabilities for lock-free data streaming, sequential data ordering, and sequence-based eve

This project serves as an educational resource for learning and implementing low-level assembly language optimizations. It provides a structured guide for developers to master hardware-specific instructions and manual performance tuning, focusing on the translation of high-level code into efficient machine-level operations for resource-constrained environments. The materials emphasize techniques for maximizing computational throughput in multimedia processing. By covering instruction-level parallelism, register management, and data parallelism, the project enables the development of software

This project is a collection of educational resources and technical guides focused on Go performance optimization. It provides instruction on improving execution speed and reducing memory usage through code and architectural refinements. The guides cover advanced strategies for low-level programming, including the use of assembly for SIMD instructions and unsafe pointers for direct memory manipulation. It also details concurrency optimization techniques such as lock sharding and cache-line padding to reduce contention and improve hardware utilization. The material encompasses broad capabilit

xxHash is a high-performance, non-cryptographic hash library designed for rapid checksum generation and data integrity verification. It functions as an incremental hashing engine, allowing for the processing of large or streaming data inputs by maintaining a persistent internal state across sequential chunks. The library is engineered as a computational framework that maximizes throughput by utilizing wide CPU registers and branchless instruction pipelining. It achieves high-speed performance by aligning data access with CPU cache lines and employing multi-stage mixing functions that ensure c

DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training and inference of large-scale neural networks. It provides specialized GPU communication kernels and a remote GPU memory interface to facilitate high-throughput data exchange between hardware nodes. The system utilizes dynamic kernel generation to compile optimized GPU kernels during execution, removing the need for separate installation compilation steps. It implements virtual-lane traffic isolation to prevent interference between different data streams and employs routing met

Acest proiect este un generator de ID-uri unice distribuit, conceput pentru a produce identificatori unici globali și sortabili pe 64 de biți pe mai multe noduri. Implementează un algoritm compatibil cu Snowflake care previne coliziunile prin combinarea timestamp-urilor, identificatorilor de worker și numerelor de secvență într-un singur întreg. Sistemul include un orchestrator de ID-uri de worker pentru a aloca și menține identități unice de mașină în timpul pornirii și migrării instanței folosind strategii bazate pe baze de date. Pentru a crește throughput-ul cererilor și a reduce latența, utilizează un strat de caching de tip ring-buffer care pre-generează identificatori și folosește un mecanism de reumplere bazat pe praguri. Software-ul oferă capabilități pentru gestionarea cheilor de baze de date distribuite, coordonarea nodurilor de sistem și configurarea distribuției biților pentru a echilibra nevoile de concurență față de durata de viață a sistemului. Include, de asemenea, utilitare pentru parsarea identificatorilor generați înapoi în componentele lor originale.

Aceasta este o implementare criptografică a funcției memory-hard Argon2, servind drept bibliotecă de hashing a parolelor și funcție de derivare a cheilor. Transformă parolele și salt-urile în hash-uri securizate și generează chei criptografice concepute pentru a rezista atacurilor de spargere hardware de mare viteză. Biblioteca utilizează un design memory-hard care necesită o cantitate semnificativă de RAM pentru a preveni accelerarea prin GPU-uri sau ASIC-uri. Încorporează accesul la memorie independent de date pentru a bloca atacurile de tip side-channel timing și suportă integrarea cheilor secrete sau a „piper-ilor” pentru a proteja împotriva atacurilor de tip brute-force asupra salt-urilor compromise. Capabilitățile suplimentare includ codificarea hash-urilor de parole pentru stocare și transport, capacitatea de a lega hash-urile de date de context specifice și gestionarea securizată a memoriei care șterge bufferele sensibile imediat după utilizare.

This project is a technical curriculum and set of educational resources focused on parallel programming, high-performance computing, and systems programming. It provides a structured course covering the implementation of parallel algorithms and multithreading techniques for processing large datasets. The project includes a systems programming guide for modern language features, a framework for lock-free concurrency patterns, and a manual for optimizing CPU and GPU performance through assembly analysis and cache management. The material covers hardware performance tuning, the implementation o

This project is a technical reference and a collection of internal analysis notes focused on the Go language runtime and compiler. It provides a detailed breakdown of the language internals, covering memory management, garbage collection, and the execution model of the scheduler. The material distinguishes itself by providing deep dives into low-level system details, including a reference for Go assembly instructions, register usage, and system call interfacing. It specifically analyzes the internal implementation of concurrency primitives, such as the goroutine scheduling mechanism, channel

JCTools is a Java concurrency library providing a collection of lock-less and wait-free data structures. It serves as a toolkit for managing thread-safe data exchange, specifically designed to optimize high-throughput messaging and producer-consumer patterns in multi-threaded applications. The library distinguishes itself by implementing specialized queue structures that minimize contention and maximize throughput. By utilizing techniques such as cache-line padding, memory-barrier-based synchronization, and relaxed-consistency memory ordering, it avoids the performance bottlenecks often assoc

perf-ninja is a collection of educational resources and curricula focused on CPU architecture, memory hierarchies, SIMD programming, and low-level performance engineering. It provides instructional material and practical labs for identifying and fixing CPU bottlenecks, such as cache misses and branch mispredictions. The project differentiates itself through specialized training in hardware-level optimizations, including the use of compiler intrinsics for SIMD vectorization and the implementation of branchless predicate execution to eliminate pipeline stalls. It also covers advanced binary-lev

Liboqs este o bibliotecă C care oferă o interfață unificată pentru algoritmi criptografici rezistenți la computerele cuantice, inclusiv mecanisme de încapsulare a cheilor și semnături digitale. Este concepută pentru a facilita integrarea securității post-cuantice în protocoalele și aplicațiile existente, asigurând protecția datelor împotriva amenințărilor viitoare din partea computerelor cuantice la scară largă. Biblioteca se distinge printr-un accent pe implementări de înaltă asigurare și rezistență la atacuri side-channel, utilizând primitive de tip constant-time pentru a preveni scurgerea de informații bazată pe timp. Oferă o configurare extinsă la momentul compilării, permițând dezvoltatorilor să selecteze algoritmi specifici și să gestioneze dimensiunea binară, ceea ce permite deployment-ul în medii diverse, variind de la servere la sisteme embedded cu resurse limitate. Proiectul include instrumente cuprinzătoare pentru benchmarking-ul performanței și injectarea de entropie agnostică față de hardware, susținând utilizarea generatoarelor de numere aleatorii personalizate. De asemenea, oferă harness-uri de testare încorporate pentru verificarea execuției constant-time și evaluarea eficienței operațiunilor criptografice pe diverse arhitecturi de procesor.